HSSD Affordance Labels and Masks

Sampled HSSD assets from the completed Stage1 label proposal and Stage2 3D-ADLLM mask run.

sampled assets: 10 full Stage2 assets: 10964 sample_assets.json

6312ba49648e2f52bd2226fa3e75a0a1d03798c3 overview thumbnail

1. showerhead.n.01

098a8e9aa72ad0663702521445f0673e7760bd9c overview thumbnail

2. chest_of_drawers.n.01

02e404b057e77e5fa66690bc55ae14a72ce0635c overview thumbnail

#	source id	name	synset/category	placement	labels	proposal sources	mean mask	file status
1	`6312ba4964`	Dawlish Recessed Shower with Premium Fixed Head and Flexible Riser Kit, Rose Gold	showerhead.n.01	wall_objects	3	behavior_taxonomy_rule:3	0.389	independent_physical_files
2	`098a8e9aa7`	Kincaid Sliding Door Chest	chest_of_drawers.n.01	large_objects	6	behavior_taxonomy_rule:6	0.082	independent_physical_files
3	`02e404b057`	Viccarbe Ace	armchair.n.01	large_objects	4	behavior_taxonomy_rule:4	0.165	independent_physical_files
4	`0a6e0fc3e8`	Moulin Noir 3-Drawer Desk	desk.n.01	large_objects	6	behavior_taxonomy_rule:6	0.125	independent_physical_files
5	`014fab638d`	Lille Lamp	table_lamp.n.01	small_objects	4	behavior_taxonomy_rule:4	0.156	independent_physical_files
6	`0039496891`	60X96 Rug, Taiga Ivory/Black	rug.n.01	large_objects	4	seqafford_style_llm:4	0.197	independent_physical_files
7	`0039699ff9`	BILD poster	wall_art.n.01	wall_objects	3	seqafford_style_llm:3	0.184	independent_physical_files
8	`159d824127`	Sony VGC-RT2SY Desktop PC + 25.5" LCD	monitor.n.04	small_objects	3	seqafford_style_llm:3	0.347	independent_physical_files
9	`34bd334b91`	Bissell Zing Canister Vacuum Cleaner	vacuum.n.04	large_objects	4	seqafford_style_llm:4	0.119	independent_physical_files
10	`3b63f00001`	Kitchen Craft Assorted Coloured Silicone Spoon	spoon.n.01	small_objects	2	seqafford_style_llm:2	0.539	independent_physical_files

1. Dawlish Recessed Shower with Premium Fixed Head and Flexible Riser Kit, Rose Gold

6312ba49648e2f52bd2226fa3e75a0a1d03798c3 · showerhead.n.01 · wall_objects

labels.json

6312ba49648e2f52bd2226fa3e75a0a1d03798c3 overview

6312ba49648e2f52bd2226fa3e75a0a1d03798c3 turn mask

turnbehavior_taxonomy_ruleratio 0.324conf 0.92

find the knob or handle region used to turn the water on or off

6312ba49648e2f52bd2226fa3e75a0a1d03798c3 press mask

pressbehavior_taxonomy_ruleratio 0.060conf 0.72

find the control region used to press or activate this object

6312ba49648e2f52bd2226fa3e75a0a1d03798c3 grasp mask

graspbehavior_taxonomy_ruleratio 0.782conf 0.72

find the handle region used to grasp this object

2. Kincaid Sliding Door Chest

098a8e9aa72ad0663702521445f0673e7760bd9c · chest_of_drawers.n.01 · large_objects

labels.json

098a8e9aa72ad0663702521445f0673e7760bd9c overview

098a8e9aa72ad0663702521445f0673e7760bd9c open mask

openbehavior_taxonomy_ruleratio 0.116conf 0.94

find the region used to open this object

098a8e9aa72ad0663702521445f0673e7760bd9c pull mask

pullbehavior_taxonomy_ruleratio 0.096conf 0.84

find the handle or edge region used to pull this object

098a8e9aa72ad0663702521445f0673e7760bd9c close mask

closebehavior_taxonomy_ruleratio 0.109conf 0.9

find the region used to close this object

3. Viccarbe Ace

02e404b057e77e5fa66690bc55ae14a72ce0635c · armchair.n.01 · large_objects

labels.json

02e404b057e77e5fa66690bc55ae14a72ce0635c overview

02e404b057e77e5fa66690bc55ae14a72ce0635c sit mask

sitbehavior_taxonomy_ruleratio 0.184conf 0.95

find the seat region used for sitting

02e404b057e77e5fa66690bc55ae14a72ce0635c support mask

supportbehavior_taxonomy_ruleratio 0.064conf 0.9

find the region used to support a seated person

02e404b057e77e5fa66690bc55ae14a72ce0635c move mask

movebehavior_taxonomy_ruleratio 0.241conf 0.65

find the region used to grasp and move this seat

4. Moulin Noir 3-Drawer Desk

0a6e0fc3e814f0e745eda939713a53d37e202702 · desk.n.01 · large_objects

labels.json

0a6e0fc3e814f0e745eda939713a53d37e202702 overview

0a6e0fc3e814f0e745eda939713a53d37e202702 support mask

supportbehavior_taxonomy_ruleratio 0.434conf 0.95

find the top surface used to support objects

0a6e0fc3e814f0e745eda939713a53d37e202702 place_on mask

place_onbehavior_taxonomy_ruleratio 0.065conf 0.93

find the surface where other objects can be placed

0a6e0fc3e814f0e745eda939713a53d37e202702 contain mask

containbehavior_taxonomy_ruleratio 0.186conf 0.88

find the region used to store or contain items

5. Lille Lamp

014fab638dc1b20201ac76b8ee36a14e92463c5a · table_lamp.n.01 · small_objects

labels.json

014fab638dc1b20201ac76b8ee36a14e92463c5a overview

014fab638dc1b20201ac76b8ee36a14e92463c5a toggle mask

togglebehavior_taxonomy_ruleratio 0.178conf 0.78

find the switch or control region used to toggle the light

014fab638dc1b20201ac76b8ee36a14e92463c5a press mask

pressbehavior_taxonomy_ruleratio 0.059conf 0.7

find the control region used to press this object

014fab638dc1b20201ac76b8ee36a14e92463c5a place_on mask

place_onbehavior_taxonomy_ruleratio 0.213conf 0.93

find the surface where other objects can be placed

6. 60X96 Rug, Taiga Ivory/Black

003949689122adfc2370d64c0615edc1b257db83 · rug.n.01 · large_objects

labels.json

003949689122adfc2370d64c0615edc1b257db83 overview

003949689122adfc2370d64c0615edc1b257db83 support mask

supportseqafford_style_llmratio 0.395conf 0.72

Identify the areas of the rug that can support a person standing or walking (the top surface).

003949689122adfc2370d64c0615edc1b257db83 place_on mask

place_onseqafford_style_llmratio 0.087conf 0.6

Segment the rug surface where objects could be placed on top (e.g., shoes, bags, small items).

003949689122adfc2370d64c0615edc1b257db83 sit mask

sitseqafford_style_llmratio 0.158conf 0.48

Mark the parts of the rug where a person could sit on it (the upper surface).

7. BILD poster

0039699ff96422d70b73397371664863025d75e8 · wall_art.n.01 · wall_objects

labels.json

0039699ff96422d70b73397371664863025d75e8 overview

0039699ff96422d70b73397371664863025d75e8 display mask

displayseqafford_style_llmratio 0.102conf 0.82

Which part of the wall art is the displayed image/poster surface that you look at?

0039699ff96422d70b73397371664863025d75e8 support mask

supportseqafford_style_llmratio 0.277conf 0.42

Identify the rigid backing or frame of the wall art that supports the poster.

0039699ff96422d70b73397371664863025d75e8 grasp mask

graspseqafford_style_llmratio 0.173conf 0.33

Which parts (edges/frame) of the poster can you grasp to pick it up or move it?

8. Sony VGC-RT2SY Desktop PC + 25.5" LCD

159d8241274cdf8c553cf43e636490086a80ca71 · monitor.n.04 · small_objects

labels.json

159d8241274cdf8c553cf43e636490086a80ca71 overview

159d8241274cdf8c553cf43e636490086a80ca71 display mask

displayseqafford_style_llmratio 0.523conf 0.86

Identify the part of the monitor where images would be displayed (the screen/display surface).

159d8241274cdf8c553cf43e636490086a80ca71 press mask

pressseqafford_style_llmratio 0.255conf 0.55

Locate the buttons you would press to power on/off or control the display (e.g., power/menu buttons).

159d8241274cdf8c553cf43e636490086a80ca71 move mask

moveseqafford_style_llmratio 0.264conf 0.46

Find the parts you could grab to safely move/reposition the monitor (outer frame/edges, not the screen).

9. Bissell Zing Canister Vacuum Cleaner

34bd334b91aa80076138650d57efa34832b9acd3 · vacuum.n.04 · large_objects

labels.json

34bd334b91aa80076138650d57efa34832b9acd3 overview

34bd334b91aa80076138650d57efa34832b9acd3 grasp mask

graspseqafford_style_llmratio 0.170conf 0.98

Which part of this handheld vacuum cleaner would you grasp to hold and use it?

34bd334b91aa80076138650d57efa34832b9acd3 move mask

moveseqafford_style_llmratio 0.120conf 0.74

Which part of this handheld vacuum cleaner is meant to be moved over a surface during cleaning?

34bd334b91aa80076138650d57efa34832b9acd3 toggle mask

toggleseqafford_style_llmratio 0.087conf 0.78

Which switch on the handheld vacuum cleaner would you toggle to control its power?

10. Kitchen Craft Assorted Coloured Silicone Spoon

3b63f000011404011c4aeb3f19bb04a112535634 · spoon.n.01 · small_objects

labels.json

3b63f000011404011c4aeb3f19bb04a112535634 overview

3b63f000011404011c4aeb3f19bb04a112535634 grasp mask

graspseqafford_style_llmratio 0.757conf 0.86

Select the part of the spoon you would grasp to pick it up and use it (the handle).

3b63f000011404011c4aeb3f19bb04a112535634 move mask

moveseqafford_style_llmratio 0.320conf 0.6

Select the entire spoon as the object that can be picked up and moved around during cooking.