PAct hard-case OT + VLM exploration
Hard subset selected from the official 100-sample non-PM diagnostics. VLM uses a local image-to-text model and all variants are compared with the same strict scorer.
Variant Summary
| variant | strict F1 | count error | axis error | tree valid |
|---|---|---|---|---|
| OT-2D | 0.000 | 2.818 | 90.00 | 1.000 |
| OT-Proto | 0.000 | 1.182 | 90.00 | 1.000 |
| NonOT-Hier | 0.000 | 1.909 | 65.45 | 1.000 |
| VLM-Seg | 0.000 | 2.909 | 90.00 | 1.000 |
| VLM-Joint | 0.000 | 1.182 | 90.00 | 1.000 |
| VLM-Struct | 0.000 | 1.909 | 65.45 | 1.000 |
Charts
Method buttons below load full transformed mesh GLBs for OT-2D, OT-Proto, NonOT-Hier, VLM-Seg, VLM-Joint, and VLM-Struct. Box proxy GLBs are retained in report.json as proxy_glb.
electronics_104011
Printer · caption: this is a vector illustration of a computer screen.
electronics_103972
Printer · caption: digital art selected for the #
electronics_103867
Printer · caption: digital art selected for the #
electronics_103978
Printer · caption: 3d model of a box
small_appliances_103043
CoffeeMachine · caption: digital art selected for the #
electronics_104020
Printer · caption: digital art selected for the #
electronics_103988
Printer · caption: a box of chocolates.................................
electronics_103878
Printer · caption: the box for the printer.
electronics_104030
Printer · caption: the box in the middle of the road
small_appliances_103016
CoffeeMachine · caption: 3d model of a box
small_appliances_103466
Toaster · caption: 3d model of a box
