Evaluation
From visual fidelity to worlds that work.
One benchmark. Three levels.
We evaluate delivered projects through native geometry, rendered views, and executed interactions. Each task has a fixed rubric tied to its creation objective.
Objects
Appearance, geometry, and the details that make an object recognizable.
- Native Blender geometry
- Multi-view render evidence
- Detail and export checks
Scenes
Coherent spaces with faithful layouts, materials, lighting, and edits.
- Scene composition
- Reference alignment
- Target edits and preservation
Interactive worlds
Connected spaces where mechanisms work and actions carry state forward.
- Independent UE5 rebuild
- Continuous traversal
- Local actions and complete workflows
Task-specific criteria. A shared score.
0.30 L1 + 0.30 L2 + 0.40 L3
Each component is normalized to 0–1; task score = 100 × Q. Task scores are averaged within each level, then combined across all 145 tasks.
Text-driven generation
Q = 0.60 V + 0.30 R + 0.10 D
- VVisual fidelity
- Fidelity to the instruction or reference and aesthetic quality.
- RRequirements
- Dimensions, part counts, geometric features, and spatial relations.
- DDetail fidelity
- Coverage and implementation accuracy across measured native-scene and export variants.
Reference reconstruction
Q = 0.20 V + 0.50 I + 0.20 L + 0.10 D
- VVisual fidelity
- Fidelity to the instruction or reference and aesthetic quality.
- IImage similarity
- Reference agreement in image structure, edges, color, and tone.
- LObject localization
- Projected object placement, size, and count.
- DDetail fidelity
- Coverage and implementation accuracy across measured native-scene and export variants.
Scene editing
Q = 0.20 V + 0.20 R + 0.10 D + 0.30 T + 0.20 P
- VVisual fidelity
- Fidelity to the instruction or reference and aesthetic quality.
- RRequirements
- Dimensions, part counts, geometric features, and spatial relations.
- DDetail fidelity
- Coverage and implementation accuracy across measured native-scene and export variants.
- TTarget modification
- Achievement of the requested edit within its designated region.
- PPreservation
- Retention of protected scene content outside the edit scope.
Interactive open worlds
Q = 0.60 A + 0.20 F + 0.05 G + 0.15 H
- AVisual quality
- Shape, construction, materials, lighting, composition, detail, and style.
- FTask fidelity
- Environment coverage and the identity of functional areas and mechanisms.
- GSpatial layout
- Support connectivity, penetration, traversability, and layout semantics.
- HInteraction correctness
- Local mechanisms, moving-part clearance, and complete stateful workflows.
Evidence at every scale
Evidence-grounded visual scoring
GPT-6-astra at medium reasoning effort judges predefined criteria against rendered evidence and task references, with generator identity hidden.
Reference alignment
Image structure, color, tone, edges, and projected object locations ground reconstruction in its reference.
Iglobal = 0.45 SSIM + 0.25 color + 0.15 tone + 0.15 edges
Native project inspection
Blender execution checks loading, editable geometry, dependencies, dimensions, and task-specific constraints. Detail fidelity follows the weakest measured variant.
Independent world evaluation
A trusted observer rebuilds the submitted Unreal Engine project in a separate workspace and records geometry, actions, trajectories, and persistent state.
Interactions in context
Local mechanisms, moving-part clearance, and continuous workflows measure whether a world supports complete activities.
H = 0.40 local mechanisms + 0.25 motion clearance + 0.35 state/workflow
Aligned with human judgments
Eight raters ranked four candidates on each of 45 reference-conditioned tasks, producing 360 rankings.
| Task family | Tasks | Kendall τb | Top-one agreement |
|---|---|---|---|
| Multi-view reconstruction | 14 | 0.753 | 77.7% |
| Single-view reconstruction | 16 | 0.659 | 78.1% |
| Reference-image scene | 8 | 0.781 | 84.4% |
| Photographic scene | 7 | 0.768 | 78.6% |