3D indoor scene agents stop guessing: teaching a builder to critique its own edits cuts five-step orientation fixes from 40% to 7%

An agent that builds 3D indoor scenes from natural language can now judge which edit is better during construction, instead of handing every decision to a general-purpose model. Earlier agentic frameworks largely relied on general-purpose models for design decisions and produced long correction trajectories; existing scene datasets supply static scenes with object poses but do not record construction actions or correction decisions. Across bedroom, living room, bathroom and office room types, Scene-Designer scores higher than the supervised-only Scene-Builder on physical validity, functional correctness and layout coherence; in living rooms realism rises from 8.0 to 9.6 and layout quality from 8.2 to 9.3.

The gain in correction efficiency shows up as fewer errors that need repeated fixing. The supervised-only Scene-Builder already learned correction strategies from construction trajectories, yet still needed unnecessary repeated attempts for some errors. Against Scene-Builder, Scene-Designer lowers the share of cases needing five or more corrective moves from 15% to 13% for door clearance occupation, 21% to 5% for window accessibility blockage, 40% to 7% for interaction orientation error, and 50% to 23% for scale mismatch.

The method runs in two stages: joint supervised fine-tuning yields a Scene-Builder that both constructs and critiques, then critic-guided reinforcement learning turns it into Scene-Designer. Raw construction trajectories average about 110 steps and contain much wasted trial and error; causal trajectory consolidation flags suspicious operations through state changes and remaining errors, then has GPT-5.2 judge their corrective role and removes only steps confirmed redundant. Consolidation cuts average trajectory length to about 30 steps and tool calls by 57%; Scene-OTA-35K holds 1K scenes and 35K observation-thought-action records, and Scene-OTC-15K holds about 15K critique preference examples.

The learned critic also selects branches at inference time and further improves scenes. General-purpose models are limited at scene critique, with GPT-5.2 reaching 72% on two-scene comparison and 15% on four-scene ranking. The jointly trained critic reaches 89% and 76% on the same test sets, and using it to filter candidate branches at inference further raises scene quality across room types.

The report covers the scope stated in the full preprint: full-parameter fine-tuning on Qwen3-VL-8B-Instruct, evaluation with the unified prompt "Design me a {room_type}", 10 generated scenes per room type, and visual-semantic quality scored by GPT-4 on realism, functionality, layout quality and completeness. Separate geometry-based accessibility metrics are reported, where Scene-Designer w/ Critic reaches the best mRR of 0.98 and FR@0.3 of 1.0. It suits readers following 3D scene generation, agentic tool use and reinforcement-learning post-training.

The design experience comes from GPT-5.2 trajectories on the SceneSmith scaffold after manual selection of high-quality scenes, so the supervision carries that generator's and scaffold's preferences. The critique test sets contain 94 two-scene comparisons and 55 four-scene rankings, and part of the perturbation ordering relies on manual annotation. Mutual reinforcement between design and critique is described as a trend rather than an established conclusion.