MIT CSAIL, Google and collaborators introduce InstructMesh, pairing TRELLIS with GPT-4 so novices can repair AI-generated 3D models and print them

InstructMesh combines a 3D generator with a language model into an interactive editing interface where users describe problems in natural language, fine-tune with sliders, and have geometry changes made in latent space. Previously, generative AI for 3D printing produced models that understand how an object should look but not how it works, and the results were hard to edit, especially for novices; this work puts editing directly in the generative model's latent space, with the user evaluating and approving changes. The text reports system design and author demonstrations: a dragon-wrapped mug, a shiny blue shell-like whistle, butterfly-wing glasses, an octopus-like dispenser, a denim-look knee brace, a shrimp-shaped bristle bot with a hidden motor, plus user-made phone stands and vases; no quantitative baseline comparison is given.

Using TRELLIS to recreate popular 3D models from Thingiverse, the team found nearly 80 percent of the generated models were structurally flawed in some way. This gives a concrete measurement for the claim that AI-generated 3D models are unreliable for fabrication, rather than a purely qualitative description. The text reports a single proportion (nearly 80 percent) without stating sample size, sampling method, or flaw criteria.

When novices who had never done 3D modeling were asked to identify and fix these structural flaws in InstructMesh, they did both about 90 percent of the time, as reviewed by an expert. This turns 'can novices spot and correct design flaws without 3D modeling experience' from an assumption into an expert-reviewed observation, suggesting intuition can partly substitute for tool expertise. Results were expert-reviewed, but the text does not give the number of novices, tasks, or reviewers and their agreement.

Users reported InstructMesh was easy to use and let them express a wide range of ideas, with sliders giving precision for tweaks such as enlarging or extruding a particular part, and the printed items worked as intended. It shifts evaluation from 'does it look right' to 'can it be edited, printed and used,' echoing the authors' 'what you see is what you get' framing. Based on user self-report and author paraphrase, i.e., subjective usability feedback; no standardized scales or control conditions are reported.

The result targets ordinary users who want to print functional items but lack 3D modeling experience, as well as designers iterating on enclosures and accessories; the setting is a workflow where a text or image prompt generates a 3D model and a human reviews and edits it before printing. The authors' proposed next steps include integrating InstructMesh into an augmented reality platform that uses surroundings as prompt context and prints rapidly (for example, a phone case matching your wallet), adding physics simulations to assess how a design behaves in specific uses (such as whether a bowl breaks when dropped and which materials work best), and incorporating the newer TRELLIS.2 to refine even smaller features.

The text does not give the number of novices, the number of tasks, the Thingiverse sampling scale, or the flaw criteria, nor does it report control conditions or statistical tests, so the robustness of the nearly 80 percent and about 90 percent figures still needs to be confirmed in the paper. In addition, physics simulation, AR integration and TRELLIS.2 remain proposed directions without results; the knee brace and robot enclosure demonstrations are individual showcases and do not by themselves establish general manufacturability.