Same object, new material prompt: physical parameters fit on contact windows, and parts simulate separately

A 3D asset can now take its physical behavior from a user-specified material rather than from appearance-based inference alone. Earlier methods infer physical properties with vision-language models or discriminative models trained on physical-property data, so predictions are largely shaped by appearance-material correlations learned from training data, making it hard to assign a user-specified material to the same object and get the corresponding response. On PhysAssets and Google Scanned Objects, PhysForge achieves the best CLIP and physical controllability scores on both common and uncommon materials, with an average physical controllability of 4.3 versus 3.8 for the strongest baseline and an average CLIP of 0.261 versus 0.249.

Physical parameters are no longer fitted to an entire generated video, but only over intervals such as contact, collision and deformation that reveal material properties. Fitting the whole generated sequence mixes material-dependent responses with nonphysical motion, because generated videos can violate physical constraints, which biases the fitted material parameters. Removing PhysFit and fitting physical parameters to the entire generated video makes the loss struggle to identify the intervals that need fitting over the longer sequence, reducing fitting performance and increasing fitting time.

Complex interactions are decomposed into individual objects and semantic components whose physical parameters are optimized independently and then composed into one simulation. Existing methods often simulate each segmented object as a single body, suppressing relative motion and interactions among its semantic components. Removing PhysComp merges semantically distinct components into a single simulated body and the relative motion and interactions disappear; with PhysComp, table, bottle and glass retain their own physical properties and motion state and collide and wobble on contact.

PhysForge targets research and engineering settings that build simulatable 3D assets from material prompts, and its output is a 3D Gaussian asset with semantic component labels that can be placed in a shared simulator. Evaluation covers the PhysAssets and Google Scanned Objects datasets, and simulation uses Taichi-based MPM and rigid-body solvers to generate 150-frame videos at 30 FPS. Material fitting runs for up to 20 rounds over detected collision events, initialized from positions and velocities obtained by video-force fitting.

PhysComp uses generated videos to segment objects from their dynamic and semantic information, and changing the material condition requires additional video generation and physical fitting for each material-condition change. Overall performance also depends on how well the generated video represents physical behavior, and PhysFit can only mitigate hallucinations from video generation to some extent. The authors suggest that more advanced non-physical motion separation techniques could provide a cleaner window with less nonphysical motion for physical parameter fitting.