Same object, new material prompt: PhysForge fits physics on contact and deformation windows

A reconstructed 3D asset can now change its physical response according to a user-specified material, not merely look like that material. Physical properties were previously inferred from appearance by vision-language models or discriminative models, so predictions were shaped by appearance-material correlations in training data and it was difficult to assign a material to the same object and obtain the corresponding response. On PhysAssets and Google Scanned Objects, PhysForge achieves the best CLIP and physical controllability scores for both common and uncommon materials, with average physical controllability 4.3 versus 3.8 for the strongest baseline and average CLIP 0.261 versus 0.249.

Complex interactions are decomposed into individual objects and semantic components whose physical parameters are optimized independently and then composed into one simulation. Existing methods often simulate each segmented object as a single body, suppressing relative motion and interactions among its semantic components, and coupled motion between components obscures which material property produced the observed response. Removing PhysComp merges semantically distinct components into a single simulated body and the relative motion and interactions disappear; with it, a table, bottle and glass retain their own physical properties and motion state and collide and wobble on contact.

Physical parameters are fitted only over time windows that reveal material properties, and motion in the generated video that does not obey physics is handled separately. Existing methods fit the entire generated video sequence, including intervals that provide little evidence of the object's physical properties, which leads to suboptimal results. Removing PhysFit and fitting the entire generated video prevents the loss from identifying the intervals that need fitting, reducing fitting performance and increasing fitting time.

The demonstrated scope is 3D asset reconstruction and physical simulation on PhysAssets and Google Scanned Objects, covering common and uncommon materials, material conditions such as liquid and sand, and separable components such as a cup and its saucer, a pot and its lid, and two books. It is aimed at researchers and engineers working on physical simulation, 3D generation and interactive content. The source proposes that more advanced non-physical motion separation techniques could provide a cleaner window with less nonphysical motion for physical parameter fitting.

PhysComp segments objects from generated videos using their dynamic and semantic information, so changing the material condition requires additional video generation and physical fitting for each change. Performance also depends on how well the generated video represents physical behavior, and PhysFit can only mitigate video-generation hallucinations to some extent.