A video-to-model framework uses U-Net and a spatio-temporal CNN to identify CBF–CLF–QP suture-thread parameters from video, reproducing thread behavior with low tracking error on unseen motions
The work presents a video-to-model framework that first extracts an ordered suture-thread node trajectory from a monocular video using U-Net segmentation and curve tracking, then uses a spatio-temporal CNN to estimate the connectivity, stiffness, and velocity-retention parameters of a structured CBF–CLF–QP model, which simulates the thread under a user-defined needle velocity; on unseen virtual materials and a non-training flower-petal needle motion, the configured model reproduces the observed thread behavior with low tracking error and reduces manual parameter tuning.