Dance scoring moves past action labels: aligning a learner to a demo phase by phase lifts expert-intermediate-beginner ordering to 81.2%
Synopsis
Automatic dance quality assessment has largely followed action recognition, judging which dance is performed rather than how well. RSDQ-Net maps target and reference performances onto a 19-node canonical skeleton, learns soft phase correspondence, models spatial execution deviation and temporal phase deviation separately, and constrains one latent quality variable with ordinal expertise labels and human pairwise preferences. Under a choreography-disjoint five-fold protocol on ImperialDance it reaches a QWK of 0.814 and ordinal MAE of 0.207, against 0.755 and 0.261 for the strongest baseline, 2T-GCN.
Interpretation
Dance quality assessment is framed as reference-conditioned execution deviation rather than action-category recognition. Two performers may complete the same choreography while differing in body alignment, joint trajectories, timing, and coordination; recognition models tend to suppress within-class variation, whereas an assessment model must preserve and interpret it. Earlier skeleton methods mainly answered which action is performed, while reference comparison often used joint-angle differences, trajectory distances, or dynamic time warping with fixed alignment independent of downstream quality supervision, and temporal warping could remove meaningful timing errors. Under the five-fold choreography-disjoint protocol on ImperialDance with 25 fold-seed evaluations, RSDQ-Net reaches 81.2% accuracy, a QWK of 0.814, and an ordinal MAE of 0.207; the strongest baseline, 2T-GCN, reaches 76.7%, 0.755, and 0.261, with paired Wilcoxon tests after Holm correction giving p=0.004 for QWK and p=0.006 for ordinal MAE.
Target and reference performances are first mapped to one 19-node canonical skeleton and encoded by a shared relation-aware spatiotemporal encoder. The three datasets start from different skeletons: AIST++ provides SMPL 3D motion, ImperialDance provides 21-node motion capture, and CDRG-UNIMAS-TIKTOK provides only RGB video that must be reconstructed into 3D body motion. The canonical topology keeps the trunk and bilateral limbs, with node features combining normalized position, temporal displacement, anatomical bone vector, and observation confidence; sequences are resampled to 30 Hz while keeping variable length, and a validity mask excludes padded frames.
Alignment does not stretch the reference onto target frames; it estimates soft correspondence in the learned motion space and keeps phase displacement as quality evidence. Fixed alignment and dynamic time warping treat correspondence as an independent preprocessing step that cannot be optimized against the downstream quality objective. Alignment weights subtract a phase-displacement term from query-key similarity, where that term is the absolute difference between the normalized target phase and the normalized reference phase, and its strength is constrained non-negative by softplus and learned jointly with the network; in ablation, removing reference alignment drops QWK to 0.724 and pairwise accuracy to 74.9%, and replacing it with DTW recovers only 0.758 and 77.2%.
Spatial execution deviation and temporal phase deviation are represented separately, then ordinal expertise levels and human pairwise preferences constrain one shared latent quality variable. The two supervision forms differ, and no common continuous rating scale can be assumed, so assigning arbitrary numerical scores would introduce unsupported assumptions. ImperialDance uses two ordered thresholds for Beginner < Intermediate < Expert, while CDRG predicts pairwise preference from the difference of calibrated latent qualities; under the subject-disjoint five-fold protocol on CDRG, pairwise accuracy is 81.4% and Kendall τb is 0.638, against 76.8% and 0.566 for the strongest baseline, CTR-GCN + RankNet.
Perspective
Evaluation covers dance-genre recognition on AIST++, choreography-disjoint ordinal quality assessment on ImperialDance, and subject-disjoint pairwise ranking on CDRG-UNIMAS-TIKTOK. ImperialDance uses an Expert demonstration of the corresponding choreography as reference; test choreographies are unseen during parameter learning but their Expert demonstrations are available at assessment time, so this is unseen-choreography reference-guided assessment. CDRG references are selected only from training subjects and fixed per fold. The model has 3.84 million parameters, 21.2 GFLOPs per sample, 8.1 ms inference, and 618 MB peak memory, suited to offline evaluation and interactive assisted training.
The three datasets differ substantially in dance style, acquisition environment, skeletal source, and annotation form; the canonical representation and shared quality formulation reduce only part of that heterogeneity, and broader validation across genres, recording conditions, and performer populations remains to be done. CDRG's 3D skeletons are reconstructed from monocular RGB video, so reconstruction errors, occlusion, and viewpoint ambiguity may affect the deviation representation. The datasets provide no frame-level or joint-level expert annotations, so the qualitative visualizations show learned deviation responses rather than validated expert error localization. The latent quality variable represents relative ordering rather than an absolute professional score, which limits interpretation where standardized numerical grading is required.
