DuET stops reading piano performances note by note: adding frame-level beats cuts MUSTER error from 6.56% to 3.64% on popular piano
Synopsis
Piano performance MIDI-to-score transcription has treated notes as the only modeling unit, yet beats can fall on frames with no note onset, so metrical information struggles to reach the notation. DuET runs a note encoder and a frame-level encoder over notes and a fixed time grid, exchanges information through bidirectional cross-attention, and lets a decoder alternate between the two representations to predict durations, bars, staves and other score attributes. On the popular-piano Gangqinpu test split it lowers MUSTER mean error from 6.56% for the strongest baseline to 3.64%, and on the classical ASAP split from 11.30% to 11.22%.
Interpretation
Piano performance transcription now learns from two cues at once, notes and a fixed time grid, so beats and downbeats are no longer just labels attached to note onsets. Earlier approaches either predicted beat labels at note onsets and used dynamic programming to infer beats without coincident notes, or conditioned on an externally supplied beat grid that required a separate beat tracker when unavailable. On the Gangqinpu test split DuET reduces MUSTER mean error from 6.56% for the strongest baseline to 3.64%; on ASAP it reaches 11.22% against the published Transformer result of 11.30%.
The model pairs a note encoder and a frame encoder with a note-aligned autoregressive decoder, and the two encoders read each other's states at every layer. The note encoder models relationships among notes, the frame encoder covers time beyond note onsets on a 20 ms grid, and the decoder alternates between note memory and frame memory across odd and even layers. Evaluation uses 59 performances from the ASAP test split and common-valid subsets of 79 performances from the Gangqinpu test split; ablations use fixed subsets of 39 ASAP and 72 Gangqinpu performances.
Perspective
Evaluation covers classical piano performances from ASAP and the authors' curated Gangqinpu dataset, which contains 18,787 performance-score pairs and focuses on popular piano; the model handles only the performance-MIDI-to-structured-score stage, leaving pitch spelling and ties to MuseScore 4.
On ASAP, DuET's missing-note and extra-note errors remain higher than several baselines; ablations show that the benefits of pitch-aware encoding and decoder memory access vary with dataset and metric, and removing input masking improves Gangqinpu note-deletion and note-insertion errors. The authors list improving downbeat transcription accuracy and extending evaluation to broader musical styles as future work.