Meeting transcription stops reading only who is talking: structured speaker conditioning steadies target recognition under overlap
Synopsis
Meeting transcription must output text for a requested speaker, but four-state activity conditioning collapses different interferer counts and hand-over histories into one target condition. Structured speaker conditioning encodes each speaker's activity track separately, combines it with session-level acoustic identity, and controls the acoustic encoder through condition-dependent low-rank cross-channel modulation. On Qwen3-ASR-0.6B with oracle diarization, the complete method uses 1.46M conditioning parameters and achieves a 6.23% mean relative cpER reduction over four-state diagonal conditioning across four meeting test sets.
Interpretation
In meeting transcription, the condition for a requested speaker no longer depends only on four activity states in the current window; it also carries each speaker track's history and an acoustic identity drawn from the whole session. Four-state conditioning summarized non-target tracks into one quantity, so different interferer counts and speaker hand-over histories could map to the same target condition. On Qwen3-ASR-0.6B with oracle diarization, the complete method achieves a 6.23% mean relative cpER reduction over four-state diagonal conditioning across AliMeeting, AISHELL-4, AMI-SDM and NOTSOFAR-1, with 1.46M conditioning parameters.
The target condition has three parts: activity history encoded per track, speaker identity taken from session audio, and a modulation operator that turns this information into encoder channel transformations. Target-centric relational encoding runs a shared causal temporal encoder over each activity track, then uses symmetric sums to compute relations between the target and one interferer and between the target and two distinct interferers, with aggregation cost linear in the number of tracks. Reference identities come from speaker-exclusive session segments outside the current window, are encoded by the first six shared encoder blocks and pooled with attentive weights; conditional low-rank modulation maps the joint condition to channel-wise affine terms and a rank-8 cross-channel residual, adding 296,064 parameters on Qwen3-ASR-0.6B.
Perspective
Evaluation covers four in-domain meeting test sets (AliMeeting, AISHELL-4, AMI-SDM, NOTSOFAR-1) and three unseen domains (MISP-Meeting, AISHELL-5, DiPCo), all with single-channel far-field audio; mechanism ablations mainly use oracle diarization, while complete-system comparisons use predicted diarization from DiariZen with sequence-to-sequence refinement.
The timestamp variant costs 0.43 percentage points of equal-dataset mean cpER relative to the text model across seven test sets; with predicted diarization, macro-average cpER across the four in-domain sets is 15.60% for text and 15.96% with timestamps. Session identity helps most under target misses and track swaps, while false-alarm interventions change AliMeeting and AMI-SDM by only 0.02 and 0.36 percentage points.