Behavior classification skips pose training: B-REAL labels raw video frame by frame under 15 ms

Behavior analysis can skip the pose-estimation stage: B-REAL reads raw video frames and outputs a behavior class per frame while acquisition is still running. The dominant pipeline first trains a pose estimator to obtain keypoints, then learns behavior classes on keypoint trajectories; such classifiers typically run offline and summarize kinematics over windows that can outlast the briefest behaviors. On held-out CalMS21 test video, B-REAL's per-frame predictions closely follow the human annotation, whereas the keypoint-based A-SOiD changes more slowly, reflecting the time bin over which it summarizes keypoint trajectories.

Annotation cost drops to minutes: a small labeled budget matches or exceeds established methods trained on full benchmark datasets. Training pose estimators previously required thousands of video frames and trained annotators, and behavior classifiers inherited a ceiling from the quality of keypoint tracking. On CalMS21, one minute of randomly sampled labels put B-REAL 5.7 and 2.5 balanced-accuracy points above A-SOiD and CBAS trained on the full dataset; at 30 seconds the margins were 25 and 32 points, with Holm-corrected p < 0.05 in all eight comparisons.

The advantage concentrates on rare, very brief behaviors. In the fly aggression dataset, charge occupies just 0.016% of scored fly-frames, and 56% of its 117 labeled bouts last a single 33 ms frame, which window-based summaries discard. From one minute of labels upward, B-REAL beat A-SOiD and CBAS on balanced accuracy and charge F1 in all eight paired draws (Wilcoxon p = 0.008); on the full training set its charge F1 was 0.40 against 0.04 for A-SOiD and 0.08 for CBAS.

Classification is fast enough to drive closed-loop experiments, with detections triggering reward, stimulation or context changes. Few online classifiers existed: real-time pose methods can only define behavior by thresholding a pose value, while video foundation models carry computational costs that force retrospective, offline classification. In a closed-loop experiment with four mice, three consecutive grooming frames triggered sweetened-water reward, and the rate of criterion grooming bouts rose 129% from early to late sessions (paired t-test p = 0.021).

The demonstrated scope covers two public benchmarks, CalMS21 mouse social behavior and Fly-vs-Fly fly aggression, plus a closed-loop grooming experiment on a single rig with one mouse strain and four animals; the platform targets behavioral neuroscience labs that need per-frame behavior readout during acquisition to trigger external devices.

Closed-loop validation used a single testing rig, one mouse strain and a cohort of four animals, which the authors frame as a functional proof-of-concept rather than an exhaustive parameter exploration; how the effect depends on the delay between detection and reinforcement was not tested. Metrics for the largest b7 backbone come from a single training run, and two of its three clip configurations showed training instability, so it is reported as an observational benchmark. A more heterogeneous pre-training corpus gave no measurable gain over CalMS21 video alone, leaving the value of corpus diversity to be established.