OncoVision's attention-driven multimodal training framework cut reading time by up to 61% and raised diagnostic confidence in a paired six-radiologist evaluation
Synopsis
OncoVision is a privileged-information training framework that uses mammography images and clinical features during training while performing inference from mammographic images alone; built on an attention-based encoder-decoder backbone, it jointly segments four regions of interest (masses, calcifications, axillary findings, and breast tissue) with accuracy exceeding the nnU-Net baseline and predicts ten structured clinical features including BI-RADS category; the authors developed two late-fusion strategies, Independent and Dependent, that integrate imaging, radiomic, and clinical information during training, with radiomic features extracted from predicted masks providing shape, intensity, and texture descriptors that complement the learned CNN representations; in a retrospective multi-re
Fig. 1 : Integrated workflow and multimodal architecture of OncoVision. (a) Integrated workflow from data acquisition and preprocessing to model training, evaluation, and web deployment. (b) Independent variant: raw clinical features and radiomic descriptors are concatenated with imaging-derived bottleneck features during training. (c) Dependent variant: clinical features are processed through a dedicated encoder before fusion with imaging and radiomic features. At inference, externally supplied clinical features are omitted, while radiomic features are generated internally from the predicted segmentation masks.
arXivInterpretation
OncoVision uses privileged-information training: mammography images and clinical features are both used during training, while inference proceeds from mammographic images alone. Unlike common multimodal approaches that still require multimodal input at inference, this design compresses the value of clinical information into the image model so that deployment needs only mammographic images. The abstract explicitly describes this asymmetric training-versus-inference setup and states it is built on an attention-based encoder-decoder backbone; no dataset size or statistical testing details are given.
The model jointly segments four regions of interest (masses, calcifications, axillary findings, and breast tissue) with accuracy exceeding the nnU-Net baseline, while also predicting ten structured clinical features including BI-RADS category. Segmentation and structured clinical feature prediction are handled within a single attention-based encoder-decoder framework rather than as separate segmentation-only or classification-only tasks. The abstract reports the comparison against the nnU-Net baseline but does not list specific segmentation metric values or confidence intervals.
The authors developed two late-fusion strategies, Independent and Dependent, that integrate imaging, radiomic, and clinical information during training, and extract shape, intensity, and texture descriptors from predicted masks to complement CNN representations. Radiomic features are derived from the model's own predicted masks, so handcrafted descriptors complement learned CNN representations rather than relying only on raw image features. The abstract describes the two fusion strategies and the radiomic feature categories but does not report quantitative comparisons between the strategies.
In a retrospective multi-reader paired assistance evaluation with six board-certified radiologists, OncoVision was associated with higher diagnostic confidence for junior and senior radiologists, reduced reading time by up to 61%, and achieved segmentation accuracy comparable to or exceeding that of radiologists for mass lesions. A multi-reader paired design examines diagnostic confidence, reading time, and segmentation accuracy together, rather than reporting only internal algorithmic metrics. The abstract gives the number of readers (six board-certified radiologists), the study type (retrospective multi-reader study), and a quantified result (reading time reduced by up to 61%), but provides no sample size, confidence intervals, or significance tests.
Perspective
The work targets mammographic interpretation: clinical features can be used during training, while deployment requires only mammographic images, making it suited to institutions where clinical features are available at training time but single-modality input is preferred at inference. Its multi-reader evaluation is a retrospective paired design with six board-certified radiologists, measuring diagnostic confidence, reading time, and segmentation accuracy, so the conclusions apply to reader assistance within that evaluation setting. The system is implemented as a secure web application deployed at a partner hospital, generating structured reports with dual-confidence scoring and attention-weighted visualizations, and the authors position it as a platform for integration into clinical workflows and for supporting screening access in underprivileged regions.
The abstract does not report dataset size, case composition, external validation, or multi-center data, so how the higher diagnostic confidence, the up-to-61% reduction in reading time, and the comparable-or-better mass segmentation accuracy hold in broader populations and equipment conditions remains an open question. The abstract also does not give a quantitative comparison between the two late-fusion strategies, the specific contribution of the radiomic descriptors, or the practical role of dual-confidence scoring and attention-weighted visualizations in clinical decisions. In addition, the abstract states that segmentation accuracy exceeds the nnU-Net baseline but does not list specific segmentation metric values, so performance differences across regions (masses, calcifications, axillary findings, and breast tissue) are not yet clear.