Innovative Research (United States)

mRNA foundation models usually average token embeddings over the whole transcript before fitting a linear probe, so signals from short regulatory regions are diluted by longer ones. A parameter-free region-aware readout instead averages within the 5′ UTR, coding sequence and 3′ UTR and concatenates the three means; across 20 frozen models it raises Pearson correlation on translation and stability tasks by 11–25% on average, with no model significantly worse on any regression task. On classification tasks AUPRC changes range from −3% to +11%.

mRNA foundation models usually average token embeddings over the whole transcript before fitting a linear probe, so signals from short regulatory regions are diluted by longer ones. A parameter-free region-aware readout instead averages within the 5′ UTR, coding sequence and 3′ UTR and concatenates the three means; across 20 frozen models it raises Pearson correlation on translation and stability tasks by 11–25% on average, with no model significantly worse on any regression task. On classification tasks AUPRC changes range from −3% to +11%.