# Autoregressive Transformer

FERMAT, a 65-million-parameter clinical foundation model pretrained on records from 3.77 million patients, was compared against itself on the same patients: patient representations predicted five-year kidney, fatty liver, and diabetes risk better, while generated trajectories predicted the diagnoses, prescriptions, and procedures recorded together on the next clinical date better.

FERMAT, a 65-million-parameter clinical foundation model pretrained on records from 3.77 million patients, was compared against itself on the same patients: patient representations predicted five-year kidney, fatty liver, and diabetes risk better, while generated trajectories predicted the diagnoses, prescriptions, and procedures recorded together on the next clinical date better.

Kronos is the first open-source foundation model built for financial candlestick (K-line) sequences, pretrained on data from more than 45 global exchanges. General time-series foundation models are not designed for the high-noise character of financial data, so Kronos first quantizes continuous multi-dimensional K-lines such as open, high, low, close and volume into hierarchical discrete tokens, then pretrains an autoregressive Transformer on those tokens as one model for diverse quantitative tasks. The release includes pretrained weights of several sizes, fine-tuning scripts and a live demo forecasting BTC/USDT over the next 24 hours.

Kronos is the first open-source foundation model built for financial candlestick (K-line) sequences, pretrained on data from more than 45 global exchanges. General time-series foundation models are not designed for the high-noise character of financial data, so Kronos first quantizes continuous multi-dimensional K-lines such as open, high, low, close and volume into hierarchical discrete tokens, then pretrains an autoregressive Transformer on those tokens as one model for diverse quantitative tasks. The release includes pretrained weights of several sizes, fine-tuning scripts and a live demo forecasting BTC/USDT over the next 24 hours.

The work casts quantum many-body dynamics as time-series forecasting on a few-qubit PXP chain realizable with Rydberg-atom arrays, training a nonlinear Transformer and a linear DLinear with the same composite loss to predict local magnetizations and connected two-point correlators from a history window; the Transformer predicts accurately in the more ergodic regime but deteriorates progressively as the initial state approaches the scarred limit, whereas DLinear stays accurate across the entire family of initial states, its main deviations being small high-frequency oscillations with little effect on overall prediction error.

The work casts quantum many-body dynamics as time-series forecasting on a few-qubit PXP chain realizable with Rydberg-atom arrays, training a nonlinear Transformer and a linear DLinear with the same composite loss to predict local magnetizations and connected two-point correlators from a history window; the Transformer predicts accurately in the more ergodic regime but deteriorates progressively as the initial state approaches the scarred limit, whereas DLinear stays accurate across the entire family of initial states, its main deviations being small high-frequency oscillations with little effect on overall prediction error.

The authors propose RRM-GPT, an autoregressive framework that generates radio resource management decisions field by field as a language model generates text: an encoder maps heterogeneous network observations into common tokens and a decoder emits each decision conditioned on the network state and the fields already committed; in a 5G NR case study, a single model generates complete scheduling grants spanning user selection, timing, link adaptation, resource allocation, and control signaling, and transfers learned behavior to an unseen scenario without adaptation.

The authors propose RRM-GPT, an autoregressive framework that generates radio resource management decisions field by field as a language model generates text: an encoder maps heterogeneous network observations into common tokens and a decoder emits each decision conditioned on the network state and the fields already committed; in a 5G NR case study, a single model generates complete scheduling grants spanning user selection, timing, link adaptation, resource allocation, and control signaling, and transfers learned behavior to an unseen scenario without adaptation.