Skip to main content
Search

# SanSi

1 related articles

Subscribe to updates
  1. arXiv

    A decision model that loops its own layers eight times: 1.4B parameters close in on a single-pass model three times its size

    Typed decision models used to return option probabilities in a single forward pass, giving every decision a fixed amount of computation. SanSi turns a language model pre-trained to loop into a typed decision model, reading the option probabilities after every loop and training each with a proper scoring rule, so one model serves every budget from one loop to eight. On 10,027 test decisions from 59 sources it reaches 72.0% accuracy, 13.5 points above a non-looped model of the same shape trained with the same recipe and 1.8 points below a single-pass model with three times its parameters, at 7.7 times the GPU time of one pass.

You’re all caught up