Sparse-autoencoder steering gets an information-theoretic design: DESSIRT beats a contrastive baseline on most question modes in Llama and Gemma
Sparse-autoencoder (SAE) steering has relied on contrastive activation statistics to pick features, although feature interpretability and steering utility are not necessarily aligned. DESSIRT represents each dataset as a vector of SAE feature firing probabilities, measures deviation from a target distribution with Jensen–Shannon divergence, and constructs steering vectors analytically from that divergence. In a seven-mode question-generation task, DESSIRT-2 beat the SDCV baseline on all but one mode in Llama 3.1-8B-Instruct and on most modes in Gemma-2-9B-IT.