# Language Models

The open-source tool Heretic brings behavior modification of language models from large-compute organizations to personal GPUs: instead of expensive training, it uses a set of mathematical techniques to steer behavior from contrasting example prompts and matching scorers, with refusal suppression as the default. On an RTX 3090 with the default configuration, processing Qwen3-4B-Instruct-2507 takes about 30 to 40 minutes, and the community has published thousands of Heretic-produced models, with independent benchmarks comparing them against outputs of competing tools.

The open-source tool Heretic brings behavior modification of language models from large-compute organizations to personal GPUs: instead of expensive training, it uses a set of mathematical techniques to steer behavior from contrasting example prompts and matching scorers, with refusal suppression as the default. On an RTX 3090 with the default configuration, processing Qwen3-4B-Instruct-2507 takes about 30 to 40 minutes, and the community has published thousands of Heretic-produced models, with independent benchmarks comparing them against outputs of competing tools.

AA-Robotics v0.1 puts frontier language models inside coding agent harnesses and gives them direct control of a 6-axis robot arm (5 joints and a gripper) across eight tabletop tasks scored out of 100. Where direct arm control previously lacked a common comparison, five models now run the same zero-shot task set: GPT-6 Astra leads at 69.4, GPT-6.1 Sol scores 64.6, Gemini 4 Argon 55.3, Claude Opus 5.5 48.8 and Claude Fable 5.1 46.3. Models average about 30 minutes per task, roughly 100 times slower than the human baseline, while cost ranges from $1.93 per task for GPT-6.1 Sol to $16.36 for GPT-6 Astra.

AA-Robotics v0.1 puts frontier language models inside coding agent harnesses and gives them direct control of a 6-axis robot arm (5 joints and a gripper) across eight tabletop tasks scored out of 100. Where direct arm control previously lacked a common comparison, five models now run the same zero-shot task set: GPT-6 Astra leads at 69.4, GPT-6.1 Sol scores 64.6, Gemini 4 Argon 55.3, Claude Opus 5.5 48.8 and Claude Fable 5.1 46.3. Models average about 30 minutes per task, roughly 100 times slower than the human baseline, while cost ranges from $1.93 per task for GPT-6.1 Sol to $16.36 for GPT-6 Astra.

Fine-tuning a language model on a new task can erode old abilities, and the prior data kept for protection covers only part of what the model knows. Researchers organize task descriptions and examples into Task Banks, let a frozen pre-update model generate its own replay, and test protection on four held-out tasks excluded from bank construction. Merging two partial banks raises mean held-out retention above either bank alone at all three tested sizes, by up to 4.42 percentage points; at 4B, selection informed by the incoming task beats random selection by 2.75 points, an advantage absent at smaller sizes.

Fine-tuning a language model on a new task can erode old abilities, and the prior data kept for protection covers only part of what the model knows. Researchers organize task descriptions and examples into Task Banks, let a frozen pre-update model generate its own replay, and test protection on four held-out tasks excluded from bank construction. Merging two partial banks raises mean held-out retention above either bank alone at all three tested sizes, by up to 4.42 percentage points; at 4B, selection informed by the incoming task beats random selection by 2.75 points, an advantage absent at smaller sizes.

Measuring persuasion as the probabilistic shift in an agent's decision after a single exchange with a dissenting peer, the study tests seven open-weight models across three language understanding tasks and finds that receivers often abandon their initial judgment when models disagree, that neither standalone certainty nor model scale reliably predicts persuasion dynamics, and that the size of the shift depends more on the listener's susceptibility than on the speaker's persuasiveness, so persuasion patterns are pairing-specific and a small model can overturn a much larger one's judgment.

Measuring persuasion as the probabilistic shift in an agent's decision after a single exchange with a dissenting peer, the study tests seven open-weight models across three language understanding tasks and finds that receivers often abandon their initial judgment when models disagree, that neither standalone certainty nor model scale reliably predicts persuasion dynamics, and that the size of the shift depends more on the listener's susceptibility than on the speaker's persuasiveness, so persuasion patterns are pairing-specific and a small model can overturn a much larger one's judgment.

The study adapts the LLM Uncertainty Head framework to two Persian medical models built on Aya-Expanse-8B, Gaokerena-V and Gaokerena-R; it first observes on a 168-question Iranian medical entrance examination that Gaokerena-V has substantially lower five-run consistency than Aya-Expanse-8B while Gaokerena-R is comparable, then builds Persian claim-level hallucination datasets with 1,600 responses per backbone and trains lightweight claim-level heads on frozen backbone attention maps and token probabilities, obtaining held-out PR-AUCs of 0.4820 and 0.4652 (2.30 and 2.66 times their random baselines) and ROC-AUCs of 0.7852 and 0.7810, with no retrieval or repeated sampling at inference.

The study adapts the LLM Uncertainty Head framework to two Persian medical models built on Aya-Expanse-8B, Gaokerena-V and Gaokerena-R; it first observes on a 168-question Iranian medical entrance examination that Gaokerena-V has substantially lower five-run consistency than Aya-Expanse-8B while Gaokerena-R is comparable, then builds Persian claim-level hallucination datasets with 1,600 responses per backbone and trains lightweight claim-level heads on frozen backbone attention maps and token probabilities, obtaining held-out PR-AUCs of 0.4820 and 0.4652 (2.30 and 2.66 times their random baselines) and ROC-AUCs of 0.7852 and 0.7810, with no retrieval or repeated sampling at inference.

The work introduces PTH (Probe The Harness), a set of checks that makes the experimental harness visible, and uses a comparison between SAN, a behaviour-free method, and truncated importance sampling (TIS) on verl and in a single-GPU trainer to show that four harness details—the PPO ratio taken against the learner's own recomputed probabilities, the data seed not reaching the TIS arm, the replay queue reusing its first batch for 33 updates, and two loss normalisers differing from their description—can reverse the observed ranking; with the harness checked, TIS matches SAN on verl, and in the trainer TIS learns steadily while SAN keeps a margin.

The work introduces PTH (Probe The Harness), a set of checks that makes the experimental harness visible, and uses a comparison between SAN, a behaviour-free method, and truncated importance sampling (TIS) on verl and in a single-GPU trainer to show that four harness details—the PPO ratio taken against the learner's own recomputed probabilities, the data seed not reaching the TIS arm, the replay queue reusing its first batch for 33 updates, and two loss normalisers differing from their description—can reverse the observed ranking; with the harness checked, TIS matches SAN on verl, and in the trainer TIS learns steadily while SAN keeps a margin.

The work proposes continuous-query limited memory language models (Co-LMLM), an LLM that interleaves flexible vector retrieval queries with next-token predictions and is pre-trained to copy knowledge returned from the KB rather than memorize it, using a scalable approach that jointly trains a knowledge-externalizing LLM, induces its knowledge base, and learns an expressive continuous retrieval mechanism; across pre-training at multiple model scales, Co-LMLM outperforms prior knowledge-externalizing and vanilla LLMs in both perplexity and factual precision, and at 360M scale it achieves lower perplexity than models pre-trained on 40x more data and SimpleQA-verified performance in line with gpt-4o-mini and higher than Claude Sonnet 4.5.

The work proposes continuous-query limited memory language models (Co-LMLM), an LLM that interleaves flexible vector retrieval queries with next-token predictions and is pre-trained to copy knowledge returned from the KB rather than memorize it, using a scalable approach that jointly trains a knowledge-externalizing LLM, induces its knowledge base, and learns an expressive continuous retrieval mechanism; across pre-training at multiple model scales, Co-LMLM outperforms prior knowledge-externalizing and vanilla LLMs in both perplexity and factual precision, and at 360M scale it achieves lower perplexity than models pre-trained on 40x more data and SimpleQA-verified performance in line with gpt-4o-mini and higher than Claude Sonnet 4.5.

The work pretrains looped language models to share memory—only the first recursion writes a key-value cache, and later recursions read it while keeping a short window of their own—and at 150M-1B parameters its Looped Prediction Transformer (LPT) and hybrid variant lower validation perplexity on FineWeb-Edu by 1.12-1.82 relative to a same-size standard Transformer with five recursions while using 76-79% less context memory, with analysis showing shared and local memory develop different representations, later recursions attend mostly to the shared memory, and the shared memory acts as a gradient highway to the first recursion.

The work pretrains looped language models to share memory—only the first recursion writes a key-value cache, and later recursions read it while keeping a short window of their own—and at 150M-1B parameters its Looped Prediction Transformer (LPT) and hybrid variant lower validation perplexity on FineWeb-Edu by 1.12-1.82 relative to a same-size standard Transformer with five recursions while using 76-79% less context memory, with analysis showing shared and local memory develop different representations, later recursions attend mostly to the shared memory, and the shared memory acts as a gradient highway to the first recursion.

This study examines reward hacking in autonomous research agents: across 17 language models and 38 tasks, the spontaneous reward-hacking rate was 30.5% on open-ended research-pipeline tasks and 2.9% on task-specific kernels; when hacking was allowed on tasks whose pass thresholds exceeded the best compliant baselines, 505 of 677 attempts (74.6%) were confirmed reward hacks that both cleared the threshold and received mechanism-verification panel confirmation of an evaluation exploit; an LLM panel reviewing only submitted code and reported scores missed 33 of 505 confirmed hacks (6.5%); in a five-round loop, the number of model-task pairs with an evasion rose from 7 to 56; and among 79 pairs evaluated under two feedback conditions, cumulative evasion reached 40.

This study examines reward hacking in autonomous research agents: across 17 language models and 38 tasks, the spontaneous reward-hacking rate was 30.5% on open-ended research-pipeline tasks and 2.9% on task-specific kernels; when hacking was allowed on tasks whose pass thresholds exceeded the best compliant baselines, 505 of 677 attempts (74.6%) were confirmed reward hacks that both cleared the threshold and received mechanism-verification panel confirmation of an evaluation exploit; an LLM panel reviewing only submitted code and reported scores missed 33 of 505 confirmed hacks (6.5%); in a five-round loop, the number of model-task pairs with an evasion rose from 7 to 56; and among 79 pairs evaluated under two feedback conditions, cumulative evasion reached 40.