AI and science frontiers · 2026-10-26

Multi-turn jailbreak attacks escalate gradually and coordinate across turns, while existing defenses either need costly retraining or judge each turn in isolation and miss how risk accumulates. THRD combines current-turn risk, historical intent escalation and the previous response's facilitation into one time-evolving risk score, deciding at inference time whether to answer normally, answer under constraints, or refuse, with no training. On Qwen2.5-7B and Llama3-8B against the X-Teaming and Tempest multi-turn attacks, THRD brings attack success rate to 0.2–4.0% while keeping MMLU and GSM8K utility loss within 1.5%.

Multi-turn jailbreak attacks escalate gradually and coordinate across turns, while existing defenses either need costly retraining or judge each turn in isolation and miss how risk accumulates. THRD combines current-turn risk, historical intent escalation and the previous response's facilitation into one time-evolving risk score, deciding at inference time whether to answer normally, answer under constraints, or refuse, with no training. On Qwen2.5-7B and Llama3-8B against the X-Teaming and Tempest multi-turn attacks, THRD brings attack success rate to 0.2–4.0% while keeping MMLU and GSM8K utility loss within 1.5%.

Organizations checking whether an LLM reply breaks internal policy have relied on fine-tuned detectors or a second LLM judging each exchange, both costly to train and to run. The new approach recasts violation detection as out-of-distribution detection in activation space: a whitening transform estimated from policy text and a few examples standardizes compliant hidden states, and the Euclidean norm of the whitened vector is compared with a calibrated threshold. On the DynaBench test set with a Qwen2.5-7B-Instruct backbone, F1 reaches 86.0%, above the fine-tuned detectors and LLM-as-a-judge baselines listed in the comparison table.

Organizations checking whether an LLM reply breaks internal policy have relied on fine-tuned detectors or a second LLM judging each exchange, both costly to train and to run. The new approach recasts violation detection as out-of-distribution detection in activation space: a whitening transform estimated from policy text and a few examples standardizes compliant hidden states, and the Euclidean norm of the whitened vector is compared with a calibrated threshold. On the DynaBench test set with a Qwen2.5-7B-Instruct backbone, F1 reaches 86.0%, above the fine-tuned detectors and LLM-as-a-judge baselines listed in the comparison table.

Binarization-based post-training quantization for LLMs has carried hidden overhead: scaling factors, group parameters and outlier indices push nominally 1-bit weights into an effective 2-4-bit range. AF1 pairs a double-factor binary decomposition with null-space compensation of discrete projection error, then allocates structural budget through hierarchical Shapley values, quantizing under a strict 1.0 effective BPW for target linear weights. On LLaMA-3-70B it reaches 61.80 average zero-shot accuracy over seven tasks and 8.91 WikiText2 perplexity, above the 3.25-bit HBLLM baseline.

Binarization-based post-training quantization for LLMs has carried hidden overhead: scaling factors, group parameters and outlier indices push nominally 1-bit weights into an effective 2-4-bit range. AF1 pairs a double-factor binary decomposition with null-space compensation of discrete projection error, then allocates structural budget through hierarchical Shapley values, quantizing under a strict 1.0 effective BPW for target linear weights. On LLaMA-3-70B it reaches 61.80 average zero-shot accuracy over seven tasks and 8.91 WikiText2 perplexity, above the 3.25-bit HBLLM baseline.

Automatic ICD coding assigns standardized diagnostic codes to discharge summaries, but sections such as hospital course and discharge diagnosis follow near-identical templates, so pretraining can latch onto document structure instead of patient-specific clinical meaning. SARL-ICD restricts contrastive pretraining to narrative sections such as chief complaint, history of present illness and past medical history, builds positive pairs from Jaccard similarity over ICD codes expanded with their hierarchy ancestors, injects LLM-augmented code descriptions through a lightweight label-text interaction module, and fuses PLM-CA with PLM-ICD through a mixture-of-experts gate. On MIMIC-IV ICD-10 the full system reaches 28.6% macro-F1 and 61.1% micro-F1, against 26.1% and 59.6% for PLM-CA.

Automatic ICD coding assigns standardized diagnostic codes to discharge summaries, but sections such as hospital course and discharge diagnosis follow near-identical templates, so pretraining can latch onto document structure instead of patient-specific clinical meaning. SARL-ICD restricts contrastive pretraining to narrative sections such as chief complaint, history of present illness and past medical history, builds positive pairs from Jaccard similarity over ICD codes expanded with their hierarchy ancestors, injects LLM-augmented code descriptions through a lightweight label-text interaction module, and fuses PLM-CA with PLM-ICD through a mixture-of-experts gate. On MIMIC-IV ICD-10 the full system reaches 28.6% macro-F1 and 61.1% micro-F1, against 26.1% and 59.6% for PLM-CA.

Vision-language models often describe objects that are not in the image, and earlier fixes intervened at the output level or across whole attention layers, which cannot separate heads that support faithful captions from heads that drive hallucination. HACP scores each attention head at inference time by the spectral change of its perturbed representation, then suppresses hallucination-prone heads and reinforces faithful ones during decoding without training. On a 1000-image multi-label COCO subset it lowers CHAIR-s/i from 66.7/18.0 to 43.4/8.6 and raises F1 from 73.0 to 77.0.

Vision-language models often describe objects that are not in the image, and earlier fixes intervened at the output level or across whole attention layers, which cannot separate heads that support faithful captions from heads that drive hallucination. HACP scores each attention head at inference time by the spectral change of its perturbed representation, then suppresses hallucination-prone heads and reinforces faithful ones during decoding without training. On a 1000-image multi-label COCO subset it lowers CHAIR-s/i from 66.7/18.0 to 43.4/8.6 and raises F1 from 73.0 to 77.0.

LLM distillation has largely kept the temperature at τ=1 and concluded that reverse KL beats forward KL. A gradient analysis shows temperature changes what forward KL learns but mainly rescales reverse KL gradients; across GPT-2 and OPT instruction-following benchmarks reverse KL leads at τ=1, forward KL overtakes at higher temperatures, and in on-policy GPT-2 Base distillation forward KL rises from 16.95 to 21.24 average ROUGE-L.

LLM distillation has largely kept the temperature at τ=1 and concluded that reverse KL beats forward KL. A gradient analysis shows temperature changes what forward KL learns but mainly rescales reverse KL gradients; across GPT-2 and OPT instruction-following benchmarks reverse KL leads at τ=1, forward KL overtakes at higher temperatures, and in on-policy GPT-2 Base distillation forward KL rises from 16.95 to 21.24 average ROUGE-L.