Rutgers, The State University of New Jersey

10 related articles

Subscribe to updates

arXiv Nov 10, 2026
SinkState stores recurrent states as adaptive low-rank factors: hybrid-model decoding peak memory falls to as little as 1/2.6 of baseline
Hybrid language models compress history into fixed-size recurrent states, yet serving systems still treat those states as dense matrices, reading and rewriting them whole at every decoding step, so memory traffic grows with batch size. SinkState exploits the concentration of queries, keys and values in linear attention into a stable low-dimensional subspace, keeping each state in an adaptive low-rank factorization throughout decoding, merging new tokens with a local SVD and consolidating the frozen part asynchronously on the CPU. Across four Mamba-2 and GDN hybrid models from 4B to 120B parameters, peak recurrent-state memory falls to 0.38–0.63 of the baseline and decoding speeds up by up to 1.58x, with accuracy maintained.
Hybrid language models compress history into fixed-size recurrent states, yet serving systems still treat those states as dense matrices, reading and rewriting them whole at every decoding step, so memory traffic grows with batch size. SinkState exploits the concentration of queries, keys and values in linear attention into a stable low-dimensional subspace, keeping each state in an adaptive low-rank factorization throughout decoding, merging new tokens with a local SVD and consolidating the frozen part asynchronously on the CPU. Across four Mamba-2 and GDN hybrid models from 4B to 120B parameters, peak recurrent-state memory falls to 0.38–0.63 of the baseline and decoding speeds up by up to 1.58x, with accuracy maintained.
University of South Florida The University of Sydney Rutgers, The State University of New Jersey
Haolin Jiang（First author）
SinkState On-the-Fly SVD of Recurrent States Linear Attention Sink Hybrid LLM Serving
bioRxiv Nov 13, 2026
Inside a protein language model, binary patterns form a jellyfish: hydrophobic content sets the main axis while heptad phase curls into a ring
In ESM2 residue embeddings, polar/nonpolar binary patterns organize into a readable geometry. Generating 50 sequences for each of all 128 heptad binary patterns, 6400 in total, and running position-preserving PCA yields a jellyfish-shaped manifold: hydrophobic content spreads along PC1, while PC2 and PC3 encode heptad phase as near sine and cosine. A charge alphabet stays ordered, but scrambled amino-acid groupings or the non-protein model ByT5 lose the cyclic organization.
In ESM2 residue embeddings, polar/nonpolar binary patterns organize into a readable geometry. Generating 50 sequences for each of all 128 heptad binary patterns, 6400 in total, and running position-preserving PCA yields a jellyfish-shaped manifold: hydrophobic content spreads along PC1, while PC2 and PC3 encode heptad phase as near sine and cosine. A charge alphabet stays ordered, but scrambled amino-acid groupings or the non-protein model ByT5 lose the cyclic organization.
Princeton University Rutgers, The State University of New Jersey
Dylan Klein（First author）Vikas Nanda（Corresponding author）
Protein Language Model Binary Patterning ESM2 Principal Component Analysis Four-helix Bundle
arXiv Oct 7, 2026
Multi-agent systems stop sharing one workflow: OOPMAS builds agents and workflows per query, reaching 89.6% on mixed tasks
Automated design of multi-agent systems has mostly worked at the task level, optimizing one fixed workflow per benchmark and applying it to every query. OOPMAS moves design to the individual query: a frozen LLM as meta-agent first generates object-style agents with roles, tools and persistent state, then an executable main-function workflow, while a dynamic skill library accumulates structured lessons from execution feedback without gradient updates. On a benchmark mixing code, math and QA, OOPMAS reaches 89.6% overall accuracy, 18.1 percentage points above the strongest baseline, ADAS.
Automated design of multi-agent systems has mostly worked at the task level, optimizing one fixed workflow per benchmark and applying it to every query. OOPMAS moves design to the individual query: a frozen LLM as meta-agent first generates object-style agents with roles, tools and persistent state, then an executable main-function workflow, while a dynamic skill library accumulates structured lessons from execution feedback without gradient updates. On a benchmark mixing code, math and QA, OOPMAS reaches 89.6% overall accuracy, 18.1 percentage points above the strongest baseline, ADAS.
University of Pittsburgh University of Maryland, College Park Rutgers, The State University of New Jersey
Qi Cheng（First author）
OOPMAS Multi-agent system Query-Level Workflow Generation
arXiv Oct 31, 2026
Story generation learns to plan reader suspense and humor: trajectory fidelity rises 12%–35% while coherence largely holds
Large language models write stories that often lack the rising and falling reader experience that makes narrative engaging. A new framework first learns how reader responses such as suspense or humor evolve in high-quality human-written stories, then uses a small model to predict the response the next segment should elicit and steers an off-the-shelf writer model with that prediction. Across suspense and humor, with GPT-5.5 and Qwen3.5-27B as writers, generated stories improve trajectory-based fidelity to human-written stories by 12%–35% while largely preserving story coherence.
Large language models write stories that often lack the rising and falling reader experience that makes narrative engaging. A new framework first learns how reader responses such as suspense or humor evolve in high-quality human-written stories, then uses a small model to predict the response the next segment should elicit and steers an off-the-shelf writer model with that prediction. Across suspense and humor, with GPT-5.5 and Qwen3.5-27B as writers, generated stories improve trajectory-based fidelity to human-written stories by 12%–35% while largely preserving story coherence.
Netflix (United States)Rutgers, The State University of New Jersey
Danrui Li（First author）
Readerly Dynamics Steering Story Generation Large language models Trajectory Prediction
arXiv Oct 12, 2026
Widening CPU vectors hits the L1 capacity wall: turning private L2 into a software-managed scratchpad gives 1.44x GEMM speedup at 2048 bits
When CPUs widen SIMD vectors from 512 to 2048 bits, compute grows but L1 data cache capacity does not, so data shuttles between L1 and L2 and throughput stalls. CacheFlex uses a lightweight ISA extension to convert part of the private L2 into a thread-private scratchpad at runtime, letting software address slots directly and bypass address translation, associative tag lookup, and coherence probes. Modeled in gem5 on Armv8-A SVE, GEMM kernels reach a 1.44x geometric-mean speedup and 40% lower memory-subsystem energy at VL=16; full layers with FlashAttention reach up to 1.78x and 45%, at 2.02% area overhead relative to the L2.
When CPUs widen SIMD vectors from 512 to 2048 bits, compute grows but L1 data cache capacity does not, so data shuttles between L1 and L2 and throughput stalls. CacheFlex uses a lightweight ISA extension to convert part of the private L2 into a thread-private scratchpad at runtime, letting software address slots directly and bypass address translation, associative tag lookup, and coherence probes. Modeled in gem5 on Armv8-A SVE, GEMM kernels reach a 1.44x geometric-mean speedup and 40% lower memory-subsystem energy at VL=16; full layers with FlashAttention reach up to 1.78x and 45%, at 2.02% area overhead relative to the L2.
University of Washington Rutgers, The State University of New Jersey
Jingqun Zhang（First author）
CacheFlex Scratchpad Memory Instruction Set Extension General Matrix Multiplication (GEMM)FlashAttention
Transactions of the Association for Computational Linguistics Oct 7, 2026
A model splits a word's two meanings early in pretraining: two heads carry 14M, while 410M spreads the job across steadier ones
During pretraining of Pythia language models, word-sense disambiguation improves in sharp steps around 1,000 to 2,000 training steps, two layer-3 attention heads in 14M track that improvement and their ablation measurably hurts it, while 410M spreads the function across more heads that hold up better under position and part-of-speech perturbations.
University of California San Diego Rutgers, The State University of New Jersey
Pamela D. Rivière（First author）
Attention Heads Word Sense Disambiguation Pythia Language Models RAW-C Dataset Mechanistic Interpretability
arXiv Oct 5, 2026
CLIMB improves multimodal RAG with a confidence-guided complementary evidence pool, consistently beating retrieval-augmented baselines on Encyclopedic-VQA and InfoSeek
The authors propose CLIMB, a training-free inference-time multimodal RAG framework that first builds a compact complementary evidence pool with an MMR-style objective balancing query relevance and passage-level redundancy, then performs confidence-controlled refinement within this fixed pool: an R/E/C critic scores passages by relevance, evidence specificity, and cross-modal alignment, while an evidence-grounded confidence estimator accepts an updated answer only when estimated confidence increases, yielding a simple stopping criterion; on Encyclopedic-VQA and InfoSeek it consistently improves over retrieval-augmented multimodal baselines, and ablations indicate complementary pooling, critic-based scoring, and iterative confidence-controlled refinement each contribute.
The authors propose CLIMB, a training-free inference-time multimodal RAG framework that first builds a compact complementary evidence pool with an MMR-style objective balancing query relevance and passage-level redundancy, then performs confidence-controlled refinement within this fixed pool: an R/E/C critic scores passages by relevance, evidence specificity, and cross-modal alignment, while an evidence-grounded confidence estimator accepts an updated answer only when estimated confidence increases, yielding a simple stopping criterion; on Encyclopedic-VQA and InfoSeek it consistently improves over retrieval-augmented multimodal baselines, and ablations indicate complementary pooling, critic-based scoring, and iterative confidence-controlled refinement each contribute.
Rutgers, The State University of New Jersey
Hang Gao（First author）
CLIMB Retrieval-Augmented Generation Visual Question Answering Multimodal Large Language Models
arXiv Oct 1, 2026
Self-evolving search agents develop "co-cheating": internal reward rises while real correctness stalls, and CrossFit cuts false-agreement mass from 6.1%/8.8% to 3.0%/3.7%
The work diagnoses a failure mode it calls co-cheating in self-evolving search agents, where a proposer and solver increasingly agree on shared errors so internal reward improves without matching external correctness, and introduces multi-sample verification (MSV) plus the main method CrossFit (splitting the proposer's source documents into folds and scoring each fold's questions with an auxiliary solver trained only on the other fold), reducing false-agreement mass from 6.1%/8.8% to 3.0%/3.7% on Qwen3.5-4B/9B and improving seven-benchmark average Cover-EM over standard coupled self-evolution by 8.8/8.4 points.
The work diagnoses a failure mode it calls co-cheating in self-evolving search agents, where a proposer and solver increasingly agree on shared errors so internal reward improves without matching external correctness, and introduces multi-sample verification (MSV) plus the main method CrossFit (splitting the proposer's source documents into folds and scoring each fold's questions with an auxiliary solver trained only on the other fold), reducing false-agreement mass from 6.1%/8.8% to 3.0%/3.7% on Qwen3.5-4B/9B and improving seven-benchmark average Cover-EM over standard coupled self-evolution by 8.8/8.4 points.
University of Michigan University of California San Diego McGill University King Fahd University of Petroleum and Minerals Rutgers, The State University of New Jersey
Meijia Chen（First author）
Co-Cheating Cross-Fitted Proposer Feedback Multi-Sample Verification Self-Evolving Search Agent
arXiv Oct 1, 2026
RIDE extrapolates RL teachers' hidden-state residuals and matches or beats the teacher on average across four base/teacher pairs
The work proposes RIDE (RL-Induced Direction Extrapolation): on student-generated trajectories it computes the layerwise hidden-state residual between an RL-trained teacher and its pre-RL checkpoint, then regresses the student's hidden states toward targets displaced beyond the teacher along that residual; across four base/RL-teacher pairs (R1-Distill-1.5B, Qwen3-4B, Llama-3.2-3B, Phi-4-mini) RIDE's mean Avg@16 on AIME24, AIME25 and AIMO approaches or exceeds the RL-trained teacher on every pair, the only compared method whose mean does so, and it consistently outperforms output-space extrapolation.
The work proposes RIDE (RL-Induced Direction Extrapolation): on student-generated trajectories it computes the layerwise hidden-state residual between an RL-trained teacher and its pre-RL checkpoint, then regresses the student's hidden states toward targets displaced beyond the teacher along that residual; across four base/RL-teacher pairs (R1-Distill-1.5B, Qwen3-4B, Llama-3.2-3B, Phi-4-mini) RIDE's mean Avg@16 on AIME24, AIME25 and AIMO approaches or exceeds the RL-trained teacher on every pair, the only compared method whose mean does so, and it consistently outperforms output-space extrapolation.
Zhejiang University Rutgers, The State University of New Jersey
Hao Li（First author）
RIDE (RL-Induced Direction Extrapolation)On-Policy Distillation Representation Distillation
Surgical Infections Sep 14, 2026
Preventing Surgical Site Infections: Ethical and Operational Imperatives for the Global Health Agenda
This article reviews evidence and implementation experience on surgical site infection (SSI) prevention across infection prevention and control (IPC), antimicrobial stewardship (AMS), antimicrobial resistance (AMR), quality improvement, surveillance, safety culture, human factors, and emerging technologies, concluding that multimodal, context-adapted IPC interventions can substantially reduce SSI incidence without major capital investment, that persistent gaps remain in infrastructure, guideline implementation, surveillance, and environmental controls, that AMR further emphasizes the need to integrate AMS into surgical safety frameworks, and that artificial intelligence and objective diagnostic technologies may improve surveillance and early SSI detection, so that SSI prevention is both a
This article reviews evidence and implementation experience on surgical site infection (SSI) prevention across infection prevention and control (IPC), antimicrobial stewardship (AMS), antimicrobial resistance (AMR), quality improvement, surveillance, safety culture, human factors, and emerging technologies, concluding that multimodal, context-adapted IPC interventions can substantially reduce SSI incidence without major capital investment, that persistent gaps remain in infrastructure, guideline implementation, surveillance, and environmental controls, that AMR further emphasizes the need to integrate AMS into surgical safety frameworks, and that artificial intelligence and objective diagnostic technologies may improve surveillance and early SSI detection, so that SSI prevention is both a
Universidad de Buenos Aires Yamagata University University of Bologna National Hospital Abuja Catholic University of Health and Allied Sciences Ospedale di Macerata Hospital Universitario La Paz Kijabe Hospital
Massimo Sartelli（First author, Corresponding author）
Infection prevention and control Antimicrobial stewardship Antimicrobial resistance Surgical site infection prevention

You’re all caught up