Skip to main content
Search

# Markov Decision Process

4 related articles

Subscribe to updates
  1. arXiv

    Multi-step agent safety stops checking each step alone: TAS caps tail risk with one shared budget, cutting average attack success to 45.15%

    Risk in multi-step LLM agents propagates along the workflow, and step-by-step independent verification does not see that propagation. TAS models workflow safety states as a Markov decision process, measures rare but high-cost tail risk with a spectral risk measure, and constrains each step's decision with one budget shared across steps. On Agent Security Bench, TAS reaches average attack success rates of 45.15% under direct prompt injection and 29.73% under indirect prompt injection, below the eleven defense baselines it compares against.

    National University of Defense Technology
    Shilong Pan（First author）
    Tail-risk-aware Agent Safety (TAS)Agent SafetySpectral Risk MeasuresMarkov Decision ProcessLLM Agents
  2. arXiv

    Image-editing agents stop guessing which model to retry: RetryRoute plans over failure history and beats the strongest prior baseline by 2.8 points at the main budget

    Web agents edit images by calling open editing models in a critique-and-retry loop, yet a retry after a rejection succeeds only 17–53% as often as a first try, while existing routers treat each retry as an independent draw. RetryRoute estimates each model's next-attempt success probability from the request's failure history, plans retries, escalations and abandonment over failure states, and charges live queue waits to the cost. At the main budget it exceeds the strongest of eleven prior baselines by 2.8 points on average over eight groups, and on a three-GPU serving testbed one setting raises the share of edits meeting the service objective by 5.0–6.3 points over the best prior baseline at every load.

    Beijing Normal UniversityNanyang Technological UniversityBeijing Normal University, Zhuhai
    Hanshuai Cui（First author）
    RetryRouteAgentic Image EditingModel RoutingMarkov Decision Process
  3. arXiv

    Data center siting is locked in for years: one multi-objective RL agent lets planners retune water, power, carbon and reliability at inference time

    AI data center siting, power procurement and cooling are still settled once by static single-point scoring on day-zero numbers, though water, power and carbon drift over a 10- to 20-year asset life and commitments bind for years. The build-out becomes a tiered Markov decision process: strategic commitments occur only in discrete windows, carry switching costs and take effect after source-dependent lead times, dispatch uses only capacity that exists, and a preference-conditioned multi-objective agent returns a policy family steerable at inference time across cost, carbon, water and reliability. On real site data, preference conditioning gives a confirmed, steerable trade-off on both objective pairs across three commitment regimes, and lock-in covers the frontier more evenly.

    New York University
    Satyapragnya Kar（First author）
    Markov Decision ProcessAI Data CenterCounterfactual Rollout Model
  4. arXiv

    Mirror-IRS-assisted hybrid RF/VLC network: two-agent DRL jointly tunes selection, power and mirror orientation, holding fairness above 0.93

    For an indoor IRS-assisted hybrid RF/VLC downlink network, this work formulates joint proportional-fairness maximization over RF/VLC technology selection, power allocation and IRS mirror roll/yaw orientation as an MDP and solves it with a cooperative two-agent DRL framework under centralized training with decentralized execution, where one agent handles selection and power and the other handles mirror orientation; simulations converge to 350 Mbps, within 10% of the 392 Mbps exhaustive-search bound, outperform DDPG and DQN, and keep Jain's fairness index above 0.93 as user count grows.

    Ahrar N. Hamad（First author）
    IRS-Aided Hybrid RF/VLC SystemsCooperative Multi-Agent Deep Reinforcement LearningIntelligent Reflecting SurfaceMarkov Decision ProcessCentralized Training with Decentralized Execution

You’re all caught up