RIDGE lets one game agent switch play styles mid-episode, so a single run unlocks both the wood pickaxe and sword

RIDGE lets one agent cover both the wood-tier crafting and combat branches in a single training run, whereas agents trained per persona each take only one branch. Automated playtesting previously paired each play style with its own fixed reward, trained one agent per persona, and took the union of behaviors at deployment, costing linearly in personas and losing the within-episode dynamics that motivated personas. On Crafter, all three conditions reach Place Table, the shared prerequisite; afterward Craftsman makes the wood pickaxe while the wood sword stays near zero, Warrior is the mirror, and RIDGE crosses the branch within a single trained agent to reach both the wood pickaxe and the wood sword, sitting below each specialist on its own branch as the expected coverage/specialization trade-off.

The blending weights are computed from game state rather than supplied externally, so the reward composition follows behavioral need. Earlier multimodal imitation and multi-objective RL treat the blending coefficient as external configuration, while RIDGE computes weights internally from slow-moving state variables such as health, hunger, crafting progress and enemy proximity. Each persona first gets a raw weight from a sigmoid centered at a per-persona bias, then a softmax normalizes the weights onto the simplex summing to one; a global sharpness parameter controls transition abruptness, collapsing to a uniform mixture as it approaches zero and approaching hard switching as it grows.

Multiple value heads isolate value-function learning from the shifting reward mixture. PPO's value function needs a quasi-stationary signal, and hard in-episode reward switches break convergence, which is why prior work avoided changing rewards within an episode. On top of a shared encoder, one value head per persona is trained with generalized advantage estimation against that persona's fixed returns; RIDGE's value loss stays comparable to Explorer-only and below Craftsman and Warrior throughout training, while Survivor-only sits an order of magnitude higher.

The sharpness ablation separates the effect of state-conditioning from the persona set itself. The uniform mixture uses the same four persona rewards with equal weights and no state-conditioning, corresponding to the uniform-weight case of linear-scalarization multi-objective RL. The zero-sharpness uniform condition clearly underperforms all state-conditioned variants throughout training; within the conditioned range, sharper settings edge out the default, though the differences sit within the seed bands.

The results come from experiments on Crafter under the benchmark's standard environment-step training budget, with four personas aligned to the collection, crafting, combat and survival achievement categories; all conditions share encoder, optimizer, rollout length, batch size, generalized advantage estimation coefficient, PPO clip and entropy bonus, differing only in reward signal, blending sharpness and critic head count. The authors position the current results as demonstrations of the blending mechanism's dynamics and plan a larger multi-seed sweep for an extended version.

The authors note substantial seed variance in PPO results and present min/max bands rather than confidence intervals, which additional seeds would replace. A hard-switch run has not been performed, so sharpness only approaches rather than reaches the switching-stability dilemma Li describes; a single-head version of RIDGE is also untested, so the separate contributions of the multi-head critic and the smooth blend remain unseparated. Crafter is currently the only tested environment.