Warmup Hurts: A Two-Stage Schedule Ablation and Physical-Perturbation Robustness Analysis of Lion in PPO

Abstract

Despite AdamW’s continued status as the default optimizer for PPO, its two-buffer moment state (2P memory) costs twice what the sign-based optimizer Lion requires (1P). Lion’s update magnitude, however, tracks the instantaneous learning rate directly and exclusively, without anything that plays the role of AdamW’s internal self-normalization when a schedule decays; this structurally predisposes it to chattering — the sliding-mode-control oscillation documented in discontinuous, sign-based control laws — and should leave its competitiveness against AdamW schedule-sensitive in a way AdamW’s is not. We isolate this sensitivity with a two-stage ablation on PPO in four MuJoCo continuous-control environments (HalfCheetah-v4, Walker2d-v4, Ant-v4, Hopper-v4; n = 15 seeds/arm, Welch’s t-test with Holm–Bonferroni correction), structurally decomposing three Lion arms (lion_raw: no schedule; lion_cosine: cosine decay from step 0; lion_warmup_cosine: linear warmup and plateau, then the same cosine decay) into two independent, additive comparisons. In the three passively stable environments (HalfCheetah-v4, Walker2d-v4, Ant-v4), decay is beneficial and warmup is significantly detrimental, reducing mean return by 39–55% and increasing chattering by 12–18% relative to decay alone, with the same directional pattern — inverted for warmup, the opposite of its usual supervised-learning justification — confirmed in post-training policy entropy. In the minimally stable Hopper-v4, no single-step ablation reaches significance, though the combined lion_raw-vs-lion_warmup_cosine contrast does. We further report an exploratory, post-hoc physical-perturbation (added mass, sensor noise) robustness analysis: mean post-fine-tune return recovery ranges from 54.7% (Walker2d-v4, worst) to 113.7% (Ant-v4, best, an outright improvement), and appears governed by a property distinct from clean-condition passive stability.