Lyapunov-Guided Deep Reinforcement Learning for Stable Low-Latency Task Offloading in Mobile Edge Computing

Abstract

Fifth-generation (5G) networks and Internet of Things (IoT) face critical challenges in balancing latency minimization and energy consumption while ensuring data queue stability in dynamic environments. This paper proposes an innovative hybrid framework that integrates stochastic optimization based on Lyapunov theory with the Deep Deterministic Policy Gradient (DDPG) algorithm—termed LG-DDPG (Lyapunov-Guided DDPG). The offloading problem is formulated as a constrained Markov Decision Process (MDP), and a theoretical upper bound for the system cost is derived. The proposed framework employs the Drift-Plus-Penalty (DPP) technique to decouple the long-term stability constraint into instantaneous subproblems, which are then solved by a DDPG-based actor-critic architecture with hidden layers (256, 128, 64) using ReLU and Tanh activations. Comprehensive simulations—averaged over 10 independent runs—demonstrate that LG-DDPG achieves a 35–45% reduction in total system cost compared to state-of-the-art baselines, with an average latency of 45.2 ± 1.3 ms and energy consumption of 2.1 ± 0.1 J, outperforming DRL-only (52.8 ms), Lyapunov-only (58.3 ms), and PSO (65.7 ms) approaches. The system scales linearly to 100+ devices with O(N) complexity, with rigorous mathematical proofs confirming queue stability and neural network convergence.