Latency‑Aware Workflow Placement in Edge‑Cloud Environments: A Reinforcement Learning Approach

Abstract

The placement of workflows in edge and cloud environments is difficult because the two have different needs (such as low latency and limited resources at the edge). Our proposed methodology to dynamically place a workflow onto a given set of resources is based on using reinforcement learning (RL) techniques to minimize end-to-end latency while adhering to resource and bandwidth constraints. We model this problem using a Markov Decision Process (MDP), where the state indicates the task dependencies, available resources, and network conditions, while the actions correspond to deciding which tasks will be mapped to either the edge nodes or cloud nodes. An RL agent is implemented using a deep Q-network (DQN) with experience replay, allowing it to learn placement policies that can adapt to changing workloads and share resources. We conducted a rigorous series of simulation studies using realistic workflow topologies (e.g. genome assembly, video analytics and IoT processing), which show that the RL-based approach reduces average latencies by 28-41% compared to heuristic-based approaches such as greedy, round-robin and HEFT, and improves on-time delivery of tasks by approximately 35%. The framework contains both a discrete-event simulator that is used during training and evaluation of the algorithm, and a fast inference module that requires less than 1 ms per decision (making it suitable for executing the dynamic placements online). We perform convergence analysis and sensitivity studies with respect to edge node heterogeneity, confirming that our methodology is robust. The implementation is available as an open-source toolkit to facilitate reproducible research.