TY  - JOUR
T1  - Reinforcement Learning-Based Scheduler with Dynamic Precedence in Deterministic Networks
AU  - JihyeRyu
AU  - GyudongPark
AU  - JuhyeokKwon
AU  - JinooJoung
JO  - The Journal of Korean Institute of Communications and Information Sciences
PY  - 2023
DA  - 2023/1/14
DO  - 10.7840/kics.2023.48.4.413
KW  - reinforcement learning
KW  - deep learning
KW  - deterministic networking
KW  - Q-learning
KW  - double deep Q-network
KW  - precedence
AB  - Smart industry, metaverse, digital-twin, and military applications require deterministic data delivery in large scale networks. This paper proposes reinforcement learning-based scheduling that assigns dynamically different precedences to the flows, in addition to the flow's class or priority, and determines the scheduling algorithm according to the flow's precedence. In the proposed reinforcement learning-based scheduling algorithm with two precedence queues, the reinforcement learning agent takes two actions that assigns the precedence of flows according to a specified criterion and selects a scheduling algorithm. Depending on the purpose of the network, any factor with high importance could be a criterion for determining the precedence. In this study, the deadline required by the flow is designated as the major factor for precedence decision. By utilizing DDQN (Double Deep Q-Network), a deep learning-based reinforcement learning model, the precedence and the scheduling algorithm are determined by observing the state of the network and selecting an action at each decision period with a fixed length. In the network simulator developed for the study, it was confirmed that the DDQN agent showed better performance than various heuristic algorithms.