N-DQN: Study on the Implementation and Research of Hierarchical Parallel Reinforcement Learning Model 


Vol. 44,  No. 10, pp. 1961-1974, Oct.  2019
10.7840/kics.2019.44.10.1961


PDF Full-Text
  Abstract

This paper considers various factors and conditions that correlate with the performance of the Reinforcement Learning, and defines the N-DQN model to improve them. N-DQN is a concept of applying and extending the architecture of HDQN, which layers multiple actors and has a structure in which operations are performed simultaneously/parallel through policy-based behavior management. Episodes acquired by each actor with action are stored in shared Replay Buffer, and various reinforcement learning enhancements such as Prioritized Experience Replay and segmentation of the reward acquisition period are applied to it. The implemented N-DQN showed about 3.5 times higher learning performance than the Q-Learning algorithm in the Reward-Sparse environment and about 1.1 times faster than DQN in attaining goal. Additionally, through the implementation of preferential experience regeneration and segmentation of reward acquisition period, problems of existing reinforcement learning models such as Positive-Bias have hardly occurred. However, due to architecture feature of using lot of actors in parallel, working on improving performance through light-weightening in the future is needed. As a cornerstone of our future work for light-weightening and improving proposed architecture, this paper describe specific detail of proposed architecture structure, various algorithm used and ways to implement it.

  Statistics
Cumulative Counts from November, 2022
Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


  Cite this article

[IEEE Style]

T. Jung, S. Kim, K. Kim, "N-DQN: Study on the Implementation and Research of Hierarchical Parallel Reinforcement Learning Model," The Journal of Korean Institute of Communications and Information Sciences, vol. 44, no. 10, pp. 1961-1974, 2019. DOI: 10.7840/kics.2019.44.10.1961.

[ACM Style]

Tack-hyun Jung, Sang-won Kim, and Keecheon Kim. 2019. N-DQN: Study on the Implementation and Research of Hierarchical Parallel Reinforcement Learning Model. The Journal of Korean Institute of Communications and Information Sciences, 44, 10, (2019), 1961-1974. DOI: 10.7840/kics.2019.44.10.1961.

[KICS Style]

Tack-hyun Jung, Sang-won Kim, Keecheon Kim, "N-DQN: Study on the Implementation and Research of Hierarchical Parallel Reinforcement Learning Model," The Journal of Korean Institute of Communications and Information Sciences, vol. 44, no. 10, pp. 1961-1974, 10. 2019. (https://doi.org/10.7840/kics.2019.44.10.1961)

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)






pISSN : 1226 - 4717
eISSN : 2287 - 3880