강화학습을 위한 3차원 물리 기반 시뮬레이터 설계 및 구현

Abstract

In recent years, the development of deep artificial neural network technology has made it possible to deal with high-dimensional information without domain knowledge. As a result, reinforcement learning has been actively researched in the field of solving problems by interacting with three-dimensional environment such as autonomous vehicle driving and drone. Recent reinforcement learning studies have been proving the validity of the results through computer simulators such as Bellemare's ALE and OpenAI's Gym. Learning and experimenting with a robotic agent in the real world is due to financial losses such as damage, accidents, and data collection costs. In the physical simulator, however, floating point or numerical integration errors can cause errors in the physical engine and bugs in various programs. The problem with these simulators is the agent can learn through physically impossible movements in reality, which means that users need to visually check and monitor them because wrong results are learned. It is also necessary to restore an agent if it learns from the wrong information. However, the tools for implementing the existing reinforcement learning environment do not provide functions related to computer graphics and do not support the restoration function. Therefore, in this paper, we provide an implementation method to visualize the screen with computer graphics, which can be reinforcement learning, and a 3D physics based simulator architecture to restore experience data and policy weights. By providing the above architecture, it is possible to reduce the cost of simulator implementation time and expect stable reinforcement learning through the restoration function.