디지털 트윈 환경에서 순환 신경망과 강화학습을 이용한 드론의 객체 추적 방법

Abstract

Object tracking in an image is a method of detecting an object in every frame of an image acquired with a photographing device and tracking it according to a change in position and size in the image. Recently, a lot of research has been done on methods of tracking objects in drone (uav) images. Unlike general photographing equipment that is fixed on the ground, drones can utilize various photographing methods with free movement in the air. Therefore, it is possible to obtain an image of the shooting result that is different from the general image captured by a camera, camcorder, or CCTV. Also, although the image-based object tracking method is a long-standing research topic, nevertheless, there are many variables to be considered and problems to be solved in addition to the characteristics of the object, such as occlusion or blur, background clutter, and changes in the shooting environment. In particular, since object tracking is a method of tracking objects by accurately recognizing objects in each frame of a continuous image, if an object cannot be recognized in a few frames. There is a problem in that object tracking fails not only in the frame but also in the entire continuous image. A lot of research has been done to solve these problems and improve performance. Mainly, many studies have been made on implementing an object tracking model through a method that can effectively learn object features, constructing a training data set with images taken from various objects and backgrounds, or increasing the amount of images. However, if the data set is increased indefinitely, overfitting occurs, which prevents proper learning of the object and degrades performance. There is a limit to constructing a training data set. In this paper, we proposed drone-based object tracking method using recurrent neural network and reinforcement learning in digital twin environment. In the proposed method, the LSTM(Long Short-Term Memory) network is used for the recurrent neural network, and the Deep Q-Network (DQN) is used for reinforcement learning. In addition, we propose a method to freely create and simulate drone images by implementing a virtual environment, one of the main functions of a digital twin, to construct a learning data set that considers the specificity of the image of drone images. First, the performance of the existing object tracking model was largely dependent on the performance or learning results of the detector because the main goal was to improve the performance of the detector that recognizes objects rather than constructing a network considering continuity in the tracking process. The proposed object tracking model learns the continuity of changes such as the size and location of the object to be tracked from the input image through the LSTM network. And reinforcement learning is performed to select an action to track an object using the learned information and to update the reward function. And as learning proceeds, an optimal target Q value for tracking an object is obtained. DQN is set as a target network that should achieve the corresponding Q value, and the predicted network is trained to be as close to the target network as possible. The learning goal of the proposed object tracking model is to learn the behavior of recognizing an object in continuous images and continuously moving and tracking the bounding box according to the change of the target object. Second, since the existing public image data set consists of images taken with general shooting equipment, it is not possible to reflect all the characteristics of the drone shot image. There is not much quantity to compose. In addition, in order to take a large amount of images with an actual drone, there are practical difficulties such as regulations and safety issues related to drone operation, and technical limitations of the drone aircraft. In this paper, we create and experiment with the training data set of the proposed object tracking model as a simulation function through a virtual environment among various functions of the digital twin. The simulation environment is a virtual city implemented in 3D, and the drone for taking images to be used for learning and experiments is also implemented in 3D. The drone flies with real physical characteristics, and takes pictures while automatically flying in a virtual city environment using an attached camera. In the virtual city, people and cars are implemented in 3D as tracking target objects. If the simulation function of the digital twin is utilized, an image can be created in a virtual environment under shooting conditions that are almost similar to the real one, or images can be created freely by setting shooting conditions that are difficult in reality. Therefore, it has an advantage in terms of data set configuration. The importance of continuity in object tracking and the utility of digital twins are explained through the object tracking model and data set creation and simulation methods proposed in this paper. and try to prove the novelty of the proposed method through experiments with the existing object tracking model. Object tracking results between the proposed method and the existing ADNet 및 ASRL method were compared with the test continuous image sets of two public data sets, VisDrone2019 and OTB-100. Compared to the ADNet 및 ASRL object tracking model, the proposed object tracking model confirmed better tracking performance in images taken in the real environment.