고차원 자료에 대한 심층 신경망에서의 초매개변수 조율 및 영향점 분석

Abstract

In deep neural network(DNN) hyper-parameters, set directly by the user, have significant effect on model performance and their optimization problem becomes more important. There have been many studies on hyper-parameter selection, but still limitations. In this study, we propose a hybrid-search that combines the design of experiment(DOE) methodology and the grid search in order to overcome the limitations of the hyper-parameter optimization problem. The method presented in this study can avoid the time-consuming problem through screening in advance the hyper-parameter values ​​using the DOE methodology. We then consider many combinations of the screened values using the grid search. Furthermore, we investigate the optimal number of hidden layers and hidden nodes in terms of the error rate for the high-dimensional data, and compare the case of all the input variables with that of the selected input variables, changing the number of hidden layers and hidden nodes. Finally, the impact of influential observations can be very significant for high-dimensional data where the number of variables is even larger than that of observations. In this study, we propose a deep neural network-influence box plot(DNN-influence box plot) as a graphical method for diagnosing the influential observations. It is shown that the influential observations can be diagnosed through three high-dimensional examples. Also, we compare the influence plots and the HIM plot which are useful graphical tools for diagnosing the influential observations in high-dimensional data.