Research on work support AR interface for people with intellectual disabilities

Abstract

We have identified four problems for mentally retarded users of augmented reality (AR) devices. The first is that they do not notice the information presented to them, and the second is that they have difficulty pressing buttons in the AR space. The third is the inability to accurately assess one's own fatigue level. The fourth is the possibility of increased fatigue and decreased safety due to the head turning in the direction of the viewer. To solve the first problem, we propose two ways to capture the user's attention: one is to place the information window near the user's gazing point and dynamically adapt the window based on eye tracking data. The second is to make the information window blink. As a solution to the second problem, we propose a method that detects the user's button- pressing gesture, and when the gesture is detected, the system presses the button on behalf of the user. As a solution to the third problem, we propose a method that monitors user information and detects changes that occur during fatigue. For the fourth problem, we propose a method to control the user’s head direction by elucidating the relationship between the position of the presented information and the change in head direction. The effectiveness of the proposed method for problems one and two was tested in a user study with seven participants with different degrees of intellectual disability. An evaluation experiment is planned to verify the effectiveness of the proposed method for problems 3 and 4.