Object Following Control of a Six-legged Walking Robot Using Kinect Camera Sensor

Abstract

The development of object following systems using camera sensors has been a topic being researched and reported widely in the field of autonomous robotics. The problems in the development of such systems are choosing robot, selecting camera for object detection, maintaining stability of the robot and designing appropriate control law to control the robotic system. To deal with these problems, this thesis proposes a new type of object following algorithm applied to the six-legged walking robot (6LR) using a Kinect camera sensor based on a Kalman filter and a backstepping control method. To do this task, the following steps are executed. Firstly, a real 6LR with four joint legs is developed with several interconnected devices such as servomotors, Micro Control Unit (MCU), Bluetooth, etc. The Kinect camera sensor is installed on the 6LR platform for image processing. In this thesis, a candidate blue colored object is used as an object for detection and following. For the 6LR, the tripod walking gait is chosen to control six legs in this thesis. The walking cycle of tripod gait is presented with swing phase of 3 legs and retract phase of other 3 legs. A graphical user interface (GUI) is developed using Visual studio C# software to send control commands to the robot system and display its results. Secondly, a system modeling is described to understand the behavior of the 6LR system as follows. First, a forward kinematic modeling of one leg with four joints and four links is presented. The Denavit-Hartenberg (DH) convention is adopted to define the modeling parameters which allow the forward kinematics function by composition of the individual coordinate transformation. Second, a kinematics of 6LR is presented to derive the homogeneous transformation matrices from body coordinate frame to each leg coordinate frame. Third, kinematic modeling of the 6LR with a candidate object is described. Fourth, kinematic modeling of 6LR to define control inputs such as translational velocity and angular velocity for object following control is presented. Thirdly, a color-based object detection algorithm is applied to the RGB images obtained by the Kinect camera sensor to detect a blue colored candidate object and to find its center coordinate in the RGB image frames. Using simple trigonometry and Kinect depth data, the pixel position coordinate is converted to the real world coordinate in mm. Then, a Kalman filter algorithm based on an object motion model is used to track the real world position and velocity of the moving candidate object. Fourthly, based on the system modeling, controllers for the 6LR are designed using backstepping method and Lyapunov stability criterion. The walking control of 6LR is designed using tripod walking gait and differential kinematics algorithm instead of inverse kinematics, and then the object following controller is designed to make the 6LR follow the candidate object. Finally, simulations and experiments are conducted to verify the effectiveness and performance of the proposed controller for following the moving candidate object. The results show that the proposed controller makes the six-legged robot follow the candidate object very well using color-based object detection algorithm and Kalman filter tracking algorithm.