Control of a Mobile Picking Robot for Path Tracking and Object Grasping Using a Kinect Stereo Camera

Abstract

Motion planning such as mobile manipulation must be able not only to move in safety through their environments but also to manipulate objects in their environments. This dissertation presents experimental validation on control of a mobile picking robot (MPR) for path tracking and object grasping using a Kinect stereo camera. To do this task, the followings are done. Firstly, the system configuration of the mobile picking robot is described. The mobile picking robot is considered as two subsystems such as a manipulator and a mobile platform. The Kinect stereo camera is mounted on the front side of the mobile platform to capture both RGB and depth (RGBD) images at the speed of 30 frames/sec. Secondly, modeling of BLDC motor is introduced. Mathematical kinematic and dynamic modelings of the mobile platform and the manipulator are presented. The Denavit-Hartenberg (DH) convention is adopted to define the modeling parameters which allow the construction of the forward kinematics function by composing the individual coordinate transformations. Thirdly, the RGBD image from the Kinect stereo camera sensor is utilized to recognize objects for obstacle avoidance, path planning and picking objects. To ensure robust and correct object identification, two algorithms to be effective for object detection are employed: Speeded Up Robust Features (SURF) algorithm first for detecting object’s features and Fast Library for Approximate Nearest Neighbors (FLANN) algorithm for matching object’s features. For object grasping, a rectangle representation algorithm based on depth map data from the Kinect stereo camera sensor is proposed. By converting RGBD image into 3D point clouds, an algorithm for localizing handle-like grasp affordances is proposed. The main idea is to search the point cloud for neighborhoods that satisfy handle-like grasp affordances and can be grasped by the end-effector of the manipulator. Fourthly, a SLAM algorithm based on EKF is adopted to solve the global localization problem. The positions of the landmarks are obtained using a Kinect stereo camera sensor. The position of the mobile platform is predicted using EKF prediction step based on data of encoder and laser sensor. D* Lite algorithm is used to generate a path for the mobile platform to go from a start point to a goal point with avoiding unknown obstacles using information obtained from the Kinect stereo camera sensor. Fifthly, a robust servo controller design based on a polynomial differential operator method is applied for the mobile platform to track the path generated by the D* algorithm. This robust servo controller is also applied for BLDC motor for speed control of the BLDC motor with a step type of disturbance and 3 types of the references such as step, ramp and parabola. The differential kinematics algorithm is applied for the end-effector of the manipulator to approach the desired grasing position. Finally, the effectiveness of the proposed algorithms and controller is verified by using simulation and experiment.