A study on Image Analysis for Human Detection Using CCD and Thermal Infrared Cameras

Abstract

In the current context of increased surveillance and security, the necessity has emerged for more sophisticated surveillance systems, for instances around buildings. The desire to provide robust and accurate surveillance information has led to considerable research on methods to integrate information from different sensors. One idea that is promising relies on the use of pairs of video (visible spectrum) and thermal infrared cameras distributed around premises of interest. Since it is not practical to have humans observing the resulting images in real-time, it is proposed to add an "intelligent fusion/detection step" to the system so that human observers are involved only in case "abnormal situations" occur. This research includes investigation of possible solutions, design, development and implementation of a pedestrian detection system, processing data from infrared and visible video sources in real-time. Design requirements include processing at full frame rate as well as low memory and system resource consumption. The memory utilization is one of the major concerns since high demand for memory resources is a critical aspect in most image processing applications. Therefore the main goal of this work is to help and improve real-time surveillance in security systems. To automate the system, a dedicated number of general purpose image processing techniques are required. These techniques include background separation, acquisition noise removal and object detection through connected component labeling. They are all discussed and addressed in individual chapters. The first step in the proposed study is to collect a database of known scenarios both indoor and outdoor with moving pedestrians. These image sequences are captured for both CCD and Thermal infrared cameras where the video and thermal data are synchronized, geometrically corrected and temperature calibrated. The next step is to develop a segmentation strategy to extract the regions of interest corresponding to pedestrians in the images. In order to achieve the region of interest, we apply Gaussian Mixture model, to eliminate the background. Next, these regions are grouped from image to image separately for both video and thermal sequences before a fusion algorithm proceeds to track and detect humans. This insures a more robust performance. Our method for thermal infrared imagery is based on background reduction and brightness saliency map classifier scheme. Brightness saliency information is combined into a contour saliency silhouette. Finally, specific criteria of size and temperature relevant to humans are introduced. For the purpose of object detection and feature extraction, a connected component labeling technique was employed, based on a single pass approach to fulfill real-time processing requirement. The system was implemented, verified and tested on an Intel i7-2600 @ 3.60GHz CPU, with 4 GB RAM. The programs were implemented using both MATLAB programming software and OpenCV in visual studio 2010. Details and limitations of the specific implementation are discussed. An overview of experimental human detection and pedestrian tracking results are provided. The thesis concludes with system analysis and suggestions for future work.