Real-Time Driver Fatigue Monitoring by Facial Image and Bio-signal Data Fusion in Android-based Smartphone Device

Abstract

For the past decade, it is well defined in the literature that fatigue is one of the most prospective factor in affecting the driver behavior. This dissertation proposed a method to monitor driver safety by analyzing information related to fatigue based on two distinct methods: eye features monitoring and bio-signal processing. The process involved fusion of attributes including eye movements, photoplethysmography (PPG) and electrocardiography (ECG) signals that are assigned as input variables to an inference analysis paradigm framework. The outcome of the inference network indicated the driver aptitude level and is updated dynamically. The monitoring system is practically designed in Android-based platform smartphone device where it can receive all the sensory information from the dedicated sensors via wireless communication. The transmission mediums can be configured to either Bluetooth or low power ZigBee wireless communication based on the system structure. However, it is critical that several sensors are integrated and synchronized for a more realistic evaluation of the driver behavior. The sensors applied included an integrated smartphone device video-sensor to capture the driver facial image and bio-signal sensors that placed on the steering wheel to acquire driver PPG and ECG signals. Several inference networks are implemented which are dynamic Bayesian network (DBN), Fuzzy Logic system (FL), and integration of both into Fuzzy Bayesian network (FBN) to derive the driver vigilance index in the system. A fake incoming call and text messaging warning services are designed to alert the drivers if they are suspiciously fell into low arousal state. The manifold testing of the system demonstrated the practical benefits of multiple features and their fusion, particularly with discrete methods to enable a more authentic and effective fatigue detection.