Driver intent and fatigue prediction system using spatio-temporal facial expression patterns and head pose dynamics

This paper introduces a strong and real-time system for predicting a driver's intention and detecting fatigue by looking at facial expressions and head movements over time. The system uses a step-by-step process that starts with preparing images. This includes making the images clearer with histogram equalization, detecting the face using MediaPipe landmarks, and normalizing the images to make them more consistent across different frames. To find the face efficiently and accurately, the system uses a lightweight MediaPipe face detection model, which is suitable for use in cars and can detect detailed facial features. From there, the system extracts features like the Eye Aspect Ratio (EAR), Mouth Aspect Ratio (MAR), blink rate, and yawning frequency. It also measures head posture parameters such as pitch, yaw, and roll using SolvePnP with 3D face modeling. These features are then modeled over time using a deep learning approach that combines CNN and LSTM networks. The CNN, based on ResNet18, captures the spatial features of each frame, while the LSTM captures the patterns over time, helping to detect different mental states like alertness, distraction, fatigue, and driving intention. The system shows good results on standard driving datasets and has great potential for use in advanced driver-assistance systems (ADAS), helping to improve road safety and intelligent transportation.