MEDIATOR is developing a system that mediates between human driver and the vehicle automation to ensure safe, real-time switching between the two, based on who is fittest to drive. Currently, the prototypes of the four main system components and their integration are getting shape. Here we briefly introduce the driver state component with the key concepts of fatigue, distraction and comfort. The development of the driver state component is a joint effort of Autoliv, Ben-Gurion University, Chemnitz University of Technology, Cygnify, Delft University of Technology, FCA, SWOV, Veoneer and VTI.

Driver fitness
As a first prerequisite, the driver state component must provide a reliable and continuous picture of the current and near future fitness of the driver: is (s)he still able to drive the car or, if the automation is in charge, to take over control if needed? To this end, the driver state component must estimate the time to driver fitness (TTDF) and the time to driver unfitness (TTDU). TTDF is the time until a driver is able again to safely perform the manual driving task. TTDU is the time until the driver is no longer able to safely perform the manual driving task.
As described in Deliverable 1.2 MEDIATOR focuses on two crucial determinants of driver fitness: fatigue and distraction.

Monitoring distraction
In-vehicle cameras will be used to decide if the driver is distracted. These cameras will look whether the driver’s eyes are on the road and whether (s)he is engaged in non-driving related tasks. Next to analysis of the driver’s face, a separate camera looks at the driver’s whole body (see figure) and analyses body, arms, hands and objects handled by the hands. The glance patterns and whether the driver is engaged in non-driving tasks determine the current and near future fitness of the driver. This assessment also takes account of relevant driving context factors such as traffic, road type and time of day.

Monitoring fatigue
The assessment of driver fatigue is based on a combination of camera data and physiological measurements. The physiological measures consist of heart inter beat interval (IBI) data. Data are collected via a small sensor mounted on the driver. The camera-based data integrate pictures of eyes, face landmarks and head position with pictures of lane changes and lane departures. In both cases, the system operates by considering the past 5 minutes of data. The fatigue estimate is produced at the end of these 5 minutes.

Driver comfort
A third element of the driver state component of the Mediator system is driver comfort. Comfort is one of the main reasons for drivers to go for higher levels of vehicles automation, for example in traffic jams. Hence, for the Mediator system to be acceptable for drivers, it is important that the system enhances or at least not reduces drivers’ feelings of comfort. Driver comfort will be predicted by known uncomfortable situations and measured based on camera data again. A MEDIATOR simulator study showed that specific facial expressions can be linked to specific emotions. Pressed and stretched lips, for example. could be linked to feelings of tension.

Integration
Integration of the distraction, fatigue and comfort findings result in an overall value for the current and near future driver’s state: the Integrated Driver State Fitness Value (IDSFV). To handle limitations in forecasting the driver state, the IDSFV will in fact consist of three values: one for the worst case, one for the most likely case and one for the best case. These three values are the output of the driver state component.