Cars That Watch Their Drivers Could Reteach the World to Drive

In most cases, driver-monitoring systems use an infrared camera or cameras on a car’s dash, steering wheel, or rearview mirror. Instead of streaming imagery to the cloud like a cellphone, they process imagery using software inside the car to track head, eye, or hand movements. Those systems have been trained on millions of images of real and synthetic people behind the wheel, their makers say, and can determine whether a driver is looking at the road ahead—or at the cellphone on their lap.

Depending on the automaker, a system that senses that the driver isn’t paying enough attention when using an automated driving feature might flash a warning, change the color of internal lights on the dashboard or steering wheel, sound an alert,  buzz the steering wheel, or some combination of all those tactics. If the driver doesn’t respond, the system will trigger the car to slowly come to a stop.

Gabi Zijderveld, the chief marketing officer of Smart Eye, a Swedish company that sells eye-tracking systems to automotive suppliers, calls the driver monitoring in cars today “fairly simplistic.” The company says its driver-monitoring products are in more than 1 million vehicles on the road and will be in Volvo’s upcoming luxury electric Polestar 3. But in the future, a car equipped with a driver-monitoring system might be able to pull off more nuanced feats, Zijderveld says.

Data from cars could be used to teach more sophisticated algorithms to recognize when a person is, say, safely scrolling through the car’s infotainment system during an easy drive on a sunny four-lane highway—or riskily fiddling with a playlist while navigating city streets in the snow. Alerts might sound or flash or buzz in the latter situation but not the former.

When a driver is trying to multitask, researchers who study the psychology and mechanics of driving tend to evaluate their distraction based on whether their eyes are coming back to the road often, and for long enough, to reestablish a sense for where their car and other vehicles, cyclists, and pedestrians are in space. Driver-monitoring systems may eventually be able to combine information from the car’s many sensors to, for example, determine that a driver isn’t sufficiently paying attention when their vehicle is about to be T-boned, and tighten their seat belt.

Drivers who have already gotten on the wrong side of existing driver-monitoring systems know that their warnings and wailing can be annoying, and they sometimes cry wolf. Automotive engineers choosing when the systems buzz or beep, and how, have to strike a tricky balance.

Experts say the key to building great driver-monitoring systems is to create software that doesn’t just tell a driver when they’re doing something wrong but supports their attention. “It's about defensive driving and avoiding conflict altogether versus avoiding crashes once you're in a conflict,” says Greg Fitch, head of safety research at Android Auto, Google’s in-vehicle app. That could mean sounding quiet but escalating tones, not a high-pitched beep, when it sees you looking off to the side—when you may be watching for pedestrians. Perhaps the system doesn’t totally disengage automatic lane-keeping when you use the wheel to hug the side of a lane but instead shares control.