ARTIFICIAL

PASSENGER
(A Sleep
Prevention Dialogue

Based Car System)

By
Kini Priyanka

Introduction
IBM

(International business machines

corporation, NY) has developed a software
that holds a conversation with the driver to
determine whether the driver can respond
alertly enough, called Artificial
Passenger.

This

was designed to make long solo

journeys safer and more bearable.

Why Such System?

According

to a national survey in UK and USA, it

is observed that the driver fatigue annually
causes
10000 crashes
1500 deaths
7100 injuries
Majority of off-road accidents observed, were
caused by eye closure of half and even 2-3
seconds, where the normal human eye blinks at
0.2-0.3 seconds.

What is an Artificial
Passenger?

Natural

language e-companion.

Sleep

preventive device in cars to

overcome drowsiness.

Life

safety system.

What Does It Do?

Detects

alarm conditions through sensors.

Broadcasts

pre-stored voice messages

over the speakers.

Captures

images of the driver.

Condition Sensor

Mobile Indicator
Device

About Artificial Passenger

an Artificial Intelligence
based companion that will be resident in
software and chips embedded in the
automobile dashboard.

The AP is

The

system has a conversation planner

that holds a profile of you, including details
of your interests and profession.

 A microphone

picks up your answer and

breaks it down into separate words with
speech-recognition software.

A camera

built into the dashboard also

tracks your lip movements to improve the
accuracy of speech recognition.

 A voice

analyzer then looks for signs of

tiredness by checking to see if the answer
matches your profile. Slow responses and
lack of attention are signs of fatigue.

If

you reply quickly and clearly, the system

judges you to be alert and tells the
conversation planner to continue the line
of questioning.

If your response is slow or doesnt make

sense, the voice analyzer assumes you
are dropping off and acts to get your
attention.

If

driver displays signs of fatigue, the

artificial passenger might be programmed
to open all the windows, sound a buzzer,
increase background music volume, or
even spray the driver with ice water.

Devices Used in Artificial

Passenger

Eye tracker
Voice recognizer or speech recognizer
Natural language processor
Driver analyzer
Conversational planner
Alarm
Microphone
Camera

Working Components
Eye-Tracker

Collecting eye movement data requires both hardware and software.

Hardware:: Head-Mounted systems or remote

systems
Both systems measure the corneal reflection of the infrared LED,
which illuminates and generates a reflection off the surface of the eye.
This action causes the pupil to appear as a bright disk in contrast to
surrounding iris and creates a small glint underneath the pupil which
is used for calibration and tracking.
Head mounted systems use multiple data points to record
eye movement and measures pupil glint from multiple angles.
Remote systems ,in contrast, measure the orientation of the eye
relative to a fixed unit such as camera.

Monitoring System

Software:

1.
2.

Data collection and analysis is handled

by eye-tracking software.
Software catalogs eye-tracking in one of
the two ways
ERICAs Eye Gaze software uses a small
red x to represent eye movement.
In other, data is stored as a series of x/y
coordinates related to specific grid points
on the computer screen

Eye Tracker

Working
Components( cont)

Automatic Speech Recognition

There are two ASRs used in the system:

First one is speaker independent and used for

decoding voice signals of the driver.

Second one operates with voice car media and

decodes tapes, audio , telephones, mails etc.

Working
Components( contd)

Natural Language Processor

Processes the decoded signals of voice or text data provided

from the ASR.

Identifies related meanings from the contents of the decoded

messages.

Produces variant of responses.

This output goes to the driver analyzer as an input.

Working
Components( contd)

Driver Analyzer
Receives

the textual data and voice data.

Measures
Time

the time response using a clock.

responses, conclusions about drivers

alertness will be passed on to the
conversational planner.

Working
Components( contd)

Conversational Planner

It is the heart of the system.

Instructs the language generator to produce the

response.

If the driver is in perfect condition CP instructs the

language generator to continue the conversation
otherwise it will be instructed to change the
conversation.

The following is a typical scenario involving Artificial

Passenger. Imagine, driver Joe returning home after
late nights. His head starts to nod

AP Hey Joe, what did you get your daughter for her

birthday?
Joe Its not her birthday!
AP You seem a little tired. Want to play a game?
Joe Yes.
AP You were a wiz at Name that Tune last time. I was
impressed. Want to try your hand at trivia?
Joe ok.
AP Pick a category: Hollywood Stars, Magic Moments or
Hall of Fame?
Joe Hall of Fame.

Working Components
(contd)

Alarm
If

the CP (conversational planner) receives

information that the driver is about to fall asleep
an alarm system is activated.

Microphone

For picking up words and separate

them by some internally used software
for conversation.

Working Components
(contd)

Camera
This

will track the lip movements of the

driver.
Used to improve the accuracy of the
speech recognition.

Showing the dashboard of the car where the whole artificial system is generally
attached.

Steps Involved in
Tracking

Automatically

initialize lips and eyes using

color predicates and connected
components.

Track

lip corners using dark line between

lips and color predicate even through large
mouth movement like yawning.

 Construct

a bounding box of the head.

Determine

rotation using distances

between eye and lip feature points and
sides of the face.

Determine

eye blinking and eye closing

using the number and intensity of pixels in
the eye region.

Determine

driver vigilance level using all

acquired information.

Detecting Driver
Aiming a single
camera at a head of the driver.
Vigilance

Detecting frequency of up and down nodding and left to

right rotations of the head within a selected time period
with the camera.

Determining frequency of eye blinking and eye closing.

Determining frequency of yawning of the driver within the

selected time period with the camera

Generating an alarm signal in real time if the frequency

value of the up and down nodding, the left to right
rotations, the eye blinking, the eye closings, the yawning
exceeds a selected threshold value.

Applications

Interface with Neighboring Cars -Determines if a driver

presents a high safety risk for ex. falling asleep, tired, inexperienced
or under the influence of alcohol and signals the cars nearby to be
careful of the driver.

Medical Application -The system can monitor a driver and

detect if they are sick ,for ex. having a stroke or heart attack.

In any problem it alerts the vehicles near by , so the driver there can
become alert.

Opens and closes the doors and windows of the car automatically.

It is also used for the entertainment.

Also used in cabins in airplanes, trains, boats etc

Future Implementation
Will

provide us with shortest time routing

based on road conditions changing
because of weather and traffic, information
about the cars on the route, destination
requirement (as-flight has been delayed
etc)

Conclusion
Method

for monitoring driver alertness

Sufficient time to avert an accident.
Successful implementation of Artificial
passenger would allow use of various
services in car like reading emails,
navigation, downloading music files,
voice games etc without compromising
on driver safety

Questions?

Thank
You