Professional Documents
Culture Documents
ABSTRACT
This project presents a method of real time detection of the human face. It outlines a novel approach for the real-time detection
of car driver drowsiness. There are number of accidents which take place due it the fatigue and alcohol drinking of driver.
Computer scanner and eye detection application is combined to an embedded system to achieve this goal. Real time faces as
well as eyes detection is performed using Haar-feature based cascade classifier and shape predictor 68 face landmarks. The
proposed system is realized with laptop camera embedded with Python-IDLE and OpenCV installed.
Keywords: Human face, Drowsiness, Embedded System, Python-IDLE
1. INTRODUCTION
With the increase in number of road accidents due to drowsiness of the driver and driving while drunk, one should
come forward with a real time embedded system which warns driver when he/she is not paying attention to the road.
This embedded system uses the combined application of Electronic systems with programming language Python. It can
also be extended and linked with Mechanical system (i.e., Disc brake) to apply automatic braking. The main objective
of this project is to warn driver about drowsiness while he/she is driving and to prevent road accidents,
But among all these features we calculated, most of them are irrelevant. For example, consider the image below. The
top row shows two good features. The first feature selected seems to focus on the property that the region of the eyes is
often darker than the region of the nose and cheeks. The second feature selected relies on the property that the eyes are
darker than the bridge of the nose. But the same windows applied to cheeks or any other place is irrelevant. So how do
we select the best features out of 160000+ features? It is achieved by Adaboost.
This, we apply each and every feature on all the training images. For each feature, it finds the best threshold which will
classify the faces to positive and negative. Obviously, there will be errors or misclassifications. We select the features
Volume 7, Issue 6, June 2019 Page 2
IPASJ International Journal of Mechanical Engineering (IIJME)
Web Site: http://www.ipasj.org/IIJME/IIJME.htm
A Publisher for Research Motivation ........ Email:editoriijme@ipasj.org
Volume 7, Issue 6, June 2019 ISSN 2321-6441
with minimum error rate, which means they are the features that most accurately classify the face and non-face images.
(The process is not as simple as this. Each image is given an equal weight in the beginning. After each classification,
weights of misclassified images are increased. Then the same process is done. New error rates are calculated. Also new
weights. The process is continued until the required accuracy or error rate is achieved or the required number of
features is found). The final classifier is a weighted sum of these weak classifiers. It is called weak because it alone
can't classify the image, but together with others forms a strong Their final setup had around 6000 features. (Imagine a
reduction from 160000+ features to 6000 features. That is a big gain). So now you take an image. Take each 24x24
window. Apply 6000 features to it. Check if it is face or not. Wow. Isn't it a little inefficient and time consuming? Yes,
it is. The authors have a good solution for that.
After importing library, we have to call cascade classifiers used in particular application. In this case, two cascade
classifiers are used. One is for face and one is for eyes. One important thing to be kept in mind is that files of cascade
classifiers are to be kept in the same folder as that of program file. As cascade classifier is called in conjunction with
OpenCV, one will have to use cv2.CascadeClassifier as shown in below fig.
Now, we have to open laptop camera and start continuous streaming of it. For that, we have to use while loop. If we
do not use while loop, it will not give video streaming and all we can see is just a capture frame at an instant of
time when laptop camera is activated. In between while loop, we have to use for loop after defining the cascade
classifiers for the detection of faces and eyes. When the camera is activated, we want that streaming frame to be shown
on laptop
Fig. 2.5 Program for eyes and face detection (By Haar cascade)
And after running the program, the result will be,
classifier takes that Tilak as eyes and shows it in square box. To remove this problem, one have to use shape
predictor in place of Haar cascade classifier.
Now that we have the eye regions, we can compute eye aspect ratio to determine if the eyes are closed:
Fig. 3.3 Compute the eye aspect ratio to determine if eyes are closed
If the eye aspect ratio indicated that the eyes have been closed for a sufficiently long enough amount time, we ll sound
an alarm to wake up the driver:
Fig. 3.4 Sound an alarm if eyes are closed for sufficiently long enough time
Now, we ll implement the drowsiness detection algorithm detailed above using OpenCV, dlib and Python.
Lines 2-12 import our required Python packages. Next, we need to define our sound_alarm function which accepts a
path to an audio file residing on disk and then plays the file:
We also need to define the eye_aspect_ratio function which is used to compute the ratio of distances between the
vertical eye landmarks and the distances between the horizontal eye landmarks
The return value of the eye aspect ratio will be approximately constant when the eye is open. The values will then rapid
decrease towards zero during a blink.
If the eye is closed, the eye aspect ratio will again remain approximately constant, but will be much smaller than the
ratio when the eye is open.
To visualize this, consider the following figure,
On the top-left we have an eye that is fully open with the eye facial landmarks plotted. Then on the top-right we have
an eye that is closed. The bottom then plots the eye aspect ratio over time. As we can see, the eye aspect ratio is
constant (indicating the eye is open), then rapidly drops to zero, then increases again, indicating a blink has taken
place.
In our drowsiness detector case, we ll be monitoring the eye aspect ratio to see if the value falls but does not increase
again, thus implying that the person has closed their eyes.
Next, let s parse our command line arguments:
Our drowsiness detector requires one command line argument followed by two optional ones, each of which is detailed
below:
--shape-predictor: This is the path to dlib s pre-trained facial landmark detector
--alarm: Here you can optionally specify the path to an input audio file to be used as an alarm.
--webcam: This integer controls the index of your built-in webcam/USB camera.
Now that our command line arguments have been parsed, we need to define a few important variables:
Line 48 defines the EYE_AR_THRESH . If the eye aspect ratio falls below this threshold, we ll start counting the
number of frames the person has closed their eyes for.
If the number of frames the person has closed their eyes in exceeds EYE_AR_CONSEC_FRAMES (Line 49), we ll
sound an alarm.
Experimentally, I ve found that an EYE_AR_THRESH of 0.3 works well in a variety of situations (although you may
need to tune it yourself for your own applications).
I ve also set the EYE_AR_CONSEC_FRAMES to be 48, meaning that if a person has closed their eyes for 48
consecutive frames; we ll play the alarm sound. You can make the drowsiness detector more sensitive by decreasing
the
EYE_AR_CONSEC_FRAMES similarly, you can make the drowsiness detector less sensitive by increasing it.
Line 53 defines COUNTER, the total number of consecutive frames where the eyeaspect ratio is below
EYE_AR_THRESH.
If COUNTER exceeds EYE_AR_CONSEC_FRAMES, then we ll update the Boolean ALARM_ON (Line 54).
The dlib library ships with a Histogram of Oriented Gradients-based face detector along with a facial landmark
predictor we instantiate both of these in the following code block:
Next, we need to decide if we are working with a file-based video stream or a live USB/webcam/Raspberry Pi camera
video stream:
If you’re using a file video stream, then leave the code as is. Otherwise, if you want to use a built-in webcam or USB
camera, uncomment Line 62.
For a Raspberry Pi camera module, uncomment Line 63.
If you have uncommented either Line 62 or Line 63, then uncomment Line 64 as well to indicate that you are not
reading a video file from disk.
Finally, we have reached the main loop of our script:
Line 89 determines the facial landmarks for the face region, while Line 90 converts these (x, y)-coordinates to a NumPy
array.
Using our array slicing techniques from earlier in this script, we can extract the (x, y)- coordinates for both the left and
right eye, respectively (Lines 94 and 95).
From there, we compute the eye aspect ratio for each eye on Lines 96 and 97. Our next code block simply handles
visualizing the facial landmarks for the eye regions themselves:
At this point we have computed our (averaged) eye aspect ratio, but we haven t actually determined if a blink has
taken place this is taken care of in the next section:
Line 111 makes a check to see if the eye aspect ratio is below our blink threshold if it is, we increment the number of
consecutive frames that indicate a blink is taking place (Line 112).
Otherwise, Line 116 handles the case where the eye aspect ratio is not below the blink threshold.
In this case, we make another check on Line 119 to see if a sufficient number of consecutive frames contained an eye
blink ratio below our pre-defined threshold. If the check passes, we increment the TOTAL number of blinks (Line 120).
We then reset the number of consecutive blinks COUNTER (Line 123).
Our final code block simply handles drawing the number of blinks on our output frame, as well as displaying the
current eye aspect ratio:
REFERENCES
1. Welcome to Python.org. (2019, May 17) (n.d.). Retrieved from https://www.python.org/
2. Ketul Patel (2017,December 30) PIP INSTALL COMMAND IN PYTHON 3.6 [Video File]
Retrieved from: https://www.youtube.com/watch?v=237dNNQhD3Q
3. Decipher Technic [Jul 31, 2016] Install OpenCV-Python on Windows PC [Video file]
Retrieved from: https://www.youtube.com/watch?v=3xAslL8htuQ
4. Decipher Technic [2018, May 18] Install Dlib Python API on Windows PC
Retrieved from: HTTPS://WWW.YOUTUBE.COM/WATCH?V=Q4_M8YTAPDG
5. PYSOURCE. (2019, JANUARY 07) EYE DETECTION - GAZE CONTROLLED KEYBOARD WITH PYTHON
AND OPENCV P.1[VIDEO FILE] RETRIEVED FROM:
HTTPS://WWW.YOUTUBE.COM/WATCH?V=VWUGKCX_KOY&LIST=PL6YC5OUGCOTLVHB5OFFLUJ90OFB
UOU5G
6. Sentdex. (2016, January 10) Haar Cascade Object Detection Face & Eye - OpenCV with Python for Image and
Video Analysis 16 [Video file] Retrieved from: https://www.youtube.com/watch?v=88HdqNDQsEk
7. Opencv.(2018,February07).Opencv/opencv. Retrieved from
https://github.com/opencv/opencv/tree/master/data/haarcascades
8. Deep Learning Haar Cascade Explained. (2018, August 17). Retrieved from http://www.willberger.org/cascade
haar-explained
9. OpenCV. (2019, May 06). Retrieved from https://opencv.org/