Professional Documents
Culture Documents
20162017
Getting Started with Python + OpenCV
Antonino Furnari http://www.dmi.unict.it/~furnari/ furnari@dmi.unict.it
Prof. Sebastiano Battiato http://www.dmi.unict.it/~battiato/ battiato@dmi.unict.it
We recommend online consultation
In this practical session we will learn:
The basics of Jupyter notebooks;
The basics of the scipy stack (Python + numpy + matplotlib);
The basics of Python + OpenCV;
1. Conventions
The goal of this practical session is to guide the reader through the understanding of theoretical and
practical concepts. To guide the understanding, the reader will be asked to answer some questions and do
some exercises.
2. Jupyter Notebooks
As reported on Jupyter website (https://jupyter.org):
The Jupyter Notebook is an opensource web application that allows you to create and
share documents that contain live code, equations, visualizations and explanatory text.
Uses include: data cleaning and transformation, numerical simulation, statistical modeling,
machine learning and much more.
The web interface connects to an IPython Kernel, which performs all the computation in background. All
the content of a Jupyter notebook is contained into "cells". A cell contains some text which is "executed"
when the the user presses SHIFT + ENTER. A cell can be of different kinds:
Code cells: contain python code which is executed when the cell is executed;
Markdown cells: contain Markdown code, which is converted to HTML and displayed when the
cell is executed.
Raw Cells: content is not processed when they are executed;
In [1]:
# this is a code cell, and, specifically, this is a Python comment
print "Hello World"
Hello World
This is a markdown cell. It can mix html and markdown code.
This is a raw cell. Its content is not processed.
It is possible to inspect a symbol (e.g., a variable) by typing symbol?:
In [2]:
a='text'
a?
Read more about Jupyter Notebooks can be found here: https://jupyter.readthedocs.io/en/latest/index.html
3. Python/Numpy basics
Python is a very flexible programming language. It is designed to be easy to learn, fast to code and to
allow to write highly readable code. Learning Python is easy and there are a lot of online resources. Here
we will quickly see some examples to revise the main concepts.
3.1 Interpreted Language
Python is interpreted. The easiest way to write a Python program consists in writing an example.py file
containing some code and running it with python example.py. The first time a script is interpreted, an
"compiled" example.pyc version is created to improve performance. Nevertheless, the program should
always be run by typing python example.py and not python example.pyc. If the content of example.py
changes, file example.pyc is updated accordingly when python example.py is executed.
3.2 Duck Typing
Python is duck typed, i.e., types are identified at run time following the duck test:
If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.
3.5 Lists and Tuples
Before starting, it is important to know that in Python there are at least two important data structures:
Lists: sequences of values of any type. Types can be also mixed.
Tuples: similar to list but immutable, i.e., they cannot be modified after initialization.
In [3]:
l = [1,2,3,4,5] #this is a list (square brackets)
t = (1,2,3,4,5) #this is a tuple (round brackets)
print l
print t
[1, 2, 3, 4, 5]
(1, 2, 3, 4, 5)
Lists and tuple are indexed like in C: the first index is zero.
In [4]:
print l[0],l[2]
print t[0],t[3]
l[0]=8
print l
1 3
1 4
[8, 2, 3, 4, 5]
In [5]:
try:
t[2]=5 #this will yield an error since tuples are immutable
except:
print "This will raise an exception!"
This will raise an exception!
Initializing a tuple with just an element, will produce a number instead:
In [6]:
t=(1)
t?
If we want to define a monodimensional tuple, we should type:
In [7]:
t=(1,)
t?
It is possible to convert tuple to list and vice versa:
In [8]:
l=[1,2,3,4,5,6,7,8]
t=(4,5,6,7,4,8,2,4)
ttl = list(t)
ltt = tuple(l)
print ttl
print ltt
[4, 5, 6, 7, 4, 8, 2, 4]
(1, 2, 3, 4, 5, 6, 7, 8)
Tuples and lists can be indexed in many different ways:
In [9]:
print "t[0:3] ‐>", t[0:3] #from index 0 (included) to index 3 (excluded)
print "l[1:3] ‐>", l[1:2] #from index 1 (included) to index 2 (excluded)
print "t[0:] ‐>", t[1:] #from index 1 (included) to the last index (second index
is omitted)
print "l[:2] ‐>", l[:2] #from index 0 (omitted) to index 2 (excluded)
print "t[0:7:1] ‐>", t[0:8:1] #from index 0 (included) to index 8 (excluded) at a st
ep of 1 (i.e., all numbers)
print "l[0:7:2] ‐>", l[0:8:2] #from index 0 (included) to index 8 (excluded) at a st
ep of 2 (i.e., every other number)
print "t[::2] ‐>", t[::2] #from index 0 (omitted) to index 8 (omitted) at a step o
f 2 (equivalent to the previous one)
print "l[‐1] ‐>", l[‐1] #last number
print "t[‐2] ‐>", t[‐2] #second‐last numebr
print "l[‐5] ‐>", l[‐5] #fifth‐last numebr
print "t[‐3:] ‐>", t[‐3:] #last three number
print "l[:‐3] ‐>", l[:‐3] #from index 0 (omitted) to third‐last index (excluded)
print "t[‐1:‐5:‐1] ‐>", t[‐1:‐5:‐1] #from the penultimate index to the fifth‐last index
(excluded) at a step of ‐1
print "l[‐1:0:‐1] ‐>", l[‐1:0:‐1] #from the penultimate index to index 0 (excluded) at
a step of ‐1
print "l[::‐1] ‐>", l[::‐1] #reverse the list
t[0:3] ‐> (4, 5, 6)
l[1:3] ‐> [2]
t[0:] ‐> (5, 6, 7, 4, 8, 2, 4)
l[:2] ‐> [1, 2]
t[0:7:1] ‐> (4, 5, 6, 7, 4, 8, 2, 4)
l[0:7:2] ‐> [1, 3, 5, 7]
t[::2] ‐> (4, 6, 4, 2)
l[‐1] ‐> 8
t[‐2] ‐> 2
l[‐5] ‐> 4
t[‐3:] ‐> (8, 2, 4)
l[:‐3] ‐> [1, 2, 3, 4, 5]
t[‐1:‐5:‐1] ‐> (4, 2, 8, 4)
l[‐1:0:‐1] ‐> [8, 7, 6, 5, 4, 3, 2]
l[::‐1] ‐> [8, 7, 6, 5, 4, 3, 2, 1]
Excercise 3.1
Define a 1dimensional tuple;
Define a 5dimensional tuple;
Define a list from a tuple;
Define a list of tuples;
Define a tuple of lists.
Excercise 3.2
Define the list [1,8,2,6,15,21,76,22,0,111,23,12,24], then:
Print the first number of the list;
Print the last number of the list;
Print the sum of the numbers with odd indices (e.g., 1,3,5,...) in the list;
Print the reverse list;
Print the average of the numbers in the list.
3.4 If statements and loops
Before talking about if statements and for loops, rememeber that indentation in Python is mandatory!
It practically substitutes brackets, therefore, if you forget to indent your code in the right way, you will get
an error!
Some examples of the if construct:
In [10]:
true_condition = True
false_condition = False
if true_condition: #always remeber about the colon
print "cool!" #indentation here is mandatory
if false_condition: #false condition should not fire
print "not cool :("
if not false_condition: #negate boolean value using "not"
print "cool again :)"
if false_condition:
print "false"
else: #note indentation and colon here
print "true"
if false_condition:
print "false"
elif 5>4: #"else if" is abbreviated as "elif"
print "5>4"
else:
print "4<5??"
cool!
cool again :)
true
5>4
It is also possible to test if a value belongs to a list as follows:
In [11]:
l=[1,6,2,5,7]
if 5 in l:
print "5 in l"
if 8 in l:
print "8 in l"
5 in l
While loop:
In [12]:
i=0
while i<5: #colon
print i #indentation
i+=1
0
1
2
3
4
For loop syntax is a little different than standard C syntax. For loops in Python are more similar to
foareach loops than standard for loops. Therefore, they require an "iterable" (e.g., a list or a tuple) to run:
In [13]:
l=[1,7,2,5]
for v in l:
print v
1
7
2
5
To write something similar to the standard C:
for (int i=1; i<5; i++) {...}
we can use the range function which generates iterables on the fly:
In [14]:
for i in range(5):
print i
0
1
2
3
4
Some tricks:
In [15]:
array=[1,7,2,4,5]
for i in range(len(array)): #standard notation
print i,"‐>",array[i]
0 ‐> 1
1 ‐> 7
2 ‐> 2
3 ‐> 4
4 ‐> 5
In [16]:
for index,value in enumerate(array): #get both index and value
print index,"‐>",value
0 ‐> 1
1 ‐> 7
2 ‐> 2
3 ‐> 4
4 ‐> 5
In [17]:
a1=[1,6,2,5]
a2=[1,8,2,7]
for v1,v2 in zip(a1,a2): #iterate twi arrays at once!
print v1,v2
1 1
6 8
2 2
5 7
3.6 Numpy Arrays
Numpy arrays are another data type which can be used to handle matrix operations. Note that
python "arrays" can actually be matrices and tensors in general. Before using numpy, we have to import
its package:
In [18]:
import numpy as np #now we can reference numpy simply by "np"
A numpy array is defined starting from a list or a tuple as follows:
In [19]:
l = [[1,2,3],[4,5,2],[1,8,3]] #this is a list of lists
print "List of lists:",l
a = np.array(l) #this is a numpy array
print "Numpy array:\n",a
print "Numpy array from tuple:",np.array((1,2,3))
List of lists: [[1, 2, 3], [4, 5, 2], [1, 8, 3]]
Numpy array:
[[1 2 3]
[4 5 2]
[1 8 3]]
Numpy array from tuple: [1 2 3]
Let's create some structures:
In [20]:
a1 = np.array([1,2,3,4]) #this is a numpy array
a2 = np.array([5,2,1,4]) #another numpy array
m1 = np.array([[1,2,3,4],[5,4,2,3],[7,5,3,2],[0,2,3,1]]) #this is a numpy 4x4 matrix
m2 = np.array([[8,2,3,1],[1,5,3,6],[1,2,7,5],[4,5,2,1]]) #another numpy 4x4 matrix
print 'Array:',a1
print 'Array:',a2
print 'Matrix:\n',m1
print 'matrix:\n',m2
Array: [1 2 3 4]
Array: [5 2 1 4]
Matrix:
[[1 2 3 4]
[5 4 2 3]
[7 5 3 2]
[0 2 3 1]]
matrix:
[[8 2 3 1]
[1 5 3 6]
[1 2 7 5]
[4 5 2 1]]
Let's perform matrix and array opeartions:
In [21]:
print "Shape:",a1.shape #print the array shape
print "Shape:",a2.shape #vectors are monodimensional array of 4 elements
print "Sum:",a1+a2 #vector sum
print "Elementwise multiplication:",a1*a2 #elementwise multiplication
print "Power of two:",a1**2 #power of two
print "Elementwise power:",a1**a2 #elementwise power operation
print "Vector product:",a1.dot(a2) #vector product
print "Minimum:",a1.min() #array minumum
print "Maximum:",a1.max() #array maximum
print "Sum:",a2.sum() #sum of all values
print "Mean:",a1.mean() #array mean
Shape: (4L,)
Shape: (4L,)
Sum: [6 4 4 8]
Elementwise multiplication: [ 5 4 3 16]
Power of two: [ 1 4 9 16]
Elementwise power: [ 1 4 3 256]
Vector product: 28
Minimum: 1
Maximum: 4
Sum: 12
Mean: 2.5
In [22]:
print "Shape:",m1.shape #matrix shape
print "Shape:",m2.shape #matrices are 4x4
print "Sum:",m1+m2 #sum
print "Elementwise product:",m1*m2 #elementwise product
print "Power of two:",m1**2 #power of two
print "Elementwise power:",m1**m2 #elementwise power
print "Matrix multiplication:",m1.dot(m2) #matrix multiplication
print "Minimum:",m1.min() #minumum
print "Maximum:",m1.max() #maximum
print "Minimum along columns:",m1.min(0) #minumum along columns
print "Minimum along rows:",m1.min(1) #minimum along rows
print "Sum:",m1.sum() #mean value
print "Mean:",m1.mean() #mean value
print "Diagonal:",m1.diagonal() #diagonal of the matrix
Shape: (4L, 4L)
Shape: (4L, 4L)
Sum: [[ 9 4 6 5]
[ 6 9 5 9]
[ 8 7 10 7]
[ 4 7 5 2]]
Elementwise product: [[ 8 4 9 4]
[ 5 20 6 18]
[ 7 10 21 10]
[ 0 10 6 1]]
Power of two: [[ 1 4 9 16]
[25 16 4 9]
[49 25 9 4]
[ 0 4 9 1]]
Elementwise power: [[ 1 4 27 4]
[ 5 1024 8 729]
[ 7 25 2187 32]
[ 0 32 9 1]]
Matrix multiplication: [[29 38 38 32]
[58 49 47 42]
[72 55 61 54]
[ 9 21 29 28]]
Minimum: 0
Maximum: 7
Minimum along columns: [0 2 2 1]
Minimum along rows: [1 2 2 0]
Sum: 47
Mean: 2.9375
Diagonal: [1 4 3 1]
Since Python is "duck typed", types are inferred at run time:
In [23]:
a=np.array([0,1,2,3,4])
b=np.array([0.2,0.5,2,0.8])
c=np.array(['l','p','ma'])
print a.dtype #array of integers
print b.dtype #array of floats
print c.dtype #array of strings the "2" indicates that the longest string has length 2
int32
float64
|S2
Numpy arrays can be indexed similarly to Python lists and tuples as follows:
In [24]:
mat = np.array(([1,5,2,7],[2,7,3,2],[1,5,2,1]))
print "mat[0,0] ‐>",mat[0,0] #first row, first column
print "mat[2][1] ‐>",mat[2][1] #third row, second column (alternate notation)
print "mat[0] ‐> ",mat[0] #first row of mat
print "mat[:,0] ‐> ",mat[:,0] #first column of mat. The ":" notation means "leave eve
rything unchanged along this dimension"
print "mat[0,:] ‐>",mat[0,:] #first row of mat (alternate notation)
print "mat[0:2,:] ‐>\n",mat[0:2,:] #first two rows of mat
print "mat[:,0:2] ‐>\n",mat[:,0:2] #first two columns of mat
print "mat[‐1] ‐>",mat[‐1] #last row of mat
mat[0,0] ‐> 1
mat[2][1] ‐> 5
mat[0] ‐> [1 5 2 7]
mat[:,0] ‐> [1 2 1]
mat[0,:] ‐> [1 5 2 7]
mat[0:2,:] ‐>
[[1 5 2 7]
[2 7 3 2]]
mat[:,0:2] ‐>
[[1 5]
[2 7]
[1 5]]
mat[‐1] ‐> [1 5 2 1]
Numpy allows to handle tensors of arbitrary dimensionality:
In [25]:
mat=np.array([[[1,2],
[2,3],
[1,2]],
[[5,2],
[‐1,4],
[0,2]],
[[0,5],
[2,‐3],
[2,1]]])
print mat.shape
(3L, 3L, 2L)
Indexing can be performed similarly:
In [26]:
print "mat[:,:,0] ‐>", mat[:,:,0] #3 x 3 matrix contained in the "first channel" of the
tensor
print "mat[:,:,1] ‐>",mat[:,:,1] #3 x 3 matrix contained in the "second channel" of the
tensor
print "mat[...,0] ‐>", mat[...,0] #3 x 3 matrix contained in the "first channel" of the
tensor (alternate notation)
print "mat[...,1] ‐>",mat[...,1] #3 x 3 matrix contained in the "second channel" of the
tensor (alternate notation)
mat[:,:,0] ‐> [[ 1 2 1]
[ 5 ‐1 0]
[ 0 2 2]]
mat[:,:,1] ‐> [[ 2 3 2]
[ 2 4 2]
[ 5 ‐3 1]]
mat[...,0] ‐> [[ 1 2 1]
[ 5 ‐1 0]
[ 0 2 2]]
mat[...,1] ‐> [[ 2 3 2]
[ 2 4 2]
[ 5 ‐3 1]]
Excercise 3.3
Define a 3x4 matrix containing float numbers, then:
Print the last row of the matrix;
Print the second column of the matrix;
Sum the first and last row of the matrix;
Sum the elements along the diagonal;
Find the number of elements in the matrix.
4. Getting Started with OpenCV
To use OpenCV library, it is necessary to import it as follows:
In [27]:
import cv2
Remember that your best when using OpenCV is the documentation. You can find it at:
http://docs.opencv.org/2.4/index.html
4.1 Loading and Saving Images
We assume that an image named 'lena.png' is available in the current directory and load it using the
imread function of cv2:
In [28]:
img=cv2.imread('lena.png')
img is an ndarray (i.e., a numpy array), we can check it by typing:
In [29]:
img?
Let's check some properties:
In [30]:
print img.shape
print img.dtype
(512L, 512L, 3L)
uint8
In [31]:
print img.min()
print img.max()
3
255
WARNING if the file does not exist, the imread function will not raise any exception! A None value will be
returned insted:
In [32]:
img2=cv2.imread('doesnotexist.jpg')
print img2
None
We can check if the image was succesfully read as follows:
In [33]:
img2=cv2.imread('doesnotexist.jpg')
if img2 is None:
print "Houston, we have a problem!"
Houston, we have a problem!
imread also supports some flags to load images in grayscale mode or to change color depth, read the doc!
Excercise 4.1
Load the lena.png image in grayscale format using the cv2.imread function
Images can be written using the imwrite function:
In [34]:
cv2.imwrite('test.jpg',img)
Out[34]:
True
The function returns True if the write operation succeeds. Imwrite will guess what format to use, given the
extension:
In [35]:
cv2.imwrite('test.png',img)
Out[35]:
True
4.2 Displaying images
Images can be displayed using built in opencv functions. In particular it is possible to create "named
windows" (i.e., windows with a title) and show images in them:
In [36]:
cv2.namedWindow('Lena')
cv2.imshow('Lena',img)
cv2.waitKey()
cv2.destroyAllWindows()
The cv.waitKey() instruction is necessary to let the system "wait" for the user to press any key before
proceeding. Without this instruction, the system would return immediately without actually displaying the
image. The cv2.destroyAllWindows() function is used to close all open windows.
When working with Jupyter notebook, it may be more convenient to use matplotlib to show images. First,
load the required packages:
In [37]:
from matplotlib import pyplot as plt
Now we can try to show the image using the plt.imshow function:
In [38]:
plt.imshow(img)
plt.show()
The plt.show instruction is necessary to actually show the image.
As you can see from the plot above, colors are not correct. This is due to the fact that OpenCV reads
images in the BGR (i.e., Blue Green and Red) format, while matplotlib expects images in the
standard RGB (i.e., Red Green and Blue) format. The problem can be solved in different ways.
4.2.3 Color conversion
We can use the cv2.cvtcolor function to convert the image from BGR to RGB.
In [39]:
img2=cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
plt.imshow(img2)
plt.show()
4.2.2 Manually swapping channels
Using powerful numpy indexing capabilities to flip the order of the channels:
In [40]:
plt.imshow(img[...,::‐1]) #this conversion can be performed inline!
plt.show()
Question 4.1
The ::1 notation can be used also along the first or second dimension. What
would be the effect of inserting the ::1 notation to invert elements along the first
dimension? What about the second one?
Excercise 4.2
The ::1 notation can be used also along the first or second dimension. Display
the image with elements along the first dimension (i.e., rows) inverted. Display the
image with elements along the second dimension (i.e., columns) inverted.
Excercise 4.3
Display only the RED channel of the lena image.
5. Basic Image Processing with OpenCV
OpenCV implements a variety of image processing operation which can be easily implemented.
5.1 Gaussian Blur
In [41]:
img_g=cv2.GaussianBlur(img,(15,15),7) #use a 15x15 Gaussial kernel with standard deviat
ion 7
plt.imshow(img_g[...,::‐1])
plt.show()
5.2 Conversion to grayscale
In [42]:
img_gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
print img_gray.shape #now the image has just one channel
plt.imshow(img_gray[...,:],cmap='gray') #cmap='gray' is required to show the image corr
ectly
plt.show()
(512L, 512L)
5.3 Image Blending
In [43]:
img_rotate=img[::‐1,...]
img_blend=img_rotate/2+img/2
plt.imshow(img_blend[...,::‐1])
plt.show()
5.4 Sobel Derivatives
In [44]:
img_sobel_x=cv2.Sobel(img_gray,‐1,1,0) #x‐derivative set to 1, y‐derivative set to 0
img_sobel_y=cv2.Sobel(img_gray,‐1,0,1) #x‐derivative set to 0, y‐derivative set to 1
print img_sobel_x.shape
print img_sobel_y.shape
plt.subplot(1,2,1)
plt.imshow(img_sobel_x[...,::‐1],cmap='gray')
plt.title('Sobel X')
plt.subplot(1,2,2)
plt.imshow(img_sobel_y[...,::‐1],cmap='gray')
plt.title('Sobel Y')
plt.show()
(512L, 512L)
(512L, 512L)
5.5 Edge Detection
In [45]:
img_canny = cv2.Canny(img,180,200)
plt.imshow(img_canny[...,::‐1],cmap='gray')
plt.show()
Excercise 5.1
Blend the image created using the Canny function with the original lena image.
6 Video I/O with OpenCV
Manipulating video with OpenCV is easy. Actually, Jupyter notebooks are not always well suited for such
kind of realtime processing. Therefore, you may consider experimenting with the code below using the
"IDE paradigm".
6.1 Acquiring Video from a Camera
To acquire a video stream from your webcam, you can write:
In [46]:
cap=cv2.VideoCapture(0) #open camera with device "0"
r,img=cap.read() #reads a frame "img" and a return value "r", set to True if there are
no errors
while r:
cv2.imshow('Frame',img) #show the acquired frame
img_c=cv2.Canny(img,180,200) #process current frame
cv2.imshow('Canny',img_c) #show processed version
a=cv2.waitKey(40) #wait for 40 ms (i.e., 25 fps)
if a!=‐1: #if any key was pressed:
r=False #set r to False
else:
r,img=cap.read() #else, read new frame
cv2.destroyAllWindows()
cap.release()
Excercise 6.1
Write the code to show the webcam video stream where each frame has been
blurred (Gaussian blur).
6.2 Saving Video to File
To write vieo to file, we first need to create a cv2.VideoWriter object:
In [47]:
vw = cv2.VideoWriter('test_video.avi',‐1,25,(512,512))
#the "‐1" on Windows allows to select the codec from the GUI
#25 stands for 25fps
#resolution is 640x480
Now we can write a new frame with the vw.write method:
In [48]:
for i in range(300):
vw.write(img_canny)
The stream is closed using the release method:
In [49]:
vw.release()
Excercise 6.2
Write the code to save to file video acquired using the webcam.
7. References
[1] Jupyter Notebooks. https://jupyter.readthedocs.io/en/latest/index.html
[2] Duck Typing. https://en.wikipedia.org/wiki/Duck_typing
[3] Python data structures. https://docs.python.org/2/tutorial/datastructures.html
[4] Python control flow. https://docs.python.org/2/tutorial/controlflow.html
[5] Numpy arrays. https://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html
[6] OpenCV doc. http://docs.opencv.org/2.4/index.html
[7] Reading and writing images and video with OpenCV.
http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html
[8] Displaying images with Python and OpenCV.
http://opencvpython
tutroals.readthedocs.io/en/latest/py_tutorials/py_gui/py_image_display/py_image_display.html
[9] OpenCV Gaussian Blur.
http://docs.opencv.org/2.4/modules/imgproc/doc/filtering.html#cv2.GaussianBlur
[10] OpenCV Color Conversion.
http://docs.opencv.org/2.4/modules/imgproc/doc/miscellaneous_transformations.html#cv2.cvtColor
[11] OpenCV Sobel Derivatives. http://docs.opencv.org/2.4/modules/imgproc/doc/filtering.html#sobel
[12] OpenCV Canny Edge Detection.
http://docs.opencv.org/2.4/modules/imgproc/doc/feature_detection.html#canny