Applications of Machine Vision

Overview and Introduction

Instructor: Chao-Ching Ho
National Yunlin University of Science and Technology

Robotic Embedded System Lab

Overview of the Machine Vision
class

 When and where to take class

 Who should take the class
 Why machine vision
 What is machine vision
 How to learn machine vision by steps
When and where to take class

 Office Hours
 10:00 -12 :00, Wednesday

 Room EM 317

 Classroom
 工程教學大樓－ EB205

 Wednesday, 14:10- 17:00 (FGH)

 Class web page

 http://sites.google.com/a/smartrobot.co.cc/robot/courses/machine
visionappl
 TA
 連志豪， EM330
Textbooks

 Lecture Notes:
 各授課業界教師
 References:
 Visual Servoing Control Based Three-Dimensional
Tracking: Theory, Algorithms, Practicalities
(Paperback),  Chao-Ching Ho, LAP, 2009.
 Ramesh Jain and R. Kasturi, Machine Vision,
McGraw Hill, April 1995.
Grading Policy

 Course participance (10%)

 Students are encouraged to exercise the
assignments and practice to present in class
 Mid-term Report (40%)
 Propose a 1or 2 page report to express the
motivation of specified machine vision topic.
 Final Report (50%)
 Based on the learned machine vision knowledge,
select one topic, study and give a presentation.
Course Topics
 Machine Vision Overview and Introduction
 機器視覺技術介紹，郭子鑫研究員 ( 業界教師 )
 光學系統應用於機器視覺，江文慶經理 ( 業界教師 )
 中鋼所的機器視覺技術介紹，何秋誼研究員 ( 業界教師 )
 自動化技術應用於機器視覺系統，林勤喻經理 ( 業界教 師 )
 泛用型機械視覺程式庫介紹 ( 一 ) ，光道視覺科技經理 ( 業界
教師 )
 泛用型機械視覺程式庫介紹 ( 二 ) ，光道視覺科技經理 ( 業界
教師 )
 機器視覺系統應用於面板檢測，戴文智博士 ( 業界教師 )
 雷射探頭系統應用於機器視覺，江文慶經理 ( 業界教師 )
 工具機技術介紹，郭子鑫研究員 ( 業界教師 )
 開放視覺程式庫介紹
 工業研究院參訪
Who should take the class

 If you’re interested in the fields of

 Image Processing
 Animation
 Computer Graphics
 Automatic Optical Inspection
 Robotic Vision
 3D Scanning technology
 Combined vision with industrial engineering
 Combined vision with consumer engineering
Why machine vision

 An image is worth 1000 words

 Many biological systems rely on vision.
 The world is 3D and dynamic.
 Cameras and computers are cheap and popular
 Sometimes, the sensor or the environment are
actively changed to make a task easier.
 This is called “Active” Vision
Application areas
 Industrial inspection, quality control, Reverse
Engineering
 Video Surveillance and security, road monitoring
 Pattern Recognition, Face/Gesture Recognition,
Human-computer interfaces
 Artificial Intelligence, unmanned vehicles
 Robotics, Visual Servoing
 Virtual Reality, tele-operations
 Medical Image analysis (MRI, CT, X-ray)
What is machine vision
 Also termed “computer vision”, “robot vision”
 A machine automatically processes an image and
reports “what is in the image”
 Recognize the content oft he image
 Locate and inspect the objects in the image
 Machine vision is vision for machine…
 Measurement of features
 Pattern classification based on those features
 Measurement of features focuses on processing the
image pixels and extract sets of measurement
 Vision = Geometry + Measurement + Interpretation
何謂機械視覺？
 『機械視覺』 (Machine Vision) 為『計算機視覺』
(Computer Vision) 在工業及生產製造的運用。

 『計算機視覺』允許計算機抽取影像中的資訊。

 機械視覺運用在生產製造中的檢查稱為自動光學檢查
(Automated Optical Inspection)

2004 國科會自動化學門研究發展規劃書 ( 范光照教授 )

Machine Vision focuses on:
 What information  Related disciplines
should be extracted?  Image processing
 How can it be  Pattern recognition
extracted?  Photogrammetry
 How should it be
 Computer graphics
represented?
 Artificial intelligence
 Projective geometry
 How can it be used to  Control theory
achieve the goal?
Sensors
Intensity Images

Light coming from the world hits the sensor.

Digital Images

are 2D arrays (matrices) of numbers:

Applications
Robot Soccer Initiative
Vision system
Host
Communication Host
computer
System computer Communication
System

Robots on the
playing field

Basic Architecture for Robot Soccer Systems

Reverse Engineering

•Combine color and range data

•Use knowledge about most likely shapes

The Pennsylvania State University

自動光學檢測 (AOI)
 What is AOI ？
 Automatic Optical Inspection 自動光學檢測
 檢測方式：非破壞、非接觸式光學檢測。
 檢測項目：運用各種不同光源進行物件的內外層瑕疵檢測或
外觀尺寸形狀色澤的分類與辨識。

 組成系統：由照明系 CCD
鏡頭
影像擷
取硬體
統、取像系統、電控 光源
系統、影像處理與分 電腦
析系統組成。
 主要元件：光 測試樣品
源、 CCD 鏡頭、影
位移平台
像擷取卡、掃描位移
平台、主控電腦等。

Copyright 2008 ITRI 工業技術研究院

AOI 重要性及未來發展
 近年來由於新興產業之興起，如 IC 、 PCB 、 LCD
、 BGA 、光通訊等等，必須做到百分之百的全檢，
少數廠商已開始致力於精密量測系統之發展。然而，
大多數仍歸屬於電子儀器類。除了電性檢驗外，外觀
及表面瑕疵檢測也是必須項目， AOI 設備的產業時代
顯然已經來臨，它具有以下特質：
 使用者：不影響線上製程、新興產業少不了它、價格昂貴、
絕大部分進口、維護更新不易。
 製造者：需要高進入技術門檻，產品少量多樣，光 - 機 - 電 -
資訊技術整合，人才取得不易，附加價值高。
 研究單位：具有設備研發人力與技術。
 學術單位：具有影像處理研發能力。

2004 國科會自動化學門研究發展規劃書 ( 范光照教授 )

AOI 定義 (廣義 )
 廣義的 AOI 設備為結合光學感測系統、訊號處
理系統及分析軟體，應用層面可有：
 宇宙探測 : 太空計畫。
 遙測 : 航空遙測、衛星遙測。
 生物醫學 :X 射線成像、同位素成像、超音波成像。
 工業生產 : 生產品質檢測。
 軍警 : 指紋比對、字形辨認 ( 筆跡 ) 、人像比對、文
件識別。
 機械視覺 : 機器人控制、無人駕駛汽車。
 多媒體技術 :AOI 教學、美工廣告、大眾傳播。

2004 國科會自動化學門研究發展規劃書 ( 范光照教授 )

常見機器視覺應用
•自動化檢測
•定位與導引
•瑕疵檢測
•二維量測
•物件檢查
•影像分析
•顯微鏡觀察
•顆粒分析
•色彩量測

National Instruments
 數值化分析

• Average intensity of a
region
• Standard deviation
• Line profile

National Instruments
Coordinate Systems
Origin of
coordinate system
Defined by the is based on a
pattern match
location and angle of a
reference point
(Origin) within the
image.
Based on the Origin,
regions of interest will
shift and rotate with
the unit under test.

National Instruments
 Identification & Classification

 1D Barcodes
 2D Barcodes
 DataMatrix

 PDF 417

 Optical Character
Recognition

National Instruments
應用案例
 IC 檢測技術

National Instruments
應用案例
 機器視覺定位控制系統

Ima
Acq ge
uisi
tion
Mo
t
Con ion
trol
D
Acq ata
ui s i
t io n
r
te
pu
m
Co

I/O, Monitoring, Control

National Instruments
檢測系統架構及流程說明 :

曝光時間 /
位移速度

Image
光源設計 人機介面 / alignment
影像訊號快速擷取及
同步位置訊號電路
光學濾光片 / 光學效率提
昇鏡組 影像資料銜
接 / 影像強化
Flat-field correction
Gain and offset

線路 / 金屬材料缺陷判斷核心
演算法
檢測樣品

機台精度及行程控制 Stage 控制器

Copyright 2008 ITRI 工業技術研究院

缺陷檢測及判斷之演算法 ( 例一 )– 線路軌跡法

Open
Open
找到的
缺陷 缺陷
缺陷
分類器
分類器 Short
Short

Mouse-bite
Mouse-bite

分類引擎
分類引擎 Etc.
Etc.

Copyright 2008 ITRI 工業技術研究院

研究領域
立體視覺追蹤伺服系統 (3D Visual Servoing Control)

USB

color camera
UART PC

encorder

dsPIC

motor drivers 5-axis manipulator

 研究領域
立體視覺追蹤伺服系統 (3D Visual Servoing Control)

 USER MODE  KERNEL MODE

 User Interface USB Camera

 Calibration WDM Driver
Robot DSP
 Intrinsic Parameters
 Extrinsic Parameters
 Hand-Eye Homogeneous
 CAMSHIFT Algorithm
 HSV Transform Device Control Block SyncFilter
 Back-Projection
 Stereo Computing
 3D Position of Fish
 3D Position Robot Arm

RS232

PID DSP
Y
KE

MODE  Class BaseFunc Function Multiply Function

研究領域
行動機器人 (Mobile Robot) USB

Tracking Target
Far color camera
UART
Obstacle
encorder PC
Middle
Obstacle

Near
Obstacle UART
dsPIC
DC motor

Wheel Mobile Robot

servo motor
dsPIC
3D Scan
3D Scan
研究領域
以視覺辨識為基礎的嵌入式智慧型監控系統
Machine vision vs. Image Processing

 “Image processing” tries to make images look

better, the output of an image processing
system is an image.
 The output of the “machine vision” system is
information about the content of the image.
機器視覺架構

+ + +
Lighting Camera Frame Grabber
Application
and or or
Software
Optics Sensor Vision System

National Instruments
 類
比
交
錯
式
相機種類

非
交
錯
式
掃
瞄
RG
B
wi
th
St
il l
Co
平 l or
行
數
位
式

IE
E E-
13
94

Ca
NI Vision Acquisition Software ™

me
ra
li n
k
彩
色
交
錯
式
線
掃
瞄
式
National Instruments
 General structure of a CV problem

Knowledge Base
Problem

Image Feature
Pre-processing Analysis
Acquisition Extraction

• Noise
Interpretation
• Sensor • Lines
• Illumination • Img enhancement
• Transform • Corners
• Contours
• Regions
• Optical flow
Keyence Machine Vision System Applications

Task: Part Identification Task: Defect Inspection

Differentiation of the buttons Inspection of burnt marks/short-
on a mobile phone circuit of connector resin

Task: Presence/Absence detection Task: Defect Inspection

Detecting defective pin plating Inspection of trimmer switch position Keyence
 Task: Part Identification Task: Presence/Absence detection
Checking correct cable assembly Detecting “reject” marks on electronic components

Dimension Measurement Task: Positioning

Measuring the coplanarity of connector pins Positioning of a CCD device

Electrical and Electronic Industries Keyence

 Task: Counting Task: Dimension Measurement
Checking BGA solder balls Positioning confirmation for silicon wafers

Task: Positioning
Positioning of LCD glass substrates Task: Counting
Detecting ink marks on a silicon wafer

Electrical and Electronic Industries Keyence

 Task: Part Identification Task: Defect Inspection
Differentiation of cylinder blocks Crankshaft porosity detection

Task: Part Identification Task: Presence/Absence detection

Differentiation of tires Detecting the presence/absence of bearing grease

Automotive and Metal Industries Keyence

 Task: Defect Inspection Task: Presence/Absence detection
Inspection of flaws on a steel plate Detecting the groove defect in a piston head

Task: Dimension Measurement Task: Positioning

Measuring the notch position of a gear Position control of a robot

Automotive and Metal Industries Keyence

Task: Counting Task: Dimension Measurement
Check correct seating of parts for die protection Measuring deformation of metal materials

Task: Positioning Task: Counting

Checking fit of body panels Counting bearing balls

Automotive and Metal Industries Keyence

Task: Part Identification Task: Defect Inspection
Inspection of different types of medicine capsulesInspection of pinholes and foreign materials on a
sheet

Task: Presence/Absence detection Task: Part Identification

Inspecting for molded products remaining in a mold Inspection of plastic cups and printing

Food, Pharmaceutical and Other Industries Keyence

Task: Defect Inspection Task: Presence/Absence detection
Inspection of stains on the bottom of beverage Detecting the presence/absence of package
cans inserts and missing items

Task: Dimension Measurement Task: Positioning

Pitch measurement of construction boards Checking mis-positioned labels

Food, Pharmaceutical and Other Industries Keyence

 Task: Counting Task: Dimension Measurement
Counting items in a carton Measuring the thickness of building materials

Task: Positioning Task: Counting

Detecting liquid level in a bottle Counting beverage cans

Food, Pharmaceutical and Other Industries Keyence