Professional Documents
Culture Documents
11-Oct-2018
Juan Carlos Niebles and Ranjay Krishna
Stanford Vision and Learning Lab
1
Stanford University
What we will learn today?
11-Oct-2018
2
Stanford University
What we will learn today
11-Oct-2018
3
Stanford University
Fitting as Search in Parametric Space
11-Oct-2018
• Computational complexity is important
– It is infeasible to examine every possible set of parameters and every possible combination
of features
4
Stanford University
Source: L. Lazebnik
Example: Line Fitting
• Why fit lines?
Many objects characterized by presence of straight lines
11-Oct-2018
bridges missing evidence?
• Noise in measured edge points,
orientations:
– How to detect true underlying
parameters?
6
Stanford University Slide credit: Kristen Grauman
Voting
• It’s not feasible to check all combinations of features by fitting a model to each
possible subset.
11-Oct-2018
• Noise & clutter features will cast votes too, but typically their votes should be
inconsistent with the majority of “good” features.
• Ok if some features not observed, as model can span multiple fragments.
7
Stanford University Slide credit: Kristen Grauman
RANSAC [Fischler & Bolles 1981]
• RANdom SAmple Consensus
• Intuition: if an outlier is chosen to compute the current fit, then the resulting line
won’t have much support from rest of the points.
11-Oct-2018
8
Stanford University Slide credit: Kristen Grauman
RANSAC [Fischler & Bolles 1981]
RANSAC loop:
11-Oct-2018
• Keep the transformation with the largest number of inliers
9
Stanford University Slide credit: Kristen Grauman
RANSAC Line Fitting Example
11-Oct-2018
10
Stanford University Slide credit: Kristen Grauman
RANSAC Line Fitting Example
11
Stanford University Slide credit: Kristen Grauman
RANSAC Line Fitting Example
12
Stanford University Slide credit: Kristen Grauman
RANSAC Line Fitting Example
11-Oct-2018
Total number of points
within a threshold of
line.
14
Stanford University Slide credit: Kristen Grauman
RANSAC Line Fitting Example
15
Stanford University Slide credit: Kristen Grauman
RANSAC Line Fitting Example
11-Oct-2018
Repeat, until we get a good
result.
16
Stanford University Slide credit: Kristen Grauman
Features and Fitting 11-Oct-2018
17
Stanford University
RANSAC: How many samples?
• How many samples are needed?
– Suppose w is fraction of inliers (points from line).
11-Oct-2018
Þ Choose k high enough to keep this below desired failure rate.
18
Stanford University Slide credit: David Lowe
RANSAC: Computed k (p=0.99)
Sample Proportion of outliers
size
n 5% 10% 20% 25% 30% 40% 50%
11-Oct-2018
7 4 8 20 33 54 163 588
8 5 9 26 44 78 272 1177
19
Stanford University Slide credit: David Lowe
After RANSAC
• RANSAC divides data into inliers and outliers and
yields estimate computed from minimal set of inliers.
11-Oct-2018
20
Stanford University Slide credit: David Lowe
RANSAC: Pros and Cons
• Pros:
11-Oct-2018
• A voting strategy, The Hough transform, can handle high
percentage of outliers
21
Stanford University
What we will learn today?
11-Oct-2018
Some background reading:
Rick Szeliski, Chapter 4.1.1; David Lowe, IJCV 2004
22
Stanford University
Image matching: a challenging problem
by swashford
24
Stanford University Slide credit: Steve Seitz
Harder Case
25
Stanford University Slide credit: Steve Seitz
Harder Still?
26
Stanford University Slide credit: Steve Seitz
Answer Below (Look for tiny colored squares)
– Articulation d dq
11-Oct-2018
φ
φ
θq
θ
– Intra-category variations
28
Stanford University
General Approach
1. Find a set of
distinctive key-
points
11-Oct-2018
m easure
descriptor from the
N pixels
normalized region
e .g . c o lo r e .g . c o lo r
No chance to match!
11-Oct-2018
We need a repeatable detector!
30
Stanford University Slide credit: Darya Frolova, Denis Simakov
Common Requirements
• Problem 1:
– Detect the same point independently in both images
11-Oct-2018
We need a reliable and distinctive descriptor!
31
Stanford University Slide credit: Darya Frolova, Denis Simakov
Invariance: Geometric Transformations
34
Stanford University Slide credit: Tinne Tuytelaars
Requirements
11-Oct-2018
• Distinctiveness: The regions should contain “interesting” structure.
35
Stanford University
Slide credit: Bastian Leibe
Many Existing Detectors Available
11-Oct-2018
• Others…
• Those detectors have become a basic building block for many recent applications in
Computer Vision.
36
Stanford University Slide credit: Bastian Leibe
What we will learn today?
11-Oct-2018
Some background reading:
Rick Szeliski, Chapter 4.1.1; David Lowe, IJCV 2004
37
Stanford University
Keypoint Localization
11-Oct-2018
– Repeatable detection
– Precise localization
– Interesting content
Þ Look for two-dimensional signal changes
38
Stanford University
Slide credit: Bastian Leibe
Finding Corners
11-Oct-2018
dominant directions
• Corners are repeatable and distinctive
11-Oct-2018
“flat” region: “edge”: “corner”:
no change in all no change along significant change
directions the edge direction in all directions 40
Stanford University Slide credit: Alyosha Efros
Corners versus edges
Large
Corner
Small
Edge
Large
11-Oct-2018
Small
Nothing
Small
41
Stanford University
Corners versus edges
??
Corner
11-Oct-2018
42
Stanford University
Harris Detector Formulation
11-Oct-2018
Window function w(x,y) = or
11-Oct-2018
Sum over image region – the area we are
with respect to y
checking for corner
M
Slide credit: Rick Szeliski 44
Stanford University
Harris Detector Formulation
11-Oct-2018
Sum over image region – the area we are
with respect to y
checking for corner
M
45
Stanford University Slide credit: Rick Szeliski
What Does This Matrix Reveal?
• This means:
– Dominant gradient directions align with x or y axis
11-Oct-2018
– If either λ is close to 0, then this is not a corner, so look for locations
where both are large.
• What if we have a corner that is not aligned with the image
axes?
46
Stanford University Slide credit: David Jacobs
What Does This Matrix Reveal?
• This means:
– Dominant gradient directions align with x or y axis
11-Oct-2018
– If either λ is close to 0, then this is not a corner, so look for locations
where both are large.
Direction of the
11-Oct-2018
slowest change
(lmax)-1/2
(lmin)-1/2
48
Stanford University adapted from Darya Frolova, Denis Simakov
Interpreting the Eigenvalues
11-Oct-2018
l 1 and l 2 are small;
E is almost constant in
“Flat” “Edge”
all directions
region l 1 >> l 2
l1 49
Stanford University Slide credit: Kristen Grauman
Corner Response Function
• Fast approximation
11-Oct-2018
– Avoid computing the
eigenvalues
“Flat” “Edge”
– α: constant
region
(0.04 to 0.06) θ<0
l1
50
Stanford University Slide credit: Kristen Grauman
Window Function w(x,y)
é I x2 IxI y ù
M = å w( x, y) ê 2 ú
êë I x I y I y úû
é I x2 I x I y ù
M = åê 2 ú
x, y ê I I I ú
– Problem: not rotationëinvariant û
x y y
1 in window, 0 outside
11-Oct-2018
– Gaussian already performs weighted sum
é I x2 IxI y ù
M = g (s ) * ê ú
êë I x I y I y2 úû Gaussian
– Result is rotation invariant
3. Gaussian
filter g( s I)
g(Ix2) g(Iy2) g(IxIy)
11-Oct-2018
4. Cornerness function – two strong eigenvalues
θ = det[M (σ I ,σ D )]− α[trace(M (σ I ,σ D ))]2
= g ( I x2 ) g ( I y2 ) - [ g ( I x I y )]2 - a[ g ( I x2 ) + g ( I y2 )]2
57
Stanford University
Slide credit: Krystian Mikolajczyk
Harris Detector – Responses [Harris88]
• Translation invariance?
• Translation invariance
11-Oct-2018
Ellipse rotates but its shape (i.e.
eigenvalues) remains the same
• Translation invariance
11-Oct-2018
Corner All points will be
classified as edges!
Not invariant to image scale!
62
Stanford University Slide credit: Kristen Grauman
What we are learned today?
11-Oct-2018
63
Stanford University