Human Detection

Mikel Rodriguez

Organization
1. Moving Target Indicator (MTI)

2. Target Classification (Human Detection)

Background models Moving region detection Target chip generation Results

Target features Support vector machines Results

Input Frame

Object Detectio n

Target Chips

Wavelet Features

SVM Classifier

MTI

Classification

Moving Target Indicator

Moving target indicator (MTI) identifies moving objects which can be potential targets

MTI Motivation
Becoming increasingly important in military

and civilian applications To minimize human involvement

Expensive Short attention spans

Computerized monitoring system Real-time capability 24/7

MTI Challenges
Different sensor modalities

LADAR, IR, EO

Targets with different dynamics Small targets Weather conditions

Illumination changes, shadows

MTI
Input Video

Background Modeling Intensity Gradient models models

Moving Target Detection Background Subtraction

dynamic update

Targets Chips Position

Hierarchical Approach to Background Modeling

Pixel level Region level Frame level

Pixel Level
Background Features
Intensity, heat index

EO

IR

Gradient 2D: magnitude, orientation

2 0 4 0 6 0 8 0 10 0 10 2 10 4 10 6 2 0 4 0 6 0 8 0 2 0 4 0 6 0 8 0 10 0 10 2

Magnitude
10 0 10 2 10 4 10 6 10 8 20 0

10 4 10 6 2 0 4 0 6 0

Orientation
8 0 10 0 10 2 10 4 10 6 10 8 20 0

Pixel Level
Background Features
Intensity, heat index Per-pixel mixture of Gaussians. Gradient based subtraction Gradient feature vector =[ m, dd]
m = f x2 + f y2
1

fy d = tan ( ) fx

Pixel Level
Moving Region Detection
Mark pixels that are different from the background intensity

model

Mark pixels that are different from the background gradient

model

Region Level
Fusion of Intensity & Gradient Results
For each color based region, presence ofedge difference pixels at the boundaries is checked.

Im e ag

Clo b o r ased

G ien rad t

Regions with small number of edge difference pixel are removed, color model is updated.

F al in

Frame Level
Model Update
Performs a high level analysis of the scene

components
If more > 50% of the intensity based background subtracted image becomes foreground. Frame level processing issues an alert Intensity based subtraction results are ignored

Structure of the MTI Class

MTI

Background
ConnectedComponents() BoundaryEdges() SetNumGaussians() SetAlpha() SetRhoMean() SetWeightThresh() SetActiveRegion() GetNumGaussians() GetAlpha() GetRhoMean() GetWeightThresh() GetActiveRegion()

Object

Chips
Centroid() ObjectArea() Height() Width()

SetBoundingBox() SetRhoLocation() SetCentroid() GetBoundingBox() GetRhoLocation() GetCentroid() IsFalseDetection()

Results

Target Classification

Classification of objects into two classes: humans and others, from target chips generated by MTI

Challenges
Small size Obscured targets Background clutter Weather conditions

Classifier Flow
Negative Positive
Training SVM Feature Extraction Wavelet MTI Chips Testing
Support Vectors

Decision

Wavelet Based Target Features

Blurred Vertical

Horizontal

Diagonal

Feature Extraction
Apply 2D Wavelet Transform

Daubechies wavelets

Apply Inverse 2D Wavelet Transform to each

of the coefficient matrices individually Rescale and vectorize output matrices

Why Wavelets?
Separability among samples

Humans can be separated from cars and background

Correlation using gradient mag.

Correlation using gray levels

Why Wavelets?
Person 11 - DB3 Wavelet Correlation

Support Vector Machines (SVM)

Classification of data into two classes

N dimensional data. Linearly separable

If not transform data into a higher dimensional space

Find separating N dimensional hyperplane

SVM
Linear Classifier
w T x +b = 0 w x +b > 0
T

hyperplane equation

N dimensional data point xi

w T x +b < 0

wT x + b r= w
Sample distance to hyperplane

SVM
Best Hyperplane?
Infinite number of hyperplanes.

Minimize ri over sample set xi Maximize margin around hyperplane

Samples inside the margin are the support vectors

w x + b1 = 0
T 1

2 w

w T x + b2 = 0 2 w T x + b = 0

SVM
Training Set

Let =1,A training set is a set of tuples: {(x1,y1),(x2,y2),(xm,ym)}.

w T x i + b 1 if yi =1

w T x j + b 1 if y j = 1
For support vectors inequality

becomes equality Unknowns are w and b

SVM
Linear Separability
Linear programming,

Separator line in 2D w1xi,1+w2xi,2+b=0.

Find w1, w2, b such that is maximized Find w1, w2, b such that (w)=wTw is minimized
yi ( w T x i +b ) 1

SVM
Solution
Has the following form:
w = i yi x i and b = yi i w T x i

Non-zero i indicates xi is support vector Classifying function is:

f ( x ) = i yi x T x + b i

Classification Class
Human Classification

TrainingFunction
ReadPositiveImages() ReadNegativeImages() AssemblePositive() AssembleNegative() AssembleMatrices()

TestingFunction
LoadSVM() ReadImages()

ExtractFeatures
ConvertToGray() ApplyWaveletFilter() ApplyInverseTrans() ResizeInverse() VectorizeInverse() Concatenate()

TrainSVM
LIBSVM

TestSVM
LIBSVM

Classification Baseline Analysis

Run time for 3.0GHz dualcore, 2GB RAM

Training: 276 training samples 8.015 seconds Testing: 24.087 chips (25 by 25) per second

Classifier size

Depends on diversity of images For 276 training samples of 25x25, classifier size is 1.101 MB

Classification Baseline Analysis

Memory requirements

Requires entire set of support vectors

M = ( # of SV ) ( 4( wavelet coefficients ) 32bits) ( # of pixels)

Current classifier
M = 30 128 bit ( 25 25 ) = 2400000 bits = 2.3Mb

Experiments
Vivid Dataset UCF Dataset

Results
Training set

300 target chips

Testing

3872 human chips 5605 vehicle and background chips

Performance 2.4% false positive (others classified as pedestrians) 3.2% false negative (pedestrian classified as others)

Future directions
MTI Detection by parts Motion clustering Classification

Various kernels for SVM Better target features

Motion, steerable pyramids, shape features (height, width) Local wavelet coefficients

Adaboost