Professional Documents
Culture Documents
Outlier detection
Process monitoring
Examples
2009 September
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Definitions
Soft sensor is a common name for software where several
measurements are processed together. There may be dozens
or even hundreds of measurements. The interaction of the
signals can be used for calculating new quantities that can not
be measured.
Soft sensors or inferential calculators are operators virtual
eyes. Soft sensors create windows to a process where physical
equivalents are unrealistic or even impossible.
Sensor output can be a control signal, advisory information for
operators, predictions of product quality, information on
process faults or outliers in data.
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Techniques
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Tasks
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
\
[
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Robust statistics
Use robust statistics (scaling, PCA, covariance matrix) to increase
effectiveness of outlier detection.
A robust estimate of scale can be obtained using the median
absolute deviation of the median (MADM) defined as
d
med|x med(x)|
0.6745
(1)
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
PCA (1)
The ith transformed variable zi , which is known as PC or score is
zi = hxpi i
(2)
M
X
zij2
j=1
sj2
(3)
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
PCA (2)
The variable
Ti2
N(N M)
M(N 2 1)
(4)
M(N 2 1)
F (M, N M)
N(N M)
(5)
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
PCA (3)
The portion of the input space corresponding to the K M
smallest eigenvalues can be monitored using the Q statistic (2 )
Q = eT e
(6)
e = (I PPT )x
(7)
where
is the residual vector. The threshold Q for the Q statistic can be
computed from the eigenvalues and the normal deviate
corresponding to the 1 percentile.
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
H matrix
hi = xi (XT X)1 xT
i
2K /N
The cut-off value for the Jackknife residual ti (next slide) is given
by t0.95 (N-K-1)the 95% quantile of the Student distribution.
Jackknife residuals
5
0
0.2
0.4
0.6
High leverage
0.8
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Measure based on
Formula
r
(NK 1)
ti = ti (NK
t 2 )
Residuals
Cut-off value
t(N K 1)
ei2
eT e
2(K 1)
N
hi = hi +
Influence Function
p
p
WKi = |ti | hi (1 hi ) 2 K /N
ti = e i
(1hi )
ei = yi ybi
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Classification techniques
Can be supervised or unsupervised. In the unsupervised case, the
detector is trained by finding a hypersphere of the minimum radius
r that contains most of the data points from a set Xt .
The function f (x) used to categorize x is:
N
X
f (x) = H (x, x) 2
i (xi , x) + T
(8)
i=1
N
X
i (xi , xi )
i=1
subject to
PN
i=1 i
N X
N
X
i j (xi , xj )
i=1 j=1
Antanas Verikas
(9)
Introduction
Outlier detection
Process monitoring
Examples
Basic information
Distance from the data center
Residual analysis, Influence measures
Classification techniques
Contradictory Data?
CDC
RA
AANN
Fuzzy Expert
Neural Network
Inlier
Outlier
Introduction
Outlier detection
Process monitoring
Examples
M
X
zj2
;
2
j=1
sj
SPEy =
K
X
(yi ybi )2
(10)
i=M+1
where sj2
zj = pT
j y
is the variance of zj .
is the jth score , where pj is the jth column of the
loading matrix P, and y is the multivariate process output.
2 is:
An upper limit (UL) for the TM
2
TUL
=
M(N 1)(N + 1)
F (M, N M)
N(N M)
Antanas Verikas
(11)
Introduction
Outlier detection
Process monitoring
Examples
ka 1=nKa
PCA and PLS based monitoring
for bk>0,
where
a=[a1 a2. . .an]T. The centered
Kernel
PCA-based
monitoring
Monitoring
by
AANN
Introduction
Outlier detection
Process monitoring
Examples
Introduction
Outlier detection
Process monitoring
Examples
= t
k=1
t + (c )
e (x, x) tT t)
(K
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Objective function F
F = SSE + (1 )
P P
s
= Si=1 Sj=i+1 siji sj
si , sj variance of ui and uj
sij covariance of ui and uj
For the ith data point
P
SPEi = R
xij )2
j=1 (xij b
Introduction
Outlier detection
Process monitoring
Examples
105
Fig. 2. Non-linear scores obtained for the 102 nominal runs (+) where each score represents a run (multiway approach). Three faulty runs () were
projected on the model (left: confidence limits assuming normal distribution, 95 and 99%; right: confidence limits based on the density estimation method,
95 and 99%).
Figure: Non-linear scores obtained for the 102 nominal runs (+). Three
faulty runs () were projected on the model (Left: confidence limits
assuming normal distribution, 95 and 99%;PRight: confidence
limits based
N
xxi
1
on the kernel density estimate b
f (x) = Nh
,
95
and
99%).
K
i=1
h
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Fig. 7. On-line monitoring, snapshot-1. Normal trending can be observed both in the scores and in the SPE plots.
Figure: Normal trending is observed in both the scores and SPE plots.
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Fig. 7. On-line monitoring, snapshot-1. Normal trending can be observed both in the scores and in the SPE plots.
Fig. 8. On-line monitoring, snapshot-2. As soon as the failure of sensor 11 is introduced, it manifests itself as dierent cluster in the lower left scores
plot. In addition, the SPE indicates this event by spotting clearly above the 95% condence limit.
Introduction
Outlier detection
Process monitoring
Examples
Fig. 9. On-line monitoring, snapshot-3. After sensor 11 got back to normal, now the scores plots point the location of the dark clusters to dierent
place, indicating a new type of failure.
Fig. 10. On-line monitoring, snapshot-4. Flooding manifests itself in more process signals as time passes. Activity in the scores plot also indicates an
abnormal trending in the system.
Introduction
Outlier detection
Process monitoring
Examples
SPE2 = kx2 b
x2 k2
.....................................
xp = [x1 (t), x2 (t), ..., xp (t 1)]
SPEp = kxp b
xp k2
Introduction
Outlier detection
Process monitoring
Examples
cov (xi , xj )
xi xj
(12)
Introduction
Outlier detection
Process monitoring
Examples
Bayesian approach
Two sources of errors: the noise on the target t2 (x), and the
variance of the output due to weights (parameters) uncertainty
w2 (x). These two terms are independent, thus:
y2 (x) = w2 (x) + t2 (x)
(13)
(14)
(15)
NN-based approach
(10)
and
form:
hi t fh
X
na
Introduction
OutlierI detection
Wijmonitoring
p t j
Process
Examples
j0
nc
X
W H h l t
Chemical industry
Forest industry
nTextile
b
X
industry
^y t k
Machinery
WikR ^
Electronics industry
k1
Paper-making industry
Printing industry
1 bH
i
il
Predicting online biomass
concentration
l1
an RNN, the in
processed (scaled
range [-1,1]. This
area of the hidd
case, the output
the range [-1,1].
Recurrent NN
to pcapture
dynamic
information
theneural
input-output
^y isinthe
where
is the neural
network
input, ^
network
and h volume,
is the hidden
feed output
rate, liquid
and neurons
dissolvedoutput.
oxygen.
data. Inputs:
Introduction
Outlier detection
Process monitoring
Examples
120
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
278
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
knots;pixels
b sound
knots;
c shakes
with feature set 1 and 3232
regions
with aand
pessimistic
boundary. Error escapes and thus non-classied defects were
2.5% of total defects.
can
conclude
that these
defectspresent
can beindetected
successThirdly,
the
wrinkles
(crease)
the fabric
under
Chemical
industry
fully. However,
the degree
of segmentation
the amount
inspection
can alter
gray-level
arrangement (or
of neighboring
Forest
industry
of noise)
ofhence
defects
varies from
pixels,
and
generate
false defect
alarm to
or defect.
noise. Three posTextile
industry
sible reasons can be attributed to this variation. Firstly, the
Machinery
texture Real-time
of fabric
under
inspection is not exactly as same as
Electronics
industry
2.5.1.
implementation
Paper-making
thatCan
used
theindustry
training
FFN. Thisincan
be due toLet
(i) us
elastictheforFFN
be implemented
real-time?
conPrinting
industry
ity
of
yarns
and/or,
(ii)
variations
in thePCA)
alignment
sider the 5/1 FFN withrandom
25 inputs
(ignoring
used
of fabric with respect to camera. Secondly, the impurities
present in the yarns of the fabric also contribute to this noise.
Thirdly, the wrinkles (crease) present in the fabric under
inspection can alter gray-level arrangement of neighboring
pixels, and hence generate false alarm or noise.
(d)
(g)
(j)
(b)
(a)
(e)
(d)
(h)
(g)
(k)
(j)
(b)
(c)
(e)
(f)
(h)
(i)
(k)
(m)
Fig. 5. Plain weave fabric samples with defect double weft, thin bar, broken ends, sleck-pick in (a), (d), (g) and (j), respectively; their
Antanas Verikas
360
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
1
2x y
2
x2
+y
x2 y2
exp 12
exp{2jfo q(x, y )}
Antanas Verikas
Outlier detection
monitoring
RProcess
pq |
Examples
Textile industry
Machinery
In stage three, the three fused imag
Electronics (4)
industry
tained, in stage two are combined u
Paper-making industry
Printing industry a single and final fused image K(x,
1 (2)
Gabor filtering-based detection of textile
defects
K (x,
y) = [K 12
(x, y) + K 23 (x,
3
Antanas Verikas
Kp (x, y ) =
P4
2
q=1 [Spq (x, y )]
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Fig. 4 Fabric Fault Segmentation Results. a Ideal Weaves: from top to bottom Ribs top, Oxford, Twill, Drill, Twill and Satin; b Faulty Weaves:
from top to bottom Double pick, End out, Multiple break, Hook damage, Selvage float and Weft crack; c Final fused images; d Final segmented
images
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
ponents
130
Outlier detection
Process monitoring
Examples
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Fig. 8. Detection results for TILDA images drawn from c1r3 texture class, = 2.5.
Fig. 6. Detection results for TILDA images drawn from c1r1 texture class, = 2.5.
Fig. 10. Detection results for TILDA images drawn from c2r2 texture class, = 2.
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Fig. 21. The MQE of three degradation processes (a) Test 1, bearing 3 (b) Test 1, bearing 4 (c
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
138
SOM is trained using data from all the operation conditions. The
U-matrix (matrix of distances between weights of SOM nodes) is
used to visualizeH.the
of the
degradation
Qiu et trajectory
al. / Advanced Engineering
Informatics
17 (2003) 127140process.
Fig. 19. The U-matrix and trajectory of degradation process of bearing 3 from test 1 (inner race failure).
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
can beindustry
integrated to produce a T value through T s
Chemical
(Eq.
(7)). Therefore, one image of 256 256 pixels ca
Forest
industry
Textile
industry
struct
a T2 distance diagram with 64 64 T2 values
Machinery
An
Electronics image
industrycan be divided into many multivaria
cessing units.
Paper-making
industryLet a multivariate processing u
Printing
M(x,industry
y) and an original image be f(i, j). Fig. 3 prese
Wavelet
characteristics
Wavelet transform
Image
synthesis
AR
Approximated
characteristic
DR
Detail
characteristic
T2 distance diagram
Z : T 2 value
Defect locations
(high energy)
Y : 255
X : 255
T 2 = n ( X X ) ' S 1 ( X X )
Fig. 2. The processes of the wavelet-based multivariate T2 model.
Antanas Verikas
Normal location
(low energy)
800
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Fig. 15. Defects in dierent texture surfaces and their detection results by the the wavelet-based T2 statistical method.
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
The task
To develop soft sensors for on-line detection of abnormal paper
formation variations in machine direction in various frequency
regions.
Forming
section
Press
section
Drying
section
Reel
section
Calender
Headbox
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Antanas Verikas
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Processing flowchart
On-Line Monitoring
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
(16)
where
(17)
(18)
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Feature extraction
Ei =
1 X
P(u)
Ni
(19)
1
max P(u)
E i uRi
(20)
uRi
Emi =
i = Ni2
Pi (u)
1 i2
Ni
uRi Pi (u)
(21)
1 X n
Pi (u) log Pin (u)
log Ni
(22)
Xh
P
uRi
Mi =
uRi
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
N
X
i (xai , xai )
i=1
N X
N
X
i j (xai , xaj )
(23)
i=1 j=1
P
subject to N
i=1 i = 1 and 0 i 1/N, i = 1, ..., N.
The function f (xa ) used to categorize the data point xa is:
N
X
i (xai , xa ) + T
f (xa ) = H (xa , xa ) 2
i=1
(24)
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Data
Data sets A and C have been recorded on production line and
correspond to 920 and 70 km of newsprint divided into
M = 9200 and 700 blocks, respectively.
The data set A was randomly split into training, validation,
and test subsets. Ten different random splits were used.
All data of the set C have been used for testing. In the 700
data blocks there were 70% and 75% of outliers in the
lowest and highest frequency region, respectively.
The abnormalities screened were located in the region of
0.2 0.4 mm and 1.6 3.2 m.
Antanas Verikas
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Set\n
Test
Test
10
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Graphical process
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Printing press
Dampening
Ink bath
Blanket cylinder
Paper path
Plate cylinder
Impression cylinder
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Colour deviation
and areas to measure
the amount of colour.
2007-11-21
Halmstad
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Estimating
the amount
ink computing
Estimating
the amount
of ink byofsoft
RGB
Neural Net
CMYK
2007-11-21
Halmstad
Antanas Verikas
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
400
500
600
Wavelength (nm)
P
R()S()F ()
D = lg P
S()F ()
Antanas Verikas
700
(25)
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Introduction
Outlier detection
Process monitoring
Examples
450
500
550
600
Wavelength (nm)
650
700
x = RGB triplet
k = (xi , x)
a = BT k
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0
400
450
500
550
600
Wavelength (nm)
650
700
Antanas Verikas
0.1
400
450
500
550
600
Wavelength (nm)
650
700
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
f (Q)g ()
(26)
where g () is a measure
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
ranking
PrintQuality
quality ranking
by man
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
Sample
Ann
Anna R
Sofia
Peter
Frank
Stellan
Bernd
Dan
6.00
7,63
7,38
3,25
3,75
1,75
1,25
Cristofer
Magnus
Wolfgang
Kerstin
Maria
Mean
Marcus
Mean
2007-11-21
6.00
7.71
7,28
3.0
3.93
4.93
1.57
1.57
Antanas Verikas
Soft Sensors for Monitoring
Halmstad
19
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
R
0.9524
0.9524
0.9762
0.9762
p
0.0003
0.0003
0.0000
0.0000
Antanas Verikas
Introduction
Outlier detection
Process monitoring
Examples
Chemical industry
Forest industry
Textile industry
Machinery
Electronics industry
Paper-making industry
Printing industry
(27)
K X
X
i=1 xj i
Antanas Verikas
dij2
(28)