Anomaly Detection in SMTP Traffic
Hao Luo, Binxing Fang, Xiaochun Yun
School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001
{luohao, bxfang, yxc}@pact518.hit.edu.cn
Abstract
We investigate an effective and robust mechanism
for detecting SMTP traffic anomaly. Our detection
method cumulates the deviation of current delivering
status from history behavior based on a weighted sum
method called the leaky integrate-and-fire model to
detect anomaly. The simplicity of our detection method
is that the method need not store history profile and
low computation overhead, which make the detection
method itself immunes to attacks. The performance is
investigated in terms of detection probability, the false
alarm ratio, and the detection delay. Our results show
that leaky integrate-and-fire method is quite effective
at detecting anomaly in the SMTP traffic. Compared
with non-parametric Cumulative Sum method, the
evaluation results show that our detection method has
lower false alarm ratio and higher detection
probability.
1. Introduction
The SMTP [1] is used as the basis for most
electronic mail. Email is the most popular Internet
service now [2], and it allows people to communicate
by exchanging electronic messages globally. These
messages can be delivered in a few seconds to a couple
of hours. An added attraction is the relatively low cost
of sending large messages. Combined, these benefits
give users a convincing argument for access to email,
and thus the connection of their systems to the Internet.
SMTP is a simple protocol and contains only a few
basic commands. There are several security threats that
associated with these commands and Denial-of-service
attack is one of the most popular threats of SMTP.
Denial-of-service attacks based on SMTP are aimed at
flooding a network or computer with massive email
messages to prevent legitimate use. In most cases a
computer is affected because it cannot handle the load
created by receiving large numbers of messages at the
same time, or running out of storage space, or cannot
handle large messages [3]. An example of Denial-of-
Proceedings of the Third International Conference on Information Technology: New Generations (ITNG'06)
0-7695-2497-4/06 $20.00 © 2006 IEEE
service attacks of SMTP is error mails bouncing back
attack [4], and a report shows on October 2003, at least
two domains in the United States had been received
hundreds of thousands of error mails from all over the
Internet [5].
Another important threat of SMTP is email-based
viruses, and email viruses have become one of the
major Internet security threats today. An email virus is
a malicious program, which hides in an email
attachment, and becomes active when the attachment is
opened. A principal goal of email virus attacks such as
Melissa is to generate a large volume of email traffic
over time, so that email servers and clients are
eventually overwhelmed with this traffic, which
effectively disrupting the usage of the email service.
Modern email viruses are more damaging, taking
actions such as creating hidden backdoors on the
infected machines that can be used to commandeer
these machines in a subsequent coordinated attack.
In this paper, we propose an effective and robust
method for detecting SMTP traffic anomaly, which is
complementary to the alert threats mentioned above.
The effect of our detection method is that the method
need not store history profile and low computation
overhead. Instead of monitoring the ongoing traffic at
the front end or the victim server, our method check
the SMTP server’s delivery log. The benefit of
checking SMTP log to detect traffic anomaly is that we
need not monitor raw traffic of the server exchanging
and make computation overhead very low and the
SMTP log provides detail information about receiving
and sending status. The key feature of our method is to
utilize the leaky integrate-and-fire model to cumulate
the deviation of current delivering status from the
history status. The leaky integrate-and-fire model is a
weighted sum model, and the newer input data will
play a more important role in the result. The old data
will be dropped from the result by a weighted factor. In
this way, our method archives high detection
probability and lower false alarm ratio. The efficacy of
our detection method is validated by simulating
experiment with real background test data.
 The remainder of this paper is organized as follows.
The section 2 shows the related works of network
anomaly detection. In section 3 we discuss the leaky
integrate-and-fire model based SMTP traffic anomaly
detection method. In section 4, we evaluate our
anomaly detection method and compare our method
with a non-parametric Cumulative Sum method.
Finally, section 5 presents our conclusions.
2. Related Works
It is possible to continuously track the behavior of
the network by online learning and statistical
approaches. Statistical analysis has been used to detect
both anomalies corresponding to network failure, as
well as network intrusions [6].
A predictive detection method [7] was used in web
server anomaly detection, by analyzing time series
measurements of the number of http operations per
second. The statistical model considered both seasonal
and trend components, which were modeled using a
Holt-Winters algorithm. Time correlations were
modeled using a second order auto-regressive model.
After removing the non-stationarities from the time
series measurements, anomalies were detected using a
generalized likelihood ratio algorithm. This method
need store history profile for future using.
A wavelet approach was proposed and implemented
by Paul and others [8], they used wavelet filter to
process four classes of network traffic anomalies:
outages, flash crowds, attacks and measurement
failures. Their results showed that wavelet filters were
quite effective at exposing the details of both ambient
and anomalous traffic. However, the authors also
mentioned that their signal analysis method could not
detect anomalies in real time.
The authors of [6] proposed approaches for
detecting SYN flooding attacks using CUSUM-type
algorithm, and this case made use of the standard
sequential change point detection approach. The
approach of [6] applied the time series measurements
of the difference of the number of SYN packets and the
corresponding FIN packets in a time interval. The
simulation results have shown that SYN flooding
attacks can be detected with high accuracy by
CUSUM-type algorithms.
3. SMTP Anomaly Detection
In this section, a real time statistical analysis
method we developed using the theory of Leaky
Integrate-and-fire model is discussed. Like most
statistical anomaly detection systems, we compare the
observed sequence with the profile in representing the
Proceedings of the Third International Conference on Information Technology: New Generations (ITNG'06)
0-7695-2497-4/06 $20.00 © 2006 IEEE
server’s normal behavior. However, unlike the
traditional network intrusion detection system that
detects the anomaly directly by the deviation of current
behavior from the profiled normal history behavior,
our method cumulates the deviation in a period to
detect the anomaly according to the method of
Integrate-and-fire model described. Compared with the
CUSUM-type algorithms, the detection algorithm
based on Integrate-and-fire model is more sensitive to
current network status.
Our method uses SMTP server’s log to detect the
anomaly. SMTP server log provides a mail server’s
receiving and sending information including failure
message such as mail with invalid account. The log
also includes delivery time of each mail. Since our
work is detecting SMTP traffic anomalies, this data
source is sufficient.
3.1. The SMTP Behavior Deviation Evaluation
Let {xn, n=0,1,…} be the serial of mail numbers that
a mail server received within one sampling period, and
let {yn, n=0,1,…} be the corresponding sent mail
numbers in the same sampling period. We define {Δn,
n=0,1,…} be the number of received mails minus that
of the corresponding sent mails collected within one
sampling period.
In general, the mean of {Δn} is dependent on the
accounts number of SMTP server, and it may also
depend on the access patterns, for example, varying
with time of the day and week. To make our detecting
algorithm more general, we should eliminate these
dependencies. Thus, {Δn} is normalized by the average
number Yn of {yn}. Yn can be computed by using an
exponentially weighted moving average (EWMA) of
previous measurements.
Yn = β Yn −1 + (1 − β ) yn (1)
Where β is the EWMA factor that represents the
memory in the estimation. Define Xn=Δn/ Yn , and { Xn }
is no longer dependent on the network size or time-of-
day.
So we can define the deviation of SMTP behavior
for a given interval n as:
Dn = X n − X n−1 (2)
Where X n is the mean rate of Xn and estimated from
measurements prior to n-1. The mean X n is also
computed by EWMA method. The deviation of SMTP
behavior Dn is used as input data of our anomaly
detection method.
3.2. The Leaky Integrate-and-fire Model
 Leaky integrate-and-fire model have been proposed Therefore:
as model of neurons for a long time. It can be used for L '( n) = exp( −
1
)( L '( n − 1) + Dn ) (8)
processing time-varying signals [9] and also can be K
used in powerful computing systems [10]. The simplest As the negative SMTP behavior deviation means no
form of integrate-and-fire model consists of a resistor anomaly in our detection, according (8), here we let
R in parallel to a capacitor C driven by an external ⎧ 1 +
current I(t). The voltage V(t) across the capacitor C is ⎪ L (n) = exp(− )( L(n − 1) + Dn ) (9)
⎨ K
compared to a threshold δ. If V(t)=δ at time t an output ⎪⎩ L (0) = 0
spike φ(t) is generated and V(t) is reset to an initial be our network status function. Where n>0 and x+ is
voltage Ur. Between spikes, the voltage of a leaky equal to x if x > 0 and x+ is equal to 0 otherwise. We
integrate-and-fire model is governed by: will use L(n) in making detection decisions. Here we
dV (t ) V (t ) I (t ) (3) call K as cumulating factor.
=− +
dt RC C Let H represents the anomaly threshold. At interval
Suppose that a spike has occurred at ti. For t>ti the n, if L(n)>H, an alarm will be signaled at time n,
stimulating current is I(t). The V(t) can be expressed otherwise the network status is normal. If the alarm is
as: signaled at time n, L(n) will be reset to 0.
t − ti 1 t −t s (4)
C ∫0
V (t ) = U exp(− )+ exp(−
i

r )I (t ) ds
RC RC
When leaky integrate-and-fire model is used to
detect SMTP anomaly, the deviations of SMTP
behavior in each interval of t>ti are inputted, and the
V(t) are tested as alarm condition. The detail of
detection algorithm will be described in Section 3.3.
3.3. Anomaly Detection Approach
In our SMTP traffic anomaly detection approach,
the SMTP health status is obtained by the output of
leaky integrate-and-fire model. In the process of
capacitor recharging, when the input current is
constant, the earlier input current will raise voltage
faster. Therefore, in our detection method, the
deviation of SMTP behavior Dn will be inputted into
leaky integrate-and-fire model from the current interval
3.4. Parameter Specification
The tuning parameters of above algorithm are the
cumulating factor K for computing the network health
status, the alarm threshold H, and the EWMA factor β.
In general, the EWMA factor β is chosen as 0.98[6],
here we also chose β=0.98 as our EWMA factor in
experiments. To implement our leaky integrate-and-
fire anomaly detection algorithm, we still need to
specify two tunable parameters: K and H. The
cumulating factor K decides how we cumulate the
SMTP status deviation to detect the anomaly, and the
alarm threshold H depends on K. Dn has different
contribution to L(n) with different K. Fig.1 shows the
percentage of (exp(-n/K) Dn) in L(50), where we set Dn
=1, n=1,2,…,50.
65

to the last spike occurred interval. That means we input 60

55 K=1
Percentage of exp(-n/K) in L(n)

current Dn first, and than the one just before current, 50 K=2
K=3
and so on. In this way, the current SMTP delivery 45
40
K=4
K=5
status will play a more important role in the detection 35 K=6
K=7
result. Because Dn is the discrete value, suppose that a 30
25
K=8
K=9
spike has occurred at interval nk, the output of leaky 20 K=10
15 K=11
integrate-and-fire model at interval n can be gotten 10

from (3) as: 5

0
n − nk 1 n−n n − nk − i + 1
k

∑ (5) 1 6 11 16 21 26 31 36 41 46
V '( n) = U exp(−
r )+ exp( − )D nk + i n
RC C i =1 RC
Fig. 1. Results of exp(-n/K) with different K
Let Ur=0, L’(n) = CV’(n) and K=RC, from (5) we
We can see clearly from Fig.1 that the smaller K,
get:
n−n
the more contribution exp(-1/K) does, and the shorter
k
n − nk − i + 1
L '( n) = ∑
exp( −
i =1
)D (6)
K
nk + i
history profile is referred. When K=1, exp(-1/5)
contributes 63.21% to L(50), and about 8 intervals are
So we have: evidently referred in L(50); when K=15, exp(-1/15)
n − nk n − nk + 1 contributes 8.79% to L(50), and all 50 intervals are
L '(n − 1) = exp( − ) Dnk +1 + exp( − ) Dnk + 2 (7)
K K referred in L(50). Here we can see when K=5,
1 3
+... + exp(− ) Dn −1
K ∑ exp(− n / 5) contributes about 45% of integrate result,
n =1

Proceedings of the Third International Conference on Information Technology: New Generations (ITNG'06)
0-7695-2497-4/06 $20.00 © 2006 IEEE
 10
and
∑ exp(−n / 5) contributes about 91% of result. This
n =1
means when we chose K=5, the calculating result not
only emphasizes the first three inputs, but also refers
enough history information. So in our detection
algorithm, we chose K=5 as our cumulating factor.
Suppose we should raise an alarm when xn increases
to 1.6 times of normal value. When we decide
cumulating factor K=5, we can calculate H by the
following algorithm:
Function GetThreshold(K)
FOR I = 1 to TO 10 DO
e = e + exp(-i/K)
RETURN e*0.6
EndFunction
When we set K=5, we can get H=2.4 following the
above algorithm.
4. Performance Evaluation
In this section, we firstly chose parameters of our
method. In order to compare our method with the
CUSUM-type algorithm described in [6], we also
chose parameters for algorithm in [6]. The algorithm of
[6] is given by
g n = [ g n −1 + ( X n − a)]+ (10)
In addition to parameters choice, we evaluate how
the parameters of our detection algorithm affect the
detecting performance.
Secondly, we investigate the performance of our
leaky integrate-and-fire method presented in the
previous section for detecting SMTP traffic anomaly.
The performance metrics considered include the
detection probability, the false alarm rate, and the
detection delay. The detection probability (DP) is the
percentage of attacks for which an alarm is raised, the
false alarm ratio (FAR) is the percentage of alarms that
did not correspond to an actual attack, and the
detection delay (DD) is the detection delay after the
attack starts.
Our experiments use actual SMTP server delivery
logs taken from our campus mail server as background
data. We use mail server’s log during 2.5 days and
measure the SMTP deliveries in one minute. Our test
set includes 120412 receiving mails information and
80358 sending mails information with average
receiving speed 33.45 mails per minutes and sending
speed 22.32 mails per minutes.
The attacks were generated synthetically, and this
allowed us to control the characteristics of the attacks,
hence to investigate the performance of the detection
algorithms for different attack intensity. The typical
attacking duration observed in the Internet is 10
minutes [11], therefore the attacks are generated with
Proceedings of the Third International Conference on Information Technology: New Generations (ITNG'06)
0-7695-2497-4/06 $20.00 © 2006 IEEE
mean duration 10 time intervals. The inter-arrival time
between consecutive attacks is random distributed in
60-180 time intervals with mean values 120 intervals.
Our detection method is not sensitive to the attacking
pattern: it can detect the attacks with both constant and
burst intensity attacks.
4.1. Parameter Selection
According to the directions described in section 3.4
and [6], we chose K=5 and H=2.4 for leaky Integrate-
and-fire anomaly detection method and chose a=1.1
and threshold TH=2.2 for CUSUM is our test set. In
order to evaluate the parameters we select, we
enumerate each possible combination of parameters of
two anomaly detection methods.
For leaky integrate-and-fire method, we test
threshold H from 2 to 4 increased by 0.1 and test
cumulating factor K from 1 to 15 increased by 1. For
CUSUM method, we test a from 0.6 to 1.6 with step
0.05 and test threshold TH from 0.6 to 6 with step 0.1.
We reserve the parameter pairs those can archive
average 100% detection probability in 10 round tests.
The test set is generated by overlapping constant
intensity attacks with the duration of 10 intervals (10
minutes). The intensity of attacks is 60% of mean
actual receiving mails rate.
For CUSUM algorithm described in [6], we select
parameter pair with a=1.1 and threshold TH=2.2, in
our test set, CUCUM method gets FAR=0.0077 and
DD=2.00. For our method, we select K=5 and H=2.4.
In the test set, our method gets FAR=0.0043 and
DD=0.7. Our select parameters have smaller false
alarm ratio and lower detection delay in all reserved
parameter pairs. Considering the tradeoff between false
alarm ratio and detection delay, the parameter pairs we
select for performance evaluation are suitable.
4.2. Evaluation of Cumulating Factor
Fig.2 shows the how the accumulating factor K
affects the false alarm ratio and detection delay, where
the threshold H is adjusted by the algorithm described
in Section 3.4. The Fig.2 is obtained by taking the
average of 10 runs.
Fig.2 shows the effect of cumulating factor K on
detection delay and false alarm ratio. As the
cumulating factor K decides the length of history that
the detection method uses. The bigger K, the longer
history is referred in making decision, and at the same
time, the bigger K, the lower weight of current
delivering status are considered in detection result.
This means the current delivering status influents
results less. The longer history may lead to long
 detection time because current delivery status is not 120
Received
Sent

Mails
80
sensitive to the final detection result. The smaller K, 40
0
the shorter history is considered and the bigger weight 0 720 1440 2160

Amplitude
27

of current networking status has, the faster we can 18

9
detect the anomaly, but at the same time, the final 0
0 720 1440 2160
detection results are more sensitive to the current 1

CUSUM
delivering status, it will make more false alarm. 0
3.5
0 720 1440 2160
Detection Delay

3.0 1

LIF
2.5
2.0
1.5 0
0 720 1440 2160
1.0
0.5 Time Intervals
b. Detection results of high intensity attacks
0.0
1 3 5 7 9 11 13
0.04 Fig.3. Detection Results
0.03 Fig.3a and Fig.3b show the results for leaky
FAR

0.02 integrate-and-fire method in the condition of low and

0.01
0.00
1 3 5 7
Fig.2. Effect of Accumulating Factor K
In our test set, when K=5, the detection results has a
good tradeoff between detection delay and false alarm
ratio.
4.3. Evaluation of Anomaly Detection
Our experiments consider attacks with constant
intensity, i.e. the attacks reach amplitude in one time
interval. Firstly we generate two types of intensity to
compare our method with CUSUM algorithm
described in [6], and there are low and high constant
intensity attacks. In low constant intensity attack, the
added attacks’ amplitude is 17 mails, and it is about
50% of mean normal SMTP receiving speed. The high
intensity attacks’ is about 77% of mean normal SMTP
receiving speed. Fig.3 shows the detection results.
100
Mails
high constant intensity attacks, respectively. The
horizontal axes in these figures are the number of time
9 11 13
K interval. In each graph, from top to bottom, we have
the SMTP deliveries trace with attacks, the attacks, and
the detecting results of CUSUM algorithm and leaky
integrate-and-fire method.
The above graphs show that our method has good
performance in both low intensity attacks and high
intensity attacks. In low intensity attacks, our method
yields a detection probability of 100% and false alarm
ratio 0.24%. In high intensity attacks, our method gets
similar results with 100% detection probability and
0.33% false alarm ratio. CUSUM algorithm has good
performance in high intensity attacks, but in low
intensity attacks, it can’t archive 100% detection
probability, and the false alarm ratio of CUSUM
algorithm is much worse than our method
Secondly we generate a serial of different intensity
attacks to evaluation our detection performance. The
detail average results with 10 runs are shown in Fig.4.
Received The horizontal axis in Fig.4 is attack mails injected per

Sent
50 interval.
0 0.008
0 720 1440 2160
Amplitude

17.0 0.006
FAR

8.5 0.004

0.0 0.002
0 720 1440 2160 15 17 19 21 23 25 27 29
1
CUSUM

2.5
2.0
1.5
DD

0 1.0
0 720 1440 2160 0.5
1
15 17 19 21 23 25 27 29
LIF

1.0

0.9
DP

0 CUSUM
0 720 1440 2160 LIF
0.8
Time Intervals
0.7
a. Detection results of low intensity attacks
15 17 19 21 23 25 27 29
Attack Amplitude

Fig.4. Average Detecting Results of different

Intensity Attack
From Fig.4, we can see clearly that the detection
probability of our method is higher than CUSUM
algorithm, but when the attack intensity is larger than
68% of mean rate of normal SMTP receiving speed (23
attack mails per interval), both can archive 100%

Proceedings of the Third International Conference on Information Technology: New Generations (ITNG'06)
0-7695-2497-4/06 $20.00 © 2006 IEEE
 detection probability. But our method gets 100%
detection probability when the attacks are larger than
50% of mean rate of normal SMTP receiving speed (17
attack mails per interval).
The detection delay of our method is similar with
CUSUM algorithm, in low intensity attacks with 15
external attack mails injected, the detection delay of
both methods are about 2.5 intervals, and in high
intensity attacks with 29 attacks mails are injected per
interval, two methods only need about 1 interval to
raise alarms.
Our method has better false alarm ratio than
CUSUM algorithm in all scenarios. The average false
alarm ratio of our method is about 0.4% and 0.7% for
CUSUM algorithm.
The difference in the performance of our detecting
method and CUSUM method is our method uses
weighted sum method to cumulate the behavior
deviation and CUSUM method treats all deviation
fairly, therefore, our method is more sensitive to
current network status than CUSUM method, therefore
our method has better detection probability and lower
false alarm ratio than CUSUM algorithm, especially in
low intensity attacks. Detection of low intensity attacks
is important because early detection of anomaly with
increasing intensity attacks will enable defensive
action to be taken earlier.
5. Conclusions
In this paper, we propose an effective and robust
mechanism for detecting SMTP traffic anomaly. Our
detection method cumulates the deviation of current
delivering status based on the leaky integrate-and-fire
model, which is a weighted sum method. The effect of
our detection method is that the method need not store
history profile and low computation overhead. Our
results show that leaky integrate-and-fire method is
quite effective at detecting attacks, especially in low
intensity attacks. Compared with non-parametric
Cumulative Sum method, the evaluation results show
that our detection method has lower false alarm ratio
and higher detection probability. Especially in low
intensity attacks, our method detecting accuracy is
Proceedings of the Third International Conference on Information Technology: New Generations (ITNG'06)
0-7695-2497-4/06 $20.00 © 2006 IEEE
much higher and the false alarm ratio is lower than
CUSUM method.
6. References
[1] J. Postel, Simple Mail Transfer Protocol, RFC 821,1982
[2] R.Caceres, P.Danzig, S.Jamin, and D. Mitzel,
”Characteristics of widearea TCP/IP conversations”,
Computer Communication Review,SIGCOMM, ACM Press,
New York, NY,USA, 1991, pp.101–112.
[3] B. Harris, R. Hunt, “TCP/IP security threats and attack
methods”, Computer Communications, Elsevier, 1999,
pp.885-897
[4] N.Yamai, K.Okayama, T.Miyashita, S.Maruyama, and
M.Nakamura, “A Protection Method against Massive Error
Mails Caused by Sender Spoofed Spam Mails”, Proceeding
of the 2005 Symposium on Application and the Internet,
2005, IEEE Computer Society, Italy, pp.384-390
[5] Brian McWilliams, “Wired News: Time-Travel Spammer
Strikes Back”, Lycos, Inc., http://www.wired.
com/news/technology/0,1282,61026,00.html, 2003.10.
[6] H. Wang, D. Zhang, and K. G. Shin, “Detecting syn
flooding attacks”, IEEE INFOCOM, New York City, NY,
2002. pp. 1530-1539
[7] J. Hellerstein, F. Zhang, and P. Shahabuddin. “A
statistical approach to predictive detection”. Computer
Networks, Elsevier, 2001, pp. 77–95.
[8] P. Barford, J. Kline, D. Plonka, and A. Ron, “A signal
analysis of network traffic anomalies”, SIGCOMM, ACM
Press, New York, NY, USA, 2002. pp. 71-82
[9] L.S. Smith. “Onset-based sound segmentation”,
Advances in Neural Information Processing Systems. MIT
Press, 1996, pp. 729-735.
[10] R.D. Patterson, M.H. Allerhand, C. Giguere, “Time-
domain Modelling of Peripheral Auditory Processing: A
Modular Architecture and a Software Platform”. Journal of
the Acoustical Society of America, 1995, pp.1890-1894
[11] D. Moore, G. Voelker and S. Savage, “Inferring Internet
Denial of Service Activity”, Proceedings of USENIX
Security Symposium 2001, 2001.