Snort Architectural Document

Based on snort-2.9.4.6 open source code

Contents

1 Introduction .......................................................................................................................................... 4
2 Snort Components ................................................................................................................................ 4
2.1 Snort Executable ........................................................................................................................... 5
2.2 DAQ ............................................................................................................................................... 5
2.3 Dynamic Preprocessors ................................................................................................................. 5
2.4 Detection Engine ........................................................................................................................... 5
2.5 Snort Text Rules ............................................................................................................................ 5
2.6 Dynamic rules ................................................................................................................................ 5
2.7 Snort Configuration ....................................................................................................................... 5
3 Snort Architectural Overview................................................................................................................ 6
4 Snort In Execution ................................................................................................................................. 7
5 Snort Example Topology ....................................................................................................................... 8
6 Snort-Initialization ................................................................................................................................. 9
6.1 InitSignal ...................................................................................................................................... 10
6.2 InitGlobal ..................................................................................................................................... 10
6.3 ParseCmdLine.............................................................................................................................. 10
6.4 RegisterOutputPlugins ................................................................................................................ 10
6.5 FileApiInit .................................................................................................................................... 10
6.6 ParseSnortConf ........................................................................................................................... 10
6.7 Load Plugins ................................................................................................................................ 11
6.8 Initialize Plugins........................................................................................................................... 11
6.9 Configure Plugins ........................................................................................................................ 11
6.10 Parse Rules .................................................................................................................................. 11
6.11 Doing Optimizations for faster packet detection........................................................................ 11
6.12 Control Socket Initialization ........................................................................................................ 11
7 DAQ ..................................................................................................................................................... 12
7.1 DAQ initialization ........................................................................................................................ 12
7.2 DAQ Start .................................................................................................................................... 13
7.3 Data Acquisition .......................................................................................................................... 14
8 Decode ................................................................................................................................................ 15
8.1 Decoding process ........................................................................................................................ 16
9 Preprocess ........................................................................................................................................... 17
9.1 Dynamic Preprocessors ............................................................................................................... 17
9.2 Initialization of Dynamic Preprocessor ....................................................................................... 17
9.3 Sequence Diagram of Initialization of Dynamic Preprocessor .................................................... 18
9.4 Non Dynamic Preprocessors ....................................................................................................... 19
10 Detection ......................................................................................................................................... 20
10.1 Rule Tree ..................................................................................................................................... 22
10.2 How Detection works .................................................................................................................. 23
11 Output Module ............................................................................................................................... 24
12 Snort Maintenance Thread ............................................................................................................. 25





1 Introduction
This document describes the snort architecture based on the open source code Snort-2.9.4.6.
2 Snort Components















Fig: 1





Snort Executable
Detection
Engine
Dynamic Rules
Dynamic
Preprocessors
DAQ
Snort
Configuration
Output
Plugin
Snort Text
Rules
2.1 Snort Executable
The main part of snort system which loads and controls other components. Snort Executable
communicates with other components to perform various tasks like data acquisition, decoding
protocols, normalizing network traffic, detecting and preventing attacks and create alerts and logs.
2.2 DAQ
Third party library used by snort executable to acquire data from network.
2.3 Dynamic Preprocessors
Shared library loaded by snort executable to normalize network traffic.
2.4 Detection Engine
Detection engine is responsible for controlling detection process. Detection engine evaluates network
packet against a known attack signatures. These signatures are contained in snort rules.
2.5 Snort Text Rules
Snort rules in the form of text files.
2.6 Dynamic rules
Snort rules in the form of shared library. These libraries are loaded by snort dynamically.
2.7 Snort Configuration
Configuration file used to configure different components in snort system.











3 Snort Architectural Overview
The flow of snort working is as shown in Fig 2.





Maintenance thread Main thread

Yes yes Yes
No No










Fig: 2
Snort
Initialization
DAQ

Decode
Preprocess
Detect
Start
Exit
Periodically checkup configuration to
inform Main thread configuration
changes
Exit
Snort started with snort initialization. Under normal IPS mode, snort works under two threads. One is
the main-thread and the other is the maintenance thread. All the major snort operations are done under
the main-thread. The maintenance thread is for performing periodic checkup for the proper functioning
of snort and also for reloading the snort configuration data and its preprocessing on error conditions
and also on snort rule updates. A set of checkup functions are executed during periodic checkups.
4 Snort In Execution


















Fig: 3


Snort Executable
Dynamic Preprocessor
Dynamic Rules
Text Rules
DAQ
USER SPACE
KERNEL SPACE

System Call
Socket
5 Snort Example Topology












Client Machine 1 Client Machine 2

Fig: 4










Snort IPS
172.20.1.186
192.168.50.1
I/F 1 I/F 2
192.168.50.2
I/F 3
Internet
Remote
Monitoring
Workstation



6 Snort-Initialization
Snort performs a series of operation while initializing. Fig: 2 show the order of operations.




















InitSignals
InitGlobals
ParseCmdLine
RegisterOutputPlugins
FileApiInit
ParseSnortConf
Initializes Plugins
Configure plugins
LoadPlugins








Fig: 5

6.1 InitSignal
Initialize the snort specific signals with the corresponding signal handler.
6.2 InitGlobal
Initialize the netmask array and protocol-names array. Netmask [] is an array of 32 elements and
protocol_names[] array contains 256 entries. Netmask[] contains the hexa decimal mask values.
Protocol_names[] contains different protocol names which will be accessed from different part of the
program by giving its corresponding ID.
6.3 ParseCmdLine
Snort command line arguments are parsed and assigned to the concerned variables.
6.4 RegisterOutputPlugins
There are many initialization functions in the preproc-list. Here the preprocessor keywords are linked to
the corresponding initialization function in the preproc-list.
6.5 FileApiInit
The FileAPI struct will be initialized from here.
6.6 ParseSnortConf
Snort config file is parsed and the information is filled in the SnortConf structure. The details of the
command line arguments are also updated in the SnortConf structure.
Parse Rules
Do optimizations for fast Packet Detection
Control Socket Initialization
6.7 Load Plugins
A link list of _DynamicEnginePlugin structure is there to store the informations of all dynamically located
plugins. The plugins handle, meta data, initialization function and its version informations are stored in
this structure. The loaded plugins includes the snort rules plugins and preprocessor plugins.
6.8 Initialize Plugins
Initialization involves mapping snort specific keywords with the corresponding initialization function
pointers in the respective plugins.
6.9 Configure Plugins
All the plugins are configured and their information is stored in a linked list. The respective Config
structure is selected from the linked list based on the keyword associated to it. Each plugin is
associated with a keyword.
6.10 Parse Rules
Using the default parser policy, the snort config file is parsed at first and information are stored in
SnortConfig structure. It then process individual snort rule and add it to the snort rule list. It
extensively uses PCRE library functions while parsing the snort rules.
6.11 Doing Optimizations for faster packet detection
Some optimization for the creation of faster Packet inspection engine is done here. Creates two groups
to categorize snort rules. The rules contain content option are grouped into one group and rules do
not contain content option form the second group. The rules with same destination ports are again
form subgroups within these two main groups.
6.12 Control Socket Initialization
The entire socket related initialization is done here.










7 DAQ
DAQ module does data acquisition from network During snort initialization, it loads all the libraries
needed for data acquisition. For DAQ activities, snort communicates with DAQ libraries through a
mediator library( shim layer ). Mediator library ( shim layer ) communicates with specific DAQ library
which is specified in the snort configuration file.
7.1 DAQ initialization
During DAQ initialization, it opens PF_PACKET raw socket to receive all network traffic which is
completely unmodified. Then it creates a bridge between two network interfaces. It then allocates the
desired memory for both the interfaces. Fig 3 shows a typical DAQ initialization process.

Fig: 6


7.2 DAQ Start
During DAQ start, each interface is bind to specific socket. It then Turns on the promiscuous mode for
the Network Interface controller. DAQ module then asks the kernel to create ring buffer for holding the
packet data. When the packet data is obtained through PF_SOCKET, it gets added to ring buffer.
DAQ Module Snort Main
DAQ_Conf ig
DAQMediator
Get DAQ Module
Load DAQ modules
Store DAQ Modules
Module Initialize
Open PF_PACKET socket
Create Bridge
Allocate Packet Memory
Store DAQ Module
Fig: 7


7.3 Data Acquisition
During data acquisition, DAQ module fetches data associated with one of the socket corresponds to a
network interface and send data to the snort application through callback function. Then it passes that
data to other interface using associated sockets. Fig: 5 show the complete data acquisition flow.
Snort Main DAQ Mediator DAQ Module
Get DAQ Module
DAQ Start
Bind Interf ace With Socket
Turn on promisous mode
Setup Ring Buf f er f or interf ace


Fig: 8

8 Decode
In this module the data link layer packets get decoded and the information are stored in respective
structures in snort
Snort Main DAQ Mediator DAQ Module
Get DAQ Module
Inv oke Data Acquire with callback
Poll data with sockets
Callback with acquired data
Pass Data Beteween interf aces
8.1 Decoding process

Fig: 9

The packets arrived from the network card have been passed to the snort decode engine by DAQ to
decode the raw Data Link Layer packets. The DAQ enabled by snort is in an endless loop watching for the
arrival of packets. When it receives the packets from the network card driver, it then runs the
SetPktProcessor() function to set the root decoder based on the datalink layer into the Grinder .Then it
calls ProcessPacket() function.
In the ProcessPacket () function, it calls the Data link layer Decoding function for decoding Data Link
Layer packet frame. During the Data Link Layer decoding, it calls the Network Layer decode function for
finding the network layer protocol type. The network layer decoding function calls the next higher
(transport) layer decode function for finding type of packet. Finally the ProcessPacket () function link all
the information decoded to appropriate data structures. The packets are now ready for detection
9 Preprocess
Preprocessors reassemble packets, do protocol specific decoding and normalization. This enables
detection engine to easily perform attack detection. If packet payload is not normalized, the probability
of getting false positives will be more. So preprocessor improve the accuracy of detection process.
9.1 Dynamic Preprocessors
The dynamic preprocessors are plugins loaded dynamically by snort executable. These preprocessors
are configurable and initialized during snort initialization.
9.2 Initialization of Dynamic Preprocessor
The interface between snort executable and a dynamic preprocessor is a data structure named
_DynamicPreprocessorData or DPD. DPD contains addresses of many functions as function pointers.
The snort executable first fills this structure with its function addresses and pass this structure to
dynamic preprocessors through preprocessors exposed function named InitializePreprocessor.
InitializePreprocessor calls preprocessors setup function using a macro DYNAMIC_PREPROC_SETUP().
Setup function gets address of function named RegisterPreprocessor from DPD and calls this function
with a call back function and a unique string as arguments. This unique string is used to identify the
preprocessor, which is an entry in snort configuration file. Snort executable uses these parameters using
the above mentioned unique string and invokes the callback function.

9.3 Sequence Diagram of Initialization of Dynamic Preprocessor
Fig 3 shows the sequence diagram showing the Initialization of Dynamic Preprocessor.
Fig: 10
Snort Executable Dy namic Preprocessor
Data
Dy namic Preprocessor
Load Dy namic Preprocessor
Fill Dy namic Preprocessor Data
InitializePreprocessor
Store PreProcessor Data
Call Setup Function
Get RegisterPreprocessor
Inv oke RegisterPreprocessor
Conf igurePreprocessor
Call init
Store Init address
Inv oke AddFuncToPrepocList
Store PreprocFunc address


The following example code shows how to register a preprocessor.
_dpd.registerPreproc(ftp_telnet, FTPTelenetInit);
The registerPreproc() function takes two parameters. The first parameter is a unique string to identify
preprocessor. This string would be entered in the snort.conf file when configuring snort to use the
preprocessor. The line added to snort configuration file is shown below.
Preprocessor ftp_telnet: global inspection_type stateful encrypted_traffic no check_encrypted
The second parameter is pointer to an initialization function. The initialization function is used to
process options in the configuration file that affect how preprocessor analyzes packet. The function
receives a pointer to a character string as function argument. Snort passes global inspection_type
stateful encrypted_traffic no check_encrypted as an argument to FTPTelenetInit. In addition to
initialization, register a preprocessor function. The preprocessor function is the function that analyze
packet and normalize traffic. The following is an example code shows adding preprocessor function.
_dpd.addPreproc(FTPTelenetChecks,PRIORITY_APPLICATION, PP_FTPTELNET, PROTO_BIT__TCP);
9.4 Non Dynamic Preprocessors
Rule-based pattern matching can often fail on protocols for which data can be represented in many
different ways. For example, Web servers accept many different ways of writing a URL. IIS, for example,
will accept backslash (\) characters in place of forward slash (/) characters in URLs. In such cases, we can
define a single right way to write the data that we are matching.
Thus Snort has relied on preprocessors that are compiled into Snort. There are three non-
dynamic preprocessors enabled in snort.conf include HTTP Inspect, RPC Decode and BO (Back Orifice).
The http_inspect preprocessor is a generic HTTP decoder that examines HTTP requests and replies.
http_inspect detects previously unknown Web servers or proxies, allowing a better understanding of
what HTTP activity are taking place on the network.
Step to activate this preprocessor, add the following in Snort configuration file:

preprocessor http_inspect: global \
iis_unicode_map unicode.map 1252
These values tell HTTP Inspect where to find the unicode.map file and tells Snort to use the United
States Unicode codemapp (1252) when decoding Unicode characters.
preprocessor http_inspect_server: server default
profile all ports { 80 8080 8180 } oversize_dir_length 500

This set of values establishes a default profile for all Web servers answering requests on ports 80, 8080
and 8180 TCP. The only other option tells Snort to generate alerts when a URL directory length exceeds
500 characters.
The RPC Decode preprocessor normalizes fragmented RPC into a single record. The default values watch
for traffic involving the portmapper (port 111) and 32711, which is only one of many possible RPC server
ports.

preprocessor rpc_decode: 111 32771

The final default preprocessor included in the stock snort.conf is the Back Orifice preprocessor, indicated
by:

preprocessor bo

BO has not been a popular back door communications channel for many years, this preprocessor should
be left out of your new configuration file.
10 Detection
The main components of detection module of snort are detection engine, snort rule set, snort dynamic
rules and PCRE library. Snort loads detection engine, rule set, snort dynamic rules and PCRE library.
Snort rule set is in the form of text files and other three are shared libraries. Using the snort
configuration file we can configure the rule set and dynamic rules. That means, user can specify the
required rules in snort configuration file. During detection, each incoming packet will be processed using
the set of rules. Fig: 8 show the general code flow during initialization of detection modules.

Fig: 11

Detection engine consists of one or more dynamic plugins which process the Ethernet packets against a
set of snort rules. Each plugin will do its own specific task. Snort by itself contain a set of rules by
default, and also snort can have dynamic rules which are loaded dynamically as per the configuration in
snort configuration file. For the efficient use of snort rules, these rules are stored in the form of a tree
structure. The header data of rule is used to build a rule tree node (RTN) and option data of rule is used
to create option tree node (OTN). All of the OTNs with a matching header are grouped together under a
single RTN. The node structure will also contains a list of function pointers that need to be executed for
Snort Main Detection Engine Dy namic Rules
InitializeEngine
Load Detection Engine
Load Dy namic Rules
Register Snort
Load PCRE
Store Dy namic Engine Data
Initialize
Get Register Rule
Register Rule
Create Rule Tree
Create Option Tree
processing the Ethernet packet for the corresponding rule. PCRE (Perl Compatible Regular Expression)
library will be used by the detection engine for regular expression pattern matching of packet data
content.
10.1 Rule Tree

Fig: 12

Rule tree is data structure used to hold all the snort rules. Each rule tree node(RTN) corresponds to a
rule header. Each option tree node(OTN) corresponds to an option header. Each RTN is associated with
different OTNs. RTNs are branched into other RTNs and thus form a rule tree. OTNs contain address of
packet evaluation functions in addition to rule option values. During detection, snort detection engine
walks through rule tree to find appropriate rule set to apply those on incoming packet.










10.2 How Detection works
After normalizing the packet, preprocessor pass packet to detection module to evaluate packet against a
known attack pattern. The detection module groups rules according to destination ports. Each group is
associated with specific attack pattern. When a packet comes, pattern match algorithm is used to find a
match against the packet data and corresponding rule set is selected for packet evaluation. The alerts
found during packet evaluation are added to event queue. These entries in event queue are later used
by output modules to generate alerts.

Fig: 13
11 Output Module

Snort output plug-ins (snort output modules) provides the administrators the ability to configure
logs and alerts in a manner that is easy to understand, read and use in their organization's
environment.
Detection Module Pattern Matcher Rule Tree PCRE Preprocessor
Ev aluate Packet
Get Ruleset f or a packet
Get rules
Create Rule Group with dest port
Match Pattern to f ind ruleset
Get Rule Option and inv oke ev aluation f untion
pcre_exec if option PCRE
Add alert to ev ent queue
The Output plug-ins are initialized, registered and configured at the time of snort initialization itself.
These output modules are executed whenever snort's alert or logging subsystems are called
following the execution of preprocessors and the packet capture engine. The packet capture engine
in snort receives packets off the wire and sends to the analysis module. If the packet or packets
trigger an alert or log event, the data is passed to the corresponding output module. The
parsecmdLine () function finds whether we need to log or alert the packet when parsing snort
config file. Depending upon the snort config run mode, the parsecmdLine () function determines
whether to log packet or print packet. If printpacket (alert) is selected as log_func, the
corresponding data will be printed on console. ie alert. If LogPacket is selected as log_func,
CallLogPlugins() function is called when data is passed to the output module. In this function
corresponding log function is handled depending upon the type. it can be in the form of default
logging, SNMP traps, Xml logging, syslog, snortdb, unified logs, barnyard etc.

12 Snort Maintenance Thread

Snort supports reloading a configuration in lieu of restarting Snort in so as to provide seamless
traffic inspection during a configuration change. A separate thread will parse and create a
swappable configuration object while the main Snort packet processing thread continues inspecting
traffic under the current configuration. When a swappable configuration object is ready for use, the
main Snort packet processing thread will swap in the new configuration to use and will continue
processing under the new configuration.
The SnortStartThreads function calls ReloadConfigThread for creating reload maintenance thread
for snort. The function ReloadConfigThread checks for new snort configurations by calling
reloadConfig() function. This function will set the new Configurations if available and thread will
wait till snort swap or exiting and when it happens it will delete old snort configuration files if
newer configuration is set, by calling Snortconffree() function. If the configurations reload is success
it will load the new preprocessor configurations and free the old ones, so any preprocessor cleanup
that requires a configuration will be using the new one unless it relies on old configurations that are
still attached to the existing sessions. Then it calls the function updatePeriodicCheck().
Maintenance thread Continuous this process till the snort exits.