2009 First International Workshop on Education Technology and Computer Science

A Multi-link Aggregate IPSec Model

Yun-he Zhang, Zhi-tang Li, Mei-zhen Wang, Ling Xiao

School of Computer Science and Technology
Huazhong University of Science and Technology
Wuhan, China
email: {yhzhang | leeying | mzwang | lingx}@mail.hust.edu.cn
AbstractInternet has become the universal information
communications infrastructure. VPN is commonly used to
implement communications over different branch intranets.
IPSec is a suit of protocols that adds security to
communications at the IP layer, and it is a popular technology
to implement VPN. On the basis of analysis on the insufficiency
of traditional IPSec systems, a multi-link aggregate IPSec
model is proposed. The new model can negotiate multiple
groups of security policies on different physical links for same
branch intranet pair, and distribute IPSec traffics over
multiple links. A prototype system of the new model which is
based on Netfilter mechanism is implemented on Linux
platform. Analysis on the test result from the prototype system
shows that the new model can work better under the
environment of multi-link, and can enhance the capability and
reliability of VPN application.
Figure 1. IPSec Architecture

Keywords multi-link; aggregate; IPSec

I.

There are three main functionalities of IPSec separated

into three protocols: Authentication Header (AH),
Encapsulating Security Payload (ESP) and Internet Key
Exchange (IKE). AH allows authentication of each IP
packets selected header fields or depending on the
selected mode of the entire IP packet. ESP allows
encryption and optionally authentication of the entire
IP packet or of the IP payload, depending on the selected
mode, namely the transport and the tunnel modes. Finally,
key management is automated through the Internet key
exchange (IKE) protocol.[1,2,3,4,5]

INTRODUCTION

Nowadays Internet has already become the universal

communications infrastructure in business. With distanceindependent rates and flat fees, the costs of corporate Internet
communications become predictable and cheaper. However,
the original design of TCP/IP, the core protocol family of
Internet, is based on the suppose of trusty communication
and not considered security threat. First, TCP/IP is lack of
effective authentication mechanism. The second problem is
the lack of secrecy instruments. Furthermore, the data
transferred over network may be modified by attackers due
to unavailability of integrety protection. Such security holes
make it more important for business to protect data from
alternation when they are transfered on Internet.
One solution is virtual private network (VPN). A VPN is
a private network on a public network infrastructure
(Internet). VPN establishes logical channel to transfer data
reliably by tunneling mechanisms and cryptographic method.
For end users, VPN tunnel is transparent. This solution takes
advantage of the cheap and ubiquitous Internet, and provides
security guarantees, and is widely used now.
IPSec, a suit of protocols which provides network-layer
security, is a common way to implement VPN now. IPSec
adds additional headers/trailers to an IP packet and can
encapsulate (tunnel) IP packets in new ones. Fig. 1 shows the
IPSec architecture.

978-0-7695-3557-9/09 $25.00 2009 IEEE

DOI 10.1109/ETCS.2009.639

II.

TRADITIONAL IPSEC MODEL

Corporation which has embranchments will dispose

IPSec gateways on their intranet boundary to create VPN for
intranet data access between each other. Normally, an IPSec
VPN gateway has only one internet connection. Traditional
IPSec model can work well under these environments.
The traditional IPSec model is shown in Fig. 2. The
primary modules of an IPSec VPN are IKE and IPSec. IKE
that working on application layer does the negotiation of key
and security policies between two devices. IPSec works on
kernel layer normally, and does encapsulation, encryption
and authentication to IP packets.

493
489

Figure 3. Maintain tunnels on each link

Figure 2. Traditional IPSec model

The initiator sends the first Phase 1 message to the

responder to initiate Phase 1 negotiation. Phase 1 negotiation
that normally in Main Mode will generate ISAKMP SA to
protect Phase 2 negotiation messages. After the Phase 1
negotiation finished, the initiator will start Phase 2
negotiation to establish IPSec SA which is used to protect
communication traffics between hosts in protected branch
intranets. IPSec SA will be written to the kernel SPD/SAD.[6]
IPSec module acquires plain IP packets from network
stack and search SPD according to the source and destination
IP address of packets. If an outbound SP group could be
found, IPSec will handle the plain IP packets according to
the found policies, such as encrypt, authenticate and
encapsulate. After the work done, IPSec send the handled
packets to network. If the packet acquired is an IPSec
cryptograph (AH or ESP packets), IPSec will search an
inbound SP in SPD according to the SPI value in AH or ESP
header.
III.

So we present our new IPSec model, multi-link aggregate

IPSec model. Firstly, the IKE module in traditional model is
replaced by M-IKE which can do negotiation on different
links for the same branch intranet pairs. After M-IKE
negotiation, a number of SA groups for the same branch
intranet pair will be written into IPSec. Because of these SA
groups are belong to one branch intranet pair and have some
same information, IPSec should organize them in some
framework in order to search exactly and conveniently while
processing packets.
When IPSec receive IP packets to process, IPSec will
search a corresponding SP in SPD according to the packets
header fields. While under the multi-link environment there
are multiple groups of SP match with packet, we must
choose one of them to apply to the packet. SA election does
this work. The whole model is described in Fig. 4.

MULTI-LINK AGGREGATE IPSEC MODEL AND

IMPLEMENTATION

A. Model
With the development of Internet applications, corporate
existing network bandwidth may not be able to meet the
business, or for fault-tolerant backup considerations, some
corporations install a number of Internet links to enhance the
capability and reliability of their network. However, using
traditional IPSec VPN system, only one of these physical
links can be used to do IPSec communication. To take full
advantage of multiple links, the better scheme is maintaining
IPSec tunnels on each physical link for the same branch
intranet pair, as Fig. 3 shown.

Figure 4. Multi-link aggregate IPSec model

According to the design of our multi-link aggregate

IPSec model, we implement a prototype system for
verification under Linux. Our prototype is based on opensource IPSec project OpenSwan which is popular used.

490
494

IPSec checks the IP packet, if the protocol of it IPSec

protocol like AH and ESP which means the packet is
incoming, IPSec will do inbound processing, otherwise,
IPSec will do outbound processing with the packet.
The outbound processing is as follow:
(1) Search SPD according to source and destination IP
address of the packet;
(2) If no SP found, pass through the packet;
(3) If a SP matched, check that if it is in a multi-link SP
chain or not;
(4) If the SP is a single-link SP, skip to next step;
(5) If the SP is in a multi-link SP chain, run SP election
algorithm to elect a SP from the chain to use;
(6) Pick up the SA chain from SAD according to the SPI
in the chosen SP and process the packet according to each
SA in the SA chain.
The inbound processing is the same with common IPSec
implementations, and as follow:
(1) Look up SA in SAD according to outer header's
destination IP address, IPSec protocol and SPI value in the
packet header;
(2) If no SA found, the packet MUST be dropped;
(3) If a SA is found, deal with the packet according to it;
(4) If the next header of the packet is still an IPSec packet,
go to (1) and do the processing again.
After IPSec outbound/inbound processing, IPSec module
gives the packets back to network protocol stack and return
to the pre-routing point. Linux kernel then continually
processes the modified packets according to the original
process. System route table will be looked up using the
destination IP address in packets, and then packets will be
inserted into a corresponding NIF sending queue. At last, the
packets will be sent by NIF. For outbound packets, choosing
different SA chain brings different encapsulation with
different destination gateway on different link. So our
prototype can utilize all outgoing links to carry out VPN
tunnel communication.

B. Multi-link IKE negotiation

M-IKE implements the following three important
functionalities:
Multi-link negotiation M-IKE creates and binds UDP
sockets on each interface corresponding physical links. For
each interface, we configure a set of security policies of
same branch intranet pair. While it negotiating on one
physical link, M-IKE sends and receives messages using
corresponding socket. M-IKE records own and peer
addresses of the negotiation socket into tunnel information,
and uses them to identify different tunnels on different links.
Because of the additional parameters, M-IKE can support
multiple tunnel instances for the same branch intranet pair at
the same time.
Link weight assignment For each link, M-IKE assigns
a weight value to it, and also saves this weight value to
corresponding tunnel information. The weight is the measure
of link capability. The link which has higher capability is
assigned greater weight value. Weight value will be written
into IPSec with the SA groups after tunnel negotiating
successfully, and IPSec will use it in SA election process. In
our prototype, weight value is manual configured by
administrator, and it will be learned automatically in future
research.
Tunnel probing While some links break down,
packets sent to it will be dropped. M-IKE must execute
tunnel probing to detect the link status. In our prototype,
DPD is used for detection, and if probing failed, M-IKE will
delete corresponding tunnel from IPSec so that IPSec can
distribute traffic to other healthy links. If the link recover its
health, M-IKE also can detect it by particular plain probing
and re-negotiate new SA on it so that IPSec can distribute
traffics to it once more.
C. IPSec processing
OpenSwan uses virtual network adapter to implement
IPSec. In that way, each packet that handled by IPSec will be
returned to IP protocol stack several times to re-routing, and
system works in low efficiently. Our prototype use Netfilter
Hook Mechanism of Linux kernel to acquire IP packets from
system, and after dealing with packet, IPSec will give the
packets back to system protocol stack to forward.[7]
We register our IPSec module on the pre-routing hook of
IP layer as Fig. 5 shown. Kernel picks up a packet from
receiving queue, and does some basic packet check, then call
the callback functions on the pre-routing hook which
contains out IPSec entrance function to process packet.

D. SP management
In our prototype, SP is in the form of eroute which is
similar to general route. The data structure for eroute is as
follow:
struct eroute {
struct rjtentry er_rjt;
struct sa_id er_said;
struct sockaddr_encap er_eaddr;
struct sockaddr_encap er_emask;
......
struct estat estat;
struct eroute *er_ma_next;
struct eroute *er_ma_prev;
uinit_32 er_weight;
struct in_addr er_src_gateway;
struct in_addr er_dst_gateway;
};
There are several important members in struct eroute:
er_rjt, er_said, er_ma_next, er_ma_prev and er_weight. The
er_rjt is used in Radix Tree fast routing algorithm which is
immigrated from BSD routing table implementation. Radix

Figure 5. IPSec hook at pre-routing of netfilter

491
495

Tree that is a Binary Search Tree based on binary key values

can provide fast and accurate subnet / netmask matching.
Our prototype uses it to implement SP matching by source
and destination IP address of packets. The er_said of eroute
is to identify an SA which is used to processing packets
matching the eroute. The er_said consists of three members:
destination host, SPI value and IPSec protocol (AH / ESP).
The data structure is as follow:
struct sa_id {
struct in_addr dst; /* A.dest. host */
uint_32 spi;
/* B. 32-bit SPI */
int proto;
/* C. protocol */
};
The other three members of er_ma_next, er_ma_prev and
er_weight are used to implement multi-link aggregation. As
mentioned above, M-IKE negotiates a number of SA on
different physical link for the same branch intranet pair and
writes them into IPSec. IPSec inserts SA into SAD and
creates corresponding SP in SPD. The SP will be inserted to
the Radix Tree by using er_rjt, and those of them which for
the same branch intranet pair will be insert an isolated loop
chain by using er_ma_next and er_ma_prev. Each branch
intranet pair has such a chain.
The insert algorithm is as follow:
(1) Search Radix Tree using new SPs source and
destination subnet information;
(2) If not found any SP, skip to next step;
(3) If a SP had the same subnet information, insert the
new SP to the loop chain of the found SP;
(4) Insert the new SP into Radix Tree.
Fig. 6 gives the sketch map of SP loop chain in Radix
Tree.

(4) If the SP found not in a chain, choose it directly;

(5) If the SP found in a chain, traverse the chain, and
n

calculate the sum of weight,

Wsum = Wi ;
i =1

(6) Pick out a number between 0 and (-1) by generating a

random number R as the choice factor, CF = R mod Wsum ;
(7) For the jth SP in the chain, if

j 1

i =1

i =1

Wi < CF Wi ,

choose it as the elected SP.

Using the above-mentioned algorithm of election, we can
distribute user traffic between the same branch intranet pair
over different IPSec tunnels on different physical links.
IV.

TESTING AND ANALYSIS

To validate the efficiency of out model, we have carried

out experiments. The testing environment consists of 2 MAIPSec VPN gateways (IBM PowerPC 405EP in 266MHz,
64MB Memory, 100BaseT Fast Ethernet Adaptor), a number
of 10Mbps ethernet hubs that used as isolated physical links
and a network tester NuStream-600.The topology map is
shown in Fig. 7.

Figure 7. Testing environment

Firstly, we test the throughput on multi-link environment.

Testing parameters are: 1400 bytes frame size, only use ESP,
and the encryption algorithm is AES-128. The result is listed
in TABLE I.
TABLE I.
Total BW
Throughput
Total BW
Figure 6. SP loop chain

Throughput

The er_weight value is passed by M-IKE, saved in SP,

and used in SP election. IPSec searches SPD according to
packet information, and runs SP election to choose one SP
from SP chain of the same branch intranet pair if necessary.
The algorithm is as follow:
(1) Search Radix Tree using source and destination IP
address of the packet;
(2) If not found any SPs, ignore this packet;
(3) If one SP found, check if it is in a SP chain;

THROUGHPUT TEST RESULT

10

20

30

40

50

9.002

17.938

26.78

35.811

42.925

60

70

80

90

100

50.5

50.5

50.5

50.5

50.5

According to the throughput testing result, we get a line

graph as Fig. 8.

492
496

the link fault by probing and delete corresponding tunnel SA.

As a result, user traffics are migrated to the other right
tunnels and the user is unaware of the occurrence of the fault.
V.

CONCLUSIONS

We designed a multi-link aggregate IPSec model which

can make use of multi-links of corporation. According to the
model, we implemented a prototype system on Linux
platform. We also carried out experiments to validate the
efficiency of the model. To improve our model, some
expansive works can be done in the future, such as
improving the measure method of weight, and making it to
adapt to the link performance automatically; improving
IPSec distribute algorithm to cooperate with connectionoriented communications.
ACKNOWLEDGMENT
Figure 8. Throughput line graph

This work was partially supported by the National

Natural Science Foundation of China No.60573120.

Through observing and analyzing the line graph, we can

find that the throughput increases with the total physical link
bandwidths increase. The throughput is not equal to the total
bandwidth due to the waste of IPSec encapsulation in IP
packets and the cost for SP election. And the cost for SP
election increases slightly with the increase of link number
(which means the increase of SP number). However, we note
that when link bandwidth up to some value, the throughput
does not increase any more. The throughput is limited by the
processing capacity of the two gateways. On the whole, our
multi-link aggregate model can make better use of the multilink bandwidth compared with general IPSec systems only
used in one link.
We also observe the traffic distribution. According to the
statistics of packets number processed by every tunnels, we
draw the conclusion that the processing rate of the ith tunnel
approximate to

REFERENCES
[1]
[2]
[3]
[4]
[5]

[6]

Wi
. This result is identical with our
Wsum

[7]

design.
Finally, we test fault tolerance of our model. We shut
down one hub when testing throughput. Then M-IKE detects

493
497

S. Kent and R. Atkinson, Security Architecture For the Internet

Protocol RFC2401, IETF RFC, 1998.
S. Kent and R. Atkinson, IP Authentication Header RFC2402,
IETF RFC, 1998.
S. Kent and R. Atkinson, IP Encapsulating Security Payload (ESP)
RFC2406, IETF RFC, 1998.
D. Harkins and D. Carrell, The Internet Key Exchange (IKE)
RFC2409, IETF RFC, 1998.
Alberto Ferrante, Vincenzo Piuri, Fabien Castanier, A QoS-enabled
Packet Scheduling Algorithm for IPSec Multi-Accelerator Based
Systems, CF05, May 46, 2005, Ischia, Italy.
Naganand Doraswamy and Dan Harkins, IPSec: The new security
standard for the internet, internet, and virtual private networks[M],
Prentice Hall PTR, 1999.
Zhao Da-Yuan, Jiang Yi-Xin and Lin Chuang. Implementation and
performance evaluation of IPSec VPN based on netfilter[J]. Wuhan
Univ J Nat Sci, 2005, 10(1): 98-102.