Software Security Flaws

based on
Buffer Overflow Vulnerabilities
Roland J. Graf
University of Applied Science and Technology - Salzburg
ITS - Information Technology and System Management
rgraf.itsb2003@fh-salzburg.ac.at
Abstract
Buffer and stack overflows are always software bugs. They are the
basis of many security vulnerabilities and todays most common security
threat. Typically, these buffer overflows result in enabling attackers to
inject code into system and to run the harming code. This paper introduces to process memory organization of computers and explains what
buffer overflows are. Each kind of overflow allows a specific method exploiting the software, so the article classifies the type of overruns. Some
lines of code and figures explain how code injection attacks work. It then
discusses some methods to detect and counter security flaws based on
buffer overflow vulnerabilities.
Keywords: buffer overflow, code injection, security flaw

Introduction

Computer and network security companies and organizations (SANS1 , CERT2 ,

etc.) or security news services (Heise Security3 , SecurityFocus4 , Computer
Crime & Intellectual Property Section5 , etc.) regularly publish online information and statistics on computer attacks, security bugs, exploited software and
software vulnerabilities. These reports advise the software industry that most
computer systems and applications are unsecure and not able to withstand the
attacks and threats from the global network (called Wild Wild Web in [1]).
Since years most security flaws are based on buffer overflow vulnerabilities.
Local Flaws and Global Threats In the past computers were working in
well-defined environments. Developers were focused to write modern, which
meant functional, fast and ergonomically designed software. In the early 1990
1 http://www.sans.org
2 http://www.cert.org
3 http://www.heise.de/security/
4 http://www.securityfocus.com/
5 http://www.cybercrime.gov/

many books were published to discuss methods to write efficient and ergonomic
user interfaces and how to write solid and stable applications [2] and maintainable code [3]. But all these introductions into the most popular computer
languages [4],[5] and classic best practices books [2],[3] didnt refer to secure
code, threat modeling and security, and didnt handle topics like How to write
Secure Code. Software had to be stable and solid, maintainable, flexible, high
configurable and easy expandable. Developers didnt care about security vulnerabilities and attacks, because mostly computers were not attached to a network
or isolated on proprietary networks. A local security flaw and software bug was
a local threat only. [1],[6],[7]
Today nearly all computers are connected over the Internet. As part of the
overarching global network, local security vulnerabilities are possible backdoors
to the whole network and connected computer systems. Security flaws and
local software bugs are always global threats for the network and all attached
devices. To write software today, developers have to understand the principles of
software security and, as described in this article, the specific danger of overflow
vulnerabilities.
Software Security Security vulnerabilities which can be abused to compromise a computer system are based on bugs inside software modules. Security
and software development professionals realize now that computer security is
almost about making software behave. Only secure software becomes generally accepted and is marketable today. Software developers have to know about
security principles, threat modeling, security vulnerabilities, secure coding techniques and security flaws in their code [1],[8],[6]. Building solid and secure software systems will become more and more important in the future.
Online services, specialized companies and many publications provide developers with information and coding guidelines how to write secure code and how
to create and administrate secure software systems. Software libraries, new
compiler options and development and testing tools used during the software
development process, should increase the software security with lower costs.
Overflow Vulnerabilities Typically a buffer overflow, usually called buffer
overrun, enables attackers to inject and run harming code into a computer
system. Buffer overruns have been identified as security problems in the 1960s.
One of the first known examples was a finger worm in 1988, written by Robert
T. Morris (Morris Worm 6 ).
Overflows are generally caused by missing or poorly coded input data validation
and error handlers. Overflows take place when an application processes data,
which are larger than the memory space allocated for it (buffer overflows) or
the processor can handle in its registers (integer overflows). These kinds of
bugs enable attackers to write data in a region of memory (buffer), which is
interpreted and used as program code later. So data manipulation and data
injection can also be emerged as code generation and harming code injection.
How to write attacks based on buffer overflows is described in detail elsewhere
since years, i.e. in [9],[10],[11],[7] and the principles in extracts in this article.
6 http://en.wikipedia.org/wiki/Morris_worm

To understand the technical basics of these attacks, the next chapter gives a
short abstract about process memory and processor registers.

Memory and Registers

A short introduction in this article explains the principles of process and memory
organization implemented in most operating systems and the types of processor
registers used to address process memory.

2.1

Process Memory

To understand the principles of attacks, which are based on buffer overflows,

it is necessary to look into the memory organization of a modern computer
system. A binary or an application is generally an executable file which is
saved on a data storage medium. Various file formats are available for such
binary files. Microsoft platforms use the Portable Executable Format (PE),
UNIX based systems mostly use the Executable and Linking Format (ELF). If
an application is called, the executable file will be loaded into the memory and
will be started there. This running process needs a memory for the instructions
and to manipulate data during runtime. The process memory shown in Figure
1 includes a Text and Data Segment, as such as a Heap and a Stack.

Figure 1: Process Memory Layout and Stack Layout

Text Segment - The text segment saves the program instructions which
describe the program flow. To avoid an intentional or unintentional memory manipulation this segment is mostly marked as read-only. Each effort
to write into this segment results in a segmentation violation exception.
Data-Segment - All data of initialized and not-initialized global variables
are stored separately in the data segment above the text segment (see
Figure 1). Uninitialized global variables are saved in the BSS (Block
Started by Symbol), initialized data in a region called Data.

 Heap - Not-initialized dynamic memory allocated during the runtime is

organized in a separated memory block called heap. Dynamic memory is
requested from the operating system by the application with application
programming interface (API) functions (malloc()). As shown in Figure
1 the heap is growing dynamically from down to top and is limited by the
global process memory size and by the stack size.
Stack - The most common scope is a function and all local variables
and buffers declared in this function are only valid inside this function.
Normally these dynamic and transient memory blocks are stored in a separated memory on the top of the process memory, onto Stack. Besides
these locally declared variables and buffers, on Intels x86 Architectures 7
(x86) also function parameters and function return addresses, frame pointers and optional the exception handler frame and callee-save registers are
stored on the stack (for more details see [12]).
The stack is growing from top to down and is limited by the global process
memory and the heap size (see Figure 1). If the stack grows too much,
also parts of the heap will be overwritten and vice versa.

2.2

Processor Registers

A processor has a very small, but mostly very fast amount of internal memory
used for program processing. Some parts of this memory are reserved for internal usage only and not accessible from outside the processor. But some storage
space is used for moving data and data manipulation, to store results, memory
addresses and to store program counters, pointers and indices. These accessible
memory segments are called Processor Registers. For this application use, all
processors have General Purpose Registers (GPR). Intels 32-bit Architecture 8
(IA-32) describes some general data registers, general address registers, floating
point stack registers, some registers for special purpose (multimedia, etc.) and
also registers to control the processor behavior (interrupt control, paging, mode
switches, etc.) [13]. The number and type of registers varies from model to
model. All modern microprocessors used in computer systems have registers,
grouped by segment, control, debug and test registers. And each register manipulation can result in an unintentional behavior and uncontrolled - or attacker
controlled - program flow.

Buffer Overflow Vulnerabilities

Buffer overflows take place when an application processes data from mostly external sources, which are larger than the memory space allocated for it. The
data is written into invalid regions, so memory is overwritten outside the memory allocation bounds. This overflow occurs only when the buffer was prepared
for a static size of data. A received data block could be too large and no length
check is implemented to handle the memory block which is larger than expected.
Overflows also result from incorrect memory calculations or the use of memory,
after the memory block has been unallocated. Overflowing memory can seem
7 http://en.wikipedia.org/wiki/X86
8 http://en.wikipedia.org/wiki/IA-32

like a harmless bug, but an overflow is always a consequence of a software bug.

Not all possible buffer overflow errors are definitely exploitable, but they might
be a gateway for the next attack. Michael Howard and David LeBlanc wrote in
Writing Secure Code [1, p.133]:
You can prove only that something is exploitable, so any given
buffer overrun either is exploitable or might be exploitable. In other
words, if you cant prove that its exploitable, always assume that
an overrun is exploitable.
So code which allows a buffer overflow is always to handle as exploitable error.
Dont fix only bugs that you think are exploitable. Just fix the bugs! [1, p.134]
Because buffer overflow attacks are based on memory and register manipulations, technical literature and publications classifies these kinds of attacks as
Memory and Register Exploits.

3.1

Memory and Register Exploits

Manipulated processor registers, stack, function and memory pointers, and the
global stack and heap management are the main points of attacks to assume
control over operating systems or applications. An attacker tries to manipulate
stack and heap variables and function pointers placed in different memory segments. If the length and type of data is not validated, as a consequence a too
large memory block could be placed onto the memory, a buffer overflows and a
part of the memory will be overwritten by the data. Each buffer overflow error
will change the intended program flow or program behavior, if function pointers
are affected.
1

int

g_i;

char g_szMsg[] = "global message text";

3
4

void foo(char *str)

static int l = 100;

static int n;

char buffer[20];

strcpy( buffer, str);

...

10
11

12
13

int main(int argc, char *argv[])

14

15

foo(argv[1]);

16

...
return 0;

17
18

Listing 1: Example application with local and global variables

As shown in the example Listing 1 above, the unsafe function strcpy copies
all characters from str into the local variable buffer. If str is longer than 19+1
bytes, strcpy overwrites the adjacent memory on the stack too. In best case
the application crashes and quits with an error. In worst case - by an attack
- the application continues with changed data, frame pointers or overwritten
function return addresses, which results in an unintentional or perhaps attacker
manipulated program flow.
To write an exploit, the attacker has to find out information about the
application and the system. Following requirements are necessary to exploit an
application with a buffer overflow:
Knowledge about buffer overflow vulnerabilities (unchecked input data,
static allocated buffers, etc.)
Knowledge about the system memory structure (heap, stack, segments,
etc.)
Knowledge about the function call mechanism and coherences (application
call stack, function parameters, function return addresses, saved frame
pointers, application pointer variables, etc.)

3.2

Overflow Error Classification

Different memory types, memory addressing methods, memory segmentation

methods and processor registers provide different methods to attack an application and for the developer to protect the application against unintentional
memory manipulation. All buffer overflow attacks are based on lacks of bounds
checking in memory handling routines. The kind of overflows depends on the
type of memory, the type of used registers or the method to provoke the program error. Regarding [9], the most common overflow methods and errors used
to compromise a system are:
Stack Overflow: occurs, if a buffer declared on the stack overflows or is
overwritten by data larger than the allocated buffer
Frame Pointer Overwrite: occurs, if a buffer declared on the stack
overflows or is overwritten by data larger than the allocated buffer
Array Indexing Errors: occurs, if a buffer overflows by one byte only
Off-by-One Overflow or an array index overflows or underflows Integer
Overflow/Underflow ; a typical error if a loop iterates one times too many
or few. See the following short example:
char buffer[20];
for(int i=0; i<=20; i++ )
buffer[i] = ...;
BSS Overflow: occurs, if static declared local buffers or uninitialized
global buffers in the BSS segment are overwritten by data larger than the
allocated buffers.

 Heap Overflow: occurs, if a dynamical allocated buffer on the heap

overflows or is overwritten by data larger than the allocated buffer
Format String Error: is not exactly a buffer overflow; occurs, if the
format string and the string format function arguments dont match.
Different overflow errors promote different methods of attacks. Some of the
overflows are easier to use to manipulate a program flow or runtime data, other
overflows are more complicated and are not trivial to abuse. The following
pages classify now some basic types of overflows in detail and their exploiting
methods.

3.3

Stack-based Overflow Vulnerabilities

The classical form of overflow exploitation is an attack to the buffers allocated

on the stack. As described in the short overview about the process memory (see
Figure 1), different data (local variables, function parameters, function return
addresses, etc.) are stored on the stack. Static buffer overflows are stack-based
buffer overflows; attacks are called also Stack Smashing Attacks. In this context
static means a local buffer inside a function with a static size (i.e. an array of
bytes), which would be placed on the stack. The attacker chooses a string larger
than the allocated buffer.
As shown in the example above, if the unchecked user input will be copied
with an unsecure function like strcpy to the statically allocated buffer, some
parts of the stack will be overwritten by the copied string. The string is chosen
by the attacker, so the overwritten return address is injected by the attackers
input string too. Because a string is only a sequence of bytes, each address
and code can be injected. The attacker tries firstly to inject harming code and
secondly to change a function return address to run this injected code. [1],[9]
To inject a code, the attacker has to find out:
the location of the function return address on the stack
the offset of the buffer to the return address
Disassemblers and debuggers help the attacker to get these information [7, p.71145]. Sometimes primitive approximations and repeating the desired return
address several times in the assumed region of the return address yields to
success [10].

3.4

Heap-based Overflow Vulnerabilities

A heap overflow is much the same programming fault as stack-based buffer

overflows. It is much trickier to exploit, but it is also possible to inject code
over a heap overflow, which is presented in publications too, i.e. in [14], [1], [6]
and [9].
The heap is a memory region inside the process memory (local buffer ), which is
available for dynamic memory allocations during the runtime. There are some
reasons to survey heap overflows in detail [14]:
No compiler extensions and tools like StackGuard do currently exist to
detect heap overflows.
7

 Some processor architecture and modern operating systems offer a nonexecutable stack; this doesnt protect against heap-based attacks
Because of common misconception of many developers, who mean that
heap overflows are not exploitable, they change their code from local buffer
to static buffer usage.
During the runtime, library functions are available to allocate and deallocate
memory blocks with arbitrary size in arbitrary order. The memory pool is
managed dynamically by the operating system or a memory manager (Heap
Management). The memory manager divides the memory in free and allocated
blocks, which are organized in complex data structures. After the application
startup, only one huge free memory block is available. But after some memory
allocations and memory return calls, the heap is fragmented in allocated and
free memory blocks with different size.

Figure 2: Heap and stack manipulation in three steps

The memory allocation is dynamically implemented in the application. But
if the memory block size requested by the program is handled statically, a buffer
overflow is possible. Memory of other blocks could be overwritten, if a heap
buffer overflows. If an application allocates a memory block with the function
malloc(), the application will retrieve a pointer to the memory with the requested size. A second call of malloc() retrieves a second pointer. All these
memory blocks are inside the process heap and have apparently a randomized
address. An attacker now could try to find out the memory block order inside
the heap. If he is able to manipulate adjacent memory blocks, he could take
control over the application or could manipulate the application data.
Figure 2 shows a combination of heap and stack manipulation in three steps.
The application allocates two memory blocks used for reading input data and
saves the pointers also on the heap. After calling the allocation functions, the
8

memory blocks are ordered as shown in Figure 2 on the heap. The attacker
redirects the pointer to Buffer2 stored in BufferX. The application passes the
unchecked user input to Buffer1, this causes a buffer overflow and the pointer to
Buffer2 gets overwritten by an address chosen by the attacker (see Figure 2-1).
The pointer he injected is a vector to a function return address, stored onto
the stack. Now (see Figure 2-2) the attacker puts in the address of the injected
code in Buffer1. Because the application writes this data to Buffer2 now and the
pointer to this buffer was redirected, the application writes the attackers chosen
address over the function return address stored on the stack. If the function
goes out of scope, the program flow will follow the function return address (see
Figure 2-3). Equivalent examples with complete source code are shown in [1]
and [9]. Because the stack could be manipulated without any stack smashing,
mechanism that might guard the stack over canaries (see section StackGuard in
this document) wont notice that the stack has been manipulated.

3.5

Format String Vulnerabilities

A format string vulnerability is not a real buffer overflow, but it acts almost
like a buffer overflow. This is the reason why format buffer vulnerabilities are
always handled like overflow errors. Format bugs occur because C and C++
support functions which take a variable number of arguments. C/C++ compilers perform limited type checking on functions that can take a variable number
of arguments, so all these functions are type unsafe. Typically functions, which
use the Cs varargs mechanism are all input and output functions, which use
format strings to describe the arguments pushed onto the stack (i.e. the printf
and scanf families of functions are examples of functions that use variable argument lists). Format string functions work with placeholders for arguments,
which should be stored onto the stack. The function assumes that the corresponding arguments with the specified type are available on the stack. If the
attacker now is able to manipulate the format string, then he can use the format
directives to write data into the process memory or to get data from the stack.
[15]
An example shows the problem. The function printf(pStr) with one argument pStr is used in the application. If the attacker is able to choose the
string stored in pStr, he is able to spy on the stack too. He could fill the string
with the format mask "%x%x%x%x%x%x%x%x%x%x%x%x". This string assumes that
there are 12 integers on the stack to put out. Because the function printf has
no further arguments, it would wrongly get other integers from the stack. This
is a fast and easy method to spy on the stack and to get a stack dump. A
code change to printf("%s", pStr) prevents this source of danger. Another
dangerous format mask is "%n". It writes an integer to an address, specified by
a parameter - so a value on the stack - at runtime. [9],[15]

Buffer Manipulation Prevention

Reasons for the occurrence of buffer overflows are mostly unchecked user input,
failed or missing bounds checks and indices. The easiest method of defense buffer
overflows is to write solid code, which means preventing buffer overruns through
writing bounds and pointer checking and a robust code. [1],[2] Michael Howard

and David LeBlanc recommend in [1] that programmers should . . . always validate all inputs - the world outside your function should be treated as hostile
and bent upon your destruction. [1, p.155]
Security analysts and developers have to identify potential code vulnerabilities
and software bugs. The performance of manual code inspection is the most
secure but not most efficient method to determine security vulnerabilities. So
software developers are also supported by compiler options, compiler extensions,
save functions and save libraries to write more secure code. Some of the most
common features and tools are introduced here.
4.0.1

Safe Types

Some of the reasons for buffer overflows are based on the programming language
C and C++, their types, pointers and the C common runtime library functions.
C/C++ handles no array bound checks, pointer checks, overflow checks and type
checks, because C and C++ are optimized for space and speed efficiency. C++
already increased the type safety. Some compilers support warning levels and
advise the developer with warning or error messages during compile time (static
tests) [16]. The compilers identify type conflicts, uninitialized variables and
possible errors and security vulnerabilities. Libraries like the Standard Template
Library9 (STL) for C++ implements string classes to handle array of characters,
pointer helper classes and templates for implement type safe collections. These
libraries encapsulate the pointer and memory handling and prefer references
instead of pointers.
4.0.2

Safe Functions

Some common runtime library (CRT) functions are marked as unsafe. Further use of these functions is not recommended. String handling is the single
largest source of buffer overruns, because the deprecated string functions cannot check the destination buffer length. For instance a call of the function
strcpy(szBuf, pInput) generates a stack overflow vulnerability, because the
function strcpy doesnt handle string length limitations. For security enhancements many functions in the CRT now have secure versions, which should be
used instead. Developers consider manually updating the code to use a secure version. The secure and recommended version to copy a string would be
strcpy_s(szBuf, _countof(szBuf), pInput).
Some of the security enhancements in the CRT to prevent buffer overflows include the following points:
Sized Buffers
Null termination
Parameter Validation
Format string syntax checking
A list of all deprecated CTR functions and their recommended replacements for
Microsoft operating systems is available under [17].
9 http://www.sgi.com/tech/stl/

10

4.1

Compiler Extensions

Modern C/C++ compilers try to check the source code during the compiler
time. But not all vulnerabilities can be detected during the development cycle.
So some compilers automatically generate code for checks during the runtime
(dynamical checks).
4.1.1

Bounds Checking

Bounds Checking is the name given to any method of detecting whether or not
an index (or pointer) given, lies within the limits of an array.
C and C++ never perform bounds checking per default, because these languages
are optimized for efficiency. Other newer languages (Java, C#, Visual Basic,
etc.) enforce runtime bounds checking. Meanwhile also some compiler patches
and extensions are available to add this feature to C and C++. One of the
available compiler extensions are BoundsChecking 10 (the newer version is called
CRED - C Range Error Detector ).
4.1.2

StackGuard

A compiler extension like StackGuard 11 patches the free Gnu-C-Compiler (GCC)

to detect and defeat stack smashing attacks. StackGuard generates a function
prolog and epilog routine, which manipulates the stack and places a canary word
prior the return address on the stack (see Figure 3.b). Before jumping to the return address, StackGuards function_epilog() checks if the canary words have
been modified. This check protects the return address on the stack from being
altered, because a buffer overflow manipulates always the memory addresses
around the return address and so the canary word too. If the canary word was
manipulated, an exception will be thrown by the function_epilog(). A big
disadvantage of StackGuard is, that the canary word is between the function
return address and the saved frame pointer. Because of this order the saved
Frame Pointer is not protected by StackGuard. [9],[18]
4.1.3

Microsoft /GS Option

Microsofts compiler option /GS, available in all C++ .NET compilers since
2002, use the same principles for buffer overrun runtime checks and provides
a security cookie (also called speed bump) between the buffer and the saved
frame pointer (see Figure 3.c) [19] The Visual C++ .NET /GS option sets
up a canary between any variable declared on the stack and the EBP pointer,
return address pointer, and the function-specific exception handler. According
to the MSDN documentation in [20], the C++ compiler makes also a copy of
the vulnerable incoming parameters after storage for local variables, because
there they are not in danger of being overwritten. [1]
10 http://sourceforge.net/projects/boundschecking/
11 http://en.wikipedia.org/wiki/StackGuard

11

Figure 3: Stack guards: (a) Unmodified stack frame; (b) StackGuard modification with a Canary Word; (c) Microsoft compiler option /GS modification with
a Security Cookie
4.1.4

StackShield

StackShield 12 is also a compiler patch for the GCC compiler, but uses other
technologies to protect function pointers. The Global-Ret-Stack Method holds
a copy of function return addresses at the beginning of the DATA segment,
which is not overflowable. The function_epilog() compares the saved pointer
with the function return pointer on the stack. A second method, called RetRange-Check Method, checks, if the Function Return Address points into the
Text Segment. A redirected jump outside the Text Segment or a manipulated
address, which is not equal to the stored address in the DATA segment, would
result in an exception. [9]

4.2

Libraries

Compiler Extensions like StackShield and StackGuard assume that the source
codes of the applications are available, because the projects have to be recompiled with the patched compiler. When the sources are not available, the
exchange of the standard libraries could increase security and reduce the risk of
overflows.
4.2.1

LibSafe

The library LibSafe 13 substitutes unsafe library functions which will be linked
dynamically to the applications. It uses the preload feature of dynamically
loadable ELF libraries to automatically and transparently load with processes
it needs to protect. [21, p.7] These new functions, which are hooked between
the application and the standard library, prevent stack overflows and stack manipulations or their negative effects. Libsafe saves a copy of the Function Return
Addresses stored on the stack in a separated memory. If a manipulation in the
12 http://www.angelfire.com/sk/stackshield/
13 http://directory.fsf.org/libsafe.html

12

stack is detected, Libsafe writes a syslog message and terminates the compromised application.

4.2.2

Libformat

The library Libformat uses the same principle as described for Libsafe to eliminate format string vulnerabilities. Libformat parses the user specified format
strings in writable segments and prevents data injections with dangerous type
conversions (i.e. %n, etc.).
A patch of Gnu C Library (GLibc) to wrap format string functions is also available under the name FormatGuard 14 . Because a static library will be patched,
the applications have to be recompiled to active the FormatGuard protection.
[9]

4.3

Other Methods

There are some other methods to prevent buffer overflows, format string flaws
and code injection, which are not discussed here in detail. Some of the concepts
are not focused on security vulnerabilities, but they help to create stable and
faultless software. These methods should ensure to write secure code, or in other
cases, they should limit the effects of security vulnerabilities, if an attacker compromises an application. The following enumeration gives a very short abstract
of further security relevant technologies, but discussed no more in this article:
High Level Programming Languages - Languages, which dont offer
pointers and low level data manipulation functions and with a high data
abstraction level; script languages, which are typically interpreted and
controlled by an interpreter during the runtime; programming languages
with bounds checking during compile and runtime, generated by the compiler (i.e. Pascal, Modula-2, Oberon, Java, C#, etc.) or implemented
in a Language Runtime Infrastructure (i.e. .NETs Common Language
Infrastructure (ISO/IEC 23271:200615 or Standard ECMA-33516 ) )
Virtual Environments and Sandboxes - Encapsulate and shield the
whole operating system (i.e. VMWare17 , etc.), user defined applications
and their data (i.e. Altiris Software Virtualization Solution18 , etc.), applications written in special languages (i.e. Java and their Java Virtual
Machine (JVM), etc.)
Non-executable Memory - Memory segments, memory pages or parts
of memory are marked as non-executable. This method can stop existing buffer overflow exploits and should make exploiting a vulnerability
harder (i.e. non-executable stack in OpenWall19 , non-executable pages
with PaX20 , ExecShield [22] etc.) [23, p.88]
14 http://www.securityfocus.com/tools/1844
15 http://www.iso.org/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=42927
16 http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-335.pdf
17 www.vmware.com
18 http://www.altiris.com
19 http://www.openwall.com
20 http://pax.grsecurity.net

13

 Process Environment Protection and Security Systems - System

to controls access to system resource and objects. Novells openSuSE authors describe their control system called AppArmor, it [. . . ] proactively
protects the operating system and applications from external or internal
threats. . . enforcing good behavior and preventing even unknown application flaws from being exploited.[24] (i.e. Mandatory Access Control
(MAC) implemented in SELinux21 , AppArmor22 , TrustedBSD23 , etc.)
Hardware based protection methods - Modern processors mark segments and part of memory as non-executable too. (i.e. in modern x86Architectures (32/64 bit) AMD supports No-Execute-Bits (NX) and Intel
called the same technology Execute-Disable-Bit (XD), both prevent code
execution in marked memory pages). [25],[13] In the near future all effective solutions and operating systems should work consequently with
hardware supported Address Space Layout Randomization (ASLR).

Conclusion

Today attacks are a constant threat for software systems. As a result of the constantly increasing danger, computer attacks, security bugs, exploited software
and software vulnerabilities are the issues for software developing today. Typically, buffer overflows are the result of design flaws, software bugs and unchecked
memory processing. With some lines of code and some knowledge about memory management principles, attackers are able to inject and run harming code or
to obtain secret data. Backdoors, unchecked pointers and memory manipulation
make it easy to board a computer system.
This article describes technical details about memory management and buffer
overflows and discusses then some methods of preventing buffer overflow vulnerabilities. Secure coding techniques, systematic code analyzing and compiler
switches are only few of the methods of how to increase code security. Safe
libraries, advanced software testing methods and checking coding standards
automatically are still or should be common in modern software development
processes. The described compiler extensions and libraries are assistive technologies to detect and prevent stack and buffer manipulations only. These tools
make overruns more difficult to exploit, but studies have shown that the effectiveness of some code injection detection methods is very limited. It is more
complicated to inject code, but it is not generally impossible. Meanwhile many
publications describe how to bypass StackGuard [26] or other protection technologies described in this document.
Also techniques undiscussed in this article are available to design robust and
secure computer systems. Binary audits, secure coding reviews, fault injections
and the principle of least-privileges are only some effective methods to improve
software security, but most of them are scarcely used by developers.[1],[9] Writing secure software is a lot of work and complex, therefore secure software is
expensive. Perhaps these are some of the reasons why software is still insecure today. Software developers need techniques which make software robust,
21 http://www.nsa.gov/selinux
22 http://de.opensuse.org/Apparmor
23 http://www.trustedbsd.org

14

solid and secure with the least effort. With an ordinary infrastructure today it
seems, that the only safe code injection avoidance assumes manual implemented,
strict input and range checks during the runtime. Tools, libraries and compiler
extensions are unfortunately add-ons only. In the future the combination of
hardware (processors, memory management units, etc.), secure environments
and operating systems (isolated processes, virtualizations, etc.), safe functions
and programming languages (manages code, etc.) give the software industry
the tools and chance to create secure software systems. A lot of technologies
are already available today. Developers should use these methods to protect
computers, otherwise attackers will abuse them for their purpose.

References
[1] Howard, M. and D.C. LeBlanc: Writing Secure Code, Second Edition. Microsoft Press, Redmond, Washington, USA, 2nd edition, 2002,
ISBN 0735617228.
[2] Maguire, S.: Writing Solid Code: Microsofts Techniques for Developing
Bug-Free C Programs (Microsoft Programming Series). Microsoft Press,
Redmond, Washington, USA, 1993, ISBN 1556155514.
[3] McConnell, S.: Code Complete - Deutsche Ausgabe der Second Edition. Microsoft Press Deutschland, Unterschleiheim, Germany, 2nd edition, 2005,
ISBN 386063593X.
[4] Kernighan, B.W. and D.M. Ritchie: Programmieren in C. ANSI C (2.
A.). Mit dem C-Reference Manual. Hanser Fachbuch, 2nd edition, 1990,
ISBN 3446154973.
[5] Stroustrup, B.: Die C ++ Programmiersprache. 2.
uberarbeitete Auflage. Addison Wesley Verlag, Bonn M
unchen Paris, 2nd edition, 1992,
ISBN 3893193863.
[6] Howard, M. and D. LeBlanc: Sichere Software programmieren. Microsoft
Press Deutschland, Unterschleiheim, Germany, 2002, ISBN 386063674X.
[7] Hoglund, G. and G. McGraw: Exploiting Software: How to Break Code
(Addison-Wesley Software Security Series). Addison-Wesley Professional,
Boston, USA, 2004, ISBN 0201786958.
[8] Swiderski, F. and W. Snyder: Threat Modeling (Microsoft Professional).
Microsoft Press, Redmond, Washington, USA, 2004, ISBN 0735619913.
[9] Klein, T.: Buffer Overflows und Format-String-Schwachstellen. Funktionsweisen, Exploits und Gegenmanahmen. dpunkt Verlag, Heidelberg, Germany, 2003, ISBN 3898641929.
[10] Aleph One: Smashing the stack for fun and profit. Phrack, 7(49), November
1996. http://www.phrack.org/phrack/49/P49-14.
[11] Zatko (alias Mudge), P.C.: How to write Buffer Overflows, 1995. http:
//insecure.org/stf/mudge_buffer_overflow_tutorial.html.

15

[12] Bray, B.: Compiler Security Checks In Depth - Anantomy of

the x86 Stack.
Visual Studio Technical Article, 2002.
http:
//msdn2.microsoft.com/en-us/library/aa290051(VS.71).aspx#
vctchcompilersecuritychecksindepthanchor3.
R 64 and IA-32 Architectures Software Developers
[13] Intel Corporation: Intel
Manuals, 2006-2007.
http://www.intel.com/products/processor/
manuals/index.htm.

[14] Conover, M. and w00w00 Security Team: w00w00 on heap overflows, 1999.
http://www.w00w00.org/files/articles/heaptut.txt.
[15] Lhee, Kyung Suk and Steve J. Chapin: Buffer overflow and format
string overflow vulnerabilities. Softw. Pract. Exper., 33(5):423460, 2003,
ISSN 0038-0644.
[16] Sutter, Herb and Andrei Alexandrescu: C++ Coding Standards:
101 Rules, Guidelines, and Best Practices (C++ in Depth Series). Addison-Wesley Professional, Boston, San Francisco, New York,
2004, ISBN 0321113586. http://www.awprofessional.com/articles/
article.asp?p=373337&seqNum=3&rl=1.
[17] Microsoft Corporation: MSDN Library - Run-Time Library Reference Deprecated CRT Functions, 2005. http://msdn2.microsoft.com/en-us/
library/ms235384(VS.80).aspx.
[18] Cowan, Crispan, Calton Pu, Dave Maier, Jonathan Walpole, Peat Bakke,
Steve Beattie, Aaron Grier, Perry Wagle, Qian Zhang, and Heather Hinton: StackGuard: Automatic adaptive detection and prevention of bufferoverflow attacks. In Proc. 7th USENIX Security Conference, pages 63
78, San Antonio, Texas, Jan 1998. http://citeseer.ist.psu.edu/
cowan98stackguard.html.
[19] Bray, B.: Compiler Security Checks In Depth - Run-Time Checks
& What /GS Does.
Visual Studio Technical Article, 2002.
http://msdn2.microsoft.com/en-us/library/aa290051(VS.71)
.aspx#vctchcompilersecuritychecksindepthanchor4.
[20] Microsoft: .NET Framework and Language Features - C++ Runtime Security Checks.
Visual Studio Professional Guided Tour,
2006.
http://msdn.microsoft.com/vstudio/tour/vs2005_guided_
tour/VS2005pro/Framework/CPlusRuntimeSecurity.htm.
[21] Baratloo, A., N. Singh, and T. Tsai: Libsafe: Protecting critical elements of stacks, dec 1999. \url{pubs.research.avayalabs.com/pdfs/
ALR-2001-019-whpaper.pdf}.
[22] van de Ven, A.: Limiting buffer overflows with execshield. Red Hat
Magazine, 9, July 2005. http://www.redhat.com/magazine/009jul05/
features/execshield.
[23] Younan, Y.: An overview of common programming security vulnerabilities
and possible solutions. Masters thesis, Vrije Universiteit Brussel, August
2003. citeseer.ist.psu.edu/younan03overview.html.
16

[24] openSuSE: The AppArmor Project, may 2007. http://en.opensuse.org/

AppArmor.
[25] Krahmer, S.: Abschirmdienst - Speicherschutztechniken von Linux. ct Magazin f
ur Computertechnik, 16, 2006.
[26] Richarte, G.: Four different tricks to bypass StackShield and StackGuard
protection, 2002.
http://downloads.securityfocus.com/library/
StackGuard.pdf.

17