OPERATING SYSTEMS

SECURITY

Jerry Breecher

15: Security

SECURITY
In This Chapter:

TheSecurityProblem
ProgramThreats
SystemandNetworkThreats
CryptographyasaSecurityTool
UserAuthentication
ImplementingSecurityDefenses
FirewallingtoProtectSystemsandNetworks
Computer-SecurityClassifications
AnExample:WindowsXP

15: Security

SECURITY
SECURITY ISSUES:

Externalprotectionofasystem.Aclassifiedsitegoestoextraordinarylengthstokeepthings
physicallytight.Amongtheissuestobeconsidered:
Unauthorized accessMechanismassuringonlyauthorizedindividualsseeclassified
materials.
Malicious
modificationordestruction
Accidental
introductionofinconsistency.
Authentication
Howdoweknowtheuseriswhoshesayssheis.Canhave
passwordsondomains.

Protection of passwordsisdifficult.Issuesinclude:

It'sveryeasytoguesspasswordssincepeopleusesimpleandeasilyremembered
words.

Needexiststochangepasswordscontinually.

Limitingnumberoftriesbeforelockingup.
15: Security

SECURITY

Security Issues

Trojan Horse:

A piece of code that misuses its environment. The program seems

innocent enough, however when executed, unexpected behavior
occurs.
Trap Doors:
Inserting a method of breaching security in a system. For instance,
somesecretsetofinputstoaprogrammightprovidespecialprivileges.
Threat monitoring: Lookforunusualactivity.Onceaccessisgained,howdoyouidentify
someoneactinginanunusualfashion?
Audit Log:
Record time, user, and type of access on all objects. Trace problems
backtosource.
Worms
Usespawningmechanism;standaloneprograms.
Internet Worm:
IntheInternetworm,RobertMorseexploitedUNIXnetworkingfeatures
(remote access) as well as bugs in finger and sendmail programs.
Grapplinghookprogramuploadedmainwormprogram.
Viruses
Fragment of code embedded in a legitimate program. Mainly effects
personal PC systems. These are often downloaded via e-mail or as
activecomponentsinwebpages.
Firewall
A mechanism that allows only certain traffic between trusted and untrusted systems. Often applied to a way to keep unwanted internet
trafficawayfromasystem.
15: Security

SECURITY

Typical Security Attacks

ATTACK METHODS:

Attacksonadistributedsysteminclude:

Passivewiretapping.(unauthorizedinterception/readingofmessages)

Activewiretapping:

Modification
Changingaportionofthemessage.

Spuriousmessages Introducingbogusmessageswithvalidaddressesand
consistencycriteria.

Siteimpersonation Claimingtobesomeotherlogicalnode.

Replay
ofprevioustransmission-repeatingpreviousvalidmessages.
(forexample,authorizationofcashwithdrawal.)

15: Security

SECURITY

Typical Security Attacks

ATTACK METHODS:

15: Security

SECURITY

Typical Security Attacks

ATTACK METHODS:

TrojanHorse
Codesegmentthatmisusesitsenvironment
Exploitsmechanismsforallowingprogramswrittenbyuserstobe
executedbyotherusers
Spyware, pop-up browser windows, covert channels
TrapDoor
Specificuseridentifierorpasswordthatcircumventsnormalsecurity
procedures
Couldbeincludedinacompiler
LogicBomb
Programthatinitiatesasecurityincidentundercertaincircumstances
StackandBufferOverflow
Exploitsabuginaprogram(overfloweitherthestackormemorybuffers)
15: Security

SECURITY

Typical Security Attacks

Example of Buffer Overflow Waiting To Happen:

#include <stdio.h>
#define BUFFER SIZE 256
int main(int argc, char *argv[])
{
char buffer[BUFFER SIZE];
int other_data;
if (argc < 2)
return -1;
else {
strcpy(buffer,argv[1]);
return 0;
}
}
15: Security

SECURITY

Typical Security Attacks

Viruses
Codefragmentembeddedinlegitimateprogram
VeryspecifictoCPUarchitecture,operatingsystem,applications
Usuallyborneviaemailorasamacro
VisualBasicMacrotoreformatharddrive
Sub AutoOpen()
Dim oFS
Set oFS =
CreateObject(Scripting.FileSystemObject)
vs = Shell(c:command.com /k format c:,vbHide)
End Sub

15: Security

SECURITY

Typical Security Attacks

A Boot Sector Virus

15: Security

10

SECURITY

Typical Security Attacks

System And Network Threats

Wormsusespawnmechanism;standaloneprogram
Internetworm
ExploitedUNIXnetworkingfeatures(remoteaccess)andbugsinfingerand
sendmailprograms.(Seenextslide)
Grappling hookprogramuploadedmainwormprogram
Portscanning
AutomatedattempttoconnecttoarangeofportsononeorarangeofIP
addresses
DenialofService
Overloadthetargetedcomputerpreventingitfromdoinganyusefulwork
Distributeddenial-of-service(DDOS)comefrommultiplesitesatonce

15: Security

11

SECURITY

Stuxnet

StuxnetisacomputerwormdiscoveredinJune2010.ItinitiallyspreadsviaMicrosoftWindows,
andtargetsSiemensindustrialsoftwareandequipment.
DifferentvariantsofStuxnettargetedfiveIranianorganizations,withtheprobabletargetwidely
suspectedtobetheuraniumenrichmentinfrastructureinIran.
ItisinitiallyspreadusinginfectedremovabledrivessuchasUSBflashdrives,andthenuses
otherexploitsandtechniquestoinfectandupdateothercomputersinsideprivatenetworksthat
arenotdirectlyconnectedtotheInternet.
Themalwarehasbothuser-modeandkernel-moderootkitcapabilityunderWindows,andits
devicedrivershavebeendigitallysignedwiththeprivatekeysoftwocertificatesthatwerestolen
fromtwoseparatecompanies.Thedriversigninghelpeditinstallkernelmoderootkitdrivers
successfullyandthereforeremainundetectedforarelativelylongperiodoftime.
OnceinstalledonWindowsStuxnetinfectsfilesbelongingto
Siemens'controlsoftware[3andsubvertsacommunicationlibrary.
Doingsointerceptscommunicationsbetweensoftwarerunningunder
WindowsandthetargetSiemensdevices.Themalwarecaninstall
itselfonPLCdevicesunnoticed.
Stuxnetmalwareperiodicallymodifiesacontrolfrequencytoandthus
SiemensSimaticS7-300
affectstheoperationoftheconnectedcentrifugemotorsbychanging
PLCCPUwiththreeI/O
theirrotationalspeed.
15: Security
12
modulesattached

Authentication

SECURITY

Passwordstealing
Easiestwayisthroughsocialmeans
fakedepositslips
easilyguessablepasswords
callingpeopleonthephoneandaskingforpasswords(orCreditCardnumbers,forthat
matter)
Technologicalapproachesalso
simpleone:leaveprogramrunningonaterminalthatfakesthelogin
sequence.Captureusernameandpasswordtoafileandthenexit
withafakeerrormessage,returningcontroltotherealloginprocess
Unixpasswordfilesusedtobeopenlyavailable(encryptedpassword).Lendsitselftobruteforcecracking.Unfortunatelysomeprogramsrequireaccesstothepasswordfiletorun(e.g.,
mail)alsounfortunatelyUnixonlyusesfirsteightcharactersofpassword

SecurIDusesapreprogrammedstringofcharacters
15: Security

13

Authentication

SECURITY

Passwordstealing
Easiestwayisthroughsocialmeans
fakedepositslips
easilyguessablepasswords
callingpeopleonthephoneandaskingforpasswords(orCreditCardnumbers,forthat
matter)
Technologicalapproachesalso
simpleone:leaveprogramrunningonaterminalthatfakesthelogin
sequence.Captureusernameandpasswordtoafileandthenexit
withafakeerrormessage,returningcontroltotherealloginprocess
Unixpasswordfilesusedtobeopenlyavailable(encryptedpassword).Lendsitselftobruteforcecracking.Unfortunatelysomeprogramsrequireaccesstothepasswordfiletorun(e.g.,
mail)alsounfortunatelyUnixonlyusesfirsteightcharactersofpassword

SecurIDusesapreprogrammedstringofcharacters
15: Security

14

NSA Exploitation

SECURITY

EdwardSnowdenmadepublicdocumentsthatrevealGovernmentagencies:
consideritessentialtobeabletoviewencrypteddata
haveadoptedabatteryofmethodsintheirassaultonthisbiggestthreats
Thosemethodsinclude
controloversettingofinternationalencryptionstandards,
theuseofsupercomputerstobreakencryptionwith"bruteforce",

Collaborationwithtechnologycompaniesandinternetserviceprovidersthemselves
Maninthemiddleattacksonthecommunicationchannelsthemselves.

15: Security

15

Cryptography

SECURITY
DEFINITIONS:

Encryption:

E( M, Ke )

E=EncypheringAlgorithm
M=Message-plaintext
Ke= Encryptionkey
C= Cypheredtext
Decryption:

= D( C, Kd )

D=DecypheringAlgorithm
Kd=Decryptionkey

15: Security

16

Cryptography

SECURITY

DEFINITIONS:

Cryptosystems are either Conventional or Public Key

Conventionalissymmetric;Ke=Kd,sothekeymustbekeptsecret.Algorithms
aresimpletodescribe,butcomplexinthenumberofoperations.
Publickeyisasymmetric;Ke!=Kd,soKecanbemadepublic.Kdissecretand
can'teasilybederivedfromKe.

Securityagainstattackiseither:
Unconditionally secure - Ke can't be determined regardless of available
computationalpower.
Computationally secure:-calculationofKdiseconomicallyunfeasible(itwould
overwhelmallavailablecomputingfacilities.)

Theonlyknownunconditionallysecuresystemincommonuse!
Involvesarandomkeythathasthesamelengthastheplaintexttobeencrypted.
The key is used once and then discarded. The key is exclusively OR'd with the
messagetoproducethecypher.
Giventhekeyandthecypher,thereceiverusesthesamemethodtoreproducethe
message.
15: Security

17

SECURITY

Data Encryption Standard

DATA ENCRYPTION STANDARD ( DES ):

TheofficialNationalInstituteofStandardsandTechnology(NIST),(formerly
theNationalBureauofStandards)encryptionforusebyFederalagencies.

The source of security is the non-linear many-to-one function applied to a

blockofdata.Thisfunctionusestranspositionandsubstitution.Thealgorithm
ispublic,butthekey(56bits)issecret.

Computationalpowertodaycancracka56bitcode.

In common use today is Triple DES in which 3 different keys are used,
makingtheeffectivekeylength168bits.

15: Security

18

SECURITY

Public Key Cryptosystems

Thegeneralprincipleisthis:
1.AnyRECEIVER AusesanalgorithmtocalculateanencryptionkeyKEaand
adecryptionkeyKDa.
2. Then the receiver PUBLICIZES KEa to anyone who cares to hear. But the
receiverkeepssecretthedecryptionkeyKDa.
3. User B sends a message to A by first encrypting that message using the
publicizedkeyforthatreceiverA,KEa.
4.SinceonlyAknowshowtodecryptthemessage,it'ssecure.

KEa
KEb
Public Key Repository
KEc

15: Security

19

SECURITY

Public Key Cryptosystems

Tobeeffective,asystemmustsatisfythefollowingrules:

a) Givenplaintextandciphertext,theproblemofdeterminingthekeysis
computationallycomplex.
b) ItiseasytogeneratematchedpairsofkeysKe,Kdthatsatisfytheproperty
D( E( M, Ke ), Kd ) = M.

Thisimpliessomesortoftrapdoor,suchthatKeandKdcanbecalculated
fromfirstprinciples,butonecan'tbederivedfromtheother.

c) TheencryptionanddecryptionfunctionsEandDareefficientandeasytouse.
d) GivenKe,theproblemofdeterminingKdiscomputationallycomplex.
Whatiscomputationallydifficult?Problemsthatcan'teasilybecalculatedinafinitetime.
Examplesinclude:factoringtheproductoftwoverylargeprimenumbers;theknapsack
problem.
TheseproblemsareNPcomplete-solutiontimesareexponentialinthesizeofthe
sample.
15: Security
20

SECURITY

Public Key Cryptosystems

Tobeeffective,asystemmustsatisfythefollowingrules:

e) Foralmostallmessagesitmustbecomputationallyunfeasibletofindciphertextkey
pairsthatwillproducethemessage.
(Inotherwords,anattackerisforcedtodiscoverthetrue(M,Ke)pairthatwas
usedtocreatetheciphertextC.)

f) Decryptionistheinverseofencryption.

E( D( M, Kd ), Ke ) = D( E( M, Ke ), Kd )

15: Security

21

SECURITY

Public Key Cryptosystems

AN EXAMPLE:

1. Twolargeprimenumberspandqareselectedusing
someefficienttestforprimality.Thesenumbersare
secret:

Let p = 3, q = 11
n = 3 * 11 = 33.

2. Theproductn=p*qiscomputed.
3. ThenumberKd>max(p,q)ispickedatrandom
fromthesetofintegersthatarerelativelyprimeto
L(n) = ( p - 1 ) ( q - 1 ) = 20.
andlessthanL(n)=(p-1)(q-1).
Choose Kd > 11 and prime to 20.
Choose Kd = 13.
4. TheintegerKe,0<Ke<L(n)iscomputedfrom
L(n)andKdsuchthatKe*Kd=1(modL(n)).
0 < Ke < 20
Ke = 17.
(since 17 * 13 = 221 = 1 ( mod 20 ) )
15: Security

22

SECURITY

Public Key Cryptosystems

AN EXAMPLE:
Separate the text to be encoded into chunks with values 0 - ( n - 1 ).
In our example, we'll use < space = 0, A = 1, B = 2, C = 3, D = 4, E = 5 >.
Then " B A D <sp> B E E " --> "21 04 00 25 05"
21 ^ 17
04 ^ 17
00 ^ 17
25 ^ 17
05 ^ 17

( mod 33 )
( mod 33 )
( mod 33 )
( mod 33 )
( mod 33 )

=
=
=
=
=

21.
16.
00.
31.
14.

21 ^ 13
16 ^ 13
00 ^ 13
31 ^ 13
14 ^ 13

( mod 33 )
( mod 33 )
( mod 33 )
( mod 33 )
( mod 33 )

=
=
=
=
=

21.
04.
00.
25.
05.

This whole operation works because, though n and Ke are known, p and q are not
public. Thus Kd is hard to guess.
[Note: recently a 100 digit number was successfully factored into two prime numbers.]

15: Security

23

SECURITY

Public Key Cryptosystems

AUTHENTICATION AND DIGITAL SIGNATURES:

Sender Authentication:
Inapublickeysystem,howdoesthereceiverknowwhosentamessage(sincethereceiver's
encryptionkeyispublic)?

SupposeAsendsmessageMtoB:

a) ADECRYPTSMusingA'sKd(A ).
b) Aattachesitsidentificationtothemessage.
c) AENCRYPTStheentiremessageusingB'sencryption,Ke(B)
C = E ( ( A, D( M, Kd(A) ) ), Ke(B) )
d)
e)

BdecryptsusingitsprivatekeyKd(A)toproducethepairA, D( M, Kd(A) ).
SincetheproclaimedsenderisA,BknowstousethepublicencryptionkeyKe(A).

Capture/Replay
Inthiscase,athirdpartycouldcapture/replayamessage.
Thesolutionistousearapidlychangingvaluesuchastimeorasequencenumberaspartof
themessage.

15: Security

24

SECURITY
Man-in-the-middleAttack
onAsymmetric
Cryptography
Herearetheattackstepsforthisscenario:
1.Senderwishestosendamessageto
Receiver.
2.SasksRforitsencryptionkey.
3.WhenRreturnskey,thatkeyis
interceptedbytheattackerwhosubstitutes
herkey.
4.Senderencryptsmessageusingthis
boguskeyandreturnsit.
5.Sincetheattackeristheownerofthis
boguskey,theattackercanreadthe
message.

Public Key Cryptosystems

Sender

Receiver

15: Security

25

Example - SSL

SECURITY

InsertionofcryptographyatonelayeroftheISOnetworkmodel(thetransportlayer)
SSLSecureSocketLayer(alsocalledTLS)
Cryptographicprotocolthatlimitstwocomputerstoonlyexchangemessageswith
eachother
Verycomplicated,withmanyvariations
Usedbetweenwebserversandbrowsersforsecurecommunication(creditcard
numbers)
Theserverisverifiedwithacertificate assuringclientistalkingtocorrectserver
Asymmetriccryptographyusedtoestablishasecuresession key(symmetric
encryption)forbulkofcommunicationduringsession
Communicationbetweeneachcomputerusessymmetrickeycryptography
15: Security

26

SECURITY

Example Windows 7

Securityisbasedonuseraccounts
EachuserhasuniquesecurityID
LogintoIDcreatessecurity access token
IncludessecurityIDforuser,forusersgroups,andspecial
privileges
Everyprocessgetscopyoftoken
Systemcheckstokentodetermineifaccessallowedordenied
Usesasubjectmodeltoensureaccesssecurity.Asubjecttracksand
managespermissionsforeachprogramthatauserruns
EachobjectinWindowsXPhasasecurityattributedefinedbyasecurity
descriptor
Forexample,afilehasasecuritydescriptorthatindicatesthe
accesspermissionsforallusers

15: Security

27

SECURITY

Security Classifications

U.S.DepartmentofDefenseoutlinesfourdivisionsofcomputer
security:A,B,C,andD.
DMinimalsecurity.
CProvidesdiscretionaryprotectionthroughauditing.
DividedintoC1andC2.C1identifiescooperatinguserswith
thesamelevelofprotection.C2allowsuser-levelaccess
control.
BAllthepropertiesofC,howevereachobjectmayhave
uniquesensitivitylabels.DividedintoB1,B2,andB3.
AUsesformaldesignandverificationtechniquestoensure
security.

15: Security

28

SECURITY
Wrap Up

In this chapter weve looked at how to secure information that may be placed in
hazardous public forums.
Data on the net is an excellent example here.

15: Security

29