INSPECTING HTTPS TRAFFIC ON GATEWAYS

October 2011

Author: Kishin Fatnani

This article was originally written for and published in Exploiting Software 02/11 - Hakin9 magazine (http://hakin9.org/)

What to expect in this article

Basics on Secure Sockets Layer (SSL) based communication How security devices inspect Hypertext Transfer Protocol Secure (HTTPS) traffic Pros and cons of enabling HTTPS inspection on Check Point R75.20

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

INTRODUCTION
In the past, security devices inspecting application content for attack patterns, misuse or malware, had been blind to encrypted traffic and due to this encrypted protocols like Hypertext Transfer Protocol Secure (HTTPS) have been a safe method used by attackers to bypass security inspection. Though reverse proxies and Web Server modules have been there for long, but they only inspect incoming traffic i.e. connections made to protected web servers in the organization. Inspecting outgoing traffic or traffic of connections made by users to outside world servers, not protected by the device, had been on the wish lists. Devices these days come with the capability to inspect Secure Sockets Layer (SSL) based outgoing traffic, however there are some concerns enabling such kind of inspection. In this article we cover some basics of SSL, the challenges in inspecting SSL traffic, and also see how Check Point's HTTPS Inspection feature is able to inspect HTTPS traffic at the gateway. After reading this article you will know the pros and cons of enabling SSL inspection on a gateway.

SECURE SOCKETS LAYER (SSL) BASICS

Protocols like HTTP, SMTP, POP3 etc are plaintext i.e. data carried by these protocols can be read by anyone who is able to intercept the traffic while in transit. Since these protocols are widely used over the Internet, there was a need to secure them in order to protect users' data. By protection of data we are looking at confidentiality, integrity and authentication. Instead of building security measures in all the application protocols, the Secure Sockets Layer (SSL) protocol provides a secure channel below the application layer thereby making it easier to make an application protocol a secure protocol.

Figure 1: SSL providing security below application and above transport layers

SSL is prominently used to protect the HTTP protocol making it HTTPS. Today it is also used to tunnel an entire network stack, creating a VPN. The SSL specification was developed by Netscape communications. After its version 3.0, a new Internet standard protocol was defined and called Transport Layer Security (TLS). The first version of TLS, version 1.0, was kind of an upgrade to SSL 3.0 with a few enhancements. Most web browsers and servers today support TLS 1.0, though versions 1.1 and 1.2 were also defined later, many browsers and applications are yet to support them. Throughout this article, we may commonly refer to both, SSL and TLS, as SSL.

HOW SSL/TLS WORK

SSL provides: Confidentiality - the data passing through SSL is encrypted using a symmetric encryption algorithm like DES, 3DES, RC4, AES etc. Integrity - a hashing algorithm like MD5, SHA1 etc. is used which transforms data of any length into a small (128.160 or 256 bits depending on the algorithm used) fixed size data called the hash. The receiving end would also create hash of the same data and match to verify the integrity Authentication - achieved by RSA signatures or DSS.
Blog: kishinf.blogspot.com Facebook: www.facebook.com/ksecure Twitter: www.twitter.com/ksecure

Page | 2

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

Key Exchange - to use symmetric encryption, the two parties, the sender and the receiver, need to share a common secret key. Exchanging this secret key over an insecure network is a challenge as someone acquiring the key can decipher the entire communication. For this reason, an asymmetric algorithm is used for the key exchange which could be RSA or DH. A set of algorithms, one for each function (Confidentiality, Integrity, Authentication and Key exchange), is known as a cipher suite. For e.g.
TLS_RSA_WITH_RC4_128_MD5 TLS_RSA_WITH_RC4_128_SHA TLS_DH_RSA_WITH_DES_CBC_SHA TLS_DHE_DSS_WITH_DES_CBC_SHA

Given above are some of the cipher suites used in TLS. For instance the cipher suite named TLS_DH_RSA_WITH_DES_CBC_SHA specifies that TLS will be used with DH for key exchange, RSA for authentication, DES_CBC for encryption and SHA for hashing.

SESSION ESTABLISHMENT
When an SSL session is initiated, the client sends its SSL version along with a list of supported cipher suites to the server. The server chooses its preferred suite and responds back to the client. If the server does not support any of the suites proposed by the client, the session fails to establish. If authentication is required, the server presents its X.509 certificate to the client and it may also request the client to send its certificate. This is not mandatory in SSL though it is very common that a server usually sends its certificate while not requesting client's certificate for authenticating the client.

WHAT IS CHECKED IN THE CERTIFICATE?

Name - When the server sends its certificate to the client, the client checks the domain name in the certificate Validity - its date of creation and expiry are checked to see if today's date falls in between Authority - the most important is the authority that signed the certificate must be a trusted one There may be a chain of certificates including the server's certificate, then that of the authority who issued the server's certificate followed by the issuer of the authority's certificate and so on till a root authority is reached which has a self signed certificate. The client will check certificates up till it finds a trusted authority's certificate.

WHICH AUTHORITIES DO WE TRUST?

Usually clients like web browsers come with a list of trusted authorities which can be seen in the options dialog in the browsers. Users may also add new authorities by adding their certificates.

CERTIFICATE STATUS
An authority may revoke a certificate before its expiry date for some reasons. A revoke certificate must not be accepted by a client, hence a client needs to check the current status of the certificate by querying the Certificate Authority (CA). The CA may provide a list of revoked certificates called the CRL or it may provide an online service, Online Certificate Status Protocol (OCSP), to check the status of the certificate. The URLs for querying are provided in the certificate itself.

Blog: kishinf.blogspot.com

Facebook: www.facebook.com/ksecure

Twitter: www.twitter.com/ksecure

Page | 3

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

VALIDATION
When a certificate authority issues a web server certificate, it first validates that the domain name actually belongs to the requester, this is called 'Domain Validation'. There is another kind of validation called 'Extended Validation' (EV), where the CA will validate other information like company, physical address etc before issuing a certificate. Many web browsers will display the company name in green color in the address field when it receives an extended validation certificate.

Figure 2: Browsers display company name in green for Extended Validation certificates

FAILURES
When any of the checks in a certificate fails, the client may fail the session or like web browsers, give a choice to the user to stop or continue without authentication. In case of client authentication, the server will do a similar checking of the client's certificate, if that is required.

THE SECRETS
If RSA is chosen for key exchange, the master secret is communicated by encrypting with the public key of the server. In case of DH, the secret is generated by both parties using their own private key and other's public key. In either case, the master secret cannot be derived without the knowledge of the private key. Using the secret and other data, finally a session key is created that is used to encrypt the data in the entire session.

APPLICATION DATA
Once the session is established, the application data is encrypted and sent in the subsequent packets. This encryption is done with the symmetric encryption algorithm in the selected cipher suite. The session key that is known to the client and server is used for this encryption.

INSPECTION GATEWAYS
Security gateways inspect user data for various purposes like virus detection, data loss prevention, intrusion detection, policy enforcement etc. If the data is in encrypted form, these devices become useless unless they can decipher the information and inspect the original data. We know that deciphering this data is possible only if we have the session key and that can be obtained only if we possess the private key. One of the ways to enable inspection of SSL data is to provide a copy of the server's private key to the inspecting device or application. This can be done only if the server is owned by the organization and hence only incoming connections can be inspected. Incoming connections can also be inspected by way of reverse proxy, where connections made to the server are actually terminated at the proxy which will have the original server certificate and private key. The outside

Blog: kishinf.blogspot.com

Facebook: www.facebook.com/ksecure

Twitter: www.twitter.com/ksecure

Page | 4

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

world will only be communicating with the proxy server which in turn forwards the requests to the real server. This way the encrypted communication is between the client and the proxy server.

OUTGOING CONNECTIONS
To inspect outgoing connections that is connections made by internal users to the outside world servers, can be done by breaking the connection or proxying, but that is not so simple when using certificates for authentication. To understand the problem, let us take the following scenario where a Web Browser (WB) is trying to establish a secure connection to the Web Server (WS) and is passed through a transparent Proxy Device (PD) as in the figure below:

Figure 3: Sample communication

As the browser makes a connection, the web server sends its certificate to the browser. On receiving the certificate from the server, the browser validates it and if it trusts the certificate, uses the public key from the certificate to encrypt the pre-master secret and send it to the server. The server will use its private key to decrypt this secret. In case a device like PD is proxying this connection, it will not be able to see this secret as it does not have the private key of the web server. The web server is an external server and does not belong to the organization or the user connecting to it, hence it may not be possible to acquire this key for the purpose of monitoring the connection. Without the encryption keys which are derived using these secrets, it will not be possible to monitor the connection. Now let us see what work around most security devices use to monitor SSL connections. When web server sends its certificate which is signed by a trusted authority, PD does all the checks on the certificate and if it passes the checks, PD creates a new certificate which is identical to the server's certificate but with a different public key and this will be signed by the PD instead of the actual issuer of the server's certificate. This new certificate is passed on to the browser for validation and key exchange. The browser must accept this certificate for the connection to succeed. As the public key used in this certificate is different that the one in the server's certificate, there will be one encrypted session between the browser and PD and another between PD and the server.

Blog: kishinf.blogspot.com

Facebook: www.facebook.com/ksecure

Twitter: www.twitter.com/ksecure

Page | 5

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

Figure 4: PD sends a new certificate to the browser

OUR CONCERNS
1) Will the client (WB in our case) accept a certificate that is signed by the proxy device (PD)? 2) Will the gateway trust all the authorities that the client is trusting? 3) Will the same cipher suite be used as the client and server would be using if connected directly? 4) What will the gateway do with client certificates, if requested by the server? 5) Does the gateway check the current status of the certificate? 6) Does it support Extended Validation certificates? 7) Can the administrator use the gateway's private key to decrypt the communication?

CHECK POINT'S HTTPS INSPECTION

To look at the above concerns, we used the HTTPS Inspection feature of Check Point which is available in its release R75.20. Please note that the product has been used just to demonstrate the concepts and not to display the features or limitations of the product. Also please note that the limitations given below are to make the readers aware as these are found in many other products providing SSL / HTTPS Inspection. Our setup is similar to Figure 3 & 4 where Check Point Security Gateway is put in place of PD. An HTTPS Inspection policy was set to inspect all traffic from the browser to the web server, see figure 5 below.

Figure 5: HTTPS Inspection policy

CERTIFICATE EXCHANGE
When the browser attempts to establish an SSL connection with the server on the other side of the Check Point gateway, the request reaches the gateway and matches the inspection rule. The gateway does not
Blog: kishinf.blogspot.com Facebook: www.facebook.com/ksecure Twitter: www.twitter.com/ksecure

Page | 6

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

forward this connection request to the server, but instead makes a new connection with the server. The cipher suites are negotiated in this new connection and the server sends its certificate to the gateway. The gateway validates all information as a web browser would have done it, however the list of trusted CAs in the user's browser may not be entirely same as the list on the gateway. This list can however be updated by the Check Point administrator. If the certificate is valid, the gateway creates a new certificate with the same information i.e. domain name, validity etc. but with new keys. This certificate is then signed by the gateway itself using the provided CA pvt. key and then the certificate is sent down to the browser. In case of an untrusted certificate, the gateway can be configured to either drop the connection or continue by creating a self-signed certificate with the server's new keys as generated by the gateway. Note, this is not signed by the CA key used on the PD. By doing this, the decision to accept or reject the connection is transferred to the user as the browser will surely not be trusting this self signed certificate.

LOOKING AT OUR CONCERNS

Concern No. 1: Will the client (WB in our case) accept a certificate that is signed by the proxy device (PD)? The user's browser may not trust this certificate by default. There are two ways to resolve this: 1) Add the gateway's certificate in the browser's trusted list 2) Configure the gateway to use an already trusted CA certificate for signing new server certificates. The private key needs to be given for signing.

Concern No. 2: Will the gateway trust all the authorities that the client is trusting? The gateway has a pre-defined list of trusted CAs which can be modified by the administrator. If a CA present in the browser is unavailable in the gateway's list, the administrator can add the same to the gateway. When a server with a certificate from an untrusted CA is accessed, the gateway still creates a new certificate, however, instead of signing it with the gateway's key, it creates a self signed certificate. This for sure will not be trusted by the browser, so the user will get an error and will be given a choice to accept it and continue or to discontinue.

Figure 6a: Browser displaying certificate signed by PD

Figure 6b: Browser displaying self-signed certificate created by PD

Concern No. 3: Will the same cipher suite be used as the client and server would be using if connected directly?
Blog: kishinf.blogspot.com Facebook: www.facebook.com/ksecure Twitter: www.twitter.com/ksecure

Page | 7

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

May not be. The point here is that the gateway can only propose the ciphers that it supports. In our lab we observed that the browser proposed 36 cipher suites, whereas the gateway on behalf of the browser actually proposed just 6 cipher suites to the actual server, which were:

Figure 7a: Proposal by browser (WB)

Figure 7b: Proposal for the same request by the gateway (PD)

As you would see in this list there is no support for DH key exchange by the gateway, so if a web server is only supporting DH, you will not be able to make a connection even though your browser supports it. This is rare as most servers will support RSA but you may be having an intranet server which is set to support DH only as per your security policy. Well the solution to this would be to bypass the inspection for this server using the HTTPS Inspection policy.

Concern No. 4: What will the gateway do with client certificates, if requested by the server? Connecting to a server that required client certificate for authentication did not work, hence the server had to be put in the bypass list.

Concern No. 5: Does the gateway check the current status of the certificate? The gateway does the CRL check however it seems at this moment the OCSP feature is not supported.

Concern No. 6: Does it support Extended Validation certificates? If you are connecting to a site having an EV certificate, don't expect to see the company name in green as the browser does not get the original certificate and the new one does not have the EV tag.

Concern No. 7: Can the administrator use the gateway's private key to decrypt the communication? Well this would be a real concern to the users as nobody would want an administrator to be able to see his personal emails or bank statements or any other sensitive information. It may seem that by using the gateway's/CA's private key in Wireshark, the administrator can decrypt the entire communication but this is not true. The gateway's private key is used to sign certificate and not to exchange secrets, however we have not gone in depths to find if the plain text data can be extracted from the kernel or daemons on the gateway.

Blog: kishinf.blogspot.com

Facebook: www.facebook.com/ksecure

Twitter: www.twitter.com/ksecure

Page | 8

Inspecting HTTPS Traffic on Gateways by Kishin Fatnani

CONCLUSION
Inspecting HTTPS is extremely useful and must be used to secure communications, however, one must understand the above issues and accordingly create policies to inspect necessary traffic and bypass traffic that is not compatible with this module but is required for business. Also it is important to make the users aware of the changes as many times the decision to continue or stop a session comes in the hands of the users.

SOURCES
As part of the research done for this article I visited a lot of sites and read quite a lot of documents. It is not possible for me to list all of them, however the ones that give me good insights are mentioned below: http://en.wikipedia.org/wiki/Secure_Sockets_Layer http://en.wikipedia.org/wiki/Transport_Layer_Security http://en.wikipedia.org/wiki/Online_Certificate_Status_Protocol http://technet.microsoft.com/en-us/library/ee658156.aspx#BKMK_HowHTTPSinspectionworks Check Point's Application Control and URL Filtering R75.20 Administration Guide

ABOUT AUTHOR
Kishin Fatnani - CISSP, GCIH GOLD, GCFA, CCSE R70, CEH, LPT, .... Kishin is the founder of K-Secure (www.ksecure.net), an IT security training company. He has over 20 years experience in IT, main focus being security, and conducts many security trainings including Packet Analysis, Snort Rule Writing, Ethical Hacking, Forensic Analysis, Web Application Security, Check Point and more. He has also been a SANS local mentor for the GIAC certification GCFA and has some contributions to the development of Check Point courseware and certification exam. He can be contacted at kishinf@ksecure.net Blog: kishinf.blogspot.com Facebook: www.facebook.com/ksecure Twitter: www.twitter.com/ksecure

Blog: kishinf.blogspot.com

Facebook: www.facebook.com/ksecure

Twitter: www.twitter.com/ksecure

Page | 9