DO NOT REPRINT

FORTINET

Web Filtering

In this lesson, we will show you how to filter users access to web sites, which is one of the most
commonly used features employed by network administrators.

DO NOT REPRINT
FORTINET

Web Filtering

After completing this lesson, you should have these practical skills. This will give you an understanding
of the various options that are available to manage and track web content.
Familiarity with website design and behavior, as well as the HTTP protocol are useful to understanding
this module.

DO NOT REPRINT
FORTINET

Web Filtering

Web filtering is simply a means of controlling, or tracking, the websites people visit. There are many
reasons why a network administrator would want to do this: preserve employee productivity; prevent
network congestion where valuable bandwidth is used for non-business purposes; prevent loss or
exposure of confidential information; decrease exposure to web-based threats; limit legal liability when
employees access or download inappropriate or offensive material; prevent copyright infringement
caused by employees downloading or distributing copyrighted materials; prevent children from viewing
inappropriate material.

DO NOT REPRINT
FORTINET

Web Filtering

Proxy-based web filtering is achieved using a transparent proxy intercepting traffic between the client
and server, and setting up a man-in-the-middle. Proxy-based provides he the most flexibility and
configuration options for inspecting web traffic because it intercepts at Layer 7, as such some features
are only available to you when using proxy-based inspection. Greater control comes at a cost, it is also
the most resource intensive in terms of memory and CPU usage, resulting in the slowest throughput.
That said, it is widely used and is a very strong solution on appropriately scaled systems.

DO NOT REPRINT
FORTINET

Web Filtering

Flow-based web filtering is achieved by caching traffic intercepted traffic between the client and server,
analyzing the TCP flow: hence flow-based. It provides less flexibility and configuration options for
inspecting web traffic, when compared to proxy-based, because it intercepts at Layer 3 and works with
the Layer 4 data. It does not recover actual files, as the proxy does, so content cannot be sent to
scanunit.

DO NOT REPRINT
FORTINET

Web Filtering

Rather than looking at the HTTP protocol, another option is to filter the DNS request that occur prior to
an HTTP Get request. This has the advantage of being very lightweight, but at a cost because it lacks
the precision of HTTP filtering. Every protocol will generate DNS requests in order to resolve a
hostname, therefore this kind of filtering will impact all of the higher level protocols that depend on DNS,
not just web traffic. For example, it could apply FortiGuard categories to DNS requests for FTP servers.
Very few web filtering features are possible beyond hostname filtering, due to the amount of data
available at the point of inspection.

DO NOT REPRINT
FORTINET

Web Filtering

Inspection mode is set in the web filter profile. When changing mode, the options displayed will change
because they are dependent on the inspection mode. When a web filter profile using proxy inspection
mode is selected in your firewall policy, a proxy options profile must also be defined. The proxy options
profile defines proxy behaviors as well as the ports to be inspected for web or DNS traffic. HTTPS
inspection port numbers, and other settings related to the handling of SSL, are defined separately in the
SSL/SSH inspection profile.

DO NOT REPRINT
FORTINET

Web Filtering

Lets summarize the different modes. Proxy-based caches traffic, so it can cause a noticeable delay
depending on the file size, oversize limit and connection speed. It does, however, support a greater
number of web filtering features. Flow-based has a much higher throughput rate, compared to proxybased, because it does not cache data so there is no transmission delay. DNS-based is very lightweight
because it handles only the nameserver lookup, but suffers from accuracy issues because it does not
see the full URL.

DO NOT REPRINT
FORTINET

Web Filtering

DNS web filtering looks at the nameserver response which typically occurs when you connect to a
website. Proxy and flow-based web filtering booth look for the HTTP 200 response returned when you
successfully access the website. Handling the response, as opposed to the DNS request or HTTP Get,
confirms the site is present.

DO NOT REPRINT
FORTINET

Web Filtering

Static URL filtering is enabled in the web filter profile. Entries in the URL filter list are checked against
the website that is visited. If a match is found, then the configured action is taken. If there is no match,
then the FortiGate will move on to the next check enabled.
Patterns set to the type Simple are exact text matches. Patterns set to the type Wildcard allow for
some flexibility in the text pattern by allowing wildcard characters and partial matching to occur. Patterns
set to the type Reg. Expression allows for the use of PCRE regular expressions to be used.

DO NOT REPRINT
FORTINET

Web Filtering

When a user visits a website, the FortiGate looks at the URL list for a matching entry. In this example,
the website matches the 3rd entry (using same list as the previous slide). This entry is a simple type, so
the match must be an exact one. There is no option for a partial match with a simple pattern. In this
case the action is to block the website so the user is presented with a block page, rather then the
website they were expecting to see.

DO NOT REPRINT
FORTINET

Web Filtering

Rather than block or allow websites individually like Static URL filtering, FortiGuard Category filtering
looks at the category that a website has been rated with. Action is taken based on that category, not the
URL itself.
FortiGuard Category filtering is a live service that requires a connection to the FortiGuard network and
active contract in order to operate. If the contract expires, there is a 7 day grace period to renew the
contract before services will be cut off. Rather then communicating to the FortiGuard network to receive
a websites category, larger FortiManager models can be used instead.
FortiGuard Category filtering and Static URL filtering have different lists of possible actions that can be
configured. The impact of selecting different actions will be covered later on.

DO NOT REPRINT
FORTINET

Web Filtering

When a user visits a web site, you can use the FortiGuard live service to find out the category for the
URL and allow or block access by category. This is a great way to perform bulk URL filtering without
having to individually define each web site.
After the 7 day grace period the FortiGate will not be able to rate websites and every visit will be treated
as a rating error. In the event of a rating error for a website there are only 2 options, block or allow.

DO NOT REPRINT
FORTINET

Web Filtering

FortiGuard category filtering is enabled in the GUI, through the Web Filter profile. Categories and subcategories are listed and can have the action to take defined individually. Actions are assigned through
right clicking the mouse and selecting from a menu.
If the feature is enabled and the unit does not have a valid contract then a warning will be displayed in
the GUI.

DO NOT REPRINT
FORTINET

Web Filtering

The FortiGate can maintain a list of recent web site rating responses in memory, so if the URL is one
that the device already knows about it will not have to send back a rating request. Two ports are
available for the unit to query FortiGuard with, port 53 and port 8888. Port 53 is the default since this is
also the port number used for DNS which is almost guaranteed to be open. However, any kind of
inspection will reveal that this traffic is not DNS and prevent the service from working. In this case, you
can switch to the alternate port 8888, but this port is not guaranteed to be open in all networks so you
will need to check this before setting this up. Port 80 is an option for FortiGuard communications, but
only if you are using a FortiManager, rather then the FortiGuard network.

DO NOT REPRINT
FORTINET

Web Filtering

Caching responses reduces the amount of time it takes to establish a rating for a website. Packets
operate on the scale of milliseconds at the fastest with Seconds, not being unusual. Memory checking is
orders of magnitude faster (nanoseconds).
This timeout defaults to 15 seconds but can be adjusted as high as 30 seconds if necessary.

DO NOT REPRINT
FORTINET

Web Filtering

Web site categories are determined by both automatic and human methods. The FortiGuard team has
automatic web crawlers that look at various aspects of the website in order to come up with a rating.
There are also people who examine websites and look into rating requests in order to determine
categories.

DO NOT REPRINT
FORTINET

Web Filtering

There is always the possibility for errors in rating, or a scenario where you simply do not agree with the
rating a site has been given. In this case, you can use the web portal to contact the FortiGuard filtering
team to submit a web site for a new rating, or to get it rated if it is not already in the database.

DO NOT REPRINT
FORTINET

Web Filtering

The Warning action is only an option when using FortiGuard Category filtering and only with Proxymode inspection. It is not available with Static URL filtering.
When someone visits a website that is in a Category with an action of warning, they are presented with a
page that warns them they may not wish to visit this website. They are given a choice to go to the
website anyway, or go back to the previous website.

DO NOT REPRINT
FORTINET

Web Filtering

The Authenticate action is only an option when using FortiGuard Category filtering and only with Proxymode inspection. It is not available with Static URL filtering.
The authentication action blocks all websites that are in that category, unless a successful passcode is
entered. This is not user authentication and putting in proper credential will not result in any kind of
login. The username/password pair is used in the same way a key is used to open a locked door.
Once this has been done successfully, access is allowed to that category for the amount of time that has
been configured. This will allow the user to visit any other websites that are in the same category for
however long has been configured. They will not be prompted again when visiting a second (or third)
website in the same category, so long as the timer has not expired.

DO NOT REPRINT
FORTINET

Web Filtering

The Exempt action is only an option when using Static URL filtering. It is not available with FortiGuard
category filtering.
The exempt action is used in order to bypass issues that may be caused by other checks. Sometimes
FortiGuard category filtering is not granular enough, sometimes a file you need is being caught by virus
scanning. Exempt gives the ability to bypass one or more checks or all further checks.

DO NOT REPRINT
FORTINET

Web Filtering

These actions are possible with FortiGuard Category filtering and Static URL filtering. Regardless of
which feature they are used with, the resulting action will be the same.

Allow Effectively defines the website as being trusted. Access to the site is permitted and no log
message is generated to record this.
Monitor Access to the website is permitted and a log message is generated to record the event
Block Prevents access to the website and displays a block page to the user instead.

Log message generation is subject to firewall policy, specifically the Logging Option setting.

DO NOT REPRINT
FORTINET

Web Filtering

When using FortiGuard category filtering, one option to allow or block access to a website is to make a
web rating override and define the website to be in a category other then what FortiGuard puts it into.
Web ratings are only for hostnames, no URLs or wildcard characters are allowed.
Category filtering is not granular, like static URL filtering. If you have a category that is blocked (or
allowed) and you need to make an exception for a particular website, this is one option that is available
to you.
If the contract expires, and the 7 day grace period passes, web rating overrides will be not be effective.
All website categories will be still be considered rating errors.

DO NOT REPRINT
FORTINET

Web Filtering

Since FortiGuard category filtering is not granular and performs actions based on the category the
websites are in there may be times when an exception needs to be made for a single website.
Rather then unblock a potentially unwanted category access can be provided an a site-by-site basis.
The reverse can also be true, with the majority of websites in a category being fine, but a single one
needs blocking.
Changing the category does not automatically result in a different action for the website. This will
depend on the settings within the Web Filter profile at the time the user is accessing that web site.

DO NOT REPRINT
FORTINET

Web Filtering

Custom categories can be created and used in conjunction with Web rating overrides. If the predefined
categories within FortiGuard are not suitable for the situation, additional customized categories can be
added.
These custom categories can be added and deleted as needed, so long as they are not in use. A
category is considered to be used if there are any Web rating overrides that have been configured to us
it. It will also be considered in use if there is an action associated with that category other then Allow in
any web filter profile.

DO NOT REPRINT
FORTINET

Web Filtering

FortiGuard quota can be used to limit the time users spend on web sites, based on the categorization.
Quota cannot redirect you once the web site is loaded in the browser. For example, if you had 45
seconds left on your quota and you visited a web site, it would likely finish loading before 45 seconds
was done. You could then spend 20 minutes browsing the information you received. You could not get
blocked or notified until the next attempt to access another one of these web sites. The reason for this is
that the connection to the web site is not generally a live stream. Once you receive the information, the
connection is closed.

DO NOT REPRINT
FORTINET

Web Filtering

Quotas are configured just below where you configure the Category actions in the Web filter profile.
There can be multiple quotas (timers) configured within this section. Each one can either be linked to a
single category, or multiple. If the Quota applies to multiple categories then it is not that amount for each
individual category, the timer applies to all of the categories that are specified.

DO NOT REPRINT
FORTINET

Web Filtering

Some Features on the FortiGate cant provide direct user feedback. FortiGuard quota wont provide any
feedback to the user until they exceed the quota they have been given, unless the Fortinet bar is
enabled.
The Fortinet Bar injects a Java applet which uses a communications port to talk to the FortiGate and get
additional information from features that would otherwise provide no direct user feedback.
FortiGuard quota provides a count down.
Other features that cant do block pages (IE: application control) will show block events in the top bar.
HTTPS pages are a lot more sensitive to injected data, so its not possible to reliably insert data, so the
Fortinet Bar is only available for HTTP websites.

DO NOT REPRINT
FORTINET

Web Filtering

Enforcing safe search can be done for Google, Bing and Yahoo. Safe search is an option that some
search engines have in order to apply their filters to the search results that are displayed. This way even
if Safe Search is disabled in the browser, the FortiGate will make sure the query is subject to whatever
settings the service decides. All the FortiGate can do is ensure that it is enabled. It cannot dictate the
behavior of this, as this task is up to the search engine providers. It works by looking for the Safe
Search string when you submit a search. If it is not there, the FortiGate unit will modify the request to
include it. This way, even if it is not enabled locally in the browser, it gets applied to the request as it
passes through the FortiGate.
YouTube EDU filtering is also available. This is a service offered by YouTube to educational institutions.
When you create an account with them they provide you with an identifier. Unlike normal Safe Search,
this does not append the URL, but adds an HTTP header into the packets. This identifies your school to
YouTube when people visit. Within your YouTube EDU account, you can configure the filters and
settings in order to limit video access.

DO NOT REPRINT
FORTINET

Web Filtering

There are several different components to web filtering, and when they are enabled, the inspection order
follows these steps.
The local static URL filter occurs first.
Second, FortiGuard category filtering determines a rating.
Finally advanced filters take place, like Safe search or removing Active X components.
After all the checks are done the information is handed off internally for virus scanning.

DO NOT REPRINT
FORTINET

Web Filtering

Heres a look at the web filter profile. Up at the top you can enable FortiGuard and assign the actions to
the various web site categories.
If you scroll down towards the bottom you will find the more advanced options that can be enabled, like
Safe Search and Static URL filtering. Once you have enabled and saved the settings you require, you
will need to apply the profile to your firewall policy to activate the options.

DO NOT REPRINT
FORTINET

Web Filtering

Web profile overrides change the rules that will be used to inspect traffic. Enabling them allows
authorized users to enter a passcode that will change the Web filter profile that inspects there traffic to
another profile. Proper configuration would mean this new profile had elevated access permissions and
allow additional websites. The new profile will be used to inspect ALL of their web traffic from that point
on, until the timer expires. Authentication must be enabled in order to use this. Once web profile
overrides are enabled, the FortiGuard block page will show an override link that users can select in order
to active this override.

Apply to Groups Select the user credentials that allow overrides.

Assign to Profile Which Web profile will be used, after a successful override.
Scope Who will be effected by the override.
Duration How long the override will last.

DO NOT REPRINT
FORTINET

Web Filtering

How the FortiGate handles HTTPS traffic is decided based on the settings of the SSL Inspection profile
that is applied to the Firewall Policy. SSL Certificate Inspection reads only unencrypted data from the
hello message, whereas Full SSL Inspection will proxy SSL, allowing for full content inspection.
SSL and Certificates are covered in more detail in the Certificate Operations module.

DO NOT REPRINT
FORTINET

Web Filtering

This is an example of the log message generated as a result of applying a web filter profile on a firewall
policy. Access details include information about the FortiGuard quota and category (if those are
enabled), which web filter profile was used to inspect the traffic, the URL and more details about the
event.

DO NOT REPRINT
FORTINET

Web Filtering

You can also view the raw log data by selecting the Download Raw Log button at the top right of the
GUI. When the downloaded file is opened, it will be a plain text file in a syslog format.

DO NOT REPRINT
FORTINET

Web Filtering

List of IPs to use for FortiGuard comes back from update server (FortiGuard Distribution Network or
FortiManager).

Weight Based on the difference in timezone between the FortiGate and this server (modified by
traffic)
RTT Return Trip Time
Flags D (IP returned from DNS), I (Contract server contacted), T (being timed), F (failed)
TZ Server timezone
Curr Lost current number of consecutive lost packets (in a row, resets to 0 when 1 packet
succeeds)
Total Lost total number of lost packets

List is a variable length, depending on the FortiGuard Distribution Network, but approximately 10 total
IPs is the average.

DO NOT REPRINT
FORTINET

Web Filtering

Logs can be used to determine the decision made by the FortiGate but this depends on the configured
settings. The firewall policy may not be set to log or the action could be set to accept. In both of those
cases no log event will be generated to record the decision.
This diagnostic shows the full URL in the output. In order to have it fit some of the output was chopped
off from this page. The source of the request, the hostname, URL, user (if authentication is enabled), the
profile used to examine the URL can all be determined by reading the output.

DO NOT REPRINT
FORTINET

Here is a review of what we discussed. We showed:

An overview of web filtering functionality
Explained the different types and modes for web filtering
How static URL filtering works
How FortiGuard category filtering works
How to submit a website for rating
Different actions that can be associated with accessing a website
How to do a rating override and create a custom category
Applying a quota to a category
Introduced the Fortinet Bar
Showed how its possible to force safe search with some common websites
Explained the order of the checks involved with inspecting websites
Explained how to configure a web profile override
Finally we covered the basics of inspecting HTTPS traffic

Web Filtering