An Introduction to

Distributed Switching
An overview of distributed switching,
its benefits and applications


















Intune Networks All Rights Reserved 2
Distributed Switching
Distributed switching is a new networking technology that can be used to build more efficient
networks than previously possible. It replaces conventional switching, transport and multiplexing
systems with a single network function.

A distributed switch has many of the characteristics we would expect in a conventional switch - it
has ports that can switch data packets between each other across a backplane. But unlike a
conventional local switch, whose ports are located in the same chassis, a distributed switch has
ports that can be spread across hundreds of kilometres, using an optical backplane. Because the
ports on a distributed switch can be many miles apart, a single switch can replace an entire
network of switches and transport links. In addition, a distributed switch provides a full-mesh of
non-blocked connectivity between each of its ports, so it is not limited by the constraints of
traditional point-to-point topologies.

What problems does it solve?
Distributed switches can be used to build networks with the same functionality as traditional
switch and transport systems, but they are particularly good at solving some specific networking
challenges:

Network and IT unification - for example deploying SDN over distance
Network virtualisation - enabling connectivity on demand
Creating fully meshed networks - without blocking traffic
High utilisation of fibre assets - with no packet loss
High resilience - with no cost premium

Conventionally, the only way to provide a full mesh of loss-less connectivity was to over-build te
network. Distributed switching solves that problem.

How does it work?
In a conventional network, switching is
electrical, and transport optical. Additional
colours of light, or wavelengths, are used to
provide a multiplexing function across
transport links.

A distributed switch, like Intune's iVX8000,
uses the optical layer to perform both
switching and transport functions
simultaneously. Multiple colours of light are
used as an address space.

Each port on the switch is automatically
provisioned with its own wavelength. Any port
wishing to send a packet to that destination
simply tunes its transmitting LASER to the
correct destination colour. The receiving port
sees all packets on the fibre optic backplane
and filters off packets with its allocated colour.


Fig 1.


Intune Networks All Rights Reserved 3
Because optical systems have such low losses, the packets can traverse many miles across the
backplane between ports, creating a distributed switch.

The system uses tuneable LASERs, which must be able to change colour fast enough to send every
packet to a different destination. Intune, for example, have built systems that can re-tune and
stabilise a LASER in nanoseconds - fast enough to optically switch packets at line rate. The system
also requires an optical collision-avoidance mechanism and scheduling mechanisms to prevent
packet-loss, and enable QoS, multicast and port-mirroring - functions you would expect on a local
electrical switch.

How does it compare to conventional architectures?
Because distributed switching systems include both switching and transport functions, they are
most meaningfully compared to an equivalent network, rather than individual components.

A significant difference, is that a distributed switch uses no ports to connect the network nodes to
each other via transport links. Every port on a distributed switch is 'service-facing' - all the
'network-facing' ports have been replaced by the optical backplane. This significantly reduces the
number of ports required, the cost, and the complexity to create the equivalent network
connectivity. See Fig 2.

Operationally, it also removes the need to plan, implement and manage each point-to-point link.

Fig 2.

Fig 3.


Intune Networks All Rights Reserved 4
And because the system acts as a single distributed switch, all of its capacity is available between
any of its ports, eliminating the risk of traffic being blocked by an individual congested link.

If the network is highly meshed then the benefit of distributed switching is even greater (see Fig 3).
For example, to provide non-blocking, fully-meshed connectivity between n nodes with a single
service port at each node, requires an additional n(n-1) internal network-connections (the 'n-
squared problem') - whereas a distributed switch only requires its n service ports. This is because
the distributed switch can send traffic from any port to any port on a per-packet basis, as if they
were connected locally. A ten node network would need 90% less switching capacity to achieve the
same amount of unblocked connectivity.

How does it scale?
Scaling a network node
If we look inside a traditional
switching node we find two
elements: an electrical L2/3
switch and an optical transport
system (DWDM or ROADM for
example). See Fig 4a.

The switch will have service ports
which may connect to customers
or content or aggregate other
switches further downstream.
However, on the 'network-side' of
the switch, further ports are
required, with a 1:1 relationship
to the transport paths required
for connectivity to other nodes. Inside the optical system there is also a 1:1 relationship between
the number of transmitters and receivers per transmission path. In a meshed network the number
of switch and optical ports
becomes an 'n-squared
problem'.

Conversely, a distributed switch
port integrates these functions
into its service-ports, which are
directly linked to an optical
backplane - with no further
connections required. See Fig 4b.
Adding another port at the node
automatically adds more optical
capacity, simultaneously
delivering both switching and
transmission capacity across the
system.

Most importantly, any optical
transmitter can be tuned to send to any receiver at any instant in time - so a single transmitter and
receiver can send any of its capacity (partially or entirely) anywhere across the entire system. By
breaking this 1:1 dependency, a single set of optical components can deliver a full set of network

Fig4a.

Fig4b.


Intune Networks All Rights Reserved 5
connectivity and create a network that benefits from the efficiency of statistical packet
multiplexing.

Adding further ports adds both i/o (input/output) and switch fabric capacity to the whole system.
Operationally this is also very simple. Because every port can reach every other port simply by
sending light with a new colour, a new port is instantly available to all previous ports with no
additions or changes required anywhere else in the system.
Scaling a switch
Distributed switching systems have multiple dimensions in which they can scale, most significantly:

Switch Fabric - the total switching capacity of the distributed system
Reach - how far the system can transport packets
Ports - how many ports can be supported and at what speeds
Switch Fabric
The switch fabric capacity can, in itself, be increased in three ways:
Increasing the bit rate of each tuneable LASER
Adding more wavelengths. Wavelengths are principally used as an addressing space, but
because adding an address also allows a new port to be added, it inherently adds more
switching capacity to the system
Using more fibres within a single system. Initial systems use a single pair of fibres but
multiple pairs of fibres could be used for multiple stages of switching.
Foreseeable advances in each of these dimensions suggest current architectures could scale to
produce a 64Tbps distributed switching fabric.
Reach
The reach between ports is determined by similar constraints to conventional optical transport
technology, and as those limits change, so the reach of a distributed switch will also increase. At
the time of writing, the technology typically supports ports spread across a 500km circumference.
Ports
The speed of the client ports is largely a market-driven consideration. A common misconception
about a distributed switch is that the speed of the optics determines the speed of the client-ports
(as they would in a transport network). The 'backplane' of a distributed switch uses multiple optical
paths clocking at a certain rate.
This is analogous to looking inside
an electrical switch backplane,
which might run multiple paths
clocking at 2-6Gbps, for example,
yet still service a 100GE i/o port.
In a distributed switch, it happens
to be efficient to match the
backplane clock speeds with the
speed of the client ports, but
there is no direct linkage between
the two. So a switch using optics
clocking at 10Gbps can provide
1Gbps or 100Gbps client ports.

Fig 5.


Intune Networks All Rights Reserved 6
Scaling a network
When a conventional electrical
switch reaches its capacity, the
network is scaled by adding an
intermediate layer of switching,
or building the central switch into
an interconnected 'fabric'. Either
way, increasing numbers of
switch ports are required to
interconnect the network
'internally', consuming ever more
resources internally than is spent
on the service-facing ports. See
Fig 5.

A distributed switch can also be
scaled in a similar way, by
introducing another layer of switching. However, because the aggregating switch ports can be
physically distributed and co-located with service switch ports, far less internal ports are required
to create a resilient network architecture. See Fig 6.

Benefits of Distributed Switching
Distributed switching offers a number of key benefits over traditional networks:
Highly Connected Networks
Distributed switches inherently provide a full-mesh of connectivity between all of their ports.
When traditional switch and transport networks create this full mesh, they suffer from the 'n-
squared problem' resulting in an exponential amount of ports. This full-mesh attribute is
particularly useful in interconnecting distributed data centres, for example.
Networks have fewer components
Traditional networks consume many switch ports interconnecting with other switches via point-to-
point transport links. But a distributed switch can present its client-ports hundreds of kilometres
apart from one another, so there is no need for intermediate switching stages or connections to
transport links. So the system requires fewer switch ports, fewer optics, and fewer electrical to
optical interfaces than with conventional architectures. Operators who have assessed this
architecture have estimated capex savings in excess of 50% in highly connected networks.
The network is easier to operate
Collapsing multiple switching and transmission systems into one single element lowers planning,
configuration and management costs. The system also uses less power than conventional
networks (because it processes packets optically rather than electrically) and consumes
significantly less rack-space. Opex savings of around 60% can be expected.
Efficient Utilisation of Fibre Assets
Distributed switches use optical fibres as their backplane. They switch packets directly in the
optical layer, rather than embedding them within semi-static point-to-point transport links, like
DWDM. This mean that a distributed switch has the efficiency of statistical packet multiplexing, so
they can effectively fill fibre assets at nearly 100%, whilst still providing QoS. Traditional networks

Fig 6.


Intune Networks All Rights Reserved 7
can only risk filling fibres at high capacity if traffic patterns are very predictable, or if packet losses
are acceptable.
High Resilience at Low Cost
Aside from the level of carrier-class resilience you'd expect in any system of this type, the
distribution of switching is fundamentally more resilient than previous architectures. Resilient
paths across the system can be instantly turned-up without the need to 'reserve' them during
normal operation. And client-connected resources (such as storage) can be spread over multiple
sites with no additional
cost of connectivity (as
every port can connect
to every other.)
Network Virtualisation
Intune's system
effectively virtualises
layers 0-2 of the
network. All the network
capacity is available to
any port, so the system
can deliver highly
dynamic traffic flows
with no impact on traffic
crossing between other
ports.
Transforming the network into a unified computing platform (with SDN)
The system abstracts two networking domains (switching and transport) into one. It also operates
as a single network element instead of many. Add to this its behaviour as a non-blocking full-mesh
of connectivity and it becomes a powerful tool when opening up the wide area network to an
external controller. Using web services or Openflow, for example, an external controller simply has
to populate the system's forwarding fabric to control the equivalent of an entire network. IT
resources can be located anywhere in the network and be made available as a single virtual pool,
which is both highly efficient, highly resilient, and delivers optimum user-experience. See Fig 7.

How is distributed switching best used?
Distributed switching has many advantages over conventional networks, but it is not suitable for
every network application. It offers only marginal benefits if the traffic volumes between ports are
low (<1Gbps), if the connectivity is only ever between two points, or if the traffic patterns are
highly stable and predictable.

However, any network that requires high levels of connectivity and high bandwidth, that needs to
cope with unpredictable traffic flows or needs to be unified with an externally controlled system,
can benefit from the use of distributed switching. Application examples include:
Distributed Data Centres
The interconnection of distributed data centres to create a single unified data centre, regardless of
the physical location of servers, storage and processing. This application particularly benefits from
the mesh of non-blocking connectivity delivered by distributed switching.


Fig 7.


Intune Networks All Rights Reserved 8
Software Defined Networks
Extending SDN across the wide area is inherently more complex than in the data centre. This is
partly because of the need to
control the additional domain of
transport as well as switching,
and partly because it is too
expensive to over-build capacity
in the wide area, as we can in the
data centre. Distributed
switching allows an entire wide
area network to be abstracted
into a single switch fabric, which
can be externally controlled and
populated in the same way as a
local switch in the data centre.
Content Distribution Networks
Distributed switching is ideal for connecting many customer-connected switches to content servers
in Content Distribution Networks (CDNs). Distributed switching allows users to access content from
many different content servers, regardless of their physical location, and allows the operator to
load-balance efficiently across content servers in different locations.
'Elastic' Ethernet Service Platform
Distributed switching provides highly dynamic connectivity - so services can be turned up and
down in an instant, without the risk of affecting other traffic. This is ideal for offering flexible or
'elastic' services that are created on-demand.
Next Generation Metro Networks
Metro network architectures are becoming increasingly challenged by the demands to deliver
more capacity as well as greater interconnectivity. Distributed switching can significantly increase
the efficiency of scaling these networks, as well as enabling SDN to transform them into giant
extended computing resources.

Distributed Switching - A Networking Break-Through
Distributed switching is the most significant development in networking technology since the
arrival of IP and fibre optic transport. It has the potential to radically reduce networking costs as
well as unify the network with computing resources. Perhaps unsurprisingly, it is already being
installed and tested by some of the world's largest network operators.

More details on distributed switching and its applications are available from Intune Networks at
www.intunenetworks.com.




About Intune Networks
Intune is a venture-backed company headquartered in Dublin, Ireland, with offices in the United
States and further research and development facilities in Belfast, UK.