EMC VNX 5400 Storage Solution

Exchange Server 2013 Solution Reviewed Program (ESRP)
EMC VNX5400 UNIFIED STORAGE

20,000 USERS VIRTUALIZED MICROSOFT
EXCHANGE SERVER 2013
MAILBOX RESILIENCY STORAGE SOLUTION
Tested with: ESRP – Storage Version 4.0
Test date: March 2014
EMC SOLUTIONS
Abstract
This white paper describes the technical validation of a 20,000-user Exchange
2013 storage solution deployed on EMC® VNX5400™ unified storage and
virtualized on Microsoft Hyper-V according to criteria specified by the Microsoft
Exchange Solution Reviewed Program (ESRP) – Storage program. The
performance results and best practices presented in this paper provide
validated guidelines for configuring the VNX5400 storage system for a large
enterprise Exchange Server 2013 environment.
March 2014
Copyright © 2014 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its

publication date. The information is subject to change without notice.
The information in this publication is provided as is. EMC Corporation makes no

representations or warranties of any kind with respect to the information in this
publication, and specifically disclaims implied warranties of merchantability or
fitness for a particular purpose. Use, copying, and distribution of any EMC
software described in this publication requires an applicable software license.
EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC
Corporation in the United States and other countries. All other trademarks used
herein are the property of their respective owners.
For the most up-to-date listing of EMC product names, see EMC Corporation
Trademarks on EMC.com.
Part Number H12997
EMC VNX5400 Unified Storage

20,000 Users Virtualized Microsoft Exchange Server 2013 2
Mailbox Resiliency Storage Solution
Table of contents
Overview .............................................................................................................................................. 5
Disclaimer ............................................................................................................................................ 5
Features ............................................................................................................................................... 6
EMC VNX series of unified storage platforms .................................................................................... 6
Architecture................................................................................................................................. 6
VNX software ............................................................................................................................... 7
VNX FAST VP ................................................................................................................................ 8
EMC Virtual Provisioning technology on VNX ............................................................................... 9
EMC VNX5400 specifications....................................................................................................... 9
Solution description ........................................................................................................................... 11

Storage design for Exchange Server 2013 ...................................................................................... 12
Building block used in this solution ............................................................................................... 13
VNX storage pools and data protection .......................................................................................... 14
Mailbox server scalability factors ................................................................................................... 15
Exchange virtualization .................................................................................................................. 15
Targeted customer profile .............................................................................................................. 15
Tested deployment ........................................................................................................................ 16
Best practices..................................................................................................................................... 19
Hyper-V virtual Fibre Channel ......................................................................................................... 19
Mailbox server virtual machines ..................................................................................................... 19
Hypervisor host .............................................................................................................................. 20
Networking .................................................................................................................................... 20
Core storage................................................................................................................................... 20
Backup strategy ............................................................................................................................. 21
Information resources ........................................................................................................................ 21
Test results summary ......................................................................................................................... 22

Storage reliability (stress test) ....................................................................................................... 22
Storage performance...................................................................................................................... 22
Individual server metrics ........................................................................................................... 22
Performance across servers ....................................................................................................... 23
Database backup and recovery performance.................................................................................. 23
Database read-only performance............................................................................................... 24
Transaction log recovery/replay performance ............................................................................ 24

Detailed test results ........................................................................................................................... 24
How to view Jetstress reports ......................................................................................................... 24
Conclusion ......................................................................................................................................... 25
Contact EMC .................................................................................................................................. 25

Overview
This document provides information about EMC® VNX5400™ unified storage array
performance with 20,000 Microsoft Exchange Server 2013 users. Performance
validation is based on Microsoft Exchange Solution Reviewed Program (ESRP)—
Storage program guidelines 1. For any questions or comments regarding the content of
this document, see Contact EMC on page 25.
Disclaimer
This document has been produced independently of Microsoft Corporation. Microsoft
Corporation expressly disclaims responsibility for, and makes no warranty, express or
implied, with respect to, the accuracy of the contents of this document.
The information contained in this document represents the current view of EMC on
the issues discussed as of the date of publication. Due to changing market
conditions, it should not be interpreted as a commitment on the part of EMC. In
addition, EMC cannot guarantee the accuracy of any information presented after the
date of publication.
1
The ESRP—Storage program was developed by Microsoft Corporation to provide a common
storage testing framework for vendors to provide information on their storage solutions for
Microsoft Exchange Server software. For more details on the Microsoft ESRP—Storage
program, refer to http://technet.microsoft.com/en-us/exchange/ff182054.aspx.

Features
EMC VNX series of The EMC® VNX™ series delivers industry-leading innovation and enterprise
unified storage capabilities for file, block, and object storage in a scalable, easy-to-use solution. This
platforms next-generation storage platform, shown in Figure 1, combines powerful and flexible
hardware with advanced efficiency, management, and protection software to meet
the demanding needs of today’s enterprises.
Figure 1. EMC VNX series of unified storage platforms
EMC VNX series unified storage systems deliver uncompromising scalability and
flexibility for the mid-tier while providing market-leading simplicity and efficiency to
minimize total cost of ownership.
Architecture
Based on the powerful new family of Intel Xeon E5-2600 (Sandy Bridge) processors,
the EMC VNX implements a modular architecture that integrates hardware
components for block, file, and object with concurrent support for native network-
attached storage (NAS), Internet small computer system interface (iSCSI), Fibre
Channel (FC), and Fibre Channel over Ethernet (FCoE) protocols. The series delivers
file (NAS) functionality via two to eight X-Blade data movers and block (iSCSI, FCoE,
and FC) storage via dual storage processors (SPs) using full 6 Gb/s Serial Attached
SCSI (SAS) disk drive topology. The system includes the patented EMC MCx™
multicore storage software operating environment that delivers unparalleled
performance efficiency. You can start with block or file functionality and easily
upgrade to unified when needed. The unified configuration includes the following
rack-mounted enclosures:
• Block Services: Disk processor enclosure (includes disk drives) or SP
enclosure (no drives included) plus standby power system. Capacity for block
or file use cases is added via disk-array enclosures (DAEs).
• File and Unified Services: One or more data mover enclosures and a control
station to deliver file protocols.

Customers can benefit from the new VNX features such as 2:
• Next-generation unified storage, optimized for virtualized applications
• Extended cache using flash drives with EMC Fully Automated Storage Tiering
(FAST™) Cache and FAST for Virtual Pools (FAST VP) that can be optimized for
the highest system performance and lowest storage cost simultaneously on
both block and file
• Multiprotocol support for file, block, and object, with object access through
EMC Atmos® Virtual Edition (Atmos VE)
• Simplified management with EMC Unisphere™, which provides a single
management framework for all NAS, SAN, and replication needs
• Up to three times faster performance with the latest Intel multicore CPUs
optimized for flash
• 6 Gb/s SAS back end supporting the latest drive technologies including a
variety of 2.5-inch and 3.5-inch drive types such as flash, SAS, and nearline
SAS (NL-SAS), of varying capacity sizes
• Expanded EMC UltraFlex™ I/O connectivity—FC, iSCSI, Common Internet File
System (CIFS), Network File System (NFS) including parallel NFS (pNFS), Multi-
Path File System (MPFS), and FCoE connectivity for converged networking over
Ethernet
VNX software
The VNX series includes six software suites and two software packs, making it easier
and simpler to attain the maximum overall benefits 3. All VNX software is managed
through Unisphere.
Software suites
The VNX software is also available in modular suites:
• FAST Suite—Automatically optimizes performance and cost to simultaneously
provide the highest system performance and the lowest storage cost
• Security and Compliance Suite—Keeps data safe from changes, deletions,

and malicious activity
• Local Protection Suite—Practices safe data protection and repurposing
• Remote Protection Suite—Protects data against localized failures, outages,

and disasters
2
Features listed are based on the VNX Operating Environment version available at the time of
this solution validation. EMC constantly improves and updates its storage fleet with new
features and functionalities. For the latest features and updates visit www.emc.com.
3
Features of Suites and Packs might vary depending on the system. See the EMC VNX Series
Software Suites data sheet for details.

• Application Protection Suite—Automates application copies and proves
compliance
• Unisphere Management Suite—Monitors and manages multiple VNX systems

with full visibility across the virtual stack
Software packs
The VNX series software is also available in two comprehensive packages to ensure
that customers have all of the necessary capabilities to protect and manage their
information.
• VNX Total Protection Pack—Includes local, remote, and application protection
suites
• VNX Total Efficiency Pack—Includes the suites in the VNX Total Protection
Pack, along with the FAST and Security & Compliance suites
For additional details about the EMC VNX series of unified storage systems, refer to
the following white papers available at www.emc.com:
• Introduction to the New EMC VNX Series—VNX5200, VNX5400, VNX5600,

VNX5800, VNX7600, & VNX8000
• Introduction to the EMC VNX Series—VNX5100, VNX5300, VNX5500,
VNX5700, & VNX7500
VNX FAST VP
The FAST VP feature, which is included in the VNX FAST Suite, can lower the total cost
of ownership (TCO) and increase performance by intelligently managing data
placement according to activity level. When FAST VP is implemented, the storage
system measures, analyzes, and implements a dynamic storage-tiering policy much
faster and more efficiently than a human analyst could ever achieve.
Storage provisioning can be repetitive and time-consuming and, when estimates are
calculated incorrectly, it can produce uncertain results. It is not always obvious how
to match capacity to the performance requirements of a workload’s data. Even when a
match is achieved, requirements change and a storage system’s provisioning
requires constant adjustments.
Storage tiering allows a storage pool to use drives of varying levels of performance.
Logical unit numbers (LUNs) use the storage capacity needed from the pool on the

devices with the required performance characteristics. FAST VP uses I/O statistics at a
256 MB slice granularity (known as sub-LUN tiering). The relative activity level of each
slice is used to determine the need to promote to higher tiers of storage. Relocation is
initiated at the user’s discretion through either manual initiation or an automated
scheduler. FAST VP removes the need for manual, resource-intensive LUN migrations
while still providing the performance levels required by the most active dataset.
EMC Virtual Provisioning technology on VNX

EMC Virtual Provisioning™ technology on VNX series systems provides pool-based
storage provisioning by implementing pool LUNs that can be either thin or thick. Thin
LUNs provide on-demand storage that maximizes the utilization of your storage by
allocating it as it is needed. Thick LUNs provide high performance and predictable
performance for your applications. Both types of LUNs benefit from the ease-of-use
features of pool-based provisioning. Pools and pool LUNs are also the building blocks
for advanced data services such as FAST VP and compression.
Virtual Provisioning technology supports features such as hot sparing, proactive

sparing, and the ability to migrate data between thin LUNs, thick LUNs, or traditional
LUNs without incurring application downtime. The ability to nondisruptively migrate
data to different LUN and disk types provides the best solution for meeting your
changing application and business requirements without downtime. This flexibility
separates Virtual Provisioning from typical thin provisioning implementations.
EMC VNX5400 specifications

The EMC VNX5400 model is a member of the VNX series next-generation storage
platform, powered by two Intel quad-core Xeon E5-2600 1.8 GHz series processors
and a 6 Gb/s SAS drive back end, providing the industry’s highest bandwidth. The
EMC VNX5400 model is designed to deliver maximum performance and scalability for
enterprises. It is a converged platform that replaces the EMC CLARiiON® and EMC
Celerra® platforms and enables organizations to dynamically grow, share, and cost
effectively manage multiprotocol file systems and multiprotocol block storage access.
The VNX Operating Environment enables Microsoft Windows and Linux/UNIX clients
to share files in multiprotocol (NFS and CIFS) environments. At the same time, it
supports iSCSI, FC, and FCoE access for high-bandwidth and latency-sensitive block
applications.
Table 1 lists VNX5400 features for block storage. For additional VNX specifications for
both block and file storage, see EMC VNX Series Unified Storage Systems at
www.emc.com.
Table 1. VNX5400 features summary (shown for block storage only)
System feature Value/Description

Number of drives per array 4 minimum, 250 maximum
Drive types Flash, SAS, NL-SAS
DAE options 25 x 2.5-in. SAS/flash drives—2U

15 x 3.5-in. SAS/flash drives—3U
RAID options 0, 1, 3, 5, 6, 1/0

System feature Value/Description
Number of SPs 2
Block protocols supported FC, iSCSI, FCoE
CPU/memory per array 2 quad-core Intel Xeon E5-2600 @

1.8 GHz/32 GB (16 GB for each SP)
Maximum block UltraFlex I/O modules per 8

array
Maximum FAST Cache 1 TB
Maximum raw capacity 750 TB
Maximum number of SAN hosts 1,024
Maximum number of pools 15
Maximum number of LUNs (pool) 1,000
Maximum number of LUNs (classic) 2,048
Maximum number of ports per array 36
2/4/8 Gb/s FC maximum number of ports 32

per array
GbE iSCSI maximum number of ports per 16

array
10 GbE iSCSI maximum number of ports 16

per array
FCoE maximum number of ports per array 16
6 Gb/s SAS maximum number of buses for 2 x 4 lane

DAE connections per array

Solution description
This solution is intended for medium-to-large enterprises that are planning to deploy
and virtualize Microsoft Exchange Server 2013 on Microsoft Hyper-V and EMC
storage. It uses the Exchange Server 2013 Database Availability Group (DAG) feature
to provide mailbox resiliency and high availability for Exchange users. The solution
design represents a virtualized Exchange Server 2013 environment supporting
20,000 users in a mailbox resiliency configuration across two VNX5400 storage
arrays. The solution is designed to provide outstanding performance and flexibility for
today’s and tomorrow’s Exchange users.
This solution employs virtualization to effectively use hardware resources that benefit
the Exchange application. In this solution, the physical hypervisor server hosts
multiple Exchange mailbox server virtual machines.
In this solution, 20,000 users are distributed across eight mailbox servers in a DAG.
The DAG has two RAID-protected copies of every Exchange database—a primary
(active) copy and a secondary (passive) copy—that are evenly split between
hypervisor hosts and storage pools on two VNX5400 arrays. Each Exchange database
replicates to an alternate mailbox server that resides on a different Hyper-V host
through the use of the Exchange native DAG host-based log shipping mechanism.
Each mailbox server virtual machine is configured to support up to 5,000 users

(2,500 active and 2,500 passive) with a 2 GB mailbox capacity and 0.10 I/O
operations per second (IOPS) per user (including an additional 20 percent I/O
headroom). This user profile corresponds to about 150 messages per user per day.
The solution is designed to eliminate a single point of failure and can handle the loss
of an array, hypervisor server, mailbox server virtual machine, database volume, host
bus adapter (HBA), or switch. For example, during a mailbox server failure or
maintenance operation, each mailbox server can handle the compute and storage
requirements of all active and passive databases. Similarly, during a hypervisor host
failure or maintenance activities, another host can support all mailbox servers
(20,000 users with four virtual mailbox servers per hypervisor host).
Note: You can compare this solution architecture to one in which Exchange Server 2013 is
deployed in a stand-alone configuration (without DAG) and a single VNX5400 storage array
with eight mailbox servers provides service to all 20,000 users.

Figure 2 illustrates the architecture of this solution.
Figure 2. EMC VNX5400 storage solution architecture for 20,000 Exchange 2013 users
Storage design for Storage design is an important element for ensuring the successful deployment of
Exchange Server Microsoft Exchange Server 2013. Sizing and configuring storage for use with
2013 Exchange Server 2013 can be a complicated process driven by many variables and
requirements that vary from organization to organization. Properly configured
Exchange storage, combined with optimally sized server and network infrastructures,
can guarantee a smooth Exchange operation. Defining a unit of measure—a mailbox
server building block—is one method that simplifies the sizing and configuration of
large amounts of storage on the EMC VNX series storage arrays for use with Exchange
Server 2013.
A mailbox server building block represents the amount of disk, network, and server
resources required to support a specific number of Exchange Server 2013 users. The
amount of required resources is derived from a specific user profile type, mailbox
size, and availability requirements. Using the building block approach simplifies the
design and implementation of Exchange Server 2013.
Once the initial building block is designed, it can be easily reproduced to support the
required number of users in your enterprise. By using this approach, EMC customers
can now create their own building blocks that are based on their company’s specific
Exchange environment requirements. This approach is helpful when future growth is
expected because it makes Exchange environment expansion simple and
straightforward. EMC best practices involving the building block approach for
Exchange Server design have proven to be highly successful in many customer
implementations.

Building block The mailbox server building block used in this solution supports up to 5,000 users
used in this per server with a 2 GB mailbox size and 0.10 IOPS per user (0.12 IOPS were validated,
solution which reflects Microsoft's recommended overhead of 20 percent).
This building block is made up of 14 x 3 TB NL-SAS drives (12 for databases and 2 for
logs) and supports I/O and capacity for a single mailbox server, which in turn
supports 5,000 users (2,500 active and 2,500 passive).
The solution uses four of these building blocks (56 disks—48 disks for databases and
8 disks for logs) on the primary storage array to scale the configuration up to 20,000
users. The four building blocks are then duplicated on the secondary storage array.
Multiple building blocks are grouped together in storage pools. In this solution we
grouped two building blocks in a RAID 1/0 storage pool with 24 x 3 TB NL-SAS disks,
thus supporting database I/O and mailbox capacity for two mailbox server virtual
machines. We also added 2 x 100 GB flash disks to the pool for FAST VP to allow for
absorption of metadata. For transactional logs, we configured separate RAID 1/0
pools with four 3 TB NL-SAS disks, each supporting two building blocks. See Figure 3
on page 14 for more details.
Table 2 summarizes the attributes of the building block.
Table 2. Mailbox server building block attributes
Item Value/Description
Number of users supported by a 5,000 (2,500 active and 2,500 passive)
single mailbox server
User profile 0.10 IOPS with a 20% headroom (0.12 IOPS

tested)
Mailbox size 2 GB
Drive type and capacity 3 TB NL-SAS drives; 7,200 rpm; 100 GB flash
RAID type RAID 1/0
Database LUN size 1,300 GB
Log LUN size 55 GB
Number of disks to support database 14 (12 for databases, 2 for logs)

and logs for a single mailbox server
Mailbox server virtual machine 6 vCPUs, 92 GB RAM, 200 GB OS volume, 2 NICs

requirements (1 or 10 Gb/s Ethernet)*
* For mailbox server CPU and memory requirements, use the Exchange 2013 Server
Role Requirements Calculator on Microsoft TechNet.

VNX storage pools In this solution, we used thin-provisioned pool-based LUNs on the VNX5400 array for
and data the Exchange databases and logs.
protection
Using storage pools on VNX systems simplifies storage provisioning. Traditional
storage provisioning with only RAID groups restricts the number of disks in a group to
16. Storage pools, on the other hand, enable you to manage potentially hundreds of
disks at a time. Such pool-based provisioning provides benefits similar to metaLUN
striping across many drives, but, unlike metaLUNs, storage pools require minimal
planning and management effort. Storage pools support the same RAID protection
levels as RAID groups do: RAID 5, RAID 6, and RAID 1/0.
By using storage pools, you can take advantage of FAST VP, which can lower the total
cost of ownership (TCO) and increase performance by intelligently managing data
placement according to activity level. Also, using storage pools provides the
capability to use VNX Snapshots for data replication and protection. EMC AppSync®
uses VNX Snapshots replication technology to provide fully automated application-
consistent replicas of Exchange data. To learn more about protecting Exchange data
with EMC AppSync and VNX Snapshots, review the white paper titled Advanced
Protection for Microsoft Exchange 2010 on EMC VNX Storage available at
www.emc.com.
Figure 3 illustrates the VNX5400 storage solution using building block methodology,
designed to provide the best performance and sufficient capacity for virtualized
Exchange 2013 with 20,000 users at 2 GB mailbox capacity.
Figure 3. Exchange storage design using EMC building-block methodology

Mailbox server The ESRP—Storage program focuses on storage solution testing to address
scalability factors performance and reliability issues with storage design. However, storage is not the
only factor that must be considered when designing a scalable Exchange solution.
Other factors that can affect server scalability are:
• Server processor utilization
• Server physical and virtual memory limitations
• Resource requirements for other applications
• Directory and network service latencies
• Network infrastructure limitations
• Replication and recovery requirements
• Client usage profiles
Any combination of these factors can affect the total number of mailboxes supported
by a single mailbox server. Because all of these factors fall outside the scope of
ESRP—Storage program solution validation, a single mailbox server might not
necessarily support 5,000 users (as validated for the solution described in this
document) in every customer deployment.
Exchange Microsoft supports Exchange Server 2013 in production on hardware virtualization

virtualization software such as Microsoft Hyper-V, VMware vSphere, and any third-party hypervisor
that has been validated under the Windows Server Virtualization Validation Program.
If Exchange mailbox servers are deployed as virtual machines, the number of users
that can be supported by each mailbox server virtual machine depends on the
hypervisor type, physical server hardware capabilities such as CPU and memory
characteristics, and high availability requirements. For more information about
Exchange Server 2013 system requirements for hardware virtualization, refer to
Exchange 2013 Virtualization on Microsoft TechNet.
For more information on identifying and addressing performance bottlenecks in an
Exchange system, refer to Troubleshooting Microsoft Exchange Server Performance
on Microsoft TechNet.
Targeted customer This solution is designed for any medium-to-large enterprise that is planning to
profile consolidate and virtualize its Exchange Server 2013 environment, and requires a
highly reliable, scalable storage solution. The design of this solution, which supports
20,000 Exchange users, has the following characteristics:
• 20,000 active mailboxes
• 0.10 IOPS per user (150 messages sent and received per user/day, with an
additional 20 percent overhead—0.12 IOPS validated)
• 2 GB mailbox size
• Eight Exchange servers per DAG, each supporting 5,000 users (2,500 active
and 2,500 passive)
• Native DAG replication mechanism for mailbox resiliency and high availability
with two database copies maintained

• 24x7 background database maintenance (BDM) enabled
• Two EMC VNX5400 storage arrays (the solution was validated with four
mailbox servers connected to a single storage array)
Note: If you require more than two database copies, EMC recommends placing two RAID-
protected DAG copies on one storage array and placing a third RAID-protected copy on
another array.
Tested deployment Table 3 through Table 8 summarize the testing environment.
Table 3. Simulated Exchange configuration
Attribute Value
Number of Exchange mailboxes 20,000
Number of DAGs 1
Number of copies/database 2
Number of servers/DAG 8
Number of active mailboxes/server 5,000
Number of databases/mailbox server 10
Number of mailboxes/database 500
User profile 0.10 IOPS with a 20% headroom (0.12 IOPS

validated)
Database LUN size 1,300 GB
Log LUN size 55 GB
Total database size for performance testing 41,360 GB (40.4 TB)

(10,340 GB per mailbox server)
Percentage of storage capacity used by 79.5% (40.4 TB/50.8 TB)

Exchange database 4 Total database size/Total formatted capacity
4
Storage performance characteristics change based on the percentage utilization of the
individual disks. Tests that use a small percentage of the storage (~25 percent) might exhibit
reduced throughput if the storage capacity utilization is significantly increased beyond what is
reported in this paper.

Table 4. Storage hardware
Component Description
Storage connectivity FCoE
Storage model and OS/firmware EMC VNX5400, VNX Block Operating Environment
revision version 05.33.000.5.038
Windows Hardware Compatibility List
Storage cache 32 GB mirrored
Number of storage processors 2
Number of storage ports used 4 (2 per SP)
Maximum bandwidth of storage 20 Gb/s (2 x 10 Gb/s FCoE ports)

connectivity to host
Switch type/model/firmware Cisco Nexus 5548UP/NXOS 6.0(2)N2(3)

revision UCS Fabric Interconnect 6248/UCS Firmware
2.1(3b)
HBA model/firmware Cisco VIC 1240/Firmware 2.1(3b)
Number of HBAs per host 4 (2 for boot and 2 for virtual Fibre Channel)
Hypervisor server specification Cisco Systems Inc. UCSB-B200-M3 with 2 Intel

Xeon CPU E5-2660 v2 @ 2.20 GHz, 20 Core, 256
GB RAM
Number of disks used in this 56 (48 for databases and 8 for logs)
solution as validated
Maximum number of disks 250

supported by this array model
Table 5. Storage software
Component Description
HBA driver 2.3.0.20
HBA Queue Depth setting Default
Multipathing EMC PowerPath® (installed on Hyper-V host and

guest VM)
Hypervisor type Microsoft Windows Server 2012 R2 Hyper-V
Exchange Server virtual machine OS Windows Server 2012 R2 Datacenter (6.3.9600)
ESE.dll file version 15.00.0847.030
Replication solution name/version N/A

Table 6. Storage disk configuration (mailbox store disks) 5
Attribute Description
Disk type and speed 3 TB NL-SAS 7,200 rpm
Raw capacity for each disk 2,751 GB
Number of physical disks in test 48
Total raw storage capacity 132,048 GB (129 TB)
RAID level RAID 1/0
Total formatted capacity 52,000 GB (50.78 TB) (13 TB per server)
Storage capacity utilization 40.3% (50.78 TB/129 TB)

(percentage) Formatted capacity/total raw capacity
Database capacity utilization 31.3% (40.4 TB/129 TB)

(percentage) Total database size/total raw capacity
Table 7. Storage disk configuration (transactional log disks)
Attribute Description
Disk type and speed 3 TB NL-SAS 7,200 rpm
Raw capacity for each disk 2,751 GB
Number of physical disks in test 8
Total raw storage capacity 22,008 GB
RAID level RAID 1/0
Total formatted capacity 2,200 GB (550 GB per server)
Table 8. Replication Configuration
Replication mechanism Exchange 2013 DAG mailbox resiliency

Number of links 2
Simulated link distance LAN
Link type IP
Link bandwidth 10 Gb/s Ethernet (converged)
5
Capacity of drives, and therefore the raw data capacity of the disk array, based on binary
representation of 1KB = 1024 bytes. This means that 1 GB = 1,073,741,824 bytes.

Best practices
Microsoft Exchange Server 2013 has changed significantly since the early versions of
Exchange, particularly with regard to I/O and storage. Many changes have been made
to the core schema and the Extensible Storage Engine (ESE) to reduce the I/O
footprint. Due to this I/O reduction, Exchange Server 2013 can be deployed on
low-cost SATA and NL-SAS disks in addition to FC, SAS, and flash. Depending on the
Exchange Server 2013 deployment model you choose, DAG or stand-alone, storage
configuration options can vary from highly reliable SAN or SCSI infrastructure to
direct-attached storage (DAS), and even to non-RAID JBOD configurations in specific
circumstances. All RAID types are supported to meet the requirements of the high-
availability model and storage configuration you choose to deploy. EMC provides
storage options for all of these configurations.
• For Exchange Server 2013 storage design best practices, refer to Exchange
2013 Storage Configuration Options on Microsoft TechNet.
• For Exchange Server 2013 sizing and capacity planning, refer to Exchange
2013 Sizing and Capacity Planning on Microsoft TechNet.
In addition to following Microsoft’s recommendations, EMC recommends that you
follow the best practices described in this section to improve EMC VNX series storage
performance with Exchange Server 2013.
Hyper-V virtual In this solution we used virtual FC within the guest OS on the Hyper-V server. The
Fibre Channel virtual FC feature in Hyper-V on Windows Server 2012 and Windows Server 2012 R2
provides virtual FC ports within the guest operating system, which allows you to
connect to virtual FC directly from within virtual machines. Therefore, a virtual
machine can connect directly to a SAN through a virtualized FC adapter.
For more information, requirements, and best practices about Hyper-V virtual FC, refer
to Hyper-V Virtual Fibre Channel Overview on Microsoft TechNet.
Mailbox server Follow these recommendations to ensure the best possible mailbox server
virtual machines performance:
• Format Windows Server NTFS volumes to be used for Exchange databases and
logs with an allocation unit size of 64 KB.
Note: Partition alignment is no longer required when running Microsoft Windows

Server 2008/2008 R2 or Windows Server 2012/2012 R2, because partitions are
automatically set to a 1 MB alignment offset.
• Apply all required OS hotfixes recommended by Microsoft, EMC, the

hypervisor vendor, and the HBA vendor.
• Reserve CPU and memory resources required for each mailbox server virtual
machine.
• In virtual environments, install virtual machine tools to optimize performance.
• Configure two network adapters, one for the management network and one
for the DAG replication network. For DAG replication, 10 Gb/s virtual network
adapters are more appropriate.

Hypervisor host To configure the hypervisor host for optimal Exchange performance follow these
recommendations:
• In SAN environments, use redundant HBAs connected to different fabrics.
• Verify that the HBA installed in the server can support the IOPS requirements,
even in a failover situation. To avoid throttling, ensure that the queue depth is
set according to EMC recommendations for the specific HBA vendor and type.
• Install EMC PowerPath for optimal path management and maximum I/O
performance. For more information on installing and configuring PowerPath,
visit www.emc.com/storage/powerpath/powerpath-multipathing.htm.
• Depending on your backup solution, configure Exchange disks as virtual disks
or SCSI pass-through disks.
• Apply all required hypervisor OS hotfixes recommended by Microsoft, EMC,
the hypervisor vendor, and the HBA vendor.
Networking For high availability deployments that use DAGs, we recommend multiple physical
NICs connected to different networks to isolate user MAPI traffic from database
replication traffic.
For iSCSI deployments, we prefer multiple network switches for fault tolerance and
performance. Where this is not possible, use VLANs to isolate iSCSI traffic.
Core storage EMC VNX5400 is already optimized for Exchange Server 2013 workloads and requires
only a few minor adjustments. Follow these guidelines to ensure optimal storage
performance:
• Isolate the Microsoft Exchange server database workload from other
I/O-intensive applications or workloads (assign the Exchange workload to its
own set of disks). This ensures the highest level of performance for Exchange
and simplifies troubleshooting in the event of a disk-related performance
issue.
• For VNX models that preceded next-generation systems (VNX5100, VNX5300,
VNX5500, VNX 5700, VNX7500), set the VNX storage array page size
parameter to 16 KB, and allocate 1 GB of available VNX array SP memory to
read cache and the remaining available memory to write cache.
Note: On next-generation VNX systems such as the VNX5400 model, you do not
have to set the page size and the memory for read and write cache. These settings
are now managed automatically.
• Ensure that write cache is enabled for all database and log LUNs.
• Plan for performance even in a failover situation. Ensure SP utilization does
not exceed 70–75 percent.
• Ensure that I/O load and bandwidth are evenly balanced and distributed
across the VNX5400 front-end ports and back-end buses for failover and load
balancing.
• When sizing storage, always prioritize IOPS over capacity. After calculating
the IOPS requirements, assess the capacity that is required.

• After establishing the required IOPS, apply a 20 percent I/O overhead factor
to build in adequate reserve throughput capability.
• Consider additional throughput requirements imposed by background
database maintenance (BDM). Exchange 2013 BDM is approximately 2.25
MB/s per database.
• For most Exchange deployments, use storage pools made up of disks of the
same type with the appropriate RAID multiplier. In configurations with high
IOPS requirements, using multitiered pools (FAST VP) might be beneficial.
• Ensure that each DAG copy is isolated in a different storage pool.
• Isolate the databases from the logs on separate sets of disks for optimal
performance.
For additional information about configuring VNX storage for best performance,
review the white paper titled EMC VNX Unified Best Practices for Performance
available on www.emc.com.
For a complete list of EMC storage best practices for Microsoft Exchange, as well as
using EMC FAST VP, EMC FAST Cache, and EMC XtremCache™ technologies with
Exchange workloads, review the white paper titled Microsoft Exchange Server Best
Practices and Design Guidelines on EMC Storage available on www.emc.com.
Backup strategy When you are using intelligent VNX storage arrays for Exchange storage, we
recommend using EMC AppSync to take application-consistent Volume Shadow Copy
Service (VSS) snapshots to protect your Exchange data.
Information resources
For more information on EMC solutions for Microsoft Exchange Server, visit
www.emc.com/exchange.

Test results summary
This section provides a high-level summary of the ESRP test results for this solution.
Note: Detailed test result reports are attached to this PDF in HTML format.
We validated this solution using Microsoft Jetstress 2013 to ensure that the storage
design satisfies the disk I/O and capacity requirements for the target profile. Jetstress
simulates Exchange Server 2013 I/O types at the database level by interacting
directly with the database Extensible Storage Engine (ESE) technology (also known as
Jet) on which Exchange is built.
Jetstress can be configured to test the maximum I/O throughput available to the disk
subsystem within the performance constraints of Exchange. Alternatively, Jetstress
can be configured to accept a specific user profile (user count, IOPS per user) and
validate that the disk subsystem is capable of maintaining an acceptable level of
performance with such a profile. Both configuration options produce similar results.
Note: The Jetstress tool is designed to test performance of an Exchange storage subsystem
before the subsystem is placed in a production environment. Jetstress is not designed to
test server CPU, memory configuration, or the impact of user client activity. To test Exchange
server functionality and end-to-end deployment, it is a best practice to use Microsoft Load
Generator (LoadGen).
The documentation for Jetstress describes how to configure and execute I/O validation
or evaluation on your server hardware. The Microsoft Jetstress tool is available at
www.microsoft.com/en-us/download/details.aspx?id=36849.
Storage reliability The reliability (stress) test runs for 24 hours. The goal of this test is to validate that
(stress test) the storage can handle high I/O load for a long period of time. Both log and database
files are analyzed for integrity after the stress tests to ensure that no database or log
corruption occurs. The 24-hour stress test results reveal that:
• No errors were reported in the saved event log file.
• No errors were reported for the database and log checksum processes.
Storage The purpose of the primary storage performance test is to exercise the storage with
performance the maximum sustainable Exchange I/O for 2 hours with 100 percent user
concurrency. The test reveals the amount of time it takes the storage to respond to
I/O requests under load.
This section presents the sum of all database transactional I/O and the average I/O
latencies for all databases and logs recorded during the 2-hour test duration. Data for
each server is presented separately. Aggregates across servers are also included.
Individual server metrics

Table 9 presents the sum of the I/O across all databases and the average latencies
across all databases and logs for each server. The configuration is designed to
achieve a target of 600 IOPS for 5,000 users on each server. This includes 20 percent
overhead, above the 0.10 IOPS user profile, for reserve throughput capability.

Table 9. Individual server metrics
Database I/O EXCH01 EXCH02 EXCH03 EXCH04

Target transactional I/O 600 600 600 600
Achieved transactional I/O 685 682 685 680
I/O database reads/sec 472 469 472 468
I/O database writes/sec 213 213 213 212
I/O database average read latency (ms) 15.69 15.88 16 15.58
I/O database average write latency (ms) 1.72 2.07 1.81 1.76
Transaction log I/O EXCH01 EXCH02 EXCH03 EXCH04
I/O log writes/sec 54 54 54 55
I/O log average write latency (ms) 0.59 0.61 0.55 0.58
Performance across servers

Table 10 presents the sum of the I/O across servers and the average latency across
servers. The configuration is designed to achieve a target of 2,400 Exchange Server
2013 IOPS for 20,000 users. The results show excellent I/O performance with 2,732
Exchange Server 2013 user IOPS achieved. This provides an additional 332 IOPS
reserved throughput capacity to absorb any unexpected spikes during extremely
heavy user activities. Note that these results already include 20 percent IOPS
overhead recommended by Microsoft.
Table 10. Performance across servers
Database I/O Value

Target transactional IOPS 2,400
Achieved transactional IOPS 2,732
I/O database reads/sec 1,881
I/O database writes/sec 851
I/O database average read latency (ms) 15.79
I/O database average write latency (ms) 1.84

Transaction log I/O Value
I/O log writes/sec 217
I/O log average write latency (ms) 0.58
Database backup Two tests are reported in this category. The first test measures the sequential read
and recovery rate of database files, and the second test measures recovery/replay (the playing of
performance transaction logs against databases) performance.

Database read-only performance
The database read-only performance test measures the maximum rate at which
databases can be backed up through the use of VSS. Table 11 shows the average rate
of backing up a single database file across four servers.
Table 11. Database read-only performance
Metric Value
MBs read/sec per database 35.18
MBs read/sec per server 351.8
Total MBs read/sec for all four servers 1,407.2
Transaction log recovery/replay performance

The transaction log recovery/replay performance test measures the maximum rate at
which log files can be played against databases. Table 12 shows the average rate for
500 log files played against a single database. Each log file is 1 MB in size.
Table 12. Transaction log recovery/replay performance
Metric Value
Average time to play one log file (in seconds) 2.78
Detailed test results

Detailed test results for all four mailbox servers are attached to this PDF in the form of
Jetstress HTML reports. Report types are as follows:
• 24-hour stress test
• Checksum for 24-hour stress test
• 2-hour performance test
• Checksum for 2-hour performance test
• Backup test
• Soft recovery test
How to view Click the paper clip icon in the left-hand pane of Adobe Reader to reveal the list of
Jetstress reports Jetstress reports (HTML files). Double-click each file to open the corresponding report
in your browser.

Conclusion
This document is developed by EMC and reviewed by the Microsoft Exchange Product
team. The storage validation performed and the data presented in this document are
based on the tests introduced in the Microsoft ESRP test framework. The results were
reviewed by the Microsoft Exchange Product team. Customers should not quote the
data directly for their predeployment verification. It is still necessary to go through the
exercises to validate the storage design for each specific customer environment. The
ESRP program is not designed to be a benchmarking program; tests are not designed
to achieve the maximum throughput for a given solution. Rather, the program is
focused on producing recommendations from vendors for Exchange applications. The
data presented in this document should not be used for making direct comparisons
between various solutions.
The testing and validation of this Exchange Server 2013 mailbox resiliency storage
solution demonstrates the following:
• The EMC VNX5400 storage array is an excellent platform for Exchange Server
2013 storage. The array provides optimal performance with adequate
capacity for deploying large Exchange mailboxes.
• The VNX5400 array is fully optimized for Exchange Server 2013 workloads.
• A building-block approach simplifies mailbox server design and facilitates
scalable, predictable performance for all mailbox servers.
EMC recommends that you use a building-block approach when designing storage
solutions for Exchange Server 2013. This solution uses a building block of 5,000
users with 0.12 IOPS per user (including 20 percent overhead) and a 2 GB mailbox
size for validation.
The configuration is scaled to 20,000 users through the utilization of four building
blocks on an EMC VNX5400 storage array. This configuration meets all of the
recommended Microsoft Exchange Server 2013 metrics for performance and capacity.
In addition, the performance (I/O and latency) test results presented in this
Microsoft-approved ESRP document demonstrate that this solution provides
significant headroom to accommodate future user growth.
EMC periodically publishes Proven Solutions white papers that demonstrate that the
EMC VNX series of storage arrays can handle extremely heavy Exchange Server 2013
workloads. For more information, visit www.emc.com/exchange.
Contact EMC EMC recommends that you consult with EMC Professional Services to assist with the
design and deployment of a similar solution. For information regarding this or any
other EMC solution, use the following numbers:
• United States: (800) 782-4362 (SVC-4EMC)
• Canada: (800) 543-4782 (543-4SVC)
• Worldwide: (508) 497-7901
For additional information on EMC products and services available to customers and
partners, visit www.EMC.com or http://Powerlink.EMC.com.


EMC VNX 5400 Storage Solution

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EMC VNX 5400 Storage Solution

Uploaded by

Copyright:

Available Formats

Exchange Server 2013 Solution Reviewed Program (ESRP)

EMC VNX5400 UNIFIED STORAGE

EMC believes the information in this publication is accurate as of its

The information in this publication is provided as is. EMC Corporation makes no

Part Number H12997

EMC VNX5400 Unified Storage

Solution description ........................................................................................................................... 11

Information resources ........................................................................................................................ 21

Test results summary ......................................................................................................................... 22

EMC VNX5400 Unified Storage

EMC VNX5400 Unified Storage

EMC VNX5400 Unified Storage

Figure 1. EMC VNX series of unified storage platforms

EMC VNX5400 Unified Storage

• Security and Compliance Suite—Keeps data safe from changes, deletions,

• Local Protection Suite—Practices safe data protection and repurposing

• Remote Protection Suite—Protects data against localized failures, outages,

EMC VNX5400 Unified Storage

• Unisphere Management Suite—Monitors and manages multiple VNX systems

• Introduction to the New EMC VNX Series—VNX5200, VNX5400, VNX5600,

EMC VNX5400 Unified Storage

EMC Virtual Provisioning technology on VNX

Virtual Provisioning technology supports features such as hot sparing, proactive

EMC VNX5400 specifications

Table 1. VNX5400 features summary (shown for block storage only)

System feature Value/Description

Drive types Flash, SAS, NL-SAS

DAE options 25 x 2.5-in. SAS/flash drives—2U

RAID options 0, 1, 3, 5, 6, 1/0

EMC VNX5400 Unified Storage

Block protocols supported FC, iSCSI, FCoE

CPU/memory per array 2 quad-core Intel Xeon E5-2600 @

Maximum block UltraFlex I/O modules per 8

Maximum FAST Cache 1 TB

Maximum raw capacity 750 TB

Maximum number of SAN hosts 1,024

Maximum number of pools 15

Maximum number of LUNs (pool) 1,000

Maximum number of LUNs (classic) 2,048

Maximum number of ports per array 36

2/4/8 Gb/s FC maximum number of ports 32

GbE iSCSI maximum number of ports per 16

10 GbE iSCSI maximum number of ports 16

FCoE maximum number of ports per array 16

6 Gb/s SAS maximum number of buses for 2 x 4 lane

EMC VNX5400 Unified Storage

Each mailbox server virtual machine is configured to support up to 5,000 users

EMC VNX5400 Unified Storage

EMC VNX5400 Unified Storage

Table 2 summarizes the attributes of the building block.

Table 2. Mailbox server building block attributes

User profile 0.10 IOPS with a 20% headroom (0.12 IOPS

RAID type RAID 1/0

Database LUN size 1,300 GB

Log LUN size 55 GB

Number of disks to support database 14 (12 for databases, 2 for logs)

Mailbox server virtual machine 6 vCPUs, 92 GB RAM, 200 GB OS volume, 2 NICs

EMC VNX5400 Unified Storage

Figure 3. Exchange storage design using EMC building-block methodology

EMC VNX5400 Unified Storage

Exchange Microsoft supports Exchange Server 2013 in production on hardware virtualization

EMC VNX5400 Unified Storage

Tested deployment Table 3 through Table 8 summarize the testing environment.

Table 3. Simulated Exchange configuration