EasyTier For OracleASM - AIX

Performance benefits of IBM Storwize V7000 with IBM Easy Tier for Oracle ASM
Configurations and performance benefits
Mayur Shetty
IBM Systems and Technology Group ISV Enablement September 2011
Copyright IBM Corporation, 2011.
Table of contents
1 Abstract.................................................................................................................................. 1 2 Introduction ........................................................................................................................... 1
2.1 Overview ............................................................................................................................................ 1 2.2 Assumptions ...................................................................................................................................... 1 2.3 Intended audience ............................................................................................................................. 2
3 IBM Storwize V7000 technology stack ................................................................................. 2

3.1 IBM Storwize V7000 overview ........................................................................................................... 2 3.1.1 IBM Storwize V7000 functionalities ................................................................................................ 2 3.1.2 IBM Storwize V7000 terminology.................................................................................................... 4
4 Oracle 11g ASM overview ..................................................................................................... 5

4.1 ASM instance..................................................................................................................................... 5 4.2 ASM disk group.................................................................................................................................. 5 4.3 ASM disk............................................................................................................................................ 5 4.4 Allocation units................................................................................................................................... 6 4.5 ASM striping....................................................................................................................................... 6
5 IBM Storwize V7000 and Easy Tier ....................................................................................... 7

5.1 Easy Tier overview............................................................................................................................. 7 5.2 Easy Tier functionality........................................................................................................................ 7 5.3 Easy Tier mechanism to move data .................................................................................................. 8 5.3.1 Automatic mode ....................................................................................................... 8 5.3.2 Manual mode............................................................................................................ 8 5.4 Data relocation options with Easy Tier .............................................................................................. 8
6 Lab setup ............................................................................................................................. 10

6.1 Test environment ............................................................................................................................. 10 6.2 Storwize V7000................................................................................................................................ 10 6.4 Host nodes....................................................................................................................................... 10 6.5 Oracle configuration......................................................................................................................... 11
7 SwingBench Oracle load generator ................................................................................... 11

7.1 Overview .......................................................................................................................................... 11 7.2 OrderEntry benchmark .................................................................................................................... 12
8 IBM Storwize V7000 with Easy Tier configuration............................................................. 12

8.1 SAN configuration planning ............................................................................................................. 12 8.2 MDisk configuration ......................................................................................................................... 12 8.3 Volume configuration ....................................................................................................................... 20 8.4 Mapping volumes to the host........................................................................................................... 22
9 Test methodology................................................................................................................ 24
9.1 Prerequisite on the AIX host ............................................................................................................ 24 9.2 Creating an ASM disk group............................................................................................................ 25 9.3 Creating an Oracle database and schema ...................................................................................... 25 9.4 Easy Tier enabled manually to collect STAT data........................................................................... 25 9.5 Running the workload for STAT data............................................................................................... 27
Performance benefits of IBM Storwize V7000 with IBM Easy Tier for Oracle ASM Copyright IBM Corporation, 2011
9.6 Easy Tier analysis using the STAT utility ........................................................................................ 28 9.7 Adding SSD to the Storwize V7000 storage pool ............................................................................ 31 9.8 Migration of hot extent to SSD MDisk.............................................................................................. 38
10 Easy Tier test results......................................................................................................... 39

10.1 SwingBench workload without Easy Tier configured..................................................................... 39 10.2 SwingBench workload with Easy Tier configured using 1 x SSD.................................................. 40
11 Summary ............................................................................................................................ 42 Resources............................................................................................................................... 43 About the author..................................................................................................................... 43 Trademarks and special notices ........................................................................................... 44
Performance benefits of IBM Storwize V7000 with IBM Easy Tier for Oracle ASM Copyright IBM Corporation, 2011
1 Abstract
This paper demonstrates the performance benefits of running an Oracle 11g database using Oracle Automatic Storage Management (ASM) on IBM Storwize V7000 with IBM Easy Tier enabled. The paper also explains online transaction processing (OLTP) workload generated using the SysBench load generator tools.
2 Introduction
This section provides an overview on the various topics explained in this paper, the assumptions made, and the intended audience that are likely to benefit from this paper.
2.1 Overview
This paper demonstrates the performance benefits that IBM Easy Tier provides by seamlessly migrating hot extents from hard disk drives (HDDs) to a higher performing solid-state drives (SSDs) within the IBM Storwize V7000 solution. This might be either to internal SSD in the IBM Storwize V7000 system or to external storage systems that are virtualized by IBM Storwize V7000 systems. The other load generator tool that has been used here is Oracle Vdbench. The objective of Vdbench is to generate a wide variety of controlled storage I/O workloads, allowing control over workload parameters such as I/O rate, logical unit number (LUN) or file sizes, transfer sizes, thread count, volume count, volume skew, read/write ratios, read and write cache hit percentages, and random or sequential workloads. The other load generator used to arrive at the configuration guidelines is the Oracle I/O Calibration Tool ORION). This tool generates I/O using the same I/O software stack used by the Oracle server software without having to install the server software and create a database. It can simulate various workload types at different load levels to arrive at performance metrics for input/output operations per second (IOPS), and latency (response time). It can also simulate the effect of striping performed by ASM. The intention of this paper is not to demonstrate the maximum possible I/O benchmark or performance number for the IBM Storwize V7000 system. Those benchmark and performance numbers are likely to be shown in the Storage Performance Council SPC-1 and SPC-2 results posted by IBM on the SPC website. This paper demonstrates how to configure Easy Tier, and explains how Easy Tier might benefit the performance for an Oracle database workload by optimizing the utilization of SSDs.
2.2 Assumptions
The team assumes that the best practices that are documented in the IBM Storwize V7000 Introduction and Implementation Guide from IBM Redbooks holds good for Oracle databases too. Another assumption is that the load generation tools, Oracles Vdbench and ORION, are appropriate for determining storage performance for Oracle I/O workloads. A detailed description of both the tools is provided in a later section. But briefly, ORION generates I/O using the same I/O software stack used by the Oracle server software without having to install the server software. By using ORION, it is possible to avoid addressing the subject of Oracle database server tuning, which is beyond the scope of this paper. Also, Vdbench generates a wide variety of controlled storage I/O workloads, allowing control over workload parameters such as I/O rate, LUN or file sizes, transfer sizes, thread count, volume count,
Performance benefits of IBM Storwize V7000 with IBM Easy Tier for Oracle ASM Copyright IBM Corporation, 2011.
volume skew, read/write ratios, read and write cache hit percentages, and random or sequential workloads.
2.3 Intended audience

The intended audience of this paper is any technical lead, system administrator, storage administrator, or an Oracle database administrator (DBA) who is planning on deploying the IBM Storwize V7000 system for running Oracle database. This paper provides a starting point for Easy Tier configuration, as well as provides an idea on how Easy Tier optimally utilizes SSDs, and provides the performance benefit for an Oracle database workload.
3 IBM Storwize V7000 technology stack

This section includes concepts related to the IBM Storwize V7000 systems.
3.1 IBM Storwize V7000 overview

The IBM Storwize V7000 solution provides a modular storage system that includes the capability to virtualize external storage area network (SAN) attached storage as well as its own internal storage. IBM Storwize V7000 provides a number of preset configuration options that are aimed at simplifying the implementation process. It also provides an active-active solution, and is a clustered, scalable, midrange storage, as well as an external virtualization device. Included with IBM Storwize V7000 is a simple and easy to use graphical user interface (GUI) that allows storage to be deployed quickly and efficiently. The GUI runs on the IBM Storwize V7000 system and therefore, there is no need for a separate console. The IBM Storwize V7000 solution consists of a control enclosure which includes the chassis, node canisters, drives, and energy sources that include batteries, and expansion enclosures which is a hardware unit that includes expansion canisters, drives, and energy sources that do not include batteries. Within each enclosure are two canisters which are the hardware units that includes the node hardware, fabric and service interfaces, and serial-attached SCSI (SAS) expansion ports, determining the role of the enclosure
3.1.1 IBM Storwize V7000 functionalities

The following functionalities are available with the IBM Storwize V7000 system: Thin provisioning: With thin provisioning, applications can grow dynamically, but consume only the space that they are actually using. Without thin provisioning, pre-allocated space is reserved irrespective of whether the application uses it or not. Volume Mirroring: Volume Mirroring provides a single volume image to the attached host systems while maintaining pointers to two copies of data in separate storage pools. Copies can be on completely separate disk storage systems that are being virtualized. IBM Tivoli Storage FlashCopy Manager: FlashCopy creates instant application copies that can be used for backup or application testing. FlashCopy makes better use of the space with incremental copy, where only changed blocks are copied. Metro Mirror: Metro Mirror provides synchronous remote mirroring function up to approximately 300 km between sites. As the host I/O completes only after the data is cached at both locations,
performance requirements might limit the practical distance. Metro Mirror is designed to provide fully-synchronized copies at both sites with zero data loss after the initial copy is completed. Global Mirror: Global Mirror provides long distance asynchronous remote mirroring function up to approximately 8,000 km between sites. With Global Mirror, the host I/O completes locally, and the changed data is sent to the remote site later. This is designed to maintain a consistent, recoverable copy of data at the remote site which lags behind the local site. Data migration: A data migration wizard can be used to import external storage system into the IBM Storwize V7000 system. Easy Tier: Provides a mechanism to seamlessly migrate hot spots to a higher performing storage pool within the IBM Storwize V7000 solution. This can be to internal drives within the IBM Storwize V7000 system or to external storage systems that are virtualized by the IBM Storwize V7000 system.
Figure 1: IBM Storwize V7000 virtualizing external and internal storage
3.1.2 IBM Storwize V7000 terminology

Table 1 lists the Storwize V7000 terminology used in the paper. For an entire list of the Storwize V7000 terminology, refer to IBM Storwize V7000 Introduction and Implementation Guide at ibm.com/redbooks/redpieces/abstracts/sg247938.html?Open IBM Storwize V7000 term Control enclosure Definition A hardware unit that includes the chassis, node canisters, drives, and energy sources including batteries. A hardware unit that includes expansion canisters, drives, and energy sources that do not include batteries. A hardware unit that includes the node hardware, fabric and service interfaces, and SAS expansion ports. A controlling process in which hosts have access to only specific volumes within a cluster. Array-managed disks and drives that are held in enclosures and nodes that are part of the cluster. A component of a storage pool that is managed by a cluster. An MDisk is either a part of a RAID array of internal storage or a Small Computer System Interface (SCSI) logical unit (LU) for external storage. An MDisk is not visible to a host system on the SAN. A collection of storage capacity that provides the capacity requirements for a volume. The ability to define a storage unit (full system, storage pool, or volume) with a logical capacity size that is larger than the physical capacity assigned to that storage unit. A discrete unit of storage on disk, tape, or other data recording medium that supports some form of identifier and parameter list, such as a volume label or I/O control. Each MDisk is divided into segments of equal size called extents. When a volume is created from a storage pool, the volume is allocated based on the number of extents required to satisfy the capacity requirements for the volume.
Expansion enclosure
Node canister
Host mapping Internal storage Managed disk (MDisk)
Storage pool
Thin provisioning or thin provisioned
Volume
Extent
Table 1: IBM Storwize V7000 terminology
4 Oracle 11g ASM overview

ASM is a volume manager and a file system for Oracle database files that supports single-instance Oracle Database and Oracle Real Application Cluster (Oracle RAC) configurations. ASM is Oracle's recommended storage management solution that provides an alternative to conventional volume managers, file systems, and raw devices. ASM uses disk groups to store data files. An ASM disk group is a collection of disks that ASM manages as a unit. Within a disk group, ASM exposes a file system interface for Oracle database files. The content of files that are stored in a disk group are evenly distributed, or striped, to eliminate hot spots and to provide uniform performance across the disks. The performance is comparable to the performance of raw devices.
4.1 ASM instance

An ASM instance is built on the same technology as an Oracle Database instance. An ASM instance has a System Global Area (SGA) and background processes that are similar to those of Oracle Database. However, because ASM performs fewer tasks than a database, an ASM SGA is much smaller than a database SGA. In addition, ASM has a minimal performance effect on a server. ASM instances mount disk groups to make ASM files available to database instances; ASM instances do not mount databases. ASM keeps track of a files layout with metadata in an extent map. An extent map is a table that maps data extents in a file to allocation units on disk. ASM metadata is the information that ASM uses to control a disk group and the metadata resides within the disk group. ASM metadata includes the following information: The disks that belong to a disk group The amount of space that is available in a disk group The names of the files in a disk group The location of disk group, data file, and data extents A redo log that records information about atomically changing data blocks
4.2 ASM disk group

A disk group is a logical grouping of one or more disks that ASM manages as a collection. Each disk group contains the metadata associated with itself. ASM disk groups are similar to logical volumes in a typical SAN. Files are allocated from disk groups. Any ASM file is completely contained within a single disk group. However, a disk group might contain files belonging to several databases and a single database can use files from multiple disk groups. Disk group components include disks, files, and allocation units.
4.3 ASM disk

ASM disks are the storage devices that are provisioned to ASM disk groups. The ASM disk must be read and write accessible by the ASM owner from the server nodes. Examples of ASM disks include:
A disk or partition from a storage array An entire disk or the partitions of a disk Logical volumes Network-attached files (NFS)
When you add a disk to a disk group, you can assign a disk name or the disk is given an ASM disk name automatically. This name is different from the name used by the operating system. In a cluster, a disk can be assigned different operating system device names on different nodes, but the disk has the same ASM disk name on all of the nodes. In a cluster, an ASM disk must be accessible from all of the instances that share the disk group. If the disks are the same size, then ASM spreads the files evenly across all of the disks in the disk group. This allocation pattern maintains every disk at the same capacity level and ensures that all of the disks in a disk group have the same I/O load. Because ASM load balances among all of the disks in a disk group, different ASM disks should not share the same physical drive.
4.4 Allocation units

Within an ASM disk, space is divided into allocation units. An allocation unit is the fundamental unit of allocation within a disk group. A file extent consists of one or more allocation units. An ASM file consists of one or more file extents.
4.5 ASM striping

ASM striping has two primary purposes: Balance loads across all of the disks in a disk group Reduce I/O latency
Coarse-grained striping provides load balancing for disk groups while fine-grained striping reduces latency for certain file types by spreading the load more widely. To stripe data, ASM separates files into stripes and spreads data evenly across all of the disks in a disk group. The stripes are equal in size to the effective allocation unit. The coarse-grained stripe size is always equal to the allocation unit size. The fine-grained stripe size always equals 128 KB; this provides lower I/O latency for small I/O operations such as redo log writes.
5 IBM Storwize V7000 and Easy Tier

This section gives an overview of the Easy Tier feature in the Storwize V7000 and mechanism in which Easy Tier moves the data between the tiers in the storage pool.
5.1 Easy Tier overview

Easy Tier is designed to determine the appropriate tier of storage based on data access requirements, and then automatically and non-disruptively move data, at the sub-volume or sub-LUN level, to the appropriate tier. This feature is designed to reduce, if not eliminate, the amount of manual effort involved. In this new dynamic environment, data movement is seamless to the host application regardless of the storage tier in which the data resides.
Figure 2: Easy Tier overview
5.2 Easy Tier functionality

Easy Tier performs the following three functions: Workload monitoring: During the workload monitoring stage, Easy Tier determines if the data on the HDD tier needs to be promoted to the SDD tier, or whether the data on the SSD tier needs to be demoted to the HDD tier. Workload analysis: Easy Tier can provide information on data access patterns that can be used for SSD capacity planning. Smart data migration: Easy Tier seamlessly relocates data to the appropriate tier. The data movement is seamless to the host application regardless of the storage tier in which the data resides.
5.3 Easy Tier mechanism to move data

There are two different mechanisms based on which data can be moved between tiers in Easy Tier. They are: Automatic mode Manual mode
5.3.1 Automatic mode

In this mode, Easy Tier monitors the access to extents on a logical volume and, based on storage usage, automatically and nondisruptively relocates the hottest data (in the form of extents) to an appropriate storage device in a managed extent pool. A storage pool that is managed by Easy Tier requires at least one SSD MDisk and one non-SSD MDisk. You can also create a mixed extent pool that is managed by Easy Tier by: Creating a storage pool with SSD and HDD MDisks. Adding SSD MDisk to an existing HDD storage pool Adding HDD MDisk to an existing SSD storage pool Merging an existing SSD and an existing HDD storage pool
After a storage pool is formed, you can add additional MDisks to the managed storage pool or additional storage pools can be merged with this storage pool. Volumes that are created in managed storage pools are allocated on HDD extents based on the monitoring process and algorithms used by Easy Tier, which might later be migrated to SSDs.
5.3.2 Manual mode

In this mode, you can dynamically relocate a logical volume between extent pools or within an extent pool to change the extent allocation method of the volume or to redistribute the volume across new MDisks that have been added. This capability is referred to as dynamic volume relocation.
5.4 Data relocation options with Easy Tier

Initial implementation will have the following two tiers: Generic SSD which is tier 0 Generic HDD which is tier 1
Internal drives have tiers chosen by IBM Storwize V7000 system, while external (SAN) MDisks have a tier assigned by the user. Easy Tier between SSD and SAS or Fibre Channel (FC) drives have been tested, but nothing in the product disallows other combinations. For example, SAS to nearline SAS is allowed but will not be tested. Using Easy Tier with other disk configurations can lead to unpredictable performance and need to be tested before production implementation. Easy Tier is designed to work with IBM Storwize V7000 internal SSDs or SAN-based external SSDs.
Easy Tier is supported in both mirrored volumes and thin-provisioned volumes. In the case of mirrored volumes, both copies are independently managed by Easy Tier, while in thin-provisioned volumes only the real storage is subject to Easy Tier management. In the case of sequential volumes, where the extents are allocated one after another in one MDisk, they can be measured but not managed by Easy Tier. Also, in image mode volumes, which are special volumes having direct relationship with one MDisk and are used to migrate existing data in and out of the V7000 cluster, the volumes can be measured but not managed by Easy Tier. Easy Tier is also supported for FlashCopy source and target volumes. Metro Mirror and Global Mirror primary and secondary volumes are also supported with Easy Tier, but the Metro Mirror and Global Mirror volumes do not inherit characteristics of the primary as part of the disaster recovery activity.
6 Lab setup
This section lists the hardware and software configurations that were used in the lab exercises.
6.1 Test environment

Figure 3 shows the Oracle database server, the Oracle database client, and the Storwize V7000 used in the test environment.
(Oracle 11g R2 DB Server)

PowerPC_POWER7 Processor 2 x 3.3 GHz 16 GB RAM
(Oracle 11g R2 Client)

PowerPC_POWER7 Processor 2.66GHZ 12core 30GB RAM
ControllerA 8GB RAM
ControllerB 8GB RAM 300GB vol02
Storwize V7000
RAID5 Mdisk 1
300GB vol01
300GB vol03 RAID5 Mdisk 2
300GB vol04
ASM Diskgroup ora_asm ( vol01,vol02,vol03,vol04) External Drive HDD DB : ASM Diskgroup B HDD Drive SSD Drive
RAID5 Mdisk Storage Pool
SSD 300GBx1 Easy Tier
Figure 3: Database server, database client, and the IBM Storwize V7000 storage used for the test
6.2 Storwize V7000

The following table lists the Storwize V7000 used in the exercise. Storage name Node model Mico code level isv7k1.storage Storwize V7000 6.1.0.7 (build 25.3.1103030000)
6.4 Host nodes

The following table lists the AIX host that were used in the exercise Server type Processor Memory Operating system IBM pSeries IBM PowerPC POWER7 16 GB 7.1
10
6.5 Oracle configuration

The following table shows the Oracle configuration used in the exercise Server version 11.2.0.1.0
7 SwingBench Oracle load generator

This section talks about the usage of the SwingBench Oracle load generator.
7.1 Overview
SwingBench is a free load generator (and benchmarks) and is designed to stress test an Oracle database (9i,10g,11g). It can be downloaded from the URL: http://www.dominicgiles.com/downloads.html SwingBench consists of a load generator, a coordinator and a cluster overview. The software enables a load to be generated and the transactions/response times to be charted. SwingBench can be used to demonstrate and test technologies such as RACs, online table rebuilds, standby databases, online backup and recovery, and so on. The code that is shipped with SwingBench includes benchmark OrderEntry, which was used in the test. OrderEntry is based on the oe schema that is shipped with Oracle11g. It has been modified so that Spatial, Intermedia schemas do not need to be installed. It can be run continuously (that is, until you run out of space). It introduces heavy contention on a small number of tables and is designed to stress interconnects and memory. It is installed using the oewizard located in the bin directory. SwingBench has three front-ends to the same kernel: SwingBench This front-end has a rich graphical interface. It displays real-time charts and allows you to edit configuration parameters. A summary of the results are placed in an XML file. It is useful for demonstrations. Minibench This front-end has a small footprint graphical interface. It has some simple real-time charting. A summary of the results are placed in an XML file. It is useful for assessing the impact of an operation, such as a backup. Charbench This has a character-based front-end. The output can be in a vmstat/sar format. There are many command line options. Charbench allows for the gathering of very detailed real-time data when doing benchmarking that fully stresses the system.
11
7.2 OrderEntry benchmark

SwingBench comes with the OrderEntry benchmarks that were used in the tests.
Benchmark name Order Entry
Description Models the classic order entry stress test. It has a similar profile to the TPC-C benchmark. This version models an online order entry system with users being required to log in before purchasing goods. The benchmark uses the following three files:
names.txt nls.txt productids.txt
Profile
Select 50% Insert 30% Update 20% Delete 0%
These contain sample data used by each of the transactions. The location of each file is specified in the swingconfig.xml file in the EnvironmentVariable section.
8 IBM Storwize V7000 with Easy Tier configuration

This section explains the configuration of various components that were involved in testing. It includes: SAN configuration, MDisks configuration, storage pools, volumes, and hosts.
8.1 SAN configuration planning

For the best practices on SAN configuration planning, refer to the IBM Storwize V7000 Introduction and Implementation Guide from IBM Redbooks.
8.2 MDisk configuration

In the Storwize V7000 system, the internal drives cannot be directly added to storage pools. They need to be included in a Redundant Array of Independent Disks (RAID) to provide protection against the failure of individual drives. A RAID array is created as an MDisk, and during the array creation, wizards and presets are available to suggest configurations to users based on the hardware attached to the system. The recommendation is to use these presets for easy configuration and best performance.
12
The following content shows how to create arrays on the system using the GUI. The Storwize V7000 command line interface (CLI) might not be needed though it is flexible in most cases. When you first set up the Storwize V7000 system, the initial setup wizard prompts the user to use the fully-automated RAID setup. Now, start from the Getting Started page on the GUI, as shown in Figure 4.
Figure 4: Getting started with the storage configuration
Figure 5 shows the internal drives with an Online status before the MDisk configuration.
13
Figure 5: All internal drive storage
You can choose to use the recommended (fully-automatic) configuration or a different configuration for customization. (Refer to Figure 6). In this example, a different configuration has been used. In this step, first select the drive class. The GUI then provides a preset list based on the drive class. You can decide whether to configure spares automatically, and optimize the array for performance or capacity. You can also indicate the number of drives to be provisioned. The configuration summary at the bottom of the page (in Figure 6) displays information on what the system will attempt to do, or indicates that it cannot meet the RAID creation requirement. The team configured the internal SAS drives, and chose the preset of RAID-5. In this example, the team used 18 drives, optimized for a capacity to create 3 x RAID-5 arrays.
14
Figure 6: HDD MDisk creation
With the Optimize for Performance option selected, all arrays in a pool will have the same performance characteristics, and potentially leave unused drives in the system. The Optimize for Capacity option uses all drives in the system, which means that some of the arrays might not meet the array width goals. It will do this by reducing the number of drives in one or more arrays to be created by an equal amount, and you will get an optimal performance layout (all arrays have the same width, or the width with one drive more or one drive less, but the width will not be the same as the width goal). Note: If the perfect number of drives exists, the Optimize for Capacity option creates the same layout as the Optimize for Performance option. Both the algorithms are explained in the IBM Redbooks, Implementing the IBM Storwize V7000 at ibm.com/redbooks/redpieces/abstracts/sg247938.html?Open Next, you need to decide whether to expand an exiting pool or create one or more new pools with the arrays. In this example, a new pool dedicated for internal storage (as shown in Figure 7) has been created.
15
Figure 7: Creating a new storage pool
Now, you will see the new storage pool creation progress and the command line result, if the Details option is turned on (as shown in Figure 8).
Figure 8: Details of the RAID arrays creation
16
Next, the test team created another RAID-5 array as shown in Figure 9.
Figure 9: Creating another MDisk
The team then added this to the existing asm_oracle storage pool, as shown in Figure 10.
17
Figure 10: Adding the MDisk to the existing storage pool
Figure 11 shows the actual command that gets executed to create the RAID5 array, and add it to the storage pool oracle_asm.
18
Figure 11: Actual command that runs in the background
On the Pools page, you can notice that a new oracle_asm pool has been created, and it has four members mdisk0, mdisk1, mdisk2 and mdisk3, all having an Online status (as shown in Figure 12).
Figure 12: Storage pools and arrays status
You have now completed managing the internal HDDs. Next, you will see how to manage the internal
SSDs.
19
8.3 Volume configuration

A volume is a logical disk that is presented to a host system by the IBM Storwize V7000 system. The IBM Storwize V7000 storage system translates this volume into a number of extents which are allocated across MDisks present in the storage pool. On the left pane, click Volumes by Pool on the pop-up menu and then click New Volume to launch the New Volume wizard (as shown in the Figure 13). Then select the storage pool from which you want to create volumes.
Figure 13: New Volume wizard
20
Enter the volume name and the size of the volume that you need to create. Click the + sign to create additional volumes, as shown in Figure 14.
Figure 14: Volume creation using the wizard
21
Figure 15 shows detailed messages that are displayed when you click Create and Map to Host in the new volume creation wizard.
Figure 15: Background command to create the volumes
8.4 Mapping volumes to the host

A host system is connected to the Storwize V7000 system through either a Fibre Channel connection or an iSCSI connection. For this paper, the team used Fiber Channel to connect host to the IBM Storwize V7000 system. Hosts are defined to IBM Storwize V7000 by identifying their WWPNs for Fibre Channel hosts. After viewing the details, when you click Continue, the Modify Mappings dialog box (as shown in Figure 17) is displayed. In this example, the volumes are mapped to the isvx7 host, which is selected from the list.
22
Figure 16: Mapping volumes to host isvx7
Click Next to navigate to the Modify Mappings screen, shown in Figure 17.
Figure 17: Mapped volumes
On clicking OK, you can see that the volumes are mapped to the host, as shown in Figure 18.
23
Figure 18: Background command for host mappings
9 Test methodology
This section explains the test methodology followed in this paper.
9.1 Prerequisite on the AIX host

On the node running AIX, detect and configure the new device. Run lspv to list the physical disks already configured on the system. isvp14_ora> lspv To configure the new devices that the team created on the Storwize V7000 system and mapped to the isvp14_ora host, run the cfgmrg command. isvp14_ora> cfgmgr Run lspv again and you can see the entry for the new disks in the output. The following command changes an available disk (hdisk13, hdisk14, hdisk15, and hdisk16) to a physical volume by assigning it a physical volume identifier (PVID), if it does not already have one. isvp14_ora> chdev -l hdisk13 -a pv=yes isvp14_ora> chdev -l hdisk14 -a pv=yes isvp14_ora> chdev -l hdisk15 -a pv=yes isvp14_ora> chdev -l hdisk16 -a pv=yes
24
Change the owner and group of the devices to oracle and dba as shown in the following example. isvp14_ora> chown oracle:dba /dev/rhdisk13 isvp14_ora> chown oracle:dba /dev/rhdisk14 isvp14_ora> chown oracle:dba /dev/rhdisk15 isvp14_ora> chown oracle:dba /dev/rhdisk16
The new LUNs are now available to create an ASM disk group.
9.2 Creating an ASM disk group

Using Oracle asmca tool, the test team created an Oracle ASM disk group +DATA using the 4 x 300GB LUNs that the team presented to the AIX host. For the +DATA disk group, the team used external redundancy.
9.3 Creating an Oracle database and schema

The test team then created an Oracle database using the General Purpose or Transactional Processing template in the Oracle dbca tool. In the dbca tool, the team specified ASM as the storage type for the database and used Oracle managed files with +DATA for the database area. Next, the team created the Order Entry schema using the oewizard tool from the SwingBench kit. The Order Entry schema creation involved the creation of the soe user, tables, indexes, data, and so on. For this exercise, the team selected a scale factor of 100 that used 100 GB of raw data plus the space required for indexes. A scale factor of 100 created an Order Entry table space of 320 GB in size, and temporary table space size of 60 GB
9.4 Easy Tier enabled manually to collect STAT data

The oracle_asm storage pool currently has HDD MDisks in it, so by default it does not have Easy Tier measurements enabled to collect the data the STAT utility requires. Therefore, this must be manually enabled. To activate the Easy Tier function on the oracle_asm storage pool, the following command has been issued from the Storwize V7000 CLI.
Figure 19: Command to manually enable Easy Tier
Using the Storwize V7000 GUI to view the oracle_asm storage pool confirms that Easy Tier has been activated and that it is now ready to produce the statistics that will help in determining how much data need to be promoted to SSDs, and how much SSD capacity is required.
25
Figure 20: Storage pool with Easy Tier active
Figure 21 shows Easy Tier status for the vol01 volume that is Measured. Also, it shows that the entire capacity of 300 GB is on HDDs, while the SSDs have 0 GB of the volume.
26
Figure 21: Easy Tier status is measured
9.5 Running the workload for STAT data

From the Oracle client machine, the SwingBench workload with 400 users was run using the charbench client. bash-3.2$ ./charbench -c swingconfig.xml -a -v users,tpm,tps,resp > results_400users_aix.txt 2> swing.err The resultant file results_400users_aix.txt includes the transaction per minute, transaction per second, and the response time. At the end of the 24 hours, a heat data file on the Storwize V7000 reported the amount of hot extents that might benefit from being promoted to the SSD MDisk. As shown in Figure 22, the file is located using the GUI on the Support page and then by clicking Show full log listing.
27
Figure 22: Heat data file
Download the data file and save it on the Microsoft Windows machine where the STAT tool has been installed.
9.6 Easy Tier analysis using the STAT utility

The IBM Storage Tier Advisor Tool is a Microsoft Windows application that analyzes heat data files produced by Easy Tier and produces a graphical display of the amount of hot data per volume. In addition, it predicts how adding SSD capacity to the measured storage pool could benefit system performance. Heat data files are produced approximately one time a day when Easy Tier is active on one or more storage pools and updates the activity per volume since the prior heat data file was produced. This heat information is added to a running tally that will reflect the heat activity to-date for the measured pools and volumes. On IBM System Storage SAN Volume Controller and Storwize V7000, the heat data file is in the /dumps directory on the configuration node and is named dpa_heat.node_name.time_stamp.data.
28
Any existing heat data file is overwritten whenever a new heat data file is produced; however, it includes all the information till date for all measured pools and volumes. The file must be off-loaded by the user and processed with the STAT utility and invoked from a Windows command prompt. The user also has the option of specifying the output directory. The STAT utility creates a set of HTML files and the user can then open the resulting index.html file in a browser to view the results. You can download SAN Volume Controller / Storwize V7000 Storage Tier Advisor Tool at: ibm.com/support/docview.wss?uid=ssg1S4000935 Installing the STAT utility places the program files, by default, in the Program Files\IBM\STAT directory on the local C:\ drive. The Storwize V7000 system produces a heat file that has to be downloaded to the directory containing the STAT executable. When you right-click the file, a dialog box with the option to download and save the file, as shown in Figure 23 is displayed.
Figure 23: Downloading the heat data file
When the heat file is in the correct directory, run the STAT.exe to generate the STAT HTML reports, as shown in Figure 24.
29
Figure 24: Generating the HTML report
Five HTML reports will be recorded in the Data Files directory of the STAT folder under Program Files\IBM. Reports as shown in Figure 25, Figure 26, and were obtained on running STAT.exe against the Storwize V7000 heat file from the previous testing with an AIX system. The System Summary HTML report shows that 132 GB of the AIX volume had enough I/O operations on specific extents to merit their promotion to SSDs. It also recommends the SSD configuration and estimated performance improvement.
Figure 25 System Summary HTML report
30
Figure 26: Volume Heat Distribution HTML report
Figure 27: Storage Pool Recommendation HTML report
9.7 Adding SSD to the Storwize V7000 storage pool

Based on the Storage Tier Advisor Tool recommendations, an SSD RAID-10 MDisk was added to the oracle_asm storage pool using the graphic user interface (GUI). Figure28 shows all the inter SSDs available in our test system.
Figure 28: Internal SSDs
31
Click on the Configure Storage button to create the SSD MDisk. The Configure Internal Storage window will pop-up as shown in the Figure 29 below.
Figure 29: Configure Internal Storage pop up window
Click on the Select a different configuration radio button to select a preset configuration as shown below in Figure 30.
32
Figure 30: Step 1 of Configuring the Internal Storage
Select SSD Easy Tier from the Preset drop down menu as shown in Figure 31.We used 2 x SSD drives with RAID 10 FOR the new MDisk, and then clicked the Next button
33
Figure 31: SSD Easy Tier preset
In Step 2 we add the newly created SSD mdisk to an existing storage pool oracle_asm as shown below in the Figure 32.
34
Figure 32: Adding the new SSD mdisk to an existing storage pool
The Figure 33 below shows the actual command that gets executed to add the newly created SSD mdisk to the existing storage pool oracle_asm.
35
Figure 33: Command to add the newly created mdisk to an existing storage pool
The Figure 34 below shows the newly created SSD MDisk mdisk7
Figure 34: Newly created SSD MDisk
36
The Figure 35 below shows the storage pool oracle_asm with the SSD MDisk, and with Easy Tier Active.
Figure 35: Storage Pool with Easy Tier Active
Move the mouse pointer over a volume name, in this case, vol01. Then right-click and click Properties. This shows the details of the volume in the three tabs: Overview, Host Maps, and Member MDisks. Under the Member MDisk tab, in the # of Extents column for vol01, it shows 300 for mdisk0, 300 for mdisk1, 300 for mdisk2, and 300 for mdisk3. The reason for this is that the size of the vol01 volume is 300 GB, and the extent size of the oracle_asm storage pool is 256 MB. Therefore, 256x300 = 76.8 GB, which is the size of vol01 on mdisk0. There are four HDD MDisks in the storage pool, and therefore 76.8x4 = 307 GB, which is roughly the size of the vol01 volume that the team had created. You will see later that as the test progresses, Easy Tier will analyze the I/O pattern and develops a migration plan for the hot extents in each of the HDD Mdisks. During the extent migration phase, you will see three more SSD MDisks in the member MDisk screen. The number under the # of Extents column starts reducing as the hot extents migrate from the HDD MDisks to the SSD MDisks, until all the SSD extents have been used. The screen shot below show the extent distribution for each of the volumes before the workload was started from the Oracle client.
IBM_2076:ISV7K1:admin>svcinfo lsvdiskextent vol01 id number_extents 3 300 4 300 5 300 6 300 IBM_2076:ISV7K1:admin>svcinfo lsvdiskextent vol02 id number_extents 3 300 4 300
37
5 300 6 300 IBM_2076:ISV7K1:admin>svcinfo lsvdiskextent vol03 id number_extents 3 300 4 300 5 300 6 300 IBM_2076:ISV7K1:admin>svcinfo lsvdiskextent vol04 id number_extents 3 300 4 300 5 300 6 300 IBM_2076:ISV7K1:admin>
9.8 Migration of hot extent to SSD MDisk

Depending on where in the Easy Tier measurement collection cycle the SSDs were added to the storage pool, it might take several hours before the hot extents are migrated to the SSD backed MDisks in the pool and another heat data file is generated. The SwingBench workload is run using the charbench client during the time the hot extents are migrated from the HDD to the SSD MDisk. At the end of the charbench run, the new Easy Tier data file can be downloaded and the report generated using the STAT tool. The screen shot below shows the extent distribution of each of the volumes after running the workload for a period for 48 hours. You can see that the hot extents of the volumes have been migrated to the SSD mdisk7.
IBM_2076:ISV7K1:admin>svcinfo id number_extents 3 243 4 245 5 242 6 242 7 228 IBM_2076:ISV7K1:admin>svcinfo id number_extents 3 244 4 243 5 242 6 244 7 227 IBM_2076:ISV7K1:admin>svcinfo id number_extents 3 243 4 242 5 245 6 244 7 226 IBM_2076:ISV7K1:admin>svcinfo id number_extents 3 245 4 243 5 244 6 243 7 225 IBM_2076:ISV7K1:admin> lsvdiskextent vol01
lsvdiskextent vol02
lsvdiskextent vol03
lsvdiskextent vol04
38
10 Easy Tier test results

This section describes the results from the tests that were performed with and without Easy Tier enabled on the IBM Storwize V7000 system.
10.1 SwingBench workload without Easy Tier configured

The following points describe the storage setup, the workload generated, and the results that were observed. For this test, four volumes were created on a storage pool that consisted of four HDD MDisks. Each MDisk was 1.4 TB in size and configured using RAID5 with (5+1) x 300 GB drives. The stripe size of the storage pool was 256 MB. The workload was generated by charbench client with 400 concurrent users. The test was run for a duration of 24 hours. The Think Time for the test was between 50 milliseconds to 100 milliseconds. As shown in Figure 36, it has been observed that the team got approximately around 35000 transactions per minute (TPM). The response time was around 100 milliseconds to 115 milliseconds.
Figure 36: Transactions per minute with no Easy Tier active
39
Figure 37: Response time graph with no Easy Tier active
10.2 SwingBench workload with Easy Tier configured using 1 x SSD

The following points describe the storage setup, the workload generated, and the results that were observed. For this test, four volumes were created on a storage pool consisting of four HDD MDisks and one SSD MDisk. Each HDD MDisk was 1.4 TB in size, and configured using RAID 5 with (5+1) x 300 GB drives. The SSD MDisk was 300 GB in size and configured using RAID 10 with 2 x 300 GB drives. The stripe size of the storage pool was 256 MB. The workload generated by charbench client with 400 clients. The test was run for a duration of 24 hours. The Think Time for the test was between 50 milliseconds to 100 milliseconds. As shown in Figure 38, it has been observed that initially the team got approximately around 33000 TPM when all the extents of the volumes were on the HDD MDisks. The team then observed the extent migration from the HDD to the SSD MDisk. During the extent migration the TPM was gradually increasing. After the completion of the extent migration to the SSDs, the test team observed around 40000 TPM. The response time initially was around 125 milliseconds to 130 milliseconds when all the extents of the volumes were on HDD MDisks. With the migration of the hot extents from the HDD to the SDD Mdisk, the response time started decreasing. At the end of the test run of 24 hours, the team observed that the response time was around 17 milliseconds.
40
Figure 38: Transactions per minute with Easy Tier active
Figure 39: Response time graph with Easy Tier active
41
11 Summary
Oracle ASM data distribution policy is capacity based ie. it spreads data evenly across all of the volumes in an ASM disk group, and maintains every volume at the same capacity level. In a real world scenario I/O activity on the volumes of an ASM disk group are different, leading to I/O hot spots. The Easy Tier feature in the Storwize V7000 removes these hot spots in an Oracle ASM disk group automatically by migrating the hot extents from the HDD drives to faster SSD drives, and thus providing higher transactions per minute and lower response time to Oracle clients. In the paper the team observed the ease with which Easy Tier can be configured and deployed for use with Oracle ASM. We also observed that with the Storage Tier Advisor Tool, the users are now able to accurately provision SSDs accurately for their Oracle workloads. From our observations we can conclude that the Easy Tier feature of the Storwize V7000 system along with Oracle ASM provides excellent cost-performance benefits to Oracle database customers by placing only the hot data onto SSDs while the rest of the data is still on lower cost HDD drives.
42
Resources
The following websites provide useful references to supplement the information contained in this paper: Performance benefits of IBM Storwize V7000 with Easy Tier for Oracle 11g workload ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101838 Deploying Oracle 11g RAC Release 2 with IBM Storwize V7000 on Red Hat Enterprise Linux ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101772 Using the Storage Tier Advisory Tool (STAT) and Easy Tier on the IBM Storwize V7000 ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101852 IBM SAN Volume Controller Performance Configuration Guidelines for Implementing Oracle Databases with Automatic Storage Management (ASM) ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101481 IBM Business Partner support and resources ibm.com/partnerworld/pwhome.nsf/weblook/index_us.html IBM Publications Center www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi?CTY=US IBM Redbooks ibm.com/redbooks IBM developerWorks ibm.com/developerWorks
About the author

Mayur Shetty is a consultant in IBM Systems and Technology Group ISV Enablement Organization. He has more than 10 years experience working with the Enterprise Server and Storage products, and Oracle database.
43
Trademarks and special notices

Copyright IBM Corporation 2011. All rights Reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml. UNIX is a registered trademark of The Open Group in the United States and other countries. DIA Data Integrity Assurance, Storewiz, Storwize, and the Storwize logo are trademarks or registered trademarks of Storwize, Inc., an IBM Company. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be
44
given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.
45

EasyTier For OracleASM - AIX

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EasyTier For OracleASM - AIX

Uploaded by

Copyright:

Available Formats

Performance benefits of IBM Storwize V7000 with IBM Easy Tier for Oracle ASM

Configurations and performance benefits

IBM Systems and Technology Group ISV Enablement September 2011

Copyright IBM Corporation, 2011.

3 IBM Storwize V7000 technology stack ................................................................................. 2

4 Oracle 11g ASM overview ..................................................................................................... 5

5 IBM Storwize V7000 and Easy Tier ....................................................................................... 7

6 Lab setup ............................................................................................................................. 10

7 SwingBench Oracle load generator ................................................................................... 11

8 IBM Storwize V7000 with Easy Tier configuration............................................................. 12

10 Easy Tier test results......................................................................................................... 39

2.3 Intended audience

3 IBM Storwize V7000 technology stack

3.1 IBM Storwize V7000 overview

3.1.1 IBM Storwize V7000 functionalities

Figure 1: IBM Storwize V7000 virtualizing external and internal storage

3.1.2 IBM Storwize V7000 terminology

Host mapping Internal storage Managed disk (MDisk)

Thin provisioning or thin provisioned

Table 1: IBM Storwize V7000 terminology

4 Oracle 11g ASM overview

4.1 ASM instance

4.2 ASM disk group

4.3 ASM disk

4.4 Allocation units

4.5 ASM striping

5 IBM Storwize V7000 and Easy Tier

5.1 Easy Tier overview

Figure 2: Easy Tier overview

5.2 Easy Tier functionality

5.3 Easy Tier mechanism to move data

5.3.1 Automatic mode

5.3.2 Manual mode

5.4 Data relocation options with Easy Tier

6.1 Test environment

(Oracle 11g R2 DB Server)

(Oracle 11g R2 Client)

ControllerA 8GB RAM

ControllerB 8GB RAM 300GB vol02

300GB vol03 RAID5 Mdisk 2

RAID5 Mdisk Storage Pool

SSD 300GBx1 Easy Tier

6.2 Storwize V7000

6.4 Host nodes

6.5 Oracle configuration

7 SwingBench Oracle load generator

7.2 OrderEntry benchmark

Benchmark name Order Entry

8 IBM Storwize V7000 with Easy Tier configuration

8.1 SAN configuration planning

8.2 MDisk configuration

Figure 4: Getting started with the storage configuration

Figure 5: All internal drive storage

Figure 6: HDD MDisk creation

Figure 7: Creating a new storage pool

Figure 8: Details of the RAID arrays creation

Figure 9: Creating another MDisk

Figure 10: Adding the MDisk to the existing storage pool

Figure 11: Actual command that runs in the background

Figure 12: Storage pools and arrays status

8.3 Volume configuration

Figure 13: New Volume wizard

Figure 14: Volume creation using the wizard