Professional Documents
Culture Documents
Do you want to set up a redundant environment for high availability but don't know how AIX can
help you? Discover PowerHA (formerly HACMP) and gain a cheat sheet on how to configure
and set up a simple two-node cluster.
There are some types of computing environments in which you can't afford downtime—the
applications and data are so important that if one machine dies, you want another to be able to
pick up and immediately take over. Fortunately, in IBM® AIX®, a special piece of software called
PowerHA can provide redundancy and high availability to meet these needs. This article provides
an introduction to PowerHA and shows how to set up and configure a simple two-node cluster.
PowerHA at work
PowerHA is designed to keep resources highly available with minimum downtime by gathering
resources in ways that allow multiple IBM System p servers to access them. PowerHA manages
disk, network, and application resources logically, passing control to individual machines based on
availability and preference. From a systems administration point of view, the main concept behind
PowerHA is to keep everything as redundant as possible to ensure that there is high availability at
all levels.
Here, two System p servers share a common set of SAN storage and communicate on two
networks. They share between them a set of IP addresses, some Logical Volume Manager (LVM)
resources, and application controls—all managed by PowerHA.
One of these servers is considered to be "active" and is in control of these resources, while the
other is idle and sits ready in case it is needed, as shown in Figure 2.
When a problem occurs with the availability of some of the physical resources, such as some wires
being accidentally unplugged, PowerHA senses the errors and makes the other server take over.
There is a momentary pause in the availability of the resources, but then everything comes up as
though it were on the original machine, and no one can tell the difference, as shown in Figure 3.
Once the hardware becomes available again, the resources can remain where they are or go back
to the original server. It is completely at the discretion of the administrator.
However, hardware failures aren't the only reason for making resources move from one server
to another. You can also use this technology for things like operating system upgrades, firmware
maintenance, or other activities that may require downtime, all of which adds to the versatility and
usefulness of PowerHA.
• Resource group: This is a logical grouping of service IP addresses, application servers, and
shared volume groups that the nodes in the cluster can manage.
• Failover: This is a condition in which resource groups are moved from one node to another.
Failover can occur when a systems administrator instructs the nodes in the cluster to do so
or when circumstances like a catastrophic application or server failure forces the resource
groups to move.
• Failback/fallback: This is the action of moving back resource groups to the nodes on which
they were originally running after a failover has occurred.
• Heartbeat: This is a signal transmitted over PowerHA networks to check and confirm
resource availability. If the heartbeat is interrupted, the cluster may initiate a failover
depending on the configuration.
Prep work
A number of steps must take place before you can configure an PowerHA cluster and make it
available. The first step is to make sure that the hardware you will be using for the two servers is
as similar as possible. The number of processors, the quantity of memory, and the types of Fibre
Channel and Ethernet adapters should all be the same. If you are using logical partition (LPAR) or
virtual I/O (VIO) technology, be consistent: Don't mix hardware strategies like logical Host Ethernet
Adapters (LHEA) on one node with standard four-port Ethernet adapters on the other.
No development servers
I have seen many environments in a number of different companies over the years in
which the decision is made to declare one of the nodes in a cluster a "production" server
and the other a "development" server. This decision is typically made because companies
decide that having a server sit idle more than 90 percent of the time in case of a disaster
is a waste of money. I cannot stress this enough: DO NOT DO THIS. When this strategy
is used, invariably differences in the two servers arise, as development causes differences
in software, applications, and operating system functions. And when the time comes that
the production resource group has to be failed over to the development server (because it's
always a matter of when, not if), those differences will prevent things from running correctly.
The second step, which should coincide with the first, is to size the environment in such a way that
each node can manage all the resource groups simultaneously. If you decide that you will have
multiple resource groups running in the cluster, assume a worst-case scenario where one node will
have to run everything at once. Ensure that the servers have adequate processing power to cover
everything.
Third, you need to assign and/or share the same set of resources to each server. If you use SAN
disks for storage, the disks for the shared volume groups need to be zoned to all nodes. The
network VLANs, subnets, and addresses should be hooked up in the same fashion. Work with your
SAN and network administrators to get addresses and disks for the boot, persistent, and service IP
addresses.
Fourth and finally, the entire operating system configuration must match between the nodes.
The user IDs, third-party software, technology levels, and service packs need to be consistent.
One of the best ways to make this happen is to build out the intended configuration on one node,
make a mksysb backup, and use that to build out all subsequent nodes. Once the servers are built,
consider them joined at the hip: make changes on both servers consistently all the time.
With all of the virtualization technology available today, it's far more worthwhile to use VIO
to create a pair of production and development LPARs on the same set of System p servers
and hardware resources than to try to save a few dollars at the expense of sacrificing the true
purpose for which PowerHA was designed. Use things like shared processor weights, maximum
transmission unit (MTU) sizes, and RAM allocation to give the production LPARs more clout than
the development LPARs. Doing so creates an environment that can handle a failover and assures
managers and accountants that finances are being used wisely.
Note: This process assumes that all IP addresses have been predetermined and that the SAN
zoning of the disks is complete. Unless otherwise stated, you must run the tasks here on each and
every node of the cluster.
This defines one network per Ethernet adapter. I prefer to use the Pre-defined option as opposed
to the Discovered path, but that is up to your discretion. Check the subnet masks for consistency.
This defines the boot IP addresses on the respective network adapters. This address should be
the same IP addresses you used in step 3. Make sure you define these addresses within the
proper respective PowerHA-defined network.
This defines the persistent IP addresses, again paying attention to pick the proper respective
PowerHA-defined network.
smitty cm_extended_config_menu_dmn
Select the Discover PowerHA-related Information from Configured Nodes option, and check
for errors to fix. Generally, rebooting each node can clear up any minor problems, and this is a
good point to test restarting each server anyway.
Run the smitty cl_vg command, and create a shared volume group. When you create a shared
volume group, you only need to select one of the nodes, because the disk is shared.
Repeat step 7, except this time, select the Discovered option and the target disk.
This defines an application server for an application that PowerHA will manage. Use the scripts
you created in step 4.
Select the Change/Show Resources and Attributes for a Resource Group option. Then,
perform these steps:
Set Automatically correct errors found during verification? to Interactive. Correct any
problems along the way.
Conclusion
PowerHA is a robust and effective tool for keeping resources available on AIX servers. Although
this article presented a simple introduction and how-to for setting up a two-node cluster, PowerHA
is capable of doing much more, including application monitoring, integrating NAS resources, and
putting logic into starting up resource groups. But if you are looking to hit the ground running, the
best advice I have is to make a test cluster and try everything you can.
Resources
• HACMP Library: Learn more about HACMP in AIX and find helpful resources from the
HACMP Library.
• IBM PowerHA SystemMirror for AIX: Learn more about IBM PowerHA for AIX version 6.1, the
replacement for PowerHA.
• PowerHA for AIX Cookbook: Learn how to install, tailor, and configure PowerHA version 5.5.
• IBM eServer pSeries HACMP V5.x Certification Study Guide Update: This guide shows
how to implement high-availability clusters with HACMP version 5.x, helps you upgrade an
existing cluster to the latest version, or prepare you for the HACMP version 5.x certification
exam to achieve IBM eServer Certified Systems Expert - pSeries HACMP 5.x for AIX 5L.
• Implementing High Availability Cluster Multi-Processing (HACMP) Cookbook: Broaden your
understanding of the HACMP and HACMP Extended Distance (HACMP/XD) architecture.
• HACMP Planning Guide: This guide provides information necessary to plan and install the
HACMP for AIX software.
• AIX and UNIX® developerWorks zone: The AIX and UNIX zone provides a wealth of
information relating to all aspects of AIX systems administration and expanding your UNIX
skills.
• New to AIX and UNIX? Visit the New to AIX and UNIX page to learn more.
• Technology bookstore: Browse the technology bookstore for books on this and other
technical topics.
• developerWorks blogs: Check out our blogs and get involved in the developerWorks
community.
• Follow developerWorks on Twitter.
• Get involved in the My developerWorks community.
• Participate in the AIX and UNIX forums:
• AIX Forum
• AIX Forum for developers
• Cluster Systems Management
• IBM Support Assistant Forum
• Performance Tools Forum
• Virtualization Forum
• More AIX and UNIX Forums
Christian Pruett is a senior UNIX systems administrator with more than 14 years of
experience with AIX, Sun Solaris, Linux, and HP/UX in a wide variety of industries,
including computing, agriculture, and telecommunications. He is the co-author of two
IBM Redbooks on AIX, has served as a UNIX book review for O’Reilly Publishing,
and has worked on several of the IBM AIX certification exams. He resides in Colorado
with his wife and two children. You can reach Christian at pruettc@gmail.com.