Eucalyptus Installation

Overview
These are the steps that were taken to create a functional Eucalyptus 3.1 system. Theses intstructions are for CentOS 6.1. They may or may not work on other distributions. (Definitely won't work for any non-RedHat OS)

Hardware
This guide assumes that you have 2 or more computers to use in order to build a Eucalyptus cloud. These computers must support hardware virtualization (Read the "Prepare System" section for more info). This guide will be documenting how to set up a Eucalyptus cloud using the MANAGED networking mode. In our case, this required the machine that is running the Cluster Controller (CC) to have a second Network Interface Card (NIC) so that all Node Controllers (NCs) were on their own subnet.

Prepare System
We need to do a few things to make sure that our system is ready for Eucalyptus to be installed. Update the BIOS is one of the things we needed to do, Here is how you can do it too: Step 1: Downloading FreeDOS to use to boot the machine and run the BIOS .EXE wget http://www.fdos.org/bootdisks/autogen/FDOEM.144.gz Then, gunzip FDOEM.144.gz to unzip the file. Step 2: Copy the BIOS flash utility and the BIOS image that needs to be used to upgrade, and mount it to a floppy disk image. Here is what you do in order to complete this step: modprobe vfat modprobe loop mkdir /tmp/floppy mount -t vfat -o loop FDOEM.144 /tmp/floppy After mounting the Floppy you want to copy the EXE. that you downloaded for the BIOS:

cp DELLBIOSVERSION.exe /tmp/floppy (not actual name of the bios) then unmount the floppy: umount /tmp/floppy Step 3 is to burn the bootable CD which emulates a floppy device. mkisofs -o bootcd.iso -b FDOEM.144 FDOEM.144 cdrecord -v bootcd.iso Now you should have a bootable CD that can boot into FreeDOS where you can then run the BIOS .EXE that you have put on it.

Disable SELinux
Eucalyptus does not work with SELinux (Security Enhanced Linux). Disable it! Open the following file in your favorite text editor (I use vim):
vim /etc/selinux/config

Change the following line:

SELINUX=enforcing

To this:
SELINUX=permissive

Then run:
setenforce 0

Open Ports (or Disable Firewall)

You have a choice here. You can either open the following ports by editing iptables, or just turn off the firewall. If you want to add firewall rules to allow traffic on those ports, the easiest way is by running...
system-config-firewall-tui

... and following the prompts to open the following ports on the machines you will be using for the CLC/CC: 1. 2. 3. 4. 8443 8773 8774 9001

Open port 8775 on all Node Controllers.

Don't feel like opening ports? Select the option to disable the firewall in `system-configfirewall-tui` and move on with your life.

Check for Hardware Virtualization

We are going to be using KVM (Kernel-based Virtual Machine) for our virtual machine instances, as is this is what RedHat is supporting now rather than Xen. KVM requires that your CPU have hardware virtualization support (Intel VT or AMD-V) in order to work. To check for that, run the following command:
egrep '^flags.*(vmx|svm)' /proc/cpuinfo

If there is NO output then your CPU does NOT have hardware virtualization support, and therefore cannot use KVM. If you're willing to sell your soul you may still be able to use VMWare...

Sync with NTP Time Server

The system clocks need to synchronized across all machines. Install the Network Time Protocol daemon for this. Install the ntp package if it is not already installed:
yum install -y ntp

Sync with time server:

ntpdate pool.ntp.org

Start the service:

service ntpd start

Install Dependencies
Install Additional Repositories
Download the following .rpm package files More info here.
wget http://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.14.noarch.rpm wget http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release6-5.noarch.rpm wget http://downloads.eucalyptus.com/devel/packages/3devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-31.el.noarch.rpm yum localinstall pgdg-centos91-9.1-4.noarch.rpm epel-release-65.noarch.rpm \ elrepo-release-6-4.el6.elrepo.noarch.rpm \

eucalyptus-nightly-release-3-1.el.noarch.rpm

Necessary Manual Edit

Note: DO THIS IF AND ONLY IF YOU ARE RUNNING A 32 BIT VERSION OF CENTOS. It is HIGHLY recommended that you use a 64 bit OS, but making this change allows you to get some .noarch packages from the Eucalyptus repository. Otherwise, the package list will be 404 Not Found. Edit /etc/yum.repos.d/eucalyptus-nightly-release.repo This is the file for the Eucalyptus 3 development repositories. The x86_64 repos had to be used because the i686/i386 repositories gave a 404 error.
[euca-3-devel] name=euca-3-devel baseurl=http://downloads.eucalyptus.com/devel/packages/3devel/nightly/centos/$releasever/x86_64/ gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY gpgcheck=1 enabled=0 [euca-3-deps] name=euca-3-deps baseurl=http://downloads.eucalyptus.com/devel/packages/3devel/centos/$releasever/x86_64/ gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY gpgcheck=1 enabled=1

The changes that should be made are in bold.

Install Packages
Source: http://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.html
yum install bzr python-boto euca2ools libvirt-devel openssl-devel gcc ant ant-nodeps \ java-1.6.0-openjdk-devel curl-devel libxslt-devel xalan-j2-xsltc wsdl4j \ backport-util-concurrent httpd postgresql91-server libvirt PyGreSQL make \ openssh-clients scsi-target-utils qemu-kvm axis2-codegen axis2-adbcodegen \ httpd httpd-devel rampartc rampartc-devel axis2c axis2c-devel axis2

Create eucalyptus User

We need to add a eucalyptus user for the Eucalyptus services to run as. It is added to the kvm group so machines that are node controllers can manipulate instances.

useradd eucalyptus passwd eucalyptus

Modify WSDL2C.sh
Open /usr/lib64/axis2c/bin/tools/wsdl2c/WSDL2C.sh in a text editor. Erase the existing lines and add the following:
#!/bin/sh java -classpath $(build-classpath axis2/codegen axis2/kernel axis2/adb \ axis2/adb-codegen wsdl4j commons-logging xalan-j2 xsltc \ backport-util-concurrent ws-commons-XmlSchema ws-commons-neethi \ ws-commons-axiom annogen ) org.apache.axis2.wsdl.WSDL2C $*

Compile Dependencies
DON'T ACTUALLY DO THIS ON A 64 BIT MACHINE. The eucalyptus repository that was installed in a previous step has all of the packages you need. This is here for historical purposes and will probably be deleted soon. Note: Compilation only needs to happen on one machine. Once compiled, the binaries can be copied to the other servers This section is copied pretty much verbatim from the Eucalyptus 2.0 Installation Instructions.

Download Source Code

Download and extract tarball (2.0.3 dependencies work for 3.0 as far as I can tell)
wget http://eucalyptussoftware.com/downloads/releases/eucalyptus2.0.3-src-deps.tar.gz tar -zxf eucalyptus-2.0.3-src-deps.tar.gz

Set shell variable for install location Dave's completely objective, non-biased Note: Installing to /opt is really ugly, but this what the Eucalyptus install guide uses, so I will too.
export EUCALYPTUS=/opt/eucalyptus

Create directory for dependency binaries

mkdir -p $EUCALYPTUS/packages/

Change directory to where the dependency sources were extracted Your path will probably be different
cd ~/Downloads/eucalyptus-src-deps/

Install Axis2
Extract and copy Axis2 No compilation here. Nice and easy.

tar -zxf axis2-1.4.tgz mv axis2-1.4 $EUCALYPTUS/packages/

Install Axis2C
export APACHE_INCLUDES=/usr/include/httpd/ export APR_INCLUDES=/usr/include/apr-1/ export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0 tar zxf axis2c-src-1.6.0.tar.gz cd axis2c-src-1.6.0 CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} --withapache2=$APACHE_INCLUDES --with-apr=$APR_INCLUDES --enable-multithread=no make make install

Install Rampart/C
export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH tar zxf rampartc-src-1.3.0-0euca2.tar.gz cd rampartc-src-1.3.0 ./configure --prefix=${AXIS2C_HOME} --enable-static=no --withaxis2=${AXIS2C_HOME}/include/axis2-1.6.0 make make install

Edit Rampart/C Configuration

Change the following in $AXIS2C_HOME/axis2.xml. In the 'inflow' section, change:
<!--phase name="Security"/-->

to
<phase name="Security"/>

In the 'outflow' section, change:

<!--phase name="Security"/-->

to
<phase name="Security"/>

Compiling Eucalyptus
Download Source Code
Fetch the latest source from launchpad
bzr branch lp:eucalyptus

Enter the source directory

cd eucalyptus

Compile
Set shell variables for Java environment:
export JAVA_HOME="/usr/lib/jvm/java-openjdk/" export JAVA="$JAVA_HOME/jre/bin/java"

Compile Eucalyptus
./configure --with-axis2c=/usr/lib64/axis2c/ --with-apache2-moduledir=/usr/lib64/httpd/modules/ make make install

Install Eucalyptus on Other Machines

Based heavily upon this wonderful blog post.

Copy Installation to All Machines

Repeat for each machine.
rsync -a $EUCALYPTUS/ root@<HOST>:$EUCALYPTUS/ of the node you are copying to) (<HOST> being the IP

Set environment variables

These are needed pretty much everywhere. A good idea would be to place these in the ~/.bashrc files for root and eucalyptus users. You WILL have commands fail if you don't set these.
export EUCALYPTUS=/opt/eucalyptus export PATH=$PATH:$EUCALYPTUS/usr/sbin

Install Packages
Follow the "Installing Additional Packages" and "Installing Dependencies" instructions at the top of this page to install needed packages onto the node

Configure
Edit /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf Find the following configuration variables and set the values accordingly
EUCALYPTUS="/opt/eucalyptus" HYPERVISOR="kvm" USE_VIRTIO_DISK="1" USE_VIRTIO_NET="1"

INSTANCE_PATH="/opt/eucalyptus/instances" VNET_BRIDGE="virbr0"

Register Startup Scripts

Create symbolic links to the daemons so that the `service` command will work with them
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

Edit Hosts
A common problem that occurs is that the `euca_conf --initialize` command will fail. This usually happens because the system hostname isn't in the /etc/hosts file. Edit hosts file:
vim /etc/hosts

Add something like the following to the bottom of the file using your machine's IP address and hostname:
10.10.10.111 myhostname

Perform Inital Setup

euca_conf --setup

Start Cloud Controller (CLC), Storage Controller (SC) and Walrus

Start Service
euca_conf --initialize service eucalyptus-cloud start

Get Credentials
mkdir euca-credentials cd euca-credentials euca_conf --get-credentials admin.zip unzip admin.zip source eucarc

If you get an error when you try to do this such as "index out of range", it means that the eucalyptus services aren't fully operational yet. Be patient and eventually you can get your credentials.

Register Walrus, CC, and SC

The component and storage flags are arbitrary names for the registered services. Change to your liking. In our specific case in CS401, Walrus and the SC are running on the same machine as the CLC, so the hostname is simply the static IP of the Morpheus server.
su eucalyptus euca_conf --register-walrus --host <hostname> --component walrus -partition mycloud euca_conf --register-sc --host <hostname> --component storage -partition mycloud euca_conf --register-cluster --host <hostname> --component cluster -partition mycloud

It is important to note that the partition name for the Storage Controller and Cluster Controller MUST BE THE SAME. If the names are different then your cloud will NOT WORK.

Cluster Controller
Start Service
service eucalyptus-cc start

Node Controllers
Run on each NC

Install Packages
yum install eucalyptus-nc

Configure Ethernet Bridge

Enter the network configuration directory
cd /etc/sysconfig/network-scripts/

Open a new file called ifcfg-br0 in your text editor of choice

vim ifcfg-br0

Here is an example bridge configuration. Essentially, move your configuration from ifcfgeth0 to ifcfg-br0. Your ethernet device configuration will resemble this:

DEVICE="eth0" TYPE="Ethernet" HWADDR="00:18:8B:81:AE:E4" BRIDGE="br0"

Your bridge configuration will resemble this:

DEVICE="br0" TYPE="Bridge" BOOTPROTO="static" BROADCAST="10.15.255.255" DNS1="10.13.1.25" GATEWAY="10.15.1.1" IPADDR="10.15.15.13" NETMASK="255.255.0.0" ONBOOT="yes"

Restart the network service:

service network restart

If all goes well your devices and bridge will come up.

Register Node with CC

From the CLC run:
euca_conf --register-nodes="nodeip1 nodeip2 etc"

Start Services
service libvirtd start service eucalyptus-nc start

Verify Installation
Verify Registered Services on CLC
euca-describe-walruses euca-describe-storage-controllers euca-describe-clusters

Sample Output:
WALRUS walrus ENABLED {} STORAGECONTROLLER ENABLED {} walrus storage 10.15.15.10 storage 10.15.15.10

CLUSTER trinity ENABLED {}

trinity

10.15.15.12

Verify Cluster is Advertising Resources

euca-describe-availability-zones verbose

Sample Output (that should be updated when the cluster actually advertises resources):
AVAILABILITYZONE trinity 10.15.15.12 arn:euca:eucalyptus:trinity:cluster:trinity/ AVAILABILITYZONE |- vm types free AVAILABILITYZONE |- m1.small 0002 AVAILABILITYZONE |- c1.medium 0002 AVAILABILITYZONE |- m1.large 0001 AVAILABILITYZONE |- m1.xlarge 0000 AVAILABILITYZONE |- c1.xlarge 0000

/ / / / / /

max 0002 0002 0001 0000 0000

cpu 1 1 2 2 4

ram 128 256 512 1024 2048

disk 2 5 10 20 20