Configuring Oracle ASM On Amazon EC2

As part of the work we did for running Astrometric Processing on Amazon EC2, we needed to configure an instance of Oracle Enterprise 11g using ASM (Automated Storage Management)

The steps in this post are based on the Oracle supplied AMI ami-7ecb2f17 which includes Oracle 11g 11.06 Enterprise Edition 64bit.

You can use the EC2 tools to launch instances, but for the purposes of this post I will use ElasticFox.

For convention in this post, I will use the following prompt symbols: $$ for your Local PC, and EC2$ for the remote EC2 instance

Step 1: Launch the instance

Launch ami-7ecb2f17 as a large instance. Once it is running you will have to wait a couple of minutes for SSH to be available. When you connect, using Putty or whichever tool you prefer, accept the license and choose a password as shown below.

At the next prompt “Would you like to create a database now”, select no.

Before you proceed, this image has a small problem. The /mnt partition isn’t mounted and as it is, we won’t have enough space to be able to create a new image based on this one.

This is an easy fix adding the following line to the end of /etc/fstab

/dev/sdb   /mnt      ext3    defaults        0 0

and executing the following in a shell

EC2$ mount /mnt

Step 2: Assign an elastic IP

In order for Oracle to work well, it’s better to have a static IP for it, and for that we will assign an Elastic IP.
Using Elastic Fox or the API tools create an Elastic IP e.g. 174.129.xxx.yyyy and assign it to the instance

Change the instance hostname to be the public DNS name

EC2$ hostname ec2-174-129-xxx-yyy.compute-1.amazonaws.com

Step 3: Prepare ASMLib

Download Oracle ASMLib from the oracle website here making sure you chose the right drivers for this kernel: 2.6.18-53.1.13.9.1.el5xen.
Ah. First problem, there aren’t any for this particular kernel, so we’ll download the nearest ones available which are the 3 drivers for 2.6.18-53.1.13.el5

Once downloaded, copy the rpm’s up to your instance and install them

EC2$ rpm -Uvh    oracleasm-support-2.1.3-1.el5.x86_64.rpm 
EC2$ rpm -Uvh  oracleasm-2.6.18-53.1.13.el5xen-2.0.4-1.el5.x86_64.rpm 
     oracleasmlib-2.0.4-1.el5.x86_64.rpm

Because of the incompatibility of the drivers the second step will give us an error:

error: Failed dependencies:
kernel-xen = 2.6.18-53.1.13.el5 is needed by oracleasm-2.6.18-53.1.13.el5xen-2.0.4-1.el5.x86_64

We have to force the install and then use the script /usr/lib/oracleasm/oracleasm_debug_link to create a link between the two dependencies. If you have Oracle MetaLink see notes 805535.1 and 4626181.1
Here we need to add a small caveat. If you do the next steps, Oracle won’t support you. I imagine that this will change soon as they are committed to EC2

EC2$ rpm -Uvh --nodeps oracleasm-2.6.18-53.1.13.el5xen-2.0.4-1.el5.x86_64.rpm  
                oracleasmlib-2.0.4-1.el5.x86_64.rpm
Preparing...                ########################################### [100%]
   1:oracleasm-2.6.18-53.1.1########################################### [100%]
   2:oracleasmlib           ########################################### [100%]
root@domU-12-31-39-00-1D-21:[/tmp]

Create the link between the oracleasm packages

EC2$ /usr/lib/oracleasm/oracleasm_debug_link 2.6.18-53.1.13.el5xen 2.6.18-53.1.13.9.1.el5xen
root@domU-12-31-39-00-1D-21:[/tmp]

Now we need to configure ASMlib using /etc/init.d/oracleasm configure

Believe it or not, that’s the hard part done. The rest is much, much easier.

Before we go any further with configuring Oracle we need some persistent storage for ASM. I’m going to attach 5 EBS (Elastic Block Storage) volumes of 100GB each to the instance using devices /dev/sdg through to /dev/sdk.

For each disk we need to create a primary partition for ASM

And then we have to create the disk for ASM

Step 4: Configure the ASM and the Database

Because these are EC2 server images, and therefore don’t have X installed, we have to execute dbca using pre-recorded response files.

EC2$ dbca -silent -responseFile dbca_ASM.rsp
 
EC2$ dbca -silent -responseFile dbca_AGISDB.rsp

You can download the response file I used for ASM here and the response file for creating the database here

Step 5: Start your databases

EC2$ export ORACLE_SID=+ASM
EC2$ echo "startup" | sqlplus -s / as sysdba
EC2$ export ORACLE_SID=AGISDB
EC2$ echo "startup" | sqlplus -s / as sysdba
EC2$ echo "AGISDB Started"

Step 6: Fix /etc/inittab

The /etc/inittab generated by oracle doesn’t have the run level in it that Amazon EC2 instances have, run level 4. To fix this edit /etc/inittab and change the following line

h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null

to

h1:345:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null

Copy /etc/inittab to /etc/inittab.fixed

Add the following to /etc/init.d/dbora

/u01/app/oracle/product/11.1.0/db_1/bin/localconfig delete
/u01/app/oracle/product/11.1.0/db_1/bin/localconfig add
cp /etc/inittab.fixed /etc/inittab
emctl start dbconsole

Optionally start your databases

Final step: Create your new image

From your PC, copy your certificates up to the running instance

$$ scp -i <path to your keypair>  <path to your certificates dir>/*.pem root@ec2-xx-xx-xx-xxx.compute-1.amazonaw s.com:/mnt/

Inside the image, create a new image bundle making sure you exclude the /mnt directory where your certificates are. If you create a public image with your certificates embedded in them, you compromise the security of your images.

EC2$ ec2-bundle-vol -d /mnt  -e /mnt,/apps -k /mnt/pk-<your pk>.pem -c /mnt/cert-<your cert>.pem -r x86_64 -u <your EC2 account id> -p mynewinstance

Upload the bundle to S3 to a suitable place, e.g. myimages

EC2$ ec2-upload-bundle -b myimages -m /mnt/mynewinstance.manifest.xml -a <your EC2 key> -s <your EC2 secret key>

Finally, register the image in EC2
$$ ec2-register myimages /mynewinstance.manifest.xml

I think I’ve included everything. If you have problems, please let me know.