Article presents quick installation of Oracle Enterprise RAC 11.2.0.3 on Oracle Enterprise Linux 5 using Oracle Virtual Box.

Architecture

Software used:

• Oracle Virtual Box – 64 bit
• Oracle Enterprise Linux 5 – 64 bit
• Oracle Database 11G Release 2(11.2.0.3) – 64 bit for Linux

Binaries 11.2.0.3

p10404530_112030_Linux-x86-64_1of7.zip - database software
p10404530_112030_Linux-x86-64_2of7.zip - database software
p10404530_112030_Linux-x86-64_3of7.zip - grid software

Two virtual machines are created:

• rac1.dbaora.com
• rac2.dbaora.com

	public	private	vip
rac1	192.168.0.50	192.168.1.60	192.168.0.70
rac2	192.168.0.51	192.168.1.61	192.168.0.71

with single client access  name (SCAN) address

	public
rac-scan	192.168.0.20 192.168.0.21 192.168.0.22

Each virtual machines consist of:

• 32G local disk (dynamic space allocation)
• 3G ram memory
• 3 network cards (2 dedicated just for RAC)

NOTE to save space rac2.dbaora.com is linked clone of rac1.dbaora.com

NOTE – to get connection to internet from virtual box be sure you are not using the same subnet as your wifi card

In my case all are different:

• 192.168.0.X is public – you can change it
• 192.168.1.X is private – you can change it
• 192.168.2.X is my internet – usually it is static for your network unless you change it on router – doesn’t make sense 🙂

Shared storage visible on each node:

• 20G shared virtual disk (fixed size)

Virtual Machine configuration

1. Start virtual box manager and click “New” button.

2. Enter virtual machine name “oel5_112_rac1”, select type “Linux”, version “Oracle(64bit)”, set memory to 3096M. Click “Create” button to allocate storage for this machine.

3. Enter storage attributes: name, size 32G, type VDI, dynamically allocated then click “Create” button.

4. Virtual machine is ready click “Settings” to change some details.

5. Select “System” option, tab “Processor” in case you want to assign more processor to your virtual machine. I have 4 core machines so I assigned to rac1.dbaora.com 2 CPU.

6. Select “Option” option, “Video” tab and assign 128MB of memory and enable 3D acceleration to improve graphics in your virtual machine.

7. Select “Network” option and select “Internal Network” for Adapter 1 and 2 and “NAT” for Adapter 3. Adapter 1 will be used as public RAC interface, Adapter 2 as private and Adapter 3 to connect to internet(it’s optional but I use it to have access to internet from my virtual machine).

8. On “Storage” tab assign iso file to “CD/DVD Drive”  to install Oracle Enterprise Linux 5 64 bit. Once it’s done click “OK” button.

9. Virtual box is ready to install so click “Start” button.

Install Oracle Enterprise Linux 5

Installation of OEL5 is described in details in following article: Install Oracle Linux 5 64-bit. The only difference is size of our disk, memory and number of network cards. Remember to assign hostname to “rac1.dbaora.com” during installation and select “Active on Boot” for each network card.

We have 3 network interfaces with ip addresses from DHCP. For RAC installation we need static addresses for eth0 and eth1. You can assign static ip addresses by click “Edit” button or you can do it later using “Network manager”.

• 192.168.0.50/255.255.255.0 for eth0 (public)
• 192.168.1.60/255.255.255.0 for eth1 (private)

“Network Manager” is available from System->Administration->Network. You can apply here static ip address for eth0 and eth1 and activate both cards. Please remember to save configuration from network manager by selection from menu File->Save.

NOTE: eth2 is NAT interface so nothing has to be change. This card is used to communicate with Internet.

Shared storage

1. To add shared storage virtual machine rac1.dbaora.com must be down. Once it’s down go to “Settings” select “Storage” then click on icon with plus on disk.

2. Click on “Create new disk” button.

3. Default option “VDI” is fine click “Next” button.

4. Select “Fixed size” then click “Next” button.

5. Enter storage name “Shared” and set size to 20G then click “Create” button.

6. Once created you should see new disk attached to our virtual machine. Click “OK” button.

8. From main menu you need to select File->Virtual Media Manager to change “Shared” disk attribute. Select “Shared.vdi” and click “Modify” button.

9. Select “Sharable” and click “OK” button. It will prepare this storage to be shareable between RAC instances. Please start virtual machine to create partitions on shared storage.

Host configuration

All commands are executed as user root.

Configure shared storage.Shared storage is visible as /dev/sdb device. Two partitions each 10G will be created using fdisk command. Both new partitions will be used for ASM.

--list devices

ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sda2  /dev/sdb

--add two partitions

fdisk /dev/sdb

The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):   
Using default value 1
Last cylinder or +size or +sizeM 
or +sizeK (1-2610, default 2610): +10240M

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1247-2610, default 1247): 
Using default value 1247
Last cylinder or +size or +sizeM or +sizeK (1247-2610, default 2610): 
Using default value 2610

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

--list new partitions

fdisk -l /dev/sdb

Disk /dev/sdb: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1246    10008463+  83  Linux
/dev/sdb2            1247        2610    10956330   83  Linux

Add groups

--required groups
/usr/sbin/groupadd -g 501 oinstall
/usr/sbin/groupadd -g 502 dba
/usr/sbin/groupadd -g 503 oper

Add user oracle

/usr/sbin/useradd -u 502 -g oinstall -G dba oracle

Change password for user oracle

passwd oracle

Add kernel parameters to /etc/sysctl.conf

--kernel parameters for 11g installation
kernel.shmmni = 4096 
kernel.shmmax = 4398046511104
kernel.shmall = 1073741824
kernel.sem = 250 32000 100 128

fs.aio-max-nr = 1048576
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586

Apply kernel parameters

/sbin/sysctl -p

Add following line for user oracle to set shell limits in file /etc/security/limits.conf

--shell limits for users oracle 11gR2
oracle   soft   nproc    131072
oracle   hard   nproc    131072
oracle   soft   nofile   131072
oracle   hard   nofile   131072
oracle   soft   core     unlimited
oracle   hard   core     unlimited
oracle   soft   memlock  50000000
oracle   hard   memlock  50000000

The “/etc/hosts” file must contain a fully qualified name for the server.

<IP-address>  <fully-qualified-machine-name>  <machine-name>

Enter following data to “/ect/hosts”

127.0.0.1     localhost.localdomain localhost
#public
192.168.0.50  rac1.dbaora.com      rac1
192.168.0.51  rac2.dbaora.com      rac2
#private
192.168.1.60  rac1-priv.dbaora.com rac1-priv
192.168.1.61  rac2-priv.dbaora.com rac2-priv
#virtual
192.168.0.70 rac1-vip.dbaora.com  rac1-vip
192.168.0.71 rac2-vip.dbaora.com  rac2-vip
#scan
192.168.0.20 rac-scan.dbaora.com  rac-scan
192.168.0.21 rac-scan.dbaora.com  rac-scan
192.168.0.22 rac-scan.dbaora.com  rac-scan

Verify network response on rac1.dbaora.com for th0 and eth1

[root@rac1 ~]# ping rac1 -c 1
PING rac1.dbaora.com (192.168.0.50) 56(84) bytes of data.
64 bytes from rac1.dbaora.com (192.168.0.50): 
icmp_seq=1 ttl=64 time=0.032 ms

--- rac1.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.032/0.032/0.032/0.000 ms

[root@rac1 ~]# ping rac1-priv -c 1
PING rac1-priv.dbaora.com (192.168.1.60) 56(84) bytes of data.
64 bytes from rac1-priv.dbaora.com (192.168.1.60): 
icmp_seq=1 ttl=64 time=0.036 ms

--- rac1-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.036/0.036/0.036/0.000 ms

Check which packages are installed and which are missing

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE}(%{ARCH})\n' binutils \
compat-libstdc++-33 \
elfutils-libelf \
elfutils-libelf-devel \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc \
libstdc++ \
libstdc++-devel \
make \
sysstat \
unixODBC \
unixODBC-devel

Install missing packages. It’s just example:

#directory with mounted Oracle Enterprise Linux 5 install disk

cd <path with Oracle Enterprise Linux 5>/Server/Packages

#install missed packages (example for package unixODBC*)

rpm -Uvh unixODBC*

Disable Secure Linux

To disable secure Linux edit the “/etc/selinux/config” file, making sure the SELINUX flag is set as follows. It requires REBOOT to be effective !!!

SELINUX=disabled

Disable Firewall

Firewall can be disabled in graphical mode or manually.

Option 1 – graphical mode

Run firewall graphical manager “System”->”Administration”->”Security Level and Firewall” and click “Disable” button and “Apply” button. As you can notice you can change SELinux on second tab as well.

Option 2 – manual mode

Run following command

service iptables stop
chkconfig iptables off

Change NTP

It’s required to disable NTP(Network Time Protocol) or modify its settings so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes.

Option 1 – disable NTP

service ntpd stop
Shutting down ntpd:                                        [  OK  ]
chkconfig ntpd off
mv /etc/ntp.conf /etc/ntp.conf.orig
rm /var/run/ntpd.pid

Option 2 – modify NTP

If you want to keep NTP, add the “-x” option in the “/etc/sysconfig/ntpd” file.

OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

and restart NTP

# service ntpd restart

Directories

Create ORACLE_BASE

mkdir -p /ora01/app/oracle

Create ORACLE_HOME for grid and database software

mkdir -p /ora01/app/oracle/product/11.2.0/db_1
mkdir -p /ora01/app/grid/product/11.2.0/grid

chown oracle:oinstall -R /ora01
chmod 775 /ora01/app/oracle

New profiles for user oracle

change oracle profile /home/oracle/.bash_profile

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

alias genv='. /home/oracle/.bash_profile_grid;envo'
alias denv='. /home/oracle/.bash_profile_database;envo'

. /home/oracle/.bash_profile_database

envo

create new profile for user oracle /home/oracle/.bash_profile_grid. This profile will be used for grid software.

# Oracle Settings
export TMP=/tmp

export ORACLE_HOSTNAME=rac1.dbaora.com
export ORACLE_UNQNAME=+ASM
export ORACLE_BASE=/ora01/app/oracle
export ORACLE_HOME=/ora01/app/grid/product/11.2.0/grid
export ORACLE_SID=+ASM1

PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;
export CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

alias cdob='cd $ORACLE_BASE'
alias cdoh='cd $ORACLE_HOME'
alias tns='cd $ORACLE_HOME/network/admin'
alias envo='env | grep ORACLE'

create new profile for user oracle /home/oracle/.bash_profile_database. This profile will be used for database software.

# Oracle Settings
export TMP=/tmp

export ORACLE_HOSTNAME=rac1.dbaora.com
export ORACLE_UNQNAME=ORA11G
export ORACLE_BASE=/ora01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_SID=ORA11G1

PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;
export CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

alias cdob='cd $ORACLE_BASE'
alias cdoh='cd $ORACLE_HOME'
alias tns='cd $ORACLE_HOME/network/admin'
alias envo='env | grep ORACLE'

umask 022

So next time you do su to oracle user you should have environment prepared for database software

[root@rac1 ~]# su - oracle
ORACLE_UNQNAME=ORA11G
ORACLE_SID=ORA11G1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/11.2.0/db_1

Number of aliases are defined in profiles to quickly managed environment

• genv – sets environment for grid software

alias genv=’. /home/oracle/.bash_profile_grid;envo’

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/11.2.0/grid

• denv – sets environment for database software

alias denv=’. /home/oracle/.bash_profile_database;envo’

[oracle@rac1 ~]$ denv
ORACLE_UNQNAME=ORA11G
ORACLE_SID=ORA11G1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/11.2.0/db_1

• cdob – change directory to $ORACLE_BASE

alias cdob=’cd $ORACLE_BASE’

• cdoh – change directory to $ORACLE_HOME

alias cdoh=’cd $ORACLE_HOME’

• tns – change directory to $ORACLE_HOME/network/admin

alias tns=’cd $ORACLE_HOME/network/admin’

• envo – displays current environments settings and filters by ORACLE word

alias envo=’env | grep ORACLE’

[oracle@rac1 ~]$ envo
ORACLE_UNQNAME=ORA11G
ORACLE_SID=ORA11G1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/11.2.0/db_1

Remember to modify owner of all profiles

chown oracle:oinstall /home/oracle/.bash*
chmod 750 oracle:oinstall /home/oracle/.bash*

As oracle user unzip database and grid software. It should create 2 directories:

• database – database software
• grid – grid software

--database software
p10404530_112030_Linux-x86-64_1of7.zip
p10404530_112030_Linux-x86-64_2of7.zip

--grid software
p10404530_112030_Linux-x86-64_3of7.zip

[oracle@rac1]$ unzip p10404530_112030_Linux-x86-64_1of7.zip
[oracle@rac1]$ unzip p10404530_112030_Linux-x86-64_2of7.zip
[oracle@rac1]$ unzip p10404530_112030_Linux-x86-64_3of7.zip

[oracle@rac1]ls
database grid

Install package cvuqdisk-1.0.9-1.rpm from grid software as user root

cd <install grid software>/rpm
[root@rac1 rpm]# rpm -Uvh cvuqdisk-1.0.9-1.rpm
Preparing...                ############################## [100%]
Using default group oinstall to install package
   1:cvuqdisk               ############################## [100%]

Configure ASM devices

To configure ASMlib first you need to download ASMLib rpms from OTN. If you are using UEK kernel all required rpms are already installed.

• oracleasm-support-2.1.8-1.el5.i386.rpm
• oracleasmlib-2.0.4-1.el5.i386.rpm
• oracleasm-[your-kernel-version].rpm

Install the packages using the following command

rpm -Uvh oracleasm*.rpm

Configure ASMlib

[root@rac1 ~]# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: 
Writing Oracle ASM library driver configuration: done

Load asm kernel module

[root@rac1 ~]# /usr/sbin/oracleasm init
Loading module "oracleasm": oracleasm
Mounting ASMlib driver filesystem: /dev/oracleasm

Add ASM disks

[root@rac1 ~]# /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# /usr/sbin/oracleasm createdisk DISK2 /dev/sdb2
Writing disk header: done
Instantiating disk: done

Scan ASM disks

[root@rac1 ~]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

List ASM disks

[root@rac1 ~]# /usr/sbin/oracleasm listdisks
DISK1
DISK2

Cloning machine

New machine rac2.dbaora.com is cloned from machine rac1.dbaora.com using “linked clone” method. It allows to speed up process of cloning. it’s especially useful for development environments.

• Full clone: In this mode all depending disk images are copied to the new VM folder. The clone can fully operate without the source VM.
• Linked clone: In this mode new differencing disk images are created where the parent disk images are the source disk images. If you selected the current state of the source VM as clone point, a new snapshot will be created implicitly.

1. Before we create “linked clone” we need to remove shared storage. On Storage tab select Shared.vdi then click “Remove Attachment”.

2. Now you should see oel5_112_rac1 without Shared storage.

3. In menu of VirtualBox select “Machine”-“Clone” option. Enter name for new machine oel5_112_rac2 and click “Next” button.

4. Select “Linked clone” option and click “Clone” button.

5. You should see new machine oel5_112_rac2

6. It’s time to attach Shared.vdi disk to both machine. Select first machine oel5_rac112_rac1 then click “Storage”. Select “Controller:SATA” and click last button “Add Hard Disk”.

7. Click “Choose existing disk”. You should see standard window to select a file. Select “Shared.vdi” to attach it to machine.

8. Once it’s done shared disk should be visible by machine oel5_112_rac1. Click “OK” button to save your configuration.

9. You need to repeat the same steps 6,7 to attach shared disk to machine oel5_112_rac2. Don’t be surprised that both machines are using the same disk “oel5_112_arc1.vdi”. New machine oel5_112_rac2 is linked clone.

Network settings for second machine

Start second linked and use “Network Manager”  System->Administration->Networkto apply new static ip address for eth0 and eth1 and activate both cards. Please remember to save configuration from network manager by selection from menu File->Save.

• 192.168.0.51/255.255.255.0 for eth0 (public)
• 192.168.1.61/255.255.255.0 for eth1 (private)

Remember to change host name as well to

• rac2.dbaora.com

Verify network between nodes

It’s good time to run verify network on each node: rac1.dbaora.com and rac2.dbaora.com

hostname
ping rac1 -c 1
ping rac2 -c 1
ping rac1-priv -c 1
ping rac2-priv -c 1

test on rac1.dbaora.com

 [root@rac1 ~]# hostname
rac1.dbaora.com
[root@rac1 ~]# ping rac1 -c 1
PING rac1.dbaora.com (192.168.0.50) 56(84) bytes of data.
64 bytes from rac1.dbaora.com (192.168.0.50): icmp_seq=1 ttl=64 time=0.035 ms

--- rac1.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.035/0.035/0.035/0.000 ms
[root@rac1 ~]# ping rac2 -c 1
PING rac2.dbaora.com (192.168.0.51) 56(84) bytes of data.
64 bytes from rac2.dbaora.com (192.168.0.51): icmp_seq=1 ttl=64 time=0.318 ms

--- rac2.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.318/0.318/0.318/0.000 ms
[root@rac1 ~]# ping rac1-priv -c 1
PING rac1-priv.dbaora.com (192.168.1.60) 56(84) bytes of data.
64 bytes from rac1-priv.dbaora.com (192.168.1.60): icmp_seq=1 ttl=64 time=0.025 ms

--- rac1-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.025/0.025/0.025/0.000 ms
[root@rac1 ~]# ping rac2-priv -c 1
PING rac2-priv.dbaora.com (192.168.1.61) 56(84) bytes of data.
64 bytes from rac2-priv.dbaora.com (192.168.1.61): icmp_seq=1 ttl=64 time=0.352 ms

--- rac2-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.352/0.352/0.352/0.000 ms

test on rac2.dbaora.com

 [root@rac2 ~]# hostname
rac2.dbaora.com
[root@rac2 ~]# ping rac1 -c 1
PING rac1.dbaora.com (192.168.0.50) 56(84) bytes of data.
64 bytes from rac1.dbaora.com (192.168.0.50): icmp_seq=1 ttl=64 time=0.342 ms

--- rac1.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.342/0.342/0.342/0.000 ms
[root@rac2 ~]# ping rac2 -c 1
PING rac2.dbaora.com (192.168.0.51) 56(84) bytes of data.
64 bytes from rac2.dbaora.com (192.168.0.51): icmp_seq=1 ttl=64 time=0.039 ms

--- rac2.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.039/0.039/0.039/0.000 ms
[root@rac2 ~]# ping rac1-priv -c 1
PING rac1-priv.dbaora.com (192.168.1.60) 56(84) bytes of data.
64 bytes from rac1-priv.dbaora.com (192.168.1.60): icmp_seq=1 ttl=64 time=0.312 ms

--- rac1-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.312/0.312/0.312/0.000 ms
[root@rac2 ~]# ping rac2-priv -c 1
PING rac2-priv.dbaora.com (192.168.1.61) 56(84) bytes of data.
64 bytes from rac2-priv.dbaora.com (192.168.1.61): icmp_seq=1 ttl=64 time=0.055 ms

--- rac2-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.055/0.055/0.055/0.000 ms

Grid software installation

Start grid software installation as user oracle, remember to run command xhost+ as root.

xhost +
access control disabled, clients can connect from any host

Set grid environment and run grid installation software

su - oracle

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/11.2.0/grid

--run installation 
cd <install grid software>
./runInstall

1. Select “Skip software updates” and click “Next” button.

2. Select “Install and Configure Oracle Grid Infrastructure for a Cluster” and click “Next” button.

3. Accept default “Typical installation” and click “Next” button

4. Enter SCAN Name “rac-scan” then click “Add” button to add second node rac2.dbaora.com

5. Enter public and virtual host name for second node and click “OK” button

6.Click on “Identify network interfaces” button

7. Ensure eth0 is public and eth1 is private interface type and click “OK” button

8. It’s time to establish SSH connection between cluster nodes. Enter OS Password for user oracle and click “Setup” button. Once configuration is done click “Next” button

9. Select “Oracle Automatic Storage Management” and enter SYSASM user password and click “Next”

10. Click on “Change Discovery Path” button

11. Enter path for ASM devices “/dev/oracleasm/disk*”

12. You should see available devices. Select first one and click “next” button

13. Accept default directory for “Inventory Directory” and click “Next” button

14. Checks are executed to show any errors that need to be fixed before installation can proceed. Ignore “Device Checks for ASM” by check “Ignore All” checkbox and click “Next” button

15. Confirm you ignore checks click “Yes” button

16. It’s the last moment to go back and make modifications. Click “Install” to proceed with installation

17. It will take some time to install software so be patient

18. When prompted run configuration scripts on each node. Once it’s done click “OK” button

19. Installation should continue …

20. You should receive information about errors related to SCAN setup, assuming you are not using DNS. Just ignore it.

21. Click “Next” Button

22. Just click “Yes” button

23. Installation of grid software is completed so click “Close” button

ASM configuration assistant

So far we have created only one ASM group DATA. I want to show you how quickly you can add extra group to ASM instance.

As user oracle start ASMCA. Remeber to start xhost + as user root to enable run graphical tools from other users like oracle.

ASMCA – ASM configuration assistant is part of grid software so proper environment must be set

su - oracle

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/11.2.0/grid

[oracle@rac1 ~]$ asmca

1. ASMCA started click “Disk Groups” tab.

2. Click “Create” button to add new ASM diskgroup.

3. Enter group name BACKUP, check checkbox for “/dev/oracleasm/disks/DISK2″ device and click “OK” button.

4. You should see following screen. Click “OK”.

4. Now you should see we have 2 ASM groups “BACKUP” and “DATA”. Click “Exit” button.

Now you can verify in sqlplus you have 2 ASM groups.

[oracle@rac1 ~]$ sqlplus

SQL*Plus: Release 11.2.0.3.0 Production on Sun Jun 16 22:39:11 2013
Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Enter user-name: / as sysasm

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit 
Production With the Automatic Storage Management option

SQL> select name from v$asm_diskgroup;

NAME
------------------------------
DATA
BACKUP

Database software installation

Let’s start with database software installation as user oracle. Set environment to database software

su - oracle

[oracle@rac1 ~]$ denv 
ORACLE_UNQNAME=ORA11G 
ORACLE_SID=ORA11G1 
ORACLE_BASE=/ora01/app/oracle 
ORACLE_HOSTNAME=rac1.dbaora.com 
ORACLE_HOME=/ora01/app/oracle/product/11.2.0/db_1

--run installation 
cd <install database software>
./runInstall

1. Uncheck checkbox “I wish to receive security updates via My Oracle Support” and click “Next” button.

2. Ignore this message and click “Yes”.

3. Select “Skip software updates” and click “Next” button.

4. Select “Create and configure a database” and click “Next” button.

5. Select “Server Class” and click “Next” button.

6. Make sure both nodes are selected, then click the “Next” button.

7. Select “Advanced install” and click “Next” button.

8. Select your languages and click “Next” button.

9. Accept default “Enterprise Edition” and click “Next” button. “Select Options” button enables to enable/disable extra options to install like:

• Partitioning – default enabled
• OLAP – default enabled
• Label Security
• Data Mining – default enabled
• Database Vault
• Real Application Testing – default enabled

10. It shows directories for database software installation. Just click “Next” button.

11. Accept default settings and click “Next” button.

12. Enter database name and click “Next” button.

13. On following tabs you can specify many options for your database like:

• memory settings
• database character sets
• turn on extra new security features
• install extra schemas

Click “Next” button to continue.

14. Just click “Next” button.

15. Select “Oracle Automatic Storage Management” and enter password for user ASMSNMP. The user is used by OEM similar to DBSNMP. Click “Next” button.

16. Accept default settings “Do not enable automated backups” or specify details for your automated backup then click “Next” button.

17. Select “DATA” diskgroup to install our new database and click “Next” button.

18. Specify passwords for users: SYS, SYSTEM, SYSMAN, DBSNMP. You can specify the same password for all of them like in example below. Click “Next” button to continue.

19. Accept default OS groups and click “Next” button.

20. Prerequisites checks are executed to verify cluster settings.

21. Ignore error related to SCAN by check checkbox “Ignore All” and click “Next” button.

22. Click “Yes” button.

23. It’s the last moment to make changes before starting installation. Click “Install” button to continue.

24. Just wait installation in progress …

25. When software installation is completed DBCA is started to create database.

26. Summary window for our new database. Just click “OK” button. it’s worth to change passwords for our important users like SYS, SYSTEM etc “Password Management”.

27. As last step run root.sh script on both nodes. when it’s completed click “OK” button.

28. Installation of database software and new database ORA11G is completed. Click “Close” button.

Verify RAC installation

[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/11.2.0/grid

[oracle@rac1 ~]$ srvctl config database -d ORA11G
Database unique name: ORA11G
Database name: ORA11G
Oracle home: /ora01/app/oracle/product/11.2.0/db_1
Oracle user: oracle
Spfile: +DATA/ORA11G/spfileORA11G.ora
Domain: dbaora.com
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: ORA11G
Database instances: ORA11G1,ORA11G2
Disk Groups: DATA
Mount point paths: 
Services: 
Type: RAC
Database is administrator managed

[oracle@rac1 ~]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): rac2,rac1

[oracle@rac1 ~]$ srvctl status asm
ASM is running on rac2,rac1

[oracle@rac1 ~]$ srvctl status database -d ORA11G
Instance ORA11G1 is running on node rac1
Instance ORA11G2 is running on node rac2

Status of your database can be visible in OEM https://rac1.dbaora.com:1158/em/

Have a fun 🙂

Tomasz