Article presents quick installation of Oracle Enterprise RAC 12C(12.1.0.1) on Oracle Enterprise Linux 6 using Oracle Virtual Box.

Architecture

Software used:

• Oracle Virtual Box – 64 bit
• Oracle Enterprise Linux 6 – 64 bit
• Oracle Database 12C Release 1(12.1.0.1) – 64 bit for Linux

Database Software 12C

Software for 12CR1 is available on OTN or edelivery

• OTN: Oracle Database 12c Release 1 (12.1.0.1) Software (64-bit).
• edelivery: Oracle Database 12c Release 1 (12.1.0.1) Software (64-bit)

Database software

linuxamd64_12c_database_1of2.zip 
linuxamd64_12c_database_2of2.zip

Grid software

linuxamd64_12c_grid_1of2.zip - grid software
linuxamd64_12c_grid_2of2.zip - grid software

Requirements

Be sure you fulfill following:

• Oracle Linux 6 with the Unbreakable Enterprise kernel: 2.6.39-200.24.1.el6uek.x86_64 or later
• Oracle Linux 6 with the Red Hat Compatible kernel: 2.6.32-71.el6.x86_64 or later

Two virtual machines are created:

• rac1.dbaora.com
• rac2.dbaora.com

	public	private	vip
rac1	192.168.0.50	192.168.56.60	192.168.0.70
rac2	192.168.0.51	192.168.56.61	192.168.0.71

with single client access name (SCAN) address

	public
rac-scan	192.168.0.20 192.168.0.21 192.168.0.22

NOTE – to avoid problems with access to internet from virtualbox public and private subnet should be different then your internet subnet in my case it is:

• public 192.168.0.x
• private 192.168.56.x
• internet 192.168.1.x

SCAN addresses are resolved by configuration of dnsmasq (simple DNS)

Each virtual machines consist of:

• 64G local disk (dynamic space allocation)
• 4G ram memory
• 3 network cards (2 dedicated just for RAC + 1 for Internet connection)

NOTE to save space rac2.dbaora.com is linked clone of rac1.dbaora.com

Shared storage visible on each node:

• 20G shared virtual disk (fixed size)

Virtual Machine configuration

1. Start virtual box manager and click “New” button.

2. Enter virtual machine name “oel6_121_rac1”, select type “Linux”, version “Oracle(64bit)”, set memory to 4096M. Click “Create” button to allocate storage for this machine.

3. Enter storage attributes: name, size 64G, type VDI, dynamically allocated then click “Create” button.

4. Virtual machine is ready click “Settings” to change some details.

5. Select “System” option, tab “Processor” in case you want to assign more processor to your virtual machine. I have 4 core machines so I assigned to rac1.dbaora.com 2 CPU.

6. Select “Option” option, “Video” tab and assign 128MB of memory and enable 3D acceleration to improve graphics in your virtual machine.

7. Select “Network” option and select “Internal Network” for Adapter 1 and 2 and “NAT” for Adapter 3. Adapter 1 will be used as public RAC interface, Adapter 2 as private and Adapter 3 to connect to internet(it’s optional but I use it to have access to internet from my virtual machine).

8. On “Storage” tab assign iso file to “CD/DVD Drive” to install Oracle Enterprise Linux 6 64 bit. Once it’s done click “OK” button.

9. Virtual box is ready to install so click “Start” button.

Install Oracle Enterprise Linux 6

Installation of OEL6 is described in details in following article: Install Oracle Linux 6 64-bit. For RAC OS requirements are following: disk(at least 64G), memory(at least 4096M) and number of network cards(2 as minimum + 1 extra for Internet connection).

RAC requires static addresses for eth0 and eth1.

• 192.168.0.50/255.255.255.0 for eth0 (public)
• 192.168.56.60/255.255.255.0 for eth1 (private)

Remember to assign hostname to “rac1.dbaora.com” during installation. If you click during installation on “Configure Network” button you should see “Network Connections” manager to configure ip address for each network card.

NOTE: eth2 is NAT interface so nothing has to be change. This card is used to communicate with Internet.

“Network Connections” manager is available from “System->Preferences->Network Connections” if you want specify static addresses later after installation of OEL6.

I usually modify storage settings to allocate 4096M for swap and rest for ‘/’

Shared storage

1. To add shared storage virtual machine rac1.dbaora.com must be down. Once it’s down go to “Settings” select “Storage” then click on icon with plus on disk.

2. Click on “Create new disk” button.

3. Default option “VDI” is fine click “Next” button.

4. Select “Fixed size” then click “Next” button.

5. Enter storage name “Shared” and set size to 20G then click “Create” button.

6. Once created you should see new disk attached to our virtual machine. Click “OK” button.

7. From main menu you need to select “File->Virtual Media Manager” to change “Shared” disk attribute. Select “Shared.vdi” and click “Modify” button.

8. Select “Sharable” and click “OK” button. It will prepare this storage to be shareable between RAC instances. Please start virtual machine to create partitions on shared storage.

Host configuration

All commands are executed as user root.

Configure shared storage

Shared storage is visible as /dev/sdb device. Two partitions each 10G will be created using fdisk command. Both new partitions will be used for ASM.

--list devices

ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sda2  /dev/sdb

--add two partitions

[root@rac1 Desktop]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, 
SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xc50ad972.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 
         will be corrected by w(rite)

WARNING: 
  DOS-compatible mode is deprecated. It's strongly recommended to
  switch off the mode (command 'c') and change display units to
  sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1): 
Using default value 1
Last cylinder, 
  +cylinders or +size{K,M,G} (1-2610, default 2610): +10240M

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1307-2610, default 1307): 
Using default value 1307
Last cylinder, +cylinders or +size{K,M,G} (1307-2610, default 2610): 
Using default value 2610

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

--list new partitions

fdisk -l /dev/sdb

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc50ad972

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1306    10490413+  83  Linux
/dev/sdb2            1307        2610    10474380   83  Linux

Add groups

--required groups
groupadd -g 54321 oinstall
groupadd -g 54322 dba
groupadd -g 54323 oper

--additional not required
#groupadd -g 54324 backupdba
#groupadd -g 54325 dgdba
#groupadd -g 54326 kmdba
#groupadd -g 54327 asmdba
#groupadd -g 54328 asmoper
#groupadd -g 54329 asmadmin

Add user oracle

/usr/sbin/useradd -u 54321 -g oinstall -G dba,oper oracle

Change password for user oracle

passwd oracle

Manual setup

Add kernel parameters to /etc/sysctl.conf – this part in fact is not required for OEL6

# kernel parameters for 12gR1 installation

fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500

Apply kernel parameters

/sbin/sysctl -p

Add following line for user oracle to set shell limits in file /etc/security/limits.conf

# shell limits for users oracle 12gR1

oracle   soft   nofile   1024
oracle   hard   nofile   65536
oracle   soft   nproc    2047
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768

Other host configuration

The “/etc/hosts” file must contain a fully qualified name for the server.

<IP-address>  <fully-qualified-machine-name>  <machine-name>

Enter following data to “/ect/hosts”. Please notice that SCAN is disabled here. It will be configured later with dnsmasq.

127.0.0.1     localhost.localdomain localhost

#public
192.168.0.50   rac1        rac1.dbaora.com
192.168.0.51   rac2        rac2.dbaora.com

#private
192.168.56.60  rac1-priv   rac1-priv.dbaora.com 
192.168.56.61  rac2-priv   rac2-priv.dbaora.com

#virtual
192.168.0.70   rac1-vip    rac1-vip.dbaora.com
192.168.0.71   rac2-vip    rac2-vip.dbaora.com

#scan
#192.168.0.20   rac-scan    rac-scan.dbaora.com
#192.168.0.21   rac-scan    rac-scan.dbaora.com
#192.168.0.22   rac-scan    rac-scan.dbaora.com

Verify network response on rac1.dbaora.com for eth0 and eth1

[root@rac1 ~]# ping rac1 -c 1
PING rac1.dbaora.com (192.168.0.50) 56(84) bytes of data.
64 bytes from rac1.dbaora.com (192.168.0.50): 
icmp_seq=1 ttl=64 time=0.032 ms

--- rac1.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.032/0.032/0.032/0.000 ms

[root@rac1 ~]# ping rac1-priv -c 1
PING rac1-priv.dbaora.com (192.168.56.60) 56(84) bytes of data.
64 bytes from rac1-priv.dbaora.com (192.168.56.60): 
icmp_seq=1 ttl=64 time=0.036 ms

--- rac1-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.036/0.036/0.036/0.000 ms

Check which packages are installed and which are missing

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE}(%{ARCH})\n' binutils \
compat-libcap1 \
compat-libstdc++-33 \
elfutils-libelf \
elfutils-libelf-devel \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc \
libstdc++ \
libstdc++-devel \
make \
libXext \
libXtst \
libX11 \
libXau \
libxcb \
libXi \
sysstat \
unixODBC \
unixODBC-devel

Install missing packages. It’s just example:

#directory with mounted Oracle Enterprise Linux 6 install disk 

cd <OEL6 disc>/Server/Packages 

#install missed packages (example for package unixODBC*) 

rpm -Uvh unixODBC*

or via internet

yum install unixODBC*

MEMORY_TARGET

During installation you can encounter typical error commonly know by DBAs

ORA-00845: MEMORY_TARGET not supported on this system

Oracle is using for automatic memory management Linux shared segemnts. Usually they are too small but you can modify it on-the-fly. For this presentation we need 2500M.

Just modify entry in “/etc/fstab” to have persistent settings between reboot of your machine

tmpfs   /dev/shm   tmpfs  size=2500M  0 0

remount it

mount -o remount tmpfs

and verify

mount | grep tmpfs
tmpfs on /dev/shm type tmpfs (rw,size=2500M)

Secure Linux

To change secure Linux edit the “/etc/selinux/config” file, making sure the SELINUX flag is set as follows. It requires REBOOT to be effective !!!

SELINUX=permissive

Disable Firewall

Firewall can be disabled in graphical mode or manually.

Option 1 – graphical mode

Run firewall graphical manager System->Administration->Firewall and click “Disable” button. Remember to apply changes File->Apply before quit this manager.

Option 2 – manual mode

Run following command

service iptables stop
chkconfig iptables save

Change NTP

It’s required to disable NTP(Network Time Protocol) or modify its settings so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes.

Option 1 – disable NTP

service ntpd stop
Shutting down ntpd:                                        [  OK  ]
chkconfig ntpd off
mv /etc/ntp.conf /etc/ntp.conf.orig
rm /var/run/ntpd.pid

Option 2 – modify NTP

If you want to keep NTP modify “/etc/sysconfig/ntpd” file to

OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

and restart NTP

# service ntpd restart

Directories

Create ORACLE_BASE

mkdir -p /ora01/app/oracle

Create ORACLE_HOME for grid and database software

mkdir -p /ora01/app/oracle/product/12.1.0/db_1
mkdir -p /ora01/app/grid/product/12.1.0/grid

chown oracle:oinstall -R /ora01
chmod 775 /ora01/app/oracle

New profiles for user oracle

change oracle profile /home/oracle/.bash_profile

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

alias genv='. /home/oracle/.bash_profile_grid;envo'
alias denv='. /home/oracle/.bash_profile_database;envo'

. /home/oracle/.bash_profile_database

envo

create new profile for user oracle /home/oracle/.bash_profile_grid. This profile will be used for grid software.

# Oracle Settings
export TMP=/tmp

export ORACLE_HOSTNAME=rac1.dbaora.com
export ORACLE_UNQNAME=+ASM
export ORACLE_BASE=/ora01/app/oracle
export ORACLE_HOME=/ora01/app/grid/product/12.1.0/grid
export ORACLE_SID=+ASM1

PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;
export CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

alias cdob='cd $ORACLE_BASE'
alias cdoh='cd $ORACLE_HOME'
alias tns='cd $ORACLE_HOME/network/admin'
alias envo='env | grep ORACLE'

create new profile for user oracle /home/oracle/.bash_profile_database. This profile will be used for database software.

# Oracle Settings
export TMP=/tmp

export ORACLE_HOSTNAME=rac1.dbaora.com
export ORACLE_UNQNAME=ORA12C
export ORACLE_BASE=/ora01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/12.1.0/db_1
export ORACLE_SID=ORA12C1

PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;
export CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

alias cdob='cd $ORACLE_BASE'
alias cdoh='cd $ORACLE_HOME'
alias tns='cd $ORACLE_HOME/network/admin'
alias envo='env | grep ORACLE'

umask 022

So next time you do su to oracle user you should have environment prepared for database software

[root@rac1 ~]# su - oracle
ORACLE_UNQNAME=ORA12C
ORACLE_SID=ORA12C1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/12.1.0/db_1

Number of aliases are defined in profiles to quickly managed environment

• genv – sets environment for grid software

alias genv=’. /home/oracle/.bash_profile_grid;envo’

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/12.1.0/grid

• denv – sets environment for database software

alias denv=’. /home/oracle/.bash_profile_database;envo’

[oracle@rac1 ~]$ denv
ORACLE_UNQNAME=ORA12C
ORACLE_SID=ORA12C1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/12.1.0/db_1

• cdob – change directory to $ORACLE_BASE

alias cdob=’cd $ORACLE_BASE’

• cdoh – change directory to $ORACLE_HOME

alias cdoh=’cd $ORACLE_HOME’

• tns – change directory to $ORACLE_HOME/network/admin

alias tns=’cd $ORACLE_HOME/network/admin’

• envo – displays current environments settings and filters by ORACLE word

alias envo=’env | grep ORACLE’

[oracle@rac1 ~]$ envo
ORACLE_UNQNAME=ORA12C
ORACLE_SID=ORA12C1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/12.1.0/db_1

Remember to modify owner of all profiles

chown oracle:oinstall /home/oracle/.bash*
chmod 750 /home/oracle/.bash*

As oracle user unzip database and grid software. It should create 2 directories:

• database – database software
• grid – grid software

--database software
linuxamd64_12c_database_1of2.zip
linuxamd64_12c_database_2of2.zip

--grid software
linuxamd64_12c_grid_1of2.zip
linuxamd64_12c_grid_2of2.zip

[oracle@rac1]$ unzip linuxamd64_12c_database_1of2.zip 
[oracle@rac1]$ unzip linuxamd64_12c_database_2of2.zip
[oracle@rac1]$ unzip linuxamd64_12c_grid_1of2.zip 
[oracle@rac1]$ unzip linuxamd64_12c_grid_2of2.zip 

[oracle@rac1]ls 
database grid

Install package cvuqdisk-1.0.9-1.rpm from grid software as user root

cd <install grid software>/rpm
[root@rac1 rpm]# rpm -Uvh cvuqdisk-1.0.9-1.rpm
Preparing...                ############################## [100%]
Using default group oinstall to install package
   1:cvuqdisk               ############################## [100%]

Configure ASM devices

To configure ASMlib first you need to download ASMLib rpms from OTN.

• oracleasmlib-2.0.4-1.el6.x86_64.rpm

ASM requires also package oracleasm-support-2.1.8-1.el6.x86_64. It should be already installed on OEL6. In case you don’t have it the oracleasm-support package can be downloaded from the Unbreakable Linux Network (ULN) if you have an active support subscription, or from http://public-yum.oracle.com if you do not.

Install the package using the following command

rpm -Uvh oracleasm*.rpm

Verify all is in place

[root@rac1 rpm]# rpm -qa | grep oracleasm
oracleasm-support-2.1.8-1.el6.x86_64
oracleasmlib-2.0.4-1.el6.x86_64

Configure ASMlib

[root@rac1 ~]# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: 
Writing Oracle ASM library driver configuration: done

Load asm kernel module

[root@rac1 rpm]# /usr/sbin/oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size 
Mounting ASMlib driver filesystem: /dev/oracleasm

Add ASM disks

[root@rac1 ~]# /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# /usr/sbin/oracleasm createdisk DISK2 /dev/sdb2
Writing disk header: done
Instantiating disk: done

Scan ASM disks

[root@rac1 ~]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

List ASM disks

[root@rac1 ~]# /usr/sbin/oracleasm listdisks
DISK1
DISK2

Configure dnsmasq for SCAN

1. To install dnsmasq run as root following command

yum install dnsmasq

2. Configure dnsmasq

create new file “/etc/racdns” with settings for SCAN

[root@rac1 ~]# cat /etc/racdns
#scan
192.168.0.20   rac-scan    rac-scan.dbaora.com
192.168.0.21   rac-scan    rac-scan.dbaora.com
192.168.0.22   rac-scan    rac-scan.dbaora.com

modify dnsmasq default configuration file “/etc/dnsmasq.conf”. One parameter addn-hosts should be changed to point to file “/etc/racdns”.

[root@rac1 ~]# cat /etc/dnsmasq.conf | grep addn-hosts
addn-hosts=/etc/racdns

3. Start dnsmasq and set the serice to be started automatically during booting your host

service dnsmasq start
chkconfig dnsmasq on

4. Next step is to resolve problem with file “/etc/resolv.conf”

I’m using third network card as NAT with DHCP so each time you restart network card or reboot host the file is overwritten with automatically generated settings. Nameserver points for 192.168.1.1 which is required to resolve internet entries but not enough to resolve SCAN settings via dnsmasq.

[root@rac1 ~]# cat /etc/resolv.conf
# Generated by Networkmanager
search dbaora.com
nameserver 192.168.1.1

As default dnsmasq is running on ip adress 127.0.0.1 so it requires following settings in “/etc/resolv.conf”.

[root@rac1 ~]# cat /etc/resolv.conf
# Generated by Networkmanager
nameserver 127.0.0.1
search dbaora.com
nameserver 192.168.1.1

You must protect the file from being automatically overwritten by host reboot, network card restart etc.

[root@rac1 ~]# chattr +i /etc/resolv.conf

Verification of DNS

It’s just running nslookup to verify all is working fine

rac-scan

[root@rac1 ~]# nslookup rac-scan
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac-scan.dbaora.com
Address: 192.168.0.22
Name:    rac-scan.dbaora.com
Address: 192.168.0.20
Name:    rac-scan.dbaora.com
Address: 192.168.0.21

[root@rac1 ~]# nslookup rac-scan
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac-scan.dbaora.com
Address: 192.168.0.20
Name:    rac-scan.dbaora.com
Address: 192.168.0.21
Name:    rac-scan.dbaora.com
Address: 192.168.0.22

[root@rac1 ~]# nslookup rac-scan
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac-scan.dbaora.com
Address: 192.168.0.21
Name:    rac-scan.dbaora.com
Address: 192.168.0.22
Name:    rac-scan.dbaora.com
Address: 192.168.0.20

rac1, rac2, rac1-priv, rac2-priv, rac1-vip, rac2-vip

[root@rac1 ~]# nslookup rac1
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac1.dbaora.com
Address: 192.168.0.50

[root@rac1 ~]# nslookup rac2
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac2.dbaora.com
Address: 192.168.0.51

[root@rac1 ~]# nslookup rac1-priv
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac1-priv.dbaora.com
Address: 192.168.56.60

[root@rac1 ~]# nslookup rac2-priv
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac2-priv.dbaora.com
Address: 192.168.56.61

[root@rac1 ~]# nslookup rac1-vip
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac1-vip.dbaora.com
Address: 192.168.0.70

[root@rac1 ~]# nslookup rac2-vip
Server:        127.0.0.1
Address:    127.0.0.1#53

Name:    rac2-vip.dbaora.com
Address: 192.168.0.71

Cloning machine

Before cloning shutdown machine rac1.dbaora.com.

New machine rac2.dbaora.com is cloned from machine rac1.dbaora.com using “linked clone” method. It allows to speed up process of cloning. it’s especially useful for development environments.

• Full clone: In this mode all depending disk images are copied to the new VM folder. The clone can fully operate without the source VM.
• Linked clone: In this mode new differencing disk images are created where the parent disk images are the source disk images. If you selected the current state of the source VM as clone point, a new snapshot will be created implicitly.

1. Before we create “linked clone” we need to remove shared storage. On Storage tab select Shared.vdi then click “Remove Attachment”.

2. Now you should see oel6_121_rac1 without Shared storage.

3. In menu of VirtualBox select “Machine”-“Clone” option. Enter name for new machine oel6_121_rac2 and click “Next” button.

4. Select “Linked clone” option and click “Clone” button.

5. You should see new machine oel6_121_rac2

6. It’s time to attach Shared.vdi disk to both machine. Select first machine oel6_rac121_rac1 then click “Storage”. Select “Controller:SATA” and click last button “Add Hard Disk”.

7. Click “Choose existing disk”. You should see standard window to select a file. Select “Shared.vdi” to attach it to machine.

8. Once it’s done shared disk should be visible by machine oel6_121_rac1. Click “OK” button to save your configuration.

9. You need to repeat the same steps 6,7 to attach shared disk to machine oel6_121_rac2. Don’t be surprised that both machines are using the same disk “oel6_121_arc1.vdi”. New machine oel6_121_rac2 is linked clone.

Network settings for second machine

Start second linked machine oel6_121_rac2 and set new static ip address for eth0 and eth1.

• 192.168.0.51/255.255.255.0 for eth0 (public)
• 192.168.56.61/255.255.255.0 for eth1 (private)

and change host name to

• rac2.dbaora.com

1. Change hostname

Modify “/etc/sysconfig/network” file

NETWORKING=yes
HOSTNAME=rac2.dbaora.com

2. Start “System”->”Preferences”->”Network Connections” tool to modify ip-addresses of your network cards.

3. Modify entries in profile of user oracle .bash_profile_database, .bash_profile_grid on machine oel2_121_rac2

ORACLE_HOSTNAME=rac2.dbaora.com
ORACLE_SID=+ASM2
ORACLE_SID=ORA12C2

/home/oracle/.bash_profile_grid

# Oracle Settings
export TMP=/tmp

export ORACLE_HOSTNAME=rac2.dbaora.com
export ORACLE_UNQNAME=+ASM
export ORACLE_BASE=/ora01/app/oracle
export ORACLE_HOME=/ora01/app/grid/product/12.1.0/grid
export ORACLE_SID=+ASM2

PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;
export CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

alias cdob='cd $ORACLE_BASE'
alias cdoh='cd $ORACLE_HOME'
alias tns='cd $ORACLE_HOME/network/admin'
alias envo='env | grep ORACLE'

/home/oracle/.bash_profile_database

# Oracle Settings
export TMP=/tmp

export ORACLE_HOSTNAME=rac2.dbaora.com
export ORACLE_UNQNAME=ORA12C
export ORACLE_BASE=/ora01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/12.1.0/db_1
export ORACLE_SID=ORA12C2

PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib;
export CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

alias cdob='cd $ORACLE_BASE'
alias cdoh='cd $ORACLE_HOME'
alias tns='cd $ORACLE_HOME/network/admin'
alias envo='env | grep ORACLE'

umask 022

Once it’s done reboot machine oel6_121_rac2 and start machine oel6_121_rac1

Verify network between nodes

It’s good time to verify network on each node: rac1.dbaora.com and rac2.dbaora.com

hostname
ping rac1 -c 1
ping rac2 -c 1
ping rac1-priv -c 1
ping rac2-priv -c 1

test on rac1.dbaora.com

[root@rac1 ~]# hostname
rac1.dbaora.com
[root@rac1 ~]# ping rac1 -c 1
PING rac1.dbaora.com (192.168.0.50) 56(84) bytes of data.
64 bytes from rac1.dbaora.com (192.168.0.50): icmp_seq=1 ttl=64 time=0.035 ms

--- rac1.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.035/0.035/0.035/0.000 ms
[root@rac1 ~]# ping rac2 -c 1
PING rac2.dbaora.com (192.168.0.51) 56(84) bytes of data.
64 bytes from rac2.dbaora.com (192.168.0.51): icmp_seq=1 ttl=64 time=0.318 ms

--- rac2.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.318/0.318/0.318/0.000 ms
[root@rac1 ~]# ping rac1-priv -c 1
PING rac1-priv.dbaora.com (192.168.56.60) 56(84) bytes of data.
64 bytes from rac1-priv.dbaora.com (192.168.56.60): icmp_seq=1 ttl=64 time=0.025 ms

--- rac1-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.025/0.025/0.025/0.000 ms
[root@rac1 ~]# ping rac2-priv -c 1
PING rac2-priv.dbaora.com (192.168.56.61) 56(84) bytes of data.
64 bytes from rac2-priv.dbaora.com (192.168.56.61): icmp_seq=1 ttl=64 time=0.352 ms

--- rac2-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.352/0.352/0.352/0.000 ms

test on rac2.dbaora.com

[root@rac2 ~]# hostname
rac2.dbaora.com
[root@rac2 ~]# ping rac1 -c 1
PING rac1.dbaora.com (192.168.0.50) 56(84) bytes of data.
64 bytes from rac1.dbaora.com (192.168.0.50): icmp_seq=1 ttl=64 time=0.342 ms

--- rac1.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.342/0.342/0.342/0.000 ms
[root@rac2 ~]# ping rac2 -c 1
PING rac2.dbaora.com (192.168.0.51) 56(84) bytes of data.
64 bytes from rac2.dbaora.com (192.168.0.51): icmp_seq=1 ttl=64 time=0.039 ms

--- rac2.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.039/0.039/0.039/0.000 ms
[root@rac2 ~]# ping rac1-priv -c 1
PING rac1-priv.dbaora.com (192.168.56.60) 56(84) bytes of data.
64 bytes from rac1-priv.dbaora.com (192.168.56.60): icmp_seq=1 ttl=64 time=0.312 ms

--- rac1-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.312/0.312/0.312/0.000 ms
[root@rac2 ~]# ping rac2-priv -c 1
PING rac2-priv.dbaora.com (192.168.2.61) 56(84) bytes of data.
64 bytes from rac2-priv.dbaora.com (192.168.2.61): icmp_seq=1 ttl=64 time=0.055 ms

--- rac2-priv.dbaora.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.055/0.055/0.055/0.000 ms

Grid software installation

Start grid software installation as user oracle, remember to run command xhost+ as root.

xhost +
access control disabled, clients can connect from any host

Set grid environment and run grid installation software

su - oracle

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/12.1.0/grid

--run installation
cd <install grid software>
./runInstall

1. Select “Skip software updates” and click “Next” button.

2. Select “Install and Configure Oracle Grid Infrastructure for a Cluster” and click “Next” button. Then select “Configure a Standard Cluster” and click “Next” button.

3. Accept default “Typical installation” and click “Next” button

4. Enter SCAN Name “rac-scan” then click “Add” button to add second node rac2

5. Enter public and virtual host name for second node and click “OK” button

6.Click on “Identify network interfaces” button

7. Ensure eth0 is public and eth1 is private interface type and click “OK” button

8. It’s time to establish SSH connection between cluster nodes. Click “SSH Connectivity” button then enter OS Password for user oracle and click “Setup” button. Once configuration is done click “Next” button

9. Select “Oracle Automatic Storage Management” and enter SYSASM user password change group OSASM to DBA and click “Next”

10. You should see available devices. Select first one “ORCL:DISK1” change Redundancy to “External” then click “Next” button

11. Accept default directory for “Inventory Directory” and click “Next” button

12. Check checkbox “Automatically run configuration scripts” and enter root user password then click “Next” button

13. Checks are executed to show any errors that need to be fixed before installation can proceed. In case you have errors you need to fix them. If all is well you should see summary window.

14. In my case one error has appeared which can be safely ignored. Oracle requires 4GB memory which was assigned but 128MB was allocated for graphics so errors is raised. So check checkbox “Ignore All” then click “Next” button.

15. Click “Yes” button to proceed with installation

16. Summary window. It’s the last moment to go back and make modifications. Click “Install” to proceed with installation

17. It will take some time to install software so be patient

18. In the end of installation you will be asked to confirm scripts to run scripts as user root. Click “Yes” button.

19. Installation should continue …

20. Installation of grid software is completed so click “Close” button

ASM configuration assistant

So far we have created only one ASM group DATA. I want to show you how quickly you can add extra group to ASM instance.

As user oracle start ASMCA. Remeber to start xhost + as user root to enable run graphical tools from other users like oracle.

ASMCA – ASM configuration assistant is part of grid software so proper environment must be set

su - oracle

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/12.1.0/grid

[oracle@rac1 ~]$ asmca

1. ASMCA started click “Disk Groups” tab.

2. On tab “Disk Groups” click “Create” button to add new ASM diskgroup.

3. Enter group name BACKUP, check checkbox for “ORCL:DISK2″ device and click “OK” button. Redundancy must be “External”.

4. You should see following screen. Click “OK”.

4. Now you should see we have 2 ASM groups “BACKUP” and “DATA”. Click “Exit” button.

Now you can verify in sqlplus that you have 2 ASM groups.

[oracle@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Sat Nov 2 01:02:02 2013

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> select name from v$asm_diskgroup;

NAME
------------------------------
BACKUP
DATA

Database software installation

Let’s start with database software installation as user oracle. Set environment to database software

su - oracle

[oracle@rac1 ~]$ denv
ORACLE_UNQNAME=ORA12C
ORACLE_SID=ORA12C1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/oracle/product/12.1.0/db_1
--run installation 
cd <install database software>
./runInstall

1. Uncheck checkbox “I wish to receive security updates via My Oracle Support” and click “Next” button.

2. Ignore this message and click “Yes”.

3. Select “Skip software updates” and click “Next” button.

4. Select “Create and configure a database” and click “Next” button.

5. Select “Server Class” and click “Next” button.

6. Select “Oracle Real Application Cluster database installation” and click “Next” button.

7.Accept default “Admin managed” and click “Next button.

8. Make sure both nodes are selected, then click the “Next” button.

9. Select “Typical install” and click “Next” button.

10. On this page you need to specify directory for database software, Storage type “Oracle Automatic Storage Management” with database diskgroup “DATA” then global name for your database “ORA12C”, password to Administer database and pluggable database “PORA12C”. Once it’s done click “Next” button.

11. Prerequisites checks are executed to verify cluster settings.

12. Summary window. It’s the last moment to make changes before starting installation. Click “Install” button to continue.

13. Just wait installation in progress …

14. In the end of binary installation you need to run script on each node as user root. then click “OK” to continue with installation.

/ora01/app/oracle/product/12.1.0/db_1/root.sh

15. After binaries installation is completed DBCA is started automatically to create your new shiny database.

16. You should see database installation summary window

17. installation is completed click “Next” button then “Close” button.

Verify RAC installation

[root@rac1 ~]# su - oracle

[oracle@rac1 ~]$ genv
ORACLE_UNQNAME=+ASM
ORACLE_SID=+ASM1
ORACLE_BASE=/ora01/app/oracle
ORACLE_HOSTNAME=rac1.dbaora.com
ORACLE_HOME=/ora01/app/grid/product/12.1.0/grid
[oracle@rac1 ~]$ srvctl config database -d ORA12C
Database unique name: ORA12C
Database name: ORA12C
Oracle home: /ora01/app/oracle/product/12.1.0/db_1
Oracle user: oracle
Spfile: +DATA/ORA12C/spfileORA12C.ora
Password file: +DATA/ORA12C/orapwora12c
Domain: 
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: ORA12C
Database instances: ORA12C1,ORA12C2
Disk Groups: DATA
Mount point paths: 
Services: 
Type: RAC
Start concurrency: 
Stop concurrency: 
Database is administrator managed

[oracle@rac1 ~]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): rac1,rac2

[oracle@rac1 ~]$ srvctl status asm
ASM is running on rac2,rac1

[oracle@rac1 ~]$ srvctl status database -d ORA12C
Instance ORA12C1 is running on node rac1
Instance ORA12C2 is running on node rac2

Status of your database can be visible in EM database express https://rac1.dbaora.com:5500/em/

Have a fun 🙂

Tomasz