Step by Step Installing Oracle 12c GI and RAC on Linux 7 Using VirtualBox

Introduction

This article describes the detailed step by step installation of Oracle Database 12c  (12.1.0.2 64-bit) GI and RAC on Linux (Oracle Linux 7.1 64-bit) using VirtualBox (5.0.10).

Contents

  • System Specifications
  • Download Software
  • Setup VirtualBox
  • OS Installation
  • Configure Bind DNS
  • Prepare Linux for Oracle
  • Create Virtualbox Shared Storage
  • Clone VirtualBox VM
  • Setup Second Linux VM
  • Configure Second Linux VM
  • VNC Server Setup
  • Install 12c Grid Infrastructure (ASM)
  • Install RAC Database Software
  • Create New Disks and Create New Diskgroup FRA
  • Create 12c RAC Database

System Specifications

Windows 10 64 bit
Capture

Download Software

Download the following software:

Capture

Setup VirtualBox

  1.   Start VirtualBox Manager.
  2.   From the main screen, choose File > Preferences > Network.

Capture3.  Double click on VirtualBox Host-Only Ethernet Adapter. Update settings as shown and click Ok twice.

Pv4 Address: 192.168.78.1

IPv4 Network Mask: 255.255.255.0

Capture

On OS X and Linux the VirtualBox Host-Only Ethernet Adapter is called vboxnet0.

4.  On your computer, create a new folder virtuallab on a location of your choice that will contain all the virtual disks.

5.  In VirtualBox, from the main screen, click the New icon in the upper left hand corner. Type in racnode1 for the Name of the VM. Choose Linux for the Type and Oracle (64 bit) for the Version and click Next.

Capture

6. Type 4096 in the Size field and click Next.

Untitled

7. Leave Create a virtual hard drive now selected and click Create.

 

Untitled

8.  Leave VDI (VirtualBox Disk Image) selected and click Next.

Untitled

9.  Leave Dynamically allocated selected and click Next.

Untitled

10.  Type in the full path or choose through the browse icon the Location for the disk file to be created. It’s better to use the common folder virtuallab previously created for all the virtual disks.

The file should be named racnode1.vdi.  Type in 30.00 GB in the Size field and click Create.

Capture

11. From the main screen, select the virtual machine racnode1 and click the Settings icon in the upper left hand corner.

Untitled

12. Open the USB sub-menu. Uncheck the Enable USB Controller check box.

Untitled

13.  Open the Storage sub-menu. Delete the Empty disk under the IDE Controller.

Untitled

14.  Click REMOVE.

Untitled

15.  Click on the Add Optical Device.

Untitled

16.   Click on the Choose disk.

Untitled

17.   Locate and open the v74844-01.iso file.

Untitled

18.  Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: drop down to Host-only Adapter.

Untitled

19. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to: drop down to Internal Network and type the name rac-priv in the Name field.

Untitled

20.  Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the Attached to: drop down to NAT.

Untitled

21. Select the Shared Folders tab. Click Add Shared Folder.

Enter the path where you have downloaded the Oracle installation media and enter a name for your folder.

Enter the Folder Name

Check Auto-mount and click OK.

Untitled

22. Review the summary of the new virtual machine.

Untitled

OS Installation

  1.  Click on Start to boot the virtual machine. If asked to select a start-up disk, choose V74844-01.iso and click Start again.Untitled

Untitled

2.  Leave English (English) selected and click Continue.

Untitled

Untitled

3.  In the hostname field, type racnode1.virtuallab

  • Click Configure Network in the bottom left corner.
  • Highlight enpOs3 and click the Configure
  • Click the Connect automatically
  • Click the IPv4 Settings tab
  • Click the Method: drop down and select Manual.
  • Click Add and type 192.168.78.51 for the address.
  • Leave Netmask at 24.
  • In the DNS servers: box, type 192.168.78.51, 192.168.78.52.
  • In the Search domains: box, type virtuallab.
  • Click the Save.

Untitled

4.  Highlight System EnpOs9 and click the Configure… button.

  • Click the Connect automatically
  • Click the IPv4 Settings 
  • Click the Method: drop down and select Automatic (DHCP) addresses only.
  • Click the Apply
  • Click Close to close the network configuration menu and click Next.

Untitled

Untitled

5.  Click Begin Installation.

Untitled

Untitled

Untitled

6.  To connect to the new created virtual machine, create a connection in Putty for the new VM called racnode1 with the IP Address of 192.168.78.51.

Untitled

Untitled

7.  Turn off and disable the firewall IPTables.

You must disable IPTABLES. Missing this step and then DNS does not work.
[root@racnode1 ~]# service iptables stop
Redirecting to /bin/systemctl stop  iptables.service
Failed to issue method call: Unit iptables.service not loaded.
[root@racnode1 ~]# chkconfig iptables off
error reading information on service iptables: No such file or directory
[root@racnode1 ~]#
[root@racnode1 ~]# chkconfig --list iptables
[root@racnode1 ~]#

8. Disable SELinux. Open the config file and change the SELINUX variable from enforcing to disabled.

[root@racnode1 ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

[root@racnode1 ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.

SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

[root@racnode1 ~]# vi /etc/selinux/config
[root@racnode1 ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.

SELINUXTYPE=targeted

9. Verify that all the network interfaces are up.

[root@racnode1 ~]# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 08:00:27:2d:25:e6 brd ff:ff:ff:ff:ff:ff

3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000

    link/ether 08:00:27:f8:0e:13 brd ff:ff:ff:ff:ff:ff

4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000

   link/ether 08:00:27:27:ed:e6 brd ff:ff:ff:ff:ff:ff

10. Install the database pre-install package oracle-rdbms-server-12cR1-preinstall.

    a) To download this RPM from your OL7 machine you first need to modify the default yum configuration file. Modify /etc/repos.d/public-yum-ol7.repo to enable ol7_addons repository: set enabled=1 instead of 0 , It was 0 initially enabled=0

[ol7_addons]
name=Oracle Linux $releasever Add ons ($basearch)
baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL7/addons/
basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

    b) #yum update
    c) #yum -y install oracle-rdbms-server-12cR1-preinstall

[root@racnode1~]# yum install oracle-rdbms-server-12cR1-preinstall

ol7_UEKR3                  | 1.2 kB  00:00:00
ol7_latest                 | 1.4 kB  00:00:00

(1/5): ol7_UEKR3/x86_64/updateinfo          |  57 kB  00:00:00
(2/5): ol7_latest/x86_64/updateinfo         | 439 kB  00:00:01
(3/5): ol7_latest/x86_64/group              | 652 kB  00:00:03
(4/5): ol7_latest/x86_64/primary            |  12 MB  00:00:23
(5/5): ol7_UEKR3/x86_64/primary             |  13 MB  00:00:26

ol7_UEKR3                                    327/327
ol7_latest                                 10502/10502

Resolving Dependencies
--> Running transaction check
---> Package oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-4.el7 will be installed
--> Processing Dependency: sysstat for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: xorg-x11-utils for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: bind-utils for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: gcc-c++ for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: compat-libcap1 for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: ksh for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: libaio-devel for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: xorg-x11-xauth for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: psmisc for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: compat-libstdc++-33 for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64
--> Processing Dependency: libstdc++-devel for package: oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64

......
......
......
......

---> Package libICE.x86_64 0:1.0.8-7.el7 will be installed
---> Package libSM.x86_64 0:1.2.1-7.el7 will be installed
---> Package libpath_utils.x86_64 0:0.2.1-24.el7 will be installed
---> Package libtalloc.x86_64 0:2.1.1-1.el7 will be installed
---> Package libtevent.x86_64 0:0.9.21-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================
 Package                                                        Arch                             Version                                          Repository                            Size

 oracle-rdbms-server-12cR1-preinstall                           x86_64                           1.0-4.el7                                        ol7_latest                            18 k

Installing for dependencies:

bind-libs                                                      x86_64                           32:9.9.4-18.el7_1.5                              ol7_latest                           1.0 M
bind-utils                                                     x86_64                           32:9.9.4-18.el7_1.5                              ol7_latest                           198 k
compat-libcap1                                                 x86_64                           1.10-7.el7                                       ol7_latest                            17 k
compat-libstdc++-33                                            x86_64                           3.2.3-72.el7                                     ol7_latest                           190 

.....
......
......
......
......

 xorg-x11-utils                                                 x86_64                           7.5-13.1.el7                                     ol7_latest                           110 k xorg-x11-xauth                                                 x86_64                           1:1.0.7-6.1.el7                                  ol7_latest                            28 k

Updating for dependencies:

bind-libs-lite                                                 x86_64                           32:9.9.4-18.el7_1.5                              ol7_latest                           713 k

bind-license                                                   noarch                           32:9.9.4-18.el7_1.5                              ol7_latest                            80 k

Transaction Summary

=============================================================================================================================================================================================

Install  1 Package  (+52 Dependent packages)
Upgrade             (  2 Dependent packages)
Total download size: 16 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
warning: /var/cache/yum/x86_64/7Server/ol7_latest/packages/bind-libs-lite-9.9.4-18.el7_1.5.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY550 kB/s | 913 kB  00:00:27 ETA

Public key for bind-libs-lite-9.9.4-18.el7_1.5.x86_64.rpm is not installed
(1/55): bind-libs-lite-9.9.4-18.el7_1.5.x86_64.rpm                                                                                                                    | 713 kB  00:00:01
(2/55): bind-license-9.9.4-18.el7_1.5.noarch.rpm                                                                                                                      |  80 kB  00:00:00
(3/55): bind-utils-9.9.4-18.el7_1.5.x86_64.rpm                                                                                                                        | 198 kB  00:00:00
(4/55): bind-libs-9.9.4-18.el7_1.5.x86_64.rpm                                                                                                                         | 1.0 MB  00:00:02
(5/55): compat-libcap1-1.10-7.el7.x86_64.rpm                                                                                                                          |  17 kB  00:00:00
......
......
......
......
......
......


(51/55): rpcbind-0.2.0-26.el7.x86_64.rpm                                                                                                                              |  55 kB  00:00:00
(52/55): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                           |  78 kB  00:00:00
(53/55): sysstat-10.1.5-7.el7.x86_64.rpm                                                                                                                              | 295 kB  00:00:00
(54/55): xorg-x11-utils-7.5-13.1.el7.x86_64.rpm                                                                                                                       | 110 kB  00:00:00
(55/55): xorg-x11-xauth-1.0.7-6.1.el7.x86_64.rpm                                                                                                                      |  28 kB  00:00:00

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                        647 kB/s |  16 MB  00:00:25
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:

 Userid     : "Oracle OSS group (Open Source Software group) <build@oss.oracle.com>"

Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03

Package    : 7:oraclelinux-release-7.1-1.0.5.el7.x86_64 (@anaconda/7.1)

From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle

Is this ok [y/N]: y

Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libXau-1.0.8-2.1.el7.x86_64                                                                                                                                              1/57
  Installing : libxcb-1.9-5.el7.x86_64                                                                                                                                                  2/57
  Updating   : 32:bind-license-9.9.4-18.el7_1.5.noarch                                                                                                                                  3/57
  Installing : libICE-1.0.8-7.el7.x86_64                                                                                                                                                4/57
  ......
......
......
......
......
.......
......

 Installing : oracle-rdbms-server-12cR1-preinstall-1.0-4.el7.x86_64                                                                                                                   54/57
Updating   : 32:bind-libs-lite-9.9.4-18.el7_1.5.x86_64                                                                                                                               55/57
Cleanup    : 32:bind-libs-lite-9.9.4-18.el7.x86_64                                                                                                                                   56/57  
Cleanup    : 32:bind-license-9.9.4-18.el7.noarch                                                                                                                                     57/57
Verifying  : libXxf86misc-1.0.3-7.1.el7.x86_64                                                                                                                                        1/57
Verifying  : libdmx-1.1.3-3.el7.x86_64                                                                                                                                                2/57
Verifying  : libXmu-1.1.1-5.1.el7.x86_64                                                                                                                                              3/57

Verifying  : libXinerama-1.1.3-2.1.el7.x86_64                                                                                                                                         4/57

Verifying  : libtalloc-2.1.1-1.el7.x86_64                                                                                                                                             5/57

 ......
......
......
......
......
......
                                                                                                                         
Verifying  : 32:bind-libs-lite-9.9.4-18.el7.x86_64                                                                                                                                   57/57

Installed:

oracle-rdbms-server-12cR1-preinstall.x86_64 0:1.0-4.el7

Dependency Installed:

  bind-libs.x86_64 32:9.9.4-18.el7_1.5          bind-utils.x86_64 32:9.9.4-18.el7_1.5         compat-libcap1.x86_64 0:1.10-7.el7            compat-libstdc++-33.x86_64 0:3.2.3-72.el7

  gcc-c++.x86_64 0:4.8.3-9.el7                  gssproxy.x86_64 0:0.3.0-10.el7                keyutils.x86_64 0:1.5.8-3.el7                 ksh.x86_64 0:20120801-22.el7_1.3

  libICE.x86_64 0:1.0.8-7.el7                   libSM.x86_64 0:1.2.1-7.el7                    libX11.x86_64 0:1.6.0-2.1.el7                 libX11-common.noarch 0:1.6.0-2.1.el7

.....
......
......

Dependency Updated:

  bind-libs-lite.x86_64 32:9.9.4-18.el7_1.5                                                      bind-license.noarch 32:9.9.4-18.el7_1.5

Complete!

[root@racnode1 ~]#

You can find a detailed log for these steps in /var/log/oracle-rdbms-server-12cR1-preinstall/backup/<timestamp>/orakernel.log:

11.  Install additional RPMs that will be used to install and administer the servers.

[root@racnode1 ~]# yum install -y tigervnc-server.x86_64 xclock man parted.x86_64 unzip.x86_64 xterm lsof bind  xorg-x11-twm

12. Uninstall NTP.

[root@racnode1 ~]#yum remove -y ntp

No Match for argument: ntp

No Packages marked for removal

[root@racnode1 ~]#

13.  Install VirtualBox Addons

[root@racnode1 cdrom]# pwd

/media/cdrom

[root@racnode1 mnt]# ./VBoxLinuxAdditions.run

Verifying archive integrity... All good.

Uncompressing VirtualBox 5.0.10 Guest Additions for Linux............

VirtualBox Guest Additions installer
Removing installed version 5.0.10 of VirtualBox Guest Additions...
Removing existing VirtualBox non-DKMS kernel modules[  OK  ]

Copying additional installer modules ...

Installing additional modules ...
Removing existing VirtualBox non-DKMS kernel modules[  OK  ]
Building the VirtualBox Guest Additions kernel modules
Building the main Guest Additions module[  OK  ]
Building the shared folder support module[  OK  ]
Building the OpenGL support module[  OK  ]
Doing non-kernel setup of the Guest Additions[  OK  ]
Starting the VirtualBox Guest Additions [  OK  ]
Installing the Window System drivers
Could not find the X.Org or XFree86 Window System, skipping.

[root@racnode1 mnt]# cd /mnt

[root@racnode1 mnt]# ls -ltr
total 51259
-r-xr-xr-x 1 root root      647 Aug 20 21:47 AUTORUN.INF
dr-xr-xr-x 2 root root     2048 Nov 11 04:02 OS2
dr-xr-xr-x 2 root root     2048 Nov 11 04:02 cert
dr-xr-xr-x 2 root root     2048 Nov 11 04:02 64Bit
dr-xr-xr-x 2 root root     2048 Nov 11 04:02 32Bit
-r-xr-xr-x 1 root root   316016 Nov 11 04:56 VBoxWindowsAdditions.exe
-r-xr-xr-x 1 root root     5519 Nov 11 04:57 runasroot.sh
-r-xr-xr-x 1 root root     6964 Nov 11 04:57 autorun.sh
-r-xr-xr-x 1 root root 10244320 Nov 11 04:57 VBoxWindowsAdditions-x86.exe
-r-xr-xr-x 1 root root  7515597 Nov 11 04:57 VBoxLinuxAdditions.run
-r-xr-xr-x 1 root root 16949040 Nov 11 05:02 VBoxWindowsAdditions-amd64.exe
-r-xr-xr-x 1 root root 17440768 Nov 11 05:58 VBoxSolarisAdditions.pkg

[root@racnode1 mnt]#

Configure Bind DNS

 1. Enable BIND DNS to start at boot time.

[root@racnode1 ~]# chkconfig named on

Note: Forwarding request to 'systemctl enable named.service'.

ln -s '/usr/lib/systemd/system/named.service' '/etc/systemd/system/multi-user.target.wants/named.service'

2.  Change named directory permissions

[root@racnode1 ~]# ls -ltr /var/named

total 16
-rw-r-----. 1 root  named  152 Jun 21  2007 named.localhost
-rw-r-----. 1 root  named  168 Dec 15  2009 named.loopback
-rw-r-----. 1 root  named  152 Dec 15  2009 named.empty
-rw-r-----. 1 root  named 2076 Jan 29  2013 named.ca

drwxrwx---. 2 named named    6 Sep  3 13:25 slaves
drwxrwx---. 2 named named    6 Sep  3 13:25 dynamic
drwxrwx---. 2 named named    6 Sep  3 13:25 data

[root@racnode1 ~]# touch /var/named/virtuallab
[root@racnode1 ~]# chmod 664 /var/named/virtuallab
[root@racnode1 ~]# chgrp named /var/named/virtuallab
[root@racnode1 ~]# chmod g+w /var/named
[root@racnode1 ~]# chmod g+w /var/named/virtuallab
[root@racnode1 ~]#

3.  Backup the BIND configuration file.

  [root@racnode1 ~]# ls -ltr /etc/named.conf

-rw-r-----. 1 root named 1582 Oct 30  2013 /etc/named.conf

[root@racnode1 ~]# ls -ltr /etc/named.conf*

-rw-r-----. 1 root named 1582 Oct 30  2013 /etc/named.conf
[root@racnode1 ~]# cp /etc/named.conf /etc/named.conf.org
[root@racnode1 ~]#

4. Change /etc/named.conf permissions.

[root@racnode1 ~]# chmod 664 /etc/named.conf

[root@racnode1 ~]#

Otherwise, the original protection may cause trouble in the restarting named step with write-protection errors in /var/log/messages. 

Run the following command or edit the /etc/named.conf file to change the named configuration manually

sed -i -e 's/listen-on .*/listen-on port 53 { 192.168.78.51; };/' \
-e 's/allow-query .*/allow-query     { 192.168.78.0\/24; localhost; };\n        allow-transfer  { 192.168.78.0\/24; };/' \
-e '$azone "virtuallab" {\n  type master;\n  file "virtuallab";\n};\n\nzone "in-addr.arpa" {\n  type master;\n  file "in-addr.arpa";\n};' \
/etc/named.conf 

options {       listen-on port 53 { 192.168.78.51; };       
listen-on-v6 port 53 { ::1; };       
directory       "/var/named";       
dump-file       "/var/named/data/cache_dump.db";       
statistics-file "/var/named/data/named_stats.txt";       
memstatistics-file "/var/named/data/named_mem_stats.txt";       
allow-query     { 192.168.78.0/24; localhost; };       
allow-transfer  { 192.168.78.0/24; };       
recursion yes;        

dnssec-enable yes;       
dnssec-validation yes;       
dnssec-lookaside auto;        

/* Path to ISC DLV key */       
bindkeys-file "/etc/named.iscdlv.key";        

managed-keys-directory "/var/named/dynamic";
}; 

logging {       channel default_debug {               
                  file "data/named.run";               
                       severity dynamic;       
        };
}; 

zone "." IN {       
       type hint;       
       file "named.ca"
;}; 

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key"; 

zone "virtuallab" {
 type master; 
 file "virtuallab";
}; 

zone "in-addr.arpa" { 
 type master; 
 file "in-addr.arpa";
};

5.  Create the zone file for the virtuallab domain on racnode1 by running the following command:

echo '$TTL 3H
@       IN SOA  racnode1        hostmaster      (
                                        101   ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
                NS      racnode1
                NS      racnode2
localhost       A       127.0.0.1
racnode1        A       192.168.78.51
racnode1-vip    A       192.168.78.61
racnode1-priv   A       172.16.100.51
racnode2        A       192.168.78.52
racnode2-vip    A       192.168.78.62
racnode2-priv   A       172.16.100.52
racnode-cluster-scan     A       192.168.78.251
racnode-cluster-scan     A       192.168.78.252
racnode-cluster-scan     A       192.168.78.253' \
> /var/named/virtuallab

[root@racnode1 etc]# cat /var/named/virtuallab

$TTL 3H
@       IN SOA  racnode1        hostmaster      (
                                        101   ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
                NS      racnode1
                NS      racnode2
localhost       A       127.0.0.1
racnode1        A       192.168.78.51
racnode1-vip    A       192.168.78.61
racnode1-priv   A       172.16.100.51
racnode2        A       192.168.78.52
racnode2-vip    A       192.168.78.62
racnode2-priv   A       172.16.100.52
racnode-cluster-scan     A       192.168.78.251
racnode-cluster-scan     A       192.168.78.252
racnode-cluster-scan     A       192.168.78.253
[root@racnode1 etc]#

6.  Create the reverse zone file racnode1.

Copy and paste below command as root:

echo '$TTL 3H
@       IN SOA  racnode1.virtuallab.      hostmaster.virtuallab. (
                                        101   ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
                NS      racnode1.virtuallab.
                NS      racnode2.virtuallab.

51.78.168.192   PTR     racnode1.virtuallab.
61.78.168.192   PTR     racnode1-vip.virtuallab.
51.100.16.172   PTR     racnode1-priv.virtuallab.
52.78.168.192   PTR     racnode2.virtuallab.
62.78.168.192   PTR     racnode2-vip.virtuallab.
52.100.16.172   PTR     racnode2-priv.virtuallab.
251.78.168.192  PTR     racnode-cluster-scan.virtuallab.
252.78.168.192  PTR     racnode-cluster-scan.virtuallab.
253.78.168.192  PTR     racnode-cluster-scan.virtuallab.' \
> /var/named/in-addr.arpa

[root@racnode1 etc]# cat /var/named/in-addr.arpa
cat: /var/named/in-addr.arpa: No such file or directory

[root@racnode1 etc]# echo '$TTL 3H
> @     IN SOA  racnode1.virtuallab.     hostmaster.virtuallab.  (
>                                         101   ; serial
>                                         1D      ; refresh
>                                         1H      ; retry
>                                         1W      ; expire
>                                         3H )    ; minimum
>                 NS      racnode1.virtuallab.
>                 NS      racnode2.virtuallab.
> 
> 51.78.168.192   PTR     racnode1.virtuallab.
> 61.78.168.192   PTR     racnode1-vip.virtuallab.
> 51.100.16.172   PTR     racnode1-priv.virtuallab.
> 52.78.168.192   PTR     racnode2.virtuallab.
> 62.78.168.192   PTR     racnode2-vip.virtuallab.
> 52.100.16.172   PTR     racnode2-priv.virtuallab.
> 251.78.168.192  PTR     racnode-cluster-scan.virtuallab.
> 252.78.168.192  PTR     racnode-cluster-scan.virtuallab.
> 253.78.168.192  PTR     racnode-cluster-scan.virtuallab.' \
> > /var/named/in-addr.arpa

[root@racnode1 etc]# cat /var/named/in-addr.arpa

$TTL 3H
@       IN SOA  racnode1.virtuallab.      hostmaster.virtuallab. (
                                        101   ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
                NS      racnode1.virtuallab.
                NS      racnode2.virtuallab.

51.78.168.192   PTR     racnode1.virtuallab.
61.78.168.192   PTR     racnode1-vip.virtuallab.
51.100.16.172   PTR     racnode1-priv.virtuallab.
52.78.168.192   PTR     racnode2.virtuallab.
62.78.168.192   PTR     racnode2-vip.virtuallab.
52.100.16.172   PTR     racnode2-priv.virtuallab.
251.78.168.192  PTR     racnode-cluster-scan.virtuallab.
252.78.168.192  PTR     racnode-cluster-scan.virtuallab.
253.78.168.192  PTR     racnode-cluster-scan.virtuallab.

[root@racnode1 etc]#

7.  Generate the rndc.key file.

[root@racnode1 etc]# rndc-confgen -a -r /dev/urandom
wrote key file "/etc/rndc.key"
[root@racnode1 etc]# chgrp named /etc/rndc.key
[root@racnode1 etc]# chmod g+r /etc/rndc.key
[root@racnode1 etc]# ls -lrta /etc/rndc.key
-rw-r----- 1 root named 77 Nov 24 21:50 /etc/rndc.key

8.  Check that the parameter PEERDNS is set to no in /etc/sysconfig/network-scripts/ifcfg- -enp0s9 to prevent the resolv.conf from being overwritten by the dhcp client, also make sure  DEFROUTE=no, PEERROUTES=no

[root@racnode1 ]#cat /etc/sysconfig/network-scripts/ifcfg-enp0s9

TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=no
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=enp0s9
UUID=0c76e053-3487-479b-bf5f-18bb26b4d154
DEVICE=enp0s9
ONBOOT=yes
PEERDNS=no
PEERROUTES=no
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_PRIVACY=no

[root@racnode1 etc]#

9.  Restart the named service.

[root@racnode1 etc]# service named restart
Redirecting to /bin/systemctl restart  named.service
[root@racnode1 etc]#

10. Restart network service.

[root@racnode1 etc]# service network restart
Restarting network (via systemctl):                      [  OK  ]

11.  Change /etc/resolv.conf and check nslookup nodes and SCAN ips working fine.

root@racnode1 ~]# cat /etc/resolv.conf

# Generated by NetworkManager
search gateway virtuallab
nameserver 192.168.78.51
nameserver 192.168.78.52
nameserver 10.0.0.138

[root@racnode1 ~]# nslookup racnode1
Server:         192.168.78.51
Address:        192.168.78.51#53

Name:   racnode1.virtuallab
Address: 192.168.78.51

[root@racnode1 ~]# nslookup racnode2
Server:         192.168.78.51
Address:        192.168.78.51#53

Name:   racnode2.virtuallab
Address: 192.168.78.52

[root@racnode1 ~]# nslookup racnode-cluster-scan
Server:         192.168.78.51
Address:        192.168.78.51#53

Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.252
Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.253
Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.251

[root@racnode1 ~]# nslookup racnode-cluster-scan
Server:         192.168.78.51
Address:        192.168.78.51#53

Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.253
Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.252
Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.251

[root@racnode1 ~]# nslookup racnode-cluster-scan
Server:         192.168.78.51
Address:        192.168.78.51#53

Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.252
Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.253
Name:   racnode-cluster-scan.virtuallab
Address: 192.168.78.251

[root@racnode1 ~]#

Prepare Linux for Oracle

  1.  Double check the required RPMs  (https://docs.oracle.com/database/121/CWLIN/prelinux.htm#CEGCECCC)

the following packages (or later versions) must be installed:

binutils-2.23.52.0.1-12.el7.x86_64 
compat-libcap1-1.10-3.el7.x86_64 
gcc-4.8.2-3.el7.x86_64 
gcc-c++-4.8.2-3.el7.x86_64 
glibc-2.17-36.el7.i686 
glibc-2.17-36.el7.x86_64 
glibc-devel-2.17-36.el7.i686 
glibc-devel-2.17-36.el7.x86_64 
libaio-0.3.109-9.el7.i686 
libaio-0.3.109-9.el7.x86_64 
libaio-devel-0.3.109-9.el7.i686 
libaio-devel-0.3.109-9.el7.x86_64 
ksh
make-3.82-19.el7.x86_64 
libXi-1.7.2-1.el7.i686 
libXi-1.7.2-1.el7.x86_64 
libXtst-1.2.2-1.el7.i686 
libXtst-1.2.2-1.el7.x86_64 
libgcc-4.8.2-3.el7.i686 
libgcc-4.8.2-3.el7.x86_64 
libstdc++-4.8.2-3.el7.i686 
libstdc++-4.8.2-3.el7.x86_64 
libstdc++-devel-4.8.2-3.el7.i686 
libstdc++-devel-4.8.2-3.el7.x86_64 
sysstat-10.1.5-1.el7.x86_64

Check  the install required package:

# rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" |grep libstdc

Some i686 packages are missing, to install

[root@racnode1 yum.repos.d]# rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" |grep libstdc
libstdc++-4.8.3-9.el7 (x86_64)
libstdc++-devel-4.8.3-9.el7 (x86_64)
compat-libstdc++-33-3.2.3-72.el7 (x86_64)
[root@racnode1 yum.repos.d]# yum install libstdc++-4.8.3-9.el7.i686

Resolving Dependencies
--> Running transaction check
---> Package libstdc++.i686 0:4.8.3-9.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================
 Package                       Arch                     Version                         Repository                      Size
=============================================================================================================================
Installing:
 libstdc++                     i686                     4.8.3-9.el7                     ol7_latest                     307 k

Transaction Summary
=============================================================================================================================
Install  1 Package

Total download size: 307 k
Installed size: 992 k
Is this ok [y/d/N]: y
Downloading packages:
libstdc++-4.8.3-9.el7.i686.rpm                                                                       | 307 kB  00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libstdc++-4.8.3-9.el7.i686                                                                                1/1
  Verifying  : libstdc++-4.8.3-9.el7.i686                                                                                1/1

Installed:
  libstdc++.i686 0:4.8.3-9.el7

Complete!

[root@racnode1 yum.repos.d]# rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" |grep libstdc
libstdc++-4.8.3-9.el7 (x86_64)
libstdc++-4.8.3-9.el7 (i686)
libstdc++-devel-4.8.3-9.el7 (x86_64)
compat-libstdc++-33-3.2.3-72.el7 (x86_64)
[root@racnode1 yum.repos.d]# yum install libstdc++-devel-4.8.3-9.el7.i686
Resolving Dependencies
--> Running transaction check
---> Package libstdc++-devel.i686 0:4.8.3-9.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================
Package                           Arch                   Version                         Repository                    Size
=============================================================================================================================
Installing:
libstdc++-devel                   i686                   4.8.3-9.el7                     ol7_latest                   1.5 M

Transaction Summary
=============================================================================================================================
Install  1 Package

Total download size: 1.5 M

Installed size: 7.8 M

Is this ok [y/d/N]: y
Downloading packages:
libstdc++-devel-4.8.3-9.el7.i686.rpm                                                                  | 1.5 MB  00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction

 Installing : libstdc++-devel-4.8.3-9.el7.i686                                                                          1/1

Verifying  : libstdc++-devel-4.8.3-9.el7.i686                                                                          1/1

Installed:

libstdc++-devel.i686 0:4.8.3-9.el7

Complete

2. Installing the cvuqdisk RPM for Linux

If you do not use an Oracle Preinstallation RPM, then you must install the cvuqdisk RPM. Without cvuqdisk, Cluster Verification Utility cannot discover shared disks, and you receive the error message “Package cvuqdisk not installed” when you run Cluster Verification Utility. Use the cvuqdisk rpm for your hardware (for example, x86_64).

To install the cvuqdisk RPM, complete the following procedure:

Locate thecvuqdisk RPM package, which is in the directory rpm on the Oracle Grid Infrastructure installation media. If you have already installed Oracle Grid Infrastructure, then it is located in the directory grid_home/cv/rpm.

  • Copy the cvuqdisk package to each node on the cluster. You should ensure that each node is running the same version of Linux.
  • Log in as root.
  • Use the following command to find if you have an existing version of thecvuqdisk package:
# rpm -qi cvuqdisk

If you have an existing version, then enter the following command to deinstall the existing version:

# rpm -e cvuqdisk
  • Set the environment variable CVUQDISK_GRP to point to the group that will own cvuqdisk, typically oinstall. For example:
    # CVUQDISK_GRP=oinstall; export CVUQDISK_GRP
  • In the directory where you have saved the cvuqdisk rpm, use the following command to install thecvuqdisk package:
# rpm -iv package

For example:

# rpm -iv cvuqdisk-1.0.9-1.rpm

3.  Install ifconfig package.

Ifcocnfig is replaced by ip already.

#ip addr

To install ifconfig :

[root@racnode1 bin]# yum install net-tools

4.  Create users ,groups and directories.

[root@racnode1 etc]# passwd oracle
Changing password for user oracle.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

[root@racnode1 etc]# groupadd -g 54323 oper
[root@racnode1 etc]# groupadd -g 54324 backupdba
[root@racnode1 etc]# groupadd -g 54325 asmdba
[root@racnode1 etc]# groupadd -g 54326 dgdba
[root@racnode1 etc]# groupadd -g 54327 kmdba
[root@racnode1 etc]# groupadd -g 54328 asmadmin
[root@racnode1 etc]# groupadd -g 54329 asmoper

[root@racnode1 etc]# useradd -u 54322 -g oinstall -G asmadmin,asmdba grid
[root@racnode1 etc]# usermod -G dba,backupdba,dgdba,kmdba,asmdba,asmoper,vboxsf   oracle

[root@racnode1 etc]# id grid
uid=54322(grid) gid=54321(oinstall) groups=54325(asmdba),54328(asmadmin),54321(oinstall)

[root@racnode1 etc]# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54322(dba),54321(oinstall)

[root@racnode1 etc]# usermod -G dba,backupdba,dgdba,kmdba,asmdba,asmoper,vboxsf oracle

[root@racnode1 etc]# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54322(dba),54324(backupdba),54325(asmdba),54326(dgdba),54327(kmdba),54329(asmoper),54321(oinstall)

[root@racnode1 etc]# id grid
uid=54322(grid) gid=54321(oinstall) groups=54325(asmdba),54328(asmadmin),54321(oinstall)

[root@racnode1 etc]# mkdir -p  /u01/app/12.1.0/grid
[root@racnode1 etc]# mkdir -p /u01/app/grid
[root@racnode1 etc]# mkdir -p /u01/app/oracle
[root@racnode1 etc]# chown -R grid:oinstall /u01
[root@racnode1 etc]# chown oracle:oinstall /u01/app/oracle
[root@racnode1 etc]# chmod -R 775 /u01/

Put “umask 022” into .bash_profile

[root@racnode1 etc]# su - grid
[grid@racnode1 ~]$ echo $SHELL

/bin/bash

[grid@racnode1 ~]$ vi .bask_profile
[grid@racnode1 ~]$ exit
logout

[root@racnode1 etc]# su - oracle
[oracle@racnode1 ~]$ echo $SHELL

/bin/bash

[oracle@racnode1 ~]$ vi .bash_profile
[oracle@racnode1 ~]$

5.  Installation Owner Resource Limit Recommended Ranges

Resource Shell Limit Resource Soft Limit Hard Limit
Open file descriptors nofile at least 1024 at least 65536
Number of processes available to a single user nproc at least 2047 at least 16384
Size of the stack segment of the process stack at least 10240 KB at least 10240 KB, and at most 32768 KB
Maximum Locked Memory Limit memlock at least 90 percent of the current RAM when HugePages memory is enabled and at least 3145728 KB (3 GB) when HugePages memory is disabled at least 90 percent of the current RAM when HugePages memory is enabled and at least 3145728 KB (3 GB) when HugePages memory is disabled

 

[root@racnode1 etc]# cat /etc/security/limits.conf

# oracle user limit
#
oracle              soft    nproc   2047
oracle              hard    nproc   16384
oracle              soft    nofile  1024
oracle              hard    nofile  65536
oracle              soft    stack   10240
oracle              soft    stack   32768

# grid user
#
grid              soft    nproc   2047
grid              hard    nproc   16384
grid              soft    nofile  1024
grid              hard    nofile  65536
grid              soft    stack   10240
grid              soft    stack   32768

6.  Preventing Installation Errors Caused by Terminal Output Commands.

During an Oracle Grid Infrastructure installation, OUI uses SSH to run commands and copy files to the other nodes. During the installation, hidden files on the system (for example, .bashrc or .cshrc) will cause makefile and other installation errors if they contain terminal output commands.

To avoid this problem, you must modify these files in each Oracle installation owner user home directory to suppress all output on STDOUT or STDERR (for example, stty, xtitle, and other such commands) as in the following examples:

Bourne, Bash, or Korn shell:

if [ -t 0 ]; then   
 
   stty intr ^C

fi

C shell:

test -t 0 

if ($status == 0) then         

  stty intr ^C        

endif

7.   Install oracle asm packages.

[root@racnode1 etc]# yum install oracleasm-support

......
......
......
......
Installed:
  oracleasm-support.x86_64 0:2.1.8-3.el7

[root@racnode1 etc]# rpm -qa|grep -i oracleasm
oracleasm-support-2.1.8-3.el7.x86_64

Install kmod-oraceasm package in either way below :

# mount /dev/cdrom  /mnt
#cd /mnt
#cd Packages
# rpm -i kmod-oracleasm-2.0.8-8.el7.x86_64.rpm

OR

# yum install kmod-oraceasm

Create Virtualbox Shared Storage

1. Shutdown virtual server.

 #shutdown –h now

2. In the VirtualBox Manager, select the machine racnode1, click Settings -> Storage. Select Controller SATA and click on the Add Hard Disk button: Click on Create New Disk:

Untitled

3.  Leave VDI (VirtualBox Disk Image) selected and click Next

Untitled

4.  For the shared storage, select Fixed size and click Next.

Untitled

5.  Change the location of the file to the folde previously created.

Enter asm1.vdi as disk name.

Enter 5.00Gb as disk size.

Click Create.

6. Now the disk is physically created with a size of 5Gb, so it can take more time to complete.

Untitled

7.  Repeat the steps to create three more disks:

asm2.vdi (5.00GB)

asm3.vdi (5.00GB)

asm4.vdi (5.00GB)

8.  Finally, four virtual disks should have been created for asm.

Click OK.

Untitled

9.  Make all the asm devices shareable: from VirtualBox Manager, click File -> Virtual Media Manager.

Untitled

10.  For each asm devices: right-click ->Modify, Select Shareable and click OK. Do it for the four asm devices.

Untitled

11.  Start the machine racnode1, you should see now four more disks named /dev/sd*:

[root@racnode1 ~]# ls -ltr /dev/sd*
brw-rw---- 1 root disk 8, 48 Feb 15 18:35 /dev/sdd
brw-rw---- 1 root disk 8, 32 Feb 15 18:35 /dev/sdc
brw-rw---- 1 root disk 8,  0 Feb 15 18:35 /dev/sda
brw-rw---- 1 root disk 8, 64 Feb 15 18:35 /dev/sde
brw-rw---- 1 root disk 8,  2 Feb 15 18:35 /dev/sda2
brw-rw---- 1 root disk 8, 16 Feb 15 18:35 /dev/sdb
brw-rw---- 1 root disk 8,  1 Feb 15 18:35 /dev/sda1

12.  Once the new disks are visible , add a primary partition on each of them with fdisk.

[root@racnode1 ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only,until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x3173fa52.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759):
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

13.  Repeat the same step for sdc, sdd and sde.

When finished, all disks have at least a partition:

[root@racnode1 ~]# ls -l /dev/sd?1
brw-rw---- 1 root disk 8, 65 Feb 15 18:46 /dev/sde1
brw-rw---- 1 root disk 8, 49 Feb 15 18:46 /dev/sdd1
brw-rw---- 1 root disk 8, 33 Feb 15 18:46 /dev/sdc1
brw-rw---- 1 root disk 8, 17 Feb 15 18:44 /dev/sdb1
brw-rw---- 1 root disk 8,  1 Feb 15 18:35 /dev/sda1

14.  Create ASM disks

[root@racnode1 /]# oracleasm createdisk ASM_DISK1 /dev/sdb1
Writing disk header: done
Instantiating disk: done

[root@racnode1 /]# oracleasm createdisk ASM_DISK2 /dev/sdc1
Writing disk header: done
Instantiating disk: done

[root@racnode1 /]# oracleasm createdisk ASM_DISK3 /dev/sdd1
Writing disk header: done
Instantiating disk: done

[root@racnode1 /]# oracleasm createdisk ASM_DISK4 /dev/sde1
Writing disk header: done
Instantiating disk: done

[root@racnode1 /]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4

Clone VirtualBox VM

1. Shutdown the VM

[root@racnode1 /]# shutdown -h now

2.  Clone the disk racnode1.vdi from VirtualBox Manager, click File -> Virtual Media Manager.

Untitled

3.  Right-click on the disk racnode1.vdi and click Copy

4. Leave the file racnode1.vdi, click Next

Untitled

5.  Leave VDI as file type. Click Next

Untitled

6.  Leave Dynamically allocated. Click Next

Untitled

7.  Type in the full path or choose through the browse icon the Location for the disk file to be created. Again, it is better to use the common folder virtuallab previously created for all the virtual disks.

The file should be named racnode2.vdi.

Click Copy.

Untitled

8.  Click Close to exit from the Virtual Media Manager.

Untitled

Setup Second Linux VM

1. In VirtualBox, click New icon in the upper left corner to create the new VM.

2.  Type in racnode2 for the Name of the VM. Choose Linux for the Type and Oracle (64 bit) for the Version and click Next

Untitled

3.  Type 4096 in the Size field and click Next.

Untitled

4.  Select Use an existing virtual hard drive file and use the folder icon to locate the file racnode2.vdi previously created.  Click Create to complete the creation of the second VM.

Untitled

5.  There are now two virtual machines racnode2 being a clone of racnode1.

From the main screen, select the virtual machine racnode2 and click the Settings icon in the upper left hand corner.

Untitled

6.  Open theUSB sub-menu. Uncheck the Enable USB Controller check box.

Untitled

7.  Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to Host-only Adapter.

Untitled

8. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to Internal Network and type the name rac-priv in the Name field

Untitled

9.   Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the Attached to: dropdown to NAT.

Untitled

10.  Select the Shared Folders  Click Add Shared Folder.

Enter the path where you’ve downloaded the Oracle installation media and enter a name for your folder.

Check Auto-mount and click OK.

Untitled

11.  Open the Storage sub-menu.

Click on Controller SATA and then on Add Disk:

Untitled

12.  Click on Choose existing disk:

Untitled

13.  Select the disk asm1.vdi and click Open.

Untitled

14.  Repeat the operation to add all remaining asm disks.

Untitled

15.  Click on Ok to save the modifications.

16.  Review the summary of the new virtual machine.

Untitled

 

17.  Start the racnode2 VM

Configure Second Linux VM

1. Change host name from racnode1 to racnode2 for second VM.

To see current hostname details:
#hostnamectl

Change hostname to racnode2
#hostnamectl –static set-hostname racnode2.virtuallab
#hostnamectl –transient set-hostname racnode2.virtuallab

2. Network configuration using “nmtui” — Network Manager Text User Interface

#yum install NetworkManager-tui

#nmcli dev status

3.  Setup or change hostname by choosing “Set system hostname”

Untitled

4.  Setup network interfaces accordingly.

Untitled

Untitled

 

Untitled

4.  Remove the udev network rules file. It will be regenerated on the next reboot with the new MAC addresses.

[root@racnode2 ~]# rm -f /etc/udev/rules.d/70-persistent-net.rules

Remove the HWADDR and UUID lines in the network adapter configuration files.

[#sed -i -e '/HWADDR/d' -e '/UUID/d' /etc/sysconfig/network-scripts/ifcfg-enp*

5.  Open a session on racnode2.

6.  Stop the DNS service.

[root@racnode2 ~]# service named stop
Redirecting to /bin/systemctl stop  named.service

7.  Remove the actual DNS files

root@racnode2 ~]# cd /var/named
[root@racnode2 named]# ls -ltr
total 24
-rw-r----- 1 root named 152 Jun 21 2007 named.localhost
-rw-r----- 1 root named 168 Dec 15 2009 named.loopback
-rw-r----- 1 root named 152 Dec 15 2009 named.empty
-rw-r----- 1 root named 2076 Jan 29 2013 named.ca
drwxrwx---. 2 named named 6 Nov 20 17:05 slaves
-rw-rw-r-- 1 root named 814 Nov 24 21:46 virtuallab
-rw-r--r-- 1 root root 917 Nov 24 21:49 in-addr.arpa
drwxrwx---. 2 named named 47 Feb 15 19:19 data
drwxrwx---. 2 named named 58 Feb 16 11:03 dynamic
[root@racnode2 named]# mv in-addr.arpa in-addr.arpa.old
[root@racnode2 named]# mv virtuallab virtuallab.old
[root@racnode2 named]#

8.  Modify the file /etc/named.conf by using the following command:

sed -i -e 's/listen-on .*/listen-on port 53 { 192.168.78.52; };/' \
-e 's/type master;/type slave;\n masters {192.168.78.51; };/' \
/etc/named.conf

so that at the end it looks like this one :

[root@racnode2 named]# cat /etc/named.conf
//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//

options {
 listen-on port 53 { 192.168.78.52; };
 listen-on-v6 port 53 { ::1; };
 directory "/var/named";
 dump-file "/var/named/data/cache_dump.db";
 statistics-file "/var/named/data/named_stats.txt";
 memstatistics-file "/var/named/data/named_mem_stats.txt";
 allow-query { 192.168.78.0/24; localhost; };
 allow-transfer { 192.168.78.0/24; };
 /*
 - If you are building an AUTHORITATIVE DNS server, do NOT enable recursion.
 - If you are building a RECURSIVE (caching) DNS server, you need to enable
 recursion.
 - If your recursive DNS server has a public IP address, you MUST enable access
 control to limit queries to your legitimate users. Failing to do so will
 cause your server to become part of large scale DNS amplification
 attacks. Implementing BCP38 within your network would greatly
 reduce such attack surface
 */
 recursion yes;

dnssec-enable yes;
 dnssec-validation yes;
 dnssec-lookaside auto;

/* Path to ISC DLV key */
 bindkeys-file "/etc/named.iscdlv.key";

managed-keys-directory "/var/named/dynamic";

pid-file "/run/named/named.pid";
 session-keyfile "/run/named/session.key";
};

logging {
 channel default_debug {
 file "data/named.run";
 severity dynamic;
 };
};

zone "." IN {
 type hint;
 file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

zone "virtuallab" {
 type slave;
 masters {192.168.78.51; };
 file "virtuallab";
};

zone "in-addr.arpa" {
 type slave;
 masters {192.168.78.51; };
 file "in-addr.arpa";
};
[root@racnode2 named]#

9.  Start the named service.

[root@racnode2 [root@racnode2 named]# service named start
Redirecting to /bin/systemctl start  named.service
[root@racnode2 named]#

10.  Check that both the master on racnode1 and slave on racnode2 DNS servers are working

[root@racnode2 ~]# dig @racnode1 racnode1.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode2.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode1-vip.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode2-vip.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode1-priv.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode2-priv.virtuallab
[root@racnode2 ~]# dig @racnode1 collabn-cluster-scan.virtuallab

[root@racnode2 ~]# dig @racnode2 racnode1.virtuallab
[root@racnode2 ~]# dig @racnode2 racnode2.virtuallab
[root@racnode2 ~]# dig @racnode2 racnode1-vip.virtuallab
[root@racnode2 ~]# dig @racnode2 racnode2-vip.virtuallab
[root@racnode2 ~]# dig @racnode2 racnode1-priv.virtuallab
[root@racnode2 ~]# dig @racnode2 racnode2-priv.virtuallab
[root@racnode2 ~]# dig @racnode2 collabn-cluster-scan.virtuallab

Please make sure :

a.  Double check that you have stopped IPTABLES.

b.  Port 53 is blocked or not to remote query, it can be tested by :

 #telnet racnode1(2) 53

c.   Make life easier, just turn off  firewalld.

[root@racnode2 named]# service firewalld stop
Redirecting to /bin/systemctl stop firewalld.service

[root@racnode2 named]# systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@racnode2 named]#

after this, all below commands work fine.

[root@racnode2 ~]# dig @racnode1 racnode1.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode2.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode1-vip.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode2-vip.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode1-priv.virtuallab
[root@racnode2 ~]# dig @racnode1 racnode2-priv.virtuallab
[root@racnode2 ~]# dig @racnode1 collabn-cluster-scan.virtuallab

VNC Server Setup

  1. Download VNC-Viewer for your desktop.
  2. Configure VNC Server with the oracle account (passwords won’t be displayed):
[root@racnode1 etc]# su - oracle
Last login: Tue Feb 2 20:37:31 AEDT 2016 on tty1
[oracle@racnode1 ~]$ vncserver :1

You will require a password to access your desktops.

Password:
Verify:
xauth: file /home/oracle/.Xauthority does not exist

New 'racnode1.virtuallab:1 (oracle)' desktop is racnode1.virtuallab:1

Creating default startup script /home/oracle/.vnc/xstartup
Starting applications specified in /home/oracle/.vnc/xstartup
Log file is /home/oracle/.vnc/racnode1.virtuallab:1.log

2.  Open a vncviewer on your laptop and fill the IP address of racnode1 followed by :1.

Untitled

3. Enter the password “virtuallab” when prompted:

Untitled

4. The graphical interface is ready for the Grid Infrastructure Installation.

Untitled

 

Install Grid Infrastructure (ASM)

1. Verify that the VirtualBox Shared folder containing the installation media is mounted correctly and that Oracle has access to it:

[root@racnode1 12.1.0.2]# pwd
/media/sf_Grid_and_Database/Linux X86_64/12.1.0.2
[root@racnode1 12.1.0.2]# ls -ltr
total 8662093
-rwxrwx--- 1 root vboxsf 646972897 Aug 25 2014 linuxamd64_12102_grid_2of2.zip
-rwxrwx--- 1 root vboxsf 1747043545 Aug 25 2014 linuxamd64_12102_grid_1of2.zip
-rwxrwx--- 1 root vboxsf 1673544724 Aug 25 2014 linuxamd64_12102_database_1of2.zip
-rwxrwx--- 1 root vboxsf 1014530602 Aug 25 2014 linuxamd64_12102_database_2of2.zip
drwxrwx--- 1 root vboxsf 4096 Nov 25 21:53 grid
drwxrwx--- 1 root vboxsf 0 Nov 25 22:27 database
-rwxrwx--- 1 root vboxsf 2838 Nov 28 12:48 README.txt
-rwxrwx--- 1 root vboxsf 567375160 Nov 28 13:29 linuxamd64_12102_gsm.zip
-rwxrwx--- 1 root vboxsf 499228127 Nov 28 13:45 linuxamd64_12102_examples.zip
-rwxrwx--- 1 root vboxsf 918555219 Nov 28 14:12 linuxamd64_12102_client.zip
-rwxrwx--- 1 root vboxsf 1802723467 Nov 28 14:35 ofm_webtier_linux_11.1.1.7.0_64_disk1_1of1.zip

2.   Once the package decompression is completed, you’ll end up with two directories, grid and database containing the installation files.

[oracle@collabn1 sf_12cR1]$ ls -l
total 6295549
drwxrwx---. 1 root vboxsf 0 Jul 3 06:38 database
drwxrwx---. 1 root vboxsf 0 Jun 10 08:15 grid
-rwxrwx---. 1 root vboxsf 1361028723 Jun 27 16:21 linuxamd64_12c_database_1of2.zip
-rwxrwx---. 1 root vboxsf 1116527103 Jun 27 16:12 linuxamd64_12c_database_2of2.zip
-rwxrwx---. 1 root vboxsf 1750478910 Jun 27 16:27 linuxamd64_12c_grid_1of2.zip
-rwxrwx---. 1 root vboxsf 201673595 Jun 27 15:23 linuxamd64_12c_grid_2of2.zip

3.  if it is not done yet, install the cvuqdisk package as root on both nodes.

[root@racnode1 12.1.0.2]# ls -ltr grid/rpm/cvuqdisk*
-rwxrwx--- 1 root vboxsf 8976 Jul 1 2014 grid/rpm/cvuqdisk-1.0.9-1.rpm
[root@racnode1 12.1.0.2]# ~]# rpm -Uvh grid/rpm/cvuqdisk-1.0.9-1.rpm
-bash: ~]#: command not found
[root@racnode1 12.1.0.2]# rpm -Uvh grid/rpm/cvuqdisk-1.0.9-1.rpm
Preparing... ################################# [100%]
 package cvuqdisk-1.0.9-1.x86_64 is already installed

4.   Start to install GI from the VNC remote session, run the installation of Grid Infrastructure as grid:

[grid@racnode1 sf_12.1.0.2]$ pwd
/media/sf_12.1.0.2
[grid@racnode1 sf_12.1.0.2]$

[grid@racnode1 ~]$ /media/sf_12.1.0.2/grid/runInstaller
Starting Oracle Universal Installer...

Checking Temp space: must be greater than 415 MB. Actual 24746 MB Passed
Checking swap space: must be greater than 150 MB. Actual 3071 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2016-02-16_08-03-43PM. Please wait ...
[grid@racnode1 ~]$

5.   The Installation starts displaying a Splash Screen:

Untitled

6.  Leave Install and Configure Oracle Grid Infrastructure for a Cluster and click Next:

Untitled

7.  Leave Configure a Standard Cluster and click Next:

Untitled

8.  Leave Typical Installation and click Next:

Untitled

9.  In the SCAN Name box, enter racnode-cluster-scan.virtuallab

In the central box, only the first node is displayed. Click Add… and enter the network names of the second node:

  • Public Hostname: racnode2.virtuallab
  • Virtual Hostname:racnode2-priv.virtuallab

Untitled

In the mean time, you can setuo SSH connectivity…

10.  Click SSH connectivity…

Untitled

Enter the password of grid user and click Setup.

The setup of SSH equivalency is initiated.

Click OK when it’s completed.

Untitled

11.   Confirm SCAN Name . Click Yes.

Untitled

12.  Click on Specify Network Interfaces…

Check that the correct addresses are set:

  • Enp0s3 is marked as Public
  • Enp0s8 marked as Private
  • Enp0s9 is marked as Do Not Use

Click OK to close the Identify Network Interfaces window.

Finally, click Next

Untitled

Untitled

13.  Leave the default Oracle Base and Software Location.

In the Cluster Registry Storage Type combo box, select Oracle Automatic Storage Management

Enter password for SYSASM password.

Select asmadmin as OSASM group.

Click Next.

Untitled

Untitled

password is too weak  and Oracle raises an alert. Click Yes to continue:

Untitled

14.   Leave DATA as Disk Group Name.

Select External redundancy.

The Candidate Disks list is empty. Click on Change Discovery Path…

Enter /dev/oracleasm/disks in the Disk Discovery Path text field.

Click Ok.

Untitled

15.  Check two disks that will be immediately allocated to the new disk group.

Click Next

Untitled

16.  Leave the default Inventory Directory. Click Next

Untitled

17.  Check Automatically run configuration script and then Use “root” user credentials.

Enter the password of root and click Next.

Untitled

18.  It takes some time to complete the prerequisite checks.

Untitled

19.  If everything is correct, you will notice checks that have failed:

  • Physical Memory
  • swap Size
  • Device Checks for ASM
  • Task resolv.conf integrity

Check “Ignore All” and click Next.

Untitled

UntitledUntitledUntitledUntitled

UDEV error has  no impact on installations

Untitled

20.  Click Yes to confirm that you want to skip some prerequisites.

Untitled

21.  The summary page appears. Optionally, click Save Response File and choose a location to have the response file saved for this installation session.

Untitled

22. Review the summary and click Install to start the installation.

Untitled

23.  The installation starts.

Untitled

24.  After a while, the installation asks for a confirmation before running some scripts as root. Click Yes to continue.

Untitled

Untitled

25.   The installation continues, again, it may take longtime to complete. Once it’s finished, click on Close to exit.

Untitled

Just found when reboot both virtual boxes, the /etc/rsolv.conf was updated, and nslookup does not work properly. So we have to make /etc/resolv.conf non-updatable by using chattr.

[root@racnode2 bin]#chattr +i /etc/resolv.conf
[root@racnode2 bin]#cat /etc/resolv.conf
# Generated by NetworkManager
search gateway virtuallab
nameserver 192.168.78.51
nameserver 192.168.78.52
nameserver 10.0.0.138

26.  You can get the status of the cluster components  after installation is complete.

[root@racnode1 bin]# ./srvctl start vip -vip racnode2
[root@racnode1 bin]# su - grid
Last login: Tue Feb 16 22:12:33 AEDT 2016 on pts/0
[grid@racnode1 ~]$
[grid@racnode1 ~]$
[grid@racnode1 ~]$ . oraenv
ORACLE_SID = [grid] ? +ASM1
The Oracle base has been set to /u01/app/grid
[grid@racnode1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
 ONLINE ONLINE racnode1 STABLE
 ONLINE ONLINE racnode2 STABLE
ora.LISTENER.lsnr
 ONLINE ONLINE racnode1 STABLE
 ONLINE ONLINE racnode2 STABLE
ora.asm
 ONLINE ONLINE racnode1 Started,STABLE
 ONLINE ONLINE racnode2 Started,STABLE
ora.net1.network
 ONLINE ONLINE racnode1 STABLE
 ONLINE ONLINE racnode2 STABLE
ora.ons
 ONLINE ONLINE racnode1 STABLE
 ONLINE ONLINE racnode2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
 1 ONLINE ONLINE racnode2 STABLE
ora.LISTENER_SCAN2.lsnr
 1 ONLINE ONLINE racnode1 STABLE
ora.LISTENER_SCAN3.lsnr
 1 ONLINE ONLINE racnode1 STABLE
ora.MGMTLSNR
 1 ONLINE ONLINE racnode1 169.254.146.121 172.
 16.100.61,STABLE
ora.cvu
 1 ONLINE ONLINE racnode1 STABLE
ora.mgmtdb
 1 ONLINE ONLINE racnode1 Open,STABLE
ora.oc4j
 1 ONLINE ONLINE racnode1 STABLE
ora.racnode1.vip
 1 ONLINE ONLINE racnode1 STABLE
ora.racnode2.vip
 1 ONLINE ONLINE racnode2 STABLE
ora.scan1.vip
 1 ONLINE ONLINE racnode2 STABLE
ora.scan2.vip
 1 ONLINE ONLINE racnode1 STABLE
ora.scan3.vip
 1 ONLINE ONLINE racnode1 STABLE
--------------------------------------------------------------------------------

Install RAC Database Software

1. In a VNC session, run the installer as oracle user.

[oracle@racnode1 sf_TEST]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ol-root 27G 9.1G 18G 35% /
devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs 2.0G 635M 1.4G 31% /dev/shm
tmpfs 1.9G 8.5M 1.8G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 497M 245M 253M 50% /boot
none 932G 66G 867G 8% /media/sf_TEST
tmpfs 370M 0 370M 0% /run/user/0
[oracle@racnode1 sf_TEST]$ cd /media/sf_TEST
[oracle@racnode1 sf_TEST]$ ls
database
grid
linuxamd64_12102_client.zip
linuxamd64_12102_database_1of2.zip
linuxamd64_12102_database_2of2.zip
linuxamd64_12102_examples.zip
linuxamd64_12102_grid_1of2.zip
linuxamd64_12102_grid_2of2.zip
linuxamd64_12102_gsm.zip
ofm_webtier_linux_11.1.1.7.0_64_disk1_1of1.zip
README.txt
[oracle@racnode1 sf_TEST]$ cd database
[oracle@racnode1 database]$ ls
install response rpm runInstaller sshsetup stage welcome.html
[oracle@racnode1 database]$ ./runInstaller
Starting Oracle Universal Installer...

Checking Temp space: must be greater than 500 MB. Actual 17779 MB Passed
Checking swap space: must be greater than 150 MB. Actual 3071 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2016-02-17_09-40-01PM. Please wait ...[oracle@racnode1 database]
$

2.  Uncheck I wish to receive security updates via My Oracle Support and click Next.

Untitled

3.  Click Yes to confirm that you do not want to be informed about security updates.

Untitled

4.  Select Install database software only and click Next.

Untitled

5.  Select Oracle Real Application Clusters database installation and click Next.

Untitled

6.  Verify that all nodes are selected and click Next.

Untitled

7.  Click SSH connectivity to setup  passwordless connectivity.

Untitled

8.  Click Next to accept the default language selection.

Untitled

9.  Select Enterprise Edition and click Next.

Untitled

Oracle Base: /u01/app/oracle

Software location: /u01/app/oracle/product/12.1.0/dbhome_1

Click Next.

Untitled

10.  Given right OS groups.

Untitled

11. The installer checks all prerequisites.

Untitled

12.  Review the installation summary, optionally save a response file, and click Install.

Untitled

13.  The installation is under going……

Untitled

14.  Before completion, the installer asks to run a script on both nodes as root. DO NOT CLICK OK before root.sh run.

Untitled

[root@racnode1 ~]# /u01/app/oracle/product/12.1.0/dbhome_1/root.sh

Performing root user operation.

The following environment variables are set as:
   ORACLE_OWNER= oracl
   ORACLE_HOME=  /u01/app/oracle/product/12.1.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

[root@racnode1 ~]#
[root@racnode2 bin]# /u01/app/oracle/product/12.1.0/dbhome_1/root.sh
Performing root user operation.
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/oracle/product/12.1.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

[root@racnode2 bin]#

click OK on the previous window, then click Close to exit the Installer.

15.  Click OK on the previous window, then click Close to exit the Installer.

Untitled

Create New Disks and Create New Diskgroup FRA

1.Shutdown racnode1 VM

2. Choose SETTING ->Storage:

Untitled

3.  Choose “Create new disk”.

Untitled

4.  Click VDI( VirtualBox Disk Image ).

Untitled

5.   Check “Fixed Size”.

Untitled

6.  Gives size 5 GB.

Untitled

7.  Click Create.

Untitled

8.  Repeat same for asm6.vdi with 5GB size as well.

9.  Go to Oracle VM VirtualBox Manager : File ->virtual Media Manager

Untitled

10.  Make asm5.vdi “Shareable”.

Untitled

11.  Go to Oracle VM VirtualBox Manager ->racnode2 ->Settings ->storage –add hard disk.

Untitled

12.  Choose “Choose existing disk”.

Untitled

13.  Choose “asm5.vdi”.

Untitled

Untitled

14.  Repeat the same thing for asm6.vdi as well.

15. Create new partitions in /dev/sdf, /edv/sdg ( asm5.vdi,asm6.vdi )

on racnode 1  only (this has to be done only on one node. )

[root@racnode1 bin]# ls -ltr /dev/sd*
brw-rw---- 1 root disk 8, 0 Feb 17 23:26 /dev/sda
brw-rw---- 1 root disk 8, 2 Feb 17 23:26 /dev/sda2
brw-rw---- 1 root disk 8, 1 Feb 17 23:26 /dev/sda1
brw-rw---- 1 root disk 8, 80 Feb 17 23:26 /dev/sdf
brw-rw---- 1 root disk 8, 96 Feb 17 23:26 /dev/sdg
brw-rw---- 1 root disk 8, 16 Feb 17 23:26 /dev/sdb
brw-rw---- 1 root disk 8, 48 Feb 17 23:26 /dev/sdd
brw-rw---- 1 root disk 8, 32 Feb 17 23:26 /dev/sdc
brw-rw---- 1 root disk 8, 64 Feb 17 23:26 /dev/sde
brw-rw---- 1 root disk 8, 17 Feb 17 23:26 /dev/sdb1
brw-rw---- 1 root disk 8, 33 Feb 17 23:26 /dev/sdc1
brw-rw---- 1 root disk 8, 49 Feb 17 23:26 /dev/sdd1
brw-rw---- 1 root disk 8, 65 Feb 17 23:26 /dev/sde1
[root@racnode1 bin]#
[root@racnode1 bin]# fdisk /dev/sdf
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x373e3026.

Command (m for help): n
Partition type:
 p primary (0 primary, 0 extended, 4 free)
 e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
First sector (2048-10485759, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759):
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 bin]# fdisk /dev/sdg
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xe5f3fc7c.

Command (m for help): n
Partition type:
 p primary (0 primary, 0 extended, 4 free)
 e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
First sector (2048-10485759, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759):
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode1 bin]# ls -ltr /dev/sd*
brw-rw---- 1 root disk 8, 0 Feb 17 23:26 /dev/sda
brw-rw---- 1 root disk 8, 2 Feb 17 23:26 /dev/sda2
brw-rw---- 1 root disk 8, 1 Feb 17 23:26 /dev/sda1
brw-rw---- 1 root disk 8, 16 Feb 17 23:26 /dev/sdb
brw-rw---- 1 root disk 8, 48 Feb 17 23:26 /dev/sdd
brw-rw---- 1 root disk 8, 32 Feb 17 23:26 /dev/sdc
brw-rw---- 1 root disk 8, 64 Feb 17 23:26 /dev/sde
brw-rw---- 1 root disk 8, 17 Feb 17 23:26 /dev/sdb1
brw-rw---- 1 root disk 8, 33 Feb 17 23:26 /dev/sdc1
brw-rw---- 1 root disk 8, 49 Feb 17 23:26 /dev/sdd1
brw-rw---- 1 root disk 8, 65 Feb 17 23:26 /dev/sde1
brw-rw---- 1 root disk 8, 80 Feb 17 23:37 /dev/sdf
brw-rw---- 1 root disk 8, 81 Feb 17 23:37 /dev/sdf1
brw-rw---- 1 root disk 8, 96 Feb 17 23:37 /dev/sdg
brw-rw---- 1 root disk 8, 97 Feb 17 23:37 /dev/sdg1
[root@racnode1 bin]#

[root@racnode1 bin]# fdisk -l /dev/sdf

Disk /dev/sdf: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x373e3026

 Device Boot Start End Blocks Id System
/dev/sdf1 2048 10485759 5241856 83 Linux
[root@racnode1 bin]# fdisk -l /dev/sdg

Disk /dev/sdg: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xe5f3fc7c

 Device Boot Start End Blocks Id System
/dev/sdg1 2048 10485759 5241856 83 Linux
[root@racnode1 bin]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
[root@racnode1 bin]# oracleasm createdisk ASM_DISK5 /dev/sdf1
Writing disk header: done
Instantiating disk: done
[root@racnode1 bin]# oracleasm createdisk ASM_DISK6 /dev/sdg1
Writing disk header: done
Instantiating disk: done
[root@racnode1 bin]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
ASM_DISK5
ASM_DISK6
[root@racnode1 bin]#

Execute the below steps on racnode2 to scan and list newly created oracleasm disk:
[root@racnode2 ~]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
[root@racnode2 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ASM_DISK5"
Instantiating disk "ASM_DISK6"
[root@racnode2 ~]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
ASM_DISK5
ASM_DISK6
[root@racnode2 ~]#  

16.   Create diskgroup FRA.

[root@racnode1 bin]# su - grid
Last login: Wed Feb 17 20:49:36 AEDT 2016 on pts/0
[grid@racnode1 ~]$ . oraenv
ORACLE_SID = [grid] ? +ASM1
The Oracle base has been set to /u01/app/grid
[grid@racnode1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Wed Feb 17 23:45:31 2016

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> select path,header_status from v$asm_disk;

PATH HEADER_STATU
---------------------------------------- ------------
/dev/oracleasm/disks/ASM_DISK3 PROVISIONED
/dev/oracleasm/disks/ASM_DISK6 PROVISIONED
/dev/oracleasm/disks/ASM_DISK4 PROVISIONED
/dev/oracleasm/disks/ASM_DISK5 PROVISIONED
/dev/oracleasm/disks/ASM_DISK2 MEMBER
/dev/oracleasm/disks/ASM_DISK1 MEMBER

5 rows selected.

SQL> CREATE DISKGROUP FRA EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/ASM_DISK5','/dev/oracleasm/disks/ASM_DISK6';

Diskgroup created.

SQL> select path,header_status from v$asm_disk;

PATH HEADER_STATU
---------------------------------------- ------------
/dev/oracleasm/disks/ASM_DISK3 PROVISIONED
/dev/oracleasm/disks/ASM_DISK4 PROVISIONED
/dev/oracleasm/disks/ASM_DISK6 MEMBER
/dev/oracleasm/disks/ASM_DISK5 MEMBER
/dev/oracleasm/disks/ASM_DISK2 MEMBER
/dev/oracleasm/disks/ASM_DISK1 MEMBER

6 rows selected.

Create 12c RAC Database

1.Run the Database Configuration Assistant:

[oracle@racnode1 ~]$ id
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),996(vboxsf),54322(dba),54323(oper),54324(backupdba),54325(asmdba),54326(dgdba),54327(kmdba),54329(asmoper)

[oracle@racnode1 ~]$ /u01/app/oracle/product/12.1.0/dbhome_1/bin/dbca

Untitled

2.  Create Database and click Next.

Untitled

3.  Select Create a database with default configuration.

Global Database Name: RACTEST.VIRTUALLAB

Storage Type: Automatic Storage Management (ASM)

Database Files Location: +DATA

Fast Recovery Area: +FRA

Database Character Set: AL32UTF8 – Unicode UTF-8 Universal character set

Administrative password: Password

Check Create As Container Database

Pluggable Database Name: PDB_A

Untitled

4.  Click Next to start Prerequisite Checks.

Untitled

Untitled

5.  Review the summary page and click Finish.

Untitled

6. Check the progress:

Untitled

7.  The database creation is finished. Click Exit, then Close.

Untitled

Advertisements
This entry was posted in 12c new features, GI and RAC and tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

4 Responses to Step by Step Installing Oracle 12c GI and RAC on Linux 7 Using VirtualBox

  1. Pingback: How to Move 12c Grid Infrastructure Management Repository ( GIMR ) to Another Diskgroup | James Huang Oracle DBA Blog

  2. Pingback: How to Move OCR, Voting Disk File, ASM SPILE to a New Diskgroup ( 12.1.0.2 ) | James Huang Oracle DBA Blog

  3. Oleg says:

    Hello James!
    First of all thank you for the amazing and very time consuming work of creating this guidance!
    I am trying to follow your steps and have few questions so far. Most of them are related to the network configuration.

    The Section “OS Installation”, Step 4
    The second screenshot “https://jameshuangsj.files.wordpress.com/2016/02/untitled28.png” shows configuration for the Enp0s8 (is it rac-priv adapter?),
    and the IP address there is “172.16.100.61”
    Please note: it is 61 there.

    Later in the section “Configure Bind DNS” step 5 it reads the IP addresses 51 and 52 in the following lines
    “racnode1-priv A 172.16.100.51”
    and later
    “racnode2-priv A 172.16.100.52”

    Also in the “step 6. Create the reverse zone file racnode1” we see 51 and 52 again
    “51.100.16.172 PTR racnode1-priv.virtuallab”
    “52.100.16.172 PTR racnode2-priv.virtuallab”

    And later in the section “Configure Second Linux VM” step “4. Setup network interfaces accordingly.”
    the screenshot “https://jameshuangsj.files.wordpress.com/2016/02/untitled72.png” shows again the IP address 61
    “172.16.100.61”

    Is that correct for the second node? Same IP address as node 1 in step 4?
    In that case why the BIND is configured for 51 and 52?

    Can you please advise what would be correct and consistent values of IP addresses there?
    Thank you!

    Like

    • james huang says:

      Hi Oleg,
      Thanks for reading my blog.
      Sorry about for replaying you late.
      Yes, Enp0s8 is rac-priv adapter, you are correct!
      Yes, you are correct, second node should be 172.16.100.62.

      It is old way ( pre 11gR2 ) for BIND is configured for VIP as 51 and 52.
      For after 11gR2, a cluster created HAIP ( 169.254.x.x ) is created and attached to Priv interface. This HAIP is used as private IP, instead of the old private host(IP). in this case, 172.16.100.51(52) will not be used for after 11gGR2 GI.

      when you run “ifconfig -a”, you will see similar:

      enp0s8: flags=4163 mtu 1500
      inet 172.16.100.61 netmask 255.255.255.0 broadcast 172.16.100.255

      ..

      enp0s8:1: flags=4163 mtu 1500
      inet 169.254.146.121 netmask 255.255.0.0 broadcast 169.254.255.255
      ether 08:00:27:f8:0e:13 txqueuelen 1000 (Ethernet)

      You won’t see any 172.16.100.51(52) in 12c GI.

      Wish my answer helps. Please let me know if still any issues.

      Thanks,
      James

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s