Infrastructure at your Service

Mouhamadou Diaw

Oracle 21c : Two nodes Grid Infrastructure Installation

Oracle 21c is actually released in the cloud, and I did some tests to setup a Grid infrastructure cluster with two nodes.
I used following two VM servers to test
racp1vm1
racp1vm2
Below the addresses I am using. Note that a dns server is setup

192.168.0.101 racp1vm1.dbi.lab racp1vm1            --public network
192.168.0.103 racp1vm2.dbi.lab racp1vm2            --public network
192.168.0.102 racp1vm1-vip.dbi.lab racp1vm1-vip    --vitual network
192.168.0.104 racp1vm2-vip.dbi.lab racp1vm2-vip    --virtual network
10.1.1.1 racp1vm1-priv.dbi.lab racp1vm1-priv       --private network
10.1.1.2 racp1vm2-priv.dbi.lab racp1vm2-priv       --private network

The scan name should be resolved in a round-robin method. Every time the nslookup command should return a different IP as first address

racp1-scan.dbi.lab :  192.168.0.105
racp1-scan.dbi.lab : 192.168.0.106
racp1-scan.dbi.lab . 192.168.0.107
[[email protected] diag]# nslookup racp1-scan
Server:         192.168.0.100
Address:        192.168.0.100#53

Name:   racp1-scan.dbi.lab
Address: 192.168.0.105
Name:   racp1-scan.dbi.lab
Address: 192.168.0.107
Name:   racp1-scan.dbi.lab
Address: 192.168.0.106

[[email protected] diag]# nslookup racp1-scan
Server:         192.168.0.100
Address:        192.168.0.100#53

Name:   racp1-scan.dbi.lab
Address: 192.168.0.107
Name:   racp1-scan.dbi.lab
Address: 192.168.0.106
Name:   racp1-scan.dbi.lab
Address: 192.168.0.105

[[email protected] diag]# nslookup racp1-scan
Server:         192.168.0.100
Address:        192.168.0.100#53

Name:   racp1-scan.dbi.lab
Address: 192.168.0.106
Name:   racp1-scan.dbi.lab
Address: 192.168.0.105
Name:   racp1-scan.dbi.lab
Address: 192.168.0.107

[[email protected] diag]#

I used udev for the ASM disks and below the contents of my udev file

[[email protected] install]# cat /etc/udev/rules.d/90-oracle-asm.rules
Oracle ASM devices
KERNEL==\”sd[b-f]1",  OWNER="grid", GROUP="asmadmin", MODE="0660"

The installation is the same that for the 19c. Unzip your software in your GRID_HOME

[[email protected] ~]$ mkdir -p /u01/app/21.0.0.0/grid
[[email protected] ~]$ unzip -d /u01/app/21.0.0.0/grid grid_home-zip.zip

And run the gridSetup.sh command

[[email protected] grid]$ ./gridSetup.sh

I got some warnings i decided to ignore and go ahead

As specified I executed the scripts on both nodes

[[email protected] ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[[email protected] ~]#

[[email protected] ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[[email protected] ~]#

Below truncated outputs of root.sh.

[[email protected] ~]# /u01/app/21.0.0.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/21.0.0.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
…
..
CRS-4256: Updating the profile
Successful addition of voting disk 39600544f8794f63bfb83f128d9a9079.
Successfully replaced voting disk group with +CRSDG.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   39600544f8794f63bfb83f128d9a9079 (/dev/sdc1) [CRSDG]
Located 1 voting disk(s).
2021/02/08 15:05:06 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2021/02/08 15:06:00 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/02/08 15:06:00 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2021/02/08 15:08:06 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/02/08 15:08:27 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[[email protected] install]# /u01/app/21.0.0.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/21.0.0.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
…
…
2021/02/08 15:15:00 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/02/08 15:15:00 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2021/02/08 15:15:20 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/02/08 15:15:27 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[[email protected] install]#

Then click OK

You can verify that the installation was fine

[[email protected] diag]# /u01/app/21.0.0.0/grid/bin/crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [21.0.0.0.0]
[[email protected] diag]#

[[email protected] diag]# /u01/app/21.0.0.0/grid/bin/crsctl check cluster -all
**************************************************************
racp1vm1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
racp1vm2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[[email protected] diag]#
[[email protected] diag]# /u01/app/21.0.0.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       racp1vm1                 STABLE
               ONLINE  ONLINE       racp1vm2                 STABLE
ora.chad
               ONLINE  ONLINE       racp1vm1                 STABLE
               ONLINE  ONLINE       racp1vm2                 STABLE
ora.net1.network
               ONLINE  ONLINE       racp1vm1                 STABLE
               ONLINE  ONLINE       racp1vm2                 STABLE
ora.ons
               ONLINE  ONLINE       racp1vm1                 STABLE
               ONLINE  ONLINE       racp1vm2                 STABLE
ora.proxy_advm
               OFFLINE OFFLINE      racp1vm1                 STABLE
               OFFLINE OFFLINE      racp1vm2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       racp1vm1                 STABLE
      2        ONLINE  ONLINE       racp1vm2                 STABLE
ora.CRSDG.dg(ora.asmgroup)
      1        ONLINE  ONLINE       racp1vm1                 STABLE
      2        ONLINE  ONLINE       racp1vm2                 STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       racp1vm2                 STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       racp1vm1                 Started,STABLE
      2        ONLINE  ONLINE       racp1vm2                 Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       racp1vm1                 STABLE
      2        ONLINE  ONLINE       racp1vm2                 STABLE
ora.cdp1.cdp
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.cdp2.cdp
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.cdp3.cdp
      1        ONLINE  ONLINE       racp1vm2                 STABLE
ora.cvu
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.qosmserver
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.racp1vm1.vip
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.racp1vm2.vip
      1        ONLINE  ONLINE       racp1vm2                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       racp1vm1                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       racp1vm2                 STABLE
--------------------------------------------------------------------------------
[[email protected] diag]#

2 Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Mouhamadou Diaw
Mouhamadou Diaw

Senior Consultant