By Mouhamadou Diaw
In this blog we are going to install an Oracle Restart 19c with ASM Filter Driver (AFD). I am using following configuration
-Oracle 19c
-Oracle Linux Server 7.6
-Kernel 4.14.35-1902.2.0.el7uek.x86_64
Once the Oracle Restart installed, I will create an ACFS filesystem
We will have to configure following disks for AFD
-/dev/sdc
-/dev/sdd
The Oracle downloaded software was unzipped to the GRID_HOME
1
|
[oracle@oraadserver u01]$ unzip -d /u01/app/19 .0.0.0 /grid LINUX.X64_193000_grid_home.zip |
After setting the variables ORACLE_HOME and ORACLE_BASE, we then use with user root the command asmcmd afd_label to provision disk devices for use with Oracle ASM Filter Driver.
1
2
3
4
|
[root@oraadserver dev] # export ORACLE_HOME=/u01/app/19.0.0.0/grid [root@oraadserver dev] # export ORACLE_BASE=/tmp [root@oraadserver dev] # /u01/app/19.0.0.0/grid/bin/asmcmd afd_label DATA /dev/sdc --init [root@oraadserver dev] # /u01/app/19.0.0.0/grid/bin/asmcmd afd_label CRS /dev/sdd --init |
We can verify the status of the disks with the command afd_lslbl
1
2
3
4
5
6
7
8
9
10
11
|
[root@oraadserver ~] # /u01/app/19.0.0.0/grid/bin/asmcmd afd_lslbl /dev/sdc -------------------------------------------------------------------------------- Label Duplicate Path ================================================================================ DATA /dev/sdc [root@oraadserver ~] # /u01/app/19.0.0.0/grid/bin/asmcmd afd_lslbl /dev/sdd -------------------------------------------------------------------------------- Label Duplicate Path ================================================================================ CRS /dev/sdd [root@oraadserver ~] # |
In my case with the kernel 4.14.35-1902.2.0.el7uek.x86_64, I had to apply the following
Patch 27494830: BUILD UEK5U2 COMPATIBLE ACFS GRID KERNEL MODULES
To avoid this error during the installation
1
2
|
Action - To proceed, do not specify or select the Oracle ASM Filter Driver option. Additional Information: - AFD-620: AFD is not supported on this operating system version: '4.14.35-1902.2.0.el7uek.x86_64' |
The patch was applied before the installation as described in the following documentation
How to Apply a Grid Infrastructure Patch Before Grid Infrastructure Configuration (before root.sh or rootupgrade.sh or gridsetup.bat) is Executed (Doc ID 1410202.1)
The opatch version was verified
1
2
3
4
5
|
[oracle@oraadserver u01]$ /u01/app/19 .0.0.0 /grid/OPatch/opatch version OPatch Version: 12.2.0.1.17 OPatch succeeded. [oracle@oraadserver u01]$ |
After the patch was unpacked
1
|
[oracle@oraadserver u01]$ unzip p27494830_193000ACFSRU_Linux-x86-64.zip |
And then the patch was applied
1
2
3
4
5
|
[oracle@oraadserver u01]$ /u01/app/19 .0.0.0 /grid/gridSetup .sh -silent -applyRU /u01/27494830/ Preparing the home to patch... Applying the patch /u01/27494830/ ... Successfully applied the patch. The log can be found at: /tmp/GridSetupActions2020-10-20_09-32-54PM/installerPatchActions_2020-10-20_09-32-54PM .log |
And now we lunch the install. Note that only some screenshots will be shown. And for the non shown pictures the default was kept.
1
2
3
|
[oracle@oraadserver grid]$ pwd /u01/app/19 .0.0.0 /grid [oracle@oraadserver grid]$ . /gridSetup .sh |
For the ASM diskgroups configuration
I got some warnings that Idecided to ignore (just a test environment)
The two scripts are executed
1
2
3
4
5
6
7
8
|
[root@oraadserver oraInventory] # /u01/app/oraInventory/orainstRoot.sh Changing permissions of /u01/app/oraInventory . Adding read ,write permissions for group. Removing read ,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@oraadserver oraInventory] # |
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
|
[root@oraadserver oraInventory] # /u01/app/19.0.0.0/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/19 .0.0.0 /grid Enter the full pathname of the local bin directory: [ /usr/local/bin ]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Creating /etc/oratab file ... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file : /u01/app/19 .0.0.0 /grid/crs/install/crsconfig_params The log of current session can be found at: /u01/app/oracle/crsdata/oraadserver/crsconfig/roothas_2020-10-20_09-58-51PM .log 2020 /10/20 21:58:55 CLSRSC-363: User ignored prerequisites during installation LOCAL ADD MODE Creating OCR keys for user 'oracle' , privgrp 'oinstall' .. Operation successful. LOCAL ONLY MODE Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root' , privgrp 'root' .. Operation successful. CRS-4664: Node oraadserver successfully pinned. 2020 /10/20 22:00:43 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' oraadserver 2020 /10/20 22:03:14 /u01/app/oracle/crsdata/oraadserver/olr/backup_20201020_220314 .olr 2451816761 2020 /10/20 22:03:49 CLSRSC-327: Successfully configured Oracle Restart for a standalone server [root@oraadserver oraInventory] # |
After the installation we can validate that the ASM was up
1
2
3
4
5
6
|
[root@oraadserver oraInventory] # crsctl check has CRS-4638: Oracle High Availability Services is online [root@oraadserver oraInventory] # ps -ef | grep pmon root 10153 1730 0 22:10 pts /0 00:00:00 grep --color=auto pmon oracle 26037 1 0 22:05 ? 00:00:00 asm_pmon_+ASM [root@oraadserver oraInventory] # |
Now using asmca, let’s configure an ACFS filesystem . Basically we
-Create a CRS diskgroup
-In CRS diskgroup we create a volume
-In this volume we create an ACFS filesystem and mount it
1
2
3
|
[oracle@oraadserver ~]$ which asmca /u01/app/19 .0.0.0 /grid/bin/asmca [oracle@oraadserver ~]$ asmca |
The ACFS filesystem is now mounted
1
2
3
4
|
[root@oraadserver ~] # df -h /share_acfs/ Filesystem Size Used Avail Use% Mounted on /dev/asm/acfs_vol-3 5.0G 319M 4.7G 7% /share_acfs [root@oraadserver ~] # |
Now let’s reboot the system and verify that the acfs filesystem is mounted.
1
2
3
4
5
6
7
8
9
10
|
[root@oraadserver ~] # df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 1.8G 8.0K 1.8G 1% /dev tmpfs 1.8G 637M 1.2G 35% /dev/shm tmpfs 1.8G 8.8M 1.8G 1% /run tmpfs 1.8G 0 1.8G 0% /sys/fs/cgroup /dev/mapper/ol-root 47G 14G 34G 29% / /dev/sda1 497M 230M 268M 47% /boot tmpfs 368M 0 368M 0% /run/user/54323 tmpfs 368M 0 368M 0% /run/user/0 |
No. When trying to manually mount it, I got this error
1
2
3
4
5
6
|
[root@oraadserver ~] # /bin/mount -t acfs /dev/asm/acfs_vol-3 /share_acfs mount .acfs: CLSU-00107: operating system function : open64; failed with error data: 2; at location: OOF_1 mount .acfs: CLSU-00101: operating system error message: No such file or directory mount .acfs: CLSU-00104: additional error information: open64 ( /dev/ofsctl ) mount .acfs: ACFS-00502: Failed to communicate with the ACFS driver. Verify the ACFS driver has been loaded. [root@oraadserver ~] # |
I then load the acfs drivers
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
[root@oraadserver ~] # /u01/app/19.0.0.0/grid/bin/acfsload start -s ACFS-9300: ADVM /ACFS distribution files found. ACFS-9314: Removing previous ADVM /ACFS installation. ACFS-9315: Previous ADVM /ACFS components successfully removed. ACFS-9294: updating file /etc/sysconfig/oracledrivers .conf ACFS-9307: Installing requested ADVM /ACFS software. ACFS-9294: updating file /etc/sysconfig/oracledrivers .conf ACFS-9308: Loading installed ADVM /ACFS drivers. ACFS-9321: Creating udev for ADVM /ACFS . ACFS-9323: Creating module dependencies - this may take some time . ACFS-9154: Loading 'oracleoks.ko' driver. ACFS-9154: Loading 'oracleadvm.ko' driver. ACFS-9154: Loading 'oracleacfs.ko' driver. ACFS-9327: Verifying ADVM /ACFS devices. ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec' . ACFS-9156: Detecting control device '/dev/ofsctl' . ACFS-9309: ADVM /ACFS installation correctness verified. [root@oraadserver ~] # |
When checking the volume, it was disabled
1
2
3
4
5
6
7
8
9
10
11
12
13
|
ASMCMD> volinfo --all Diskgroup Name: CRS Volume Name: ACFS_VOL Volume Device: /dev/asm/acfs_vol-3 State: DISABLED Size (MB): 5120 Resize Unit (MB): 64 Redundancy: UNPROT Stripe Columns: 8 Stripe Width (K): 1024 Usage: ACFS Mountpath: /share_acfs |
So I enable enable it
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
ASMCMD> volenable –all ASMCMD> volinfo --all Diskgroup Name: CRS Volume Name: ACFS_VOL Volume Device: /dev/asm/acfs_vol-3 State: ENABLED Size (MB): 5120 Resize Unit (MB): 64 Redundancy: UNPROT Stripe Columns: 8 Stripe Width (K): 1024 Usage: ACFS Mountpath: /share_acfs ASMCMD> |
And then I run the mount command
1
2
3
4
5
6
|
[root@oraadserver trace] # /bin/mount -t acfs /dev/asm/acfs_vol-3 /share_acfs [root@oraadserver trace] # df -h /share_acfs/ Filesystem Size Used Avail Use% Mounted on /dev/asm/acfs_vol-3 5.0G 319M 4.7G 7% /share_acfs [root@oraadserver trace] # |
It was strange to manually mount the ACFS filesystem at system reboot but it is mentioned in Oracle documentation
Starting with Oracle Database 12c, Oracle Restart configurations do not support the Oracle ACFS registry.
This means that the Oracle ACFS file system resource is supported only for Oracle Grid Infrastructure cluster configurations; it is not supported for Oracle Restart configurations. When using ACFS on Oracle Restart, we have to mount the filesystem manually mount and umount the ACFS filesystem
Conclusion
When using ACFS with Oracle Restart, we have to manually mount and umount the ACFS filesystem when the system stops or starts. This can be done by some customized scripts.