Infrastructure at your Service

In this blog we are going to install an Oracle Restart 19c with ASM Filter Driver (AFD). I am using following configuration
-Oracle 19c
-Oracle Linux Server 7.6
-Kernel 4.14.35-1902.2.0.el7uek.x86_64

Once the Oracle Restart installed, I will create an ACFS filesystem

We will have to configure following disks for AFD
-/dev/sdc
-/dev/sdd

The Oracle downloaded software was unzipped to the GRID_HOME

[[email protected] u01]$ unzip -d /u01/app/19.0.0.0/grid LINUX.X64_193000_grid_home.zip

After setting the variables ORACLE_HOME and ORACLE_BASE, we then use with user root the command asmcmd afd_label to provision disk devices for use with Oracle ASM Filter Driver.

[[email protected] dev]# export ORACLE_HOME=/u01/app/19.0.0.0/grid
[[email protected] dev]# export ORACLE_BASE=/tmp
[[email protected] dev]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_label DATA /dev/sdc --init
[[email protected] dev]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_label CRS /dev/sdd --init

We can verify the status of the disks with the command afd_lslbl

[[email protected] ~]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_lslbl /dev/sdc
--------------------------------------------------------------------------------
Label                     Duplicate  Path
================================================================================
DATA                                  /dev/sdc
[[email protected] ~]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_lslbl /dev/sdd
--------------------------------------------------------------------------------
Label                     Duplicate  Path
================================================================================
CRS                                   /dev/sdd
[[email protected] ~]#

In my case with the kernel 4.14.35-1902.2.0.el7uek.x86_64, I had to apply the following
Patch 27494830: BUILD UEK5U2 COMPATIBLE ACFS GRID KERNEL MODULES
To avoid this error during the installation

Action - To proceed, do not specify or select the Oracle ASM Filter Driver option.  Additional Information:
 - AFD-620: AFD is not supported on this operating system version: '4.14.35-1902.2.0.el7uek.x86_64' 

The patch was applied before the installation as described in the following documentation
How to Apply a Grid Infrastructure Patch Before Grid Infrastructure Configuration (before root.sh or rootupgrade.sh or gridsetup.bat) is Executed (Doc ID 1410202.1)
The opatch version was verified

[[email protected] u01]$ /u01/app/19.0.0.0/grid/OPatch/opatch version
OPatch Version: 12.2.0.1.17

OPatch succeeded.
[[email protected] u01]$

After the patch was unpacked

[[email protected] u01]$ unzip p27494830_193000ACFSRU_Linux-x86-64.zip

And then the patch was applied

[[email protected] u01]$ /u01/app/19.0.0.0/grid/gridSetup.sh -silent -applyRU /u01/27494830/
Preparing the home to patch...
Applying the patch /u01/27494830/...
Successfully applied the patch.
The log can be found at: /tmp/GridSetupActions2020-10-20_09-32-54PM/installerPatchActions_2020-10-20_09-32-54PM.log

And now we lunch the install. Note that only some screenshots will be shown. And for the non shown pictures the default was kept.

[[email protected] grid]$ pwd
/u01/app/19.0.0.0/grid
[[email protected] grid]$ ./gridSetup.sh

For the ASM diskgroups configuration

The groups of my installation

I got some warnings that Idecided to ignore (just a test environment)



The two scripts are executed

[[email protected] oraInventory]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[[email protected] oraInventory]#
[[email protected] oraInventory]# /u01/app/19.0.0.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.0.0.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/19.0.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/oraadserver/crsconfig/roothas_2020-10-20_09-58-51PM.log
2020/10/20 21:58:55 CLSRSC-363: User ignored prerequisites during installation
LOCAL ADD MODE
Creating OCR keys for user 'oracle', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node oraadserver successfully pinned.
2020/10/20 22:00:43 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'

oraadserver     2020/10/20 22:03:14     /u01/app/oracle/crsdata/oraadserver/olr/backup_20201020_220314.olr     2451816761
2020/10/20 22:03:49 CLSRSC-327: Successfully configured Oracle Restart for a standalone server
[[email protected] oraInventory]#


After the installation we can validate that the ASM was up

[[email protected] oraInventory]# crsctl check has
CRS-4638: Oracle High Availability Services is online
[[email protected] oraInventory]# ps -ef | grep pmon
root     10153  1730  0 22:10 pts/0    00:00:00 grep --color=auto pmon
oracle   26037     1  0 22:05 ?        00:00:00 asm_pmon_+ASM
[[email protected] oraInventory]#

Now using asmca, let’s configure an ACFS filesystem . Basically we
-Create a CRS diskgroup
-In CRS diskgroup we create a volume
-In this volume we create an ACFS filesystem and mount it

[[email protected] ~]$ which asmca
/u01/app/19.0.0.0/grid/bin/asmca
[[email protected] ~]$ asmca

The ACFS filesystem is now mounted

[[email protected] ~]# df -h /share_acfs/
Filesystem           Size  Used Avail Use% Mounted on
/dev/asm/acfs_vol-3  5.0G  319M  4.7G   7% /share_acfs
[[email protected] ~]#

Now let’s reboot the system and verify that the acfs filesystem is mounted.

[[email protected] ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             1.8G  8.0K  1.8G   1% /dev
tmpfs                1.8G  637M  1.2G  35% /dev/shm
tmpfs                1.8G  8.8M  1.8G   1% /run
tmpfs                1.8G     0  1.8G   0% /sys/fs/cgroup
/dev/mapper/ol-root   47G   14G   34G  29% /
/dev/sda1            497M  230M  268M  47% /boot
tmpfs                368M     0  368M   0% /run/user/54323
tmpfs                368M     0  368M   0% /run/user/0

No. When trying to manually mount it, I got this error

[[email protected] ~]# /bin/mount -t acfs /dev/asm/acfs_vol-3 /share_acfs
mount.acfs: CLSU-00107: operating system function: open64; failed with error data: 2; at location: OOF_1
mount.acfs: CLSU-00101: operating system error message: No such file or directory
mount.acfs: CLSU-00104: additional error information: open64 (/dev/ofsctl)
mount.acfs: ACFS-00502: Failed to communicate with the ACFS driver.  Verify the ACFS driver has been loaded.
[[email protected] ~]#

I then load the acfs drivers

[[email protected] ~]#  /u01/app/19.0.0.0/grid/bin/acfsload start -s
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9309: ADVM/ACFS installation correctness verified.
[[email protected] ~]#

When checking the volume, it was disabled

ASMCMD> volinfo --all
Diskgroup Name: CRS

         Volume Name: ACFS_VOL
         Volume Device: /dev/asm/acfs_vol-3
         State: DISABLED
         Size (MB): 5120
         Resize Unit (MB): 64
         Redundancy: UNPROT
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage: ACFS
         Mountpath: /share_acfs

So I enable enable it

ASMCMD> volenable –all
ASMCMD> volinfo --all
Diskgroup Name: CRS

         Volume Name: ACFS_VOL
         Volume Device: /dev/asm/acfs_vol-3
         State: ENABLED
         Size (MB): 5120
         Resize Unit (MB): 64
         Redundancy: UNPROT
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage: ACFS
         Mountpath: /share_acfs

ASMCMD>

And then I run the mount command

[[email protected] trace]# /bin/mount -t acfs /dev/asm/acfs_vol-3 /share_acfs

[[email protected] trace]# df -h /share_acfs/
Filesystem           Size  Used Avail Use% Mounted on
/dev/asm/acfs_vol-3  5.0G  319M  4.7G   7% /share_acfs
[[email protected] trace]#

It was strange to manually mount the ACFS filesystem at system reboot but it is mentioned in Oracle documentation
Starting with Oracle Database 12c, Oracle Restart configurations do not support the Oracle ACFS registry.
This means that the Oracle ACFS file system resource is supported only for Oracle Grid Infrastructure cluster configurations; it is not supported for Oracle Restart configurations. When using ACFS on Oracle Restart, we have to mount the filesystem manually mount and umount the ACFS filesystem

Conclusion

When using ACFS with Oracle Restart, we have to manually mount and umount the ACFS filesystem when the system stops or starts. This can be done by some customized scripts.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Mouhamadou Diaw
Mouhamadou Diaw

Consultant