Recently we wanted to create an ACFS filesystem on a brand new 12.1.0.2 GI installation on Oracle Linux 7.1. According to the documentation this should not be an issue as “Oracle Linux 7 with the Unbreakable Enterprise kernel: 3.8.13-33.el7uek.x86_64 or later” is supported.
The “or later” is the important point. In our case:
uname -a Linux racp1vm1 3.8.13-55.1.6.el7uek.x86_64 #2 SMP Wed Feb 11 14:18:22 PST 2015 x86_64 x86_64 x86_64 GNU/Linux
Should be fine as this is later than “3.8.13-33.el7uek.x86_64”. So let’s start by creating a diskgroup:
[grid@racp1vm2 software]$ sqlplus / as sysasm SQL*Plus: Release 12.1.0.2.0 Production on Fri Jul 10 14:37:32 2015 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> set lines 200 SQL> col path for a30 SQL> select name,path,header_status from v$asm_disk; NAME PATH HEADER_STATU ------------------------------ ------------------------------ ------------ /dev/sde1 CANDIDATE /dev/sdd1 CANDIDATE /dev/sdc1 CANDIDATE DATA_0001 /dev/sdb1 MEMBER DATA_0000 /dev/sda1 MEMBER SQL> create diskgroup DG_ACFS external redundancy disk '/dev/sde1'; Diskgroup created. SQL> exit Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options [grid@racp1vm2 software]$ asmcmd lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 4096 1048576 15998 9179 0 9179 0 Y DATA/ MOUNTED EXTERN N 512 4096 1048576 4999 4949 0 4949 0 N DG_ACFS/ [grid@racp1vm2 software]$ The next step is to create a volume on top of the diskgroup: [grid@racp1vm2 software]$ asmcmd volcreate -G DG_ACFS -s 4g vol_acfs ORA-15032: not all alterations performed ORA-15221: ASM operation requires compatible.asm of 11.2.0.0.0 or higher (DBD ERROR: OCIStmtExecute) Ok, this is easy to fix: SQL> alter diskgroup DG_ACFS set attribute 'compatible.asm'='12.1'; Diskgroup altered.
Let’s try again:
[grid@racp1vm2 software]$ asmcmd volcreate -G DG_ACFS -s 4g vol_acfs ORA-15032: not all alterations performed ORA-15477: cannot communicate with the volume driver (DBD ERROR: OCIStmtExecute)
Hm. Seems something is wrong with the kernel modules:
[grid@racp1vm2 software]$ lsmod | grep -i acfs [grid@racp1vm2 software]$
We can try to load the drivers manually:
su - root [root@racp1vm2 ~]# export ORACLE_HOME=/u01/app/12.1.0/grid [root@racp1vm2 ~]# cd $ORACLE_HOME/bin [root@racp1vm2 bin]# ./acfsload start ACFS-9459: ADVM/ACFS is not supported on this OS version: 'unknown' [root@racp1vm2 bin]#
What? According to the documentation it _is_ supported. Back in 2012 I wrote a small note on that message and how one may tweak it. But this is definitely not recommended.
So what is the cause of this? You can check the available kernel modules for the GI with:
[root@racp1vm2 grid]# find . -name *.ko ... ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleadvm.ko ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleacfs.ko ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleafd.ko ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleoks.ko
Nothing about OL7. Hm. Maybe we need to apply a PSU that provides the required modules. I am on the 12.1.0.2 base release:
$ORACLE_HOME/OPatch/opatch lsinv [grid@racp1vm2 software]$ $ORACLE_HOME/OPatch/opatch lsinv Oracle Interim Patch Installer version 12.1.0.1.8 Copyright (c) 2015, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/12.1.0/grid Central Inventory : /u01/app/oraInventory from : /u01/app/12.1.0/grid/oraInst.loc OPatch version : 12.1.0.1.8 OUI version : 12.1.0.2.0 Log file location : /u01/app/12.1.0/grid/cfgtoollogs/opatch/opatch2015-07-10_14-50-34PM_1.log Lsinventory Output file location : /u01/app/12.1.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2015-07-10_14-50-34PM.txt -------------------------------------------------------------------------------- Local Machine Information:: Hostname: racp1vm2 ARU platform id: 226 ARU platform description:: Linux x86-64 Installed Top-level Products (1): Oracle Grid Infrastructure 12c 12.1.0.2.0 There are 1 products installed in this Oracle Home. There are no Interim patches installed in this Oracle Home. Patch level status of Cluster nodes : Patching Level Nodes -------------- ----- 0 racp1vm2,racp1vm1 --------------------------------------------------------------------------------
Let’s check what the psu provides:
unzip gi_121023_p20485724_121020_Linux-x86-64.zip find 20485724/ -name *.ko | grep EL7 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleacfs.ko 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleafd.ko 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleadvm.ko 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleoks.ko 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleacfs.ko 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleafd.ko 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleadvm.ko 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleoks.ko
Looks much better. Once the PSU is applied…:
[grid@racp1vm1 ~]$ /u01/app/12.1.0/grid/OPatch/opatch lsinv | grep Patch Oracle Interim Patch Installer version 12.1.0.1.8 OPatch version : 12.1.0.1.8 Patch 20299023 : applied on Fri Jul 10 15:03:44 CEST 2015 Unique Patch ID: 18703022 Patch description: "Database Patch Set Update : 12.1.0.2.3 (20299023)" Sub-patch 19769480; "Database Patch Set Update : 12.1.0.2.2 (19769480)" Patch 20299022 : applied on Fri Jul 10 15:03:15 CEST 2015 Unique Patch ID: 18582504 Patch description: "OCW Patch Set Update : 12.1.0.2.3 (20299022)" Patch 20299018 : applied on Fri Jul 10 15:02:20 CEST 2015 Unique Patch ID: 18582442 Patch description: "ACFS Patch Set Update : 12.1.0.2.3 (20299018)" Patch 19872484 : applied on Fri Jul 10 15:01:44 CEST 2015 Unique Patch ID: 18291456 Patch description: "WLM Patch Set Update: 12.1.0.2.2 (19872484)" OPatch succeeded. [grid@racp1vm1 ~]$
… let’s check the kernel modules again:
[root@racp1vm1 ~]# lsmod | grep -i acfs oracleacfs 3335385 0 oracleoks 499875 2 oracleacfs,oracleadvm Looks good, now. Can we create the volume?: [grid@racp1vm1 ~]$ asmcmd volcreate -G DG_ACFS -s 4g vol_acfs [grid@racp1vm1 ~]$ asmcmd volinfo --all Diskgroup Name: DG_ACFS Volume Name: VOL_ACFS Volume Device: /dev/asm/vol_acfs-288 State: ENABLED Size (MB): 4096 Resize Unit (MB): 64 Redundancy: UNPROT Stripe Columns: 8 Stripe Width (K): 1024
Perfect. Now the acfs filesystem can be created:
su - [root@racp1vm1 ~]# mkfs.acfs /dev/asm/vol_acfs-288 mkfs.acfs: version = 12.1.0.2.0 mkfs.acfs: on-disk version = 39.0 mkfs.acfs: volume = /dev/asm/vol_acfs-288 mkfs.acfs: volume size = 4294967296 ( 4.00 GB ) mkfs.acfs: Format complete.
Hope this helps …