Infrastructure at your Service

Recently we wanted to create an ACFS filesystem on a brand new 12.1.0.2 GI installation on Oracle Linux 7.1. According to the documentation this should not be an issue as “Oracle Linux 7 with the Unbreakable Enterprise kernel: 3.8.13-33.el7uek.x86_64 or later” is supported.

Read More
The “or later” is the important point. In our case:

uname -a
Linux racp1vm1 3.8.13-55.1.6.el7uek.x86_64 #2 SMP Wed Feb 11 14:18:22 PST 2015 x86_64 x86_64 x86_64 GNU/Linux

 

Should be fine as this is later than “3.8.13-33.el7uek.x86_64”. So let’s start by creating a diskgroup:

[[email protected] software]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.2.0 Production on Fri Jul 10 14:37:32 2015

Copyright (c) 1982, 2014, Oracle. All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> set lines 200
SQL> col path for a30
SQL> select name,path,header_status from v$asm_disk;
NAME PATH HEADER_STATU
------------------------------ ------------------------------ ------------

/dev/sde1 CANDIDATE
 /dev/sdd1 CANDIDATE
 /dev/sdc1 CANDIDATE
 DATA_0001 /dev/sdb1 MEMBER
 DATA_0000 /dev/sda1 MEMBER
SQL> create diskgroup DG_ACFS external redundancy disk '/dev/sde1';
Diskgroup created.

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

[[email protected] software]$ asmcmd lsdg
 State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
 MOUNTED EXTERN N 512 4096 1048576 15998 9179 0 9179 0 Y DATA/
 MOUNTED EXTERN N 512 4096 1048576 4999 4949 0 4949 0 N DG_ACFS/
 [[email protected] software]$
The next step is to create a volume on top of the diskgroup:

[[email protected] software]$ asmcmd volcreate -G DG_ACFS -s 4g vol_acfs
 ORA-15032: not all alterations performed
 ORA-15221: ASM operation requires compatible.asm of 11.2.0.0.0 or higher (DBD ERROR: OCIStmtExecute)
Ok, this is easy to fix:

SQL> alter diskgroup DG_ACFS set attribute 'compatible.asm'='12.1';
Diskgroup altered.

Let’s try again:

[[email protected] software]$ asmcmd volcreate -G DG_ACFS -s 4g vol_acfs
 ORA-15032: not all alterations performed
 ORA-15477: cannot communicate with the volume driver (DBD ERROR: OCIStmtExecute)

Hm. Seems something is wrong with the kernel modules:

 [[email protected] software]$ lsmod | grep -i acfs
 [[email protected] software]$

We can try to load the drivers manually:

su - root
 [[email protected] ~]# export ORACLE_HOME=/u01/app/12.1.0/grid
 [[email protected] ~]# cd $ORACLE_HOME/bin
 [[email protected] bin]# ./acfsload start
 ACFS-9459: ADVM/ACFS is not supported on this OS version: 'unknown'
 [[email protected] bin]#

What? According to the documentation it _is_ supported. Back in 2012 I wrote a small note on that message and how one may tweak it. But this is definitely not recommended.

So what is the cause of this? You can check the available kernel modules for the GI with:

[[email protected] grid]# find . -name *.ko ...
 ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleadvm.ko
 ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleacfs.ko
 ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleafd.ko
 ./usm/install/Novell/SLES11/x86_64/SP2/3.0.13-0.27/xen/bin/oracleoks.ko

Nothing about OL7. Hm. Maybe we need to apply a PSU that provides the required modules. I am on the 12.1.0.2 base release:

$ORACLE_HOME/OPatch/opatch lsinv
 [[email protected] software]$ $ORACLE_HOME/OPatch/opatch lsinv
Oracle Interim Patch Installer version 12.1.0.1.8
Copyright (c) 2015, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/12.1.0/grid
 Central Inventory : /u01/app/oraInventory
 from : /u01/app/12.1.0/grid/oraInst.loc
 OPatch version : 12.1.0.1.8
 OUI version : 12.1.0.2.0
 Log file location : /u01/app/12.1.0/grid/cfgtoollogs/opatch/opatch2015-07-10_14-50-34PM_1.log
Lsinventory Output file location : /u01/app/12.1.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2015-07-10_14-50-34PM.txt
 
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: racp1vm2
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1):

Oracle Grid Infrastructure 12c 12.1.0.2.0
There are 1 products installed in this Oracle Home.

There are no Interim patches installed in this Oracle Home.

Patch level status of Cluster nodes :

Patching Level Nodes
-------------- -----
0 racp1vm2,racp1vm1

--------------------------------------------------------------------------------

Let’s check what the psu provides:

unzip gi_121023_p20485724_121020_Linux-x86-64.zip
 find 20485724/ -name *.ko | grep EL7
 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleacfs.ko
 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleafd.ko
 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleadvm.ko
 20485724/20299018/files/usm/install/Oracle/EL7UEK/x86_64/3.8.13-35/3.8.13-35-x86_64/bin/oracleoks.ko
 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleacfs.ko
 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleafd.ko
 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleadvm.ko
 20485724/20299018/files/usm/install/Oracle/EL7/x86_64/3.10.0-123/3.10.0-123-x86_64/bin/oracleoks.ko

Looks much better. Once the PSU is applied…:

[[email protected] ~]$ /u01/app/12.1.0/grid/OPatch/opatch lsinv | grep Patch
 Oracle Interim Patch Installer version 12.1.0.1.8
 OPatch version : 12.1.0.1.8
 Patch 20299023 : applied on Fri Jul 10 15:03:44 CEST 2015
 Unique Patch ID: 18703022
 Patch description: "Database Patch Set Update : 12.1.0.2.3 (20299023)"
 Sub-patch 19769480; "Database Patch Set Update : 12.1.0.2.2 (19769480)"
 Patch 20299022 : applied on Fri Jul 10 15:03:15 CEST 2015
 Unique Patch ID: 18582504
 Patch description: "OCW Patch Set Update : 12.1.0.2.3 (20299022)"
 Patch 20299018 : applied on Fri Jul 10 15:02:20 CEST 2015
 Unique Patch ID: 18582442
 Patch description: "ACFS Patch Set Update : 12.1.0.2.3 (20299018)"
 Patch 19872484 : applied on Fri Jul 10 15:01:44 CEST 2015
 Unique Patch ID: 18291456
 Patch description: "WLM Patch Set Update: 12.1.0.2.2 (19872484)"
 OPatch succeeded.
 [[email protected] ~]$

… let’s check the kernel modules again:

[[email protected] ~]# lsmod | grep -i acfs
 oracleacfs 3335385 0
 oracleoks 499875 2 oracleacfs,oracleadvm
Looks good, now. Can we create the volume?:
 [[email protected] ~]$ asmcmd volcreate -G DG_ACFS -s 4g vol_acfs
 [[email protected] ~]$ asmcmd volinfo --all
 Diskgroup Name: DG_ACFS
Volume Name: VOL_ACFS
 Volume Device: /dev/asm/vol_acfs-288
 State: ENABLED
 Size (MB): 4096
 Resize Unit (MB): 64
 Redundancy: UNPROT
 Stripe Columns: 8
 Stripe Width (K): 1024

Perfect. Now the acfs filesystem can be created:

su -
 [[email protected] ~]# mkfs.acfs /dev/asm/vol_acfs-288
 mkfs.acfs: version = 12.1.0.2.0
 mkfs.acfs: on-disk version = 39.0
 mkfs.acfs: volume = /dev/asm/vol_acfs-288
 mkfs.acfs: volume size = 4294967296 ( 4.00 GB )
 mkfs.acfs: Format complete.

Hope this helps …

2 Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Daniel Westermann
Daniel Westermann

Principal Consultant & Technology Leader Open Infrastructure