Infrastructure at your Service

Jérôme Dubar

Reimaging an old X3-2/X4-2 ODA

Introduction

X3-2 and X4-2 ODAs are still very capable pieces of hardware in 2018. With 256GB of RAM and at least 16 cores per node, and with 18TB RAW disk capacity as a standard these appliances are far from obsolete even you probably don’t have any more support on the hardware from Oracle.
If you own several ODAs of this kind, hardware support may not really be a problem. If something fails, you can use the other ODA for spare parts.

You probably missed some patches on your old ODAs. Why? Maybe because it’s not so easy to patch, and it’s even more difficult if you don’t patch regularly. Or maybe just because you don’t want to add more tasks to your job (applying each patch is just like never stop patching).

So if you want to give a second life to your ODA, you’d better reimage it.

Reimaging: how to do?

Reimaging is the best way to do the cleanup of your ODA. Current deployment packages are certified for all the ODAs, except from V1 (first generation before the X3-2).

You first have to download all the needed files from MOS. Pay attention to download the deployment packages for OAKCLI stack because ODACLI is limited to lite and newer ODAs.

Assuming you’re using a bare metal configuration and you want to deploy the latest ODA version 12.2.1.4.0, you will need the following files :

  • 12999313 : ISO for reimaging
  • 28216780 : patch for OAKCLI stack (because reimaging actually does not update bioses and firmwares)
  • 12978712 : appliance server for OAKCLI stack
  • 17770873,  19520042 et 27449599 : rdbms clones for database 11.2.0.4, 12.1.0.2 and 12.2.0.1

Network configuration and disk configuration didn’t change: you still need to provide all the IPs, VIPs, DNS and so on for the network, and disk configuration is still not so clear with external backup meaning that you will go for 85/15 repartition between DATA and RECO instead of the default 40/60 split. Don’t forget that you can change the redundancy level for each ASM diskgroup: DATA, RECO and REDO can use high redundancy, but normal redundancy will give you 50% more free space (18TB RAW is 9TB usable in normal redundancy and 6TB usable in high redundancy).

Step 1 – Connect the ISO as a CDROM through ILOM interface and reimage the servers

I won’t give you the extensive procedure for this part: nothing has changed regarding ODA reimaging during last years.

First step is to connect to the ILOM and virtually plug the ISO image on the server. Then, select the CDROM as the next boot device, and do a power cycle of the server. You’ll have to repeat this on the other node too. Reimaging lasts about 1h and is fully automatic. The latest step is still the longest one (post-installation procedure). Once the reimaging is done, each node should have a different default name: oak1 for node 0 and oak2 for node 1 (weird). If the nodes are both oak1, please check the cables connected to the shared storage: they must be connected according to the setup poster.

Step 2 – Configure basic network settings

Reimaging is always ending by a reboot, and depending on the appliance, it will ask you the kind of network you plan to use: Copper of Fiber. Then, through the ILOM, you need to launch the configure firstnet script:

/opt/oracle/oak/bin/oakcli configure firstnet

Repeat this configuration step on the second node. Now your nodes are visible through the network.

Step 3 – Deploy, cleanup, deploy…

Reimaging was so easy… But from now it will be a little more tricky. You now need to deploy the appliance: understand configure the complete network settings, install all the Oracle stack with Grid Infrastrucure, ASM, latest database engine and eventually create a first database. And you will need a graphical interface to configure all these parameters and launch the deployment. So, from the ILOM session, let’s unpack the necessary files, start a graphical session of Linux and launch the deployment GUI.

oakcli unpack -package /opt/dbi/p12978712_122140_Linux-x86-64_1of2.zip
oakcli unpack -package /opt/dbi/p12978712_122140_Linux-x86-64_2of2.zip
oakcli unpack -package /opt/dbi/p27449599_122140_Linux-x86-64.zip
startx
oakcli deploy

Graphical interface will help you to configure all the parameters, but don’t deploy straight away from now. Backup the configuration file and then edit it:

vi /opt/dbi/deploy_oda01

Review all the parameters and adjust them to perfectly match your needs (most of these parameters cannot be changed afterwards).

Now you can launch the real deployment and select your configuration file in the graphical interface:

oakcli deploy

First try will fail and it’s a normal behaviour. Failure is because of the ASM headers: they are still writen on the disks in the storage shelf. Reimaging did nothing on these disks. And already having ASM disks configured will make the deployment process to fail. Now you can exit the deployment and do a cleanup of the failed attempt.

/opt/oracle/oak/onecmd/cleanupDeploy.pl

Unfortunatly you cannot do the cleanup if nothing is already deployed, so you need this first failing attempt. Alternatively, you can do the cleanup before reimaging, or manually clean all the disks headers and partitions on the 20 disks before trying to deploy (with a dd), but it probably won’t be faster.

When the cleanup is done, the ODA will reboot and you’ll have to configure again the firstnet from the ILOM on both nodes.

/opt/oracle/oak/bin/oakcli configure firstnet

Finally, with a new graphical session you can restart the deployment, and this time, if your parameter file is OK, it will be succesful. Yes!

startx
oakcli deploy

Step 4 – Patch the server

It seems weird but reimaging actually doesn’t update the firmware, bios, ilom of the servers, nor the firmware of the disks in the storage shelf. Understand that reimaging is only a software reimaging of the nodes. This is an example of an ODA X4-2 configuration just after reimaging and deploying the appliance:

oakcli show version -detail
System Version  Component Name            Installed Version         Supported Version
--------------  ---------------           ------------------        -----------------
12.2.1.4.0
Controller_INT            11.05.03.00               Up-to-date
Controller_EXT            11.05.03.00               Up-to-date
Expander                  0018                      Up-to-date
SSD_SHARED                944A                      Up-to-date
HDD_LOCAL                 A720                      A7E0
HDD_SHARED {
[ c2d0,c2d1,c2d2,c2d      A720                      A7E0
3,c2d4,c2d5,c2d6,c2d
7,c2d8,c2d9,c2d11,c2
d12,c2d13,c2d14,c2d1
5,c2d16,c2d17,c2d18,
c2d19 ] [ c2d10 ]                 A7E0                      Up-to-date
}
ILOM                      3.2.4.46.a r101689        4.0.2.27.a r123795
BIOS                      25030100                  25060300
IPMI                      1.8.12.4                  Up-to-date
HMP                       2.4.1.0.11                Up-to-date
OAK                       12.2.1.4.0                Up-to-date
OL                        6.9                       Up-to-date
GI_HOME                   12.2.0.1.180417(2767      Up-to-date
4384,27464465)
DB_HOME                   12.2.0.1.180417(2767      Up-to-date
4384,27464465)

Hopefully you can apply the patch even if your ODA is already in the same software version as your patch. Well done Oracle.

So let’s register the patch files and do the patching of the servers (server will probably reboot):


oakcli unpack -package /opt/dbi/p282166780_122140_Linux-x86-64_1of3.zip
oakcli unpack -package /opt/dbi/p282166780_122140_Linux-x86-64_2of3.zip
oakcli unpack -package /opt/dbi/p282166780_122140_Linux-x86-64_3of3.zip
oakcli update -patch 12.2.1.4.0 --server
...


oakcli show version -detail

System Version  Component Name            Installed Version         Supported Version
--------------  ---------------           ------------------        -----------------
12.2.1.4.0
Controller_INT            11.05.03.00               Up-to-date
Controller_EXT            11.05.03.00               Up-to-date
Expander                  0018                      Up-to-date
SSD_SHARED                944A                      Up-to-date
HDD_LOCAL                 A7E0                      Up-to-date
HDD_SHARED {
[ c2d0,c2d1,c2d2,c2d      A720                      A7E0
3,c2d4,c2d5,c2d6,c2d
7,c2d8,c2d9,c2d11,c2
d12,c2d13,c2d14,c2d1
5,c2d16,c2d17,c2d18,
c2d19 ] [ c2d10 ]                 A7E0                      Up-to-date
}
ILOM                      4.0.2.27.a r123795        Up-to-date
BIOS                      25060300                  Up-to-date
IPMI                      1.8.12.4                  Up-to-date
HMP                       2.4.1.0.11                Up-to-date
OAK                       12.2.1.4.0                Up-to-date
OL                        6.9                       Up-to-date
GI_HOME                   12.2.0.1.180417(2767      Up-to-date
4384,27464465)
DB_HOME                   12.2.0.1.180417(2767      Up-to-date
4384,27464465)

Great, our servers are now up-to-date. But storage is still not OK.

Step 4 – Patch the storage

Patching the storage is quite easy (server will probably reboot):

oakcli update -patch 12.2.1.4.0 --storage
...

oakcli show version -detail

System Version  Component Name            Installed Version         Supported Version
--------------  ---------------           ------------------        -----------------
12.2.1.4.0
Controller_INT            11.05.03.00               Up-to-date
Controller_EXT            11.05.03.00               Up-to-date
Expander                  0018                      Up-to-date
SSD_SHARED                944A                      Up-to-date
HDD_LOCAL                 A7E0                      Up-to-date
HDD_SHARED                A7E0                      Up-to-date
ILOM                      4.0.2.27.a r123795        Up-to-date
BIOS                      25060300                  Up-to-date
IPMI                      1.8.12.4                  Up-to-date
HMP                       2.4.1.0.11                Up-to-date
OAK                       12.2.1.4.0                Up-to-date
OL                        6.9                       Up-to-date
GI_HOME                   12.2.0.1.180417(2767      Up-to-date
4384,27464465)
DB_HOME                   12.2.0.1.180417(2767      Up-to-date
4384,27464465)

Everything is OK now!

Conclusion – A few more things

  • When redeploying, consider changing the redundancy of the diskgroups and the partitionning of the disk if needed. This can only be configured during deployment. Disks parameters are located in the deployment file (DISKGROUPREDUNDANCYS and DBBackupType)
  • Always check that all the components are up-to-date to keep your ODA in a consistent state. Check on both nodes because local patching is also possible, and it could make no sense if the nodes are running different level of patch
  • Don’t forget to check/apply your licenses on your ODA because using Oracle software is for sure not free
  • You have to know that a freshly redeployed ODA will have 12.2 database compatibility on diskgroups, making the use of acfs mandatory for your old databases. For me it’s a real drawback considering that acfs is adding useless complexity to ASM
  • Don’t forget to deploy the other dbhomes according to your needs

Leave a Reply

Jérôme Dubar
Jérôme Dubar

Consultant