Infrastructure at your Service

Marc Wagner

Upgrade AHF and TFA on an ODA

By December 22, 2021 Oracle 4 Comments

TFA (Trace File Analyzer) is part of AHF (Autonomous Health Framework). Those tools are preinstalled and part of ODA (Oracle Database Appliance). As you might know patching and upgrading are normally always going through ODA global Bundle patches. AHF can, without any problem, be upgraded independently. In this blog I wanted to share with you how I upgraded TFA with latest v21.4 version. The upgrade is performed with root user. This version addresses CVE-2021-45105/CVE-2021-44228/CVE-2021-45046. For reminder Apache Log4j Vulnerabilities are covered by CVE-2021-44228 and CVE-2021-45046.

Check current version of TFA

First we can check if TFA is up and running and which version is currrently used.

[[email protected] ~]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status
WARNING - TFA Software is older than 180 days. Please consider upgrading TFA to the latest version.

.------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID  | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 5388 | 5000 | 20.1.3.0.0 | 20130020200429161658 | COMPLETE         |
'-----------+---------------+------+------+------------+----------------------+------------------'


As we can see we are currently running TFA/AHF 20.1.3.0.0 version.

Check running processes

We can check as well the TFA running processes.

[[email protected] ~]# ps -ef | grep -i tfa | grep -v grep
root      4536     1  0 Oct18 ?        00:18:06 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root      5388     1  0 Oct18 ?        02:55:07 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
[[email protected] ~]#


Check the location of AHF

It is important to check in which directory AHF is currently installed in order to provide appropriate directory in the upgrade command option. Thus the setup script will be able to see that there is a current version installed and will suggest to upgrade it. Otherwise another new AHF installation will be performed.

[[email protected] ~]# cat /etc/oracle.ahf.loc
/opt/oracle/dcs/oracle.ahf


AHF is installed on the ODA in the /opt/oracle/dcs/oracle.ahf directory.

Backup of the current AHF version

Before doing any modification it is important to backup current AHF version for fallback if needed. I have been doing a tar of the currrent installation directory.

[[email protected] ~]# cd /opt/oracle/dcs

[[email protected] dcs]# ls -ltrh
total 25M
drwxr-xr-x.  3 root   root     4.0K Jul  2  2019 rdbaas
drwxr-xr-x.  3 root   root     4.0K Jul  2  2019 scratch
drwx------   2 root   root     4.0K Jul  2  2019 dcsagent_wallet
drwxr-xr-x   2 root   root     4.0K Jul  4  2019 ft
drwxr-xr-x   2 root   root     4.0K Aug 11  2019 Inventory
drwxr-xr-x   4 root   root     4.0K May 17  2020 dcs-ui
-rwxr-xr-x   1 root   root     6.8K May 21  2020 configuredcs.pl
-rw-r--r--   1 root   root      25M May 21  2020 dcs-ui.zip
drwxr-xr-x   4 root   root     4.0K Sep  2  2020 repo
-rw-r--r--   1 root   root        0 Sep  2  2020 dcscontroller-stderr.log
-rw-r--r--   1 root   root     6.7K Sep  3  2020 dcscontroller-stdout.log
drwxr-xr-x   6 oracle oinstall  32K Sep  3  2020 commonstore
drwxr-xr-x  12 root   root     4.0K Sep  3  2020 oracle.ahf
drwxr-xr-x.  2 root   root     4.0K Sep  3  2020 agent
drwxr-xr-x.  2 root   root     4.0K Sep  3  2020 sample
drwxr-xr-x   4 root   root     4.0K Sep  3  2020 java
drwxr-xr-x.  3 root   root     4.0K Sep  3  2020 conf
drwxr-xr-x.  3 root   root     4.0K Sep  3  2020 dcscli
drwxr-xr-x.  2 root   root     4.0K Sep  3  2020 bin
drwx------.  5 root   root      20K Dec 21 00:00 log

[[email protected] dcs]# mkdir /root/backup_ahf_for_upgrade/

[[email protected] dcs]# tar -czf /root/backup_ahf_for_upgrade/oracle.ahf.20.1.3.0.0.tar ./oracle.ahf

[[email protected] dcs]# ls -ltrh /root/backup_ahf_for_upgrade
total 1.3G
-rw-r--r-- 1 root root 1.3G Dec 21 14:26 oracle.ahf.20.1.3.0.0.tar


Download new AHF version

You can download latest AHF version through my oracle support portal. Download patch 30166242 :
Patch 30166242: PLACEHOLDER – DOWNLOAD LATEST AHF (TFA and ORACHK/EXACHK)

I have created a directory on the ODA to upload the patch :

[[email protected] dcs]# mkdir /u01/app/patch/TFA


Upgrade AHF on the ODA

In this part we will see the procedure to upgrade AHF on the ODA. We first need to unzip the AHF-LINUX_v21.4.0.zip file and run ahf_setup. The installation script will recognise the existing 20.1.3 version and suggest to upgrade it.

[email protected] dcs]# cd /u01/app/patch/TFA

[[email protected] TFA]# ls -ltrh
total 394M
-rw-r--r-- 1 root root 393M Dec 21 10:16 AHF-LINUX_v21.4.0.zip

[[email protected] TFA]# unzip -q AHF-LINUX_v21.4.0.zip

[[email protected] TFA]# ls -ltrh
total 792M
-r-xr-xr-x 1 root root 398M Dec 20 19:28 ahf_setup
-rw-r--r-- 1 root root  384 Dec 20 19:30 ahf_setup.dat
-rw-r--r-- 1 root root 1.5K Dec 20 19:31 README.txt
-rw-r--r-- 1 root root  625 Dec 20 19:31 oracle-tfa.pub
-rw-r--r-- 1 root root 393M Dec 21 10:16 AHF-LINUX_v21.4.0.zip

[[email protected] TFA]# ./ahf_setup -ahf_loc /opt/oracle/dcs -data_dir /opt/oracle/dcs

AHF Installer for Platform Linux Architecture x86_64

AHF Installation Log : /tmp/ahf_install_214000_58089_2021_12_21-14_30_06.log

Starting Autonomous Health Framework (AHF) Installation

AHF Version: 21.4.0 Build Date: 202112200745

AHF is already installed at /opt/oracle/dcs/oracle.ahf

Installed AHF Version: 20.1.3 Build Date: 202004291616

Do you want to upgrade AHF [Y]|N : Y

Upgrading /opt/oracle/dcs/oracle.ahf

Shutting down AHF Services
Stopped OSWatcher
Nothing to do !
Shutting down TFA
/etc/init.d/init.tfa: line 661: /sbin/stop: No such file or directory
. . . . .
Killing TFA running with pid 5388
. . .
Successfully shutdown TFA..

Starting AHF Services
Starting TFA..
Waiting up to 100 seconds for TFA to be started..
. . . . .
. . . . .
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands


Do you want AHF to store your My Oracle Support Credentials for Automatic Upload ? Y|[N] : N

AHF is successfully upgraded to latest version

.-----------------------------------------------------------------.
| Host      | TFA Version | TFA Build ID         | Upgrade Status |
+-----------+-------------+----------------------+----------------+
| ODA01     |  21.4.0.0.0 | 21400020211220074549 | UPGRADED       |
'-----------+-------------+----------------------+----------------'

Moving /tmp/ahf_install_214000_58089_2021_12_21-14_30_06.log to /opt/oracle/dcs/oracle.ahf/data/ODA01/diag/ahf/

[[email protected] TFA]#


Check new AHF version

We can check that the new version of AHF is 21.4.

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl version

AHF version: 21.4.0


Check TFA running processes

We can check that TFA is up and running.

[[email protected] TFA]# ps -ef | grep -i tfa | grep -v grep
root      4536     1  0 Oct18 ?        00:18:06 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root     61938     1 62 14:31 ?        00:01:36 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
[[email protected] TFA]#


After the upgrade script is completed there might still be some TFA processes running in order to rebuild the inventory :

root     15469 15077  0 14:58 ?        00:00:00 sh -c /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl rediscover -mode full > /dev/null 2>&1
root     15470 15469  0 14:58 ?        00:00:00 /bin/sh /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl rediscover -mode full
root     15505 15500  0 14:58 ?        00:00:00 /bin/sh /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl.tfa rediscover -mode full
root     15524 15505  1 14:58 ?        00:00:00 /u01/app/19.0.0.0/grid/perl/bin/perl /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl.pl rediscover -mode full


Make sure all those processes are completed successfully (not existing any more) before stopping AHF. Otherwise your inventory status will end with a STOPPED status.

Check status of AHF

We can check AHF status.

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf


.-------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+-------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 61938 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+-------+------+------------+----------------------+------------------'


No scheduler for any ID

orachk daemon is not running

[[email protected] TFA]#


TFA is running. No AHF scheduler. No orachk daemon.

Stop AHF and TFA

To check all is working as expected, let’s stop AHF.

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl stopahf

Stopping TFA from the Command Line
Stopped OSWatcher
Nothing to do !
Please wait while TFA stops
Please wait while TFA stops
TFA-00002 Oracle Trace File Analyzer (TFA) is not running
TFA Stopped Successfully
Successfully stopped TFA..

orachk scheduler is not running


There is still one process for TFA, the one from init.d :

[[email protected] TFA]# ps -ef | grep -i tfa | grep -v grep
root      4536     1  0 Oct18 ?        00:18:06 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
[[email protected] TFA]#


We are going to stop it :

[[email protected] TFA]# /etc/init.d/init.tfa stop
Stopping TFA from init for shutdown/reboot
Nothing to do !
TFA Stopped Successfully
Successfully stopped TFA..


And there is no more TFA processes up and running :

[[email protected] TFA]# ps -ef | grep -i tfa | grep -v grep
[[email protected] TFA]#


Start AHF

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl startahf

Starting TFA..
Waiting up to 100 seconds for TFA to be started..
. . . . .
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands

INFO: Starting orachk scheduler in background. Details for the process can be found at /opt/oracle/dcs/oracle.ahf/data/ODA01/diag/orachk/compliance_start_211221_143845.log


We can check TFA status :

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status

.-------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+-------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 87371 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+-------+------+------------+----------------------+------------------'


We can check AHF status as well and see that we have now scheduler and orachk daemon up and running :

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf


.-------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+-------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 87371 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+-------+------+------------+----------------------+------------------'

------------------------------------------------------------

Master node = ODA01

orachk daemon version = 21.4.0

Install location = /opt/oracle/dcs/oracle.ahf/orachk

Started at = Tue Dec 21 14:38:57 CET 2021

Scheduler type = TFA Scheduler

Scheduler PID:  87371

------------------------------------------------------------
ID: orachk.autostart_client_oratier1
------------------------------------------------------------
AUTORUN_FLAGS  =  -usediscovery -profile oratier1 -dball -showpass -tag autostart_client_oratier1 -readenvconfig
COLLECTION_RETENTION  =  7
AUTORUN_SCHEDULE  =  3 2 * * 1,2,3,4,5,6
------------------------------------------------------------
------------------------------------------------------------
ID: orachk.autostart_client
------------------------------------------------------------
AUTORUN_FLAGS  =  -usediscovery -tag autostart_client -readenvconfig
COLLECTION_RETENTION  =  14
AUTORUN_SCHEDULE  =  3 3 * * 0
------------------------------------------------------------

Next auto run starts on Dec 22, 2021 02:03:00

ID:orachk.AUTOSTART_CLIENT_ORATIER1


We can also check TFA processes :

[[email protected] TFA]#  ps -ef | grep -i tfa | grep -v grep
root     86989     1  0 14:38 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root     87371     1 19 14:38 ?        00:00:13 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
root     92789 87371 38 14:39 ?        00:00:00 /u01/app/19.0.0.0/grid/perl/bin/perl /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl.pl availability product Europe/Zurich
[[email protected] TFA]#


Stop AHF and TFA

We will stop AHF and TFA again.

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl stopahf

Stopping TFA from the Command Line
Nothing to do !
Please wait while TFA stops
Please wait while TFA stops
TFA-00002 Oracle Trace File Analyzer (TFA) is not running
TFA Stopped Successfully
Successfully stopped TFA..

Stopping orachk scheduler ...
Removing orachk cache discovery....
No orachk cache discovery found.



Unable to send message to TFA



Removed orachk from inittab


Stopped orachk


AHF status checks :

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf

TFA-00002 Oracle Trace File Analyzer (TFA) is not running


No scheduler for any ID

orachk daemon is not running


TFA status checks :

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status
TFA-00002 Oracle Trace File Analyzer (TFA) is not running


Check processes and stop TFA init.d :

[[email protected] TFA]#  ps -ef | grep -i tfa | grep -v grep
root     86989     1  0 14:38 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null

[[email protected] TFA]# /etc/init.d/init.tfa stop
Stopping TFA from init for shutdown/reboot
Nothing to do !
TFA Stopped Successfully
Successfully stopped TFA..

[[email protected] TFA]#  ps -ef | grep -i tfa | grep -v grep
[[email protected] TFA]#


Restart only TFA

Finally we only want to keep TFA up and running. No AHF scheduling or orachk daemon. So we are only going to start TFA.

[[email protected] TFA]# /etc/init.d/init.tfa start
Starting TFA..
Waiting up to 100 seconds for TFA to be started..
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands


Final checks

TFA running processes :

[[email protected] TFA]#  ps -ef | grep -i tfa | grep -v grep
root      5344     1  0 14:43 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null
root      5732     1 77 14:43 ?        00:00:11 /opt/oracle/dcs/oracle.ahf/jre/bin/java -server -Xms512m -Xmx1024m -Djava.awt.headless=true -Ddisable.checkForUpdate=true -XX:HeapDumpPath=/opt/oracle/dcs/oracle.ahf/data/ODA01/diag/tfa -XX:ParallelGCThreads=5 oracle.rat.tfa.TFAMain /opt/oracle/dcs/oracle.ahf/tfa
[[email protected] TFA]#


TFA status :

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/tfactl status

.------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID  | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 5732 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+------+------+------------+----------------------+------------------'
[[email protected] TFA]#


AHF status :

[[email protected] TFA]# /opt/oracle/dcs/oracle.ahf/bin/ahfctl statusahf


.------------------------------------------------------------------------------------------------.
| Host      | Status of TFA | PID  | Port | Version    | Build ID             | Inventory Status |
+-----------+---------------+------+------+------------+----------------------+------------------+
| ODA01     | RUNNING       | 5732 | 5000 | 21.4.0.0.0 | 21400020211220074549 | COMPLETE         |
'-----------+---------------+------+------+------------+----------------------+------------------'


No scheduler for any ID

orachk daemon is not running

[[email protected] TFA]#


Cleanup

We can still keep previous AHF version backup a few days just in case and remove it later.

The AHF installation files can be deleted :

[[email protected] ~]# cd /u01/app/patch/

[[email protected] patch]# ls -l TFA
total 810144
-rw-r--r-- 1 root root 411836201 Dec 21 10:16 AHF-LINUX_v21.4.0.zip
-r-xr-xr-x 1 root root 416913901 Dec 20 19:28 ahf_setup
-rw-r--r-- 1 root root       384 Dec 20 19:30 ahf_setup.dat
-rw-r--r-- 1 root root       625 Dec 20 19:31 oracle-tfa.pub
-rw-r--r-- 1 root root      1525 Dec 20 19:31 README.txt

[[email protected] patch]# rm -rf TFA

[[email protected] patch]# ls
[[email protected] patch]#


4 Comments

  • Hi, is there a Oracle doc to back this statement: “AHF can, without any problem, be upgraded independently.”?
    Also, check out the ‘ahfctl setupgrade’ option with staging, it’s very handy!

    • Marc Wagner says:

      Hi Ian,
      Thanks for your feedback.
      You can mainly refer to Doc ID 2550798.1.
      If not mistaken, installing and upgrading AHF is part of the user guide in the engineered systems section as well.
      More over when you open a SR and needs to provide TFA traces, and if you get the mention that your version is older than 180 days, support team usually recommends you to upgrade it.
      Finally have a look to Oracle Database Appliance: TheTrace File Analyzer: Examples using TFA (Doc ID 2287257.1). It is stating :
      “Note: While the TFA is bundled with each ODA software, we strongly recommend downloading the most current TFA.
      The most current version of the TFA can be downloaded from the following note:
      Autonomous Health Framework (AHF) – Including TFA and ORAchk/EXAChk (Doc ID 2550798.1)”
      Regards,
      Marc

  • Thanks!
    One reason why I asked is it looked like on the ODA 19.12 deployment, after update the schedules are gone.
    After an SR, one must look at the env file (location `orachk -showenvfile`) to see if the line `RAT_PERLEXE=/u01/app/19.12…` is NOT present.
    Regards, Ian

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Marc Wagner
Marc Wagner

Senior Consultant