I recently had problems with Enterprise Manager 13.1.0.0, receiving such alerts:
EM Event Critical hostname Memory Utilization is 93,205 % crossed warning (80%) or critical (90%)
When we have a look at the EM 13c console for the host:
On the system the free -m command displays:
oracle@host:~/24437699/ [agent13c] free -m total used free shared buffers cached Mem: 48275 44762 3512 0 205 37483 -/+ buffers/cache: 7073 41201 Swap: 8189 2397 5791
Em 13c does not take into account the buffer / cached component.
In fact the memory calculation has changed from EM 12.1.0.5 and EM 13.1.0.0. According to Metalink Note 2144976.1:
“While the total Memory available in the host target is displayed correctly after applying the latest PSU # 23030165 (Agent-Side 13.1.0.0.160429), the formula used for Memory Utilization is (100.0 * (realMem-freeMem) / realMem) and does not consider Buffers / Cached component for the calculation.”
To solve the problem we have to patch the OMS and the different agents:
For the oms: use the patch 23134365
For the agents : use the patch 24437699
Watch out, when you want to apply the 23134365 patch for oms, we have to install the latest version of omspatcher. We download Patch 19999993 of Release 13.1.0.0.0 from MOS.
We backup the OMSPatcher directory in the $ORACLE_HOME oms13c environment:
oracle:OMS_HOME:/ [oms13c] mv OMSPatcher/ OMSPatcher_save
then we copy and unzip the p19999993_131000_Generic.zip from the $ORACLE_HOME directory:
oracle:$OMS_HOME/ [oms13c] unzip p19999993_131000_Generic.zip Archive: p19999993_131000_Generic.zip creating: OMSPatcher/ creating: OMSPatcher/oms/ inflating: OMSPatcher/oms/generateMultiOMSPatchingScripts.pl creating: OMSPatcher/jlib/ inflating: OMSPatcher/jlib/oracle.omspatcher.classpath.jar inflating: OMSPatcher/jlib/oracle.omspatcher.classpath.unix.jar inflating: OMSPatcher/jlib/omspatcher.jar inflating: OMSPatcher/jlib/oracle.omspatcher.classpath.windows.jar creating: OMSPatcher/scripts/ creating: OMSPatcher/scripts/oms/ creating: OMSPatcher/scripts/oms/oms_child_scripts/ inflating: OMSPatcher/scripts/oms/oms_child_scripts/omspatcher_wls.bat inflating: OMSPatcher/scripts/oms/oms_child_scripts/omspatcher_jvm_discovery inflating: OMSPatcher/scripts/oms/oms_child_scripts/omspatcher_jvm_discovery.bat inflating: OMSPatcher/scripts/oms/oms_child_scripts/omspatcher_wls inflating: OMSPatcher/scripts/oms/omspatcher inflating: OMSPatcher/scripts/oms/omspatcher.bat inflating: OMSPatcher/omspatcher creating: OMSPatcher/wlskeys/ inflating: OMSPatcher/wlskeys/createkeys.cmd inflating: OMSPatcher/wlskeys/createkeys.sh inflating: OMSPatcher/omspatcher.bat inflating: readme.txt inflating: PatchSearch.xml
We check the OMSPatcher version:
oracle:/ [oms13c] ./omspatcher version OMSPatcher Version: 13.6.0.0.1 OPlan Version: 12.1.0.2.2 OsysModel build: Wed Oct 14 06:21:23 PDT 2015 OMSPatcher succeeded.
We download from Metalink the p23134265_131000_Generic-zip file, and we run:
oracle@host:/home/oracle/23134365/ [oms13c] omspatcher apply -analyze OMSPatcher Automation Tool Copyright (c) 2015, Oracle Corporation. All rights reserved. OMSPatcher version : 13.6.0.0.1 OUI version : 13.6.0.0.0 Running from : /u00/app/oracle/product/13.1.0.0/middleware Log file location : /u00/app/oracle/product/13.1.0.0/middleware/ cfgtoollogs/omspatcher/opatch2016-09-26_11-06-30AM_1.log OMSPatcher log file: /u00/app/oracle/product/13.1.0.0/middleware/ cfgtoollogs/omspatcher/23134365/omspatcher_2016-09-26_11-06-34AM_analyze.log Please enter OMS weblogic admin server URL(t3s://hostname:7102):> Please enter OMS weblogic admin server username(weblogic):> Please enter OMS weblogic admin server password:> Configuration Validation: Success Running apply prerequisite checks for sub-patch(es) "23134365" and Oracle Home "/u00/app/oracle/product/13.1.0.0/middleware"... Sub-patch(es) "23134365" are successfully analyzed for Oracle Home "/u00/app/oracle/product/13.1.0.0/middleware" Complete Summary ================ OMSPatcher succeeded.
We stop the oms and we run:
oracle@hostname:/home/oracle/23134365/ [oms13c] omspatcher apply OMSPatcher Automation Tool Copyright (c) 2015, Oracle Corporation. All rights reserved. OMSPatcher version : 13.6.0.0.1 OUI version : 13.6.0.0.0 Running from : /u00/app/oracle/product/13.1.0.0/middleware Please enter OMS weblogic admin server URL(t3s://hostname:7102):> Please enter OMS weblogic admin server username(weblogic):> Please enter OMS weblogic admin server password:> Configuration Validation: Success … OMSPatcher succeeded.
We finally restart the OMS:
oracle@hostname:/home/oracle/ [oms13c] emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 1 Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved. Starting Oracle Management Server... WebTier Successfully Started Oracle Management Server Successfully Started Oracle Management Server is Up JVMD Engine is Up Starting BI Publisher Server ... BI Publisher Server Already Started BI Publisher Server is Up
Now we apply the patch to the agents:
After downloaded and unzipped the p24437699_131000_Generic.zip, we stop the management agent and we run:
oracle@hostname:/home/oracle/24437699/ [agent13c] opatch apply Oracle Interim Patch Installer version 13.6.0.0.0 Copyright (c) 2016, Oracle Corporation. All rights reserved. Oracle Home : /u00/app/oracle/product/13.1.0.0/agent/agent_13.1.0.0.0 Central Inventory : /u00/app/oraInventory OPatch version : 13.6.0.0.0 OUI version : 13.6.0.0.0 OPatch detects the Middleware Home as "/u00/app/oracle/product/13.1.0.0/agent" Verifying environment and performing prerequisite checks... OPatch continues with these patches: 24437699 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Backing up files... Applying interim patch '24437699' to OH '/u00/app/oracle/product/13.1.0.0/agent/agent_13.1.0.0.0' Patching component oracle.sysman.top.agent, 13.1.0.0.0... Patch 24437699 successfully applied. OPatch succeeded.
Finally we restart the agent with the emctl start agent command.
After the patches have been applied, the memory used displayed is correct:
And we do not receive critical alerts anymore :=)