Documentum – How to really configure the D2 JMS logs starting with JBoss/WildFly

If you are working with Documentum for quite some time, you are probably familiar with the logback.xml file that can be used to configure the D2 logs. In this blog, I will be talking only about the Content Server side of this configuration. As you probably know, Documentum upgraded the JMS to use JBoss 7.1.1 “recently” (several years already) and even WildFly9.0.1 with the CS 7.3+. On this blog, I will only use “JBoss” but it refers to both JBoss and WildFly versions.  With these recent versions, the logback.xml file stopped working on linux environments (I’m not sure about Windows, I only work on linux). Therefore you will face an issue: the D2 JMS logs cannot really be configured properly, by default. Of course you will still be able to configure the JBoss and JMS logs properly because that is done through the logging.properties file (for the boot.log), through the standalone.xml file (for the server.log) and through all log4j.properties files for each JMS Applications (ServerApps, ACS, BPM, aso…). But if you are using D2, then all the D2 JMS logs (previously stored on D2-JMS.log) will also be added to the server.log as well a console output.

Unfortunately for us, the D2 JMS logs are using DEBUG by default for everything so it might represent some big files at the end of the day as soon as you start working more than XXX concurrent users. Worse than that, the D2 JMS logs, which are in DEBUG, are considered as INFO from the JBoss point of view and therefore, if you are using JBoss with INFO log level, it will print all the DEBUG information from the D2 JMS logs. Of course you could still set the JBoss level to WARN so it would remove all the DEBUG but in this case, you will also be missing the INFO from the JBoss as well as the D2 JMS sides which might include some pretty important information like for example the assurance that the D2.Lockbox can be read properly (no problems with the passwords and/or fingerprint).

So what to do about it? Well there is a JVM parameter that can actually be used to force the JBoss Server to read and use a specific logback.xml file. For that, simply update the startMethodServer.sh script as done below. I will use the logback.xml file that is present by default right under ServerApps.ear and that I will customize to get the best out of it.

First of all, I’m updating the content to add some things. Here is a template for this file:

[dmadmin@content_server_01 ~]$ cd $DOCUMENTUM_SHARED/wildfly9.0.1/server/
[dmadmin@content_server_01 server]$ logback_file="$DOCUMENTUM_SHARED/wildfly9.0.1/server/DctmServer_MethodServer/deployments/ServerApps.ear/logback.xml"
[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ # Here I'm updating the content of the default file to add custom patterns, log level, console output, aso...
[dmadmin@content_server_01 server]$ vi ${logback_file}
[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ cat ${logback_file}
<?xml version="1.0" encoding="UTF-8"?>

<configuration scan="true" scanPeriod="60 seconds">

  <appender class="ch.qos.logback.core.rolling.RollingFileAppender" name="RootFileAppender">
    <file>/tmp/D2-JMS.log</file>
    <append>true</append>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>debug</level>
    </filter>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>/tmp/D2-JMS-%d{yyyy-MM-dd}.log.zip</fileNamePattern>
      <MaxHistory>5</MaxHistory>
    </rollingPolicy>
    <layout class="ch.qos.logback.classic.PatternLayout">
      <pattern>%d{"yyyy-MM-dd HH:mm:ss,SSS z"} [%-5p] (%t) - %-45(%C{44}) : %m%n</pattern>
    </layout>
  </appender>

  <appender class="ch.qos.logback.core.ConsoleAppender" name="RootConsoleAppender">
    <layout>
      <pattern>[%-5p] - %-45(%C{44}) : %m%n</pattern>
    </layout>
  </appender>

  <root>
    <level value="${logLevel:-info}"/>
    <appender-ref ref="RootFileAppender"/>
    <appender-ref ref="RootConsoleAppender"/>
  </root>

</configuration>
[dmadmin@content_server_01 server]$

Then once you have your template logback.xml file, you need to force JBoss to load and use it otherwise it will just be ignored. As mentioned above, here is the JVM parameter to be added:

[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ grep "JAVA_OPTS=" startMethodServer.sh
JAVA_OPTS="$USER_MEM_ARGS -Djboss.server.base.dir=$JBOSS_BASE_DIR -Dorg.apache.coyote.http11.Http11Protocol.SERVER=MethodServer"
[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ sed -i 's,^JAVA_OPTS="[^"]*,& -Dlogback.configurationFile=$JBOSS_BASE_DIR/deployments/ServerApps.ear/logback.xml,' startMethodServer.sh
[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ grep "JAVA_OPTS=" startMethodServer.sh
JAVA_OPTS="$USER_MEM_ARGS -Djboss.server.base.dir=$JBOSS_BASE_DIR -Dorg.apache.coyote.http11.Http11Protocol.SERVER=MethodServer -Dlogback.configurationFile=$JBOSS_BASE_DIR/deployments/ServerApps.ear/logback.xml"
[dmadmin@content_server_01 server]$

Once done, you can customize some values like the path and name of the log file, the number of files to keep, the log level you want to use, aso. Here are some commands to do just that:

[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ d2_log="$DOCUMENTUM_SHARED/wildfly9.0.1/server/DctmServer_MethodServer/logs/D2-JMS.log"
[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ # Commands to update some values on this pattern file using the ${d2_log} variable
[dmadmin@content_server_01 server]$ sed -i "s,<file>.*</file>,<file>${d2_log}</file>," ${logback_file}
[dmadmin@content_server_01 server]$ sed -i "s,<fileNamePattern>.*</fileNamePattern>,<fileNamePattern>${d2_log}-%d{yyyy-MM-dd}.zip</fileNamePattern>," ${logback_file}
[dmadmin@content_server_01 server]$ sed -i "s,<MaxHistory>.*</MaxHistory>,<MaxHistory>180</MaxHistory>," ${logback_file}
[dmadmin@content_server_01 server]$ sed -i "s,<level>.*</level>,<level>info</level>," ${logback_file}
[dmadmin@content_server_01 server]$

With the above done, you can just restart the JMS and afterwards, you will have a new file created D2-JMS.log at the location specified and with the log level specified.

[dmadmin@content_server_01 server]$ $JMS_HOME/server/stopMethodServer.sh
{"outcome" => "success"}
[dmadmin@content_server_01 server]$
[dmadmin@content_server_01 server]$
[dmadmin@content_server_01 server]$
[dmadmin@content_server_01 server]$ $JMS_HOME/server/startJMS.sh
Starting the JMS...
The JMS process has been started.
[dmadmin@content_server_01 server]$ sleep 30
[dmadmin@content_server_01 server]$ 
[dmadmin@content_server_01 server]$ cat ${d2_log}
2018-06-16 17:16:48,652 UTC [INFO ] (default task-6) - com.emc.d2.api.methods.D2Method               : D2Method Main method com.emc.d2.api.methods.D2SubscriptionMethod arguments: {-user_name=dmadmin, -method_trace_level=0, -dcbase_name=Repo1.Repo1, -class_name=com.emc.d2.api.methods.D2SubscriptionMethod, -job_id=080f123450001612}
2018-06-16 17:16:49,668 UTC [INFO ] (default task-6) - com.emc.d2.api.methods.D2Method               : ==== START ======================================================================
2018-06-16 17:16:49,670 UTC [INFO ] (default task-6) - com.emc.d2.api.methods.D2Method               : D2-API v4.7.0070 build 186
2018-06-16 17:16:49,674 UTC [INFO ] (default task-6) - com.emc.d2.api.methods.D2Method               : DFC version : 7.3.0040.0025
2018-06-16 17:16:49,675 UTC [INFO ] (default task-6) - com.emc.d2.api.methods.D2Method               : file.encoding : ANSI_X3.4-1968
2018-06-16 17:16:49,676 UTC [INFO ] (default task-6) - com.emc.d2.api.methods.D2Method               : Arguments : {-user_name=dmadmin, -method_trace_level=0, -docbase_name=Repo1.Repo1, -class_name=com.emc.d2.api.methods.2SubscriptionMethod, -job_id=080f123450001612}
[dmadmin@content_server_01 server]$

Here you have a working D2-JMS.log file with INFO only information and no DEBUG.

Some tips regarding the logback.xml configuration (I put an example of each in the template configuration file above):

• If you want to display the full date, time (with milliseconds) and timezone in the logs, you will need to add quotes like that: <pattern>%d{“yyyy-MM-dd HH:mm:ss,SSS z”} …</pattern>. This is simply because the comma (,) is used normally to separate the timeformat from the timezone you want to display the logs on (E.g.: %d{HH:mm:ss.SSS, UTC}) but it won’t display the timezone on the logs, in this case. So you if want the seconds to be separated from the milliseconds using a comma, you need to quote the whole string. If you want the current timezone to be displayed on the logs, you can usually do it using the “z” (with not too old Java versions)
• By default, you cannot use parentheses in the pattern to enclose parameters (like “%-5p”, “%t”, aso…). This is because parentheses are used to group parameters together to apply formatting to them as part of a group. If you really want to use parentheses on the output, then you have to escape them
• You can define the minimum length of a specific pattern parameter using the “%-X” where X is the number of characters. Using that, you can align the logs as you want (E.g.: “%-5p” for the log level in 5 chars => “DEBUG”, “INFO “, “WARN “, “ERROR”)
• You can also shorten a specific pattern parameter using {X} where X is again the number of characters you would want the output string to be reduced to. It is not an exact value but the logger will do its best to reduce the length to what you want.
• You can use different appenders to redirect the logs to different outputs. Usually you will want a file appender to store everything in a file but you can also add a console appender so it gets stored in your default console output (be it your shell, a nohup file or the server.log). If you do not want the console appender so it gets stored only on the D2-JMS.log file, you can just comment the line ‘<appender-ref ref=”RootConsoleAppender”/>’

You might be wondering why this JVM parameter is not added by default by the Documentum installer since it is a valid solution for this issue, right? Well, I would simply reply that it’s Documentum. 😉