Infrastructure at your Service

Morgan Patou

Documentum – Not able to install IndexAgent with xPlore 1.6

In the last few months, we already did some tests with the Content Server 7.3, D2 4.7 and xPlore 1.6. As often, we found some funny/interesting behaviors and we faced some issues. In this blog I will talk about one issue I faced when I tried to install a new IndexAgent with xPlore 1.6. The Content Server used was a CS 7.3 (no patch available when I did this installation). This issue was present in both Silent and non-Silent installations.


So what was the issue? While trying to install a first IndexAgent, it looked like the installer wasn’t able to contact the Content Server. The GUI installer was stuck on the “Connection Broker Information”. After filling the information and clicking on “Next”, the Full Text Installer was stuck and nothing happened anymore.


Basically on this step of the installer, it is trying to connect to the docbroker using the information provided (docbroker hostname and port). Of course the docbroker and the docbase were running and responding properly.


To verify what was happening, I just checked the xPlore processes:

[[email protected]_text_server_01 ~]$ ps -ef | grep xplore
xplore    2423     1  0 08:04 ?        00:00:00 /bin/sh ./
xplore    2425  2423  0 08:04 ?        00:00:00 /bin/sh $XPLORE_HOME/wildfly9.0.1/bin/
xplore    2572  2425  5 08:04 ?        00:03:01 $XPLORE_HOME/java64/1.8.0_77/bin/java -D[Standalone] -server -XX:+UseCompressedOops -server -XX:+UseCompressedOops -XX:+UseCodeCacheFlushing -XX:ReservedCodeCacheSize=196m -Xms4096m -Xmx4096m -XX:MaxMetaspaceSize=512m -Xss1024k -XX:+UseCompressedOops -XX:+DisableExplicitGC -XX:-ReduceInitialCardMarks -Djava.awt.headless=true -Djboss.server.base.dir=$XPLORE_HOME/wildfly9.0.1/server/DctmServer_PrimaryDsearch -Dorg.jboss.boot.log.file=$XPLORE_HOME/wildfly9.0.1/server/DctmServer_PrimaryDsearch/log/server.log -Dlogging.configuration=file:$XPLORE_HOME/wildfly9.0.1/server/DctmServer_PrimaryDsearch/configuration/ -jar $XPLORE_HOME/wildfly9.0.1/jboss-modules.jar -mp $XPLORE_HOME/wildfly9.0.1/modules -Djboss.home.dir=$XPLORE_HOME/wildfly9.0.1 -Djboss.server.base.dir=$XPLORE_HOME/wildfly9.0.1/server/DctmServer_PrimaryDsearch
xplore    3977  2572  0 08:06 ?        00:00:00 $XPLORE_HOME/dsearch/cps/cps_daemon/bin/CPSDaemon $XPLORE_HOME/dsearch/cps/cps_daemon/PrimaryDsearch_local_configuration.xml Daemon0 9322 NORMAL
xplore    4069  2572  0 08:06 ?        00:00:00 $XPLORE_HOME/dsearch/cps/cps_daemon/bin/CPSDaemon $XPLORE_HOME/dsearch/cps/cps_daemon/PrimaryDsearch_local_configuration.xml QDaemon0 9323 QUERY
xplore    9591  9568  0 08:45 pts/3    00:00:00 /bin/sh ./
xplore    9592  9591  0 08:45 pts/3    00:00:08 $XPLORE_HOME/java64/1.8.0_77/jre/bin/java -Xmx262114000 -Xms262114000 com.zerog.lax.LAX /tmp/install.dir.9592/temp.lax /tmp/ "-f" ""
xplore   11099  9592  0 08:51 pts/3    00:00:02 $XPLORE_HOME/java64/1.8.0_77/jre/bin/java -cp /tmp/342186.tmp/testDocbrokerUtil.jar:$XPLORE_HOME/dfc/dfc.jar com.documentum.install.shared.util.TestDocbrokerUtil content_server_01 1489
xplore   11928  9515  0 08:59 pts/1    00:00:00 ps -ef
xplore   11929  9515  0 08:59 pts/1    00:00:00 grep xplore


As you can see above, the PrimaryDsearch is up & running so that’s good but there are two additional interesting processes:

xplore    9592  9591  0 08:45 pts/3    00:00:08 $XPLORE_HOME/java64/1.8.0_77/jre/bin/java -Xmx262114000 -Xms262114000 com.zerog.lax.LAX /tmp/install.dir.9592/temp.lax /tmp/ "-f" ""
xplore   11099  9592  0 08:51 pts/3    00:00:02 $XPLORE_HOME/java64/1.8.0_77/jre/bin/java -cp /tmp/342186.tmp/testDocbrokerUtil.jar:$XPLORE_HOME/dfc/dfc.jar com.documentum.install.shared.util.TestDocbrokerUtil content_server_01 1489


The most interesting one is the second line: this is the command that the installer is starting to check that the docbroker is responding properly and this process is stuck. To do some testing, I just put the utilities and the needed elements to a test folder so I would be able to execute this command myself whenever I wanted. Then I killed the installer since it was stuck and I tried myself to contact the docbroker:

[[email protected]_text_server_01 ~]$ time $XPLORE_HOME/java64/1.8.0_77/jre/bin/java -cp /tmp/my.tmp/testDocbrokerUtil.jar:$XPLORE_HOME/dfc/dfc.jar com.documentum.install.shared.util.TestDocbrokerUtil content_server_01 1489
real    7m43.669s
user    0m1.618s
sys     0m0.153s


As you can see above, after 7min and 43 seconds, I just killed the process (^C) because it still didn’t return anything. Just to be sure, I checked our monitoring tool and it was able to connect to the docbase, remotely, without issue so the issue here was link somehow to the Full Text Server.


To go deeper, the only thing I could do was to perform some thread dumps of the java process to try to see where the bottleneck was. After doing that, it actually became quite clear that the server had an issue with the entropy generation because the threads were holding and waiting for the rsa libraries to get a random seed from the system. If you are running Documentum (or any other thing) on a Virtual Machine you will most probably want to change the default random generation because the entropy isn’t sufficient usually on VMs. We are always doing that for all our customer’s environments but since this was just a sandbox server to play with the newly released xPlore 1.6, this wasn’t setup at that time. With xPlore 1.5, this wouldn’t have caused any issue but it looks like xPlore 1.6 is much more demanding on the random seed generations. This has most probably something to do with some security improvements done in xPlore 1.6.


So to solve this issue, you just need to increase the entropy of your server. This can be done as follow (as root):

  1. Install the rng tools package
  2. Update the options in the /etc/sysconfig/rngd file
  3. Setup the service to start automatically at OS boot


You can basically use the following commands for that (I’m using a RHEL so it’s yum based):

[[email protected]_text_server_01 ~]# yum -y install rng-tools.x86_64
Loaded plugins: product-id, search-disabled-repos, security, subscription-manager
Setting up Install Process
Resolving Dependencies
--> Running transaction check
Transaction Test Succeeded
Running Transaction
  Installing : rng-tools-5-2.el6_7.x86_64                                                                                                                                                                                     1/1
  Verifying  : rng-tools-5-2.el6_7.x86_64                                                                                                                                                                                     1/1

  rng-tools.x86_64 0:5-2.el6_7

[[email protected]_text_server_01 ~]# rpm -qf /etc/sysconfig/rngd
[[email protected]_text_server_01 ~]#
[[email protected]_text_server_01 ~]# sed -i 's,EXTRAOPTIONS=.*,EXTRAOPTIONS=\"-r /dev/urandom -o /dev/random -t 0.1\",' /etc/sysconfig/rngd
[[email protected]_text_server_01 ~]# cat /etc/sysconfig/rngd
# Add extra options here
EXTRAOPTIONS="-r /dev/urandom -o /dev/random -t 0.1"
[[email protected]_text_server_01 ~]#
[[email protected]_text_server_01 ~]# chkconfig --level 345 rngd on
[[email protected]_text_server_01 ~]# chkconfig --list | grep rngd
rngd            0:off   1:off   2:off   3:on    4:on    5:on    6:off
[[email protected]_text_server_01 ~]#
[[email protected]_text_server_01 ~]# service rngd start
Starting rngd:                                             [  OK  ]
[[email protected]_text_server_01 ~]#


When this is done, you need to force xPlore to use this new random generation mechanism. For that, just put in the profile of the xplore user one environment variable (as xplore):

[[email protected]_text_server_01 ~]$ echo 'export DEVRANDOM=/dev/urandom' >> ~/.bash_profile


When this has been done, you can just re-execute the java command and in something like 1 second the command should return a result like that:

[[email protected]_text_server_01 ~]$ $XPLORE_HOME/java64/1.8.0_77/jre/bin/java -cp /tmp/my.tmp/testDocbrokerUtil.jar:$XPLORE_HOME/dfc/dfc.jar com.documentum.install.shared.util.TestDocbrokerUtil content_server_01 1489
0 [main] ERROR com.documentum.fc.common.impl.logging.LoggingConfigurator  - Problem locating log4j configuration
1 [main] WARN com.documentum.fc.common.impl.logging.LoggingConfigurator  - Using default log4j configuration
[[email protected]_text_server_01 ~]$


These two messages aren’t important, they are just printed because there is no log4j properties file in the command line so it is using the default one. The only important thing here is that the command returns something so it is working! To confirm that, you can just run the installer once more and this time the screen shouldn’t be stuck on the Connection Broker Information.



  • Thanks, Morgan, coincidently “rngd on” was the solution to a random problem I had on out Oracle 12cR2 workshop infrastructure. I’ll blog about that soon 🙂

  • Rajesh says:

    what are major challenges between Documentum xplore 16.7 and 1.6?
    Differences between Documentum Administrator 16.7 and 7.3?

    • Morgan Patou says:

      Hi Rajesh,

      Changes and known issues are documented in the Release Note of each product so you can find the response to your questions there. If I had one thing to say, it would be regarding the DFC version. A lot of changes happened on the DFC layer starting from 16.7 so it is usually not compatible at all with previous versions (therefore, only 16.7 can communicate with 16.7 usually).


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Morgan Patou
Morgan Patou

Technology Leader ECM & Senior Consultant