Documentum story – Status of IndexAgent not visible in HTTPS (xPlore)

In one of our project, we were using CS 7.2 P02 (upgraded later with P05) in correlation with xPlore 1.5 P02. With these versions, we wanted to setup the IndexAgents in HTTPS to have a completely secured environment. We choose to follow the xPlore documentation and use the Groovy script that EMC is now providing to facilitate the setup in HTTPS of the IndexAgents & Primary DSearch.

The first thing to note is that the documentation is describing some parts of how to setup the SSL in xPlore but it is not the complete setup and some other parts are missing! You can use one of my previous blog to know how to setup an IndexAgent in HTTPS using the Groovy Script: click here. The second thing to note is that after setting our IndexAgents in HTTPS, we faced an issue where the status of the IndexAgents weren’t available anymore (it was working in HTTP)! After following the steps described in the other blogs, please take a look at the explications below in case you are having the same issue.

So after doing what is described in the link above, the status of the IndexAgents in Documentum Administrator was “Not responding” and we weren’t able to start/stop them. Same behavior via iapi, the status command returned a 200 code. Unlike the HTTP response codes, the IndexAgents codes are:

• 0 => Everything is running & working
• 100 => The JBoss instance hosting the IndexAgent is running BUT the IndexAgent isn’t started (E.g.: not in the status “Start in Normal Mode”)
• 200 => The JBoss instance hosting the IndexAgent isn’t running OR not responding

Now to check the status of the IndexAgent, you can do the following using iapi:

API> ?,c,select object_name, index_name from dm_ftindex_agent_config;
object_name                                      index_name
-----------------------------------------------  ------------------------------------------------
xplore_server_01_9200_IndexAgent                  DOCBASE_ftindex_01
(1 row affected)

API> apply,c,,FTINDEX_AGENT_ADMIN,NAME,S,DOCBASE_ftindex_01,AGENT_INSTANCE_NAME,S,xplore_server_01_9200_IndexAgent,ACTION,S,status
...
q0
API> next,c,q0
...
OK
API> dump,c,q0
...
USER ATTRIBUTES
 
  name                         [0]: xplore_server_01_9200_IndexAgent
  status                       [0]: 200

SYSTEM ATTRIBUTES


APPLICATION ATTRIBUTES
 

INTERNAL ATTRIBUTES
 

API> exit
Bye

You can see above a status 200… I can assure you that the JBoss instance and IndexAgent are up, running & working properly (we had to start it using the GUI since it was the only way to check the status)! Then why is a status code 200 returned? Well that’s the purpose of this blog!

Like I said before, we choose to setup the SSL for the IndexAgents using the Groovy script and that’s the reason why we faced this issue in the first place… The Groovy script is setup to automatically update the file “standalone.xml” to add some configuration for the SSL. This is what the script will add:

[xplore@xplore_server_01 ~]$ grep -A4 "connector name="https"" /app/xPlore/jboss7.1.1/server/DctmServer_Indexagent_DOCBASE/configuration/standalone.xml
    <connector name="https" protocol="HTTP/1.1" scheme="https" socket-binding="https" secure="true">
        <ssl name="https" password="***********"
            certificate-key-file="/app/xPlore/dsearch/admin/./../../jboss7.1.1/server/DctmServer_Indexagent_DOCBASE/configuration/my.keystore"
            cipher-suite="TLS_RSA_WITH_AES_128_CBC_SHA"/>
    </connector>

Now what is wrong with this configuration? It actually looks quite good… We were working on this specific issue for a few months with EMC without any special findings (no errors in logs, nothing in the DFC/RPC/HTTP_POST traces, aso…) when we noted that the Groovy script only added a “normal” Cipher to the standalone.xml file but our Content Servers were setup at that time using the default SSL mode of Documentum which is commonly named “Anonymous SSL”. As shown above, the SSL Cipher that the Groovy script added is “TLS_RSA_WITH_AES_128_CBC_SHA”. The problem is that the JBoss instance hosting the IndexAgent is restricting the Ciphers available but the docbase on the other side isn’t able to use this Cipher because it is using the “Anonymous SSL”. Therefore there is mismatch in the Ciphers available and the communication can’t be completed.

So what is the solution to that? Well we simply need to extend the Cipher list to find a Cipher that is available for both the docbase and the IndexAgent. To solve this issue, you just have to replace the “cipher-suite” as follow:

[xplore@xplore_server_01 ~]$ grep -A4 "connector name="https"" /app/xPlore/jboss7.1.1/server/DctmServer_Indexagent_DOCBASE/configuration/standalone.xml
    <connector name="https" protocol="HTTP/1.1" scheme="https" socket-binding="https" secure="true">
        <ssl name="https" password="***********"
            certificate-key-file="/app/xPlore/dsearch/admin/./../../jboss7.1.1/server/DctmServer_Indexagent_DOCBASE/configuration/my.keystore"
            cipher-suite="TLS_DH_anon_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA"/>
    </connector>

With this setup, you just have to restart the JBoss instance hosting the IndexAgent, reinitialize the Content Server and after doing that, the status will be available again:

API> apply,c,,FTINDEX_AGENT_ADMIN,NAME,S,DOCBASE_ftindex_01,AGENT_INSTANCE_NAME,S,xplore_server_01_9200_IndexAgent,ACTION,S,status
...
q0
API> next,c,q0
...
OK
API> dump,c,q0
...
USER ATTRIBUTES

  name                         [0]: xplore_server_01_9200_IndexAgent
  status                       [0]: 100

SYSTEM ATTRIBUTES
 

APPLICATION ATTRIBUTES
 

INTERNAL ATTRIBUTES


API> apply,c,,FTINDEX_AGENT_ADMIN,NAME,S,DOCBASE_ftindex_01,AGENT_INSTANCE_NAME,S,xplore_server_01_9200_IndexAgent,ACTION,S,start
...
q1
API> next,c,q1
...
OK
API> dump,c,q1
...
USER ATTRIBUTES

  name                         [0]: xplore_server_01_9200_IndexAgent
  status                       [0]: 0

SYSTEM ATTRIBUTES


APPLICATION ATTRIBUTES
 

INTERNAL ATTRIBUTES
 

API> exit
Bye

As you might know, the Cipher that we added contains “_anon_”. This means Anonymous (no authentication) and this also means that it is not the most secure of all Ciphers but this is necessary if you are using the default SSL mode in Documentum (without specifically setting up your own SSL Certificates). That’s why EMC recommends you to setup a non-Anonymous SSL mode but that’s a little bit more difficult to setup than just using the default one. Might be the subject of another blog 🙂