Beginning of this year, we were doing our first silent installations of the new Documentum stack. I already created a few blogs to talk about some issues with CS 7.3 and xPlore 1.6. This time, I will talk about xCP 2.3 and in particular the installation on a CS 7.3. The Patch of xCP as well as the patch for the CS 7.3 doesn’t matter since all versions are affected. Please just note that the first supported patch on a CS 7.3 is xCP 2.3 P03 so you shouldn’t be installing a previous patch on 7.3.

So, when installing an xCP 2.3 on a Content Server 7.3, you will get a pop-up in the installer with the following error message: “Installation of DARs failed”. You will only have an “OK” button on this pop-up which will close the installer. Ok so there is an issue with the installation of the DARs but what’s the issue exactly?

 

On the installation log file, we can see the following:

[dmadmin@content_server_01 ProcessEngine]$ cat logs/install.log
13:44:45,356  INFO [Thread-8] com.documentum.install.pe.installanywhere.actions.PEInitializeSharedLibrary - Done InitializeSharedLibrary ...
13:44:45,395  INFO [Thread-10] com.documentum.install.appserver.jboss.JbossApplicationServer - setApplicationServer sharedDfcLibDir is:$DOCUMENTUM_SHARED/dfc
13:44:45,396  INFO [Thread-10] com.documentum.install.appserver.jboss.JbossApplicationServer - getFileFromResource for templates/appserver.properties
13:44:45,532  WARN [Thread-10] com.documentum.install.pe.installanywhere.actions.DiWAPeInitialize - init-param tags found in Method Server webapp:

<init-param>
      <param-name>docbase_install_owner_name</param-name>
      <param-value>dmadmin</param-value>
</init-param>
<init-param>
      <param-name>docbase-GR_DOCBASE</param-name>
      <param-value>GR_DOCBASE</param-value>
</init-param>
<init-param>
      <param-name>docbase-DocBase1</param-name>
      <param-value>DocBase1</param-value>
</init-param>
<init-param>
      <param-name>docbase-DocBase2</param-name>
      <param-value>DocBase2</param-value>
</init-param>
13:44:58,771  INFO [AWT-EventQueue-0] com.documentum.install.pe.ui.panels.DiWPPELicenseAgreementPanel - UserSelection: "I accept the terms of the license agreement."
13:46:13,398  INFO [AWT-EventQueue-0] com.documentum.install.appserver.jboss.JbossApplicationServer - The batch file: $DOCUMENTUM_SHARED/temp/installer/wildfly/dctm_tmpcmd0.sh exist? false
13:46:13,399  INFO [AWT-EventQueue-0] com.documentum.install.appserver.jboss.JbossApplicationServer - The user home is : /home/dmadmin
13:46:13,405  INFO [AWT-EventQueue-0] com.documentum.install.appserver.jboss.JbossApplicationServer - Executing temporary batch file: $DOCUMENTUM_SHARED/temp/installer/wildfly/dctm_tmpcmd0.sh for running: $DOCUMENTUM_SHARED/java64/1.8.0_77/bin/java -cp $DOCUMENTUM_SHARED/wildfly9.0.1/modules/system/layers/base/emc/documentum/security/main/dfc.jar:$DOCUMENTUM_SHARED/wildfly9.0.1/modules/system/layers/base/emc/documentum/security/main/aspectjrt.jar:$DOCUMENTUM_SHARED/wildfly9.0.1/modules/system/layers/base/emc/documentum/security/main/DctmUtils.jar com.documentum.install.appserver.utils.DctmAppServerAuthenticationString $DOCUMENTUM_SHARED/wildfly9.0.1/server/DctmServer_MethodServer jboss
13:46:42,320  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeInstallActions - starting DctmActions
13:46:42,724  INFO [installer] com.documentum.install.appserver.jboss.JbossApplicationServer - user name = admin
13:46:42,724  INFO [installer] com.documentum.install.appserver.jboss.JbossApplicationServer - Server DctmServer_MethodServer already exists!
13:46:42,725  INFO [installer] com.documentum.install.appserver.jboss.JbossApplicationServer - Deploying to Group MethodServer... bpm (bpm.ear): does not exist!
13:46:42,725  INFO [installer] com.documentum.install.appserver.jboss.JbossApplicationServer - resolving $DOCUMENTUM_SHARED/wildfly9.0.1/server/DctmServer_MethodServer/deployments/bpm.ear/APP-INF/classes/dfc.properties
13:46:42,725  INFO [installer] com.documentum.install.appserver.jboss.JbossApplicationServer - resolving $DOCUMENTUM_SHARED/wildfly9.0.1/server/DctmServer_MethodServer/deployments/bpm.ear/APP-INF/classes/log4j.properties
13:46:42,725  INFO [installer] com.documentum.install.appserver.jboss.JbossApplicationServer - resolving $DOCUMENTUM_SHARED/wildfly9.0.1/server/DctmServer_MethodServer/deployments/bpm.ear/bpm.war/WEB-INF/web.xml
13:46:42,727  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeInstallActions - Finished DctmActions.
13:46:44,885  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - Start to deploy dars for docbase: DocBase2
13:52:20,931  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - End to deploy dars for repository: DocBase2
13:52:20,932  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - Start to deploy dars for docbase: DocBase1
13:57:59,510  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - End to deploy dars for repository: DocBase1
13:57:59,511  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - Start to deploy dars for docbase: GR_DOCBASE
14:04:03,231  INFO [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - End to deploy dars for repository: GR_DOCBASE
14:04:03,268 ERROR [installer] com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars - Installation of DARs failed
com.documentum.install.shared.common.error.DiException: 3 DAR(s) failed to install.
        at com.documentum.install.shared.common.services.dar.DiDocAppFailureList.report(DiDocAppFailureList.java:39)
        at com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars.deployDars(DiPAPeProcessDars.java:123)
        at com.documentum.install.pe.installanywhere.actions.DiPAPeProcessDars.setup(DiPAPeProcessDars.java:71)
        at com.documentum.install.shared.installanywhere.actions.InstallWizardAction.install(InstallWizardAction.java:75)
        at com.zerog.ia.installer.actions.CustomAction.installSelf(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.GhostDirectory.install(Unknown Source)
        at com.zerog.ia.installer.InstallablePiece.install(Unknown Source)
        at com.zerog.ia.installer.Installer.install(Unknown Source)
        at com.zerog.ia.installer.actions.InstallProgressAction.ae(Unknown Source)
        at com.zerog.ia.installer.actions.ProgressPanelAction$1.run(Unknown Source)
14:04:03,269  INFO [installer]  - The INSTALLER_UI value is SWING
14:04:03,269  INFO [installer]  - The env PATH value is: /usr/xpg4/bin:$DOCUMENTUM_SHARED/java64/JAVA_LINK/bin:$DOCUMENTUM/product/7.3/bin:$DOCUMENTUM/dba:$ORACLE_HOME/bin:$DOCUMENTUM_SHARED/java64/JAVA_LINK/bin:$DOCUMENTUM/product/7.3/bin:$DOCUMENTUM/dba:$ORACLE_HOME/bin:$DOCUMENTUM/product/7.3/bin:$ORACLE_HOME/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/dmadmin/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin
[dmadmin@content_server_01 ProcessEngine]$

 

It is mentioned that three DARs failed to be installed but since there are three docbases here, that’s actually one DAR per docbase. The only interesting information we can find from the install log file is that some DARs were installed properly so it’s not a generic issue but more likely an issue with one specific DAR. The next step is therefore to check the log file of the DAR installation:

[dmadmin@content_server_01 ProcessEngine]$ grep -i ERROR logs/dar_logs/GR_DOCBASE/peDars.log | grep -v "^[INFO].*ERROR"
[INFO]  dmbasic.exe output : dmbasic: Error 35 in line 585: Sub or Function not defined
[ERROR]  Unable to install dar file $DOCUMENTUM/product/7.3/install/DARsInternal/BPM.dar
com.emc.ide.installer.InstallException: Error handling controllable object Status = New; IsInstalled = true; com.emc.ide.artifact.bpm.model.bpm.impl.ActivityImpl@5e020dd1 (objectTypeName: null) (objectName: DB Inbound - Initiate, title: , subject: , authors: [], keywords: [], applicationType: , isHidden: false, compoundArchitecture: , componentLabel: [], resolutionLabel: , contentType: xml, versionLabel: [1.0, CURRENT], specialApp: DB-IN-IN.GIF, languageCode: , creatorName: null, archive: false, category: , controllingApp: , effectiveDate: [], effectiveFlag: [], effectiveLabel: [], expirationDate: [], extendedProperties: [], fullText: true, isSigned: false, isTemplate: false, lastReviewDate: null, linkResolved: false, publishFormats: [], retentionDate: null, status: , rootObject: true) (isPrivate: false, definitionState: installed, triggerThreshold: 0, triggerEvent: , execType: manual, execSubType: inbound_initiate, execMethodName: null, preTimer: 0, preTimerCalendarFlag: notusebusinesscal, preTimerRepeatLast: 0, postTimer: 0, postTimerCalendarFlag: notusebusinesscal, postTimerRepeatLast: 0, repeatableInvoke: true, execSaveResults: false, execTimeOut: 0, execErrHandling: stopAfterFailure, signOffRequired: false, resolveType: normal, resolvePkgName: , controlFlag: taskAssignedtoSupervisor, taskName: null, taskSubject: , performerType: user, performerFlag: noDeligationOrExtention, transitionMaxOutputCnt: 0, transitionEvalCnt: trigAllSelOutputLinks, transitionFlag: trigAllSelOutputLinks, transitionType: prescribed, execRetryMax: 0, execRetryInterval: 0, groupFlag: 0, template: true, artifactVersion: D65SP1);  Object ID = 4c0f123450002b1e;
Caused by: DfException:: THREAD: main; MSG: Error while making activity uneditable: com.emc.ide.artifactmanager.model.artifact.impl.ArtifactImpl@4bbc02ef (urn: urnd:com.emc.ide.artifact.bpm.activity/DB+Inbound+-+Initiate?location=%2FTemp%2FIntegration&name=DB+Inbound+-+Initiate, locale: null, repoLocation: null, categoryId: com.emc.ide.artifact.bpm.activity, implicitlyCreated: false, modifiedByUser: true); ERRORCODE: ff; NEXT: null
Caused by: DfException:: THREAD: main; MSG: [DM_WORKFLOW_E_NAME_NOT_EXIST]error:  "The dm_user object by the name 'dm_bps_inbound_user' specified in attribute performer_name does not exist."; ERRORCODE: 100; NEXT: null
[ERROR]  Failed to install DAR
Caused by: com.emc.ide.installer.InstallException: Error handling controllable object Status = New; IsInstalled = true; com.emc.ide.artifact.bpm.model.bpm.impl.ActivityImpl@5e020dd1 (objectTypeName: null) (objectName: DB Inbound - Initiate, title: , subject: , authors: [], keywords: [], applicationType: , isHidden: false, compoundArchitecture: , componentLabel: [], resolutionLabel: , contentType: xml, versionLabel: [1.0, CURRENT], specialApp: DB-IN-IN.GIF, languageCode: , creatorName: null, archive: false, category: , controllingApp: , effectiveDate: [], effectiveFlag: [], effectiveLabel: [], expirationDate: [], extendedProperties: [], fullText: true, isSigned: false, isTemplate: false, lastReviewDate: null, linkResolved: false, publishFormats: [], retentionDate: null, status: , rootObject: true) (isPrivate: false, definitionState: installed, triggerThreshold: 0, triggerEvent: , execType: manual, execSubType: inbound_initiate, execMethodName: null, preTimer: 0, preTimerCalendarFlag: notusebusinesscal, preTimerRepeatLast: 0, postTimer: 0, postTimerCalendarFlag: notusebusinesscal, postTimerRepeatLast: 0, repeatableInvoke: true, execSaveResults: false, execTimeOut: 0, execErrHandling: stopAfterFailure, signOffRequired: false, resolveType: normal, resolvePkgName: , controlFlag: taskAssignedtoSupervisor, taskName: null, taskSubject: , performerType: user, performerFlag: noDeligationOrExtention, transitionMaxOutputCnt: 0, transitionEvalCnt: trigAllSelOutputLinks, transitionFlag: trigAllSelOutputLinks, transitionType: prescribed, execRetryMax: 0, execRetryInterval: 0, groupFlag: 0, template: true, artifactVersion: D65SP1);  Object ID = 4c0f123450002b1e;
Caused by: DfException:: THREAD: main; MSG: Error while making activity uneditable: com.emc.ide.artifactmanager.model.artifact.impl.ArtifactImpl@4bbc02ef (urn: urnd:com.emc.ide.artifact.bpm.activity/DB+Inbound+-+Initiate?location=%2FTemp%2FIntegration&name=DB+Inbound+-+Initiate, locale: null, repoLocation: null, categoryId: com.emc.ide.artifact.bpm.activity, implicitlyCreated: false, modifiedByUser: true); ERRORCODE: ff; NEXT: null
Caused by: DfException:: THREAD: main; MSG: [DM_WORKFLOW_E_NAME_NOT_EXIST]error:  "The dm_user object by the name 'dm_bps_inbound_user' specified in attribute performer_name does not exist."; ERRORCODE: 100; NEXT: null
[dmadmin@content_server_01 ProcessEngine]$

 

With the above, we know that the only failed DAR is the BPM.dar and it looks like we have the reason for this: the DAR needs a user named “dm_bps_inbound_user” to proceed with the installation but couldn’t find it and therefore the installation failed. But actually that’s not the root cause, it’s only a consequence. The real reason why the DAR installation failed is displayed in the first line above.

[INFO]  dmbasic.exe output : dmbasic: Error 35 in line 585: Sub or Function not defined

 

For some reason, a function couldn’t be executed because not defined properly. This function is the one that is supposed to create the “dm_bps_inbound_user” user but with a CS 7.3 this function cannot be executed properly. As a result, the user isn’t created and then the DAR installation fail. For more information, you can refer to the BPM-11223.

 

This issue will – according to EMC – not be fixed in any patch of the xCP 2.3, even if this issue has been spotted quickly after the release of the xCP 2.3. Therefore, if you want to avoid this issue, you will have to wait several months for the xCP 2.4 to be released (not really realistic ;)) or you will need to create this user manually before installing the xCP 2.3 on a CS 7.3. You don’t need special permissions for this user and you don’t need to know its password so it’s rather simple to create it for all installed docbases in a few simple commands:

[dmadmin@content_server_01 ProcessEngine]$ echo "?,c,select r_object_id, user_name, user_login_name from dm_user where user_login_name like 'dm_bps%';" > create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "create,c,dm_user" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "set,c,l,user_name" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "dm_bps_inbound_user" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "set,c,l,user_login_name" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "dm_bps_inbound_user" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "save,c,l" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$ echo "?,c,select r_object_id, user_name, user_login_name from dm_user where user_login_name like 'dm_bps%';" >> create_user.api
[dmadmin@content_server_01 ProcessEngine]$
[dmadmin@content_server_01 ProcessEngine]$ cat create_user.api
?,c,select r_object_id, user_name, user_login_name from dm_user where user_login_name like 'dm_bps%';
create,c,dm_user
set,c,l,user_name
dm_bps_inbound_user
set,c,l,user_login_name
dm_bps_inbound_user
save,c,l
?,c,select r_object_id, user_name, user_login_name from dm_user where user_login_name like 'dm_bps%';
[dmadmin@content_server_01 ProcessEngine]$
[dmadmin@content_server_01 ProcessEngine]$
[dmadmin@content_server_01 ProcessEngine]$ sep="***********************"
[dmadmin@content_server_01 ProcessEngine]$ for docbase in `cd $DOCUMENTUM/dba/config; ls`;do echo;echo "$sep";echo "Create User: ${docbase}";echo "$sep";iapi ${docbase} -Udmadmin -Pxxx -Rcreate_user.api;done

***********************
Create User: GR_DOCBASE
***********************


        EMC Documentum iapi - Interactive API interface
        (c) Copyright EMC Corp., 1992 - 2016
        All rights reserved.
        Client Library Release 7.3.0000.0205


Connecting to Server using docbase GR_DOCBASE
[DM_SESSION_I_SESSION_START]info:  "Session 010f12345001c734 started for user dmadmin."


Connected to Documentum Server running Release 7.3.0000.0214  Linux64.Oracle
Session id is s0
API> r_object_id     user_name             user_login_name                                                                                                                                                             
-------------------  --------------------- ---------------------

(0 row affected)

API> ...
110f12345000093c
API> SET> ...
OK
API> SET> ...
OK
API> ...
OK
API> r_object_id     user_name             user_login_name                                                                                                                                                             
-------------------  --------------------- ---------------------
110f12345000093c     dm_bps_inbound_user   dm_bps_inbound_user
(1 row affected)

API> Bye

***********************
Create User: DocBase1
***********************


        EMC Documentum iapi - Interactive API interface
        (c) Copyright EMC Corp., 1992 - 2016
        All rights reserved.
        Client Library Release 7.3.0000.0205


Connecting to Server using docbase DocBase1
[DM_SESSION_I_SESSION_START]info:  "Session 010f234560052632 started for user dmadmin."


Connected to Documentum Server running Release 7.3.0000.0214  Linux64.Oracle
Session id is s0
API> r_object_id     user_name             user_login_name                                                                                                                                                             
-------------------  --------------------- ---------------------

(0 row affected)

API> ...
110f234560001532
API> SET> ...
OK
API> SET> ...
OK
API> ...
OK
API> r_object_id     user_name             user_login_name                                                                                                                                                             
-------------------  --------------------- ---------------------
110f234560001532     dm_bps_inbound_user   dm_bps_inbound_user                                                                                                                                                            
(1 row affected)

API> Bye

***********************
Create User: DocBase2
***********************


        EMC Documentum iapi - Interactive API interface
        (c) Copyright EMC Corp., 1992 - 2016
        All rights reserved.
        Client Library Release 7.3.0000.0205


Connecting to Server using docbase DocBase2
[DM_SESSION_I_SESSION_START]info:  "Session 010f345670052632 started for user dmadmin."


Connected to Documentum Server running Release 7.3.0000.0214  Linux64.Oracle
Session id is s0
API> r_object_id     user_name             user_login_name                                                                                                                                                             
-------------------  --------------------- ---------------------

(0 row affected)

API> ...
110f345670001532
API> SET> ...
OK
API> SET> ...
OK
API> ...
OK
API> r_object_id     user_name             user_login_name                                                                                                                                                             
-------------------  --------------------- ---------------------
110f345670001532     dm_bps_inbound_user   dm_bps_inbound_user                                                                                                                                                            
(1 row affected)

API> Bye
[dmadmin@content_server_01 ProcessEngine]$
[dmadmin@content_server_01 ProcessEngine]$ rm create_user.api
[dmadmin@content_server_01 ProcessEngine]$

 

The users have been created properly in all docbases so just restart the xCP installer and this time the BPM.dar installation will succeed.