Infrastructure at your Service

Morgan Patou

Documentum – DA fail to load if accessed after D2 because of ESAPI

In some specific cases, you might have faced an issue where Documentum Administrator is loading properly but then suddenly it’s not working anymore and all you did in between was accessing D2. I don’t remember seeing that before DA 16.x but maybe it’s just my memory that is playing me tricks! When this happen, you will see the usual pop-up on Documentum Administrator asking you to refresh/reload your screen but without any details of what’s wrong.

 

Fortunately, the logs of Documentum Administrator are more useful since it will print something like that (it’s an old log but the same still apply with the latest DA Patches as well):

2019-05-18 11:40:12,680 UTC  WARN [[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] IntrusionDetector - [SECURITY FAILURE Anonymous:[email protected] -> /webtop/IntrusionDetector] Invalid input: context=HTTP cookie name: XSS: WebformTag, type(HTTPCookieName)=^[a-zA-Z0-9\-_]{1,50}$, input=https%3A%2F%2Fweblogic_server_01.domain.com%3A8085%2FD2%2Fx3_portal%2Ftheme
org.owasp.esapi.errors.ValidationException: HTTP cookie name: XSS: WebformTag: Invalid input. Please conform to regex ^[a-zA-Z0-9\-_]{1,50}$ with a maximum length of 1024
        at org.owasp.esapi.reference.validation.StringValidationRule.checkWhitelist(StringValidationRule.java:144)
        at org.owasp.esapi.reference.validation.StringValidationRule.checkWhitelist(StringValidationRule.java:160)
        at org.owasp.esapi.reference.validation.StringValidationRule.getValid(StringValidationRule.java:284)
        at org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:214)
        at org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:185)
        at com.documentum.web.security.validators.WDKESAPIValidator.getValidCookieName(WDKESAPIValidator.java:136)
        at com.documentum.web.form.WebformTag.getJsessionidCookieName(WebformTag.java:401)
        at com.documentum.web.form.WebformTag.renderExtnJavaScript(WebformTag.java:326)
        at com.documentum.web.form.WebformTag.doStartTag(WebformTag.java:175)
        at jsp_servlet._wdk._system._errormessage.__errormessage._jsp__tag2(__errormessage.java:466)
        at jsp_servlet._wdk._system._errormessage.__errormessage._jsp__tag1(__errormessage.java:431)
        at jsp_servlet._wdk._system._errormessage.__errormessage._jspService(__errormessage.java:152)
        at weblogic.servlet.jsp.JspBase.service(JspBase.java:35)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:286)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:260)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:137)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:350)
        at weblogic.servlet.internal.ServletStubImpl.onAddToMapException(ServletStubImpl.java:489)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:376)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:247)
        at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:630)
        at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:502)
        at com.documentum.web.form.FormProcessor.dispatchURL(FormProcessor.java:2283)
        at com.documentum.web.formext.component.URLDispatchBridge.dispatch(URLDispatchBridge.java:108)
        at com.documentum.web.formext.component.ComponentDispatcher.mapRequestToComponent(ComponentDispatcher.java:505)
        at com.documentum.web.formext.component.ComponentDispatcher.doGet(ComponentDispatcher.java:354)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        at com.documentum.web.formext.component.ComponentDispatcher.doService(ComponentDispatcher.java:328)
        at com.documentum.web.formext.component.ComponentDispatcher.serviceAsNonController(ComponentDispatcher.java:166)
        at com.documentum.web.formext.component.ComponentDispatcher.service(ComponentDispatcher.java:147)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:286)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:260)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:137)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:350)
        at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at com.documentum.web.servlet.ResponseHeaderControlFilter.doFilter(ResponseHeaderControlFilter.java:351)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at com.documentum.web.servlet.CompressionFilter.doFilter(CompressionFilter.java:96)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at com.documentum.web.env.WDKController.processRequest(WDKController.java:239)
        at com.documentum.web.env.WDKController.doFilter(WDKController.java:184)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3706)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3672)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:328)
        at weblogic.security.service.SecurityManager.runAsForUserCode(SecurityManager.java:197)
        at weblogic.servlet.provider.WlsSecurityProvider.runAsForUserCode(WlsSecurityProvider.java:203)
        at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:71)
        at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2443)
        at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2291)
        at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2269)
        at weblogic.servlet.internal.ServletRequestImpl.runInternal(ServletRequestImpl.java:1705)
        at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1665)
        at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:272)
        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)
        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)
        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)
        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)
        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)
2019-05-18 11:40:12,681 UTC ERROR [[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] com.documentum.web.common.Trace - Encountered error in error message component jsp
com.documentum.web.security.exceptions.SecurityWrapperRuntimeException: HTTP cookie name: XSS: WebformTag: Invalid input. Please conform to regex ^[a-zA-Z0-9\-_]{1,50}$ with a maximum length of 1024
        at com.documentum.web.security.validators.WDKESAPIValidator.getValidCookieName(WDKESAPIValidator.java:141)
        at com.documentum.web.form.WebformTag.getJsessionidCookieName(WebformTag.java:401)
        at com.documentum.web.form.WebformTag.renderExtnJavaScript(WebformTag.java:326)
        at com.documentum.web.form.WebformTag.doStartTag(WebformTag.java:175)
        at jsp_servlet._wdk._system._errormessage.__errormessage._jsp__tag2(__errormessage.java:466)
        at jsp_servlet._wdk._system._errormessage.__errormessage._jsp__tag1(__errormessage.java:431)
        at jsp_servlet._wdk._system._errormessage.__errormessage._jspService(__errormessage.java:152)
        at weblogic.servlet.jsp.JspBase.service(JspBase.java:35)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:286)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:260)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:137)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:350)
        at weblogic.servlet.internal.ServletStubImpl.onAddToMapException(ServletStubImpl.java:489)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:376)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:247)
        at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:630)
        at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:502)
        at com.documentum.web.form.FormProcessor.dispatchURL(FormProcessor.java:2283)
        at com.documentum.web.formext.component.URLDispatchBridge.dispatch(URLDispatchBridge.java:108)
        at com.documentum.web.formext.component.ComponentDispatcher.mapRequestToComponent(ComponentDispatcher.java:505)
        at com.documentum.web.formext.component.ComponentDispatcher.doGet(ComponentDispatcher.java:354)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        at com.documentum.web.formext.component.ComponentDispatcher.doService(ComponentDispatcher.java:328)
        at com.documentum.web.formext.component.ComponentDispatcher.serviceAsNonController(ComponentDispatcher.java:166)
        at com.documentum.web.formext.component.ComponentDispatcher.service(ComponentDispatcher.java:147)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:286)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:260)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:137)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:350)
        at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at com.documentum.web.servlet.ResponseHeaderControlFilter.doFilter(ResponseHeaderControlFilter.java:351)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at com.documentum.web.servlet.CompressionFilter.doFilter(CompressionFilter.java:96)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at com.documentum.web.env.WDKController.processRequest(WDKController.java:239)
        at com.documentum.web.env.WDKController.doFilter(WDKController.java:184)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:78)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3706)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3672)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:328)
        at weblogic.security.service.SecurityManager.runAsForUserCode(SecurityManager.java:197)
        at weblogic.servlet.provider.WlsSecurityProvider.runAsForUserCode(WlsSecurityProvider.java:203)
        at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:71)
        at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2443)
        at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2291)
        at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2269)
        at weblogic.servlet.internal.ServletRequestImpl.runInternal(ServletRequestImpl.java:1705)
        at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1665)
        at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:272)
        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)
        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)
        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)
        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)
        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)

 

So what’s that? Well the logs are pretty much self-explanatory but in short, the issue is a mismatch on the ESAPI layer. For several years, Documentum applications (DA, D2, D2-Config, D2-Rest, aso…) are all using the OWASP Enterprise Security API (aka ESAPI). This is an open source library that can be used to apply/enforce security controls on Web Applications. As you can see above, the issue in this case is there is a Cookie Name that doesn’t match the expected regex (regular expression) defined in the ESAPI.properties file of Documentum Administrator:

[[email protected]_server_01 ~]$ cd $APPLICATIONS
[[email protected]_server_01 apps]$ 
[[email protected]_server_01 apps]$ grep "CookieName" da/WEB-INF/classes/ESAPI.properties
Validator.HTTPCookieName=^[a-zA-Z0-9\\-_]{1,50}$
[[email protected]_server_01 apps]$

 

When accessing DA in a new/clean browser window (E.g.: in private mode), then DA will setup its own stuff and you will have the following cookies:

Cookie: appname=da;
        wdk_sess_cookie_0=eJwFGztVFUQJkM4lg1NzQysNTQzBJlpYZuCrQEA00NLck1hcxTMoNnKMLI0TexNUwNjMTG39asqdHK0CCs3Lg3NcXXNyC/1Cq4odTHKdAq1tMhyjHdN8vTc1xTK0MD1DUNTS1NcvdSzRDUDAnrIeYA..;
        JSESSIONID_da=Nxs2AYuOOuD2iBZqBA8Vw3UuuUlEjA_EbESx98AthouJ-elAeyWP!-1595571209

 

Then if you access any application like D2-Config, D2-REST, D2-Smartview or a WebLogic Admin Console for example, then each of these application will add their own cookies but usually cookies are removed when you leave the application because not needed anymore, except for the session ones (JSESSIONID). So, after accessing D2-Config and back to DA, then you should see something like that:

Cookie: appname=da;
        wdk_sess_cookie_0=eJwFGztVFUQJkM4lg1NzQysNTQzBJlpYZuCrQEA00NLck1hcxTMoNnKMLI0TexNUwNjMTG39asqdHK0CCs3Lg3NcXXNyC/1Cq4odTHKdAq1tMhyjHdN8vTc1xTK0MD1DUNTS1NcvdSzRDUDAnrIeYA..;
        JSESSIONID_da=Nxs2AYuOOuD2iBZqBA8Vw3UuuUlEjA_EbESx98AthouJ-elAeyWP!-1595571209;
        JSESSIONID_D2Config=RR5rnoSBFAAHOR342UkkiRV2GtCqZ9TyYbwrsDTD9K058yON4SAj!491149461

 

Going back to DA after that doesn’t show any issue because the application specific cookies aren’t there anymore. The problem is only for D2. The reason is that D2 uses internally the Sencha GXT and a specific cookie is setup by that toolkit. This cookie isn’t removed when you leave the application and therefore, since the name of this cookie doesn’t follow the restriction of the DA ESAPI, then DA isn’t working anymore. Loading D2 and DA afterwards will result in the following cookies:

Cookie: appname=da;
        wdk_sess_cookie_0=eJwFGztVFUQJkM4lg1NzQysNTQzBJlpYZuCrQEA00NLck1hcxTMoNnKMLI0TexNUwNjMTG39asqdHK0CCs3Lg3NcXXNyC/1Cq4odTHKdAq1tMhyjHdN8vTc1xTK0MD1DUNTS1NcvdSzRDUDAnrIeYA..;
        JSESSIONID_da=Nxs2AYuOOuD2iBZqBA8Vw3UuuUlEjA_EbESx98AthouJ-elAeyWP!-1595571209;
        JSESSIONID_D2Config=RR5rnoSBFAAHOR342UkkiRV2GtCqZ9TyYbwrsDTD9K058yON4SAj!491149461
        JSESSIONID_D2=SxeOGoLdAWHtfOhs2D6g-525FcVyUQJFIqUfsJJx5Au6-yjFop3P!78206796;
        https%3A%2F%2Fweblogic_server_01.domain.com%3A8085%2FD2%2Fx3_portal%2Ftheme=%7B%22state%22%3A%7B%22id%22%3A%22s%3Aslate%22%2C%20%22file%22%3A%22s%3Aresources%2Fthemes%2Fslate%2Fcss%2Fxtheme-x3.css%22%7D%7D

 

You can see above the same cookie name as in the DA logs: “https%3A%2F%2Fweblogic_server_01.domain.com%3A8085%2FD2%2Fx3_portal%2Ftheme“. That’s a strange name for a cookie (it’s the encoded URL of the D2 theme) but that’s how it… Because this cookie isn’t removed when you leave D2 and because DA restrict the length and allowed characters in the name of cookies, then you have this issue. A few months ago, we opened the OpenText SR#4366426 to ask whether they could fix that but the feedback we got so far is that no it’s not possible because it’s coming from the Sencha GXT and they have apparently no control over that. Therefore, the only solution I can see would be to update the ESAPI regex to allow this cookie. It’s not really a good idea to update the ESAPI regex, unfortunately, I don’t see another solution/workaround… Here is an example on how it could potentially be done:

[[email protected]_server_01 apps]$ grep "CookieName" da/WEB-INF/classes/ESAPI.properties
Validator.HTTPCookieName=^[a-zA-Z0-9\\-_]{1,50}$
[[email protected]_server_01 apps]$ 
[[email protected]_server_01 apps]$ sed -i 's,^Validator.HTTPCookieName=\^.*\$,Validator.HTTPCookieName=\^\[a-zA-Z0-9%._:\\\\/\\\\-\]\{1\,100\}\$,' da/WEB-INF/classes/ESAPI.properties
[[email protected]_server_01 apps]$ 
[[email protected]_server_01 apps]$ grep "CookieName" da/WEB-INF/classes/ESAPI.properties
Validator.HTTPCookieName=^[a-zA-Z0-9%._:\\/\\-]{1,100}$
[[email protected]_server_01 apps]$

 

Basically what I added above is to allow the additional characters that are part of the CookieName/URL (%, ., : and /) and I also allowed more chars because by default it’s only 50 but the URL is longer. With this new definition, redeploy/update DA and the issue should be gone.

 

What I still don’t understand is why I can’t see the same issue for D2 itself or with an older version of DA (7.3 for example)… Both D2 and DA uses the ESAPI but I can only see DA 16.x fail, older versions as well as D2 are both working fine without error while their regex is even more strict: “^[a-zA-Z0-9\\-_]{1,32}$“. This is the same pattern as for DA 16.x but it limits to 32 chars max, which is even less. There must have been a bug in earlier versions that prevented this property from actually doing anything. It might have been a change on the ESAPI layer as well but I assume both DA and D2 ESAPI are in sync? I guess I will try to do some research on that.

 

Normally the issue shouldn’t happen anymore on the 20.4 release because D2 will force the Sencha GXT to use web storage instead of cookies. This will also be applied on the next patches of earlier D2 versions normally (E.g.: 16.5.1 P06, planned for end of April 2020).

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Morgan Patou
Morgan Patou

Technology Leader ECM & Senior Consultant