dbi Blog

This is part II of the 2-part article. See for part I of this article.

Testing

We will test on the host machine named docker that hosts 2 containers, container01 and container011. All 3 machines run a repository. Its name is respectively dmtest on docker (shortly, dmtest@docker:1489), dmtest01@container01:1489 (dmtest01@container01:2489 externally) and dmtest01@container011:1489 (dmtest01@container011:5489 externally). Incidentally, the enhanced syntax is also a good way to uniquely identify the repositories.
The current dfc.properties file on the host docker:
$ grep docbroker /app/dctm/config/dfc.properties dfc.docbroker.host[0]=docker dfc.docbroker.port[0]=1489
This is used for the local docbase dmtest.
Let’s tag all the docbases for an easy identification later:
$ iapi dmtest -Udmadmin -Pdmadmin <<eoq retrieve,c,dm_docbase_config set,c,l,title dmtest on docker host VM save,c,l eoq
Idem from within container01 with its default dfc.properties file:
$ iapi dmtest01 -Udmadmin -Pdmadmin <<eoq retrieve,c,dm_docbase_config set,c,l,title dmtest01 created silently on container01 save,c,l eoq
Idem from within container011:
$ iapi dmtest01 -Udmadmin -Pdmadmin <<eoq retrieve,c,dm_docbase_config set,c,l,title dmtest01 created silently on container011 save,c,l eoq
First, let's access container01.dmtest01 from the containers' host VM with the current dfc.properties file:
$ idql dmtest01 -Udmadmin -Pdmadmin OpenText Documentum idql - Interactive document query interface Copyright (c) 2018. OpenText Corporation All rights reserved. Client Library Release 16.4.0070.0035 Connecting to Server using docbase dmtest01 Could not connect [DM_DOCBROKER_E_NO_SERVERS_FOR_DOCBASE]error: "The DocBroker running on host (docker:1489) does not know of a server for the specified docbase (dmtest01)"
As expected, it does not work because container01.dmtest01 does not project to the host’s docbroker. Now, let’s turn to widql:
$ ./widql dmtest01@docker:2489 -Udmadmin -Pdmadmin --keep <<eoq select title from dm_docbase_config go eoq OpenText Documentum idql - Interactive document query interface Copyright (c) 2018. OpenText Corporation All rights reserved. Client Library Release 16.4.0070.0035 Connecting to Server using docbase dmtest01 [DM_SESSION_I_SESSION_START]info: "Session 0100c350800011bb started for user dmadmin." Connected to OpenText Documentum Server running Release 16.4.0000.0248 Linux64.Oracle title ------------------------------------------ dmtest01 created silently on container01
It works.
We used ––keep, therefore the dfc.properties file has changed:
$ grep docbroker /app/dctm/config/dfc.properties dfc.docbroker.host[0]=docker dfc.docbroker.port[0]=2489
Indeed.
That docbase can also be reached by the container’s IP address and internal port 1489:
$ docker exec -it container01 ifconfig eth0 | head -3 eth0: flags=4163 mtu 1500 inet 192.168.33.101 netmask 255.255.255.0 broadcast 192.168.33.255 ether 02:42:c0:a8:21:65 txqueuelen 0 (Ethernet) $ ./widql [email protected]:1489 -Udmadmin -Pdmadmin <<eoq select title from dm_docbase_config go eoq ... Connecting to Server using docbase dmtest01 [DM_SESSION_I_SESSION_START]info: "Session 0100c350800011b5 started for user dmadmin." ... title ------------------------------------------ dmtest01 created silently on container01
Is the local dmtest docbase still reachable ?:
$ idql dmtest -Udmadmin -Pdmadmin ... Could not connect [DM_DOCBROKER_E_NO_SERVERS_FOR_DOCBASE]error: "The DocBroker running on host (docker:2489) does not know of a server for the specified docbase (dmtest)"
Not with that changed dfc.properties file and the standard tools. But by using our nifty little tool:
$ ./widql dmtest@docker:1489 -Udmadmin -Pdmadmin <<eoq select title from dm_docbase_config go eoq ... Connected to OpenText Documentum Server running Release 16.4.0080.0129 Linux64.Oracle title ---------------------- dmtest on host VM
Fine !
Is container011.dmtest01 reachable now ?
$ ./widql dmtest01 -Udmadmin -Pdmadmin <<eoq select title from dm_docbase_config go eoq ... Connecting to Server using docbase dmtest01 ... Connected to OpenText Documentum Server running Release 16.4.0000.0248 Linux64.Oracle title ------------------------------------------- dmtest01 created silently on container01
This is container01.dmtest01, not the one we want, i.e. the one on container011.
Note that ./widql was called without the extended syntax so it invoked the standard idql directly.
Let try again:
$ ./widql dmtest01@docker:5489 -Udmadmin -Pdmadmin <<eoq select title from dm_docbase_config go eoq ... Connecting to Server using docbase dmtest01 [DM_SESSION_I_SESSION_START]info: "Session 0100c3508000059e started for user dmadmin." ... title ------------------------------------------ dmtest01 created silently on container011
Here we go, it works !
The same using the container’s IP address and its docbroker’s internal port:
$ docker exec -it container011 ifconfig eth0 | head -3 eth0: flags=4163 mtu 1500 inet 192.168.33.104 netmask 255.255.255.0 broadcast 192.168.33.255 ether 02:42:c0:a8:21:68 txqueuelen 0 (Ethernet) $ ./widql [email protected]:5489 -Udmadmin -Pdmadmin <<eoq select title from dm_docbase_config go eoq ... Connecting to Server using docbase dmtest01 [DM_SESSION_I_SESSION_START]info: "Session 0100c35080000598 started for user dmadmin." ... title ------------------------------------------ dmtest01 created silently on container011
Try now the same connection but with ––append and ––keep:
$ ./widql dmtest01@docker:5489 -Udmadmin -Pdmadmin --append --keep <<eoq select title from dm_docbase_config go eoq ... Connecting to Server using docbase dmtest01 ... Connected to OpenText Documentum Server running Release 16.4.0000.0248 Linux64.Oracle title ------------------------------------------- dmtest01 created silently on container011
What is the content of dfc.properties now ?
$ grep docbroker /app/dctm/config/dfc.properties dfc.docbroker.host[0]=docker dfc.docbroker.port[0]=2489 dfc.docbroker.host[1]=docker dfc.docbroker.port[1]=5489
Both options have been taken into account as expected.
Let’s try to reach the VM host’s repository:
$ ./widql dmtest -Udmadmin -Pdmadmin <<eoq select title from dm_docbase_config go eoq ... Connecting to Server using docbase dmtest Could not connect [DM_DOCBROKER_E_NO_SERVERS_FOR_DOCBASE]error: "The DocBroker running on host (docker:2489) does not know of a server for the specified docbase (dmtest)"
Specify the docbroker’s host and the ––verbose option:
$ ./widql dmtest@docker -Udmadmin -Pdmadmin --verbose <<eoq select title from dm_docbase_config go eoq changing to docker:... requested changes: # removed: dfc.docbroker.host[0]=docker # removed: dfc.docbroker.port[0]=2489 # removed: dfc.docbroker.host[1]=docker # removed: dfc.docbroker.port[1]=5489 # added: dfc.docbroker.host[0]=docker # added: dfc.docbroker.port[0]=2489 diffs: 12,13d11 < dfc.docbroker.host[1]=docker < dfc.docbroker.port[1]=5489 calling original: /app/dctm/product/16.4/bin/idql dmtest -Udmadmin -Pdmadmin ... Connecting to Server using docbase dmtest Could not connect [DM_DOCBROKER_E_NO_SERVERS_FOR_DOCBASE]error: "The DocBroker running on host (docker:2489) does not know of a server for the specified docbase (dmtest)"
Since the port was not specified, the wrapper took the first port found in the dfc.properties to supply the missing value, i.e. 2489 which is incorrect as dmtest@docker only projects to port docker:1489.
Use an unambiguous command now:
$ ./widql dmtest@docker:1489 -Udmadmin -Pdmadmin ––verbose <<eoq select title from dm_docbase_config go eoq changing to docker:1489... requested changes: # removed: dfc.docbroker.host[0]=docker # removed: dfc.docbroker.port[0]=2489 # removed: dfc.docbroker.host[1]=docker # removed: dfc.docbroker.port[1]=5489 # added: dfc.docbroker.host[0]=docker # added: dfc.docbroker.port[0]=1489 diffs: 11,13c11 < dfc.docbroker.port[0]=2489 < dfc.docbroker.host[1]=docker dfc.docbroker.port[0]=1489 calling original: /app/dctm/product/16.4/bin/idql dmtest -Udmadmin -Pdmadmin ... Connecting to Server using docbase dmtest ... Connected to OpenText Documentum Server running Release 16.4.0080.0129 Linux64.Oracle title -------------------- dmtest on host VM
Looks OK.
Let’s try wdmawk now. But first, here is the test code twdmawk.awk:

$ cat twdmawk.awk 
BEGIN {
   print "repo_target=" repo_target, "docbase=" docbase
   session = dmAPIGet("connect," docbase ",dmadmin,dmadmin")
   print dmAPIGet("getmessage," session)
   dmAPIGet("retrieve," session ",dm_docbase_config")
   print dmAPIGet("get," session ",l,title")
   dmAPIExec("disconnect," session)
   exit(0)
}

Line 3 displays the two variables automatically passed to dmawk by the wrapper, repo_target and docbase.
The test script connects to the docbase which was silently passed as command-line parameter by wdmawk through the -v option after it extracted it from the given target parameter docbase[@host[:port]], as illustrated below with the ––verbose option.
Let’s see the invocation for the repository on the host VM:
$ ./wdmawk dmtest@docker:1489 -f ./twdmawk.awk --verbose changing to docker:1489... requested changes: # removed: dfc.docbroker.host[0]=docker # removed: dfc.docbroker.port[0]=2489 # removed: dfc.docbroker.host[1]=docker # removed: dfc.docbroker.port[1]=5489 # added: dfc.docbroker.host[0]=docker # added: dfc.docbroker.port[0]=1489 diffs: 11,13c11 < dfc.docbroker.port[0]=2489 < dfc.docbroker.host[1]=docker –– > dfc.docbroker.port[0]=1489 calling original: /app/dctm/product/16.4/bin/dmawk -v docbase=dmtest -f ./twdmawk.awk repo_target= docbase=dmtest [DM_SESSION_I_SESSION_START]info: "Session 0100c3508000367b started for user dmadmin." dmtest on host VM
Let’s acces the container01’s repository :
$ ./wdmawk dmtest01@docker:2489 -f ./twdmawk.awk [DM_SESSION_I_SESSION_START]info: "Session 0100c35080001202 started for user dmadmin." dmtest01 created silently on container01
A small typo in the port number and …
dmadmin@docker:~$ ./wdmawk dmtest01@docker:3489 -f ./twdmawk.awk [DFC_DOCBROKER_REQUEST_FAILED] Request to Docbroker "docker:3489" failed [DM_SESSION_E_RPC_ERROR]error: "Server communication failure" java.net.ConnectException: Connection refused (Connection refused)
Note the stupid error message “… Connection refused …”, very misleading when investigating a problem. It’s just that there nobody listening on that port.
Let’s access the container011’s repository:
dmadmin@docker:~$ ./wdmawk dmtest01@docker:5489 -f ./twdmawk.awk [DM_SESSION_I_SESSION_START]info: "Session 0100c350800005ef started for user dmadmin." dmtest01 created silently on container011
Effect of the -v option:
dmadmin@docker:~$ ./wdmawk -v dmtest01@docker:5489 -f ./twdmawk.awk --verbose ... calling original: /app/dctm/product/16.4/bin/dmawk -v repo_target=dmtest@docker:1489 -v docbase=dmtest -f ./twdmawk.awk repo_target=dmtest@docker:1489 docbase=dmtest [DM_SESSION_I_SESSION_START]info: "Session 0100c35080003684 started for user dmadmin." dmtest on host VM
A repo_target parameter with the extended syntax has been passed to dmawk.
Let’s now quickly check the wrapper from within the containers.
Container01
The host’s docbase:
[dmadmin@container01 scripts]$ ./wiapi dmtest@docker:1489 -Udmadmin -Pdmadmin<<eoq retrieve,c,dm_docbase_config get,c,l,title eoq ... Connecting to Server using docbase dmtest ... dmtest on host VM
The container011’s docbase:
[dmadmin@container01 scripts]$ ./wiapi dmtest01@container011:1489 -Udmadmin -Pdmadmin<<eoq retrieve,c,dm_docbase_config get,c,l,title eoq ... Connecting to Server using docbase dmtest01 ... dmtest01 created silently on container011 ...
Container011
The host’s docbase:
dmadmin@container011 scripts]$ ./wiapi dmtest@docker:1489 -Udmadmin -Pdmadmin<<eoq retrieve,c,dm_docbase_config get,c,l,title eoq ... Connecting to Server using docbase dmtest ... Connected to OpenText Documentum Server running Release 16.4.0080.0129 Linux64.Oracle ... dmtest on host VM ...
The docbase on container01:
dmadmin@container011 scripts]$ ./wiapi dmtest01@container01:1489 -Udmadmin -Pdmadmin<<eoq retrieve,c,dm_docbase_config get,c,l,title eoq ... ... Connecting to Server using docbase dmtest01 ... dmtest01 created silently on container01 ...
Let’s briefly test the usage of the sourced configuration file. Here is a snippet of the file shown earlier in this article:

# repository connection configuration file;
# must be sourced prior so the environment variables can be resolved;
# this is a enhancement over the dfc.properties file syntax used by the dctm_wrapper utility:
# docbroker.host[i]=...
# docbroker.port[i]=...
# it supports several syntaxes:
# docbroker only definition [[docbroker_host]:[port]];
#    usage: ./widql dmtest@$dmtest
# full definition docbase[@[docbroker_host]:[port]]
#    usage: ./widql $test
# alternate ':' separator docbase:[[docbroker_host]:[docroker_port]];
#    usage: ./widql $dmtestVM
# alias literal;
#    usage: ./widql test
# in order to resolve alias literals, the wrapper will source the configuration file by itself;
...
# container011.dmtest01;
# docbroker only definition docbroker_host:port;
d_dmtest011=container011:5489
di_dmtest011=192.168.33.104:1489
# full definition;
f_dmtest011=dmtest01@container011:2489
[email protected]:1489

With a good name convention, the variables can be easily remembered which saves a lot of typing too.
Note on lines 9 and 10 how the whole extended target name can be specified, including the repository name.
A few tests:
dmadmin@docker:~$ ./widql dmtest01@$d_dmtest011 -Udmadmin -Pdmadmin --verbose current_cmd=[./widql dmtest01@container011:5489 -Udmadmin -Pdmadmin --verbose] ... dmadmin@docker:~$ ./widql dmtest01@$dip_dmtest011 -Udmadmin -Pdmadmin --verbose current_cmd=[./widql [email protected]:1489 -Udmadmin -Pdmadmin --verbose] ... dmadmin@docker:~$ ./widql $f_dmtest011 -Udmadmin -Pdmadmin --verbose current_cmd=[./widql dmtest01@container011:2489 -Udmadmin -Pdmadmin --verbose] ... dmadmin@docker:~$ ./widql $fip_dmtest011 -Udmadmin -Pdmadmin --verbose current_cmd=[./widql [email protected]:1489 -Udmadmin -Pdmadmin --verbose] ...
The variables have been expanded by the shell prior to entering the wrapper, no programming effort was needed here, which is always appreciated.

Possible Enhancements

As shown precedently, the alternate configuration file lists aliases for the couples docbroker:port and even repository@docbroker:port. In passing, the wrapper also supports the version repository:docbroker:port.
Now, in order to better match Documentum syntax, is it possible to be even more transparent by removing dollar signs, colons and at-signs while still accessing the extended syntax ? E.g.:
$ ./widql dmtest -Udmadmin ....
Yes it is. The trick here is to first look up the alias in the configuration file, which incidentally becomes mandatory now, and re-execute the program with the alias resolved. As we are all lazy coders, we will not explicitly code the looking up but instead rely on the shell: the wrapper will source the file, resolve the target and re-execute itself.
If the alias has not been defined in the file, then the wrapper considers it as the name of a repository and falls back to the usual command-line tools.
A good thing is that no new format has to be introduced in the file as the target is still the name of an environment variable.
Since the changes are really minimal, let’s do it. Hereafter, the diff output showing the changes from the listing in part I:

71c71,74
<  
---
> 
> # this variable points to the target repositories alias file and defaults to repository_connections.aliases;
> REPO_ALIAS=${REPO_ALIAS:-~/repository_connections.aliases}
> 
121a125
> [[ bVerbose -eq 1 ]] && echo "current configuration file=[$REPO_ALIAS]"
253a258,265
> else
>    [[ -f $REPO_ALIAS ]] && . $REPO_ALIAS
>    definition=${!1}
>    [[ $bVerbose -eq 1 ]] && echo "alias lookup in $REPO_ALIAS: $1 = $definition"
>    if [[ ! -z $definition ]]; then
>       new_cmd=${current_cmd/$1/$definition}
>       [[ $bVerbose -eq 1 ]] && echo "invoking $new_cmd"
>       exec $new_cmd

On line 6, the target configuration file pointed to by the REPO_ALIAS environment variable gets sourced if existing. $REPO_ALIAS defaults to repository_connections.aliases but can be changed before calling the wrapper.
Note on line 13 how bash can dereference a variable 1 containing the name of another variable 2 to get variable 2’s value (indirect expansion), nice touch.
On line 18, the script execs itself with the resolved repository target.
To apply the patch in-place, save the diffs above in diff-file and run the following command:
patch old-file < diff-file
Testing
For conciseness, the tests below only show how the target is resolved. The actual connection has already been tested abundantly earlier.
dmadmin@docker:~$ ./widql f_dmtest -Udmadmin -Pdmadmin --verbose current_cmd=[./widql f_dmtest -Udmadmin -Pdmadmin --verbose] alias lookup in /home/dmadmin/repository_connections.aliases: f_dmtest = dmtest@docker:1489 invoking ./widql dmtest@docker:1489 -Udmadmin -Pdmadmin --verbose current_cmd=[/home/dmadmin/widql dmtest@docker:1489 -Udmadmin -Pdmadmin --verbose] ... dmadmin@docker:~$ ./widql fip_dmtest01 -Udmadmin -Pdmadmin --verbose current_cmd=[./widql fip_dmtest01 -Udmadmin -Pdmadmin --verbose] alias lookup in /home/dmadmin/repository_connections.aliases: fip_dmtest01 = [email protected]:1489 invoking ./widql [email protected]:1489 -Udmadmin -Pdmadmin --verbose current_cmd=[/home/dmadmin/widql [email protected]:1489 -Udmadmin -Pdmadmin --verbose] ... dmadmin@docker:~$ ./widql fip_dmtest011 -Udmadmin -Pdmadmin --verbose current_cmd=[./widql fip_dmtest011 -Udmadmin -Pdmadmin --verbose] alias lookup in /home/dmadmin/repository_connections.aliases: fip_dmtest011 = [email protected]:1489 invoking ./widql [email protected]:1489 -Udmadmin -Pdmadmin --verbose current_cmd=[/home/dmadmin/widql [email protected]:1489 -Udmadmin -Pdmadmin --verbose]
Note how the targets are cleaner now, no curly little fancy shell characters in front.

Conclusion

As I was testing this little utility, I was surprised to realize how confortable and natural its usage is. It feels actually better to add the docbroker’s host and port than to stop at the docbase name, probably because it makes the intented repository absolutely unambiguous. The good thing is that is almost invisible, except for its invocation but even this can be smoothed out by using command aliases or renaming the symlinks.
When one has to work with identically named docbases or with clones existing in different environments, dctm-wrapper brings a real relief. And it was quick and easy to put together too.
As it modifies an essential configuration file, it is mainly aimed at developers or administrators on their machine, but then those constitute the targeted audience anyway.
As always, if you have any ideas for some utility that could benefit us all, please do no hesitate to suggest them in the comment section. Feedback is welcome too of course.

Post Views: 769