How to stop Documentum processes in a docker container, and more (part II)

ok, Ok, OK, and the docker part ?

In a minute.
In part I of this 2-part article, we showed how traps could be used to control a running executable from the outside. We also presented a bash test script to try out and play with traps. Now that we are confident about that simulation script, let’s dockerize it and try it out in this new environment. We use the dockerfile Dockerfile-dctm to create the CS image and so we include an ENTRYPOINT clause as follows:

FROM ubuntu:latest
RUN apt-get update &&      
    apt-get install -y gawk
COPY dctm.sh /root/.
ENTRYPOINT ["/root/dctm.sh", "start"]

The above ENTRYPOINT syntax allows to run the dctm.sh script with PID 1 because the initial bash process (which runs with PID 1 obviously) performs an exec call to load and execute that script. To keep the dockerfile simple, the script will run as root. In the real-world, CS processes run as something like dmadmin, so this account would have to be set up in the dockerfile (or through some orchestration software).
When the docker image is run or the container is started, the dctm.sh script gets executed with PID 1; as the script is invoked with the start option, it starts the processes. Afterwards, it justs sits there waiting for the SIGTERM signal from the docker stop command; once received, it shuts down all the running processes under its control and exits, which will also stop the container’s process. Additionaly, it can listen and react to some other signals, just like when it runs outside of a container.

Testing

Let’s test this approach with a container built using the above simple Dockerfile-dctm. Since the container is started in interactive mode, its output is visible on the screen and the commands to test it have to be sent from another terminal session; as before, for clarity, the commands have been inserted in the transcript as comments right before their result.

docker build -f Dockerfile-dctm --tag=dctm .
Sending build context to Docker daemon 6.656kB
Step 1/5 : FROM ubuntu:latest
---> 1d9c17228a9e
Step 2/5 : RUN apt-get update && apt-get install -y gawk
---> Using cache
---> f550d88161b6
Step 3/5 : COPY dctm.sh /root/.
---> e15e3f4ea93c
Step 4/5 : HEALTHCHECK --interval=5s --timeout=2s --retries=1 CMD grep -q OK /tmp/status || exit 1
---> Running in 0cea23cec09e
Removing intermediate container 0cea23cec09e
---> f9bf4138eb83
Step 5/5 : ENTRYPOINT ["/root/dctm.sh", "start"]
---> Running in 670c5231d5d8
Removing intermediate container 670c5231d5d8
---> 27991672905e
Successfully built 27991672905e
Successfully tagged dctm:latest
 
# docker run -i --name=dctm dctm
process started with pid 9 and random value 32760057
process started with pid 10 and random value 10364519
process started with pid 11 and random value 2915264
process started with pid 12 and random value 3744070
process started with pid 13 and random value 23787621
5 processes started
 
started processes:
1 9 1 1 ? -1 S 0 0:00 I am number 32760057
1 10 1 1 ? -1 S 0 0:00 I am number 10364519
1 11 1 1 ? -1 S 0 0:00 I am number 2915264
1 12 1 1 ? -1 S 0 0:00 I am number 3744070
1 13 1 1 ? -1 S 0 0:00 I am number 23787621
 
send signal SIGURG for help
send signal SIGPWR to start a few processes
send signal SIGUSR1 to start a new process
send signal SIGUSR2 for the list of started processes
send signal SIGINT | SIGABRT to stop all the processes
send signal SIGHUP | SIGQUIT | SIGTERM to shutdown the processes and exit the container
 
# docker kill --signal=SIGUSR2 dctm
status of running processes at 2019/04/06 14:56:14
showing 32760057
random value 32760057 is used by process with pid 9 and pgid 1
showing 10364519
random value 10364519 is used by process with pid 10 and pgid 1
showing 2915264
random value 2915264 is used by process with pid 11 and pgid 1
showing 3744070
random value 3744070 is used by process with pid 12 and pgid 1
showing 23787621
random value 23787621 is used by process with pid 13 and pgid 1
5 processes found
 
# docker kill --signal=SIGURG dctm
send signal SIGURG for help
send signal SIGPWR to start a few processes
send signal SIGUSR1 to start a new process
send signal SIGUSR2 for the list of started processes
send signal SIGINT | SIGABRT to stop all the processes
send signal SIGHUP | SIGQUIT | SIGTERM to shutdown the processes and exit the container
 
# docker kill --signal=SIGUSR1 dctm
starting a new process at 2019/04/06 14:57:30
process started with pid 14607 and random value 10066771
 
# docker kill --signal=SIGABRT dctm
shutting down the processes at 2019/04/06 14:58:12
stopping 32760057
stopping 10364519
stopping 2915264
stopping 3744070
stopping 23787621
stopping 10066771
6 processes stopped
 
# docker kill --signal=SIGUSR2 dctm
status of running processes at 2019/04/06 14:59:01
0 processes found
 
# docker kill --signal=SIGTERM dctm
shutting down the container at 2019/04/06 14:59:19
 
shutting down the processes at 2019/04/06 14:59:19
0 processes stopped
 
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

We observe exactly the same behavior as the stand-alone dctm.sh, that’s comforting.
Moreover, when the container is stopped, the signal is trapped correctly by the proxy:

...
random value 14725194 is used by process with pid 29 and pgid 1
showing 12554300
random value 12554300 is used by process with pid 30 and pgid 1
5 processes found
 
# date -u +"%Y/%m/%d %H:%M:%S"; docker stop dctm
# 2019/04/10 22:51:47
# dctm
shutting down the container at 2019/04/10 22:51:47
 
shutting down the processes at 2019/04/10 22:51:47
stopping 36164161
stopping 6693775
stopping 11404415
stopping 14725194
stopping 12554300
5 processes stopped

The good thing is that if the docker daemon is stopped at the host level, either interactively or at system shut down, the daemon first sends a SIGTERM to every running container:

date --utc +"%Y/%m/%d %H-%M-%S"; sudo systemctl stop docker
2019/04/06 15-02-18
[sudo] password for docker:

and on the other terminal:

shutting down the container at 2019/04/06 15:02:39
 
shutting down the processes at 2019/04/06 15:02:39
stopping 17422702
stopping 30251419
stopping 14451888
stopping 14890733
stopping 1105445
5 processes stopped

so each container can process the signal accordingly to its needs. Our future Documentum container is now ready for a clean shutdown.

Doing something useful instead of sitting idle: light monitoring

As said, the proxy script waits for a signal from within a loop; the action performed inside the loop is waiting for an input from stdin, which is not particularly useful. Why not taking advantage of this slot to make it do something useful like a monitoring of the running processes ? Such a function already exists in the script, it’s status_all(). Thus, let’s set this up:

# while true; do read; done
# do something useful instead;
while true; do
status_all
sleep 30
done

We quickly notice that their processing is not so briskly any more. In effect, bash waits before processing a signal until the command currently executing completes, here any command inside the loop, so a slight delay is perceptible before our signals get care of, especially if we are in the middle of a ‘sleep 600’ command. Moreover, incoming signals are not stacked up but they replace each one another until the most recent one only is processed. In practical conditions, this is not a problem for it is still possible to send signals and have them processed, just not in burst mode. If a better reactivity to signals is needed, the sleep duration should be shortened and/or a separate scheduling of the monitoring be introduced (started asynchronously from a loop in the entrypoint or from a crontab inside the container ?).
Note that the status send to stdout from within a detached container (i.e. started without the -i for interactive option, which is generally the case) is not visible outside a container. Fortunately, and even better, the docker logs command makes it possible to view on demand the status output:

docker logs --follow container_name

In our case:

docker logs --follow dctm
status of running processes at 2019/04/06 15:21:21
showing 8235843
random value 8235843 is used by process with pid 8 and pgid 1
showing 16052839
random value 16052839 is used by process with pid 9 and pgid 1
showing 1097668
random value 1097668 is used by process with pid 10 and pgid 1
showing 5113933
random value 5113933 is used by process with pid 11 and pgid 1
showing 1122110
random value 1122110 is used by process with pid 12 and pgid 1
5 processes found

Note too that the logs commands also has a timestamps option for prefixing the lines output with the time they were produced, as illustrated below:

docker logs --timestamps --since 2019-04-06T18:06:23 dctm
2019-04-06T18:06:23.607796640Z status of running processes at 2019/04/06 18:06:23
2019-04-06T18:06:23.613666475Z showing 7037074
2019-04-06T18:06:23.616334029Z random value 7037074 is used by process with pid 8 and pgid 1
2019-04-06T18:06:23.616355592Z showing 33446655
2019-04-06T18:06:23.623719975Z random value 33446655 is used by process with pid 9 and pgid 1
2019-04-06T18:06:23.623785755Z showing 17309380
2019-04-06T18:06:23.627050839Z random value 17309380 is used by process with pid 10 and pgid 1
2019-04-06T18:06:23.627094599Z showing 13859725
2019-04-06T18:06:23.630436025Z random value 13859725 is used by process with pid 11 and pgid 1
2019-04-06T18:06:23.630472176Z showing 26767323
2019-04-06T18:06:23.633304616Z random value 26767323 is used by process with pid 12 and pgid 1
2019-04-06T18:06:23.635900480Z 5 processes found
2019-04-06T18:06:26.640490424Z

This is handy, but still not perfect, for those cases where lazy programmers neglect to date their logs’ entries.
Now, since we have a light-weight monitoring in place, we can use it in the dockerfile’s HEALTHCHECK clause to show the container’s status through the ps command. As the processes’ status is already determined in the wait loop of the dctm.sh script, it is pointless to compute it again. Instead, we can modify status_all() to print the overall status in a file, say in /tmp/status, so that HEALTHCHECK can read it later every $INTERVAL period. If status_all() is invoked every $STATUS_PERIOD, a race condition can occur every LeastCommonMultiple($INTERVAL, $STATUS_PERIOD), i.e. when these 2 processes will access the file simultaneously, the former in reading mode and the latter in writing mode. To avoid this nasty situation, status_all() will first write into /tmp/tmp_status and later rename this file to /tmp/status. For the sake of our example, let’s decide that the container is unhealthy if there are no dummy processes running, and healthy if there is at least one running (in real conditions, the container would be healthy if ALL the processes are responding, and unhealthy if ANY of them is not but it also depends on the definition of health). Here is the new dctm.sh’s status_all() function:

status_all() {
   echo; echo "status of running processes at $(date +"%Y/%m/%d %H:%M:%S")"
   nb_processes=0
   for no in $(ps -ef | grep "I am number " | grep -v grep | gawk '{print $NF}'); do
      echo "showing $no"
      func status $no
      (( nb_processes++ ))
   done
   echo "$nb_processes processes found"
   if [[ $nb_processes -eq 0 ]]; then
      printf "status: badn" > /tmp/tmp_status
   else
      printf "status: OKn" > /tmp/tmp_status
   fi
   mv /tmp/tmp_status /tmp/status
}

Here is the new dockerfile:

FROM ubuntu:latest
RUN apt-get update &&      
    apt-get install -y gawk
COPY dctm.sh /root/.
HEALTHCHECK --interval=10s --timeout=2s --retries=2 CMD grep -q OK /tmp/status || exit 1
ENTRYPOINT ["/root/dctm.sh", "start"]

Here is what the ps commands shows now:

# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
64e22a8f75cd dctm "/root/dctm.sh start" 38 minutes ago Up 2 seconds (health: starting) dctm
 
...
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
64e22a8f75cd dctm "/root/dctm.sh start" 38 minutes ago Up 6 seconds (healthy) dctm

A new column, STATUS, is displayed showing the container’s current health status.
If a new built is unwanted, the clause can be specified when running the image:

docker run --name dctm --health-cmd "grep -q OK /tmp/status || exit 1" --health-interval=10s --health-timeout=2s --health-retries=1 dctm

Note how these parameters are now prefixed with “health-” so they can be related to the HEALTHCHECK clause.
Now, in order to observe how the status is updated, let’s play with the signals INT and PWR to respectively stop and launch processes inside the container:

# current situation:
docker logs dctm
status of running processes at 2019/04/12 14:05:00
showing 29040429
random value 29040429 is used by process with pid 1294 and pgid 1
showing 34302125
random value 34302125 is used by process with pid 1295 and pgid 1
showing 2979702
random value 2979702 is used by process with pid 1296 and pgid 1
showing 4661756
random value 4661756 is used by process with pid 1297 and pgid 1
showing 7169283
random value 7169283 is used by process with pid 1298 and pgid 1
5 processes found
 
# show status:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ff25beae71f0 dctm "/root/dctm.sh start" 55 minutes ago Up 55 minutes (healthy) dctm
 
# stop the processes:
docker kill --signal=SIGINT dctm
# wait up to the given health-interval and check again:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ff25beae71f0 dctm "/root/dctm.sh start" 57 minutes ago Up 57 minutes (unhealthy) dctm
 
# restart the processes:
docker kill --signal=SIGPWR dctm
 
# wait up to the health-interval and check again:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ff25beae71f0 dctm "/root/dctm.sh start" About an hour ago Up About an hour (healthy) dctm

The healthstatus command works as expected.
Note that the above HEALTHCHECK successful tests were done under Centos Linux release 7.6.1810 (Core) with docker Client Version 1.13.1 and API version 1.26, Server Version 1.13.1 and API version 1.26 (minimum version 1.12).
The HEALTHCHECK clause looks broken under Ubuntu 18.04.1 LTS with docker Client Version 18.09.1 and API version 1.39, Server Engine – Community Engine Version 18.09.1 and API version 1.39 (minimum version 1.12). After a change of status, HEALTHCHECK sticks to the unhealthy state and “docker ps” always shows “healthy” no matter the following changes in the running processes inside the container. It looks like the monitoring cycles until an unhealthy condition occurs, then it stops cycling and stays in the unhealthy state, as it is also visible in the timestamps by inspecting the container’s status:

docker inspect --format='{{json .State.Health}}' dctm
{"Status":"unhealthy","FailingStreak":0,"Log":[{"Start":"2019-04-12T16:04:18.995957081+02:00","End":"2019-04-12T16:04:19.095540448+02:00","ExitCode":0,"Output":""},
{"Start":"2019-04-12T16:04:21.102151004+02:00","End":"2019-04-12T16:04:21.252025292+02:00","ExitCode":0,"Output":""},
{"Start":"2019-04-12T16:04:23.265929424+02:00","End":"2019-04-12T16:04:23.363387974+02:00","ExitCode":0,"Output":""},
{"Start":"2019-04-12T16:04:25.372757042+02:00","End":"2019-04-12T16:04:25.471229004+02:00","ExitCode":0,"Output":""},
{"Start":"2019-04-12T16:04:27.47692396+02:00","End":"2019-04-12T16:04:27.580458001+02:00","ExitCode":0,"Output":""}]}

The last 5 entries stop being updated.
While we are mentioning bugs, “docker logs –tail 0 dctm” under Centos displays the whole log available so far, so specify 1 at least to reduce the output of the log history to a minimum. Under Ubuntu, it works as expected though. However, the “–follow” option works under Centos but not under Ubuntu. So, there is some instability here; be prepared to comprehensively test every docker’s feature to be used.

Using docker’s built-in init process

As said above, docker does not have a full-fledged init process like systemd but still offers something vaguely related, tini, which stands for “tiny init”, see here. It wont’t solve the inability of Documentum’s processes to respond to signals and therefore the proxy script is still needed. However, in addition to forwarding signals to its child process, tini has the advantage of taking care of defunct processes, or zombies, by reaping them up regularly. Documentum produces a lot of them and they finish up disappearing in the long run. Still, tini could speeds this up a little bit.
tini can be invoked from the command-line as follows:

docker run -i --name=dctm --init dctm

But it is also possible to integrate it directly in the dockerfile so the −−init option won’t be needed any longer (and shouldn’t be used otherwise tini will not be PID 1 and its reaping feature won’t be possible anymore, making it useless for us):

FROM ubuntu:latest
COPY dctm.sh /root/.
ENV TINI_VERSION v0.18.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN apt-get update &&      
    apt-get install -y gawk &&      
    chmod +x /tini
HEALTHCHECK --interval=10s --timeout=2s --retries=2 CMD grep -q OK /tmp/status || exit 1
ENTRYPOINT ["/tini", "--"]
# let tini launch the proxy;
CMD ["/root/dctm.sh", "start"]

Let’s build the image with tini:

docker build -f Dockerfile-dctm --tag=dctm:with-tini .
Sending build context to Docker daemon 6.656kB
Step 1/8 : FROM ubuntu:latest
---> 1d9c17228a9e
Step 2/8 : COPY dctm.sh /root/.
---> a724637581fe
Step 3/8 : ENV TINI_VERSION v0.18.0
---> Running in b7727fc065e9
Removing intermediate container b7727fc065e9
---> d1e1a17d7255
Step 4/8 : ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
Downloading [==================================================>] 24.06kB/24.06kB
---> 47b1fc9f82c7
Step 5/8 : RUN apt-get update && apt-get install -y gawk && chmod +x /tini
---> Running in 4543b6f627f3
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
...
Step 6/8 : HEALTHCHECK --interval=5s --timeout=2s --retries=1 CMD grep -q OK /tmp/status || exit 1
---> Running in d2025cbde647
Removing intermediate container d2025cbde647
---> a17fd24c4819
Step 7/8 : ENTRYPOINT ["/tini", "--"]
---> Running in ee1e10062f22
Removing intermediate container ee1e10062f22
---> f343d21175d9
Step 8/8 : CMD ["/root/dctm.sh", "start"]
---> Running in 6d41f591e122
Removing intermediate container 6d41f591e122
---> 66541b8c7b37
Successfully built 66541b8c7b37
Successfully tagged dctm:with-tini

Let’s run the image:

docker run -i --name=dctm dctm:with-tini
 
starting a few processes at 2019/04/07 11:55:30
process started with pid 9 and random value 23970936
process started with pid 10 and random value 35538668
process started with pid 11 and random value 12039907
process started with pid 12 and random value 21444522
process started with pid 13 and random value 7681454
5 processes started
...

And let’s see how the container’s processes look like with tini from another terminal:

docker exec -it dctm /bin/bash
ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 11:55 ? 00:00:00 /tini -- /root/dctm.sh start
root 6 1 0 11:55 ? 00:00:00 /bin/bash /root/dctm.sh start
root 9 6 0 11:55 ? 00:00:00 I am number 23970936
root 10 6 0 11:55 ? 00:00:00 I am number 35538668
root 11 6 0 11:55 ? 00:00:00 I am number 12039907
root 12 6 0 11:55 ? 00:00:00 I am number 21444522
root 13 6 0 11:55 ? 00:00:00 I am number 7681454
root 174 0 0 11:55 ? 00:00:00 /bin/bash
root 201 6 0 11:55 ? 00:00:00 sleep 3
root 208 174 0 11:55 ? 00:00:00 ps -ef
...

So tini is really running with PID == 1 and has started the proxy as its child process as expected.
Let’s test the container by sending a few signals:

docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5f745485a907 dctm:with-tini "/tini -- /root/dctm…" 8 seconds ago Up 7 seconds (healthy) dctm
 
# docker kill --signal=SIGINT dctm
shutting down the processes at 2019/04/07 11:59:42
stopping 23970936
stopping 35538668
stopping 12039907
stopping 21444522
stopping 7681454
5 processes stopped
 
status of running processes at 2019/04/07 11:59:42
0 processes found
 
status of running processes at 2019/04/07 11:59:45
0 processes found
 
# docker kill --signal=SIGTERM dctm
shutting down the processes at 2019/04/07 12:00:00
0 processes stopped

and then the container gets stopped. So, the signals are well transmitted to tini’s child process.
If one prefers to use the run’s −−init option instead of modifying the dockerfile and introduce tini as the ENTRYPOINT, it is even better because we will have only one version of the dockerfile to maintain. Here is the invocation and how the processes will look like:

docker run --name=dctm --init dctm
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1d9fa0d98817 dctm "/root/dctm.sh start" 4 seconds ago Up 3 seconds (health: starting) dctm
docker exec dctm /bin/bash -c "ps -ef"
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 12:11 ? 00:00:00 /dev/init -- /root/dctm.sh start
root 6 1 0 12:11 ? 00:00:00 /bin/bash /root/dctm.sh start
root 9 6 0 12:11 ? 00:00:00 I am number 23850948
root 10 6 0 12:11 ? 00:00:00 I am number 19493606
root 11 6 0 12:11 ? 00:00:00 I am number 34535435
root 12 6 0 12:11 ? 00:00:00 I am number 32571187
root 13 6 0 12:11 ? 00:00:00 I am number 35596440
root 116 0 1 12:11 ? 00:00:00 /bin/bash
root 143 6 0 12:11 ? 00:00:00 sleep 3
root 144 116 0 12:11 ? 00:00:00 ps -ef

It looks even better; tini is still there – presumably – but hidden behind /dev/init so the container will be immune to any future change in the default init process.

Adapting dctm.sh for Documentum

Adapting the proxy script to a real Documentum installation with its own central stop/start/status script, let’s name it dctm_stop_start.sh, is easy. The main changes are limited to the func() function; now, it just relays the commands to the script dctm_stop_start.sh:

#!/bin/bash
# launches in the background, or stops or queries the status of, the Documentum dctm_start_stop.sh script;
# Usage:
#   ./dctm.sh stop | start | status
# e.g.:
# $ ./dctm.sh start
# cec - dbi-services - April 2019
#
trap 'help'         SIGURG
trap 'start_all'    SIGPWR
trap 'status_all'   SIGUSR2
trap 'stop_all'     SIGINT SIGABRT
trap 'shutdown_all' SIGHUP SIGQUIT SIGTERM

verb="sleep"
export id_prefix="I am number"

func() {
   cmd="$1"
   case $cmd in
      start)
         ./dctm_start_stop.sh start &
         ;;
      stop)
         ./dctm_start_stop.sh stop &
         ;;
      status)
         ./dctm_start_stop.sh status
         return $?
         ;;
   esac
}

help() {
   echo
   echo "send signal SIGURG for help"
   echo "send signal SIGPWR to start the Documentum processes"
   echo "send signal SIGUSR2 for the list of Documentum started processes"
   echo "send signal SIGINT | SIGABRT to stop all the processes"
   echo "send signal SIGHUP | SIGQUIT | SIGTERM to shutdown the Documentum processes and exit the container"
}

start_all() {
   echo; echo "starting the Documentum processes at $(date +"%Y/%m/%d %H:%M:%S")"
   func start
}

status_all() {
   echo; echo "status of Documentum processes at $(date +"%Y/%m/%d %H:%M:%S")"
   func status
   if [[ $? -eq 0 ]]; then
      printf "status: badn" > /tmp/tmp_status
   else
      printf "status: OKn" > /tmp/tmp_status
   fi
   mv /tmp/tmp_status /tmp/status
}

stop_all() {
   echo; echo "shutting down the Documentum processes at $(date +"%Y/%m/%d %H:%M:%S")"
   func stop
}

shutdown_all() {
   echo; echo "shutting down the container at $(date +"%Y/%m/%d %H:%M:%S")"
   stop_all
   exit 0
}

# -----------
# main;
# -----------

# starts a few dummy processes;
[[ "$1" = "start" ]] && start_all

# make sure the container stays up and waits for signals;
while true; do status_all; sleep 3; done

Here is a skeleton of the script dctm_start_stop.sh:

#!/bin/bash
   cmd="$1"
   case $cmd in
      start)
         # insert here your Documentum installation's start scripts, e.g.
         # /app/dctm/dba/dm_launch_Docbroker
         # /app/dctm/dba/dm_start_testdb
         # /app/dctm/shared/wildfly9.0.1/server/startMethodServer.sh &
         echo "started"
         ;;
      stop)
         # insert here your Documentum installation's stop scripts, e.g.
         # /app/dctm/shared/wildfly9.0.1/server/stopMethodServer.sh
         # /app/dctm/dba/dm_shutdown_testdb
         # /app/dctm/dba/dm_stop_Docbroker
         echo "stopped"
         ;;
      status)
         # insert here your statements to test the Documentum processes's health;
         # e.g. dmqdocbroker -c -p 1489 ....
         # e.g. idql testdb -Udmadmin -Pxxx to try to connect to the docbase;
         # e.g. wget http://localhost:9080/... to test the method server;
         # 0: OK, 1: NOK;
         exit 0
         ;;
   esac

Let’s introduce a slight modification in the dockerfile’s entrypoint clause: instead of having the Documentum processes start at container startup, the container will start with only the proxy running inside. Only upon receiving the signal SIGPWR will the proxy start all the Documentum processes:

ENTRYPOINT ["/root/dctm.sh", ""]

If the light-weight monitoring is in action, the container will be flagged unhealthy but this can be an useful reminder.
Note that the monitoring could be activated or deactivated through a signal as showed in the diff output below:

diff dctm-no-monitoring.sh dctm.sh
21a22
> trap 'status_on_off' SIGCONT
24a26
> bStatus=1
119a122,125
> status_on_off() {
>    (( bStatus = (bStatus + 1) % 2 ))
> }
> 
132c138
    [[ $bStatus -eq 1 ]] && status_all

This is more flexible and better matches the reality.

Shortening docker commands

We have thrown a lot of docker commands at you. If they are used often, their verbosity can be alleviated through aliases, e.g.:

alias di='docker images'
alias dpsas='docker ps -as'
alias dps='docker ps'
alias dstatus='docker kill --signal=SIGUSR2'
alias dterm='docker kill --signal=SIGTERM'
alias dabort='docker kill --signal=SIGABRT'
alias dlogs='docker logs --follow'
alias dstart='docker start'
alias dstarti='docker start -i'
alias dstop="docker stop"
alias drm='docker container rm'

or even bash functions for the most complicated ones (to be appended into e.g. your ~/.bashrc):

function drun {
image="$1"
docker run -i --name=$image $image
}
 
function dkill {
signal=$1
container=$2
docker kill --signal=$signal $container
}
 
function dbuild {
docker build -f Dockerfile-dctm --tag=$1 .
}

The typical sequence for testing the dockerfile Dockerfile-dctm to produce the image dctm and run it as the dctm container is:

dbuild dctm
drm dctm
drun dctm

Much less typing.

Conclusion

At the end of the day, it is no such a big deal that the Documentum CS does not process signals sent to it for it is easy to work around this omission and even go beyond the basic stops and starts. As always, missing features or shortcomings become a source of inspiration and enhancements !
Containerization has lots of advantages but we have noticed that docker’s implementations vary between versions and platforms so some features don’t always work as expected, if at all.
In a future blog, I’ll show a containerization of the out of the box CS that includes signal trapping. In the meantime, live long and don’t despair.