As you might have seen in my previous blog posts, I am a real fan of everything that can help automate because of two main reasons:

  1. It saves some time for others non-repetitive tasks
  2. It avoids any error that could occur by manual steps

Ansible triggered my attention. Despite a few LinkedIn training I followed, I did not feel comfortable to develop my own playbooks.
That changed few weeks back, when I attended the Ansible workshop on one day with Jérome Witt.
This one gave me the last push to start writing useful playbooks for my day to day activities.

Context

As Service Delivery Manager (SDM), we are preparing VM to be installed as Kubernetes nodes or Documentum servers (WebServer, FullText and Content Server). Previously, we were running a prerequisite script manually which generated a report of what is conform or not on each server one by one which implies:

  1. Copy of script
  2. Execute it
  3. Collect result

We are not allowed to fix most of the non-conformity ourselves, but we have to request the responsible team to have it done for us (Local users, Linux settings). Then, we can rerun the script to check if it was correctly implemented.
We also have a script to test that servers are whitelisted to the SMTP host by sending a test email. We will start with this one in the next chapter.
Also because of the growing Kubernetes environment (there is batch of 10 VMs to prepare), we must find a way to automated these two tasks (prerequisite and SMTP).

Directory structure

Here is the view of the ansible tree:

├── ansible.cfg
├── appdir.yml
├── inventories
│ ├── DEV.yml
│ └── TEST.yml
├── out
│ ├── DEV
│ │ ├── Dserver1-K8s-issues.csv
│ │ ...
│ │ └── Dserver10-K8s-issues.csv
│ └── TEST
│ ├── Tserver1-K8s-issues.csv
│ ...
│ └── Tserver10-K8s-issues.csv
├── prereqsK8s.yml
├── scripts
│ ├── check_prerequisites_rhel7.sh
│ └── testmail.sh
└── testmail.yml

To avoid long page, I summarized repetitions by “…”.

Test Mail

To make it consistent with past implementations, I decided to keep the existing script for email testing. I will proceed as listed above: copy of script, execute it. If testmail.sh script does not return ‘Message accepted for delivery’, playbook will raise an error for the concerned host. If it is just “update”, it will be considered successful. And here is the playbook:

---
- name: SMTP Testing
  hosts: all
  gather_facts: no
  tasks:
    - name: Copy testmail.sh
      ansible.builtin.copy:
        src: scripts/testmail.sh
        dest: testmail.sh
        mode: '0700'
    - name: Run testmail.sh
      shell: ./testmail.sh | grep 'Message accepted for delivery'
      register: output

Example of outputs for command ‘ansible-playbook -i inventories/DEV.yml testmail.yml’:

PLAY [SMTP Testing] **********************************************************************************************************************************************************************************

TASK [Copy testmail.sh] ******************************************************************************************************************************************************************************
changed: [Dserver1]
...
changed: [Dserver10]

TASK [Run testmail.sh] *******************************************************************************************************************************************************************************
changed: [Dserver1]
...
changed: [Dserver10]

PLAY RECAP *******************************************************************************************************************************************************************************************
Dserver1  : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
...
Dserver10 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

As per idempotence, if I run script second time, task [Copy testmail.sh] will be ok, but not changed.
[Run testmail.sh] will always run and will always have changed state in PLAY RECAP.

Prerequisites

For same reason as SMTP, we will keep the current prerequisite check bash script. Also because, it will be faster to have valid playbook running. The additional step, compared to email testing, will be to download script result from distant servers to local for easier analyzes. check_prerequisites_rhel7.sh need to be run as root, that why I enabled “become” option. Here is the playbook:

---
- name: K8s workers prereqs checks
  hosts: all
  gather_facts: no
  tasks:
    - name: Copy check_prerequisites_rhel7.sh
      ansible.builtin.copy:
        src: scripts/check_prerequisites_rhel7.sh
        dest: check_prerequisites_rhel7.sh
        mode: '0707'
    - name: Run check_prerequisites_rhel7.sh
      shell: "./check_prerequisites_rhel7.sh K8s"
      register: output
      become: yes
      environment:
        PATH: '/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin'
    - name: Get result files
      fetch:
        src: "{{ inventory_hostname | lower }}-K8s-issues.csv"
        dest: "out/{{ inventory_file | basename | splitext | first }}/"
        flat: yes

Similarly as testmail, I simply need to call the playbook with the command “ansible-playbook -i inventories/DEV.yml prereqsK8s.yml”:

PLAY [K8s workers prereqs checks] ********************************************************************************************************************************************************************

TASK [Copy check_prerequisites_rhel7.sh] *************************************************************************************************************************************************************
ok: [Dserver1]
...
ok: [Dserver10]

TASK [Run check_prerequisites_rhel7.sh] **************************************************************************************************************************************************************
...
changed: [Dserver10]
...
changed: [Dserver1]
...

TASK [Get result files] ******************************************************************************************************************************************************************************
ok: [Dserver10]
...
ok: [Dserver1]
...

PLAY RECAP *******************************************************************************************************************************************************************************************
Dserver1             : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
Dserver10            : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Then, I have to look into the local copy of the results in out/dev/ directory.
From the Ansible log, I can already deduce interesting facts:

  1. First task (COPY) will be executed only at first run of the playbook, unless script is updated
  2. Second task (RUN) will be executed every time I run the playbook, thus changed will be 1 or more
  3. Third task (GET) will be executed only if result of bash script is different from previous playbook run

Simple Change

A simple prerequisite is to have /app and /appdocker owned by docker with proper permissions set. The playbook for this is:

---
- name: Create /app and /app/docker directories
  hosts: all
  gather_facts: no
  become: yes
  tasks:
    - name: Create directory {{ item }}
      ansible.builtin.file:
        path: "{{ item }}"
        state: directory
        owner: docker
        group: docker
        mode: '750'
      loop:
        - "/app"
        - "/app/docker"

As highlighted, I only have to say that directory must exist, the owner and group as well as permissions. To avoid duplication of the task for /app/docker folder, I am using the loop statement. To run the playbook, I execute ansible-playbook -i inventories/TEST.yml appdir.yml:

PLAY [Create /app and /app/docker directories] ***************************************************************************************************************

TASK [Create directory {{ item }}] ***************************************************************************************************************************
changed: [Dserver1]  => (item=/app)
...
changed: [Dserver1]  => (item=/app/docker)
...
changed: [Dserver10] => (item=/app)
...
changed: [Dserver10] => (item=/app/docker)

PLAY RECAP ***************************************************************************************************************************************************
Dserver1  : ok=0 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
...
Dserver10 : ok=0 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

A second run will have this RECAP which means nothing was changed (changed=0) as it was conform:

PLAY RECAP ***************************************************************************************************************************************************
Dserver1  : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
...
Dserver10 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Next Steps

As possible improvements, I could think of converting each checks in preprerquisites script to an Ansible task but keeping in mind that most of the checks must not be implemented by the playbook.