Infrastructure at your Service

Daniel Westermann

Building a PostgreSQL EC2 AMI for testing the latest commits

Testing the latest features or bug fixes in PostgreSQL can easily be done by pulling the latest commits from the git repository and compile PostgreSQL from source code. I am doing that frequently on my local sandbox VM but this becomes limited when I need more CPUs or more memory. This is where the public cloud really shines, as you can easily deploy new VMs, do your tests and get rid of it afterwards with just a view clicks or commands. Bringing up a Debian VM in AWS, which already has the latest development snapshot up and running is something that can be done quite fast.

As a starting point I’ve used the official Debian AMI:

For creating your own customized AMI you could go with a micro or even a nano instance, but if you want to compile fast, more CPUs are better:

Your VPC should already be configured (or just use the default one) and if you want to connect from outside AWS you’ll need a public IP address:

How much storage you’ll need, depends on the tests you want to do. For now it is not really important as you can change that later on, once the AMI is ready:

Tags are always a good idea, especially if you have many instances running:

Of course you should configure the security group to allow SSH access:

Review, provide your key pair and launch the instance:


Once it is running grab the public IP address and connect (the user for Debian is “admin”, not “ec2-user”):

[email protected]:~$ ssh -i /home/dwe/Documents/aws/dwe-key-pair.pem [email protected]
The authenticity of host '3.122.206.167 (3.122.206.167)' can't be established.
ECDSA key fingerprint is SHA256:E/W08OSWhym0k2SVQafxu3rCljqEVg/VC744sz3ilog.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '3.122.206.167' (ECDSA) to the list of known hosts.
Linux ip-10-0-1-35 4.19.0-16-cloud-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

Updating the OS to the latest release is always something I am doing as the first step:

[email protected]:~$ sudo apt update && sudo apt dist-upgrade -y

Install all packages you need for the compilation from source code (and some more in the list below):

[email protected]:~$ sudo apt install libldap2-dev libpython-dev libreadline-dev libssl-dev bison flex libghc-zlib-dev libcrypto++-dev libxml2-dev libxslt1-dev tcl tclcl-dev bzip2 wget screen libpam0g-dev libperl-dev make unzip libpam0g-dev python libsystemd-dev sudo llvm-7 llvm-7-dev clang pkg-config gcc g++ liblz4-dev pkg-config git -y

Configure the OS group and user you’ll use for the PostgreSQL installation:

[email protected]:~$ sudo groupadd postgres
[email protected]:~$ sudo useradd -g postgres -m -s /bin/bash postgres

Clone the PostgreSQL git repository:

[email protected]:~$ sudo su - postgres -c "git clone git://git.postgresql.org/git/postgresql.git"
Cloning into 'postgresql'...
remote: Enumerating objects: 836887, done.
remote: Counting objects: 100% (836887/836887), done.
remote: Compressing objects: 100% (121635/121635), done.
remote: Total 836887 (delta 719772), reused 828577 (delta 712107), pack-reused 0
Receiving objects: 100% (836887/836887), 255.06 MiB | 32.09 MiB/s, done.
Resolving deltas: 100% (719772/719772), done.

This is the small script that will do all the work for us:

[email protected]:~$ sudo su - postgres
[email protected]:~$ cat compile.sh 
#!/bin/bash

rm -rf /home/postgres/pgdata
cd /home/postgres/postgresql
git pull
make distclean
./configure --prefix=/home/postgres/pgdev
make -j4 all
make install
cd contrib
make -j4 install
/home/postgres/pgdev/bin/initdb -D /home/postgres/pgdata
/home/postgres/pgdev/bin/pg_ctl -D /home/postgres/pgdata start -l /dev/null
/home/postgres/pgdev/bin/psql -c "select version()"

Once PostgreSQL is installed create a new AMI from the EC2 instance:

Having the new AMI ready we can easily launch an instance based on the AMI:

In the “Advanced Details” section provide the call to the script:

When the instance started up, you can follow the progress in the cloud init log file:

Happy testing 🙂

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Daniel Westermann
Daniel Westermann

Principal Consultant & Technology Leader Open Infrastructure