When you run PostgreSQL workloads in production you must have a backup and restore implementation. Even for development instances, which are like production for the developers, a well-tested backup and restore procedure sometimes must be in place. Community PostgreSQL comes with pg_basebackup to help you with creating a full consistent backup of your PostgreSQL cluster. If you want to be able to do point in time recovery (PITR) you need to archive the WAL segments which can later be used to roll forward from a base backup. WAL segments go to the pg_wal directory, but if you have closer look into this directory you will see more than just the WAL segments in case you have enabled archiving.

IF you look at a freshly initialized PostgreSQL cluster the content of pg_wal looks like this:

postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal
total 16396
drwx------  3 postgres postgres     4096 Jul  4 11:47 .
drwx------ 20 postgres postgres     4096 Jul  4 11:47 ..
-rw-------  1 postgres postgres 16777216 Jul  4 11:52 000000010000000000000001
drwx------  2 postgres postgres     4096 Jul  4 11:47 archive_status

There is one WAL segment and a directory called “archived_status” which is empty right now:

postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/
total 8
drwx------ 2 postgres postgres 4096 Jul  4 11:47 .
drwx------ 3 postgres postgres 4096 Jul  4 11:47 ..

To see what’s going on in this directory we need to enable archiving and tell PostgreSQL what do when a WAL segment needs to be archived:

postgres@debian11:/home/postgres/ [pgdev] mkdir /tmp/archived_wal
postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_mode='on'" postgres
ALTER SYSTEM
postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_command='test ! -f /tmp/archived_wal/%f && cp %p /tmp/archived_wal/%f'" postgres
ALTER SYSTEM

Of course you should not archive to /tmp and you also should not use cp, as this does not guarantee that the archived segment really is written to disk. For the scope of this post it is fine like that.

Changing the archive command can be done online, but enabling or disabling archiving requires a restart:

postgres@debian11:/home/postgres/ [pgdev] pg_ctl restart

Let’s see what happens if we force the current segment to be archived:

postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_switch_wal()" postgres
 pg_switch_wal 
---------------
 0/167AF68
(1 row)
postgres@debian11:/home/postgres/ [pgdev] ls -la /tmp/archived_wal/
total 16392
drwxr-xr-x  2 postgres postgres     4096 Jul  4 12:08 .
drwxrwxrwt 10 root     root         4096 Jul  4 12:07 ..
-rw-------  1 postgres postgres 16777216 Jul  4 12:08 000000010000000000000001

As expected we see the archived segment in the directory we specified in the archive command. In addition PostgreSQL generated a “.done” file in the archive_status directory in pg_wal:

postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/
total 8
drwx------ 2 postgres postgres 4096 Jul  4 12:08 .
drwx------ 3 postgres postgres 4096 Jul  4 12:08 ..
-rw------- 1 postgres postgres    0 Jul  4 12:08 000000010000000000000001.done

So, whenever a segment was successfully archived you get the corresponding “.done” file. When do you see “.ready” files then? The answer is simple: Exactly when the opposite happens: Archiving of a segment failed for whatever reason:

postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_command='/bin/false'" postgres
ALTER SYSTEM
postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_reload_conf()";
 pg_reload_conf 
----------------
 t
(1 row)
postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_switch_wal()" postgres
 pg_switch_wal 
---------------
 0/2000160
(1 row)
postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/
total 8
drwx------ 2 postgres postgres 4096 Jul  4 12:13 .
drwx------ 3 postgres postgres 4096 Jul  4 12:12 ..
-rw------- 1 postgres postgres    0 Jul  4 12:13 000000010000000000000002.ready

This file will be there until archiving succeeds again:

postgres@debian11:/home/postgres/ [pgdev] psql -c "alter system set archive_command='test ! -f /tmp/archived_wal/%f && cp %p /tmp/archived_wal/%f'" postgres
ALTER SYSTEM
postgres@debian11:/home/postgres/ [pgdev] psql -c "select pg_reload_conf()";
 pg_reload_conf 
----------------
 t
(1 row)
postgres@debian11:/home/postgres/ [pgdev] ls -la $PGDATA/pg_wal/archive_status/
total 8
drwx------ 2 postgres postgres 4096 Jul  4 12:15 .
drwx------ 3 postgres postgres 4096 Jul  4 12:12 ..
-rw------- 1 postgres postgres    0 Jul  4 12:13 000000010000000000000002.done

When you see many “.ready” files this can either mean your archive command is currently failing or load on the system is so high, that archiving can not keep up.