The last posts looked at how the logging system and the pg_stat_statements extension can be used to monitor sql statements in PostgreSQL. This post will introduce pg_activity which is very similar to htop.
There are some dependencies which need to be installed before we can start installing pg_activity. The first one is python. As I am on a redhat based distribution this is quite easy:
[[email protected] ~] yum install -y python
Then we need to install psycopg which is the database adapter for PostgreSQL for the python language (Note: if you have installed PostgreSQL not in the default location edit the setup.cfg script and provide the path to pg_config. Otherwise the install will fail):
[email protected]:/var/tmp/ [dummy] tar -axf psycopg2-2.6.1.tar.gz [email protected]:/var/tmp/ [dummy] cd psycopg2-2.6.1 [email protected]:/var/tmp/psycopg2-2.6.1/ [dummy] python setup.py build [email protected]:/var/tmp/psycopg2-2.6.1/ [dummy] sudo python setup.py install
The next (and last) thing we need to have available is psutils, a python library for querying os statistics:
[email protected]:/var/tmp/ [dummy] tar -axf psutil-3.3.0.tar.gz [email protected]:/var/tmp/ [dummy] cd psutil-3.3.0 [email protected]:/var/tmp/ [dummy] sudo python setup.py install
That’s it. Now we can install pg_activity:
[email protected]:/var/tmp/ [dummy] unzip pg_activity-master.zip [email protected]:/var/tmp/ [dummy] cd pg_activity-master [email protected]:/var/tmp/ [dummy] sudo python setup.py install
Quite easy. Lets see what we can do with it. If you are locally on the server where your PostgreSQL instance runs you can just start pg_activity (I fired a sql statement so that you can see at least one sql in the screenshot):
[email protected]:/home/postgres/ [PG3] pg_activity
There is a nice summary on the top like in top/htop. The different statements which are currently executing are displayed below.
Hitting “h” for help shows you the various options:
The “F1/2/3” switches are very nice when you want to display blocking queries only, running queries only or waiting queries only. Another great features is that you do not need to install pg_activity on the server where PostgreSQL is running. The same connections options as in e.g. psql are there so that you can connect to any remote PostgreSQL instance you have access to:
[email protected]:/home/postgres/ [PG3] pg_activity --help Usage: pg_activity [options] htop like application for PostgreSQL server activity monitoring. Options: --version show program's version number and exit -U USERNAME, --username=USERNAME Database user name (default: "postgres"). -p PORT, --port=PORT Database server port (default: "5432"). -h HOSTNAME, --host=HOSTNAME Database server host or socket directory (default: "localhost"). -d DBNAME, --dbname=DBNAME Database name to connect to (default: "postgres"). -C, --no-color Disable color usage. --blocksize=BLOCKSIZE Filesystem blocksize (default: 4096) --rds Enable support for AWS RDS --help Show this help message and exit. --debug Enable debug mode for traceback tracking. Display Options, you can exclude some columns by using them : --no-database Disable DATABASE. --no-user Disable USER. --no-client Disable CLIENT. --no-cpu Disable CPU%. --no-mem Disable MEM%. --no-read Disable READ/s. --no-write Disable WRITE/s. --no-time Disable TIME+. --no-wait Disable W.
Conclusion: pg_activity is small but very useful tool for monitoring an PostgreSQL instance. In the next posts I’ll look into some more feature rich monitoring solutions that are around for monitoring PostgreSQL instances.
Well about this extension. Actually it’s a very great explanation but the problem I’m facing is the queries are getting refreshed in the shot span unable to read it completely. So what should I do to get all the queries executed as a list or atleast increase the refresh interval
Hi Pavan,
when go to help “h” there are options:
+: increase refresh time (max:3)
“-: decrease refresh time (min:1)”
You can increase the refresh interval to 3 seconds, not more.
Cheers,
Daniel
Hi Daniel,
I’m glad that I came to know about pg_activity by your post. But this is not addressing my database pain area.
I’m using a PostgreSQL 9.1 database version with default checkpoint params. And the wal_level as ‘archive’.
But to my wonder the archive logs are generating in a huge number, say 2000 files a day(31.25GB) from this Jan 1’st 2018.
My main concern is to find all the DML’s which are responsible for this archive log generation, as I am running with serious disk space outage.
How to get all the DML’s who are the culprits for this disk space problem. In other words, kindly help me to resolve this problem…
Thanks in Advance.
Cheers,
Pavan
Hi Pavan,
you can set log_statements=’DML’ to have all the DML statements logged to the logfile. Another way is to look at pg_stat_statements.
Cheers,
Daniel
Thank you so much Daniel. Kudoes
Hi Daniel,
Yes I found many DML’s that are being run against the database. Now what should be the best plan of action to reduce huge WAL archiving. Really it is causing a serious disk problem nearly 1.35GB archive files are being generated per hour and I’m unable to investigate further in a better way.
Please let me know if there is any better approach. Thanks in Advance.
Regards,
Pavan
Hi Daniel,
I’m glad that I came to know about pg_activity by your post. But this is not addressing my database pain area.
I’m using a PostgreSQL 9.1 database version with default checkpoint params. And the wal_level as ‘archive’.
But to my wonder the archive logs are generating in a huge number, say 2000 files a day(31.25GB) from this Jan 1’st 2018.
My main concern is to find all the DML’s which are responsible for this archive log generation, as I am running with serious disk space outage.
How to get all the DML’s who are the culprits for this disk space problem. In other words, kindly help me to resolve this problem…
Thanks in Advance.
Cheers,
Pavan
Hello!
First, change the super user permissions to the admin user, as shown below:
ALTER ROLE admin WITH SUPERUSER;
I opened a console and placed:
sudo su
su postgres
pg_activity -h localhost -U admin -d database
Then, in another console I placed:
sudo su
su postgres
psql -h localhost -U admin -d database
database= # SELECT * FROM person;
But nothing is shown in pg_activity. Please help!