InfluxDB

InfluxDB is a powerfull open source time series database (TSDB) developped by InfluxData. It’s a database optimized for time-stamped or time series data. Time series data are simply measurements or events that are tracked.
It is particully interresting for metric tracking like server cpu, ram, application performances and so on.

It is similar to SQL databases but different in many ways. The key here is the time. The database engine is optimized for high ingest and data compression. It is wrtten in Go and compiles in one single binary without any dependencies.
We can use it through a cli but the most interesting part is it’s HTTP API which will allow us to perform actions on the measurements without any third party api.
It can be installed on a single server or in the cloud. It is highly compatible with Telegraf, a time series data collector, but we will discuss it on another blog.

Installation

I did the installation on a CentOS but it can be installed on other linux distributions, OS X and windows. You can download the rpm/zip/tar on the download page or you can install it directly like following:

Add the repo:

cat <<EOF | sudo tee /etc/yum.repos.d/influxdb.repo
[influxdb]
name = InfluxDB Repository - RHEL $releasever
baseurl = https://repos.influxdata.com/rhel/$releasever/$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key
EOF

Ensure the cache is up to date:

sudo yum makecache fast

Install the database, we install curl in order to test the HTTP API:

sudo yum -y install influxdb vim curl

We enable the HTTP API:

sudo vim /etc/influxdb/influxdb.conf
[http]
  enabled = true
  bind-address = ":8086"
  auth-enabled = true
  log-enabled = true
  write-tracing = false
  pprof-enabled = true
  pprof-auth-enabled = true
  debug-pprof-enabled = false
  ping-auth-enabled = true

We start and enable the service:

sudo systemctl start influxdb && sudo systemctl enable influxdb

We need to open the port used by InfluxDB:

sudo firewall-cmd --add-port=8086/tcp --permanent
sudo firewall-cmd --reload

Now we create an admin account:

curl -XPOST "http://localhost:8086/query" --data-urlencode "q=CREATE USER username WITH PASSWORD 'password' WITH ALL PRIVILEGES"

If the query executed correctly you shouldn’t have any response.

You can now test your connection with the cli:

influx -username 'username' -password 'password'

You can list the databases with:

SHOW DATABASES

To create a database do the following:

CREATE DATABASE mymetrics

And to navigate to your created DB:

USE mymetrics

Once you’ll have some entries you will be able to list them like:

SHOW MEASUREMENTS

And then you can execute basic SQL like queries (only when you have measurements):

SELECT * FROM metric_cpu_load

Measurements via Java

Now that we have our DB setup and a user to connect, we will generate measurements with Java. But you can do it with any HTTP request compatible languages. It exists a Java api to discuss with the database directly (here) but I wanted to do it through the HTTP API.

You will need httpclient.jar and httpcore.jar from Apache Commons to use the latest http api. The version I used was:

• httpclient5-5.0.1.jar
• httpcore5-5.0.1.jar

String data = "metric_cpu_load value=0.5"
// We create a default HTTP client
HttpClient httpclient = HttpClients.createDefault();
// We create the URI with all informations
URI uri = new URIBuilder()
        .setScheme("http") // Can be https if ssl is enabled
        .setHost("localhost") // The hostname of the database
        .setPort(8086) // Default InfluxDB port
        .setPath("/write") // We call write to put data into the database
        .setParameter("db", "mymetrics") // The name of the database we created
        .setParameter("u", "username") // The username of the account we created
        .setParameter("p", "password") // The password of the account
        .build();
HttpPost httppost = new HttpPost(uri); // We create a POST request in order to add data
StringEntity entity = new StringEntity(data, ContentType.create("plain/text")); // The metric name and value is set as the body of the post request
httppost.setEntity(entity); // We link the entity to the post request
httpclient.execute(httppost); // Executes the post request, the result is an HttpResponse object, if the query went well, you should have a code 204

InfluxDB stores measurements in “tables”. One measure type is a table, you can specify several columns in the table, but note that the “value” column is mandatory. Here is the explanation of the body of the post request:

“table_name, col1=value1, col2=value2 value=value0 timestamp”

• table_name is the name of the metric, if the table doesn’t exist it will be created, if it exists, a new row will be add to this “table”
• colx are optionnal, they are tags or info that will specify this row, you can even have some rows with theses options while others don’t
• value is the mandatory field, it must be specified or the post request will fail
• timestamp is the timestamp of the row, you can specify it or remove it. If not specified, it will be set to the time when the row was added

Example

Here a simple example

We register several values:

[POST] http://localhost:8086/write?db=mymetrics&u=username&p=password
BODY: "metric_cpu_load,hostname=localhost value=0.1"

[POST] http://localhost:8086/write?db=mymetrics&u=username&p=password
BODY: "metric_cpu_load,hostname=localhost  value=0.4"

[POST] http://localhost:8086/write?db=mymetrics&u=username&p=password
BODY: "metric_cpu_load,hostname=localhost  value=0.8"

[POST] http://localhost:8086/write?db=mymetrics&u=username&p=password
BODY: "metric_cpu_load,hostname=localhost  value=0.5"

Or in CURL we create the DB if it doesn’t exist:

curl -i -XPOST 'http://localhost:8086/query?u=username&p=password' --data-urlencode "q=CREATE DATABASE mymetrics" -> returns code 200 OK
curl -i -XPOST 'http://localhost:8086/write?db=mymetrics&u=username&p=password' --data-binary 'metric_cpu_load,hostname=localhost value=0.1' -> returns 204 NO CONTENT
curl -i -XPOST 'http://localhost:8086/write?db=mymetrics&u=username&p=password' --data-binary 'metric_cpu_load,hostname=localhost value=0.4'
curl -i -XPOST 'http://localhost:8086/write?db=mymetrics&u=username&p=password' --data-binary 'metric_cpu_load,hostname=localhost value=0.8'
curl -i -XPOST 'http://localhost:8086/write?db=mymetrics&u=username&p=password' --data-binary 'metric_cpu_load,hostname=localhost value=0.5'

Now you can see the result from the cli:

influx -username 'username' -password 'password'

Connected to http://localhost:8086 version 1.8.1
InfluxDB shell version: 1.8.1

> SHOW DATABASES
name: databases
name
----
_internal
mymetrics

> USE mymetrics
Using database mymetrics

> SHOW MEASUREMENTS
name: measurements
name
----
metric_cpu_load

> SELECT * FROM metric_cpu_load
name: metric_cpu_load
time                hostname  value
----                --------  -----
1595318363624096578 localhost 0.1
1595318430152497136 localhost 0.4
1595318433209434527 localhost 0.8
1595318436384650878 localhost 0.5

We can also get it from HTTP:

http://localhost:8086/query?db=mymetrics&u=username&p=password&q=select * from metric_cpu_load

{"results":
  [{
    "statement_id":0,
    "series":[{
       "name":"metric_cpu_load",
       "columns":["time","hostname","value"],
       "values":[
           ["2020-07-21T07:59:23.624096578Z","localhost",0.1],
           ["2020-07-21T08:00:30.152497136Z","localhost",0.4],
           ["2020-07-21T08:00:33.209434527Z","localhost",0.8],
           ["2020-07-21T08:00:36.384650878Z","localhost",0.5]]}]}]}

Next steps

And now what? We have a time based database where we can list and add measurements through Java and CURL, what can we do with this?

Next time we will see how to display the results in shiny graphs thanks to Grafana.