As already announced on Twitter and LinkedIn here is the blog post to describe our OpenDB appliance in more detail. I am sure you wonder what this is about so let me explain why we are doing this. What we do see day by day at our customers is that more and more databases get consolidated on to a VMWare deployment. This is not only true for the smaller ones of those but also for the critical, potentially much bigger ones. What makes it complicated, especially for smaller companies that do not necessarily have the knowhow for the specific database, is that you need to apply the best practices not only to the database deployment but also to the operating system and the VMWare deployment. But even if you have this already in place: Do you know how to deploy the PostgeSQL binaries, how to setup a PostgreSQL instance, how to monitor and how to backup and restore all that? Do you know how to do this with MySQL/MariaDB, MongoDB, Cassandra? If your answer to this is no but you need to have a PostgreSQL/MySQL/MariaDB/MongoDB/Cassandra instance ready quite fast then the dbi OpenDB Appliance might be the solution for you. Let’s dig into some details.
A typical use case: You are forced to support an application which is running on a specific database. What do you do? Quickly setup a Linux VM, download the installer, clicking next, next, next and hopefully make the application connect to what you just installed and then cross your fingers and hope that never ever something goes wrong? You laugh? There are deployments out there which got setup in exactly this way. Another option would be to hire someone who is experienced in that area. This will not help you either as you’d at least need two people (because people tend to want to go to holidays from time to time). The next option would be to work together with external consultants which probably will work as long as you work with the right ones. Completely outsourcing the stuff is another option (or even going to the cloud), if you want to do that. With the dbi OpenDB Appliance you’ll get another option: We deliver a fully pre-configured VMWare based virtual machine image which you can easily plug into your existing VMWare landscape. Can that work? Let me explain what you would get:
As said just before you get an image which you can import into your VMWare ESX. I said this image is pre-configured, what does that mean? Well, when you start it up it boots into a CentOS 7.3 x64 Linux operating system. No magic, I know 🙂 Additionally you’ll get four pre-configured disks:
/ 15GB The Linux operating system
/boot 1GB The boot images (kernels)
/u01 50GB All files belonging to the OpenDB itself
All required DMK packages
All source files (PostgreSQL, MariaDB, MongoDB, Cassandra)
The Linux yum repositories
The HOMEs of all product installations
The admin directories for the initialized products
/u02 10GB The data files belonging to the initialized products
/u03 10GB The redo/wal files belonging to the initialized products
/u04 10GB Backups
You are not supposed to touch the root, /boot and /u01 partitions but of course you will be able to resize /u02 to /u04. The 10GB provided initially are just meant as minimum setup. Resize your VMWare disk images (vmdks) and the dbi OpenDB command line utility offers you to resize the file systems as well with just a single call. At this point you probably wonder what the dbi OpenDB command line utility is about. In short this is a wrapper around our various DMK packages. Using one of the various DMK packages you can deploy and monitor databases even today. The command line utility makes use of that and wraps around the various DMKs. The interface is menu driven to make it as easy as possible for you and helps you with initializing the appliance (setting the hostname, network configuration and disk resizing). In addition you can install the products we support and create database instances on top of that without knowing the details. We take care of implementing the best practices in the background (kernel settings, file system layout, initialization parameters, …). But that is not all: We’ll go a step further and implement monitoring, alerting and backup procedures as well. The idea is that you really do not need to take care of such things: It just comes when you setup a product.
To give you an idea you’ll get something like this when you fire up the command line utility:
============================================================================================== = = = = = _ _ _ ___ ___ ___ _ _ _ = = __| | |__(_) / _ \ _ __ ___ _ _ | \| _ ) /_\ _ __ _ __| (_)__ _ _ _ __ ___ = = / _ | '_ \ | | (_) | '_ \/ -_) ' \| |) | _ \ / _ \| '_ \ '_ \ | / _ | ' \/ _/ -_) = = \__,_|_.__/_| \___/| .__/\___|_||_|___/|___/ /_/ \_\ .__/ .__/_|_\__,_|_||_\__\___| = = |_| |_| |_| = = = = = = Please make a selection from the menu below (type 'q' to exit): = = = = 1. Deploy a database home = = 2. List the deployed database homes = = 3. Setup a database instance = = 4. List the deployed database instances = = = = 10. Stop and remove a database instance = = 11. Remove a database home = = = = = = 99. Initialize the appliance = = = = = ============================================================================================== Your input please:
You would start by “Initialize the appliance” to set your preferred host name, to initialize the network and to provide the monitoring credentials. Once done you can go on and start deploying product homes (e.g. PostgreSQL) and instances on top of that. Of course you can deploy multiple instances on the same home and you can install several homes of the same product version.
What do we mean by a “product”? A product is what we support with a specific release of the appliance. Initially this probably will be:
- PostgreSQL 9.6.5
- PostgreSQL 9.5.9
So the menu would offer you something like this for deploying the binaries:
============================================================================================== = = = = = _ _ _ ___ ___ ___ _ _ _ = = __| | |__(_) / _ \ _ __ ___ _ _ | \| _ ) /_\ _ __ _ __| (_)__ _ _ _ __ ___ = = / _ | '_ \ | | (_) | '_ \/ -_) ' \| |) | _ \ / _ \| '_ \ '_ \ | / _ | ' \/ _/ -_) = = \__,_|_.__/_| \___/| .__/\___|_||_|___/|___/ /_/ \_\ .__/ .__/_|_\__,_|_||_\__\___| = = |_| |_| |_| = = = = = = Please make a selection from the menu below (type 'q' to exit, 'b' to go back): = = = = = = 000 - PostgreSQL 9.6.5 = = 001 - PostgreSQL 9.5.9 = = = = = ============================================================================================== Your input please:
Once you have deployed the homes you require you can list them:
============================================================================================== = = = = = _ _ _ ___ ___ ___ _ _ _ = = __| | |__(_) / _ \ _ __ ___ _ _ | \| _ ) /_\ _ __ _ __| (_)__ _ _ _ __ ___ = = / _ | '_ \ | | (_) | '_ \/ -_) ' \| |) | _ \ / _ \| '_ \ '_ \ | / _ | ' \/ _/ -_) = = \__,_|_.__/_| \___/| .__/\___|_||_|___/|___/ /_/ \_\ .__/ .__/_|_\__,_|_||_\__\___| = = |_| |_| |_| = = = = = = Please make a selection from the menu below (type 'q' to exit, 'b' to go back): = = = = = = The following homes are available for deploying instances on: = = = = = = pg965:/u01/app/opendb/product/PG96/db_5/:dummy:9999:D = = PG959:/u01/app/opendb/product/PG95/db_9/:dummy:9999:D = = PG959_1:/u01/app/opendb/product/PG95/db_9_0:dummy:9999:D = = PG965_1:/u01/app/opendb/product/PG96/db_5_0:dummy:9999:D = = = = = ============================================================================================== Your input please:
Here you can see that you can have multiple homes of the same release (two for PostgreSQL 9.6.5 and two for PostgreSQL 9.5.9 in this case). The path and naming for a home follow our best practices and are generated automatically. Having the homes you can start deploying you instances:
============================================================================================== = = = = = _ _ _ ___ ___ ___ _ _ _ = = __| | |__(_) / _ \ _ __ ___ _ _ | \| _ ) /_\ _ __ _ __| (_)__ _ _ _ __ ___ = = / _ | '_ \ | | (_) | '_ \/ -_) ' \| |) | _ \ / _ \| '_ \ '_ \ | / _ | ' \/ _/ -_) = = \__,_|_.__/_| \___/| .__/\___|_||_|___/|___/ /_/ \_\ .__/ .__/_|_\__,_|_||_\__\___| = = |_| |_| |_| = = = = = = Please make a selection from the menu below (type 'q' to exit, 'b' to go back): = = = = = = Please specify an alias for your new instance = = The alias needs to be at least 4 characters = = The alias needs to be at most 8 characters = = = = = = = ============================================================================================== Your input please: MYINST1
What happens in the background then is that the PostgreSQL cluster is initialized, started and added to the auto start configuration (systemd) so that the instance will properly shutdown when the appliance is stopped and comes up when the appliance is started. Listing the deployed instances is possible, too, of course:
============================================================================================== = = = = = _ _ _ ___ ___ ___ _ _ _ = = __| | |__(_) / _ \ _ __ ___ _ _ | \| _ ) /_\ _ __ _ __| (_)__ _ _ _ __ ___ = = / _ | '_ \ | | (_) | '_ \/ -_) ' \| |) | _ \ / _ \| '_ \ '_ \ | / _ | ' \/ _/ -_) = = \__,_|_.__/_| \___/| .__/\___|_||_|___/|___/ /_/ \_\ .__/ .__/_|_\__,_|_||_\__\___| = = |_| |_| |_| = = = = = = Please make a selection from the menu below (type 'q' to exit, 'b' to go back): = = = = = = The following instances are currently deployed: = = = = = = MYINST1:/u01/app/opendb/product/PG96/db_5/:/u02/opendb/pgdata/MYINST1:5432:Y = = = = = ============================================================================================== Your input please:
The cronjobs for monitoring, alerting and backup have been created as well:
[opendb@opendb ~]$ crontab -l 00 01 * * * /u01/app/opendb/local/dmk/dmk_postgres/bin/dmk-pg-dump.sh -s MYINST1 -t /u04/opendb/pgdata/MYINST1/dumps >/dev/null 2>&1 58 00 * * * /u01/app/opendb/local/dmk/dmk_postgres/bin/dmk-pg-badger-reports.sh -s MYINST1 >/dev/null 2>&1 */10 * * * * /u01/app/opendb/local/dmk/dmk_postgres/bin/dmk-check-postgres.sh -s MYINST1 -m >/dev/null 2>&1
With every new release/update of the appliance we plan to include more products such as MariaDB/MongoDB/Cassandra, provide patch sets for the existing ones and update the Linux operating system. Updates will be delivered as tarballs and the command line utility will take care of the rest, you do not need to worry about that. You can expect updates twice a year.
To visualize this:
/u02 will hold all the files that contain your user data. /u03 is there for redo/wal/binlog where required and /u04 is for holding the backups. This is fixed and must not be changed. Independent of which product you choose to deploy you’ll get a combination of pcp (Performance Co-Pilot) and vector to do real time performance monitoring of the appliance (of course configured automatically).
Alerting will be done by a combination of third party (open source) projects and DMK. The tools we’ll use for PostgreSQL will be check_postgres and pgbadger, for example. For the other products we’ll announce what we will use when it will be included in a future release.
In addition to the VMWare template you can have the appliance also in the Hidora Cloud as a pay as you go service (although that is not fully ready).
If you have any questions just send as an email to: opendb[at]dbi-services[dot]com
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/DWE_web-min-scaled.jpg)