Icon Rufen Sie uns an
+49 441.309197-69 +49 441.309197-69

the idb!

Posted by Felix Kronlage on Thursday, December 01, 2016

idb logo

Awesome - I've been looking forward for this to happen for quite some time! Our infrastructure database has become an fully open source product and project.

We've been talking about our approach on how to handle the ever growing base of machines, instances, routers, switches and so on few times on this blog already. The journey towards the idb started more than four years ago and last year we've reached the point, where we thought it's more than just an internal tool, it has become a product. A bit more than a year ago, we've announced the availability of the idb (German blogpost), Before and after I gave various talks at conferences about what the idb does and what it will and should do at some point.

The talk at this years open source event in Kiel has been recorded and is available on YouTube (again: German content, sorry ;) At the european Ubucon I gave the talk titled Dr. Restless which is a nice summary of the current status quo of the idb. The English slides are available.

For a couple of weeks now the idb itself and the surrounding adapters are now open source and available through GitHub. We've put it into its own project scope in order not to just "throw it as source over the fence", but to be able to grow and foster a community around it.

the idb - tl;dr

The idb is a set of applications to (more or less) automatically collect data from various tools and APIs to give you a central place to keep your infrastructure in view. The central piece is idb-core which is a Ruby on Rails web application. Personally I see the idb as the application that pulls together all the strings that are already interwoven between applications like puppet, the foreman, vcenter, software and license management (opsi comes to mind), monitoring (icinga, icinga2) and of course the ever growing use of cloud providers. The idb is what enables a team to keep infrastructure coming and going in mind.

Click to enlarge


To make things easier, I want to clarify some terms. In the idb universe there are:

  • modules, which are part of idb-core and extend the functionality, for example the inventory module or the cloud provider addition we're working on currently.
  • adapters, which are the beef and connect to the various systems in your landscape to collect data. Adapters are meant to run on a regular base.
  • importers, which are meant to run once to import data from a certain source

are we there yet?

No, definitely not. While we have a nice set of adapters there are many more planned and in the pipeline. Another big leap will be to have more docs at hand - especially for getting everything up and running. Initially, since it was an internal project, we've bootstrapped everything with puppet, these bits however are tightly coupled with the rest of our stuff at bytemine. Part of the work over the last months was to make sure, the idb can be bootstrapped easily outside of our world.

down the road

The current version is 1.6.0, and once our issues are open you can see what is planned for the next minor and major releases. To give a slight overview, for the 1.6.x line we're working on making software<->machine mapping visible, so that you can search for things like:

  • show me all machines that run Ubuntu 16.04 and OpenSSL and php and WordPress

Furthermore we want to make sure one can see which data came through which adapter, as well as making sure the idb REST JSON API exposes all data. Topics like the UCS integration, i18n and multi tenacy are on the map for the 1.7 and 1.8 line. Personally I would like to see an integration with the icinga2 api, so that you can hover a mouse over an idb item and get the current status within icinga2 for that.

Of course we have several adapters planned, these include but are not limited to:

  • nedi - in order to find machines via nedi and hook them into the idb
  • oxidized - to crawl the configs of your router and switches and attach them to the relating idb objects
  • opsi - to get software and machines from opsi into the idb

packages, support and everything

While you can go ahead and grab the idb from GitHub, we offer ready-to-go packages for either Ubuntu or RHEL/CentOS (and integration into the Univention Corporate Server is planned) as part of a commercial subscription. We will publish more on the commercial side of the idb in the upcoming weeks.

Since this afternoon we've moved all our issue tracking to github as well!