Icon Rufen Sie uns an
+49 441.309197-69 +49 441.309197-69
 
EN

Integrating monitoring with Request Tracker - JSON-RT-API

Posted by Felix Kronlage on Monday, June 29, 2015

Through a posting on the icinga-users mailinglist I noticed that we have a small piece of software that we've been using for a while, that might be useful for others as well. At one of our hackathons Daniel and Bernd wrote a JSON API to interact with Request Tracker command line interface. Request Tracker has a REST API but we wanted to have something very simple that we can call from our monitoring (and possibly other components).

Since someone asked on the icinga-users list about integrating icinga with request tracker, we're publishing the JSON API on github. The API is written in Ruby and meant to be compiled into a jar with JRuby. This makes the deployment very simple. The code makes use of Puma to offer its API via HTTP. Likely the API is called from management networks and not exposed to the wild, still it might be wise to protect it with SSL/TLS.

On the monitoring side (I will stick to describing the icinga case) we handle this via a specific contact. We used to do this via the global event handler, but that way scheduled downtimes are not honored (since the global event handler always triggers).

define contact {
        use             bytemine-rt-contact
        contact_name    rt-api
        alias           RT API
        contact_groups  bytemine.email
}

This contact is defined as follows with the specific commands:

define contact {
        register                        0
        name                            bytemine-rt-contact
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options    w,u,c,r
        host_notification_options       d,u,r
        service_notification_commands   service-notify-by-rt-api
        host_notification_commands      host-notify-by-rt-api
}

The commands look like this - the notify_rt.sh script is bundled with the code on GitHub and originates from here:

http://iambenjaminlong.com/software-projects/notify_rt/. That script wraps around the Request Tracker CLI interface - we adopted it to make use of our JSON API instead.

define command {
        command_name    service-notify-by-rt-api
        command_line    /etc/icinga/bin/notify_rt.sh --NOTIFICATION_METHOD rt_service --RTURL https://json-rt-api.example.com/icinga/service_notification --SERVICEPROBLEMID $SERVICEPROBLEMID$ --LASTSERVICEPROBLEMID $LASTSERVICEPROBLEMID$ --SERVICE_STATE_TYPE $SERVICESTATETYPE$ --HOSTNAME $HOSTNAME$ --SERVICEDESC '$SERVICEDESC$' --SERVICESTATE $SERVICESTATE$ --SERVICEOUTPUT='$SERVICEOUTPUT$' --RTAUTH 'rt-api:secret_password'
}

define command {
        command_name    host-notify-by-rt-api
        command_line    /etc/icinga/bin/notify_rt.sh --NOTIFICATION_METHOD rt_host --RTURL https://json-rt-api.example.com/icinga/host_notification --HOSTNAME "$HOSTNAME$" --HOSTSTATE $HOSTSTATE$ --HOST_STATE_TYPE $HOSTSTATETYPE$ --HOSTPROBLEMID $HOSTPROBLEMID$ --LASTHOSTPROBLEMID $LASTHOSTPROBLEMID$ --HOSTOUTPUT="$HOSTOUTPUT$" --RTAUTH "rt-api:secret_password"
}

On the Request Tracker side the jar-file needs to be deployed and hooked into system startup. With the code we ship an example for an upstart config file. Basic steps to build the jar on a system equipped with JRuby:

$ gem install bundler
$ bundle install
$ rake jar

This assumes that your system fulfills the specs defined in the Gemfile. If you don't want to build from source, there is a jar file (sha256: f34d198d0f0b0c15d55752c3f68640c68238f33a65d64f0e89f563dcd2100bbb) available for download. The json-rt-api service itself needs a small configfile, again: an example is shipped with the code.

# RT-API example configuration.
---
bin: "/usr/bin/rt"
url: "https://rt.example.com/"
autoresolve: true
default_queue: "icinga"
queues:
  customerA: "customerA-support"
username: ""
password: ""
host: '127.0.0.1'
port: 3000

As you can see this is fairly self-explanatory. This configuration assumes that you put nginx, apache or another webserver in front of this service, since it will only listen on port 3000 on localhost.

The username/password is a user that needs to be configured within the Request Tracker.