Icinga Vagrant Boxes 2.0: OpenStack provider and enhanced scenarios

It’s been a while since the last Vagrant box update and release, so here are the highlights of the past months combined into a new shiny 2.0 release 🙂
 

New provider: OpenStack

In the past week I’ve been playing with OpenStack. Johan was so kind to send a PR nearly 2 years ago, now the NETWAYS NWS cloud is ready for OpenStack and my tests. Small things were needed to adopt the box provisioning with floating IPs. If you are an experienced OpenStack user, you’ll know how to source everything into your environment. Everyone else just can follow the newly written docs 🙂
Huge shout-out to Johan from CSC for his contribution!
(more…)

Monthly Snap April: Vagrant boxes, Logstash & Foreman

Greetings from sunny Upper Austria – I’m enjoying some days back at home, away from lovely Nuremberg. It is getting warmer over here after we’ve successfully welcomed Winter again during Easter 😉
What happened with Icinga in April? Blerim blogged about the new log processor for Logstash and further introduced automated monitoring with Foreman. The webinar was moved to upcoming Thursday, 4.5.2017 16:00-17:00 +02:00.
We’ve also revamped our Vagrant boxes a bit, more details in this blog post. Our fellow community member Carsten takes the Grafana module for Icinga Web 2 to the next level, make sure to check it out! 🙂
Go for a walk outside, and meet a baby cow like our “Icinga”, if you can. Your monitoring keeps you safe to enjoy nature and the hopefully lovely weather at your place. See you soon! 🙂

Vagrant box updates: Elastic Stack, InfluxDB & Icinga Web 2 modules

A while back we’ve written about changes inside our Vagrant box demo environments – and many things happened ever since.
There are a couple of new Icinga Web 2 modules directly integrated into the Vagrant boxes (Director, Grafana, Cube, Globe). In terms of metrics and event collecting we’ve integrated Elastic Stack with Icingabeat and also InfluxDB with Grafana. We are happy to release v1.3.0 today.
(more…)

vim, Puppet, Vagrant and Icinga 2 syntax highlighting

vim is my preferred editor after all. Even I hack Icinga 2 code with vim way faster than any other interface. vim requires some configuration love of course.
Many distributions do not install vim or set it as default editor. In the Git training sessions on Debian/Ubuntu we’ll have a lab for making vim the default for example. Inside the CentOS base box used for the Icinga 2 Vagrant boxes I’ll also ensure via puppet manifest to install the vim package.
Now for the configuration bits – one keeps a local $HOME/.vimrc with own customizations. That works fine for your own workstation but what if you want to automate such setups and distribute a custom vimrc config on all your servers? You could keep just a file and sync that using Puppet manifests of course. But what about the mentioned default editor settings? And you do not want to break anything with a syntax error?
I’ve found this puppet-vim module pretty nifty and helpful. The most recent commit inside the Vagrant boxes uses it already.

class { 'vim':
  opt_bg_shading => 'light',
}

There are many other options available such as setting it as default editor on Debian. To mention some more:

  • opt_nocompatible
  • opt_bg_shading – I prefer a light background, also easier on a beamer for an Icinga Camp demo
  • opt_lastposition – jump to the last position when opening a file. I definitely want that everywhere.
  • opt_syntax – enable syntax highlighting. On older systems or Debian this is disabled by default.
  • opt_misc – sets some interesting default values for incremental search, etc.:
    [‘hlsearch’,’showcmd’,’showmatch’,’ignorecase’,’smartcase’,’incsearch’,’autowrite’,’hidden’]

In case you ask – why do you care so much about vim? I wanted to integrate the new Icinga 2 vim syntax highlighting packages into the Vagrant boxes. Just install “vim-icinga2” (currently snapshot packages, available with 2.5.0) and you’re ready to go. This is also part of our Icinga 2 training sessions helping you to understand keywords and object definitions far more easy.

  package { 'vim-icinga2':
    ensure => 'latest',
    require => [ Class['icinga_rpm'], Class['vim'] ],
    alias => 'vim-icinga2'
  }

Enjoy vim everywhere 🙂
vagrant_icinga2_vim_syntax_highlighting

Awesome Dashing dashboards with Icinga 2

vagrant_dashingWe at NETWAYS are using Dashing on our office dashboards already. This blog post solely targets integrating yet another new API providing data – the Icinga 2 REST API introduced in v2.4.
The following instructions were taken from the existing Vagrant boxes and their puppet manifests to allow faster installation. Doing it manually shouldn’t be an issue though 😉

Requirements

Ensure that the following packages are installed, example for RHEL 7 with EPEL enabled:

package { [ 'rubygems', 'rubygem-bundler', 'ruby-devel', 'openssl', 'gcc-c++', 'make', 'nodejs' ]:
  ensure => 'installed',
  require => Class['epel']
}

Furthermore put a specific /etc/gemrc file which disables installing the documentation for gems – this can take fairly long and is not required by default. Especially not when provisioning a Vagrant box or a Docker container.

dashing-icinga2

I’ve created that project as demo for Icinga Camp Portland with the help of the existing Icinga 1.x dashing scripts from Markus, and a new job for fetching the Icinga 2 status data from its REST API.
Clone the git repository somewhere applicable. You don’t need any webserver for it, Dashing uses Thin to run a simple webserver on its own.

vcsrepo { '/usr/share/dashing-icinga2':
  ensure   => 'present',
  path     => '/usr/share/dashing-icinga2',
  provider => 'git',
  revision => 'master',
  source   => 'https://github.com/Icinga/dashing-icinga2.git',
  force    => true,
  require  => Package['git']
}

Install the dashing gem

The installation might take pretty long when it tries to install the gem’s documentation files. Therefore the flags “–no-rdoc” and “–no-ri” ensure that this isn’t done and only the dashing gem and its dependencies are installed into the system.

exec { 'dashing-install':
  path => '/bin:/usr/bin:/sbin:/usr/sbin',
  command => "gem install --no-rdoc --no-ri dashing",
  timeout => 1800
}

Install the gems for dashing-icinga2

Next to the dashing application itself the project requires additional gems, such as a rest client for communicating with the Icinga 2 REST API (check the Gemfile for details). Additionally the bundled gems are not installed into the system’s library but locally into the dashing-icinga2 git clone underneath the “binpaths” directory (this is to prevent conflicts with rubygem packages in the first place).

exec { 'dashing-bundle-install':
  path => '/bin:/usr/bin:/sbin:/usr/sbin',
  command => "cd /usr/share/dashing-icinga2 && bundle install --path binpaths", # use binpaths to prevent 'ruby bundler: command not found: thin'
  timeout => 1800
}

Dashing startup script

Put a small startup script somewhere executable to (re)start the Dashing application.

file { 'restart-dashing':
  name => '/usr/local/bin/restart-dashing',
  owner => root,
  group => root,
  mode => '0755',
  source => "puppet:////vagrant/files/usr/local/bin/restart-dashing",
}

Dashing runs as Thin process which puts its pid into the local tree. It is merely all about killing the process, removing the pid and then starting dashing again. “-d” puts the process into daemonize mode (not foreground) as well as “-p 8005” tells the application where to listen for browsers connecting to. Adjust that for your needs 🙂

#!/bin/bash
cd /usr/share/dashing-icinga2
kill -9 $(cat tmp/pids/thin.pid)
rm -f tmp/pids/thin.pid
/usr/local/bin/dashing start -d -p 8005

Now run Dashing.

exec { 'dashing-start':
  path => '/bin:/usr/bin:/sbin:/usr/sbin',
  command => "/usr/local/bin/restart-dashing",
  require => Service['icinga2'],
}

Configure the Icinga 2 API

The dashing job script just requires read-only access to the /v1/status endpoint. Being lazy I’ve just enabled everything but you should consider limited access 🙂

object ApiUser "dashing" {
  password = "icinga2ondashingr0xx"
  client_cn = NodeName
  permissions = [ "*" ]
}

Configure the Dashing job

There’s a bug in Dashing where job scripts ignore the settings from the config.ru file so there is no other way than to put the Icinga 2 REST API credentials and PKI paths directly into the jobs/icinga2.rb file.

$node_name = Socket.gethostbyname(Socket.gethostname).first
if defined? settings.icinga2_api_nodename
  node_name = settings.icinga2_api_nodename
end
#$api_url_base = "https://192.168.99.100:4665"
$api_url_base = "https://localhost:5665"
if defined? settings.icinga2_api_url
  api_url_base = settings.icinga2_api_url
end
$api_username = "dashing"
if defined? settings.icinga2_api_username
  api_username = settings.icinga2_api_username
end
$api_password = "icinga2ondashingr0xx"
if defined? settings.icinga2_api_password
  api_password = settings.icinga2_api_password
end

Modifications?

You really should know your HTML and Ruby foo before starting to modify the dashboards. The main widget used inside the dashboards/icinga2.erb file is “Simplemon” defined as data-view attribute. It is already provided inside the dashing-icinga2 repository. data-row and data-col define the location on the dashboard matrix.

    <li data-row="2" data-col="2" data-sizex="1" data-sizey="1">
      <div data-id="icinga-host-down" data-view="Simplemon" data-title="Hosts Down"></div>
    </li>

The important part is the data-id attribute – that’s the value coming from the icinga2 job defined in jobs/icinga2.erb.
The job update interval is set to 1 second in jobs/icinga2.erb:

SCHEDULER.every '1s' do

Connecting to the Icinga 2 REST API, fetching the status data as JSON and then iterating over these dictionaries is pretty straight forward. Additional programming examples can be found inside the Icinga 2 documentation.
Take the “hosts down” example from above:

hosts_down = status["num_hosts_down"].to_int

Now send the event to dashing by calling the send_event function providing the previosuly extracted value and the demanded color.

  send_event('icinga-host-down', {
   value: hosts_down.to_s,
   color: 'red' })

In case you’re wondering which values are fetched, let dashing run in foreground and print the “status” dictionary to get an idea about possible keys and values. Or query the Icinga 2 REST API with your own client first.

More?

You can play around with an already pre-installed environment inside the icinga2x Vagrant box and if you’re interested in an automated setup, check the puppet provisioner manifest.