Using Command Endpoint to execute Checks on Agents

Getting started using Icinga in a single node setup can already be quite a challenge for new users. Configuring checks on multiple nodes can seem at lot harder. With this blog post I will show you how easy it is to configure hosts and services in such an environment. In this example our cluster will consist of one master and two agents and we will use the command endpoint method to execute checks on our agents.

Prerequisites

– Basic Icinga 2 knowledge – already configured a single node setup with some hosts and services
– Icinga 2 installed on all three nodes and configured into a cluster (documentation on that topic)

Hosts

First we’ll configure one host for every agent endpoint, on which we will later apply services.

icinga@master-1:/$ cat /etc/icinga2/zones.d/master/hosts.conf
object Host "agent-1" {
  check_command = "hostalive"
  address = "192.168.56.101"
  vars.agent_endpoint = name // Using the hosts name, because that's also the name of our agent
}
object Host "agent-2" {
  check_command = "hostalive"
  address = "192.168.56.102"
  vars.agent_endpoint = name // Using the hosts name, because that's also the name of our agent
}

We have added vars.agent_endpoint which will later be used to fill the command_endpoint attribute on our services.

Services

After adding our hosts, we can now begin adding services to them. The easiest way of doing that is using service apply rules.

icinga@master-1:/$ cat /etc/icinga2/zones.d/master/services.conf
apply Service "disk" {
  check_command = "disk"
  command_endpoint = host.vars.agent_endpoint
  assign where host.vars.agent_endpoint
}
apply Service "load" {
  check_command = "load"
  command_endpoint = host.vars.agent_endpoint
  assign where host.vars.agent_endpoint
}

Here we’re using the command_endpoint attribute to tell Icinga that this service should be executed on the agent. To make sure we only apply those services to hosts that actually have an endpoint, we assign on host.vars.agent_endpoint which only matches if that variable isn’t empty.
We just added two apply rules to check things locally on our agents. But you of course can still add services that check things from remote by just emitting the `command_endpoint` attribute.

icinga@master-1:/$ cat /etc/icinga2/zones.d/master/services.conf
...
apply Service "ssh" {
  check_command = "ssh"
  assign where host.address
}

After that you should end up with three services per host. Two of those are checked on their agents and one on the master.
If you want to learn more about configuring clusters, take a look at the distributed monitoring chapter in the Icinga 2 documentation.