Analyzing configuration problems with Icinga 2

by | Jan 31, 2024

Today, I want to showcase an old, but still very useful, tool when it comes to analyzing and debugging an Icinga 2 configuration: the icinga2 object list command. It can be helpful in a variety of situations, for example when you want to verify that a config change has the desired effect, but also for finding out where something is set in the configuration.

The command can simply be run without additional flags. In this case, it prints all objects known to Icinga 2:

# icinga2 object list
Object 'notification' of type 'NotificationComponent':
  % declared in '/etc/icinga2/features-enabled/notification.conf', lines 5:1-5:43
  * __name = "notification"
  * enable_ha = true
  * name = "notification"
  * package = "_etc"
  * source_location
    * first_column = 1
    * first_line = 5
    * last_column = 43
    * last_line = 5
    * path = "/etc/icinga2/features-enabled/notification.conf"
  * templates = [ "notification" ]
    % = modified in '/etc/icinga2/features-enabled/notification.conf', lines 5:1-5:43
  * type = "NotificationComponent"
  * zone = ""

Object 'checker' of type 'CheckerComponent':
  % declared in '/etc/icinga2/features-enabled/checker.conf', lines 5:1-5:33
  * __name = "checker"
  * concurrent_checks = 0
  * name = "checker"
  * package = "_etc"
  * source_location
    * first_column = 1
    * first_line = 5
    * last_column = 33
    * last_line = 5
    * path = "/etc/icinga2/features-enabled/checker.conf"
  * templates = [ "checker" ]
    % = modified in '/etc/icinga2/features-enabled/checker.conf', lines 5:1-5:33
  * type = "CheckerComponent"
  * zone = ""

Object 'icingaadmin' of type 'User':
  % declared in '/etc/icinga2/conf.d/users.conf', lines 6:1-6:25
  * __name = "icingaadmin"
  * display_name = "Icinga 2 Admin"
    % = modified in '/etc/icinga2/conf.d/users.conf', lines 9:3-9:33
  [...]

[...]

Note the lines containing % declared in and % = modified in; these show where something was set, including for individual attributes. So this is a great tool to figure out where some attribute value is coming from, as this will also show if it was set in a template for example.

With a growing number of objects in the configuration, the output will become quite large. If you’re only interested in a specific object, this can easily be filtered with the command line flags -t for type and -n for name. For example, the configuration for a particular service can be printed like this:

# icinga2 object list -t Service -n 'master-1!icinga-health'
Object 'master-1!icinga-health' of type 'Service':
  % declared in '/etc/icinga2/zones.d/global-templates/services.conf', lines 1:0-1:28
  * __name = "master-1!icinga-health"
  * action_url = ""
  * check_command = "icinga"
    % = modified in '/etc/icinga2/zones.d/global-templates/services.conf', lines 3:2-3:25
  * check_interval = 30
    % = modified in '/etc/icinga2/zones.d/global-templates/templates.conf', lines 12:2-12:21
  * check_period = ""
  * check_timeout = null
  * command_endpoint = "master-1"
    % = modified in '/etc/icinga2/zones.d/global-templates/templates.conf', lines 18:2-18:29
  * display_name = "icinga-health"
  * enable_active_checks = true
  [...]

There’s one thing that has changed recently: starting with Icinga 2.14, you may see a warning like the following:

# icinga2 object list -t Host -n master-1
Object 'master-1' of type 'Host':
  [...]
warning/cli: This data is 22 hours, 50 minutes and 4 seconds older than the last Icinga config (re)load. It may be outdated. Consider running 'icinga2 daemon -C --dump-objects' first.

The reason behind this warning is that the icinga2 object list command works by reading a file that was generated beforehand by Icinga 2 when it loaded its configuration. Older versions wrote this file each time the configuration was loaded, even though it’s often never read, wasting resources on generating this file. This was changed in Icinga 2.14 so that this file is only updated when requested by additionally passing the new --dump-objects flag to icinga2 daemon -C.

You May Also Like…

Subscribe to our Newsletter

A monthly digest of the latest Icinga news, releases, articles and community topics.