How to resolve circular dependency in Icinga2

We recently announced the general availability of Icinga 2.14, which most of you might have noticed, and with that in mind, I’d like to show you how you can easily troubleshoot and eliminate some of the dependencies headaches known as the dependency cycle.

What does circular dependency mean?

A circular dependency in Icinga 2 is a mutual relation between two or more hosts and/or services that are dependent on each other, either directly or indirectly, in order to behave properly. Cyclic dependency also occurs when you have a checkable that depends on itself. This kind of problems can be easily caused by different methods. However, they all have one thing in common, which happens only when you define the other one as parent_host/service attribute in two or more checkables in their dependency config. The latest Icinga 2 version detects this when loading the configs and fails immediately with such a mysterious error message.

[2023-07-25 17:14:24 +0000] critical/config: Error: Dependency cycle:
Host 'DSL-router'
-> Dependency 'DSL-router!internet'
-> Host 'DSL-router'
[2023-07-25 17:14:24 +0000] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config

I won’t cover here all the possible scenarios in which this problem might arise. However, here is another error message that is quite  similar to the first one, though slightly different.

[2023-07-26 08:22:21 +0000] critical/config: Error: Dependency cycle:
Host 'DHCP-server'
-> Dependency 'DHCP-server!intranet'
-> Host 'DNS-server'
-> Dependency 'DNS-server!internet'
-> Host 'DSL-router'
-> Dependency 'DSL-router!intranet'
-> Host 'DNS-server'
[2023-07-26 08:22:21 +0000] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config.

How to troubleshoot?

First of all, you need to understand how dependencies work in Icinga 2 in order to solve such a problem effortlessly. When a host depends on another, it means that you don’t want to receive notifications for the child host (the host that depends on another), for instance, while the parent host is still in problem state. This logic works only if the child and the parent host aren’t the same. If we now look at my first example, what could be the root cause? According to the Icinga 2 error logs there is only one host and dependency involved. How could this trigger dependency cycle, you say? Well, let’s analyse the dependency apply rule config.

Here is a very simple dependency where the parent_host_name attribute is statically set to DSL-router and will match to all hosts that have the custom var internet set.

apply Dependency "internet" to Host {
  parent_host_name = "DSL-router"
  disable_checks = true
  disable_notifications = true

  assign where host.vars.internet
}

This config snippet seems at first glance, quite normal. However, the problem here is, that my DSL-router host has set this custom var as well. Consequently, this lead to a host being dependent on itself. The simplest solution to this is to remove the custom variable from that host. However, you can adjust your assign where condition to never match the statically defined host as well.

assign where host.name != "DSL-router" && host.vars.internet

In the second example from above are involved three hosts and dependencies, but the problem is still the same. I have two dependencies in which each defines the other host as parent and whose where condition applies to the respective parent host as well.

apply Dependency "internet" to Host {
  parent_host_name = "DSL-router"
  assign where host.vars.intranet
}

apply Dependency "intranet" to Host {
  parent_host_name = "DNS-server"
  assign where host.vars.my_network
}

This is how my host configuration looks like. Keep in mind that only the minimum and relevant information for this example is included.

object Host "DSL-router" {
  address = "192.168.1.0"
  vars.my_network = true
}

object Host "DHCP-server" {
  address = "192.168.1.1"
  vars.my_network = true
}

object Host "DNS-server" {
  address = "8.8.8.8"
  vars.intranet = true
}

Solution!

We have a DSL-router host that acts as a parent for the first dependency, and sets the my_network custom var. We have also a DNS-server host that acts as a parent for the second dependency and sets the custom variable intranet. So, thinking about it for a moment, how would you solve this problem? It’s pretty simple once you understand the purpose of your dependencies in the first place.