Icinga for Windows without an Icinga 2 agent

I’ve already dropped a hint at this topic in a previous post of mine which reflected the history of Icinga on Windows:

Icinga 2.14 will be able to directly speak to the PowerShell daemon REST API without even fork(2) or exec(3) for a subprocess (…). In fact this could make the IfW daemon the new “Icinga agent” for Windows. An Icinga agent without an Icinga agent, so to say. Schrödinger’s agent.

And this time I’m going to prove this concept, since both required components have been released by now: Icinga 2.14 and IfW 1.11. Precisely speaking, an existing Icinga master will run checks remotely on Windows, directly via the IfW REST API – without an intermediate agent.

Provide a certificate

The bridge between an Icinga 2 node and the IfW API, the ifw-api check command, demands its TLS communication to be authenticated by a valid X.509 certificate. Such can be directly issued by Icinga 2 itself and the Icinga CA is the default one ifw-api trusts. That can be created via icinga2 pki new-ca, but most users have likely run icinga2 api setup or even icinga2 node wizard anyway.

Create a key and a CSR

This intermediate step is required if Icinga 2 itself issues a certificate:

icinga2 pki new-cert --cn desktop-abul3so --key /var/lib/icinga2/certs/desktop-abul3so.key --csr /var/lib/icinga2/certs/desktop-abul3so.csr

Create the certificate itself

icinga2 pki sign-csr --csr /var/lib/icinga2/certs/desktop-abul3so.csr --cert /var/lib/icinga2/certs/desktop-abul3so.crt

For security reasons, a such certificate will be valid for 397 days and has to be renewed regularly. I personally prefer fire-and-forget ones, e.g. valid for 10 years (-days 3660):

openssl x509 -req -in /var/lib/icinga2/certs/desktop-abul3so.csr -out /var/lib/icinga2/certs/desktop-abul3so.crt -CA /var/lib/icinga2/ca/ca.crt -CAkey /var/lib/icinga2/ca/ca.key -days 3660 -extensions ext -extfile <(printf '[ext]\nbasicConstraints=critical,CA:FALSE\nsubjectAltName=DNS:desktop-abul3so')

Pack both together

Windows wants certificates as .pfx files. Such can be created via the openssl CLI, but not Icinga 2 itself:

openssl pkcs12 -in /var/lib/icinga2/certs/desktop-abul3so.crt -inkey /var/lib/icinga2/certs/desktop-abul3so.key -export -passout pass: -out /var/lib/icinga2/certs/desktop-abul3so.pfx

Now the resulting file should be copied over to the target Windows machine.

Set up Icinga for Windows

For the following steps open a PowerShell window as administrator.

Set-ExecutionPolicy

For security reasons, the execution of PowerShell scripts is restricted by default. The fire-and-forget method here is just disabling this restriction in order for IfW to run:

Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope LocalMachine

Install Icinga for Windows components

First install IfW itself, but don’t open the management console. It doesn’t provide a wizard for the following steps. Instead, manually add a repository and install the required components from it:

• Install-IcingaComponent -Name plugins
• Install-IcingaComponent -Name service

Set up the REST API

Of course an Icinga 2 node must be able to reach the API from a non-loopback NIC:

Register-IcingaBackgroundDaemon -Command Start-IcingaWindowsRESTApi -Arguments @{ '-Address' = '0.0.0.0' };

Don’t forget to open port 5668 for the same reason:

New-NetFirewallRule -DisplayName IfwApi -Direction Inbound -Protocol TCP -LocalPort 5668 -Action Allow

As already mentioned, the REST API needs a certificate. At best copy it to the usual location:

Copy-ItemSecure -Path C:\Users\Administrator\Downloads\desktop-abul3so.pfx -Destination "$(Get-IcingaFrameworkRootPath)\certificate\icingaforwindows.pfx"

But the API doesn’t allow checks out-of-the-box, so they have to be enabled as well:

Add-IcingaRESTApiCommand -Command 'Invoke-IcingaCheck*' -Endpoint apichecks

Resolve all problems with -ResolveProblems!

Now is a good time to run Test-IcingaForWindows to verify whether everything is alright. Spoiler – it isn’t:

[Failed]: The specified user "NT Authority\NetworkService" is not allowed to run as service
[Failed]: Directory "C:\Program Files\WindowsPowerShell\Modules\icinga-powershell-framework\cache" is not accessible by the Icinga Service User "NT Authority\NetworkService"
\_ Please run the following command to fix this issue: Set-IcingaAcl -Directory 'C:\Program Files\WindowsPowerShell\Modules\icinga-powershell-framework\cache'
[Failed]: Directory "C:\Program Files\WindowsPowerShell\Modules\icinga-powershell-framework\config" is not accessible by the Icinga Service User "NT Authority\NetworkService"
\_ Please run the following command to fix this issue: Set-IcingaAcl -Directory 'C:\Program Files\WindowsPowerShell\Modules\icinga-powershell-framework\config'

Instead of solving all of those problems manually, simply run Test-IcingaForWindows -ResolveProblems. Then verify whether everything is alright now, again via Test-IcingaForWindows. Only these two should remain:

[Failed]: The Icinga for Windows service is currently not running
[Failed]: The Icinga for Windows REST-Api responded with an error on "https://localhost:5668/v1"

A simple Restart-IcingaForWindows should fix them and finally bring the IfW REST API up. Yet another Test-IcingaForWindows run should confirm this.

Generate Icinga 2 check command config

If not already done, run the following command and copy the resulting files, PowerShell_Base.conf and PowerShell_CheckCommands_07-11-2024-09-59-4261.conf, over to the Icinga 2 node – /etc/icinga2/zones.d/global-templates is a good place (for a cluster).

Get-IcingaCheckCommandConfig -IcingaConfig -OutDirectory C:\Users\Administrator\Documents

But modify PowerShell_Base.conf manually – add a new line import "ifw-api" just after import "plugin-check-command". Otherwise Icinga 2 will try to run a local PowerShell and not communicate to any API.

Let the checks begin!

At this point Icinga for Windows is ready to be used for monitoring. Yet another template in /etc/icinga2/zones.d/global-templates should avoid repeating host metadata in a well configured setup:

template Host "ifw-api-remote" {
  vars.ifw_api_host = "$host.address$"
  vars.ifw_api_expected_san = "$host.name$"
}

Precisely speaking, this sets custom vars for the ifw-api check command, assuming the following host config style:

object Host "desktop-abul3so" { // (!) Name matches certificate CN
  import "ifw-api-remote"

  check_command = "hostalive"
  address = "10.27.0.147" // (!) Address is given
}

object Service "Invoke-IcingaCheckCPU" {
  host_name = "desktop-abul3so"
  check_command = "Invoke-IcingaCheckCPU"
}

And that’s all – everything is up and checking:

[OK] CPU Load
\_ [OK] Top 10 Process CPU usage
   \_ [OK] Process System with id 4: 5.00%
   \_ [OK] Process WmiPrvSE with id 1816: 5.00%

Conclusion

As you see, our monitoring solutions are constantly evolving. But our mission is still, as Bernd says: making you happy.

Whether you prefer the “good” old C++ plugins or the shiny new PowerShell ones, whether your employer wants an Icinga 2 agent on the Windows machines or not and whether your company is in Europe or the USA – we and our partners can help you to setup your Windows monitoring as you wish. Don’t hesitate to ask!