Icinga reaches Debian

It’s been a while since Christoph Maser joined Team Icinga sharing his knowledge about creating RPMs. Those packages can be found in RPMForge 🙂
There were a lot of questions about getting Debian packages for Icinga and finally, we are happy to welcome Alexander Wirt onto Team Icinga!
He is Debian packager for Nagios and now Icinga and did a really great job to bring fresh Icinga Debian packages to the upstream.
Currently, Debian lenny, sid/squeeze and Ubuntu Karmic are supported. They can be found here – please check it out and tell us about it!
Take a look at README.Debian after installing IDOUtils and make sure to enable the Event Broker Module in icinga.cfg – patching configs during package install is against packaging policy. But again, we are already working on a satisfying solution for that – check #162.
Icinga’s journey is not ending – are you working on BSD ports or any other applicable operating systems repository? Then please contact us and we will make sure to enlighten the path together for Icinga 🙂
Update 2010-04-09: Icinga got accepted in Debian sidhttp://packages.debian.org/sid/icinga – fire up apt and enjoy =)

Icinga IDOUtils – More Improvements Part III

One last shot this time for upcoming Icinga 1.0.1 and IDOUtils:
After getting several core patches into the master and also fixing duplicated service/hoststatus updates being sent to the neb module (thanks to Matthieu Kermagoret) there will be more improvements for IDOUtils.
Since the threaded housekeeper is doing fine, it is possible to periodically clean more tables. By popular demand, the following options have been added to ido2db.cfg

They can be used for your likings, by default they are not set.
If you want to help us test for the upcoming release, you are very welcome to do so!
To help you with GIT, we now have a quite detailed tutorial how to use GIT based on Icinga in our Developer Wiki =)

Icinga IDOUtils – More Improvements Part II

As mentioned in the last post, there are other improvements for Icinga and IDOUtils.
This time, I want to give you a deeper look onto database performance and the housekeeping stuff.
As you might know, selecting, updating or even deleting a row from a table heavily depends on the row count. If table size grows bigger e.g. in the historical tables from IDOUtils, those queries will be slower and hold back the main process. Current approach of IDOUtils is one forked child of ido2db for one idomod connection – working sequentially on the gotten data.
So even one select taking longer will slow down the data processing and worst case the socket will get blocking and idomod complains about writing to data sink.
But how to resolve those issues?
First of all there were several approaches originally found in mysql-mods.sql – setting indexes on table columns which are being used within the WHERE clause. Regarding the fact that ido2db is not just an insert application, but also deletes historical data on demand (table trimming options), selects objects for caching and furthermore updates existing rows (service/hoststatus e.g.) we decided to apply most useful indexes on the table creation statements. It does slow down an insert a bit, but the overall benefit is much bigger than that 🙂
Also the upcoming Icinga Web benefits from that – e.g. the logentries tables select performs a lot faster when using the API and a RDBMS.
But that’s not all – indexes are only one approach of improvement. In the last few months, Hendrik, Christoph and myself discussed a lot about the periodic housekeeping. The basic approach was to remove housekeeping function from the main data processing. Simply because historical deletes on large tables will take even longer and prevent new data being written to the database.
There have been discussions about a cronjob and seperated forked processes for housekeeping, but we wanted something within ido2db and simple to use. So Hendrik came up with the idea to create an own thread within each ido2db child which runs completely seperated from the main data processing flow – the so-called threaded housekeeper.
The thread just waits for the appropriate instance getting connected and then performs the periodic housekeeping – independant from the main flow. And it does not interfere with the normal data processing. So to speak it resolves a big performance issue within IDOUtils.
Basically, this is the way it performs:

  • sleep a while after creation and intialization
  • idle wait for database connection and connected instance from main process
  • perform periodic maintenance not interferring with main process
  • will be terminated when ido2db shuts down

Best thing so far – it has been implemented and tested and improved quite a while. Mostly done in our own git branches, but the final solution is within current git master and will be one of the outstanding new features for Icinga IDOUtils in the upcoming Icinga 1.0.1 release.
Stay tuned for more updates!
… and prepare for Icinga 1.0.1! =)

Icinga development visualized by Gource

Hi there,
Icinga and the fork happened not that long ago but during this period of time a lot of nice things happened.
Providing Icinga Core with integrated IDOUtils supporting MySQL/Postgres/Oracle, fresh docbook format and therefore enhanced documentation, a completely new Icinga API based on IDOUtils and providing data for the new upcoming Icinga Web. Also lots of other improvements and enhancements.
Writing a historical overview would get boring soon. So we decided to catch up on another Idea: gource.
It’s a small program fetching all commits within our git repositories (core, doc, api, web) and presenting the timeline and changes using rendered pictures.
But that’s not all, it is possible to convert that to nice looking movies.
But there is so much to tell…
Not this time!
Just relax and watch 🙂
Icinga Core

Icinga Doc

Icinga API

Icinga Web

Fixes for Icinga IDOUtils MySQL

Hi there,
just wanted to give you some updates regarding several fixes for Icinga IDOUtils. There were reports about doubled rows within several tables, where data only gets inserted and not updated. During my analysis it came up that there are several mistaken unique constraint definitions within the table creation for MySQL.
The unique constraint makes sure that if an INSERT will try to insert updated data, that this will create an internal exception which is caught within the ON DUPLICATE KEY clause. If caught, an UPDATE will be issued and everything is fine.
Regarding the table servicecheck, this was missing. So the start time of the servicecheck was inserted to the database, and when the servicecheck was complete, the end time also was inserted as own row into the database. Kind of useless 2 rows isn’t it? 😉
Today i did some further investigations on that since this happened with systemcommands too. It came up that tables timedevents and timedeventqueue had that “feature” too.
This is really bad because there are lots of those queries issued and data will grow fast. Not this time because there has been another modification to idomod.cfg – the data processing options haven been modified to ignore timedevents by default. It will improve IDOUtils a bit, but your feedback is as always very welcome!
Back to topic – those missing unique constraints have been added to actual GIT Master (fixing #173 and #181 – check for analysis and comparison) so make sure you get the latest and the greatest! 🙂
For those who are using Postgresql or Oracle – I have implemented and debugged them in deep. And they have own WHERE clauses for UPDATE – so no worries about that, everything is fine! =)