As part of my QA work on several projects, months ago I was looking for a continuous integration server. I looked at several, but most of them seemed really scary judging from the documentation. I finally went for CruiseControl.rb, and I have been really happy with it all this time. It’s a really nice, very simple continuous integration server written in Rails. I had it up and running before I even understood how to install the others I looked.
Even though is a really cool piece of software, I was missing some better test result reporting. It was actually there, but only for Rails projects, and unfortunately we don’t have any Rails (or Ruby, for that matter) projects at work. So, I just had a look at the sources to see if I could hook my own reporting there, and the code turned out to be impressively easy to understand (especially taking into account that it’s a rather non-standard Rails application, as it has builders running as daemons, it doesn’t really use a database, etc).
The result is a patch for CC.rb, already submitted to their BTS, that adds a plugin-based result reporting, that can be extended to understand any kind of testsuite. It’s basically a parser that collects all the test passes and test failures from the testsuite output log.
Also, the other day I had another need, which was even easier to make because it could be implemented as a simple CC.rb notification plugin. It depends on the above patch, and it collects all the bugs in the current build, searches in the history of the project, finds out who made the commits that produced the regressions, and bugs all those people by e-mail, pointing out which failures were supposedly made by them, and which build they started failing (so it’s easier to locate the offending code).
It’s not perfect, and it cannot be, but it’s a nice addition to continuous integration. This notification plugin is not public yet, but it might be in the future (especially if they accept my patch as part of upstream), so stay tuned if you’re interested.