So I started to work on my own suite of tools for it, and recently I decided to release what I’ve done so far. It’s by no means complete, but it’s very useful for me and I thought it would be useful for others. And, with a bit of luck, someone will help me improving it.

So what is it? Arepa (it stands for “Apt REPository Assistant”, but obviously I called it like that after the yummy Venezuelan sandwiches) is a suite of tools that allow you to manage an APT repository. It contains two command-line tools and a web interface, and its main features are:

• Manages the whole process after a package arrives to the upload queue: from approving it to re-building from source to signing the final repository.
• It allows you to “approve” source packages uploaded to some “incoming” directory, via a web interface.
• It only accepts source packages, and those are re-compiled automatically in the configured autobuilders. It can even “cross-compile” for other distributions (treated like binNMUs).
• Far from reinventing (many) wheels, it integrates tools like reprepro, GPG, Rsync, debootstrap and sbuild so you don’t have to learn all about them.

The approval via some web interface was actually sort of the driving force for the project. One of my pet peeves was that there wasn’t an easy way to have an upload queue and easily approve/reject packages with the tools I knew. From what I had seen, the tools were either for “single person” repositories (no approval needed because the package author is the owner of the repository) or full-blown distribution-size tools like dak and such. My use-case, however, is the following:

• You have an installation of Arepa for an entire organisation (say, a whole company or a big department).
• People inside that organisation upload packages to the upload queue (possibly using dput; the point is, the end up in some directory in the machine hosting Arepa).
• Someone (or a small group of people) are the “masters” of the repository, and they’ll have access to the web interface. From time to time they check the web UI, and they’ll approve (or not) the incoming source packages.
• If they’re approved, the source will be added to the repository and it’ll be scheduled for compilation in the appropriate combination(s) of architectures and distributions.
• A cronjob compiles pending packages every hour; when the compilation is successful, they’re added to the repository.
• At this point, the repository hosted by the Arepa installation has the new packages, but you probably want to serve the repository from a different machine. If that’s the case, Arepa can sync the repository to your production machine with a simple command (“arepa sync”).

I imagine that a lot of people have the same need, so I uploaded all the code to CPAN (you can see it with the rest of the contributions by Opera Software). Sadly there’s a silly bug in the released code (I wanted to release ASAP to be able to focus on other things, and I ended up rushing the release), but it has both a workaround and a patch. So, please give it a try if you’re interested and tell me if you would like to contribute. I haven’t released the code in GitHub or similar yet, but I’ll probably do if there’s interest.

The two modules I’ve released so far are:

• Parse::Debian::PackageDesc, a module for parsing both .dsc and .changes files from Debian. This is actually a support module for something bigger that I hope I’ll release soon-ish.
• Migraine, a still somewhat primitive database change manager inspired on the Ruby on Rails migration system.

As I feel that Migraine could be useful to a lot of people, but it’s easy to misunderstand what it really does (unless you already know Rails migrations of course), I’ll elaborate a bit. Imagine that you are developing some application that uses a database. You design the schema, write some SQL file with it, and everybody creates their own databases from that file. Now, as your application evolves, your schema will evolve too. What do you do now to update all databases (every developer installation, testing installations, and don’t forget the production database)? One painful way to do it could be documenting which SQL statements you have to execute in order to have the latest version of the schema, and expect people to apply copying-and-pasting from the documentation. However, it’s messy, confusing, and it needs someone to know both which databases to update and when.

Migraine offers a simpler, more reliable way to keep all your databases up to date. Basically, you write all your changes (“migrations”) in some files in a directory, following a simple version number naming convention (e.g. 001-add_users_table.sql, 002-change_passwd_field_type.sql), and migraine will allow you to keep your databases up to date. In the simplest, most common case, you call migraine with a configuration file specifying which database to upgrade, and it will figure out which migrations are pending to apply, if any, and apply them. The system currently only supports raw SQL, but it should be easy to extend with other types.

In principle, you shouldn’t need to write any Perl code to use migraine (it has a Perl module that you can use to integrate with your Perl programs if you like, but also a command-line tool), so you can use it even in non-Perl projects. Of course, some modern ORMs have their own database migration system, but very often you have to maintain legacy code that doesn’t use any fancy ORM, or you don’t like the migration system provided by the ORM, or you prefer keeping a single system for schema and data migrations… I think in those cases Migraine can help a lot reducing chaos and keeping things under control. Try it out and tell me what you think :-)

In a couple of days I’ll blog again about other contributions to free software I’ve made lately, but this time in the form of Opera widgets…

However, recently I watched a really funny and interesting talk at TED, Are we in control of our own decisions?, by Dan Ariely. In the talk he mentions his book, Predictably Irrational, which funnily enough a friend had already mentioned to me.

Well, I just finished the book and I have to say it was very interesting and eye-opening. It’s interesting how it shows our minds are biased for certain kinds of decisions or behaviour, even though they are often not the best for us. Some of the experiments are truly brilliant and they show totally unexpected (at least before starting reading the book ;-P) outcomes. One of the experiments that got me thinking was this:

Research on stereotypes shows […] that stereotyped people themselves react differently when they are aware of the label that they are forced to wear […] One stereotype of Asian-Americans, for instance, is that they are especially gifted in mathematics and science. A common stereotype of females is that they are weak in mathematics […] In a remarkable experiment, […] asked Asian-American women to take an objective math exam. But first they divided the women into two groups. The women in one group were asked questions related to their gender […] The women in the second group were asked questions related to their race […] The performance of the two groups differed in a way that matched the stereotypes of both women and Asian-Americans. Those who had been reminded that they were women performed worse than those who had been reminded that they were Asian-American.

I can’t stop thinking about the implications this has to working conditions and productivity in different countries, and also to project management.

The point now is that, while working on it, I needed some module to parse command-line options and “subcommands” (like git commit, svn update, etc.). As it’s written in Perl, I had a look at CPAN to see if I could see anything. The most promising module was App::Rad, but it lacked a couple of things that were very important for me: my idea was “declaring” all the possible commands and options and have the module do all the work for me (generating the help pages and the default --help implementation, generate the program help subcommand and so on). App::Rad didn’t have that, and it didn’t seem to me like that was the direction they wanted to go to with the module. But I figured I’d drop the author an e-mail anyway and see if he liked the idea so I could start adding support for all that…

And boy was that a good idea. He replied a couple of days later, and said that they had liked the idea so much that they had implemented it already (that’s why he took a couple of days to reply), and he sent me an example of the new syntax they had introduced and asked if that was what I was thinking. And not only that, but they added me to the list of contributors just for giving the idea! That completely made my day, free software rocks!

In short, the talks were good. Not fantastic on average, but good. In particular, Damian Conway’s Keynote on Thursday morning was really funny, and had food for thought. It was about contexts and the Contextual::Return module (BTW, does anyone know which system he uses for the slides?). Wednesday’s keynote by Larry Wall was about Perl 6, a bit too much into details. It had some interesting ideas about programming language extensibility, but it was a bit too much for a Wednesday morning (without much sleep). Prophet (“a grounded, semirelational, peer to peer replicated, disconnected, versioned, property database with self-healing conflict resolution”) looks really cool, I’ll see if I can have a look soon. Also some ideas about QA and automated testing, to think about, explore, and share with other people.

Many of the lightning talks were very very funny. One of the funniest was the talk about implementing lolcode in Perl6. Really impressive when you think about it, and really funny too. Others, like the Trailer Theory from Adam Kennedy were very funny too.

All in all, we had a really good time, we learned some things, we have some things written down to investigate later, and we met some new people. Yay for the YAPC::Europe!