Maybe the testers in his team are doing too good of a job?

Because, you know, if testers are finding enough of “those bugs” (the ones that should be caught and controlled by unit tests and not by testers weeks after the code was originally written), maybe some developers are just not “feeling the pressure” and can’t really get that they should be writing tests for their code. If testers are very good, things just work out fine in the end… sort of. And of course, the problem here is the trailing “sort of”.
I know I’m biased, but in my view there is a ton of bugs that should never be seen by someone that is not the developer itself. Testers should deal with more complex, interesting, user-centred bugs. Non-trivial cases. Suboptimal UIs. Implementation disagreements between developers and stakeholders. That kind of thing. It’s simply a waste of time and resources that testers have to deal with silly, easy-to-avoid bugs on a daily basis. Not to mention that teams shouldn’t have to wait for days or weeks until they find basic bugs via exploratory testing. Or that a lot of those are much, much quicker to test with unit tests than having to create the whole fixture/environment for them to be found with exploratory testing.
My current conclusion is that pushing on the UI/usability side is not only good for the user, but it’s likely to produce better code as it will be, on average, more complex and will have to be better controlled by QA (code review, less ad-hoc design, …) and automated tests. Maybe developers will start hating me for that, but hopefully users will thank me.

The talks themselves went pretty good I think, although I’d have preferred more people attending. I guess it was normal that there were less people than I’m used to, since the Linköping office is much smaller. But anyway. The Debian course went quite well and some people got started packaging stuff almost right away. The other talks were an introduction to automated testing (advocacy and arguments for it, advice, basic examples and small rant about a different kind of QA), which went ok, and an entry-level talk about unit testing in Python (thanks Ask and Batiste for the information and reviewing the slides!), which went very well. I’ll try to get the slides for all the talks available somewhere.

About the city itself, it’s a charming little part of Sweden where:

• Restaurants have insanely different prices for food whether it’s for lunch or dinner. Typical prices for lunch are 80 SEK (around 8 EUR) and typical prices for dinner are around 250 SEK just the main course!
• Restaurants usually serve some Swedish dish for lunch… and I mean every restaurant, meaning all the Greek, Vietnamese, etc. Considering “real” Swedish restaurants are very expensive, you usually go to those foreign cuisine ones when you actually want to eat Swedish food.
• Restaurants typically have some salad (that you have to take yourself) while you wait for the food… and some coffee, tea and cookies (that obviously you have to take yourself) for the end.
• Related to this, restaurants are usually very self-service. I thought service in Norway sucked, but boy was I wrong, at least there is some service. And: there were typically long but pretty-fast-moving queues, and there was this one place where you didn’t even get the food on the table after ordering at the bar; instead, you were given some gadget with some wireless receiver, and when your food was ready it’d beep so you knew you had to go to some special place and fetch your food. Is it really cheaper maintaining some gadgets than hiring a waiter? I guess so.
• The restrictions on the amount of alcohol that can be bought outside the special Government booze stores are even harder than in Norway. You can only buy booze with up to 3.5% alcohol outside “Systembolaget”. Now that is sad. And I was complaining about Norway’s 5%.
• Partly because of that (I assume/hope) the Swedish “cider” you get in Sweden is even sweeter and worse and the Swedish cider you get in Norway.
• We went to this nice student pub… which was literally for students. They actually checked your student id, but each student could bring one non-student along. Once you were “identified” as a non-student-coming-with-a-student, you’d get a stamp on your hand so you wouldn’t have to bring along the student when you ordered again. Also, the place was so very slow it was almost funny. One of the good sides was that they had what I thought it was the only decent Swedish cider… but after checking just now, it seems it’s actually American. Bummer. And the name of it was funny too: “Hardcore Cider”.
• Right before leaving the office on Friday there was a small gathering in the canteen (the “Friday Beer”), where they had a Dreamcast with one of the most awesome games I’ve seen in a long while: The Typing of the Dead, a version of The House of the Dead 2 in which you kill the zombies by typing words that appear on the screen, instead of aiming and shooting with a gun:

That got me thinking. No matter how expert one person is, he can’t take all the right decisions without help and feedback: a second opinion is always useful and can save the team from embarrassing (or, in some cases, fatal) consequences. A second opinion can give perspective or aspects not thought of by the first person.

If you apply this to software development, I can’t help thinking that one of the roles of QA fulfils this need: being experts in the field that provide second opinions and critiques on anything the team decides or produces. And they shouldn’t feel afraid to speak up because… well, it’s their job after all. And while yes, fellow developers could serve as “second opinion” too, having a more or less formal position for a “Quality Assurance Engineer” is helpful for a variety of reasons. First, as I said the chances of being afraid to speak up are much lower, because it’s their job. Second, not producing the result themselves gives some perspective that people having to fight with everyday details can have, but usually don’t; at least not as much. And last but probably important, it’s their job so they can focus on it and they don’t stop doing it because “they have deliveries soon” or because “they don’t have time”.

Finally, there is another blog post, linked from the above, that also supports my vision of QA: Toyota “Stop the Line” mentality. But this one is about processes and taking a step back when something is wrong, trying to find the root cause instead of an immediate solution. Enjoy the blog posts :-)

One chapter that made me think a lot was chapter 7: “Your boss knows more about innovation than you”. It explores the relation between (traditional) management and innovation, and claims that managers can work against innovation if they just try to increase efficiency and keep things under control. In that sense, quality assurance engineers can be like those project managers, so I wondered a lot about my role and my duties with regards to innovation. On the one hand, you do have to control things that are being done and be conservative to a certain extent. On the other hand, innovation is such an important part of an IT company (particularly if it’s Internet-related) that you really don’t want to risk blocking or stifling it.

Fortunately, it also explains how to keep the workplace open to innovation, including things like having toys and “funny” things at the office. It turns out that they’re not there to spoil the employees, but to provide an environment where people feel free to “think different” and are not afraid of new ideas or to say what they think.

All in all, I think it’s a great book. Recommended!

But. I. don’t. test. my. programs.

By hand, that is. And sometimes (usually) the coverage is not enough, and I end up making embarrassing mistakes. It usually happens outside of work, although at work I also have my share. The last one was with the Debian package dhelp, where trying to fix an issue before Lenny is released, I ended up making it even worse. The story goes like this:

There was some problem with the indexing of documents on installation/upgrade (namely, it would take ages for most people upgrading to Lenny, and they would think the upgrade process had hung). So, I go and change the indexing code so it ignores documents on installation/upgrade. Also, as suggested by someone, I created some small example utility to reindex documentation for certain packages. I test installation, upgrades, upgrade of the dhelp package itself, the utility, searching for keywords before and after all that… and everything worked.

Only that I made a typo. A typo that would make all indexing to be ignored (except for the example utility, because it was a bit lower level). And I didn’t realise, because it “only” broke some cronjob, a completely different part of the package. And it happens that the cronjob reindexed everything weekly, to make sure that you had reasonably up-to-date search indices. And it also happens that, given that the documentation reindexing was being ignored on package installation/upgrade, the weekly total reindex process was the only thing that could provide the user with indexed documentation. But I screwed up. Oh well.

Someone filed a bug yesterday, and I fixed more or less right away. But this time I spent a couple of hours thinking of test paths and ways to make it fail, and actually doing all that testing. Thanks to that, I found some potential bug in the example utility, that I fixed just in case. So hopefully everything is fine now, if I can convince the Release Masters to allow the new, less broken update to dhelp to be accepted for Lenny.

I think I need personal QA. Anyone up to the task?

The other news is that finally the Opera QA blog is online, and has the first non-hello-world-article (written by yours truly), “Continuous Integration: Team Testing”. I’m very excited about this, because it’s the first time I’ll participate directly in a company blog, and because the IT world needs more (and better) QA, so hopefully we’ll be able to spread the word and make the world a better place :-D

My current theory is that computer work is just too hard (or tools not advanced enough?), and there is too much pressure and too hard time constraints to allow people to step back and think about the big picture from time to time, to make sure everything makes sense.

And perhaps that’s why you hire and have QA people, perhaps that is the real purpose of QA. At least I feel that now. I mean, what’s the use of something that has a high “technical quality”, if it just doesn’t make sense? That is actually a big part of the quality, “making sense”. Because of that I’m starting to feel like my job is being a developer that does important things that “nobody has time to do”, because they’re too busy fighting with details. Not in the sense of a “manager”, but in the sense of some “responsible” developer. It’s a really strange job position I think :-)

It’s really hard to measure the impact of good QA in a software project, but I’m sure that is high, probably higher than people use to think. For me, thinking about software projects without QA is a bit like thinking about programming without a Version Control System: I wonder how I had done it in the past, and feel really unconfident without it. How many projects have failed (both from the resources and goals point of view, and from the business point of view) for not having good QA? How many projects have been delayed, or even cancelled, because they lacked someone caring about the Big Picture?

EDIT (2008-5-18): I have disabled comments in this post due to insane amount of spam. If you want to comment, please comment in some other entry :-/