Posts Tagged “qa”
Aug 19, 2020
About a year ago I decided to make a comic with rvr about Quality Engineering/Quality Assurance, explaining what it is and why it is important. There were a couple of breaks in between, partly because of this virus you might have heard of, but it was finally published recently. This post explains a bit the creative process behind it.
As soon as we decided we were going to make the comic, we started brainstorming to figure out how exactly we were going to explain it. We were mostly thinking of metaphors to describe QE people, but also added some associations and aesthetic ideas to the list:
- Wizards, detectives
- Team players, drummers
- Mad scientists making robot helpers
- Creativity, more freedom to build what is needed and adapt
- Build for developer, build for yourself
- James Bond’s Q / Q branch
- Proactive Mr. Wolf
- Rick and Morty
- Something like Mouse Guard? See also paper crafts.
- “Continuous disintegration”.
- SDETs are the real 10x engineers
One of the ideas for the script was to use some kind of Starship Troopers metaphor (“people killing bugs”, or “people making the weapons/tools needed for the soldiers to kill bugs”). We had a certain tension between being serious and in-depth, and being “fun” and grabbing the potential reader’s eye, and we ended up with the idea that we could open with a Starship Troopers scene, but then reveal that it’s just someone’s imagination, and let the rest of the comic be “serious” and more on the explanatory side.
At first we were discussing how to iterate with the storyboard: we were looking for some online collaboration tool that allowed us to build the storyboards, comment on them, and modify them. However, we didn’t find anything I was happy with, and I prefer working on paper for those things, so I decided to just draw a very crude storyboard and take a picture of it. The initial storyboard was like this (click to enlarge):
We discussed it a bit, and after the feedback I create new panels to replace some of the initial ones, and stitched them together in a Frankenstein monster fashion, like so (again, click to enlarge):
If you want to know more about the rationale behind the panels, notice a few things:
- Each line has a meaning, like a sentence in written language. Namely, the first line is opening/attention grabbing (“what happens when there is no QE”), the second expands a bit on the first one but from a serious perspective, the third explains why a development team needs QE, and the last explains the details of how QE achieves what they do and closes.
- Every line ends with a “cliffhanger” to make the reader want to read the next line, and introduce what it is going to be about.
At this point we decided that the storyboard was stable enough to start figuring out how the art would be.
Now that we had a stable storyboard we could look into the final art. We figured that we would still have to iterate, partly because once we used the final art, final font, and final sizes for things… we would probably see things differently: some panel might have too much text, some idea that seems to work in the abstract doesn’t work that well with the final art, etc. So from there we iterated further, but mostly on the language and on details that were relatively easy to change.
The first version with the kind of art we were going to use was this, in black and white:
This gave us a good sense of scale and available space, both for text and for illustrations, so it was much easier to tweak and improve. After a few iterations, we reached the first version with colour! You can see here that the art here has improved, and is at the level we would use in the final version.
Then we kept iterating and, after all the tweaks, reached the final, published version. Notice the difference in the last two panels, and also the text in 3-3:
There you have it! I don’t claim to know what I’m doing, but I love reading about (and writing about) creative processes, and I thought some of you might share my passion for that.
Now the idea is to make at least one more comic related to QE, expanding on more specific problems QE helps solve, or on specific facets on Quality Engineering. We’ll see what we come up with…
Dec 6, 2009
The other day I was in a conversation with some developer that was complaining about some feature. He claimed that it was too complex and that it had led to tons of bugs. In the middle of the conversation, the developer said that the feature had been so buggy that he ended up writing a lot of unit tests for it. To be honest I don’t think there were a lot of bugs after those tests were written, so that made me think:
Maybe the testers in his team are doing too good of a job?
Because, you know, if testers are finding enough of “those bugs” (the ones that should be caught and controlled by unit tests and not by testers weeks after the code was originally written), maybe some developers are just not “feeling the pressure” and can’t really get that they should be writing tests for their code. If testers are very good, things just work out fine in the end… sort of. And of course, the problem here is the trailing “sort of”.
I know I’m biased, but in my view there is a ton of bugs that should never be seen by someone that is not the developer itself. Testers should deal with more complex, interesting, user-centred bugs. Non-trivial cases. Suboptimal UIs. Implementation disagreements between developers and stakeholders. That kind of thing. It’s simply a waste of time and resources that testers have to deal with silly, easy-to-avoid bugs on a daily basis. Not to mention that teams shouldn’t have to wait for days or weeks until they find basic bugs via exploratory testing. Or that a lot of those are much, much quicker to test with unit tests than having to create the whole fixture/environment for them to be found with exploratory testing.
My current conclusion is that pushing on the UI/usability side is not only good for the user, but it’s likely to produce better code as it will be, on average, more complex and will have to be better controlled by QA (code review, less ad-hoc design, …) and automated tests. Maybe developers will start hating me for that, but hopefully users will thank me.
Sep 13, 2009
I spent the whole last week (or this week; after all it’s Sunday… and Sunday is obviously the last day of the week, not the first, right?) in Linköping, Sweden. The idea was repeating some Debian course I gave here in Oslo, giving two more talks about automated testing since I was there anyway, and attend two more talks. It was lots of fun, partly thanks to my “host” (thanks Gerald!), and surprisingly I found a bunch of things that seemed plain weird to me… or at least quite different from Oslo.
The talks themselves went pretty good I think, although I’d have preferred more people attending. I guess it was normal that there were less people than I’m used to, since the Linköping office is much smaller. But anyway. The Debian course went quite well and some people got started packaging stuff almost right away. The other talks were an introduction to automated testing (advocacy and arguments for it, advice, basic examples and small rant about a different kind of QA), which went ok, and an entry-level talk about unit testing in Python (thanks Ask and Batiste for the information and reviewing the slides!), which went very well. I’ll try to get the slides for all the talks available somewhere.
About the city itself, it’s a charming little part of Sweden where:
Restaurants have insanely different prices for food whether it’s for lunch or dinner. Typical prices for lunch are 80 SEK (around 8 EUR) and typical prices for dinner are around 250 SEK just the main course!
Restaurants usually serve some Swedish dish for lunch… and I mean every restaurant, meaning all the Greek, Vietnamese, etc. Considering “real” Swedish restaurants are very expensive, you usually go to those foreign cuisine ones when you actually want to eat Swedish food.
Restaurants typically have some salad (that you have to take yourself) while you wait for the food… and some coffee, tea and cookies (that obviously you have to take yourself) for the end.
Related to this, restaurants are usually very self-service. I thought service in Norway sucked, but boy was I wrong, at least there is some service. And: there were typically long but pretty-fast-moving queues, and there was this one place where you didn’t even get the food on the table after ordering at the bar; instead, you were given some gadget with some wireless receiver, and when your food was ready it’d beep so you knew you had to go to some special place and fetch your food. Is it really cheaper maintaining some gadgets than hiring a waiter? I guess so.
The restrictions on the amount of alcohol that can be bought outside the special Government booze stores are even harder than in Norway. You can only buy booze with up to 3.5% alcohol outside “Systembolaget”. Now that is sad. And I was complaining about Norway’s 5%.
Partly because of that (I assume/hope) the Swedish “cider” you get in Sweden is even sweeter and worse and the Swedish cider you get in Norway.
We went to this nice student pub… which was literally for students. They actually checked your student id, but each student could bring one non-student along. Once you were “identified” as a non-student-coming-with-a-student, you’d get a stamp on your hand so you wouldn’t have to bring along the student when you ordered again. Also, the place was so very slow it was almost funny. One of the good sides was that they had what I thought it was the only decent Swedish cider… but after checking just now, it seems it’s actually American. Bummer. And the name of it was funny too: “Hardcore Cider”.
Right before leaving the office on Friday there was a small gathering in the canteen (the “Friday Beer”), where they had a Dreamcast with one of the most awesome games I’ve seen in a long while: The Typing of the Dead, a version of The House of the Dead 2 in which you kill the zombies by typing words that appear on the screen, instead of aiming and shooting with a gun:
Jul 19, 2009
Those who know me professionally know that I care a lot about software quality assurance. I think it’s a mostly misunderstood field, and generally “the world” would be better off with more QA (and/or better QA). Of course, I’m always looking for more arguments to support my view :-D and the last one I found came from a very interesting blog post, Plane Crashes, Software Failures, and other Human Errors. This post explains how mistakes are made in the aviation and healthcare industries, and claims something that sounds shocking but actually makes quite a bit of sense: “errors occur most often when a senior, experienced person is performing”. The reason why it doesn’t happen as often when the less experience person is performing (again according to the blog post): “because it means the second pilot isn’t going to be afraid to speak up”.
That got me thinking. No matter how expert one person is, he can’t take all the right decisions without help and feedback: a second opinion is always useful and can save the team from embarrassing (or, in some cases, fatal) consequences. A second opinion can give perspective or aspects not thought of by the first person.
If you apply this to software development, I can’t help thinking that one of the roles of QA fulfils this need: being experts in the field that provide second opinions and critiques on anything the team decides or produces. And they shouldn’t feel afraid to speak up because… well, it’s their job after all. And while yes, fellow developers could serve as “second opinion” too, having a more or less formal position for a “Quality Assurance Engineer” is helpful for a variety of reasons. First, as I said the chances of being afraid to speak up are much lower, because it’s their job. Second, not producing the result themselves gives some perspective that people having to fight with everyday details can have, but usually don’t; at least not as much. And last but probably important, it’s their job so they can focus on it and they don’t stop doing it because “they have deliveries soon” or because “they don’t have time”.
Finally, there is another blog post, linked from the above, that also supports my vision of QA: Toyota “Stop the Line” mentality. But this one is about processes and taking a step back when something is wrong, trying to find the root cause instead of an immediate solution. Enjoy the blog posts :-)
Dec 17, 2008
That’s the title of a really good book by Scott Berkun, the fella that was project manager for Internet Explorer when it could still be called a browser ;-) The Myths of Innovation is very easy to read, funny and has some food for thought. It dissects a bunch of myths about innovation and innovators, points out typical difficulties and dangers that innovators face, and analyses why these myths are common, why people like them, and why they are so handy to refer to the history and reality of innovation, which is of course much more complex.
One chapter that made me think a lot was chapter 7: “Your boss knows more about innovation than you”. It explores the relation between (traditional) management and innovation, and claims that managers can work against innovation if they just try to increase efficiency and keep things under control. In that sense, quality assurance engineers can be like those project managers, so I wondered a lot about my role and my duties with regards to innovation. On the one hand, you do have to control things that are being done and be conservative to a certain extent. On the other hand, innovation is such an important part of an IT company (particularly if it’s Internet-related) that you really don’t want to risk blocking or stifling it.
Fortunately, it also explains how to keep the workplace open to innovation, including things like having toys and “funny” things at the office. It turns out that they’re not there to spoil the employees, but to provide an environment where people feel free to “think different” and are not afraid of new ideas or to say what they think.
All in all, I think it’s a great book. Recommended!
Oct 21, 2008
I admit it. I’m a terrible developer. I write code, sometimes even write tests.
But. I. don’t. test. my. programs.
By hand, that is. And sometimes (usually) the coverage is not enough, and I end up making embarrassing mistakes. It usually happens outside of work, although at work I also have my share. The last one was with the Debian package
dhelp, where trying to fix an issue before Lenny is released, I ended up making it even worse. The story goes like this:
There was some problem with the indexing of documents on installation/upgrade (namely, it would take ages for most people upgrading to Lenny, and they would think the upgrade process had hung). So, I go and change the indexing code so it ignores documents on installation/upgrade. Also, as suggested by someone, I created some small example utility to reindex documentation for certain packages. I test installation, upgrades, upgrade of the
dhelppackage itself, the utility, searching for keywords before and after all that… and everything worked.
Only that I made a typo. A typo that would make all indexing to be ignored (except for the example utility, because it was a bit lower level). And I didn’t realise, because it “only” broke some cronjob, a completely different part of the package. And it happens that the cronjob reindexed everything weekly, to make sure that you had reasonably up-to-date search indices. And it also happens that, given that the documentation reindexing was being ignored on package installation/upgrade, the weekly total reindex process was the only thing that could provide the user with indexed documentation. But I screwed up. Oh well.
Someone filed a bug yesterday, and I fixed more or less right away. But this time I spent a couple of hours thinking of test paths and ways to make it fail, and actually doing all that testing. Thanks to that, I found some potential bug in the example utility, that I fixed just in case. So hopefully everything is fine now, if I can convince the Release Masters to allow the new, less broken update to
dhelpto be accepted for Lenny.
I think I need personal QA. Anyone up to the task?
Jul 22, 2008
Some time ago, Opera announced the Opera Web Standards Curriculum project. It’s a very interesting collection of articles that can be used as “curriculum” to learn about web development. It gets extra geeks points for using a Creative Commons license for the articles themselves. Even the W3C mentioned it
:-)I just found some time to have a look at it, that’s why I’m posting now
The other news is that finally the Opera QA blog is online, and has the first non-hello-world-article (written by yours truly), “Continuous Integration: Team Testing”. I’m very excited about this, because it’s the first time I’ll participate directly in a company blog, and because the IT world needs more (and better) QA, so hopefully we’ll be able to spread the word and make the world a better place
Nov 18, 2007
Lately I have been thinking a lot, not to say “obsessed”, with the big picture. I can’t but wonder if that is a general IT industry problem, that big picture. I mean missing it.
My current theory is that computer work is just too hard (or tools not advanced enough?), and there is too much pressure and too hard time constraints to allow people to step back and think about the big picture from time to time, to make sure everything makes sense.
And perhaps that’s why you hire and have QA people, perhaps that is the real purpose of QA. At least I feel that now. I mean, what’s the use of something that has a high “technical quality”, if it just doesn’t make sense? That is actually a big part of the quality, “making sense”. Because of that I’m starting to feel like my job is being a developer that does important things that “nobody has time to do”, because they’re too busy fighting with details. Not in the sense of a “manager”, but in the sense of some “responsible” developer. It’s a really strange job position I think
It’s really hard to measure the impact of good QA in a software project, but I’m sure that is high, probably higher than people use to think. For me, thinking about software projects without QA is a bit like thinking about programming without a Version Control System: I wonder how I had done it in the past, and feel really unconfident without it. How many projects have failed (both from the resources and goals point of view, and from the business point of view) for not having good QA? How many projects have been delayed, or even cancelled, because they lacked someone caring about the Big Picture?
EDIT (2008-5-18): I have disabled comments in this post due to insane amount of spam. If you want to comment, please comment in some other entry