Posts Tagged “projects”
Aug 19, 2020
About a year ago I decided to make a comic with rvr about Quality Engineering/Quality Assurance, explaining what it is and why it is important. There were a couple of breaks in between, partly because of this virus you might have heard of, but it was finally published recently. This post explains a bit the creative process behind it.
As soon as we decided we were going to make the comic, we started brainstorming to figure out how exactly we were going to explain it. We were mostly thinking of metaphors to describe QE people, but also added some associations and aesthetic ideas to the list:
- Wizards, detectives
- Team players, drummers
- Mad scientists making robot helpers
- Creativity, more freedom to build what is needed and adapt
- Build for developer, build for yourself
- James Bond’s Q / Q branch
- Proactive Mr. Wolf
- Rick and Morty
- Something like Mouse Guard? See also paper crafts.
- “Continuous disintegration”.
- SDETs are the real 10x engineers
One of the ideas for the script was to use some kind of Starship Troopers metaphor (“people killing bugs”, or “people making the weapons/tools needed for the soldiers to kill bugs”). We had a certain tension between being serious and in-depth, and being “fun” and grabbing the potential reader’s eye, and we ended up with the idea that we could open with a Starship Troopers scene, but then reveal that it’s just someone’s imagination, and let the rest of the comic be “serious” and more on the explanatory side.
At first we were discussing how to iterate with the storyboard: we were looking for some online collaboration tool that allowed us to build the storyboards, comment on them, and modify them. However, we didn’t find anything I was happy with, and I prefer working on paper for those things, so I decided to just draw a very crude storyboard and take a picture of it. The initial storyboard was like this (click to enlarge):
We discussed it a bit, and after the feedback I create new panels to replace some of the initial ones, and stitched them together in a Frankenstein monster fashion, like so (again, click to enlarge):
If you want to know more about the rationale behind the panels, notice a few things:
- Each line has a meaning, like a sentence in written language. Namely, the first line is opening/attention grabbing (“what happens when there is no QE”), the second expands a bit on the first one but from a serious perspective, the third explains why a development team needs QE, and the last explains the details of how QE achieves what they do and closes.
- Every line ends with a “cliffhanger” to make the reader want to read the next line, and introduce what it is going to be about.
At this point we decided that the storyboard was stable enough to start figuring out how the art would be.
Now that we had a stable storyboard we could look into the final art. We figured that we would still have to iterate, partly because once we used the final art, final font, and final sizes for things… we would probably see things differently: some panel might have too much text, some idea that seems to work in the abstract doesn’t work that well with the final art, etc. So from there we iterated further, but mostly on the language and on details that were relatively easy to change.
The first version with the kind of art we were going to use was this, in black and white:
This gave us a good sense of scale and available space, both for text and for illustrations, so it was much easier to tweak and improve. After a few iterations, we reached the first version with colour! You can see here that the art here has improved, and is at the level we would use in the final version.
Then we kept iterating and, after all the tweaks, reached the final, published version. Notice the difference in the last two panels, and also the text in 3-3:
There you have it! I don’t claim to know what I’m doing, but I love reading about (and writing about) creative processes, and I thought some of you might share my passion for that.
Now the idea is to make at least one more comic related to QE, expanding on more specific problems QE helps solve, or on specific facets on Quality Engineering. We’ll see what we come up with…
Jun 7, 2017
It hasn’t quite been one year since I wrote the first post on Elm, but it’s not that far off and the title was clearer that way. In this time I have written three programs in Elm, each one more complex than the last:
NARROWS, a storytelling system halfway between online role-playing games and improvised Choose Your Own Adventure books.
What I like about Elm
It’s a simple language with few concepts, easy to learn and understand. After my initial (small) struggles, I have mostly loved it since.
The Elm Architecture is really nice and simple, and it’s nice that it’s integrated into the environment. It’s like having a framework for free with the language.
Everything immutable: the easy way is the right way.
Nice, clear, useful compiler error messages.
Generally nice toolchain.
Static types help you a lot, mostly without being annoying.
Newer major versions of the language simplify things further and make things clearer and less error prone (I’ve been through two major updates).
What I don’t like about Elm
The compiler seems slow and often seems to compile older versions of the code. This might be my own fault, as I have a custom Brunch configuration with an Elm plugin and I don’t even understand Brunch all that well. In comparison, the TypeScript compiler was amazing.
Newer major versions of the language break compatibility and you have to rewrite parts of your application, which is really annoying… but also worth it.
I’m not that fond of currying. Sometimes I feel like it makes some compilation error messages harder to understand.
I’m really happy I gave Elm a try almost a year ago. Although I’m looking forward to going back to ClojureScript for some project, I really, really enjoy Elm, and it has almost become my defacto front-end language/framework.
Jun 20, 2016
Around two months ago I started a new pet project. As always, I built it partly to solve a problem, and partly to learn some new language or technology. The problem I wanted to solve was showing images and maps to players when playing table-top role-playing games (and, while at it, manage my music from the same place). The language and technology were TypeScript and to a lesser extent ES2015. As always, I learned some things I didn’t quite expect or plan, like HTML drag-and-drop, Riot, a bit more Flexbox, and some more canvas image processing. The result is the first public version of Lyre, my program to help storytellers integrate music and images into their stories (especially useful for semi-improvised or interactive stories).
But the reason for this post is to talk a little bit about the technology. I admit that I haven’t really studied TypeScript thoroughly (I mostly learned bits and pieces while programming), but I think I like it to the point that it might become my front-end language of choice when I cannot use ClojureScript or similar.
Apart from enums, you can create union types that eg. are as simple as a choice between two strings, like type Mode = “off” “on”.
- Interfaces can be used to specify the properties, not just methods, that should be available in an object. Among other things, it’s possible to specify that an object should have certain specified properties plus any number of extra properties as long as their values are of a given type.
For actual editing I’m of course using Emacs, in this case with TIDE. Although the refactoring capabilities are very limited, the rest worked quite well and I’m very happy with it.
The other bigger thing I learned was Riot. Which, sadly, I didn’t like as much: I found it very confusing at times (what does this point to in the templates? it seems to depend on the nesting level of the ifs or loops), ended up with lots of rebinding of methods so they could be safely passed around in templates, and generally felt that I spent too much time fighting with it rather than writing my application. Now, some of these problems might have been caused by Riot-TS and not Riot itself, but still the experience wasn’t that great and I don’t expect to use it again in the future. Again, bear in mind that I mostly tried to learn on the go, so maybe I was doing everything wrong :-)
In conclusion, I love these projects because I end up with something useful and because I always learn a bunch of things. In this case in particular, I even learned a language that positively surprised me, even if I’m not a big fan of static typing. Again, if you want to have a look at the result, you can learn about Lyre and download the code from GitHub.
Nov 30, 2015
Lately I’ve worked on several small projects, mostly to learn new technologies. The newest one is music-related: a piano that shows scales and chords “in context”, to learn and explore music theory. The idea came about because my first instrument was the guitar, and music theory is pretty hard to make sense of when you’re playing the instrument. It’s just too hard to remember all the notes you’re playing, let alone realise when two chords in the same song are repeating notes because those notes might be played in different positions (eg. one chord might use E on the open sixth string, and another might use E on the second fret of the fourth string).
I remembered that when I started playing around with a piano, and I could figure out how to play a couple of chords, it was painfully obvious that they were repeating notes because they are in the same positions. In the same way, it felt much more natural and easier to figure out on a piano which chords fitted a scale, so I decided to write Music Explorer, and ended up even buying music-explorer.org to host it. I don’t have a particularly grand plan for it, but I’ll probably add at least some small improvements here and there.
Feb 15, 2012
This is the second part of my unit testing advice. See the first part on this blog.
If you need any introduction you should really read the first part. I’ll just present the other three ideas I wanted to cover.
Focusing on common cases
This consists of testing only/mostly common cases. These tests rarely fail and give a false sense of security. Thus, tests are better when they also include less common cases, as they’re much more likely to break inadvertently. Common cases not only break far less often, but will probably be caught reasonably fast once someone tries to use the buggy code, so testing them has comparatively less value than testing less common cases.
The best example I found was in the
wrap_stringtests. The relevant example was adding the string “A test of string wrapping…”, which wraps not to two lines, but three (the wrapping is done only on spaces, so “wrapping…” is taken as a single unit; in this sense, my test case could have been clearer and use a very long word, instead of a word followed by ellipsis). Most of the cases we’ll deal with will simply wrap a given word in two lines, but wrapping in three must work, too, and it’s much more likely to break if we decide to refactor or rewrite the code in that function, with the intention to keep the functionality intact.
See other examples of this in aa20bce (no tests with more than one consecutive newline, no tests with lines of only non-printable characters), b248b3f (no tests with just dots, no valid cases with more than one consecutive slash, no invalid cases with content other than slashes), 5e771ab (no directories or hidden files), f8ecac5 (invalid hex characters don’t fail, but produce strange behaviour instead; this test actually discovered a bug), 7856643 (broken escaped content) and 87e9f89 (trailing garbage).
Not trying to make the tests fail
This is related to the previous one, but the emphasis is on trying to choose tests that we think will fail (either now or in the future). My impression is that people often fail to do this because they are trying to prove that the code works, which misses the point of testing. The point is trying to prove the code doesn’t work. And hope that you fail at it, if you will.
The only example I could find was in the
strcasecmpendtests. Note how there’s a test that checks that the last three characters of string “abcDEf” (ie. “DEf”) is less than “deg” when compared case-insensitively. That’s almost pointless, because if we made that same comparison case-sensitively (in other words, if the “case” part of the function breaks) the test still passes! Thus it’s much better to compare the strings ”abcdef” and “Deg”.
Addendum: trying to cover all cases in the tests
There’s another problem I wanted to mention. I have seen several times before, although not in the Tor tests. The problem is making complicated tests that try to cover many/all cases. This seems to stem from the idea that having more test cases is good by itself, when actually more tests are only useful when they increase the chances to catch bugs. For example, if you write tests for a “sum” function and you’re already testing
[5, 6, 3, 7], it’s probably pointless to add a test for
[1, 4, 6, 5]. A test that would increase the chances of catching bugs would probably look more like
[-4, 0, 4, 5.6]or
So what’s wrong with having more tests than necessary? The problem is they make the test suite slower, harder to understand at a glance and harder to review. If they don’t contribute anything to the chance of catching bugs anyway, why pay that price? But the biggest problem is when we try to cover so many test cases than the code produces the test data. In this cases, we have all the above problems, plus that the test suite becomes almost as complex as production code. Such tests become much easier to introduce bugs in, harder to follow the flow of, etc. The tests are our safety net, so we should be fairly sure that they work as expected.
And that’s the end of the tips. I hope they were useful :-)
Feb 14, 2012
When reviewing tests written by other people I see patterns in the improvements I would make. As I realise that these “mistakes” are also made by experienced hackers, I thought it would be useful to write about them. The extra push to write about this now was having concrete examples from my recent involvement in Tor, that will hopefully illustrate these ideas.
These ideas are presented in no particular order. Each of them has a brief explanation, a concrete example from the Tor tests, and, if applicable, pointers to other commits that illustrate the same idea. Before you read on, let me explicitly acknowledge that (1) I know that many people know these principles, but writing about them is a nice reminder; and (2) I’m fully aware that sometimes I need that reminder, too.
Edit: see the second part of this blog.
Tests as spec
Tests are more useful if they can show how the code is supposed to behave, including safeguarding against future misunderstandings. Thus, it doesn’t matter if you know the current implementation will pass those tests or that those test cases won’t add more or different “edge” cases. If those test cases show better how the code behaves (and/or could catch errors if you rewrite the code from scratch with a different design), they’re good to have around.
I think the clearest example were the tests for the
eat_whitespace*functions. Two of those functions end in
_no_nl, and they only eat initial whitespace (except newlines). The other two functions eat initial whitespace, including newlines… but also eat comments. The tests from line 2280 on are clearly targeted at the second group, as they don’t really represent an interesting use case for the first. However, without those tests, a future maintainer could have thought that the
_no_nlfunctions were supposed to eat whitespace too, and break the code. That produces confusing errors and bugs, which in turn make people fear touching the code.
See other examples in commits b7b3b99 (escaped ‘%’, negative numbers, %i format string), 618836b (should an empty string be found at the beginning, or not found at all? does “\n” count as beginning of a line? can “\n” be found by itself? what about a string that expands more than one line? what about a line including the “\n”, with and without the haystack having the “\n” at the end?), 63b018ee (how are errors handled? what happens when a %s gets part of a number?), 2210f18 (is a newline only \r\n or \n, or any combination or \r and \n?) and 46bbf6c (check that all non-printable characters are escaped in octal, even if they were originally in hex; check that characters in octal/hex, when they’re printable, appear directly and not in octal).
Boundaries of different kinds are a typical source of bugs, and thus are among the best points of testing we have. It’s also good to test both sides of the boundaries, both as an example and because bugs can appear on both sides (and not necessarily at once!).
The best example are the tor_strtok_r_impl tests (a function that is supposed to be compatible with
strtok_r, that is, it chops a given string into “tokens”, separated by one of the given separator characters). In fact, these extra tests discovered an actual bug in the implementation (ie. an incompatibility with
strtok_r). Those extra tests asked a couple of interesting questions, including “when a string ends in the token separator, is there an empty token in the end?” in the “howdy!” example. This test can also be considered valuable as in “tests as spec”, if you consider that the answer to be above question is not obvious and both answers could be considered correct.
See other examples in commits 5740e0f (checking if
tor_snprintfcorrectly counts the number of bytes, as opposed the characters, when calculating if something can fit in a string; also note my embarrassing mistake of testing
snprintf, and not
tor_snprintf, later in the same commit), 46bbf6c (check that character 21 doesn’t make a difference, but 20 does) and 725d6ef (testing 129 is very good, but even better with 128—or, in this case, 7 and 8).
Testing implementation details
Testing implementation details tends to be a bad idea. You can usually argue you’re testing implementation details if you’re not getting the test information from the APIs provided by whatever you’re testing. For example, if you test some API that inserts data in a database by checking the database directly, or if you test the result of a method call was correct by checking the object’s internals or calling protected/private methods. There are two reasons why this is a bad idea: first, the more implementation details you tests depend on, the less implementation details you can change without breaking your tests; second, your tests are typically less readable because they’re cluttered with details, instead of meaningful code.
The only example I encountered of this in Tor were the compression tests. In this case it wasn’t a big deal, really, but I have seen this before in much worse situations and I feel this illustrates the point well enough. The problem with that deleted line is that it’s not clear what’s it’s purpose (it needs a comment), plus it uses a magic number, meaning if someone ever changes that number by mistake, it’s not obvious if the problem is the code or the test. Besides, we are already checking that the magic number is correct, by calling the
detect_compression_method. Thus, the deleted
memcmpdoesn’t add any value, and makes our tests harder to read. Verdict: delete!
I hope you liked the examples so far. My next post will contain the second half of the tips.
Oct 24, 2011
About the technology, I used the Markdown format for the pages thanks to the markdown-js library (it’s really nice that the module has an intermediate tree format that you can parse to add your own stuff before converting to HTML, like e.g. wikipage links!), express for the whole application structure and js-test-driver + jsautotest + a bit of syntax sugar from Sinon.js for the tests (but looking forward to trying out Buster.js when it’s released!). The deployment to Joyent’s Node.js SmartMachine was reasonably easy. Actually, it was pretty easy once I figured the following:
You must not forget to listen in the correct port, with
server.listen(process.env.PORT || 8001)
There are a couple of pretty useful Node.js-related command-line utilities to check logs, restart applications and so on
The configuration of the application can be done via
npm config, see npm integration on Joyent’s Wiki
If you’re curious to see the code, play with it or use it yourself, take a peek to the Wiki-Toki repository on GitHub. Happy hacking!
Mar 22, 2010
For some time now I had been frustrated by the tools to manage APT repositories. The only ones I knew of either covered too little (only adding/removing packages from a repository and such, like reprepro) or were way too complex (like the official tools used by Debian itself). Maybe/probably I’m a moron and I just didn’t know of some tool that would solve all my problems, but now it’s kind of late ;-) And before you say it, no, Launchpad is not what I was looking for as far as I understand it.
So I started to work on my own suite of tools for it, and recently I decided to release what I’ve done so far. It’s by no means complete, but it’s very useful for me and I thought it would be useful for others. And, with a bit of luck, someone will help me improving it.
So what is it? Arepa (it stands for “Apt REPository Assistant”, but obviously I called it like that after the yummy Venezuelan sandwiches) is a suite of tools that allow you to manage an APT repository. It contains two command-line tools and a web interface, and its main features are:
Manages the whole process after a package arrives to the upload queue: from approving it to re-building from source to signing the final repository.
It allows you to “approve” source packages uploaded to some “incoming” directory, via a web interface.
It only accepts source packages, and those are re-compiled automatically in the configured autobuilders. It can even “cross-compile” for other distributions (treated like binNMUs).
The approval via some web interface was actually sort of the driving force for the project. One of my pet peeves was that there wasn’t an easy way to have an upload queue and easily approve/reject packages with the tools I knew. From what I had seen, the tools were either for “single person” repositories (no approval needed because the package author is the owner of the repository) or full-blown distribution-size tools like dak and such. My use-case, however, is the following:
You have an installation of Arepa for an entire organisation (say, a whole company or a big department).
People inside that organisation upload packages to the upload queue (possibly using dput; the point is, the end up in some directory in the machine hosting Arepa).
Someone (or a small group of people) are the “masters” of the repository, and they’ll have access to the web interface. From time to time they check the web UI, and they’ll approve (or not) the incoming source packages.
If they’re approved, the source will be added to the repository and it’ll be scheduled for compilation in the appropriate combination(s) of architectures and distributions.
A cronjob compiles pending packages every hour; when the compilation is successful, they’re added to the repository.
At this point, the repository hosted by the Arepa installation has the new packages, but you probably want to serve the repository from a different machine. If that’s the case, Arepa can sync the repository to your production machine with a simple command (“arepa sync”).
I imagine that a lot of people have the same need, so I uploaded all the code to CPAN (you can see it with the rest of the contributions by Opera Software). Sadly there’s a silly bug in the released code (I wanted to release ASAP to be able to focus on other things, and I ended up rushing the release), but it has both a workaround and a patch. So, please give it a try if you’re interested and tell me if you would like to contribute. I haven’t released the code in GitHub or similar yet, but I’ll probably do if there’s interest.
Jul 1, 2009
The initial idea of the loanmeter widget was to show where in the world Kiva was offering loans. However, as I used the widget myself, I realised that the location in the world was less important for me, and I was more interested in knowing what the person was going to use the money for. So, I added some options to filter by “sector” and I figured that having some graphs comparing how much money was requested and already funded, for each sector, would be a very quick and visual way to get the information I wanted. I started playing with flot, and I have to say that except for a couple of relatively minor problems, it was quite easy to use. I don’t have screenshots showing the graphs, but feel free to try the widget itself and have a look (hint: you have two buttons at the bottom right corner to switch between “map view” and “graph view”).
The other widget I have been working on is a monitor widget for projects in CruiseControl.rb (a really simple and neat continuous integration server we use at Opera). More than one year ago, my colleague Nico had written a very quick & dirty widget for monitoring the result of the test runs of the My Opera functional testsuite. There were a couple of things I wanted to change, and I also wanted to monitor other projects, so I figured that I’d rewrite the widget to have a more maintainable codebase and then make it generic, so you could configure which CC.rb installation and which project to monitor. I’m moderately happy with the result of the refactoring, and happy enough with the final result. I know it has several issues, and I expect that once anyone outside our team starts using it, there will be things to improve and fix :-) If you use CruiseControl.rb, give it a try!
Jun 29, 2009
UPDATE: Bubug hasn’t been maintained for a long time and is now deprecated, sorry. The closest equivalent I have to a TODO application is Wiki-toki, my personal Wiki program.
When writing yesterday about the Perl modules, I realised that I hadn’t written anything about the TODO application since “The ultimate TODO app”. Well, a lot has happened to it actually. I’m glad to announce that:
It does have a (lame) name now: Bubug (supposedly stands for “Barely Unconventional Bug Untracking Gizmo”. Whatever).
It has improved a lot here and there, and it now has authentication and multi-user support, not to mention a lot of UI bling bling and goodies.
The development has moved to BitBucket, an excellent free service built by ex-Opera’s Jesper Noehr, where you can follow it more easily, comment on, check the Wiki, fork, or whatever you want. You even have a screenshot there ;-)
As you can guess from the last point, for this project I’ve been using Mercurial instead of Git. Although I certainly don’t have sophisticated needs, so YMMV (heavily), I find Git more pleasant to use. Which is kind of surprising, because I always thought that Git’s UI was a pain in the ass. Oh, well. That doesn’t mean that Mercurial is hard to use, though. I think it’s more that I’m used to Git now, and there are a couple of things that I find more convenient: the coloured diff (possible in Hg, but you have to install some extension for it, and only thinking about installing some Python extension that is not even packaged for Debian makes me want to switch to Git) and the staging area are the most important ones I can think of.
So, if you thought I had abandoned the TODO application thing, you were wrong ;-) If you’re interested, have a look at the Bubug BitBucket project page, download it, play with it, and tell me what you think.
May 10, 2009
I’ve been working on something lately that I hope I will publish sometime next month: it’s a set of tools to manage an APT package repository. The idea is that, given an upload queue (you can set it up as an anonymous FTP, or some directory accessible via SSH/SCP, or whatever floats your boat in your setup and team), you’ll have a web interface to approve those packages, a set of integrated autobuilders building the approved packages in whatever combination of architectures and distributions you want, and all that integrated with reprepro to keep your repository updated. I’ll write more about it when I have released something.
The point now is that, while working on it, I needed some module to parse command-line options and “subcommands” (like
svn update, etc.). As it’s written in Perl, I had a look at CPAN to see if I could see anything. The most promising module was App::Rad, but it lacked a couple of things that were very important for me: my idea was “declaring” all the possible commands and options and have the module do all the work for me (generating the help pages and the default
--helpimplementation, generate the
program help subcommandand so on).
App::Raddidn’t have that, and it didn’t seem to me like that was the direction they wanted to go to with the module. But I figured I’d drop the author an e-mail anyway and see if he liked the idea so I could start adding support for all that…
And boy was that a good idea. He replied a couple of days later, and said that they had liked the idea so much that they had implemented it already (that’s why he took a couple of days to reply), and he sent me an example of the new syntax they had introduced and asked if that was what I was thinking. And not only that, but they added me to the list of contributors just for giving the idea! That completely made my day, free software rocks!
Feb 9, 2009
UPDATE: Bubug hasn’t been maintained for a long time and is now deprecated, sorry. The closest equivalent I have to a TODO application is Wiki-toki, my personal Wiki program.
I have been quite frustrated by TODO applications for some months now. They’re usually either too simple, or almost too complex and without features that I think are really valuable. In particular, there are two things that I don’t remember having seen in any TODO application:
Possibility to “postpone” a task, so it doesn’t appear in the main view for a defined time.
Possibility to associate a task to a “person to nag”.
When you have a lot of small tasks to do, and they are not the kind of things you put in a BTS (say, stuff that you have to do that is not really connected to some project’s code) I think these two features are really useful, and I was surprised that no applications I saw seemed to have those. I mean, don’t people have the same problems as me?
That’s why, as I had mentioned, I started writing my own TODO application: I’d have what I wanted, and I’d learn a thing or two about Merb, DataMapper and jQuery. The application has several design limitations that I used to simplify things, like not having any notion of users (single user app without authentication) or supporting only a “title” for the tasks, without any longer description. It isn’t something I really plan to publish for other people to use (I mean, the code is in my Darcs repo, I’m just not going to make a project page for it or anything like that), so I don’t really care how much it fits other people’s needs :-)