GIMP Scripting

The last month or so I have been working on a project that I will talk about shortly. For that I had to do some simple image edits often, and after the third time or so I figured it was better to script them. I had done a little GIMP scripting before so I figured it wouldn’t be hard to write a couple of scripts to do what I wanted (which was very simple to start with). However, every time I had to do something with GIMP scripting I forget the details and I need to check again, so I figured it would be useful to leave some notes for myself and for others who might be interested.

Scripting languages

There are two main scripting engines you can use in GIMP: Script-Fu (which uses Scheme, a Lisp dialect), and python-fu (which uses Python). I have used both at some point but this tutorial will use Script-Fu. Unfortunately, making an introduction to Scheme as a language would be too long for this blog post, but you can check the Script-Fu tutorial in the official documentation.

Note that in the example I’m using some function defined in GIMP 2.10 and later, so the example code will not work as-is with older versions of the software (in particular, you will have to delete the calls to the functions gimp-context-set-stroke-method and gimp-context-set-line-width, and change gimp-drawable-edit-stroke-selection to gimp-edit-stroke… but if you do that, the script will lose some functionality).

Types of scripts/uses

There are two typical uses for these scripts: adding extra menu entries so we can call our script from within GIMP itself, and adding functions that we can call from the command-line. The former typically receive images or layers, and the latter typically receive filenames. In both cases we define functions that do what we want (from images/layers or from filenames) and then we either register them in the menu, or leave them as-is so we can call them from the command-line.

Defining a simple function

Create a new file add-border-to-image.scm inside your GIMP scripts/ directory. If you don’t know where that is, go to Edit ➝ Preferences ➝ Folders ➝ Scripts. You can use any of the folders in that list, or even add a new one.

In that file, enter the following text:

(define (add-border-to-image image)
  (let* ((drawable (car (gimp-image-get-active-layer image))))
    ;; The context push allows us to change settings (like the
    ;; foreground colour) and go back to the previous settings
    ;; when we pop it. The gimp-image-undo-group-* makes sure
    ;; that all the operations are considered only one with
    ;; regards to undo.
    (gimp-context-push)
    (gimp-image-undo-group-start image)

    (gimp-selection-all image)
    (gimp-selection-shrink image 2)

    (gimp-context-set-stroke-method STROKE-LINE)
    (gimp-context-set-line-width 4)
    (gimp-context-set-foreground "#657487")
    (gimp-drawable-edit-stroke-selection drawable)

    (gimp-image-undo-group-end image)
    (gimp-context-pop)))

This code defines a function called add-border-to-image which receives an image and paints a 4-pixel border on it. It also makes sure that all the operations are considered only one for undo purposes.

Once we have that function, we can add a menu entry for it, or we can prepare it so it’s easy to call from the command-line.

Functions already defined in GIMP

When you write these scripts you will need to check which functions are already defined in GIMP (like gimp-selection-all or gimp-drawable-edit-stroke-selection here). You do that by going to Filters ➝ Script-Fu ➝ Console and then clicking on the “Browse…” button. You will see a list of functions and a search box you can use to search for them. Note that by default the search only looks for functions with those names, so you might want to change that to “by description”.

Adding a menu entry for the function

If we want to be able to paint borders on an image we have open in GIMP, we can add the following code at the end of the file, then either restart GIMP or go to Filters ➝ Script-Fu ➝ Refresh Scripts:

(script-fu-register "add-border-to-image"
                    "<Image>/Edit/Add border"
                    "Paints a 4-pixel border"
                    "Esteban Manchado Velázquez"
                    "Esteban Manchado Velázquez"
                    "2020"
                    "RGB*"
                    SF-IMAGE "Image" 0)

This will add a new menu entry under Edit called “Add border”. The entry will be grayed out if you don’t have any image open. If you do, you will be able to click on the new option and see the newly added border.

Note: for some reason that I haven’t been able to find out, you will need to click anywhere on the image for the border to appear. This is only a problem when using it from the menu.

Calling the function from the command-line

To call the function from the command-line we will have to create a new function that receives a file path pattern (could be a file path or something like images*.png), opens the file(s), calls the function we defined, and then saves the file(s) somewhere. Add this at the top of add-border-to-image.scm:

(define (add-border-to-file file-pattern)
  (let* ((filelist (cadr (file-glob file-pattern 1))))
    (while (not (null? filelist))
           (let* ((filename (car filelist))
                  (image (car (gimp-file-load RUN-NONINTERACTIVE
                                              filename
                                              filename)))
                  (drawable
                   (car (gimp-image-get-active-layer image)))
                  (output-filename
                   (string-append (car (strbreakup
                                        filename "."))
                                  "-focused.png")))
             (add-border-to-image image)

             (gimp-file-save RUN-NONINTERACTIVE
                             image drawable
                             output-filename output-filename)
             (gimp-image-delete image))
           (set! filelist (cdr filelist)))))

This will receive one parameter, namely file-pattern, interpret it as a file pattern with possible wildcards (with the file-glob function) and go through every file that matches that pattern. Then, for every file, it will open it, calculate the output filename for that file, call add-border-to-image with the open image, and then save the result in the calculated output file and close the image.

Calling the function from the command-line

The command-line incantation to call this function is not trivial and I always forget it. We need to call the gimp program with the -i option (so that the user interface doesn’t load) and the -b option (for batch) and pass it some Scheme to call the function we want (in this case, add-border-to-file). We also need to call the special function gimp-quit so that GIMP quits after the function has finished. We do so by calling it like this:

gimp -i -b '(add-border-to-file "img*.png")' -b '(gimp-quit 0)'

Conclusions

Automating repetitive tasks with Script-Fu can be extremely useful and save us a bunch of time. We can expose the functionality we create in two ways: via GIMP’s own menus, and via the command-line’s “batch mode”. If one is used to Lisp dialects, using Script-Fu with Scheme is not too difficult; otherwise, Python-Fu can be a better alternative. Both languages have an interactive console inside of GIMP to try things out, plus a documentation browser to see and search for available Script-Fu functions.

If one wants to learn more details, the official documentation has a Script-Fu tutorial.

• gimp
• scripting
• programming
• art

Making the Quality Engineering comic

About a year ago I decided to make a comic with rvr about Quality Engineering/Quality Assurance, explaining what it is and why it is important. There were a couple of breaks in between, partly because of this virus you might have heard of, but it was finally published recently. This post explains a bit the creative process behind it.

High-level concepts

As soon as we decided we were going to make the comic, we started brainstorming to figure out how exactly we were going to explain it. We were mostly thinking of metaphors to describe QE people, but also added some associations and aesthetic ideas to the list:

• Wizards, detectives
• Team players, drummers
• Mad scientists making robot helpers
• Creativity, more freedom to build what is needed and adapt
• Build for developer, build for yourself
• James Bond’s Q / Q branch
• Proactive Mr. Wolf
• Rick and Morty
• Something like ​Mouse Guard? See also paper crafts.
• “Continuous disintegration”.
• SDETs are the real 10x engineers

Script ideas

One of the ideas for the script was to use some kind of Starship Troopers metaphor (“people killing bugs”, or “people making the weapons/tools needed for the soldiers to kill bugs”). We had a certain tension between being serious and in-depth, and being “fun” and grabbing the potential reader’s eye, and we ended up with the idea that we could open with a Starship Troopers scene, but then reveal that it’s just someone’s imagination, and let the rest of the comic be “serious” and more on the explanatory side.

Storyboard iterations

At first we were discussing how to iterate with the storyboard: we were looking for some online collaboration tool that allowed us to build the storyboards, comment on them, and modify them. However, we didn’t find anything I was happy with, and I prefer working on paper for those things, so I decided to just draw a very crude storyboard and take a picture of it. The initial storyboard was like this (click to enlarge):

We discussed it a bit, and after the feedback I create new panels to replace some of the initial ones, and stitched them together in a Frankenstein monster fashion, like so (again, click to enlarge):

If you want to know more about the rationale behind the panels, notice a few things:

1. Each line has a meaning, like a sentence in written language. Namely, the first line is opening/attention grabbing (“what happens when there is no QE”), the second expands a bit on the first one but from a serious perspective, the third explains why a development team needs QE, and the last explains the details of how QE achieves what they do and closes.
2. Every line ends with a “cliffhanger” to make the reader want to read the next line, and introduce what it is going to be about.

At this point we decided that the storyboard was stable enough to start figuring out how the art would be.

Art

Now that we had a stable storyboard we could look into the final art. We figured that we would still have to iterate, partly because once we used the final art, final font, and final sizes for things… we would probably see things differently: some panel might have too much text, some idea that seems to work in the abstract doesn’t work that well with the final art, etc. So from there we iterated further, but mostly on the language and on details that were relatively easy to change.

The first version with the kind of art we were going to use was this, in black and white:

This gave us a good sense of scale and available space, both for text and for illustrations, so it was much easier to tweak and improve. After a few iterations, we reached the first version with colour! You can see here that the art here has improved, and is at the level we would use in the final version.

Then we kept iterating and, after all the tweaks, reached the final, published version. Notice the difference in the last two panels, and also the text in 3-3:

Conclusions

There you have it! I don’t claim to know what I’m doing, but I love reading about (and writing about) creative processes, and I thought some of you might share my passion for that.

Now the idea is to make at least one more comic related to QE, expanding on more specific problems QE helps solve, or on specific facets on Quality Engineering. We’ll see what we come up with…

• qa
• quality
• comics
• storytelling
• programming
• projects

First impressions of Elm

For my next pet project I decided to learn Elm. Elm is a functional language similar to Haskell, that compiles to JavaScript. These are my first impressions so take them with a grain of salt.

Syntax

Elm’s syntax is similar to Haskell’s. I had tried to learn Haskell a long time ago but failed miserably because I couldn’t understand the types. I did not find Elm’s syntax to be a problem, and it was nice and simple, especially compared to Elixir, the last language I had learned. However, I think it was the first time in my life that I wished I had had a small syntax description or guide before diving too deep into the language. For me there were two confusing things:

1. Sometimes I saw a bunch of words that were separated by spaces or by commas, and it took me a bit of time to realise that function arguments are separated by spaces, but elements in a list or similar are separated by commas.

2. Compound types that have more than one word, like List Int. That one is easy to figure out of course, but when I saw Html Msg I had no idea what it meant. I’m still not completely sure, in fact.

The first point in particular has an implication: if you use the result of a function call as an argument for second function you must enclose the first function call in parentheses. This all seems super obvious in retrospect, but when staring at code that uses a DSL-like library to generate HTML in a language you’re starting to learn… well, it would have helped to have the first point spelled-out. Example:

    ul []
     (List.map (tagView sectionPage)
        (ModelUtils.getSectionTags section))

Here we have a call to the function ul that has two arguments: an empty list and another list, namely the result of the call to List.map. Note how the whole List.map must be enclosed in parentheses: otherwise, it would be interpreted as a call to ul with four arguments, not two. In the same way, both arguments to List.map must be enclosed in parentheses, too.

Elm Architecture

Although strictly speaking you don’t have to use it, the Elm Architecture is a fundamental part of Elm. It’s the high-level pattern for application architecture when writing Single-Page Applications in Elm, the main usecase for the language. The pattern is very similar to redux and friends in React, but it’s nicer and more natural in a functional, statically-typed language like Elm.

In short, it means separating your application in sub-applications that have a model, a React-style view produced from the model, a set of messages the view can send, and an update function that receives messages and changes the model. Besides, Elm supports commands and subscriptions, which gives a nice, clean interface for WebSockets and other things.

Conclusion

Although I’ve been looking forward to going back to ClojureScript, and in particular learn and play with Om Next, Elm is certainly a worthy contender and totally worth checking out, especially if you’re using React and you want to go one step further.

I admit I did get frustrated now and then with the static types, and at first a bit with the syntax (see the two points above) and the indentation. However, all in all I enjoyed learning and using Elm a lot, and it feels very clean, productive, and just nice to program in.

The application I wrote was very small, though, and I didn’t quite get to explore patterns in how to split an application in several sub-applications. I did read a bit about it but didn’t get to use anything fancy.

And if you want to see what I built, head over http://rpg.hcoder.org/ for the site, or https://github.com/emanchado/rpg-catalog for the code.

• elm
• frontend
• functional programming
• programming
• programming languages
• web
• webapp

First impressions of Elixir + Phoenix

I had been curious about Elixir for some time. After all, the promise of having the best of Erlang with a more palatable syntax was very attractive indeed.

A couple of days ago I finally finished a small project in Elixir using the Phoenix web framework, which is a sort of “Elixir on Rails”. These are my first impressions of both Elixir as a language and Phoenix as a framework. Take all this with a grain of salt: most of it is pretty subjective, and it’s from the perspective of a total Elixir/Erlang noob.

Elixir

I used Introducing Elixir for learning, which turned out to be a bad choice because it can feel like an intro to functional programming using Elixir, not so much an in-depth book about Elixir for someone who knows functional programming. In fact, the book preface says:

If you’re already familiar with functional languages, you may find the pacing of this gentle introduction hopelessly slow. Definitely feel welcome to jump to another book or online documentation that moves faster if you get bored.

Elixir is a bit of a mindfuck for me in that it looks like Ruby, but it’s not object-oriented at all. The language also seems to value convenience a bit too much for my taste (sacrificing simplicity or consistency). In particular, I find the extra, convenience syntax for symbols in maps extremely annoying:

%{"blah" => 1}  # string as a map key
%{blah: 1}      # symbol as a map key (instead of %{:blah => 1})

Another case of different syntax options is the if statement. These two are equivalent:

if x > 10 do :large else :small end
if x > 10, do: :large, else: :small

I seem to recall this has something to do with macros, but all that syntax sugar feels weird. And all those colons, sometimes before, sometimes after a word, look ugly and confusing to me.

I have other, smaller peeves, but they’re subjective or unimportant. However, they strengthened the impression that I didn’t like the syntax.

In conclusion, the syntax reminded me of ES2015: syntax and exceptions for convenience, which makes it feel inconsistent, complex, and hard to remember. It constantly reminded me of the fat arrow function in ES2015.

Phoenix

Phoenix came with its own, related mindfuck: it looks like Rails and often feels like it, but there aren’t classes. That confused me a couple of times, but I guess it’s just a matter of getting used to it.

I think I liked it generally, and it felt productive, but I also felt that there was too much magic and generated code. Not as bad as with how I remember Rails (from many years ago), but enough to make me feel uncomfortable. Also, take into account that my project was a perfect fit for the framework: a small, mostly CRUD application.

I did get to try both tasks and channels, which were really cool, for example with the automatic reconnect in channels (they are implemented using WebSockets) without having to write any special code.

Conclusions

It was interesting to learn Elixir, but I’m curious about Erlang now. As in, I like the concepts behind Elixir (which are mostly Erlang concepts) and I’m not in love with Elixir’s syntax, so if I had to build a system that needed that kind of scalability and reliability I would consider Erlang.

• elixir
• functional programming
• phoenix
• programming
• programming languages
• web
• webapp

Functional Programming Is Not Weird

Recently I read a blog post titled “Functional Programming Is Not Popular Because It Is Weird”. I strongly disagree with the assessment and Twitter was too short for an answer, hence this blog post :-)

The blog post says that “Writing functional code is often backwards and can feel more like solving puzzles than like explaining a process to the computer”. I don’t agree at all, and I think this is simply a result of seeing functional programming in imperative programming terms. The first thing that comes to mind is that programming should not feel like “explaining a process” to the computer. Why should it? Imperative programming is centred around the process, the how. Functional programming is more focused on the what (which indeed is backwards compared to imperative, but I don’t think it’s backwards for humans).

Second, I think the recipe example is terrible and doesn’t show anything about the two styles of programming. The point of reading a program is not to be able to follow it step by step and produce the correct result (the computer does that), it’s about understanding the intent. Food recipes are about mixing things and obtaining the result. Programming is not about mixing things. In fact, the easier to separate and recombine they are, the better. This is very unlike a recipe.

Third, there is the “Imperative languages have this huge benefit of having implicit state. Both humans and machines are really good at implicit state attached to time”. Hahaha, what? State is the source of many, many problems in programming. Avoiding state, or at least separating the state-juggling bits, generally makes for better programs. Humans are terrible at keeping state in their minds, or understanding how a certain state was reached, or realising what produced changes in the state. To give an more concrete example of this: the post claims that “you know that after finishing the first instruction the oven is preheated, the pans are greased and we have mixed a batter”. No, you don’t. In a real program, you would be reading the source code for the second point, and it wouldn’t be obvious what the current state is at that moment. You would have to read, understand and play in your head the whole first point (and remember that, in real programs, there would be many ways to reach point 2, yikes!). And when anyone changed the first point, suddenly your understanding of the second point would be wrong, and you wouldn’t even know because the second point’s source code hasn’t changed. That is one of the reasons why state is generally best avoided.

The C++ example using templates as a functional language I’ll just skip as I don’t think is relevant at all for real functional programming. But right after, the author claims “I spend too much time trying to figure out how to say things as opposed to figuring out what to say”. I can relate to this while I was learning functional programming, but that’s entirely normal: you are, in a way, re-learning how to program. Many of your instincts and mental tools aren’t valid any more, and in some sense you have to start over. It’s frustrating, but the fact that it’s hard means that you’re being forced to think in a different, and in my view better, way.

Finally, at the end there’s “if functional programming languages want to become popular, they have to be less about puzzle solving”. If it wasn’t hard for imperative programmers to learn it, it wouldn’t be a different style, it would simply be a different syntax. No point in learning it! The reason why it feels like “puzzle solving” is because it’s different, not because it’s hard.

Now, I’m not saying functional programming is the best tool for every single problem, but most gripes imperative programmers have about functional programming are about “it’s different”, whether they realise it or not. Don’t be afraid to learn functional programming when it seems hard: that’s the whole point, you are learning a different way to solve problems!

• functional
• functional programming
• programming

Music Explorer

Lately I’ve worked on several small projects, mostly to learn new technologies. The newest one is music-related: a piano that shows scales and chords “in context”, to learn and explore music theory. The idea came about because my first instrument was the guitar, and music theory is pretty hard to make sense of when you’re playing the instrument. It’s just too hard to remember all the notes you’re playing, let alone realise when two chords in the same song are repeating notes because those notes might be played in different positions (eg. one chord might use E on the open sixth string, and another might use E on the second fret of the fourth string).

I remembered that when I started playing around with a piano, and I could figure out how to play a couple of chords, it was painfully obvious that they were repeating notes because they are in the same positions. In the same way, it felt much more natural and easier to figure out on a piano which chords fitted a scale, so I decided to write Music Explorer, and ended up even buying music-explorer.org to host it. I don’t have a particularly grand plan for it, but I’ll probably add at least some small improvements here and there.

If you are interested in the technical side of it, Music Explorer is written in JavaScript (EcmaScript 6, to be more exact). I learned to use React with JavaScript, Browserify, Sass/SCSS, Ramda, AVA, the fantastic JavaScript music library teoria and a bit more CSS while writing this so it was definitely a useful learning experience for me. The full source code lives in GitHub and it’s licensed under the MIT license so you can go have a look!

• chords
• es6
• free software
• javascript
• music
• music theory
• open source
• programming
• projects
• react
• scales
• web
• webapp

Pet projects

I’ve been writing several pet projects in the last months. I wrote them mostly to learn new languages, techniques or libraries, and I’m unsure as to how much I’ll use them at all. Not that it matters. All three are related to role-playing games. In case you’re interested:

• Character suite: a program to help create characters for RPGs. It’s designed to make it easy to add new rule systems for new games. It’s written in Clojure and ClojureScript, and with it I learned devcards, macros, Clojure’s core.async, figwheel and PDF generation with PDFBox. The code is messy in parts, but I learned a lot and I have a usable result, which is the important thing.

• Nim-DR: a tiny command-line program to roll dice, including adding aliases for rolls (eg. alias “pc” for “1d100”). It doesn’t even support mixing kinds of dice, or adding numbers to the result. I mostly wrote it to get a taste of Nim. While I’m not a big fan of statically typed languages, the type inference really helped, and I liked what I saw. I may try more experiments in Nim in the future.

• Map discoverer: a program to uncover a map bit by bit. You can use it online. I wrote it to learn ES6, the new JavaScript standard, and a bit about the HTML5 canvas element. I used Babel to translate ES6 to regular JavaScript (I could have used Node 4, but Babel supports much more ES6) and es6-features.org as a reference to learn the new bits. I really liked ES6: it still feels like JavaScript, but it improves things here and there. I just wished let didn’t allow re-assignment of variables.

While writing the last one I even learned about the pointer-events CSS property, which allows you to mark an element as not receiving events like mouseclick, mousemove, etc. Very useful!

• clojure
• clojurescript
• es6
• functional programming
• javascript
• nim
• programming
• programming languages

Book review: Clojure Cookbook

I just finished “Clojure Cookbook”, a book full of recipes for all kinds of things you can do with the Clojure programming language. I got the book through O’Reilly’s Reader Review Program. Obviously the book is not for you if you don’t know any Clojure, but I think it’s great if you know some Clojure but still wonder how to do relatively basic tasks, or want to see how common libraries are used, or want to see idiomatic Clojure solutions to everyday problems.

The book is divided in different sections ranging from basic data structures to network I/O, performance and production tips or testing. Each section has a number of recipes in the form of small problems together with solutions and a detailed explanation of the solution, as well as caveats or things you should take into account when solving that kind of problem. In a book like this is inevitable that you’re not going to care much about certain sections (in my case, I didn’t care much for distributed computation and I don’t find database access that exciting), but in general the book is very clear, the explanations concise, the code to the point and the recipes useful and varied.

In a nutshell, very good if you know some Clojure but want to learn more about how to solve everyday programming problems elegantly with Clojure.

• books
• clojure
• fp
• functional programming
• programming
• programming languages

Learning Clojure

I have always had a thing for functional programming. Not so much for functional languages maybe, but definitely for functional programming principles. I had learned a tiny bit of Lisp at the uni and have fond memories of reading “On Lisp” by Paul Graham, but I never really tried to learn any Lisp for real. Or, I tried to learn Common Lisp again by reading “Land of Lisp” but gave up very quickly. I have tried to learn other functional languages with varying degrees of success (ranging from “read a tutorial or two and got the general idea” to “solved a couple of exercises and/or write trivially small programs”), but for some reason none of them really stuck.

One of those times that I decided I would try to learn a new language, I tried Clojure. I had read some hype about it but remembered Common Lisp as annoying so I was sceptical. Although this is probably extremely unfair, and I don’t really have any experience with any Lisp that is not Common Lisp (and then again that was only a bit of uni stuff), I got this impression that Clojure had all the good bits of Lisp while avoiding a lot of stuff that really bothered me.

So, in case you have avoided Clojure because you (think you) hate Lisp, you should know that:

• Clojure makes the syntax a bit easier to scan (because it uses other characters, like [] and {}, for some data structures) while keeping all the advantages of “code = data”.

• Another way it which Clojure feels more modern is function names: let*, setf, car, cdr, and others I hated are not there. To be fair this is both subjective and might be exclusive of Common Lisp.

• As it runs on the JVM, there are many, many things you can take for granted, like available libraries, and well-known, documented and relatively sane way to install new libraries and figure out what is available.

• Leiningen is a really nice tool for “project management” (run the project, install dependencies, run tests, etc.), so don’t be afraid if you hate Maven/Ant, the authors of Leiningen did, too grin

• Clojure really insists on using immutable data structures, and has some very, very cool implementation for all the basic data structures that allow immutability while having excellent performance (in a nutshell, different copies share all the common items). Of course you do have mutable versions of those data structures for special cases, but they’re rarely needed.

• There is a thing called ClojureScript, which is a language very, very similar to Clojure (exactly the same for most stuff) that compiles to Javascript, so you can use Clojure for both the front- and back-end of web applications. This is actually one of the reasons that convinced me to try Clojure, although I haven’t really used ClojureScript yet.

• My impression is that it has more IDEs to choose from that the average Lisp: apart from VIM and Emacs, you have Eclipse, Lighttable, and many others.

If you’re interested in learning Clojure, the book I used was Clojure Programming, which I found really nice and informative, and I totally recommend. Although I haven’t completely groked all the concepts yet because I haven’t had the chance to use them in real settings, I have a basic understanding of most Clojure stuff and I know I can go back to the book and re-read parts I need.

And while I haven’t really written anything big in Clojure yet, I have had some fun learning it making two (very) small projects:

• cludoku, a simple sudoku solver made to learn more than anything.

• clj-flickr-memories, a small utility that fetches pictures from Flickr and sends them via e-mail. The idea is that is picks photos that were taken several years ago, “on a day like this”.

I’m looking forward to using and learning more Clojure, and hopefully using it at work someday…

• clojure
• functional programming
• jvm
• programming
• programming languages

Javascript for n00bs

Recently a friend asked me for an easy way to learn JavaScript. I can’t remember how I learned myself, but I do remember some things that were surprising coming from other languages, or that I had guessed wrong or took me a while to understand for whatever reason, despite not really being complicated at all.

As I wrote the list for her anyway, I figured it could be useful for other people, too, so here it goes, somewhat edited:

• It’s object-oriented, but it doesn’t have classes (instead, it’s prototype-based). The Java-like syntax often makes you think it’s a normal object-oriented language, but it’s not.

• What are called “objects” in Javascript are hashes, really. It just happens that the values for some of the keys are functions. Related: object.prop and object['prop'] are equivalent.

• The for (x in obj) { ... } statement is strictly an object (not an array) thing. If you need to traverse an array with a “for” loop, you have to use the C-style form: for (var i = 0, len = obj.length; i < len; i++) { ... }.

• There are no methods in the sense of other programming languages. When you do object.blah(...) in JS, you’re simply calling the blah function with the “context” (ie. the value of the reserved word “this”) set. For example, you could do var f = obj.blah; f(...). That would (a) be perfectly legal, and (b) set “this” inside that function call to be undefined, not obj. The “call” function allows you to set that binding explicitly, like so: f.call(obj, ...).

• This implies something that I find ugly: if you have two levels of functions, you need to save the value of the this of the outer function in some variable if you want to use it in the inner function. See example below:

• JavaScript has functional language features: functions are first class citizens, and higher-order functions are possible and even common.

• You should use the operators === and !== instead of == and != (the latter are too liberal with type casting).

• You should get used to always using semicolon at the end. Although the interpreter adds them in most cases, in some other cases it might be a big surprise.

• Also, JavaScript is full of small design problems and quirks. Learn them, learn how to avoid them, and you use tools like “jshint” or “jslint”. That way it’s much easier to enjoy the language :-)

• Maybe I’m weird, but I have always found programming “inside” the browser to be awkward and clunky. To fool around with the language, you might want to use Node instead.

Example of two levels of functions:

// Imagine this function is a method
function foo(widgetList) {
    // Save the value of "this" for later
    var that = this;
    widgetList.forEach(function() {
        // Here, "this" points to widgetList. If we need
        // the object the "foo" method was called on, we
        // need to refer to "that", saved above
    }, widgetList);
}

Resources (that I haven’t read myself, but seem useful):

• JavaScript Jump Start for C++ programmers.

• JavaScript Guide in the Mozilla Developer Network.

• Javascript: the Good Parts (or anything by Doug Crockford I guess).

• javascript
• learning
• node
• programming
• resources
• web

Exceptions in Node

Whoa, boy. It seems I haven’t written for a good while now. Let’s fix that. One of the things I had in my list of possible posts was my experiments (and frustrations) with Javascript exception classes under Node, so here we go:

I needed to have several exception classes in Javascript (concretely, for RoboHydra, which works under Node). My first attempt looked something like this:

function NaiveException(name, message) {
 Error.call(this, message);
 this.name = name;
 this.message = message;
}
NaiveException.prototype = new Error();

That seemed to work well, except that the stacktrace generated by such a class doesn’t contain the correct name or the message (notice how I even try to set the message after inheriting from Error, to no avail). My second-ish attempt was to try and cheat in the constructor, and not inherit but return the Error object instead:

function ReturnErrorException(name, message) {
    var e = Error.call(this, message);
    e.name = name;
    return e;
}
ReturnErrorException.prototype = new Error();

That did fix the stacktrace problem, but breaks instanceof as the object will be of class Error, not ReturnErrorException. That was kind of a big deal for me, so I kept trying different things until I arrived at this monster:

function WeirdButWorksException(name, message) {
    var e = new Error(message);
    e.name = name;
    this.stack = e.stack;
    this.name = name;
    this.message = message;
}
WeirdButWorksException.prototype = new Error();

This is the only code that seems to do what I want (except that the stack trace is slightly wrong, as it contains an extra line that shouldn’t be there). I tried in both Node 0.6 and Node 0.8 and the behaviour seems to be the same in both. In case you’re interested, here’s my testing code showing the behaviour of the different approaches:

// NO WORKY (breaks stacktrace)
function NaiveException(name, message) {
    Error.call(this, message);
    this.name = name;
    this.message = message;
}
NaiveException.prototype = new Error();
 
// NO WORKY (breaks instanceof; also stacktrace w/ 2 extra lines)
function ReturnErrorException(name, message) {
    var e = Error.call(this, message);
    e.name = name;
    return e;
}
ReturnErrorException.prototype = new Error();
 
// WORKS (but has extra stacktrace line)
function WeirdButWorksException(name, message) {
    var e = new Error(message);
    e.name = name;
    this.stack = e.stack;
    this.name = name;
    this.message = message;
}
WeirdButWorksException.prototype = new Error();
 
[NaiveException,
 ReturnErrorException,
 WeirdButWorksException].forEach(function(eClass) {
    var e = new eClass('foo', 'bar');
 
    console.log(e.stack);
    console.log(e instanceof eClass);
    console.log(e instanceof Error);
});

It feels really strange to have to do this to get more or less proper exceptions under Node, so I wonder if I’m doing anything wrong. Any tips, pointers or ideas welcome!

• exceptions
• javascript
• node
• programming

Pragmatic Thinking & Learning, Wikis and Javascript

After so much “slacking” (just posting book summaries) I’m trying to go back to regular blogging. Remember my summary of Pragmatic Thinking & Learning? There are many exercises and pieces of advice in that book that I have been trying to practice. One of the things I decided to go for was having a personal wiki. One of the reasons being, in all honesty, that I had always wanted to have one. Another reason being that my pet TODO application, Bubug, had finally died after some Debian update (some retarded Ruby module broke compatibility with the version I was using, or something; couldn’t care to investigate). And yet another reason, well, to have a new small pet project and follow my obsession with learning Javascript, and especially Node. And that I wanted to give Joyent’s free Node service a try!

But enough with the reasons. It’s starting to look like it was a pretty useful mini-project. Not just because I learned a bit more Javascript, the excellent express web development framework and other things, but also because the result itself, even though it didn’t take long to develop (and it was pretty fun, even!), feels useful. It feels like a nice place to put all my notes, TODOs, random ideas for projects, etc. A similar feeling of freedom as when I started using my first Moleskine. Not that I would ditch paper for computer-_anything_, but it’s useful and freeing in its own way, for specific purposes.

About the technology, I used the Markdown format for the pages thanks to the markdown-js library (it’s really nice that the module has an intermediate tree format that you can parse to add your own stuff before converting to HTML, like e.g. wikipage links!), express for the whole application structure and js-test-driver + jsautotest + a bit of syntax sugar from Sinon.js for the tests (but looking forward to trying out Buster.js when it’s released!). The deployment to Joyent’s Node.js SmartMachine was reasonably easy. Actually, it was pretty easy once I figured the following:

• You must not forget to listen in the correct port, with server.listen(process.env.PORT || 8001)

• There are a couple of pretty useful Node.js-related command-line utilities to check logs, restart applications and so on

• The configuration of the application can be done via npm config, see npm integration on Joyent’s Wiki

If you’re curious to see the code, play with it or use it yourself, take a peek to the Wiki-Toki repository on GitHub. Happy hacking!

• books
• development
• github
• javascript
• markdown
• pet
• programming
• projects
• wikis

Book review: Javascript Web Applications

This is my review of “Javascript Web Applications” by Alex MacCaw, part of the O’Reilly Blogger Review Program (in a nutshell: you can choose an ebook from a given selection, and you get it for free if you make a review and post it in any consumer site). It’s a book about using Javascript to write (mostly) client-side web applications. The book cover says “jQuery Developers’ Guide to Moving State to the Client”, which is somewhat misleading: although most examples that could be written with jQuery are written with jQuery, it’s a book that anyone interested in Javascript can use, enjoy and learn from, regardless of their library of choice. It doesn’t even assume you know jQuery, and there’s a whole appendix dedicated to introducing the reader to the library, should she need it.

Structure

The book teaches how to write web applications using Javascript, always following the MVC pattern. It’s divided in four parts:

1. The first two chapters serve as an introduction to both the MVC pattern and the Javascript language. Although this book is not aimed at total Javascript newbies, you don’t have to know that much to follow the book. For example, it explains prototypes and constructor functions.

2. Chapters 3 to 5 cover the implementation details of MVC in Javascript (one chapter for the Model, another for the Controller and the last one about the View).

3. Chapters 6 to 10 cover many practicalities of client-side web development, like dependency management, unit testing, debugging, interesting browser APIs and deployment tips.

4. The last three chapters cover Javascript libraries: Spine, Backbone and JavascriptMVC.

Additionally, there are three appendices covering jQuery, Less and CSS3.

Highlights and references

• Chapter 10 (“Deploying”) is full of very good tips and information.

• Both the Backbone and the JavascriptMVC chapters were brilliant, looking forward to use any of them soon.

• All the example code is on GitHub.

• Page 24: “The secret to making large Javascript applications is not make large Javascript applications”.

• HJS plugin for jQuery for a nice syntax to create classes.

• ES5-shim for browsers that don’t support Ecmascript 5 yet.

• Chapter 2 was a very good introduction about events. removeEventListener (p. 41), stopPropagation/preventDefault (p. 43), list of properties (p. 44), load vs. DOMContentLoaded (p. 45), delegating events (p. 46) and custom events (p. 47-49), among others.

• Reference to blog post about namespacing.

• Object.create discussed on page 55.

• Using URL hash for URLs on pages 82, 83.

• Didn’t really understand the explanation for the HTML5 history API on p. 85. Alternatively, see the HTML5 history API on Dev Opera.

• Very interesting file API on p. 103 and p. 111. Forget the drag-n-drop (reason) and the copy/paste.

• Tips about when to load your Javascript on p. 156.

• The JavascriptMVC chapter was brilliant, see p. 208-213 for the class syntax (nicer and more compact, supports this._super()), p. 210 for instrospection and namespaces, p. 211, 212 for model attributes and observables, and p. 213 for setters. Very cool server encapsulation on p. 215. Type conversion and CRUD events on p. 218. JMVC views on p. 219. Templated actions and final example on p. 226-228.

Note that all page references are pages in the PDF file, not pages in the book!

Wrapping up

This book is packed with very practical information and a lot of code that will teach you how to write applications in Javascript. It builds up from relatively simple code to more advanced stuff, including tips, use of libraries, etc. It’s one of those books that makes you want to play with all the stuff you’re learning, and try it all in your next project.

However, sometimes the amount of code makes the book hard to read. Some parts (eg. beginning of the chapter about controllers) are a bit tiring as you have to read and understand so much code, esp. if you’re not that used to reading more-or-less advanced Javascript. It also lacks information about some important tools like Dragonfly (it almost feels like there’s nothing for developing with Opera) or js-test-driver.

In summary, this is the perfect book if you know a bit of Javascript and want to learn modern techniques and libraries that will get you started in serious client-side programming. Especially if you are one of those server-side programmers that don’t like Javascript but has to use it anyway (because despite all its warts, it’s a really nice language!). If you’re a Javascript wizard and you have been developing client-side code for years, this book may not be for you.

• books
• javascript
• programming
• review
• web

First steps with Scala

So I had promised to write a bit more about my initial experience with Scala. Here it is.

In my previous post I had explained why Scala in the first place, and I had mentioned that almost all of my knowledge comes from having read a bit of “Programming Scala”. Some of my highlights:

• It has a REPL for experiments.

• Immutable variables (see “Variable Declarations”).

• Type inference (see “Inferring Type Information”).

• The Option type (see “Option, Some, and None: Avoiding nulls”).

• The handy import syntax (see “Importing Types and Their Members”).

• The flexible syntax that allows for DSLs (see “Methods Without Parentheses and Dots” and “Domain-Specific Languages”).

• The flexible for loop (see “Scala for Comprehensions”).

• Pattern matching (see “Pattern Matching”, and make sure you don’t miss the examples with types, sequences, tuples with guards or case classes).

• Traits (see chapter 4, “Traits”).

• The nice XML library that comes with Scala (see chapter 10, “Herding XML in Scala”).

And note that I haven’t read that much of the book. In particular, I expect to like a lot of things about the functional aspects of Scala (described in chapter 8, “Functional Programming in Scala”).

That’s about learning the basics of the language. When I tried to make a small application to access the Flickr API, things worked really smoothly, which is always encouraging: using scalaj-http for the HTTP requests is a breeze, and parsing the resulting XML to get the interesting pieces was also pretty straightforward. Another nice surprise was the ScalaTest library.

Only when I had to start writing a bit more “real-world” code, I had to use some Java libraries. And I have to say, that was by far the worst part of programming in Scala. I only had to make some very simple date calculations, but that turned out to take more or less as long as the rest of the code I had written. Or at least, much more frustration. To write that small piece of code, I had to learn some API that didn’t make any sense to me; it took me a while to find the right, non-deprecated way of doing things; and all along I felt that the designers of that API were more focused on how proud they were of their “correct”, decoupled design that on making it simple and practical for the actual programmers using that library. My impression of the JavaMail library wasn’t actually much better, but at least the first thing I copied and pasted worked well enough.

And before I finish, I wanted to mention a couple of things about tools. Although I’m currently using Emacs, I did give Eclipse and NetBeans a try. Probably those tools are not for me, so take my experience with a pinch of salt, but I found them really confusing or they didn’t work at all for some reason. However, Scala mode and ensime for Emacs worked well for my, for now, limited needs, and frankly, I’d rather stay with Emacs that having to edit code in Eclipse or some other editor.

I still have several things pending, like finishing the Scala book, trying out sbt, experiment more with ensime and the Scala mode and write a library to access the Opera Link API. But it looks like it’s going to be a lot of fun :-)

• languages
• programming
• scala

The quest to learn a new programming language

Some time ago I realised I wasn’t all that excited about any programming language. All the languages I knew were, for some reason or another, annoying and I didn’t really feel like having any pet projects. That, combined with the idea that learning new stuff is good, pushed me to try and learn some new programming language.

Choosing one was actually kind of a problem: it had to be “mainstream” enough that it wouldn’t just be a theoretical exercise (I wanted to use it to write actual code, not just to learn it) and yet different enough to what I was used to. It also had to be “stable” enough that I didn’t have to install it from sources or follow the development of the compiler/interpreter. That didn’t really leave me a lot of options, I thought. The only ones I could think of, really, were Haskell, Go, Lisp and Scala.

Haskell I had tried to learn, and I more or less failed. I did learn the basics and I tried to start writing a Sudoku solver with it, but I got demoralised quite quickly. I felt it was a bit too exotic for actual work, and it was a bit of a downer that it took me so long to write some simple unit tests for some basic code I started to write (I couldn’t get my head around the type system and I was fighting with something really silly for many hours). Go, well, I didn’t even start learning because the Go installation instructions totally freaked me out. Not that I didn’t understand them, but the idea of fetching the sources of the compiler just to learn a programming language turned me off. And don’t get me started with the name of the compiler (dependent on the architecture, no less!), the separate linking step or the 90s-style “.out” file name for the binaries. So that left me with Lisp and Scala.

Lisp, I did know a bit. I had learned some basic Lisp at the university, and liked it quite a bit back then. I had also read part of the excellent “On Lisp” and I thought it was really cool. I still had my doubts I could use for actual work, but I was willing to give it a try. So I borrowed the (also excellent) book “Land of Lisp” from a friend and started reading, trying to learn a bit. The process wasn’t too bad, but I had the increasing feeling that in the end it would be too exotic for me, and I found the syntax too annoying. I was learning some practical Lisp, but it was taking really long to learn interesting things. And when those interesting things came, I felt they were too obscure for me, and I needed a lot of thinking to even understand the examples. But I decided to give it a try anyway, and I went ahead and tried to write some simple code to use some web service (the final goal was to write some example code for the Link API). In this case, the deal breaker was that the OAuth library I found depended on an obscene number of Lisp packages, many of which were recommended to be downloaded directly from GitHub (srsly? fuck this shit).

That left me with Scala. I had mixed feelings about it. At a first glance it looked interesting, but it was compiled, related to Java and more or less centred on concurrency. I tried to learn Scala by reading “Programming Scala”, which turned out to be more fun and productive than I had anticipated. I’m considering buying the “dead tree” version, but I have so many books to read that I don’t know when I’ll do that. So, what did I like about Scala so much? It made me feel like when I learned Ruby: it had fairly powerful features (pattern matching, traits, type inference, others I forget about) but with a readable, easy to understand syntax. It’s like the Robin Hood of programming languages, stealing features only available in impossible-to-understand languages, and bringing them to the masses. It also felt good liking a statically typed language, I didn’t think that was possible for me anymore :-)

But enough for now. Some other day I’ll write some more details about Scala and about my pet project FlickrMemories.

• functional
• go
• haskell
• languages
• lisp
• programming
• scala

Sucky Typo update

The other day I was talking about upgrading Typo. The update itself went well, true, and the site was up and running without too much downtime, but then I started using it again… and I have noticed two things so far (both about writing posts) that I really dislike:

First, the good old editor is not there anymore: the Typo editor used to be really good, because on the left hand side you had a very reliable and easy to use textarea with Wiki syntax (you can choose which exact syntax you want), and on the right hand side you had a “live preview” of your post, automatically updated with Ajax, that showed you how the post was going to look like. Well, that’s gone. Now there are two options: some retarded WYSIWYG box, that I tried to use and failed, and some good old textarea… without the damn live preview. That sucks big time, because there is no other preview (that I have seen: please enlighten me if there is indeed one), so I just blindly write things in a Wiki format, and hope that it’s going to look OK when I press “Publish”.

Second, I was playing with the Wiki format for the articles, and I changed it to “Markdown” (I always mix “Textile” with “Markdown”, and never remember which is which; the one I prefer is Textile). After I hit “Save”, not only the next article was parsed in Markdown format by default, but every single blog post. It’s like, you select the parser the system is going to use to interpret your whole blog. How retarded is that? Once you have written posts, it doesn’t make sense to change their syntax (unless you do it manually editing the post itself). Clearly the format is a property of each blog post, not of the whole blog installation.

Not everything is bad though: it seems that now you finally have a “Draft” concept, so I can start writing a blog post and just save as a draft, instead of unticking the “Online” property and saving as a normal post. Also, the drafts are saved automatically, so I don’t have to remember to hit “Save” from time to time just in case the browser crashes or I hit something stupid and erase the contents of the post. Yay for that.

• design
• programming
• rants
• typo
• upgrading
• woes