Music Explorer

Lately I’ve worked on several small projects, mostly to learn new technologies. The newest one is music-related: a piano that shows scales and chords “in context”, to learn and explore music theory. The idea came about because my first instrument was the guitar, and music theory is pretty hard to make sense of when you’re playing the instrument. It’s just too hard to remember all the notes you’re playing, let alone realise when two chords in the same song are repeating notes because those notes might be played in different positions (eg. one chord might use E on the open sixth string, and another might use E on the second fret of the fourth string).

I remembered that when I started playing around with a piano, and I could figure out how to play a couple of chords, it was painfully obvious that they were repeating notes because they are in the same positions. In the same way, it felt much more natural and easier to figure out on a piano which chords fitted a scale, so I decided to write Music Explorer, and ended up even buying music-explorer.org to host it. I don’t have a particularly grand plan for it, but I’ll probably add at least some small improvements here and there.

If you are interested in the technical side of it, Music Explorer is written in JavaScript (EcmaScript 6, to be more exact). I learned to use React with JavaScript, Browserify, Sass/SCSS, Ramda, AVA, the fantastic JavaScript music library teoria and a bit more CSS while writing this so it was definitely a useful learning experience for me. The full source code lives in GitHub and it’s licensed under the MIT license so you can go have a look!

• chords
• es6
• free software
• javascript
• music
• music theory
• open source
• programming
• projects
• react
• scales
• web
• webapp

Pet projects

I’ve been writing several pet projects in the last months. I wrote them mostly to learn new languages, techniques or libraries, and I’m unsure as to how much I’ll use them at all. Not that it matters. All three are related to role-playing games. In case you’re interested:

• Character suite: a program to help create characters for RPGs. It’s designed to make it easy to add new rule systems for new games. It’s written in Clojure and ClojureScript, and with it I learned devcards, macros, Clojure’s core.async, figwheel and PDF generation with PDFBox. The code is messy in parts, but I learned a lot and I have a usable result, which is the important thing.

• Nim-DR: a tiny command-line program to roll dice, including adding aliases for rolls (eg. alias “pc” for “1d100”). It doesn’t even support mixing kinds of dice, or adding numbers to the result. I mostly wrote it to get a taste of Nim. While I’m not a big fan of statically typed languages, the type inference really helped, and I liked what I saw. I may try more experiments in Nim in the future.

• Map discoverer: a program to uncover a map bit by bit. You can use it online. I wrote it to learn ES6, the new JavaScript standard, and a bit about the HTML5 canvas element. I used Babel to translate ES6 to regular JavaScript (I could have used Node 4, but Babel supports much more ES6) and es6-features.org as a reference to learn the new bits. I really liked ES6: it still feels like JavaScript, but it improves things here and there. I just wished let didn’t allow re-assignment of variables.

While writing the last one I even learned about the pointer-events CSS property, which allows you to mark an element as not receiving events like mouseclick, mousemove, etc. Very useful!

• clojure
• clojurescript
• es6
• functional programming
• javascript
• nim
• programming
• programming languages

Javascript for n00bs

Recently a friend asked me for an easy way to learn JavaScript. I can’t remember how I learned myself, but I do remember some things that were surprising coming from other languages, or that I had guessed wrong or took me a while to understand for whatever reason, despite not really being complicated at all.

As I wrote the list for her anyway, I figured it could be useful for other people, too, so here it goes, somewhat edited:

• It’s object-oriented, but it doesn’t have classes (instead, it’s prototype-based). The Java-like syntax often makes you think it’s a normal object-oriented language, but it’s not.

• What are called “objects” in Javascript are hashes, really. It just happens that the values for some of the keys are functions. Related: object.prop and object['prop'] are equivalent.

• The for (x in obj) { ... } statement is strictly an object (not an array) thing. If you need to traverse an array with a “for” loop, you have to use the C-style form: for (var i = 0, len = obj.length; i < len; i++) { ... }.

• There are no methods in the sense of other programming languages. When you do object.blah(...) in JS, you’re simply calling the blah function with the “context” (ie. the value of the reserved word “this”) set. For example, you could do var f = obj.blah; f(...). That would (a) be perfectly legal, and (b) set “this” inside that function call to be undefined, not obj. The “call” function allows you to set that binding explicitly, like so: f.call(obj, ...).

• This implies something that I find ugly: if you have two levels of functions, you need to save the value of the this of the outer function in some variable if you want to use it in the inner function. See example below:

• JavaScript has functional language features: functions are first class citizens, and higher-order functions are possible and even common.

• You should use the operators === and !== instead of == and != (the latter are too liberal with type casting).

• You should get used to always using semicolon at the end. Although the interpreter adds them in most cases, in some other cases it might be a big surprise.

• Also, JavaScript is full of small design problems and quirks. Learn them, learn how to avoid them, and you use tools like “jshint” or “jslint”. That way it’s much easier to enjoy the language :-)

• Maybe I’m weird, but I have always found programming “inside” the browser to be awkward and clunky. To fool around with the language, you might want to use Node instead.

Example of two levels of functions:

// Imagine this function is a method
function foo(widgetList) {
    // Save the value of "this" for later
    var that = this;
    widgetList.forEach(function() {
        // Here, "this" points to widgetList. If we need
        // the object the "foo" method was called on, we
        // need to refer to "that", saved above
    }, widgetList);
}

Resources (that I haven’t read myself, but seem useful):

• JavaScript Jump Start for C++ programmers.

• JavaScript Guide in the Mozilla Developer Network.

• Javascript: the Good Parts (or anything by Doug Crockford I guess).

• javascript
• learning
• node
• programming
• resources
• web

Exceptions in Node

Whoa, boy. It seems I haven’t written for a good while now. Let’s fix that. One of the things I had in my list of possible posts was my experiments (and frustrations) with Javascript exception classes under Node, so here we go:

I needed to have several exception classes in Javascript (concretely, for RoboHydra, which works under Node). My first attempt looked something like this:

function NaiveException(name, message) {
 Error.call(this, message);
 this.name = name;
 this.message = message;
}
NaiveException.prototype = new Error();

That seemed to work well, except that the stacktrace generated by such a class doesn’t contain the correct name or the message (notice how I even try to set the message after inheriting from Error, to no avail). My second-ish attempt was to try and cheat in the constructor, and not inherit but return the Error object instead:

function ReturnErrorException(name, message) {
    var e = Error.call(this, message);
    e.name = name;
    return e;
}
ReturnErrorException.prototype = new Error();

That did fix the stacktrace problem, but breaks instanceof as the object will be of class Error, not ReturnErrorException. That was kind of a big deal for me, so I kept trying different things until I arrived at this monster:

function WeirdButWorksException(name, message) {
    var e = new Error(message);
    e.name = name;
    this.stack = e.stack;
    this.name = name;
    this.message = message;
}
WeirdButWorksException.prototype = new Error();

This is the only code that seems to do what I want (except that the stack trace is slightly wrong, as it contains an extra line that shouldn’t be there). I tried in both Node 0.6 and Node 0.8 and the behaviour seems to be the same in both. In case you’re interested, here’s my testing code showing the behaviour of the different approaches:

// NO WORKY (breaks stacktrace)
function NaiveException(name, message) {
    Error.call(this, message);
    this.name = name;
    this.message = message;
}
NaiveException.prototype = new Error();
 
// NO WORKY (breaks instanceof; also stacktrace w/ 2 extra lines)
function ReturnErrorException(name, message) {
    var e = Error.call(this, message);
    e.name = name;
    return e;
}
ReturnErrorException.prototype = new Error();
 
// WORKS (but has extra stacktrace line)
function WeirdButWorksException(name, message) {
    var e = new Error(message);
    e.name = name;
    this.stack = e.stack;
    this.name = name;
    this.message = message;
}
WeirdButWorksException.prototype = new Error();
 
[NaiveException,
 ReturnErrorException,
 WeirdButWorksException].forEach(function(eClass) {
    var e = new eClass('foo', 'bar');
 
    console.log(e.stack);
    console.log(e instanceof eClass);
    console.log(e instanceof Error);
});

It feels really strange to have to do this to get more or less proper exceptions under Node, so I wonder if I’m doing anything wrong. Any tips, pointers or ideas welcome!

• exceptions
• javascript
• node
• programming

Pragmatic Thinking & Learning, Wikis and Javascript

After so much “slacking” (just posting book summaries) I’m trying to go back to regular blogging. Remember my summary of Pragmatic Thinking & Learning? There are many exercises and pieces of advice in that book that I have been trying to practice. One of the things I decided to go for was having a personal wiki. One of the reasons being, in all honesty, that I had always wanted to have one. Another reason being that my pet TODO application, Bubug, had finally died after some Debian update (some retarded Ruby module broke compatibility with the version I was using, or something; couldn’t care to investigate). And yet another reason, well, to have a new small pet project and follow my obsession with learning Javascript, and especially Node. And that I wanted to give Joyent’s free Node service a try!

But enough with the reasons. It’s starting to look like it was a pretty useful mini-project. Not just because I learned a bit more Javascript, the excellent express web development framework and other things, but also because the result itself, even though it didn’t take long to develop (and it was pretty fun, even!), feels useful. It feels like a nice place to put all my notes, TODOs, random ideas for projects, etc. A similar feeling of freedom as when I started using my first Moleskine. Not that I would ditch paper for computer-_anything_, but it’s useful and freeing in its own way, for specific purposes.

About the technology, I used the Markdown format for the pages thanks to the markdown-js library (it’s really nice that the module has an intermediate tree format that you can parse to add your own stuff before converting to HTML, like e.g. wikipage links!), express for the whole application structure and js-test-driver + jsautotest + a bit of syntax sugar from Sinon.js for the tests (but looking forward to trying out Buster.js when it’s released!). The deployment to Joyent’s Node.js SmartMachine was reasonably easy. Actually, it was pretty easy once I figured the following:

• You must not forget to listen in the correct port, with server.listen(process.env.PORT || 8001)

• There are a couple of pretty useful Node.js-related command-line utilities to check logs, restart applications and so on

• The configuration of the application can be done via npm config, see npm integration on Joyent’s Wiki

If you’re curious to see the code, play with it or use it yourself, take a peek to the Wiki-Toki repository on GitHub. Happy hacking!

• books
• development
• github
• javascript
• markdown
• pet
• programming
• projects
• wikis

Book review: Javascript Web Applications

This is my review of “Javascript Web Applications” by Alex MacCaw, part of the O’Reilly Blogger Review Program (in a nutshell: you can choose an ebook from a given selection, and you get it for free if you make a review and post it in any consumer site). It’s a book about using Javascript to write (mostly) client-side web applications. The book cover says “jQuery Developers’ Guide to Moving State to the Client”, which is somewhat misleading: although most examples that could be written with jQuery are written with jQuery, it’s a book that anyone interested in Javascript can use, enjoy and learn from, regardless of their library of choice. It doesn’t even assume you know jQuery, and there’s a whole appendix dedicated to introducing the reader to the library, should she need it.

Structure

The book teaches how to write web applications using Javascript, always following the MVC pattern. It’s divided in four parts:

1. The first two chapters serve as an introduction to both the MVC pattern and the Javascript language. Although this book is not aimed at total Javascript newbies, you don’t have to know that much to follow the book. For example, it explains prototypes and constructor functions.

2. Chapters 3 to 5 cover the implementation details of MVC in Javascript (one chapter for the Model, another for the Controller and the last one about the View).

3. Chapters 6 to 10 cover many practicalities of client-side web development, like dependency management, unit testing, debugging, interesting browser APIs and deployment tips.

4. The last three chapters cover Javascript libraries: Spine, Backbone and JavascriptMVC.

Additionally, there are three appendices covering jQuery, Less and CSS3.

Highlights and references

• Chapter 10 (“Deploying”) is full of very good tips and information.

• Both the Backbone and the JavascriptMVC chapters were brilliant, looking forward to use any of them soon.

• All the example code is on GitHub.

• Page 24: “The secret to making large Javascript applications is not make large Javascript applications”.

• HJS plugin for jQuery for a nice syntax to create classes.

• ES5-shim for browsers that don’t support Ecmascript 5 yet.

• Chapter 2 was a very good introduction about events. removeEventListener (p. 41), stopPropagation/preventDefault (p. 43), list of properties (p. 44), load vs. DOMContentLoaded (p. 45), delegating events (p. 46) and custom events (p. 47-49), among others.

• Reference to blog post about namespacing.

• Object.create discussed on page 55.

• Using URL hash for URLs on pages 82, 83.

• Didn’t really understand the explanation for the HTML5 history API on p. 85. Alternatively, see the HTML5 history API on Dev Opera.

• Very interesting file API on p. 103 and p. 111. Forget the drag-n-drop (reason) and the copy/paste.

• Tips about when to load your Javascript on p. 156.

• The JavascriptMVC chapter was brilliant, see p. 208-213 for the class syntax (nicer and more compact, supports this._super()), p. 210 for instrospection and namespaces, p. 211, 212 for model attributes and observables, and p. 213 for setters. Very cool server encapsulation on p. 215. Type conversion and CRUD events on p. 218. JMVC views on p. 219. Templated actions and final example on p. 226-228.

Note that all page references are pages in the PDF file, not pages in the book!

Wrapping up

This book is packed with very practical information and a lot of code that will teach you how to write applications in Javascript. It builds up from relatively simple code to more advanced stuff, including tips, use of libraries, etc. It’s one of those books that makes you want to play with all the stuff you’re learning, and try it all in your next project.

However, sometimes the amount of code makes the book hard to read. Some parts (eg. beginning of the chapter about controllers) are a bit tiring as you have to read and understand so much code, esp. if you’re not that used to reading more-or-less advanced Javascript. It also lacks information about some important tools like Dragonfly (it almost feels like there’s nothing for developing with Opera) or js-test-driver.

In summary, this is the perfect book if you know a bit of Javascript and want to learn modern techniques and libraries that will get you started in serious client-side programming. Especially if you are one of those server-side programmers that don’t like Javascript but has to use it anyway (because despite all its warts, it’s a really nice language!). If you’re a Javascript wizard and you have been developing client-side code for years, this book may not be for you.

• books
• javascript
• programming
• review
• web

LeakFeed and

A couple of week ago I discovered LeakFeed, an API to fetch cables from Wikileaks. I immediately thought it would be cool to play a bit with it and create some kind of application. After a couple of failed ideas that didn’t really take off, I decided to exploit my current enthusiasm for Javascript and build something without a server. Other advantages were that I knew Angular, an HTML “power up” written in Javascript (what else?), which I knew it would ease the whole process a lot, and I even got the chance to learn how to use Twitter’s excellent Bootstrap HTML and CSS toolkit.

What I decided to build is a very simple interface to search for leaked cables. I called it LeakyLeaks (see the code on GitHub). Unfortunately the LeakFeed API is quite limited, so I had to limit my original idea. However, I think the result is kind of neat, especially considering the little effort. To build it, I started writing support classes and functions using Test-Driven Development with Jasmine. Once I had that basic functionality up and running I started building the interface with Bootstrap and, at that point, integrating the data from LeakFeed with Angular was so easy it’s almost ridiculous. And as LeakFeed can return the data using JSONP, I didn’t even need a server: all my application is simply a static HTML file with some Javascript code.

All this get-data-from-somewhere-and-display-it is astonishingly simple in Angular. There’s this functionality (“resources”) to declare sources of external data: you define the URLs and methods to get the data from those resources, and then simply call some methods to fetch the data. E.g.: you can get the list of all tags in LeakFeed from http://api.leakfeed.com/v1/cables/tags.json (adding a GET parameter callback with a function name if you want JSONP). Similarly, you can get the list of all offices in LeakFeed from http://api.leakfeed.com/v1/cables/offices.json. In Angular, you can declare a resource to get all this information like this:

this.LeakFeedResourceProxy = $resource(
 'http://api.leakfeed.com/v1/cables/:id.json',
 { callback: 'JSON_CALLBACK' },
 { getTags:    {method: 'JSON', isArray: true, params: {id: 'tags'}},
   getOffices: {method: 'JSON', isArray: true, params: {id: 'offices'}}}
);

Once you have declared it, using it is as simple as calling the appropriate method on the object. That is, you can get the tags by calling this.LeakFeedResourceProxy.getTags(), and the offices by calling this.LeakFeedResourceProxy.getOffices(). And when I say “get the tags”, I mean get a list of Javascript objects: no JSON, text or processing involved. If you assign the result of those functions to any property (say, this.availableOffices), you’ll be able to show that information like so (the |officename is a custom filter to show the office names with a special format):

<select id="office" name="office"> \
  <option value=""> ()</option> \
</select>

The cool thing is, thanks to Angular’s data binding, anytime the value of that variable changes, the representation in the HTML changes, too. That is, if you assign another value to this.availableOffices the select box will be automatically updated to have the new set of options! But the data binding is two-way, so any changes you make in the UI also get reflected back in the Javascript variables. This further simplifies many tasks and makes programming with Angular a breeze. There are other nice things about Angular (and many nice things about Bootstrap and Jasmine of course!), but I think that’s enough for today :-)

• angular
• automated tests
• bootstrap
• css
• html
• jasmine
• javascript
• tdd
• tests

Facebook privacy scanner (ReclaimPrivacy)

Summary: there’s a simple tool that will tell you which Facebook sharing options are “too open” in your account. I’d like you to help me by trying it out and telling me what you think (if you had problems using it, if you would like extra/other information to be shown, if you found any bugs, etc.). Skip to “how to use it” below if you’re not interested in the details for developers. Thanks!

Some time ago I discovered a neat Javascript tool called ReclaimPrivacy. It was a very simple program that scanned your Facebook privacy settings and told you if you had “too open” settings so you could review and fix them. I really liked the tool and thought it was a great idea, but after Facebook changed the layout of the privacy settings, the tool stopped working.

Weeks passed and the tool didn’t get any update, so I decided to step in and try to help the original programmer adapt the tool so it worked again. The ReclaimPrivacy code is in GitHub so it was pretty easy to make my own fork and start hacking away. It didn’t take me long to adapt the first things to the new privacy settings layout, and after some more time I was much more comfortable with the code, had made more things work, added tests and even added new features. Now that it’s starting to get close to something we could release as the new official ReclaimPrivacy version, I’d like your feedback.

How to use it: add a new bookmark for this link. You usually just have to drag and drop it to your browser toolbar, or alternatively add a new bookmark (typically you can do that by pressing Ctrl-D) and make sure the address is the above link. Go to the ReclaimPrivacy help page if you have trouble (but use my link, not the one provided there!). Once you have the bookmark, go to Facebook and click on the bookmark. It will show you some information about your Facebook privacy settings on top of the page. Just leave a comment here or drop me an e-mail with your opinion, thanks! You can skip the rest of the post if you are not interested in Javascript programming and/or software automated testing ;-)

During my hacking I made a lot of different changes: I split the source file into several different files, I made the code (more) testable, I added tests, and I added more features. I’m really into testing and testability, so one of the first things I did with the code was trying to decouple it from the network calls so I could write tests for it. As you may know, I think that code that doesn’t have tests is very hard to work with, and I even consider it’s not “true code”. Now, I’m no Javascript expert, so some of my techniques might not be very… idiomatic. That said, some of the code change highlights you may be interested in:

• The getInformationDropdownSettings method, renamed to getSettingInformation, is now shorter, more readable, more testable and has more features. The changes are: (1) making it receive an object with the relevant part of the DOM, instead of a window object; (2) supporting, in principle, any kind of setting, not only dropdowns; (3) allowing each setting to have its own idea of what “too open” means (see the settings array); (4) allowing the caller of the method to specify its own list of recognised settings and acceptable privacy levels; (5) passing the number of open and total sections to the handler, instead of just a boolean stating whether or not there’s any “too open” setting.

• I made the old getUrlForV2Section more testable by extracting the most interesting (read: likely to break or need maintenance) code to its own method, _extractUrlsFromPrivacySettingsPage, and making the new getUrlForV2Section work with both real URLs (checking Facebook with an Ajax call) and fake HTML dumps representing what those URLs would return.

• I made the old withFramedPageOnFacebook, a very important method used in several places, more flexible by accepting not just URLs, but also functions or data structures (new withFramedPageOnFacebook).

• Now we have some basic tests (with fixtures even), without which doing some of these changes would have been such a pain, I wouldn’t have bothered making them in the first place.

http://github.com/emanchado/reclaimprivacy/blob/master/javascripts/utils.js#LID42

• facebook
• javascript
• privacy
• refactoring
• testability
• testing

Recent pet projects + Git + Github

I had mentioned that I was learning Javascript to write a Kiva Opera widget. Some time ago I released the first version of my World Loanmeter widget, and I have uploaded two more since. Not much has happened between the first and the third release from the user POV, but a couple of things were interesting when developing it:

• I learned QUnit, which I used to write some really useful unit tests. It’s quite nice to be able to write Javascript unit tests easily.

• I made some heavy refactoring (see above) which made me learn some more Javascript and made the code much more flexible, so now the widget is not limited to a single Kiva API page of results, but to as many pages as needed to fetch whatever number of loans the user wants. Not to mention that the data source need not be a URL.

• Now the widget actually has some configuration. Namely, the number of loans to show in the map. It also stores it persistently using the preference store, which is quite nice.

As I said, I used Git for it. I don’t “hate” it anymore, but I still find some things annoying, like the horrible, confusing names some options have (I’m thinking about “git checkout “ to revert the local changes, or “git diff –cached” to see the contents of the index/staging area; seriously guys, W-T-F?). I used to be skeptical about the “git add” for changes and then “git commit”, but I actually find it quite nice: it’s easier to plan a commit that way, and if you don’t want to plan it, you can always just “git commit “ directly. Also “git add -p” is really nice to commit just parts of a file (at last, someone copies some of the good stuff Darcs had had for ages!). Apart from Git itself, it’s cool that there is GitHub, so it’s easy to share your repositories without having to rsync to some web server or similar… not to mention that your project is much more visible that way.

But the World Loanmeter wasn’t the only pet project I was working on these past weeks: I also wrote a simple sudoku solver, demisus, in Ruby. The reason? Writing a prototype of a sudoku solver in a language I’m fluent with, to play with the design and get something interesting and easy to maintain… to rewrite it in Haskell. I have been trying to learn some functional language for some years now, but I never find a “project” that is interesting enough to write some “real world program” in the language and I end up not learning anything. After starting reading Real World Haskell, I really felt like trying to learn the language once and for all, and I figured that a sudoku solver was easy enough to write, something I know enough about, and something math-y enough to be reasonably easy to implement in Haskell.

So, if you’re interested in any of them, you can have a look in Github and even contribute ;-)

• darcs
• demisus
• git
• github
• haskell
• javascript
• kiva
• loanmeter
• opera
• Ruby
• sudoku
• widget