Bye Flickr, hi trovebox!

For some time now I’ve become more and more interested in running my own services and using less and less external services. The latest has been Flickr, which I had been a Pro member of for over 8 years now (yikes!). I used it less and less, and was more wary of uploading pictures to it, so I thought it made sense to just paying for it. However, there was one thing I was still using Flickr for, a program I wrote called clj-flickr-memories (now clj-photo-memories): every week, that program searches for pictures taken this-week-several-years-ago and sends you an e-mail with the results, if any.

The first possible alternative I found was MediaGoblin, but the API is too basic and it doesn’t even save the picture details (such as the date the photo was taken) in the database so I couldn’t even improve the API to support that. I was close to giving up when I found Trovebox: it’s a hosted service you can pay for, like Flickr, but it’s also open source so you can host it yourself if you want. And although its API documentation leaves a bit to be desired, it could do what I wanted, so I got cracking and modified my clj-flickr-memories to support both Flickr and Trovebox.

If you want to switch from Flickr to a self-hosted Trovebox and want to import your photos, there are two things you should know:

  1. If you self-host and use HTTPS (and you should!) you need to include the “https://” both in the “host” key in the command-line tool configuration file and in the Android app.
  2. You can easily import all your Flickr photos in two steps: first you use export-flickr to fetch information about your Flickr photos (only works with Flickr Pro accounts, though!) and second you use import to import those photos into Trovebox. Note that the first step will leave a directory “fetched/” with one file per photo, so you can choose which photos to import to Trovebox, eg. based on the date.

Trovebox also has mobile apps, but at least the Android one requires internet access to it’s not great for me (you can’t browse photos offline).

Book summary: The Information Diet (III)

This is the third and last part of my summary of “The Information Diet” by Clay Johnson (you can also read the first and second parts). This part covers part III: Social Obesity.

Social Obesity

The only way we can solve the problem of information obesity is to change the economics of information (the information that is the worst for us is the easiest/cheapest to obtain), because they have changed in a way that not only stupid people are getting duped anymore. We need to demand an end to factory-farmed content, and demonstrate a willingness to pay for content like investigative journalism and a strong, independent public press.

Ideas: share this book; organize in infogroups; focus and be civil, keeping focus on the goal of improving digital literacy; meet face to face; learn, eg. from the reports by Knight Comission; act, producing useful outcomes in your local communities, including children.

Participation gap

The participation gap is the gap between people and the mechanics of power in their governments. Its cause is our desire to focus on large, emotionally resonant issues over practical problems that can be solved. Related to this is the “sportsification” of politics, which makes us treat elections like athletic rivalries, vilifying the other team at the expense of doing what’s right.

The first cause is scale: the underlying structures of government aren’t designed to handle our present population. Transparency is overrated as solution to this, plus it has disadvantages like allowing dishonest people to appear honest. Two big lessons about this: (1) there’s a gigantic gap between the skills to win an election and the skills to govern a country, and (2) many of the nonprofits and advocacy groups are more interested in staying relevant than solving problems (as a result, these advocacy groups tend to focus on larger problems that can go unsolved for years; also, after working for one such group, the author assures that online petitions are not meant, primarily, to cause change, but to get your email address so that you can later be bombarded by emails asking for money).

Start sweating the small stuff at the expense of some of the big stuff. If you’re interested in making government more accountable, work on making it so that the government’s listening tools and policies are modernized. Every issue has hundreds of small, nonpolitical, operational problems. Fixing these can have a huge impact compared to combating a vague foreverwar.

Special note: Dear Programmer

Programmers: take your role in society seriously.  Dedicate some portion of your time to issues you care about. You needn’t ask for permission to do this, or wait for a nonprofit or advocacy group to ask you donate your time (and while it’s useful to partner with organizations, it’s likely that they’re more interested in your skills to help them fundraise than they are to solve problems). This isn’t a call for you to build apps for your favourite nonprofit, because unless you’re willing to support and maintain each application, and help constantly ensure its usage and adoption, you’re wasting your time.

And this is it. As for my review of the book, I got to say I was a bit disappointed by it. It was much more focused on American politics than I expected, which sometimes made it hard to relate to. There are definitely many interesting parts in this book, and a fair amount of food for thought, but some of the advice feels pretty demanding: it feels like it’s enough to keep certain things in mind and make certain changes without really measuring everything objectively (I have the impression that the author has a much bigger faith than me in numbers, measurement and “objectivity”). In summary, good book, but not as universal or life-changing for me as I had hoped.

Book summary: The Information Diet (II)

This is the second part of my summary of “The Information Diet” by Clay Johnson (you can also read the first part). This part covers part II: The Information Diet.

EDIT: the third and final part is finally published.

The Information Diet

First of all: fasting is not dieting. It’s good to disconnect, but unplugging is just a way to avoid our bad habits. Second: the diet is based on the author’s experience, and is not backed by science. Third: it’s a list of recommendations, and every person has to find what works for them. Summary: Consume deliberately. Information over affirmation.

The author coins the term infovegan, a person that consumes consciously. This requires knowing where to get appropriate data and what to do with it. Check the ingredients of “processed information” (when reading news on a new medicare proposal, take a look at the bill itself). It’s also a moral choice: opting out of a system that’s at least morally questionable, shunning factory farmed information, politically charged affirmations, and choosing to support organizations providing information consumers with source-level information and containing more truth than point-of-view.

Data Literacy

Our concept of literacy changes with every major IT shift. Now, filtering and sorting through all the available information is very valuable. Proposal for a modern data literacy:

  1. Know how to search: not just Google and Bing, but specific engines for patents, scientific papers, laws, budgets (eg. USASpending.gov), etc.
  2. Know how to filter and process: need to find the most reliable and accurate information sources, and learn how to process them with tools like spreadsheets, or else we’re unable to draw accurate conclusions.
  3. Know how to produce: knowing how to publish information (text, audio or video) and the ability to take feedback are both critical skills.
  4. Know how to synthesize: we must be able to synthesize the ideas and concepts of others back into our ideas.

The Diet

First of all, figure out how much information you’re consuming daily (the average is 11 hours). A long-term goal could be to reduce that to 6, turning the rest into information production, social time with friends, exercising, etc. Things to avoid:

  • Mass affirmation: avoid the suppliers that make a living telling you how right you are. Eg. no more than 30 minutes a day of mass affirmation. For liberals, that’d mean choosing between Stephen Colbert and Jon Stewart.
  • Overprocessed information: consume locally or try to remove distance to the things that you investigate.
  • Advertisements: the economics of advertisement-based media results in sensationalism. We have to reward our honest, nutritious content providers with financial success.
  • Our own fanaticism: keep an eye on your own fanaticism and challenge your beliefs. Keep a list of stuff you find to be absolute, like firm positions and values, and look to find data and people that challenge your biases, prescribing yourself enough time to encounter them.

And this is the end of the second part of my summary. The next one will be the last, covering Part III: Social Obesity.

Book summary: The Information Diet (I)

This is the first part of my summary of “The Information Diet” by Clay Johnson, which I got through the O’Reilly Reader Reviews program. The book is about the information we consume, and by drawing parallels to food diets, come up with ways to be consume information in a more concious and healthier way. The book is very focused on American politics, but can be applied to other topics. This part of the summary only covers the first third of the book, the introduction.

EDIT: parts two and three are now published.

Introduction

As we’re hard wired to love salt, sugars, and fats, we’re also hard wired to love affirmation and the confirmation of our beliefs. Food companies learned to sell a lot of cheap calories, by packing them with salt, fat, and sugar. And media companies learned that affirmation sells a lot better than information. Driven by profits, they produce information as cheaply as possible. As a result, they provide affirmation and sensationalism over balanced information.

When viewed through the health lens, the information abundance problem appears to be a matter of health and survival. The first step is realizing that there is a choice involved.

The media is using technology to figure out what it is that people want, and finding the fastest way to give it to them. Eg. Huffington Post shows two headlines during the first 5 minutes and keeps the one that got more clicks, and AOL’s policy says that each editor should use four factors to decide what to cover: traffic potential, revenue potential, turn-around time, and lastly, editorial quality. All editorial content staff are expected to write 5 to 10 stories per day.

“Information obesity” is what makes people not know basic facts or believe falsehoods. This doesn’t stem from a lack of information, but from a new kind of ignorance that results in the selection and consumption of information that is demonstrably wrong. We don’t trust “the news” but we do trust “our news”. Tobacco companies have figured out that “Doubt is our product since it is the best means of competing with the ‘body of fact’ that exists in the mind of the general public”. Information obesity has three flavors:

  • Agnotology: manufactured ignorance and culturally induced doubt, particularly through the production of seemingly factual data. The more informed someone is, the more hardened their beliefs become, whether or not they’re right.
  • Epistemic closure: dismissal of any information that doesn’t come from a network of interconnected and cross promoting media because it comes from “the other side”, and is therefore ipso facto not to be trusted (how do you know they’re liberal? Well, they disagree with the conservative media!).
  • Filter failure:  The friends we choose and the places we go all give us a new kind of bubble within which to consume information.

The next part will cover “Part II: The Information Diet”.

Role-playing games and drawing

In the last couple of months I have become obsessed with role-playing games again. I used to play a lot in my teens and some in my early twenties, and then completely stopped until around one year ago. Some months ago I decided to be the narrator (as opposed to regular player) and the obsession kicked back in *grin*. The game I chose was Call of Cthulhu, partly because I was familiar with the setting (stories written by H.P. Lovecraft set in the 1920s) and with the rules, and partly because I think it’s a very flexible game, as in you can centre the games on combat, on investigation, or, my favourite, on emotional tension and social interaction between characters.

In any case, all this Call of Cthulhu, the 1920s and such sparked my imagination and served as a pretty good inspiration to draw characters, different expressions and poses, clothes, hairstyles, etc. As you may know I’m still trying to learn how to “draw properly”, meaning something better than stick figures, so I need the inspiration and the practice. I’ve been drawing a lot lately and finally I decided to make a website with my drawings so that I can see how I improve over time. Some of my favourite drawings follow, but you can see all of them in the new http://drawings.hcoder.org.

Cthulhu-inspired drawings

Cthulhu-inspired drawings

And finally, if you’re interested in trying out role-playing games and Call of Cthulhu in particular, Chaosium has published a quick-start guide for the upcoming 7th edition of the game that you can download for free from their website!

Book review: Clojure Cookbook

I just finished “Clojure Cookbook“, a book full of recipes for all kinds of things you can do with the Clojure programming language. I got the book through O’Reilly’s Reader Review Program. Obviously the book is not for you if you don’t know any Clojure, but I think it’s great if you know some Clojure but still wonder how to do relatively basic tasks, or want to see how common libraries are used, or want to see idiomatic Clojure solutions to everyday problems.

The book is divided in different sections ranging from basic data structures to network I/O, performance and production tips or testing. Each section has a number of recipes in the form of small problems together with solutions and a detailed explanation of the solution, as well as caveats or things you should take into account when solving that kind of problem. In a book like this is inevitable that you’re not going to care much about certain sections (in my case, I didn’t care much for distributed computation and I don’t find database access that exciting), but in general the book is very clear, the explanations concise, the code to the point and the recipes useful and varied.

In a nutshell, very good if you know some Clojure but want to learn more about how to solve everyday programming problems elegantly with Clojure.

Digital Rights Management on the web

I strongly dislike Digital Rights Management (DRM), and often refuse to buy certain things simply on the grounds of them having DRM. So, what I do have against DRM? For the sake of clarity, I’ll assume we’re talking about videos/films (but all these arguments apply to any kind of content):

  • Any platform (= “device” or “operating system”) not supported by the producer will not have a legal way to watch the video: as DRM needs special video players (that enforce certain conditions, like not allowing you to share the film with other people), you can only watch the film if there’s a player for your platform.
  • Because of the point above, free software is left out almost by definition (eg. under Linux, DVDs have their protection cracked in order to watch them because there’s no legal player; this is not only cumbersome, but illegal in some countries even if you have bought the DVD; Netflix is not officially available on Linux, although there are workarounds).
  • It gives unnecessary control to the producers of the content (eg. Amazon can delete books you have bought from your Kindle; you maybe not be able to lend the film/book to a friend; many DVDs don’t allow you to watch the film directly without watching the trailers first).
  • It’s often inconvenient for the paying customer, plus it doesn’t even work to fight piracy (music examples / film examples).

So, can’t you just not buy any DRM-“protected” films/books if you don’t like DRM? That’s more or less what I do, but I’m worried now that the w3c is planning on introducing DRM as part of the web, which will encourage more companies to use DRM. And I said “encourage”, not “allow”, because it’s perfectly possible to do so now (eg. Netflix does it). As I’m opposed to DRM, I think it’s ok that’s it’s painful or awkward to implement: there’s no need to make something bad easier and more common.

Some pro-DRM people seem to have this misconception that people who oppose DRM want the content to be free of charge. That’s ridiculous. I want to pay for the content, I just want to watch/read that content in whatever way is comfortable for me, and in whatever devices/operating systems I happen to use. In fact, I do spend a lot of money on music (but only when it’s DRM-free).

The only pro-DRM arguments I could take seriously were brought by the BBC (hat-tip to Bruce Lawson for sending me the link). They have very good points in the sense that in many cases it’s not even their choice to offer unlimited access. The thing is, again, that:

  1. If we agree that DRM (eg. limiting the content to certain kinds of devices) is bad, why make it easier? It’s already possible, let’s not make that the default if we think it’s bad. Keeping thing that are bad-but-necessary-right-now hard, but possible, sounds like a good strategy to me…
  2. Non-supported platforms would be excluded, why make it easier to have content on the web that discriminates against certain people (eg. people who have non-mainstream devices, free software)?

Finally, the EFF has a page about owning vs. renting that talks about other reasons why I don’t like DRM.

Making comics

Almost one year ago now I finished reading “See What I Mean” (summary part 1, part 2), the book that got me started making comics. Now I’m reading the excellent “Drawing Words & Writing Pictures” and I’m learning much more about techniques and visual language. The first book I recommend if you are intrigued by the idea of making comics but always thought you couldn’t because you can’t draw, and also if you are intrigued by the idea of using comics at work (eg. see my comics explaining different use cases for my project RoboHydra). The second I recommend if you have some artistic background and/or after reading the first.

I thought it would be fun to document the way I’m making comics now, hence this post.

Idea

You start with an idea of what you want to tell, then divide it in pages. So far I’ve worked knowing the amount of pages beforehand, but I guess you could just fill pages until you’re done telling the story. The key here is that you have to think of pages because you need to know how they begin and end (eg. you don’t normally end a page in the middle of some action), and because the last page has to be full, you can’t finish halfway!

In this example, I wrote down the story and divided the scenes into the six pages I was going to use. This is also the time to design the characters, but I didn’t do it until after the thumbnails because I only discovered I could draw after I had finished them. What I did do was quick studies for the locations:

locations

Thumbnails

Once you have the story divided in pages, you need to design every page: decide the amount, shape and position of the panels, the action, the text, etc. This serves several different purposes:

  1. Confirms that you can tell the story the way you thought.
  2. Gives you a way to check the rhythm of the story and see if it works.
  3. Lets you play with the shape and size of the panels (in this case I went with a pretty conservative grid; the only exception is the title).
  4. Sets the more or less final text in each panel.
  5. Gives you a way to decide the art for each panel (composition, perspective, etc) before you spend a lot of time making the final art.

As this is the first time I worked with thumbnails, some things are not quite right: the text is sometimes quite different from the final version, some panels on the first page are quite different (I realised they didn’t work in the thumbnails and instead of reworking them I made the changes directly in the final version), and the art is completely different. This last bit is actually due to the fact that I originally planned to make this comic with stick figures, but after finishing the thumbnails I realised that I could draw better and decided to give it a go.

You can see all the thumbnails here:

squash1 squash2 squash3 squash4 squash5 squash6

Final comic

When you’re happy with the thumbnails, you’ll have a pretty good idea of how the comic is, and the only step left is to make the final art. I use a Wacom Bamboo drawing tablet and I don’t have high standards for the final result (as you’ll see *grin*), but for people making “real” comics the process is a bit more involved, as they need to do pencils first, then ink.

You can see the final result of my comic here:

sc1 sc2 sc3 sc4 sc5 sc6

You can compare each page between the thumbnail and the final result, for fun, to see how much it has changed. However, remember that the difference between thumbnails and final comic should not be that big! The only reason why the thumbnails are stick figures and the final comic it better is that I realised too late that I could draw better than stick figures.

In any case, I hope you enjoyed this post. As I said, if you’re interested in making comics but thought you couldn’t do it, go read “See What I Mean”.

Book summary: Coding Freedom

These are my notes for “Coding Freedom“, a sociology/anthropology book that analyses the free software community. You can download it for free from its website, or buy a paper version. These notes cover only the history of free software, which I found very interesting even if I basically knew it already.

1970-1984: The commodification of software

During the 1960s and part of the 1970s, most hardware was sold with software and there was no software patent or copyright protection. Programmers in university labs routinely read and modified the computer source code of software produced by others.

In 1976, just as companies began to assert copyrights over software, Gates wrote a letter to a group of hobbyists chastising them for, as he saw it, stealing his software: they had freely distributed copies of Gates’ BASIC interpreter at one of their meetings.

In the late 1970s and early 1980s the US software industries dominated internationally. Amid fears of losing ground to foreigners, US legislators launched an aggressive campaign to develop and fund the high-tech and knowledge economic sector and encountered little friction when accepting software patents in 1980.

1984-1991: Hacking and its discontents

In the late 1970s and the 1980s, corporations started to deny university-based hackers access to the source code to their corporate software, even if the hackers only intended to use it for personal or noncommercial use. This infuriated Richard Stallman, who became a “revenge programmer” (whole, fascinating story in p.68) and ultimately founded the Free Software Foundation in 1985 and then wrote the first draft of the General Public License in 1989. In 1984 he actually said “I am the last survivor of a dead culture. And I don’t really belong in the world anymore. And in some ways I feel like I ought to be dead”.

1991-1998: Silent Revolutions

Trade groups intensified their efforts to change intellectual property law largely through international treaties. They worked with law enforcement to strike against “pirates”, pursued civil court remedies against copyright infringers, launched moral education campaigns about the evils of piracy, and pushed aggressively for the inclusion of intellectual property provisions in the multilateral trade treaties of the 1990s. For example, through TRIPS, patents had to ultimately be open to all technological fields.

In the meantime, Linux would gain momentum in companies: managers would say they were not using Linux, but techies would say “yes… but don’t tell my boss”.

1998-2004: Triumph of open source and ominous DMCA

The term “open source” (less philosophical and more technical) was created and won, and the DMCA was passed, which criminalised all attempts to circumvent access control measures (ie. DRM), practically giving copyright owners technological control over digitized copyright material.

Misc final notes

For most developers, acceptance of free software rarely led to political opposition producers of proprietary software, but made them develop a critical eye toward practices such as abuse of intellectual property law and tendency to hide problems from costumers: “Free software encourages active participation. Coporate software encourages consumption”.

One of the most profound political effects of free software has been to weaken the hegemonic status of intellectual property law; copyright and patents now have company.

And that’s it. I hope you enjoy it. Go download the book if it sounds interesting or you want to learn more about hacker culture and free software.

Learning Clojure

I have always had a thing for functional programming. Not so much for functional languages maybe, but definitely for functional programming principles. I had learned a tiny bit of Lisp at the uni and have fond memories of reading “On Lisp” by Paul Graham, but I never really tried to learn any Lisp for real. Or, I tried to learn Common Lisp again by reading “Land of Lisp” but gave up very quickly. I have tried to learn other functional languages with varying degrees of success (ranging from “read a tutorial or two and got the general idea” to “solved a couple of exercises and/or write trivially small programs”), but for some reason none of them really stuck.

One of those times that I decided I would try to learn a new language, I tried Clojure. I had read some hype about it but remembered Common Lisp as annoying so I was sceptical. Although this is probably extremely unfair, and I don’t really have any experience with any Lisp that is not Common Lisp (and then again that was only a bit of uni stuff), I got this impression that Clojure had all the good bits of Lisp while avoiding a lot of stuff that really bothered me.

So, in case you have avoided Clojure because you (think you) hate Lisp, you should know that:

  • Clojure makes the syntax a bit easier to scan (because it uses other characters, like [] and {}, for some data structures) while keeping all the advantages of “code = data”.
  • Another way it which Clojure feels more modern is function names: let*, setf, car, cdr, and others I hated are not there. To be fair this is both subjective and might be exclusive of Common Lisp.
  • As it runs on the JVM, there are many, many things you can take for granted, like available libraries, and well-known, documented and relatively sane way to install new libraries and figure out what is available.
  • Leiningen is a really nice tool for “project management” (run the project, install dependencies, run tests, etc.), so don’t be afraid if you hate Maven/Ant, the authors of Leinigen did, too *grin*
  • Clojure really insists on using immutable data structures, and has some very, very cool implementation for all the basic data structures that allow immutability while having excellent performance (in a nutshell, different copies share all the common items). Of course you do have mutable versions of those data structures for special cases, but they’re rarely needed.
  • There is a thing called ClojureScript, which is a language very, very similar to Clojure (exactly the same for most stuff) that compiles to Javascript, so you can use Clojure for both the front- and back-end of web applications. This is actually one of the reasons that convinced me to try Clojure, although I haven’t really used ClojureScript yet.
  • My impression is that it has more IDEs to choose from that the average Lisp: apart from VIM and Emacs, you have Eclipse, Lighttable, and many others.

If you’re interested in learning Clojure, the book I used was Clojure Programming, which I found really nice and informative, and I totally recommend. Although I haven’t completely groked all the concepts yet because I haven’t had the chance to use them in real settings, I have a basic understanding of most Clojure stuff and I know I can go back to the book and re-read parts I need.

And while I haven’t really written anything big in Clojure yet, I have had some fun learning it making two (very) small projects:

  • cludoku, a simple sudoku solver made to learn more than anything.
  • clj-flickr-memories, a small utility that fetches pictures from Flickr and sends them via e-mail. The idea is that is picks photos that were taken several years ago, “on a day like this”.

I’m looking forward to using and learning more Clojure, and hopefully using it at work someday…