Lazy Blogger finds Asynchronous Network Spellbook

So yeah, I haven’t posted anything in a while.  My workplace, in the wisdom of a large, unwieldy bureaucracy, blacklisted tumblr in the web proxy, and can’t seem to undo the setting even after being asked to.

So I’m stuck emailing myself a list of links at the end of every day, as if it were 1994.  I am going to start posting to tumblr via email, once I have it configured.

However, there are a couple of things that I’ve found recently that are too cool not to post.

About two weeks ago, I was doing some research on how to get node.js to talk to Neo4j.  I thought about Clojure/JRuby => Java => JNI => C++ => V8 Javascript, and immediately recoiled in horror.  I began writing a node.js wrapper for neo4j-rest, and got it all spec’d out, but then wondered if I would negate the performance gains of node.js by incurring network latency for every datastore transaction.  I pondered and researched about sockets, and was encouraged by how straightforward they seemed.  However, I would have to do lots of low-level stuff to get that to work: designing a protocol, error checking, managing concurrency and locks, etc. and it seemed like the learning curve would be up there with JNI.

Then today I found ZeroMQ.  ZeroMQ is billed as being many things: a very fast message queue, a “Scalability Layer” for the web, an async network programming library (like node.js) with many language bindings, a concurrency framework.  But it’s also just a really nice abstraction on top of BSD sockets for lazy programmers like me who like to avoid the low-level details and stick to the more interesting aspects of the business domain.  So if I wanted to, I could use Node.js/express.js as my web server for my JSON REST API (so as to stay in the Javascript paradigm for both request and response), and then speak via 0MQ to a Java/Clojure/JRuby JVM process acting as my persistence layer talking to Neo4j.

Or I could skip straight to Mongrel2 instead of Node.js, and because Mongrel2 is built on ZeroMQ, I can have any number of application components written in any language running on any number of machines all providing functionality to my webapp.  That makes me excited.  I am imagining adding lots of cool stuff to what was going to be simply a RESTful data service and some mobile and desktop clients: XMPP chat, Websocket notifications, etc.  I can even imagine a business process engine (perhaps that implementation of YAWL I’ve been pondering for a few years) that can dynamically notify participants in a variety of ways—email, chat, application pop-up notification, text message, carrier pigeon.

Of course all of this is possible using a combination of other existing technologies.  But it is nice to combine a low-config web server with this flexible network or interprocess programming fabric.  It feels like these two technologies abstract away some of the nastier aspects of systems integration, and leave me free to focus on the business problem using languages and frameworks I’m already comfortable in.  Bravo to the good folks behind these libraries.