Clojure and Google App Engine

I recently finished an MVP version of Edinburgh Festival Roulette, the first web app I’ve written using Google App Engine and Clojure, and I thought I’d share some of my thoughts and experiences.

Clojure and Compojure were definitely a good choice; I was able to get something up and running quickly despite having limited experience with both. For HTML I used Hiccup, but in retrospect Enlive would probably would have been a better fit (although I guess the two aren’t mutually exclusive).

I’m a bit more conflicted about GAE. It definitely seems responsive enough and I certainly can’t complain about the cost. However, there are a few gotchas that Clojure developers should be aware of:

  • Leiningen with the Ring plug-in will produce an “uber war” for you that you can deploy straight into a Java servlet container such as Tomcat. But GAE isn’t quite as straightforward. To deploy to GAE you need to use their command line tool which expects a directory rather than a war, and needs one or more special files depending on your needs (everyone needs appengine-web.xml and I use a cron.xml as well). To get around this I wrote a simple shell script which unzips the directory and copies the files across.
  • GAE manages all the scaling, which sounds like a good thing – and in many ways is – but means you lose control of things you might not have expected. You need to understand how GAE will respond to latency in your app and under which conditions it will spin up a new instance.
  • It’s extremely important to be aware of which libraries and APIs you can use. There is a whitelist which contains the list of JRE classes the app will have access to. Whilst you can add your own libraries, you need to know if they are dependent on a JRE class not in the list. This hit me, as I used clj-http which doesn’t obey this rule (or rather the Apache lib it wraps doesn’t). I ended up rolling my own as a wrapper around java.net.HttpURLConnection. I don’t think you could even use the deprecated http.agent library as although it uses HttpURLConnection it falls foul of the next point…
  • You can’t do your own threading. This is a major issue. It is understandable, as Google want to have control over any background work, but it creates serious problems when working with Clojure. In my case it meant that I had to drop my idiomatic (well, reasonably idiomatic) Clojure code that used agents to update a cache, as agents are of course entirely dependent on threads. One of the aims of Clojure is to make concurrent programming as painless as possible, but GAE takes a lot of this of control away from you. Worse, you can’t use any libraries (such as http.agent) that use agents, futures, or other thread based constructs. But how do you know if a library uses threading or a non-whitelisted class? I’m not convinced GAE can be considered a safe platform for Clojure when it is so hard to tell if your code will run, or if an update to a third party library will break everything.

If you still want to use Clojure and GAE, be aware that are some nice looking libs for working with GAE and Clojure e.g. appengine-magic. I haven’t tried any of them though.

Finally, please have a look at the Edinburgh Festival Roulette site and tweet if you like it. It might not make much sense to you unless you’ve been to Edinburgh during festival time though!

3 Responses to “Clojure and Google App Engine”

  1. eternity Says:

    Related links:
    http://www.infoq.com/presentations/Clojure-in-the-Clouds
    https://github.com/slagyr/gaeshi

  2. Kevin Noonan Says:

    Hi Adrian,

    I’m a developer based in Dublin and I’m working on a side-project of
    my own in Clojure.

    You wrote:
    “I ended up rolling my own as a wrapper around java.net.HttpURLConnection.”

    Would you mind sharing that wrapper code with me? (Or posting it on
    Github, say?)

    Thanks in advance for any help you can offer!

    Kevin.

  3. Adrian Mouat Says:

    Hi Kevin,

    I sent you an e-mail with the code. I don’t think it’s worth posting here (or on github) as all I did was pretty trivial. (Just thought I’d update this page so it didn’t look like I’d ignored you!)

    Hope your project worked out well.

Leave a Reply