Yet another Clojure + Compojure + Google App Engine post

Have I mentioned I'm kinda late to this show? Well, better late than sorry. Anyway, I think you can't have too much of these, specially in times where google application engine (GAE) is growing and so is Clojure.

The scenario was like most others: I wanted to build my webapp using GAE and clojure, and also stay on the bleeding edge of each technology. So lets see what I've got. I started by taking a look at the "literature" available as blog posts and group discussions (such as this and this).

I installed lein, and created a standard project.clj (By force of habit, forgot to use "lein new", but it's practically the same). I added the following directives on defproject:

:source-path "src"
:compile-path "war/WEB-INF/classes/"
:library-path "war/WEB-INF/lib/"
;; namespaces will trigger AOT compilation to bytecode. you want this!
:namespaces       [ginasiosunderground.servlet]

My goal is to produce a war directory, to upload to gae servers. So I told lein to compile to war/WEB-INF/classes, using war/WEB-INF/lib as a classpath dir. (More to come later on that :namespaces directory). The final project.clj is the following:


You can see the list of dependencies. I'm using the latest clojure 1.2, compojure 0.4, hiccup 0.2.4, a ring 0.2 (specifically adapted to use with gae servers) and appengine-clj (from r0man's fork). I also use clojure-json (to make my life easier on jQuery client side ajax calls). I'd like to mention the libraries from google. As of now, a default lein setup can't find those up to date versions (they are available, but I didn't want to toy around with maven repositories to make the magic happen). So I got those jars from the gae java sdk (download it from gae's site) and placed them on my war/WEB-INF/lib folder. Now be ware, you should have a copy of them somewhere else, because each lein clean you do will delete them from the lib directory! (eventually I'll get around to automate this process. I also know there's a "lein gae" plugin somewhere, but haven't tried yet, I'm not sure if it does what I want). One last thing about lein, you'll have to remove commons-fileupload-1.2.1.jar from your lib dir, otherwise gae will complain about it (it did, for me!).
Let's move on, then. my-app.servlet loads some stuff I need for the servlet, generates an HttpServlet class, and imports some classes so that I may load a jetty server and use gae's services (datastore, etc) in interactive mode (read, using SLIME with swank-clojure). Here's the namespace:


The magic that's left is, after issuing the defroutes and defservice command, the following (thanks to r0man for the tip!):


So here we are, running a webapp that can be deployed to gae (if you have read the linked posts, you'll see that you also need a web.xml and a appengine-web.xml to deploy, and you'll also need to comment out the ring.adapter.jetty from your namespace - gae already has its own application server :). Now by running the local server via (start-server) and not via gae's dev_appserver.sh shell script, you can happily work, on slime, redefine your servlet actions and refresh the browser to see your changes. Yay!
One problem, though. How do I use google's services? appengine-clj to the rescue, browse the code (see the test-suite for examples, specially on how you can test them locally using macros like appengine-clj.users/with-user-info). I was basically a happy camper until I got to try out user logins. GAE has this method, createLoginURL and createLogoutURL (applied to a UserService and a destination page string), that takes us to a dummy login screen. To be able to use this Service, I wrote the following code (loaded before starting the jetty server!):


But while this compiles perfectly, I noted that the login screen is not loaded, i.e., clicking the login link yields a 404 page. Not good. But it should be loadable, since the dev_server has it. Google's documentation isn't much help here, and I can't seem to find how on earth I access that screen (I need it to try out administration pages!). So feel free to drop a comment if you happen to know the solution for this!
To conclude, I'm not going to publish my application as a "blank example", as there are already some around. That said, if you have a question about this setup, just ask and I'll be as helpful as I can!

On Clojure error messages, II: ask and ye shall receive

I now realize my previous post carried some strong emotions, most of them not too good, and I want to humbly apologize for it. Let me state (again) that I'm enjoying working with clojure (for the most part, at least).

I wrote about the useless (wrong?) error messages, specifically when passing a list instead of a vector (ISeq) as arguments for a defn. After experimenting a bit, I was even more confused: (defn x () 1) complaints about the integer, so does (defn x () [] 1) and (defn x () [1]), clojure just doesn't know how to create an ISeq from that integer. I now understand this makes sense because you can specify multiple arity functions, like this:

(defn x

  ([] (x 0)

  ([x] (+ x 1)))

So in this case, if a vector isn't found where the arguments are expected, clojure complains, as he can't make an ISeq.

But none of this matters anymore (to the language user). Stuart Halloway (author of the book that I used to learn about Programming Clojure, which by the way I advise you to read, if you haven't already) was almost to fast picking up this issue! I have just received his tweet with a fix to this problem: clojure (the snapshot version) now detects if the arguments are a proper vector, and if not warns us with a much clearer message: "Parameter declaration arg1 should be a vector". This should satisfy all with the same issue I tackled on the previous post; more advanced users dealing with multiple arity defns should already be aware of the syntax, so it's safe for both cases.

This was a positive outcome in more than one way. Not only clojure got a bit better, but I got reminded of how I should had followed a much nicer "netiquette". I made a public complaint, and Stuart was impeccable to pick it up, hacking a fix and letting me know about it (everyone should learn from him, including myself!). But instead of posting my complaint (again, sorry if my tone was to harsh), I should had (1) ask on a discussion list or on other social platform about this; (2) take a look into the source code, see where the issue could be fixed, write a patch, and submit it to approval to the developers - alternatively, take the oportunity to learn how to do those things, and at least let the maintainers know about it, so a bug report is issued. This is the organization that most projects require to evolve properly.

On Clojure error messages.

I know. I'm late to this train. And there are lots of resources to clarify this. But I have to manifest myself somewhere (and twitter's char limit won't do this time).

Here's the deal. I know I've written a fair share of Clojure lines of code. But I come from the land of the ancient parenthesis (specifically, Scheme, Common Lisp). So whenever I try to teach clojure to someone, it's inevitable to tell them that it's a Lisp, so it's ok to see those rounded delimiters everywhere.

This is where I start by asking them to define a simple function, and instead of defun/define, use defn. Something like this will do:

(defn xpto ()

  (+ 2 3))

Simple enough, right? Wrong. (I know it's wrong, now, off course). But for the beginner, the first thing to do is to read the error message. So let's take a look:

Don't know how to create ISeq from: clojure.lang.Symbol

  [Thrown class java.lang.IllegalArgumentException]

Yeeeah, isn't that obvious? Clojure wants to make a sequence and I'm trying to use a symb... wait, what?? the only symbol I know I'm using is xpto, and that's about right on the syntax! The Java exception isn't much help either. What argument? The ones I'm not using on the functions (but weren't those parameters?) Or the ones I'm using on the function call (but 2 and 3 are numbers, not strings!! *help*!) Ok, breath. let's take a look into the stack trace, it is usually helpful on java, at least to trace where the error is:

Backtrace:

  0: clojure.lang.RT.seqFrom(RT.java:469)

  1: clojure.lang.RT.seq(RT.java:450)

  2: clojure.lang.RT.first(RT.java:538)

  3: clojure.core$first.invoke(core.clj:53)

  4: clojure.core$sigs$asig__3331.invoke(core.clj:202)

  5: clojure.core$sigs.invoke(core.clj:214)

  6: clojure.core$defn.doInvoke(core.clj:268)

  7: clojure.lang.RestFn.invoke(RestFn.java:495)

  8: clojure.lang.Var.invoke(Var.java:381)

  9: clojure.lang.AFn.applyToHelper(AFn.java:180)

10: clojure.lang.Var.applyTo(Var.java:482)

11: clojure.lang.Compiler.macroexpand1(Compiler.java:5215)

12: clojure.lang.Compiler.macroexpand(Compiler.java:5270)

13: clojure.lang.Compiler.eval(Compiler.java:5338)

14: clojure.lang.Compiler.eval(Compiler.java:5320)

15: clojure.core$eval.invoke(core.clj:2366)

16: swank.commands.basic$eval_region.invoke(basic.clj:47)

17: swank.commands.basic$eval_region.invoke(basic.clj:37)

18: swank.commands.basic$eval801$interactive_eval__802.invoke(basic.clj:66)

19: clojure.lang.Var.invoke(Var.java:365)

20: myproject.core$eval3896.invoke(NO_SOURCE_FILE)

21: clojure.lang.Compiler.eval(Compiler.java:5353)

22: clojure.lang.Compiler.eval(Compiler.java:5320)

23: clojure.core$eval.invoke(core.clj:2366)

24: swank.core$eval_in_emacs_package.invoke(core.clj:94)

25: swank.core$eval_for_emacs.invoke(core.clj:241)

26: clojure.lang.Var.invoke(Var.java:373)

27: clojure.lang.AFn.applyToHelper(AFn.java:169)

28: clojure.lang.Var.applyTo(Var.java:482)

29: clojure.core$apply.invoke(core.clj:536)

30: swank.core$eval_from_control.invoke(core.clj:101)

31: swank.core$spawn_worker_thread$fn__453$fn__454.invoke(core.clj:300)

32: clojure.lang.AFn.applyToHelper(AFn.java:159)

33: clojure.lang.AFn.applyTo(AFn.java:151)

34: clojure.core$apply.invoke(core.clj:536)

35: swank.core$spawn_worker_thread$fn__453.doInvoke(core.clj:296)

36: clojure.lang.RestFn.invoke(RestFn.java:398)

37: clojure.lang.AFn.run(AFn.java:24)

38: java.lang.Thread.run(Thread.java:637)

Read it all? Want your 3 minutes back? (ok, 30 if you actually went looking for documentation on the API, the source of Compiler, core.clj or worse if you went looking for slime's or Thread :) ). Sick, isn't it? not a single line is useful for our cause.

I'm afraid I have no good news for you, except to tell you what the problem was and hope you'll never forget that. I know I didn't. (until I swapped to a bunch of different programming languages, including common lisp, for a while, and returned with my auto-pilot coding style turned on :) ).

The problem is that clojure expects a sequence of parameters, not a list. And thus, the proper syntax is to use [] instead of (). I know, a list is not a symbol. Or so I thought... I could even go theorize about () being translated to nil in compile time, and so it was a symbol. But evaluating (symbol? nil) quickly clarifies that for us...

Pledge - please, oh please start adding useful semantic to clojure's core language errors... The language entry barrier will lower significantly, I'm sure!