Monday, April 14, 2008

Moooing away in Common Lisp

I'v recently found myself needing to use a task manager to make some bulk operations (to be more precise, I wanted to copy all of my tasks to a device with no built-in synchronization mechanism for those). I happen to make use of the great Remember The Milk Web application (I've previously worked with Emacs Planner, but I had to drop it in favor of a more mobile version of my life).

Enter Common Lisp. Perhaps the fastest way to do some scripting in my mind, but I've yet to make a library to use a Web-based API. Turned out it was quite clean, and in about 5 hours of work I had the whole API grokked, coded, tested and hosted in GoogleCode here. (I looked into, but found the google's approach to hosting much more featured for the fast start I was looking for. If this has enough momentum I can look into it again, with more time - the main advantage would be asdf-installability, I suppose.)

I already had experience making project "skeletons" (asdf project file, package declaration, etc.). So the hardest thing I had to do was look into the web API call mechanism and into the md5 encryption. If you don't need the detailed description, tha answers are, respectively, Drakma and Ironclad. ( I also use cl-json to parse RTM's responses into a lisp-friendly format.)

To call a Web-based method, we need to make an HTTP request, and be able to read the response. using drakma (asdf installable and loaded in the usual way), all I had to do was something like:

(http-request rtm-api-endpoint :method :post :parameters '(("param-name" . "param-value")))

With some parameter abstractions, RTM's was easily tamed. A call to one of their methods is now made with the following macro:

(defun rtm-api-call-method (method &optional key-value-pairs &key with-timeline with-authentication (format "json"))
  "Calls `METHOD'.
 - Optionally passes pairs of strings in `KEY-VALUE-PAIRS' in the form
 - Keyword `WITH-TIMELINE', if not null, allows the method to be called within
 - Keyword `WITH-AUTHENTICATION', if not null, allows the method call to be
authenticated with a valid `*RTM-API-TOKEN*'.
 - Keyword `FORMAT' is one of
\"json\" (the default value) or \"rest\", and
specifies the server reply format."

  (declare (special *current-timeline*))
  (let* ((parameters `(("api_key"    . ,rtm-api-key)
                       ("method"     . ,method)
                       ("format"     . ,format)
                       ,@(when with-timeline
                                 `(("timeline" . ,*current-timeline*))

                       ,@(when with-authentication
                            `(("auth_token" . ,*rtm-api-token*))


         (api-sig (compute-rtm-api-sig parameters))

    (multiple-value-bind (result)
        (http-request rtm-api-endpoint
                      :method :post
                      :parameters `(
                                    ("api_sig" . ,api-sig)

      (let* ((response (json-bind (rsp) result rsp))
             (stat (assoc :stat response))

          ((string=  (cdr stat) "ok")
           (rest response)

          ((string=  (cdr stat) "fail")
           (let ((err-info (cdr (assoc :err response))))
             (error "RTM error code ~a: ~a~%" (cdr (assoc :code err-info))
                    (cdr (assoc :msg err-info))

With this (rather big, for the sake of readability, or mayhap due to my lack of experience in mental pretty printing) function, I define a call to an RTM module simply by coding:

(defun rtm-api-tasks-complete (list-id taskseries-id task-id)
  (rtm-api-call-method "rtm.tasks.complete"
                       `(("list_id"       . ,list-id)
                         ("taskseries_id" . ,taskseries-id)
                         ("task_id"       . ,task-id)

                       :with-authentication t
                       :with-timeline t

The only call in the first function definition that cannot be easily guessed how they work is the compute-rpm-api-sig-parameters. This is the function that performs the algorithm required by RTM to encode all parameters and guarantee authenticity in each request (along with a masked API key). Basically it does some mambo-jumbo concatenation with each parameter's name and value, and then performs an MD5 to get the equivalent hash code. Easy, right? But I'm a lazy programmer. That, and I truly believe there's no point in reinventing the wheel, so I looked for encryption functions in Lisp, and I found Ironclad, a collection of most of these functions in Lisp. So producing an md5 functions was as easy as the following:

(defun md5 (string)
  "MD5 uses ironclad to encode `STRING' into an hexadecimal digest string."
   (ironclad:digest-sequence :md5 (ironclad:ascii-string-to-byte-array string))

And that's about it. The rest was toying around, the API works perfectly (kudos to the RTM dev team!), and I was thrilled to have my tasks exported in a jiffy. If you want to find more info, feel free to snoop through the code (it's quite easy to grasp, I think, and relatively small), in the Google Code site: . So now, what would you, gentle reader, do with this API? Drop me some comments with your ideas.

Until the next time!


jmbr said...

Hi Edgar,

I just wanted to point out that you don't need to host your project in for asdf-installability. It suffices to write a page (with an appropriate link) in


Edgar Gonçalves said...

Thanks for the tip, I hadn't realize it was that simple before! It's done, it should be asdf-installable, I suppose!