Showing posts with label PhD. Show all posts
Showing posts with label PhD. Show all posts

Wednesday, January 28, 2009

Lisp macros, eval and packages

Today I spent about an hour trying to find out the source of a bug in the code for my PhD prototype. To make the context short, I need to have a class definition to be stored, and loaded afterwards. And I was using Gary King's time-saver metatilities:defclass* to reduce the required coding. The problem was when I wanted to load such a definition in a generic method, I was evaluating the definition expression - one of the acceptable uses of `eval', correct me if I'm wrong, is to interpret something that was just read (in this case, from a serialized string, from other servers). The code in question was something like this:

(defmacro load-data (name attributes)
  "Defines a domain concept, using metatilities:defclass* syntax."
  `(progn
     (metatilities:defclass* ,name (data-item)
       (,@attributes)
       (:metaclass profiled-metaclass)
       (:name-prefix  ,(format nil "~(~A~)" name)))
     (info "Data type loaded: ~A" ',name)))

(defmethod load-definition ((def data-item-definition))
  (eval (macroexpand `(load-data ,(definition-name def)
                                                     ,(data-item-definition-attributes def)))))

What I was getting as a result is that the auxiliary methods created as accessors by defclass* weren't being imported in my packaged, but in CL-USER. Hence, lots and lots of compiler warnings and subsequent errors when trying to use those accessors. I fiddled around, tried to see if I could remove the eval, but the solution, so obvious as I know can see, is quite simple. Just place your evaluated expression after setting the package. The fixed method is the following:

(defmethod load-definition ((def data-item-definition))
  (eval `(progn (in-package :dshow)
                ,(macroexpand `(load-data ,(definition-name def)
                                          ,(data-item-definition-attributes def))))))

Hope you can see this before spending too much time on a similar bug!

Friday, February 01, 2008

User interface paradigms for data search and offline data access

I've stumbled upon this interesting and illustrated article on data lookup user interface design patterns (on both web and desktop applications, finally we seem to be starting to forget there's a difference!):

Here are two design paradigms for handling large amounts of data, not to be confused (or combined) as web design meets desktop in rich Internet applications.

[From Seek or Show: Two Design Paradigms for Lots of Data « Theresaneil’s Weblog]

The post describes two main groups of patterns, based on the way a user chooses what he/she wants to search (pictures linked from the article mentioned above):

  • The Seek Paradigm - The user has to input a search criteria in order to access some search results.
  • The Show Paradigm - The user is presented with several search results (or categories), and he may drill-down the search by clicking on the desired categories or selecting certain results to eliminate unwanted items.

This got me thinking in terms of offline work, and I can easily qualify both approaches as orthogonal to the online vs. offline decision. It is merely a user interface option, which is a good thing. To understand this, picture a lookup for book titles on a library. To go offline, one would have the client application to fetch them all from the server, thus making both kind of interface patterns possible without contacting the server. Similarly, if we wanted to know how many clients had already rented a book (assuming the algorithm implied a run through each client's details to look for their rental - I know, it's not the smartest way to do that, but it's only an illustrative example!), the client wouldn't be able to fetch all the other clients' details, hence not being able to use any of the search paradigms.

So the effort that has to be done, from the application logic design point of view, is related to how the services (those accessed by the user interface) access the information stored in the database. Let's draw some pseudo-code* for the book rental example above (the second example).

  • Approach 1: The all-powerful service. This is the kind of service we would have written up until a few months ago, considering any sort of offline execution to be an academic dream or an edgy unfeasible mostly unsuccessful experiment. Basically it assumes that it is being executed with super-user privileges, hence able to access all stored data at any time with no constraints whatsoever.
  • Aproach 2: The helpful, aware and conscious service. This is the service that knows that its code may be executed in many places, and so it tries to make a scarce usage of the stored data as much as it can.

Using approach 1 we could code a service like this:

(define-service get-number-of-book-rentals (book)
(let ((all-clients (get-storage-data "clients"))
(result 0))
(dolist (client all-clients)
(set result (+ result
(client-book-rentals client book))))
result))

As you can see, while this would work perfectly on the server, the client would never be able to to the (get-storage-data "clients")call, given its lack of permission to fetch them all to the client-side storage. This is the sort of services that must be thought over and rewritten - or, better yet, automatically translated to something more useful!

By using approach 2, we would instead code something in the lines of:

(define-service get-number-of-book-rentals
(:inputs (book)
:storage-data (books-rent-by-all-clients (get-total-rentals-from-clients)))
(let ((number-of-books 0))
(dolist (client-rentals (filter book-rent-by-all-clients
:field "book"
:value book))
(set number-of-books (+ number-of-books client-rentals)))))
number-of-books))

Here it's clear that the service knows what data is going to be needed from the storage. With this information, the server can fetch it for the client when it wants to disconnect, and the client is now able to perform the same functionality without having to run through the clients (read, without having to have access to all clients' personal details). Relevant, noteworthy changes between the examples:

  • the service has to be declared with more information (like the storage-data requirements)
  • the storage data requirements code is always performed on the server. The client counterpart consists of a lookup on the client-side storage for the previously fetched information. This way there is no risk of information not being available.
  • If the service is to be run online, it also runs equally well, with no performance penalties.
  • There must be either a mentality change or a tool to translate services made with the approach 1 to the approach 2!

I'm already laying out some code to achieve something like this. Expect news in the following week or two.

*Disclaimer - despite being in a lisp pseudo-dialect, all examples could exist in the language of your choice. It's really just a matter of personal preference!

Thursday, January 10, 2008

The search for functionality dependency

Having had my PhD thesis proposal (previously presented here) approved by my university, it's time to move along to the next step: thorough description of the solution I propose, followed by its implementation.


The first important aspect I need to address is the grouping of functionalities that may be executed by a given user while offline. The ideal situation would be to pick up a specification of a Web application, then automatically produce a graph of available functionalities, with their dependency hierarchy. From here, the developer (or even the project engineer) could inspect what would be desirable to be performed offline, according to the requirements artifacts. Since we're also aiming for ease of use, on the produced tree, it would be nice to see the characteristics of each functionality, namely when related to access control permissions and data dependencies. This way we could gray out some of the available functionalities that could never be performed offline (e.g., we may want to force every functionality requiring a specific role to be accessed online only). By imposing restrictions, one could easily make a map of he needed functionalities. It seems almost to easy. What we're forgetting is how we're going to get the dependency graph. Yes, I intended to get those via a workflow graph. But let's face it, most Web applications are built without a prior workflow specification artifact ever seeing the light of day. Specifically, my case study doesn't have one (yet, however). So how shall I do this?

  • Write the aforementioned workflow specification by hand. This seems a lot of work to ask of the project development team, specially since all they want is some functionalities (they don't know exactly what or how many in a low-level, only the high-level use-case names are written in the requests, usually). It's also not possible to write it automatically, since it needs domain-specific information, usually more than what is already present (albeit very scattered) in the existing application. So this is a no-go.
  • Code walking. If we inspect each interface user action and the accessed data and services, we could obviously draw a graph. This however poses other problems. Are all programmers doing the right thing? It's not that uncommon to write services that check on other services without needing them, just to see some futile/temporary information, and forgetting about it later on. I know that doesn't happen in a perfect world, but "real" and "perfect" don't always pair along! But there is another show-stopping issue with this approach: we would have to start the "walk" somewhere, so we need to know all the starting functionality calls, along with the list of services itself.
  • User experience tracing. This is quite interesting in theory, if we ask users of several roles to perform their tasks, we can obviously trace the execution flow perfectly by reading the logs only. But we have no means to guarantee the users are going to access all the functionalities they are entitled to perform in our tracing period. That is only possible in a closed, controlled experience, by having users following scripts. But this kills the main purpose - those scripts are as hard to write as the workflow description itself, so it's of no use to us.
  • Functionality reification. By reengineering the application to have the functionality calls represented as objects, we can have the immediate dependencies graph automatically by scanning all user interfaces and listing the graphs of all calls triggered by buttons, links or other events (XHR based, for instance).

I'm going to put my efforts on this last one. There are other advantages with this approach, being studied by some colleagues of mine at INESC-ID and in the Fenix development team. The main ones are related to he presentation mechanism allowed - the organization of the Web application interface can be composed with such objects. I'll get into more details afterwards.


As usual, I appreciate any comments, tips or any other kind of experience sharing from you, dear reader!

Thursday, September 27, 2007

QUATIC07 - the humanistic version of the story

On the last post I mentioned my findings from QUATIC07, and specifically from the input received from the software engineering doctoral consortium workshop. Now it's time to clear out what I learned from the whole experience.



Let's start this thing to mention this is my second event of this sort. I've attended to CAPSI 05, to present a paper entitled "Desduplicação sobre um conjunto de nomes próprios" (in english, "Duplicate removal in a set of first names"). The paper was presented by Helena Galhardas, that despite being the third author - my teacher of decision support systems at the time - she had both the recognition and the experience to make a proper presentation without the beginner's jitters...). So, despite having a published paper, I hadn't present any, so far. Enter the QUATIC event. I'm now faced with the opportunity to show my PhD expectations to other students and teachers of the same area. And present a poster to the entire academia/industrial audience on the conference's hallways. And have my thesis proposal published as a paper on a IEEE proceedings! I must say I was pretty excited about the whole deal!



Day one. I arrived at the conference to present the thesis to the doctoral workshop, SEDES. The conference was in portuguese, and the first interesting fact I noticed was the geographical tendencies of the participants: I was the only one who lived/studied/worked below the northern part of portugal. The rest were all from Coimbra, Aveiro, Oporto, so Lisbon (!) was a strange city among them :) But everyone was really looking for a chance to get feedback on their work, so there wasn't much time nor motivation for picking around that! Well, after receiving my reviews, I met personally one of the reviewers, and found out he is going to be part of my PhD jury. I had lunch with all of them, and proceeded to the remainder of the thesis discussions. And that's about it.



Day 2 and 3. The poster. I placed the poster on a wall in the hallway, and got a nice desk to sit at. This way, I got to show some supporting images to some enquiring passerby. I made an english 10 minute presentation about it, to the industrial world, and got some positive feedback on the subject (the technical questions were a bit far from what I was expecting; the academia is surely more prepared to criticize my work, as it is still a novelty to the others). On the last day I got to see a teaching perspective on software engineering. Miller's presentation about TSP and PSP were most enlightening and entertaining. The day ended with a dinner in a Brazilian restaurant with live music. I got to mingle a bit with people I had met but rarely had the chance to talk to. It was fun, but I have to admit I was pretty tired. I went to bed and slept for a good 7 hours, before waking up to finish another presentation (7 hours seemed a lot, as I had been sleeping from 3 to 5 each night until then!)



Last thoughts. I had a great time, the preparation was quite helpful in organizing my mind schemes, and the feedback was the cherry on the top of the cake. But now I need to evolve my work so that I can go to a IEEE well known conference - not that this wasn't a good one, but I need some heavyweight recognition when I'm facing a fulminating jury! Advice for the PhD newcomers? Don't miss out opportunities like this! It can be expensive, obviously, but there are organizations able to fund your research and your publications and presentations!



Going back to Emacs (is there any other TeX editor worth reverting to, when you're an emacsen?)