Hi, [This is the big JS & friends mail, as promised. It took forever to write, and got longer than I expected, for which I apologise in advance.]
I've been doing some thinking (and discussing) recently, and there are things which I'd like to know but I couldn't find a clear answer, or indeed any answer at all. It's possible that they haven't been asked before. Note that they're all pretty high-level and not necessarily concerning any single piece of code; rather they are about "what does Weblocks want to be?". The first question is: what is the ultimate goal, and how much of what we have today is set in stone? Ie. is it "be the best way to write webapps we can achieve", and thus if a new great idea arises that needs changes in the fundamentals we change them, or is it pretty similar to what we have today, and so big changes are to be avoided? I will assume it's the former for now. Now, I believe the general idea of "the best way possible" is that the way the traditional GUI apps on the desktop are written works pretty well, and that's where we look for inspiration. So what we want is to give the programmer the "single GUI app" model of programming, which we take as strictly better[1] than the "loose collection of pages split between the client and the server", aka. the web model. However, we still fall short in several ways. We do a pretty good job of hiding the client/server separation as long as the stock widgets are used. But that breaks down as soon as a custom widget is needed; then we're immediately back to dealing with raw HTML. Automatically dirtying widgets helps, but that's the extent of it. There's no language to speak about data and no language to speak about the client- side: either you resign yourself to the very crude automatic AJAX consisting of "send me new HTML", or do more sophisticated manual JS manipulations, but then you're on your own in ensuring the thing still works without JS. There's no established protocol (that I know of) for doing this kind of glue. There are problems with both options. The manual glue's shortcomings are pretty obvious: we're exactly where we were before Weblocks existed. However, the "thin" (aka crude) AJAX is also suboptimal -- it pretty much turns the client into a kind of a remote desktop viewer, limiting us to little more than whatever the server can do in the returned HTML, and making every transition dependent on the server. This has been tried before by Rails guys, and they discovered pretty quickly that it didn't scale. That's of course not a big problem for many websites, but nevertheless it puts a strict upper bound on how successful you can be with Weblocks, and how smart your AJAX can get. Even worse, the JS we have today is completely unorganised and basically completely ad-hoc. Last I checked there was still inline JS being constructed as strings. To solve that, I believe we first need to agree on what we want possible. Personally, I think the following properties are valuable in whatever system we come up with: 1. Continues to function as it does today as long as you don't step outside of the existing widgets. Still no thinking about HTML or JS involved. 2. Can get the "thin" AJAX for free. 3. Continues to degrade gracefully for non-JS clients. 4. Can get semi-smart AJAX for free or very cheap. 5. Fully smart AJAX is easy/possible to do and with a protocol that makes it obvious how to proceed without necessarily knowing the exact structure of the HTML generated. 6. Transparent to the programmer -- no need to write separate client- side and server side versions of the same code. #1-#2 are simply no regression from today. #3 is particularly important for several reasons. One is that I believe in (and occasionally do) browsing without Javascript. Another is that this strict separation keeps us from committing one of the most hateful sins of the web, namely reimplementing (crappily) what browsers already do better (the typical example being reimplementing <a href=...> with onClicks, which prevents tabs from being used). And lastly, it forces us to develop a cleaner and more modular approach, which ultimately leads to better abstractions. #4 needs explaining what "semi-smart" means. I mean by this the kind of JS that results (using a widget we're working on right now as an example) in the HTML for a full month grid for August out of JSON data containing the year and month number, number of days in the month and the first week day. #5 is a step above from that -- the kind of full JS apps a'la GMail which only communicate with the server using a non- HTML protocol and do all rendering on the client. It won't be trivial to get that, but ideally it should be possible and easier than just doing it by hand. It should also be obvious where to hook in if you want to go down that route and reasonably easy to keep it functioning in non-JS environment. Lastly, #6 is what it's all about -- we should maintain the illusion of the single app inside the widget code, the same way widgets and continuations do it for application code. The remaining part of this mail is concentrated on how to make it possible. I should say up-front that I don't have a clear vision or a surefire way to solve everything, but I do have some ideas gathered here and there. One of them which will no doubt come into play is some kind of a data and transformation language. I've seen several interesting projects in this regard. One of them is WUI (http://common-lisp.net/project/cl- dwim/, unfortunately no better info/demo page is available right now), with an interesting possibility of exporting to CSV/XLS/PDF built into every component. Another is the Core Server (http://labs.core.gen.tr), with its DEFMETHOD/LOCAL and DEFMETHOD/REMOTE. I must admit that I haven't looked into them in enough detail to be able to tell how exactly they deal with things like what is allowed in the transformed code, etc. It seems clear, however, that you can't just allow the full language inside, since that turns into a problem of arbitrary complexity. One way to tackle at least #4 above would be to introduce delimited "data transformation" blocks which'd take a list of all variables they depend on / close over them automatically, and then using a specified subset of CL inside, get transformed into two versions: AJAX request + rendering in JS and full HTML response generated in ordinary CL. Something along the lines of: (with-data-block (first-weekday days-in-month month year) ; This results in a JSON block of {year: 2009, ; month: 8, daysInMonth: 31, firstWeekday: 1} ; being sent to the client (iter (for i from 0 to 5) (htm (:tr :class "view" (iter (for j from 0 to 6) (let* ((current-day (- (+ (* i 7) j 1) first- weekday))) (if (or ...)))))))) The point here being that this doesn't merely generate the HTML and send it to the client, but instead defines an operation twice, on both the client and the server, which then gets invoked as needed depending on whether AJAX is active or not, and if it is, generates the HTML locally. This would be a significant shift from our current dumb HTML receiver approach, yet shouldn't be excessively hard to achieve. The above approach should allow us to define transitions that do not require contacting the server at all. This is another significant boost compared to the current situation. One bump this introduces is that now we can't rely on the server and client state being synchronised at all times, and that AJAX operations might result in fewer actions being called on the server than non-AJAX ones, so we need a protocol to inform the server about what happened the next time we talk to it, but it shouldn't be that hard if all transitions are expressed in terms of data blocks which are known to both parties. Another benefit of thinking in data blocks is that if their reprenstation was sufficiently general, it could be a stepping stone towards the above-mentioned non-HTML renderers a'la WUI, especially when coupled with the views rewrite. Once we get to that point, it should be possible to progress to #5, but that's something better left for once we are there, this mail is long enough already. I apologise for the somewhat unstructured form, but it's a bit difficult to express all my thoughts in a clear and organised way. Hopefully it was readable enough to get the basic idea across; I'll be awaiting your feedback and comments. Cheers, Maciej [1] With the single exception of bookmarking, ie. the web's ability to name and point to various places inside the app. But the new nav system largely addresses this point already. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "weblocks" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/weblocks?hl=en -~----------~----~----~----~------~----~------~--~---
