Hi Rob,
Thanks again, I appreciate the ideas. By the way, I think you’re right -- I meant “synchronous” as in “blocking” and “asynchronous” as in “non-blocking.” And I mean this from the perspective of the server API. In other words, using the “post/redirect/get” pattern is “non-blocking” in terms of receiving output (the “processing report”) -- the client has to make a second API call to get the output, and it may not be ready right away. So it’s like a <http://en.wikipedia.org/wiki/Future_(programming)> future -- the processing happens asynchronously from the initial submission of the data set, and the submission (the POST) is non-blocking in terms of it not having to wait for the output. Anyway, just wanted to clear up my meaning; I am probably not using the best terms. I’m still curious if anyone has any ideas for detecting and handling a client-side “Control-C” gracefully, or any practical tips for implementing long-running and transactional POST operations. - Lu From: Rob Heittman [mailto:rob.heitt...@solertium.com] Sent: Wednesday, February 25, 2009 5:45 PM To: discuss@restlet.tigris.org Subject: Re: detecting that client aborted or timed out Okay. I suspected you might mean not just synchronously implemented in a client API, but also "single request from any simple client." What I proposed sure isn't going to work from curl, though you could maybe make it work from a browser by making the GETs return something that triggered the loop, like a meta refresh or javascript. (yuk) <pedantic> To quibble over a really academic detail, the polling code I described isn't actually asynchronous -- there is no open callback pattern involved, just waiting and polling. Waiting and polling are stuff that happen inside the synchronous I/O libraries across the board; here you're just doing it at a higher level of abstraction, but I'm pretty sure anything that blocks in-process till completion is technically defined as synchronous. </pedantic> I shut up now. It sounds like what you're aiming for is a very simple one-step solution to a problem that normally would be solved by a more complex and multi-step protocol. Given a couple compromises ... okay ... The only way I know of to make it work *simply* for a client like curl, as I think we are both repeating, is to incrementally feed data to the client. In Restlet, you can do this with one of the stream representations. Somebody with more Restlet internals know-how than me would need to say whether or when this is guaranteed to send bits in real time to the client under a JEE container, I'm a bit afraid of things that might want to buffer my stream. To avoid the bad network speed dependency behavior you described, I think you'd have to move the general polling pattern I described before to your resource. Hand the submitted POST body to a new thread or farm of worker threads, and have the responding resource poll status, queueing and feeding chunked status report data, as it becomes available, to the client at whatever rate the client can accept. The transaction may finish before the client receives all the results. This would have the additional benefit -- to my mind -- of ensuring that a successfully submitted transaction completes even if the client disconnects. But maybe you DON'T want this to happen -- e.g. for Ctrl-C in curl to cancel the transaction? (If so, I don't know how) - Rob _____ On Wed, Feb 25, 2009 at 7:51 PM, Luis Saenz <lsa...@taxcient.com> wrote: Hi Rob, Thanks! Yeah, your suggestion to use the post/redirect/get pattern is a great way to solve this issue, but it violates my first stated constraint -- a synchronous solution. :) The only reason I want a synchronous solution is for simplicity -- to support lowest common denominator clients such as curl, without requiring the user of the API to write a script with logic such as what you propose below. However, no argument that adding this asynchronous complexity would solve the problem. Also, I forgot to mention that my imaginary resource specifies a POST response body with a report of the result of the POST operation (such as a list of status codes or error messages, one for each record in the data set). Given all this, here is another solution I thought of: Modify the implementation of the resource so that it doesn't stream the entire request body from the client to the server as a first step, and then process the data as a second step. Instead, break up the processing into "chunks" by reading N records from the request body and processing only those records, storing the result of processing that chunk in memory, and so on, all within a single transaction. The good news is that this would put a ceiling on the "wait time" from the client's perspective. The bad news is that this would extend the transaction demarcation to include the network -- if the server encounters an extremely slow (say, dial-up) client, it could have the detrimental effect of keeping a data store connection and transaction open for much longer than necessary. (This only matters because the whole operation is transactional.) Anyone else have any thoughts? - Lu ------------------------------------------------------ http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=1234025
