On Sun, Jan 24, 2010 at 2:25 PM, David Richardson <[email protected]> wrote: > That's exactly what creates the high associated procedure cost - the > documents are submitted by mobile workers who connect for very short times, > sometimes very infrequently. If the document is accepted with invalid entries > and processed later, the opportunity to alert the author of the problem(s) > during the request/response cycle is lost. This means someone else must fix > the mistake (extra work), and a very unwieldy process (usually manual) must > be instituted to alert the author (training/feedback is harder and less > effective). > > I'm now willing to accept extra moving parts living in front of the db - > there are other things that are easier to accomplish there too. > The cron async process smells like Greenspun's 10th law, extrapolated to > transaction managers and message brokers ;-) >
The advantage of having an asynchronous document state machine handler is a whole lot of decoupling, as the queue is just a side effect of the database. >>> I've been trying to eliminate, or at least vastly reduce the need for >>> middleware in a specific class of applications, but this problem probably >>> cuts that task short. I'm wary of the procedural-controller middleware approach because I'm afraid it encourages you to program as though you have transactions. (Eg: change these 4 documents at the same time and hope for the best.) If you are careful with asynchronous handlers, you can be careful to pass the state through the database _changes feeds in a way that doesn't give devs a false sense of multi-document transactions. This is why I think an asynchronous node.js _changes handler (or browser ajax handlers) or event based programming in any language is such a good fit for CouchDB. If you transmit all of your important events through the database, even real-time apps can scale by using replication. Each doc update is atomic, so there's room to model larger transactions as a series of atomic updates. Chris >> On Sun, Jan 24, 2010 at 12:06 PM, David Richardson >> <[email protected]> wrote: >>> I've been trying to eliminate, or at least vastly reduce the need for >>> middleware in a specific class of applications, but this problem probably >>> cuts that task short. >>> >>> We have incoming documents that need to be validated against data stored in >>> other docs (rate tables, technical abbreviations, inspection codes). There >>> is a high procedural cost to allowing documents in and validating them >>> after saving (queue processing), but validation functions don't allow >>> bringing in external data (correct me if I'm wrong, please.) Without >>> writing middleware or sideware (an external handler), I don't see a way to >>> do this. >>> >>> What are the arguments against including a native 'lookup' function >>> available to validation functions only. This seems to be a side-effect-free >>> usage case to me. Reads are cheap, correct? >>> If my memory hasn't completely failed, this was a significant problem in >>> Lotus Notes until such a function was added, somewhere around version 3.3 >>> or 4.0. >>> >>> David Richardson >> >> The problem with this approach is that validation is run during >> replication as well, so any multi-doc data dependencies become >> problematic in ad-hoc clusters. >> >> The way to do this without middleware is to have a backend >> asynchronous process (maybe node.js) that consumes _changes and acts >> on particular updates. So a user saves the doc with a pending state >> and the _changes handler sees that and initiates validation. >> >> (One trick you can do with this is use _changes heartbeat to generate >> a cron trigger, so your async process can also do things based on >> time.) >> >> I'd like to see this async/cron functionality bundled with Couch >> itself, but there's only so much time in the day and I don't think >> it's worth delaying any releases for. Patches extremely welcome, but I >> think this one might require more discussion before anyone would be >> ready to run off and write it. >> >> Chris >> >> >> >> -- >> Chris Anderson >> http://jchrisa.net >> http://couch.io > > -- Chris Anderson http://jchrisa.net http://couch.io
