Re: [zeta-dev] Proposal: ezcWorkflow CouchDB Backend
Hi On Mon, Jul 26, 2010 at 9:14 PM, Benjamin Eberlei wrote: > Hello everyone, > > i want to propose the development of a CouchDB Storage Tiein for the > Workflow Engine. It would be a considerable improvement over the > Database backend, for various reasons: Maybe abstract it to have easy tie ins for other document oriented databases? > > 1. Several variables are saved as PHP serialize() output in LONGTEXT > fields in both the Workflow and Execution parts of the Engine. This is > rather problematic (and ugly). Workflow just has many "unrelational" > parts such as node configuration, execution state/variables and thread > information. CouchDB could just nest them as sub-objects in JSON. +1, and also return JSON constructs for potential use in view layers > > 2. Database backend uses serialized php data for Execution::$variables > and Execution::$waitingFor, both variables that are interesting for > querying the set of existing workflows. Views can be easily written > using any of the two as keys to select for. Yes! > > 3. Both a workflow and an execution could be saved in a single document > each. > > Why a workflow? Workflows are never updated, only saved with a new ID. > Otherwise you could seriously destroy currently existing executions of a > changed workflow. Therefore no problems exists with read/write > contention or version conflicts. > Fair enough, is your use-case involving "long-lived" workflows? About the execution, I see MongoDB as an attractive alternative too, because it allows updates in place for a single key/field, further minimizing the overhead for tracking this execution part > Why an execution? It has to be resurrected from persistence completely > in any case, the nested data is never of any interest in an incomplete > context. You always need the whole data. Indeed, at least for as far I dived into and understand the workflow component at this stage > > What do you think of the idea? Is there interest in this development? I think so, but I am biased towards doc-oriented backends anyway my 0.02€ Paul
Re: [zeta-dev] Proposal: ezcWorkflow CouchDB Backend
Hey Paul, i already have a buggy and untested prototype of the CouchDB backend that is as small as 300 loc. I think an additional abstraction of the document layer may only complicate the implementation or just lead to all API methods being delegated to yet another backend. Its probably easier to just have an additional ezcWorkflowMongoTiein that is equally thin. Regarding the non-Update only Insert, ezcWorkflowDatabaseTiein operates exactly the same. Yes i have very long running workflows that may change often, so this is an important requirements (for me). But i think this behaviour is important for short workflows of just some minutes execution also. greetings, Benjamin On Tue, 27 Jul 2010 00:06:00 +0200, Paul Borgermans wrote: > Hi > > On Mon, Jul 26, 2010 at 9:14 PM, Benjamin Eberlei > wrote: >> Hello everyone, >> >> i want to propose the development of a CouchDB Storage Tiein for the >> Workflow Engine. It would be a considerable improvement over the >> Database backend, for various reasons: > > Maybe abstract it to have easy tie ins for other document oriented > databases? > >> >> 1. Several variables are saved as PHP serialize() output in LONGTEXT >> fields in both the Workflow and Execution parts of the Engine. This is >> rather problematic (and ugly). Workflow just has many "unrelational" >> parts such as node configuration, execution state/variables and thread >> information. CouchDB could just nest them as sub-objects in JSON. > > +1, and also return JSON constructs for potential use in view layers > >> >> 2. Database backend uses serialized php data for Execution::$variables >> and Execution::$waitingFor, both variables that are interesting for >> querying the set of existing workflows. Views can be easily written >> using any of the two as keys to select for. > > Yes! > >> >> 3. Both a workflow and an execution could be saved in a single document >> each. >> >> Why a workflow? Workflows are never updated, only saved with a new ID. >> Otherwise you could seriously destroy currently existing executions of a >> changed workflow. Therefore no problems exists with read/write >> contention or version conflicts. >> > > Fair enough, is your use-case involving "long-lived" workflows? > > About the execution, I see MongoDB as an attractive alternative too, > because it allows updates in place for a single key/field, further > minimizing the overhead for tracking this execution part > >> Why an execution? It has to be resurrected from persistence completely >> in any case, the nested data is never of any interest in an incomplete >> context. You always need the whole data. > > Indeed, at least for as far I dived into and understand the workflow > component at this stage > >> >> What do you think of the idea? Is there interest in this development? > > I think so, but I am biased towards doc-oriented backends anyway > > my 0.02€ > > Paul
Re: [zeta-dev] Proposal: ezcWorkflow CouchDB Backend
On Tue, 2010-07-27 at 09:52 +0200, Benjamin Eberlei wrote: > Hey Paul, > > i already have a buggy and untested prototype of the CouchDB backend > that is as small as 300 loc. I think an additional abstraction of the > document > layer may only complicate the implementation or just lead to all API > methods being delegated to yet another backend. Its probably easier to > just have an additional ezcWorkflowMongoTiein that is equally thin. I second this - especially because of the quite fundamental differences between CouchDB and MongoDB it sounds more feasible to write another driver / tiein for that. I like the idea of storing the workflows in CouchDB very much - as you know I might have a usecase for that in the near future, too :). I can't gauge your design decisions regarding the implementation, though - since I just don't know enough about the Workflow component. Kind regards, Kore -- Kore Nordmann(GPG 0xDDC70BBB) - What I do: http://kore-nordmann.de/portfolio.html - Motivate me: http://wishlist.kore-nordmann.de/ signature.asc Description: This is a digitally signed message part
Re: [zeta-dev] Proposal: ezcWorkflow CouchDB Backend
Hi Benjamin, On 07/27/2010 09:52 AM, Benjamin Eberlei wrote: > i already have a buggy and untested prototype of the CouchDB backend > that is as small as 300 loc. I think an additional abstraction of the > document > layer may only complicate the implementation or just lead to all API > methods being delegated to yet another backend. Its probably easier to > just have an additional ezcWorkflowMongoTiein that is equally thin. > > Regarding the non-Update only Insert, ezcWorkflowDatabaseTiein operates > exactly the same. Yes i have very long running workflows that may change > often, so this is an important requirements (for me). But i think this > behaviour is important for short workflows of just some minutes execution > also. I'd love to see your CouchDB back end for Workflow as a WorkflowCouchDBTiein or something. I agree with you that abstracting the back end once more introduces to much complication. Workflow already provides the interface methods, so that should work out fine. Could you provide small requirements and design docs? Regards, Toby -- Tobias Schlitthttp://schlitt.infoGPG Key: 0xC462BC14 Want to hire me? Need quality assurance?http://qafoo.com eZ Components are Zeta Components now! http://bit.ly/9S7zbn
