Hi Mike,
From: <Cayanan>, "Michael D (388J)" <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, March 13, 2013 10:40 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Workflow2 - State Persistence Hey Chris, Sounds awesome. +1 to the additional features. I'm pretty sure I'll be able to help out as we go further along in developing this pipeline service for AMMOS. They eventually would like this feature and are asking me how much effort is required to do this. I think from their end, what they want, from what I understand, is the following use-case scenario: 1) Run a command that will take a snapshot of the Workflow. 2) Bring the system down. Don't wait for anything to finish. 3) Upon restart, the Workflow should resume from where you last took the snapshot. >> So, right now today, snapshots are taken at WorkflowLifecycleStates, as the >> Instance moves through its lifecycle. The only thing that doesn't happen is to have the Engine pick not done workflows and then restart them (#1 on mine below). The AMMOS folks are worried that consistently doing state persistence will be a performance hog and so by having the ability to save the state on demand, they'll be able to have things run more efficiently. >> We have some pretty clear evidence of this over the last 7 years doing data >> processing with Apache OODT that this is not the case, and though the >> performance of this matters, that can be horizontally scaled out at minimum >> overhead and complexity to support such a persistence. Hopefully we can tag up soon to go over this so that I can let them know how much effort this implementation would take. >> Yep hopefully this week! :) Cheers, Chris Cheers, Mike From: <Mattmann>, Chris Mattmann <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Tuesday, March 12, 2013 8:32 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Workflow2 - State Persistence Hey Mike, Workflow information (state, etc.) is persisted to the WorkflowInstanceRepository. In Wengine/Workflow2 support in trunk it makes heavy use of this, but the support is evolving. I think for 0.6 I'll add some functionality and features (and would appreciate any help) to allow users to: 1. Specify in workflow.properties whether or not prior not completed (or not "finished" category state) workflows should be cleared out and/or should be restarted and run on startup. Maybe a property like org.apache.oodt.cas.workflow.engine.unfinishedWorkflows.complete=true or false 2. Finish support in WorkflowInstanceRepository for persisting all new Wengine state information Those are the 2 items that should be done for 0.6 to support this feature. It's on the roadmap just not fully done yet. Cheers, Chris From: <Cayanan>, "Michael D (388J)" <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, March 7, 2013 8:41 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Workflow2 - State Persistence Hi Chris and/or Brian, I'm currently on a project where PDS and AMMOS are joining forces to create a pipeline service and we'd like to use the latest Workflow2 to do that. They asked a question at last week's meeting regarding if the latest and greatest OODT Workflow will be able to do state persistence. I can let Paul Ramirez chime in any further details since he was at the meeting as well, but can the latest and greatest Workflow do that? And if so, under which engine? I'm currently using the following engine: workflow.engine.factory = org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory Thanks, Mike
