Hey Chris, On 3/6/12 10:38 PM, "Mattmann, Chris A (388J)" <[email protected]> wrote:
>Hey Mike, > >On Mar 5, 2012, at 8:13 AM, Cayanan, Michael D (388J) wrote: >>>> [..snip..] >>> >>> I'm not sure I follow you? The workflow manager's >>> ThreadPoolWorkflowEngine needs to evaluate preconditions, on a per >>> workflow-instance basis. >>> The only way to shut that down, is to shut down the workflow engine, >>>and >>> thus implicitly the WM. >> >> In other words, if the pre condition continually fails, will the >>workflow >> ever timeout where the engine shuts down automatically? > >In the 0.3 version of the workflow manager, no. However, the work around >is to use the --resumeWorkflowInst command from the command line >workflow mgr client, or simply to call the Java API and call >resumeWorkflowInst. >That will unpause the workflow and get around the condition. Very cool. Didn't know this feature existed. > >> >> [..snip...] >>> time to whatever is configured for the PGE. I'd severely caution >>>against >>> doing so, b/c you're really giving the developer or >>> workflow person an overt amount of control over an aspect of the system >>> that will affect (perceived) performance. >> >> This was more of a curiosity question than anything. Personally, I don't >> see the need to have a configurable pre condition wait time. In SMAP >> world, there will be a case where a workflow requires two inputs, where >> the inputs are coming from the outputs of 2 different workflows. >> Furthermore, 1 of these inputs isn't expected to arrive like 10 hours >> later after the first input arrives. The assumption here is that 1 input >> will trigger the workflow and thus the pre condition would be to check >>for >> the existence for that other input. So, if you have your precondition >>wait >> time to some small interval like 30 seconds, would system performance be >> affected in this case knowing that the precondition isn't expected to be >> satisfied until 10 hours later? > >It depends on how you separate the control flow of this workflow. Let's >say >w1 is triggered and can process its data right away, but w2 is the >workflow >that needs to wait 10 hours to get its data, and w3 is waiting on w1 and >w2 >to finish. > >Here's how I would wire it up: > >1. w1 is kicked off right away, by the ops event, from a GUI, whatever. >w1 >processes its data right away, and then uses CAS-PGE and crawler >to cat/archive the outputs and met. > >2. Then I would tie a workflow event via Crawler actions to an event >that kicks off a single workflow that is the logical combination of w2 and >w3, in sequence. That way, once the crawler ingestions, it will fire an >event that will then fire w2, w2 will complete (data is there) and since >w3 is in sequence, knowing that w2 is done, we'd be set. You could >wire up a precondition on w2 to not execute unless w1 (for that dataset, >day, orbit, etc.) has be processed, but it's likely it will have by then. > >HTH! > >Cheers, >Chris Thanks for the suggestion. I will def keep this in mind when wiring up the workflows. -Mike > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Senior Computer Scientist >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 171-266B, Mailstop: 171-246 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Assistant Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >
