Ross you still having this issue? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message----- From: Ross Laidlaw <rlaidlaw.o...@gmail.com> Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org> Date: Wednesday, May 13, 2015 at 2:37 AM To: "dev@oodt.apache.org" <dev@oodt.apache.org> Subject: Re: Request for advice on workflows >Hi Chris, > >As a first step I got an XMLWorkflowRepository workflow running. I then >tried using the PackagedWorkflowRepository to create a workflow with the >same tasks. I got a sequential workflow up and running, similar to the >following: > ><cas:workflows ...> > <sequential id="urn:test:MainWorkflow" name="MainWorkflow"> > <task id-ref="urn:test:TaskA"/> > <task id-ref="urn:test:TaskB"/> > <task id-ref="urn:test:TaskC"/> > <task id-ref="urn:test:TaskD"/> > <task id-ref="urn:test:TaskE"/> > <task id-ref="urn:test:TaskF"/> > <task id-ref="urn:test:TaskG"/> > </sequential> > > <!-- task definitions --> > <task>...</task> > ></cas:workflows> > > >I found that I could run the workflow using ./wmgr-client --url >[WorkflowManagerUrl] --sendEvent --eventName [WorkflowId] >e.g. ./wmgr-client --url http://localhost:9001 --sendEvent --eventName >urn:test:MainWorkflow > > >After getting the sequential workflow running, I wanted to run tasks C and >D in parallel so I then tried adding a <parallel> section as follows: > ><cas:workflows ...> > <sequential id="urn:test:MainWorkflow" name="MainWorkflow"> > <task id-ref="urn:test:TaskA"/> > <task id-ref="urn:test:TaskB"/> > <parallel id="urn:test:ParallelWorkflow" name="ParallelWorkflow"> > <task id-ref="urn:test:TaskC"/> > <task id-ref="urn:test:TaskD"/> > </parallel> > <task id-ref="urn:test:TaskE"/> > <task id-ref="urn:test:TaskF"/> > <task id-ref="urn:test:TaskG"/> > </sequential> > ... ></cas:workflows> > > >The parallel section generates an additional task to be submitted called >'Redirector Task' that wants to be executed in the 'high' queue using the >BranchRedirector class: > >INFO: Task: [Redirector Task] has no required metadata fields >May 13, 2015 1:27:29 AM >org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob >FINEST: >[ > { > job.queueName=high, > job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, > job.name=redirector-322db9d2-4663-4b89-9af0-f8f97ad74c80, > job.id=, job.status=, job.load=2, > job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput > }, > { > >task.instance.class=org.apache.oodt.cas.workflow.examples.BranchRedirector >, > task.config={}, > task.metadata={...} > } >] > > >I wasn't sure how to configure my setup to deal with this task. From >reading OODT-70 I gather that it might be trying to start up other >wmgr-clients on batch stubs to run these tasks in separate workflows. I >tried setting up a local batch stub that accepts jobs from the 'high' >queue. When running the full workflow I get the following exception from >the batch stub when it tries to execute the redirector task: > >INFO: Assigning job: [redirector-8561de94-2681-477b-8bcb-331d83224d5b] to >node: [local-redirector] >May 13, 2015 1:36:51 AM >org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub >genericExecuteJob >INFO: stub attempting to execute class: >[org.apache.oodt.cas.workflow.structs.TaskJob] >org.apache.xmlrpc.XmlRpcClientException: Failure writing request > at >org.apache.xmlrpc.XmlRpcClientRequestProcessor.encodeRequest(XmlRpcClientR >equestProcessor.java:76) > at >org.apache.xmlrpc.XmlRpcClientRequestProcessor.encodeRequestBytes(XmlRpcCl >ientRequestProcessor.java:102) > at >org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:70) > at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) > at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) > at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) > at >org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.sendEvent( >XmlRpcWorkflowManagerClient.java:477) > at >org.apache.oodt.cas.workflow.examples.BranchRedirector.run(BranchRedirecto >r.java:62) > at >org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > at >org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run >(XmlRpcBatchStub.java:229) > at java.lang.Thread.run(Thread.java:745) >Caused by: org.apache.xmlrpc.XmlRpcException: null values not supported by >XML-RPC > at org.apache.xmlrpc.XmlWriter.writeObject(XmlWriter.java:231) > at >org.apache.xmlrpc.XmlRpcClientRequestProcessor.encodeRequest(XmlRpcClientR >equestProcessor.java:72) > ... 10 more >org.apache.oodt.cas.resource.structs.exceptions.JobInputException: Failed >to run task > at >org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104) > at >org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run >(XmlRpcBatchStub.java:229) > at java.lang.Thread.run(Thread.java:745) >Caused by: >org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcept >ion: >Failure writing request > at >org.apache.oodt.cas.workflow.examples.BranchRedirector.run(BranchRedirecto >r.java:64) > at >org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > ... 2 more >May 13, 2015 1:36:51 AM >org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run >SEVERE: Job execution failed for jobId '2015-05-13T01:36:38.746-07:00' : >batchstub.executeJob returned false >java.lang.Exception: batchstub.executeJob returned false > at >org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchM >grProxy.java:125) > > >It looks like my sequential and parallel workflows have the required names >and IDs. Do you know what could be going wrong? Maybe I'm missing a few >vital configuration steps. > >Many thanks for your help, > > >Ross > > > > > > > >On 8 May 2015 at 19:31, Ross Laidlaw <rlaidlaw.o...@gmail.com> wrote: > >> Hi Chris, >> >> Many thanks for your advice. I'll check out the >> PackagedWorkflowRepository and will let you know how it goes. >> >> Ross >> >> >> >> > On May 8, 2015, at 12:58 PM, Mattmann, Chris A (3980) < >> chris.a.mattm...@jpl.nasa.gov> wrote: >> > >> > Hey Ross: >> > >> > Great work! Try the PackagedWorkflowRepository. The best documentation >> > that exists of it is in OODT-70 [1]. >> > >> > I would use that to support the type of workflows that you’re doing. >> > If this works we should make it the default since it’s supposed to >> > be back compat with the XMLWorkflowRepositoryManager. >> > >> > Cheers, >> > Chris >> > >> > P.S. see some of the ASO scripts I wrote to check on specific task/ >> > PGE status, etc. >> > >> > >> > [1]https://issues.apache.org/jira/browse/OODT-70 >> > >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > Chris Mattmann, Ph.D. >> > Chief Architect >> > Instrument Software and Science Data Systems Section (398) >> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> > Office: 168-519, Mailstop: 168-527 >> > Email: chris.a.mattm...@nasa.gov >> > WWW: http://sunset.usc.edu/~mattmann/ >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > Adjunct Associate Professor, Computer Science Department >> > University of Southern California, Los Angeles, CA 90089 USA >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> > >> > >> > >> > >> > >> > >> > -----Original Message----- >> > From: Ross Laidlaw <rlaidlaw.o...@gmail.com> >> > Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org> >> > Date: Friday, May 8, 2015 at 8:14 AM >> > To: "dev@oodt.apache.org" <dev@oodt.apache.org> >> > Subject: Request for advice on workflows >> > >> >> Hi All, >> >> >> >> I'm working on an OODT setup (v0.8.1) with several workflow tasks >> defined >> >> and working well individually. I can use the wmgr-client to fire off >> jobs >> >> separately and all appears to be working as planned. I'd like to >> combine >> >> these tasks into a workflow so that I don't have to run each set of >> jobs. >> >> For example, in the workflow I'd like to run several 'task A' jobs >>and >> >> when >> >> all these have finished I'd like the workflow to automatically start >> >> several 'task B' jobs, etc. >> >> >> >> I found some example workflows in the distribution, plus several >>guides >> up >> >> on the wiki. Is there a current recommended way of setting up a >>simple >> >> workflow? Should I use workflow v2 or is it best to stick with v1? >> Once >> >> I've set up a workflow, how do I start it off? It looks like it >>might >> be >> >> possible to nest workflows, is that functionality available yet? >> >> >> >> Additionally, what's the best way to check on the status of currently >> >> running jobs? I've used the pcs_stat tool a lot and it's great for >> >> checking on the overall status, but is there a recommended way to >>check >> >> e.g. if any 'task A' jobs are still running for a certain workflow >>and >> >> what >> >> state they're currently in? >> >> >> >> Many thanks in advance for your advice, >> >> >> >> >> >> Ross >> > >>
