Hey Mike, Don't specify any size for the queue and just set unlimitedQueue to be true.
That should do the trick. Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: <Cayanan>, "Michael D (388J)" <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Tuesday, April 2, 2013 1:29 PM To: "[email protected]" <[email protected]> Subject: Re: Workflow Questions >Hey Chris, > >Another question I had is that for Workflow, I didn't want there to be a >queue, so I set the queue size in the workflow.properties to 0 and put the >unlimited queue size flag to false. When I did that, I got this exception: > >% ./wmgr restart > >Shutting down cas workflow manager: OK >Starting cas workflow manager: OK >Apr 2, 2013 1:03:58 PM >org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init> >INFO: Loading Workflow Manager Configuration Properties from: >[/home/mcayanan/apps-workflow/etc/workflow.properties] >Exception in thread "main" java.lang.IllegalArgumentException > at EDU.oswego.cs.dl.util.concurrent.BoundedBuffer.<init>(Unknown Source) > at >org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngine.<init>(Thread >P >oolWorkflowEngine.java:113) > at >org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory.create >W >orkflowEngine(ThreadPoolWorkflowEngineFactory.java:107) > at >org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.<init>(XmlRpcWor >k >flowManager.java:115) > at >org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.main(XmlRpcWorkf >l >owManager.java:611) > >The behavior that I was going for is by setting the queue size to '0', >when I run the 'sendEvent' command, the Workflow Manager server would >return 'false' to me when it has reached its max pool size. > >I ended up upping the queue size to 1 and the exception went away. This >probably isn't a big issue of having a queue size of 1 to get around this >exception, but I'm kind of curious as to why having size '0' as the queue >size is a bad thing. >Any thoughts? > >Thanks, >Mike > > > >On 4/2/13 12:52 PM, "Cayanan, Michael D (388J)" ><[email protected]> wrote: > >>Hi Mike I. and Chris, >> >>Thank you to both for your inputs. I will play around with the >>JobSubmitter Tool and the Client API and let you know if I have more >>questions. >> >>Cheers, >>Mike C. >> >>On 4/2/13 10:30 AM, "Mattmann, Chris A (388J)" >><[email protected]> wrote: >> >>>Hi Mike's :) >>> >>>Mike I is correct you can submit jobs through the JobSubmitter tool. >>>Also in RM if you are using Java, RM has an XmlRpcResourceManagerClient >>>API and associated set of interfaces to submit jobs. >>> >>>Mike C. -- if you don't want the Job to go into the Resource Manager >>>queue, >>>then use you must check the load first, and be sure that the job will be >>>tasked immediately. Not sure why you wouldn't just put the job in both >>>queues >>>RM and RabbitMQ.. let me know if that works. >>> >>>Cheers, >>>Chris >>> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>Chris Mattmann, Ph.D. >>>Senior Computer Scientist >>>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>>Office: 171-266B, Mailstop: 171-246 >>>Email: [email protected] >>>WWW: http://sunset.usc.edu/~mattmann/ >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>Adjunct Assistant Professor, Computer Science Department >>>University of Southern California, Los Angeles, CA 90089 USA >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> >>> >>> >>> >>> >>>-----Original Message----- >>>From: <Iwunze>, "Michael C [NOAA-JPSS] (GSFC-4700)" >>><[email protected]> >>>Reply-To: "[email protected]" <[email protected]> >>>Date: Tuesday, April 2, 2013 8:40 AM >>>To: "[email protected]" <[email protected]> >>>Subject: Re: Workflow Questions >>> >>>>Hey Chris, >>>> >>>> I have submitted Jobs to the Resource Manager without using >>>>Workflow. >>>>There is >>>>JobSumitter tool in OODT that can do this for you. >>>> >>>> >>>>Mike >>>> >>>>On 4/2/13 11:32 AM, "Cayanan, Michael D" >>>><[email protected]> >>>>wrote: >>>> >>>>>Hey Chris, >>>>> >>>>>I suppose I could. For some reason I didn't think that it was possible >>>>>to >>>>>submit a job to the Resource Manager other than through the Workflow. >>>>>In my case, I'm using a listener to pull files out of the RabbitMQ. >>>>>After >>>>>I pull a file out of the queue, I'd like to submit a job to the >>>>>Resource >>>>>Manager to trigger a Workflow Event. How would one submit a job this >>>>>way? >>>>>Under the Resource Manager trunk, in src/main/resources/examples/jobs, >>>>>I >>>>>see 2 xml files that appear to be config files. Do I need to make use >>>>>of >>>>>these to do what I want to do? >>>>> >>>>>Also, I know that the Resource Manager has it's own queue where it >>>>>places >>>>>jobs when all of its compute nodes are full. In my case (this is for >>>>>Costin's pipeline BTW), I don't want jobs being placed in the Resource >>>>>Manager queue. I would like for them to remain in the Rabbit Queue. Is >>>>>the >>>>>Resource Manager's "getNodeLoad()" method the best way to check if a >>>>>node >>>>>is full before submitting a job? >>>>> >>>>>Thanks in advance for your help! >>>>> >>>>>-Mike >>>>> >>>>> >>>>>On 4/1/13 10:15 PM, "Mattmann, Chris A (388J)" >>>>><[email protected]> wrote: >>>>> >>>>>>Hey Mike, >>>>>> >>>>>>I'll bite, can't you use the Resource Manager and its queues/nodes to >>>>>>do >>>>>>this? >>>>>> >>>>>>Cheers, >>>>>>Chris >>>>>> >>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>Chris Mattmann, Ph.D. >>>>>>Senior Computer Scientist >>>>>>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>>>>>Office: 171-266B, Mailstop: 171-246 >>>>>>Email: [email protected] >>>>>>WWW: http://sunset.usc.edu/~mattmann/ >>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>Adjunct Assistant Professor, Computer Science Department >>>>>>University of Southern California, Los Angeles, CA 90089 USA >>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>-----Original Message----- >>>>>>From: <Cayanan>, "Michael D (388J)" >>>>>><[email protected]> >>>>>>Reply-To: "[email protected]" <[email protected]> >>>>>>Date: Monday, April 1, 2013 8:11 PM >>>>>>To: "[email protected]" <[email protected]> >>>>>>Subject: Workflow Questions >>>>>> >>>>>>>Hi all, >>>>>>> >>>>>>> >>>>>>>I'm trying to see if anyone knows of a cool, easy to use "Resource >>>>>>>Manager" that can control the amount of Workflows running >>>>>>>concurrently >>>>>>>on >>>>>>>a node. Basically, my set up is like this: >>>>>>> >>>>>>> >>>>>>>- I have a Workflow Manager Server configured to run at most 10 >>>>>>>Workflow >>>>>>>Events concurrently. >>>>>>>- I have a bunch of files sitting on a RabbitMQ queue. >>>>>>>- I have a listener that is continually monitoring this particular >>>>>>>queue. >>>>>>>As soon as it sees a file in this queue, it will trigger a Workflow >>>>>>>event. >>>>>>> >>>>>>> >>>>>>>Under this scenario, normally if I have 30 files in the RabbitMQ, my >>>>>>>listener will trigger 30 Workflow events. However, the Workflow >>>>>>>Server >>>>>>>would put 20 of these events in its repository queue while the other >>>>>>>10 >>>>>>>are running. >>>>>>> >>>>>>> >>>>>>>What I want some "Resource Manager" component to do is just run the >>>>>>>10 >>>>>>>Workflow events only and have the other 20 sitting in the RabbitMQ. >>>>>>>As >>>>>>>soon as 1 event has finished, then trigger another workflow event to >>>>>>>keep >>>>>>>the number of concurrent workflows running >>>>>>> at 10. >>>>>>>Eventually, we will want this "Resource Manager" component to be >>>>>>>able >>>>>>>to >>>>>>>point to another node, node 2, where it can decide to trigger >>>>>>>another >>>>>>>set >>>>>>>of 10 Workflow events while node 1 is still running it's 10 Workflow >>>>>>>events. >>>>>>> >>>>>>> >>>>>>>Hope this makes sense. Any helpful tips would be much appreciated. >>>>>>> >>>>>>> >>>>>>>Thanks, >>>>>>>Mike >>>>>> >>>>> >>>> >>> >> >
