Hey Chris, I suppose I could. For some reason I didn't think that it was possible to submit a job to the Resource Manager other than through the Workflow. In my case, I'm using a listener to pull files out of the RabbitMQ. After I pull a file out of the queue, I'd like to submit a job to the Resource Manager to trigger a Workflow Event. How would one submit a job this way? Under the Resource Manager trunk, in src/main/resources/examples/jobs, I see 2 xml files that appear to be config files. Do I need to make use of these to do what I want to do?
Also, I know that the Resource Manager has it's own queue where it places jobs when all of its compute nodes are full. In my case (this is for Costin's pipeline BTW), I don't want jobs being placed in the Resource Manager queue. I would like for them to remain in the Rabbit Queue. Is the Resource Manager's "getNodeLoad()" method the best way to check if a node is full before submitting a job? Thanks in advance for your help! -Mike On 4/1/13 10:15 PM, "Mattmann, Chris A (388J)" <[email protected]> wrote: >Hey Mike, > >I'll bite, can't you use the Resource Manager and its queues/nodes to do >this? > >Cheers, >Chris > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Senior Computer Scientist >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 171-266B, Mailstop: 171-246 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Assistant Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > >-----Original Message----- >From: <Cayanan>, "Michael D (388J)" <[email protected]> >Reply-To: "[email protected]" <[email protected]> >Date: Monday, April 1, 2013 8:11 PM >To: "[email protected]" <[email protected]> >Subject: Workflow Questions > >>Hi all, >> >> >>I'm trying to see if anyone knows of a cool, easy to use "Resource >>Manager" that can control the amount of Workflows running concurrently on >>a node. Basically, my set up is like this: >> >> >>- I have a Workflow Manager Server configured to run at most 10 Workflow >>Events concurrently. >>- I have a bunch of files sitting on a RabbitMQ queue. >>- I have a listener that is continually monitoring this particular queue. >>As soon as it sees a file in this queue, it will trigger a Workflow >>event. >> >> >>Under this scenario, normally if I have 30 files in the RabbitMQ, my >>listener will trigger 30 Workflow events. However, the Workflow Server >>would put 20 of these events in its repository queue while the other 10 >>are running. >> >> >>What I want some "Resource Manager" component to do is just run the 10 >>Workflow events only and have the other 20 sitting in the RabbitMQ. As >>soon as 1 event has finished, then trigger another workflow event to keep >>the number of concurrent workflows running >> at 10. >>Eventually, we will want this "Resource Manager" component to be able to >>point to another node, node 2, where it can decide to trigger another set >>of 10 Workflow events while node 1 is still running it's 10 Workflow >>events. >> >> >>Hope this makes sense. Any helpful tips would be much appreciated. >> >> >>Thanks, >>Mike >
