Hi Mike I. and Chris, Thank you to both for your inputs. I will play around with the JobSubmitter Tool and the Client API and let you know if I have more questions.
Cheers, Mike C. On 4/2/13 10:30 AM, "Mattmann, Chris A (388J)" <[email protected]> wrote: >Hi Mike's :) > >Mike I is correct you can submit jobs through the JobSubmitter tool. >Also in RM if you are using Java, RM has an XmlRpcResourceManagerClient >API and associated set of interfaces to submit jobs. > >Mike C. -- if you don't want the Job to go into the Resource Manager >queue, >then use you must check the load first, and be sure that the job will be >tasked immediately. Not sure why you wouldn't just put the job in both >queues >RM and RabbitMQ.. let me know if that works. > >Cheers, >Chris > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Senior Computer Scientist >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 171-266B, Mailstop: 171-246 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Assistant Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > >-----Original Message----- >From: <Iwunze>, "Michael C [NOAA-JPSS] (GSFC-4700)" ><[email protected]> >Reply-To: "[email protected]" <[email protected]> >Date: Tuesday, April 2, 2013 8:40 AM >To: "[email protected]" <[email protected]> >Subject: Re: Workflow Questions > >>Hey Chris, >> >> I have submitted Jobs to the Resource Manager without using Workflow. >>There is >>JobSumitter tool in OODT that can do this for you. >> >> >>Mike >> >>On 4/2/13 11:32 AM, "Cayanan, Michael D" <[email protected]> >>wrote: >> >>>Hey Chris, >>> >>>I suppose I could. For some reason I didn't think that it was possible >>>to >>>submit a job to the Resource Manager other than through the Workflow. >>>In my case, I'm using a listener to pull files out of the RabbitMQ. >>>After >>>I pull a file out of the queue, I'd like to submit a job to the Resource >>>Manager to trigger a Workflow Event. How would one submit a job this >>>way? >>>Under the Resource Manager trunk, in src/main/resources/examples/jobs, I >>>see 2 xml files that appear to be config files. Do I need to make use of >>>these to do what I want to do? >>> >>>Also, I know that the Resource Manager has it's own queue where it >>>places >>>jobs when all of its compute nodes are full. In my case (this is for >>>Costin's pipeline BTW), I don't want jobs being placed in the Resource >>>Manager queue. I would like for them to remain in the Rabbit Queue. Is >>>the >>>Resource Manager's "getNodeLoad()" method the best way to check if a >>>node >>>is full before submitting a job? >>> >>>Thanks in advance for your help! >>> >>>-Mike >>> >>> >>>On 4/1/13 10:15 PM, "Mattmann, Chris A (388J)" >>><[email protected]> wrote: >>> >>>>Hey Mike, >>>> >>>>I'll bite, can't you use the Resource Manager and its queues/nodes to >>>>do >>>>this? >>>> >>>>Cheers, >>>>Chris >>>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>Chris Mattmann, Ph.D. >>>>Senior Computer Scientist >>>>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>>>Office: 171-266B, Mailstop: 171-246 >>>>Email: [email protected] >>>>WWW: http://sunset.usc.edu/~mattmann/ >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>Adjunct Assistant Professor, Computer Science Department >>>>University of Southern California, Los Angeles, CA 90089 USA >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> >>>> >>>> >>>> >>>> >>>> >>>>-----Original Message----- >>>>From: <Cayanan>, "Michael D (388J)" <[email protected]> >>>>Reply-To: "[email protected]" <[email protected]> >>>>Date: Monday, April 1, 2013 8:11 PM >>>>To: "[email protected]" <[email protected]> >>>>Subject: Workflow Questions >>>> >>>>>Hi all, >>>>> >>>>> >>>>>I'm trying to see if anyone knows of a cool, easy to use "Resource >>>>>Manager" that can control the amount of Workflows running concurrently >>>>>on >>>>>a node. Basically, my set up is like this: >>>>> >>>>> >>>>>- I have a Workflow Manager Server configured to run at most 10 >>>>>Workflow >>>>>Events concurrently. >>>>>- I have a bunch of files sitting on a RabbitMQ queue. >>>>>- I have a listener that is continually monitoring this particular >>>>>queue. >>>>>As soon as it sees a file in this queue, it will trigger a Workflow >>>>>event. >>>>> >>>>> >>>>>Under this scenario, normally if I have 30 files in the RabbitMQ, my >>>>>listener will trigger 30 Workflow events. However, the Workflow Server >>>>>would put 20 of these events in its repository queue while the other >>>>>10 >>>>>are running. >>>>> >>>>> >>>>>What I want some "Resource Manager" component to do is just run the 10 >>>>>Workflow events only and have the other 20 sitting in the RabbitMQ. As >>>>>soon as 1 event has finished, then trigger another workflow event to >>>>>keep >>>>>the number of concurrent workflows running >>>>> at 10. >>>>>Eventually, we will want this "Resource Manager" component to be able >>>>>to >>>>>point to another node, node 2, where it can decide to trigger another >>>>>set >>>>>of 10 Workflow events while node 1 is still running it's 10 Workflow >>>>>events. >>>>> >>>>> >>>>>Hope this makes sense. Any helpful tips would be much appreciated. >>>>> >>>>> >>>>>Thanks, >>>>>Mike >>>> >>> >> >
