Hey Chris,
Another question I had is that for Workflow, I didn't want there to be a
queue, so I set the queue size in the workflow.properties to 0 and put the
unlimited queue size flag to false. When I did that, I got this exception:
% ./wmgr restart
Shutting down cas workflow manager: OK
Starting cas workflow manager: OK
Apr 2, 2013 1:03:58 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Loading Workflow Manager Configuration Properties from:
[/home/mcayanan/apps-workflow/etc/workflow.properties]
Exception in thread "main" java.lang.IllegalArgumentException
at EDU.oswego.cs.dl.util.concurrent.BoundedBuffer.<init>(Unknown Source)
at
org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngine.<init>(ThreadP
oolWorkflowEngine.java:113)
at
org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory.createW
orkflowEngine(ThreadPoolWorkflowEngineFactory.java:107)
at
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.<init>(XmlRpcWork
flowManager.java:115)
at
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager.main(XmlRpcWorkfl
owManager.java:611)
The behavior that I was going for is by setting the queue size to '0',
when I run the 'sendEvent' command, the Workflow Manager server would
return 'false' to me when it has reached its max pool size.
I ended up upping the queue size to 1 and the exception went away. This
probably isn't a big issue of having a queue size of 1 to get around this
exception, but I'm kind of curious as to why having size '0' as the queue
size is a bad thing.
Any thoughts?
Thanks,
Mike
On 4/2/13 12:52 PM, "Cayanan, Michael D (388J)"
<[email protected]> wrote:
>Hi Mike I. and Chris,
>
>Thank you to both for your inputs. I will play around with the
>JobSubmitter Tool and the Client API and let you know if I have more
>questions.
>
>Cheers,
>Mike C.
>
>On 4/2/13 10:30 AM, "Mattmann, Chris A (388J)"
><[email protected]> wrote:
>
>>Hi Mike's :)
>>
>>Mike I is correct you can submit jobs through the JobSubmitter tool.
>>Also in RM if you are using Java, RM has an XmlRpcResourceManagerClient
>>API and associated set of interfaces to submit jobs.
>>
>>Mike C. -- if you don't want the Job to go into the Resource Manager
>>queue,
>>then use you must check the load first, and be sure that the job will be
>>tasked immediately. Not sure why you wouldn't just put the job in both
>>queues
>>RM and RabbitMQ.. let me know if that works.
>>
>>Cheers,
>>Chris
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Senior Computer Scientist
>>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>Office: 171-266B, Mailstop: 171-246
>>Email: [email protected]
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Adjunct Assistant Professor, Computer Science Department
>>University of Southern California, Los Angeles, CA 90089 USA
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>
>>
>>-----Original Message-----
>>From: <Iwunze>, "Michael C [NOAA-JPSS] (GSFC-4700)"
>><[email protected]>
>>Reply-To: "[email protected]" <[email protected]>
>>Date: Tuesday, April 2, 2013 8:40 AM
>>To: "[email protected]" <[email protected]>
>>Subject: Re: Workflow Questions
>>
>>>Hey Chris,
>>>
>>> I have submitted Jobs to the Resource Manager without using
>>>Workflow.
>>>There is
>>>JobSumitter tool in OODT that can do this for you.
>>>
>>>
>>>Mike
>>>
>>>On 4/2/13 11:32 AM, "Cayanan, Michael D"
>>><[email protected]>
>>>wrote:
>>>
>>>>Hey Chris,
>>>>
>>>>I suppose I could. For some reason I didn't think that it was possible
>>>>to
>>>>submit a job to the Resource Manager other than through the Workflow.
>>>>In my case, I'm using a listener to pull files out of the RabbitMQ.
>>>>After
>>>>I pull a file out of the queue, I'd like to submit a job to the
>>>>Resource
>>>>Manager to trigger a Workflow Event. How would one submit a job this
>>>>way?
>>>>Under the Resource Manager trunk, in src/main/resources/examples/jobs,
>>>>I
>>>>see 2 xml files that appear to be config files. Do I need to make use
>>>>of
>>>>these to do what I want to do?
>>>>
>>>>Also, I know that the Resource Manager has it's own queue where it
>>>>places
>>>>jobs when all of its compute nodes are full. In my case (this is for
>>>>Costin's pipeline BTW), I don't want jobs being placed in the Resource
>>>>Manager queue. I would like for them to remain in the Rabbit Queue. Is
>>>>the
>>>>Resource Manager's "getNodeLoad()" method the best way to check if a
>>>>node
>>>>is full before submitting a job?
>>>>
>>>>Thanks in advance for your help!
>>>>
>>>>-Mike
>>>>
>>>>
>>>>On 4/1/13 10:15 PM, "Mattmann, Chris A (388J)"
>>>><[email protected]> wrote:
>>>>
>>>>>Hey Mike,
>>>>>
>>>>>I'll bite, can't you use the Resource Manager and its queues/nodes to
>>>>>do
>>>>>this?
>>>>>
>>>>>Cheers,
>>>>>Chris
>>>>>
>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>Chris Mattmann, Ph.D.
>>>>>Senior Computer Scientist
>>>>>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>Office: 171-266B, Mailstop: 171-246
>>>>>Email: [email protected]
>>>>>WWW: http://sunset.usc.edu/~mattmann/
>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>Adjunct Assistant Professor, Computer Science Department
>>>>>University of Southern California, Los Angeles, CA 90089 USA
>>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>-----Original Message-----
>>>>>From: <Cayanan>, "Michael D (388J)" <[email protected]>
>>>>>Reply-To: "[email protected]" <[email protected]>
>>>>>Date: Monday, April 1, 2013 8:11 PM
>>>>>To: "[email protected]" <[email protected]>
>>>>>Subject: Workflow Questions
>>>>>
>>>>>>Hi all,
>>>>>>
>>>>>>
>>>>>>I'm trying to see if anyone knows of a cool, easy to use "Resource
>>>>>>Manager" that can control the amount of Workflows running
>>>>>>concurrently
>>>>>>on
>>>>>>a node. Basically, my set up is like this:
>>>>>>
>>>>>>
>>>>>>- I have a Workflow Manager Server configured to run at most 10
>>>>>>Workflow
>>>>>>Events concurrently.
>>>>>>- I have a bunch of files sitting on a RabbitMQ queue.
>>>>>>- I have a listener that is continually monitoring this particular
>>>>>>queue.
>>>>>>As soon as it sees a file in this queue, it will trigger a Workflow
>>>>>>event.
>>>>>>
>>>>>>
>>>>>>Under this scenario, normally if I have 30 files in the RabbitMQ, my
>>>>>>listener will trigger 30 Workflow events. However, the Workflow
>>>>>>Server
>>>>>>would put 20 of these events in its repository queue while the other
>>>>>>10
>>>>>>are running.
>>>>>>
>>>>>>
>>>>>>What I want some "Resource Manager" component to do is just run the
>>>>>>10
>>>>>>Workflow events only and have the other 20 sitting in the RabbitMQ.
>>>>>>As
>>>>>>soon as 1 event has finished, then trigger another workflow event to
>>>>>>keep
>>>>>>the number of concurrent workflows running
>>>>>> at 10.
>>>>>>Eventually, we will want this "Resource Manager" component to be able
>>>>>>to
>>>>>>point to another node, node 2, where it can decide to trigger another
>>>>>>set
>>>>>>of 10 Workflow events while node 1 is still running it's 10 Workflow
>>>>>>events.
>>>>>>
>>>>>>
>>>>>>Hope this makes sense. Any helpful tips would be much appreciated.
>>>>>>
>>>>>>
>>>>>>Thanks,
>>>>>>Mike
>>>>>
>>>>
>>>
>>
>
