Re: container allocation

By default, the ResourceManager will try give you a container on that node, rack or anywhere (in that order). We recently added ability to whitelist or blacklist nodes to allow for more control. Arun On Jun 12, 2013, at 8:03 AM, John Lilley wrote: > If I request a container on a node, and tha

RE: container allocation

about this all wrong... perhaps I should ask for containers, see what nodes they are on, and then assign the data splits to them once I see the set of available containers? john From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: Thursday, June 13, 2013 12:27 AM To: user@hadoop.apache.org Su

Re: container allocation

e the set of available containers? > > john > > *From:* Arun C Murthy [mailto:a...@hortonworks.com] > *Sent:* Thursday, June 13, 2013 12:27 AM > *To:* user@hadoop.apache.org > *Subject:* Re: container allocation > > ** ** > > By default, the ResourceManag

Re: Container allocation fails randomly

Can you give more information? logs (complete) will help a lot around this time frame. Are the containers getting assigned via scheduler? is it failing when node manager tries to start container? I clearly see the diagnostic message is empty but do you see anything in NM logs? Also if there were ru

Re: Container allocation fails randomly

Hi Omkar, Thanks for the quick reply, and sorry for not being able to get the required logs that you have asked for. But in the meanwhile I just wanted to check if you can get a clue with the information I have now. I am seeing the following kind of error message in AppMaster.stderr whenever

Re: Container allocation fails randomly

This is clearly an AM bug. are you using MR-AM or custom AM? you should check AM code which is computing progress. I suspect there must be some float computation problems. If it is an MR-AM problem then please file a map reduce bug. Thanks, Omkar Joshi *Hortonworks Inc.*

Re: Container allocation fails randomly

Hi Omkar, It is my own custom AM that I am using, not the MR-AM. But I am still not able to believe how can a negative value go from the getProgress() call which is always calculated from division of positive numbers, but might be some floating point computation problems as you are saying. Than

Re: Container allocation on the same node

You can request containers with the local host name as the required host, and perhaps reject and re-request if they aren't designated to be on that one until you have sufficient. This may take a while though. On Wed, Jun 12, 2013 at 6:25 PM, Krishna Kishore Bonagiri wrote: > Hi, > > I want to g

Re: Container allocation on the same node

Hi Harsh, What will happen when I specify local host as the required host? Doesn't the resource manager give me all the containers on the local host? I don't want to constrain myself to the local host, which might be busy while other nodes in the cluster have enough resources available for me.