RE: MapReduce task-worker assignment

2013-10-08 Thread John Lilley
Thanks! From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: Monday, October 07, 2013 6:07 PM To: user@hadoop.apache.org Subject: Re: MapReduce task-worker assignment Short version: MR provides all the info it can to (about all it's tasks locations) and the YARN scheduler deals

Re: MapReduce task-worker assignment

2013-10-07 Thread Arun C Murthy
Short version: MR provides all the info it can to (about all it's tasks locations) and the YARN scheduler deals with providing good locality with even assignment. I don't have a handy link to a doc, but here is an ancient version:

MapReduce task-worker assignment

2013-10-05 Thread John Lilley
Is there a description of how MapReduce under Hadoop 2.0 assigns mapper tasks to preferred nodes? I think that someone on the list mentioned previously that it attempted to assign one HDFS block per mapper task, but given that there can be multiple block instances per data split, how does