Re: enable snappy on hadoop 1.1.1

2013-10-05 Thread bharath vissapragada
whats the output of ldd on that lib? Does it link properly? You should compile natives for your platforms as the packaged ones may not link properly. On Sat, Oct 5, 2013 at 2:37 AM, java8964 java8964 wrote: > I kind of read the hadoop 1.1.1 source code for this, it is very strange > for me now.

MapReduce task-worker assignment

2013-10-05 Thread John Lilley
Is there a description of how MapReduce under Hadoop 2.0 assigns mapper tasks to preferred nodes? I think that someone on the list mentioned previously that it attempted to assign "one HDFS block per mapper task", but given that there can be multiple block instances per data split, how does Map

RE: Non data-local scheduling

2013-10-05 Thread John Lilley
Is this option set on a per-application-instance basis or is it a cluster-wide setting (or both)? Is this a MapReduce-specific issue, or a YARN issue? I don't understand how the problem arises in the first place. For example, if I have an idle cluster with 10 nodes and each node has four contain