how to build hadoop-hdfs trunk locally

2010-12-24 Thread sravankumar
Hi, I am facing problems in building hadoop-hdfs It is trying to download hadoop-common-timestamp.jar Does anyone know how to overcome this problem. ivy:resolve] WARNINGS [ivy:resolve] module not found: org.apache.hadoop#hadoop-common;0.23.0-SNAPSHOT

Help: Could not obtain block: blk_ Exception

2010-12-24 Thread Tali K
Hi All, I am getting Could not obtain block: blk_2706642997966533027_4482 file=/user/outputwc425729652_0/part-r-0 I checked the file is actually there. What I should do? Please help. Could not obtain block: blk_2706642997966533027_4482 file=/user/outputwc425729652_0/part-r-0

RE: EXTERNAL:Tasktracker failing and getting black listed

2010-12-24 Thread Black, Michael (IS)
#1 Check CPU fan is working. A hot CPU can give flakey errorsespecially during high CPU load. #2 Do memtest on the machine. You might have a bad memory stick that is getting hit (though I would tend to think it would be a bit more random). I've used memtest86 before to find such problems.

Re: Hadoop/Elastic MR on AWS

2010-12-24 Thread Otis Gospodnetic
Hello Amandeep, - Original Message From: Amandeep Khurana ama...@gmail.com To: common-user@hadoop.apache.org Sent: Fri, December 10, 2010 1:14:45 AM Subject: Re: Hadoop/Elastic MR on AWS Mark, Using EMR makes it very easy to start a cluster and add/reduce capacity as and

Custom input split

2010-12-24 Thread Black, Michael (IS)
Using hadoop-0.20 I'm doing custom input splits from a Lucene index. I want to split the document ID's across N mappers (I'm testing the scalabilty of the problem across 4 nodes and 8 cores). So the key is the document# and they are not sequential. At this point I'm using splits.add to add

Re: Hadoop/Elastic MR on AWS

2010-12-24 Thread Ted Dunning
EMR instances are started near each other. This increases the bandwidth between nodes. There may also be some enhancements in terms of access to the SAN that supports EBS. On Fri, Dec 24, 2010 at 4:41 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: - Original Message From: