How run Aggregator wordcount?

2013-06-22 Thread Pedro Sá da Costa
Aggregator wordcount accept multiple folders as input? e.g. bin/hadoop jar hadoop-*-examples.jar aggregatewordcount inputfolder1 inputfolder2 inputfolder3 outfolder1 -- Best regards,

Re: how to turn on NativeIO.posixFadviseIfPossible

2013-06-22 Thread Jun Li
Hi Leo, Thanks for the detailed instruction. I started from the clean downloaded hadoop1.1.12 copy and then I issue the command: ant -Dcompile.native=true. And with the configure.ac that comes with the downloaded package, without any changes, the following is the compilation output:

How can a YarnTask read/write local-host HDFS blocks?

2013-06-22 Thread blah blah
Hi all *Disclaimer* I am creating a prototype Application Master. I am using old Yarn development version. Revision 1437315, from 2013-01-23 (SNAPSHOT 3.0.0). I can not update to current trunk version, as prototype deadline is soon, and I don't have time to include Yarn API changes. My cluster

Re: How can a YarnTask read/write local-host HDFS blocks?

2013-06-22 Thread Harsh J
Hi, On Sat, Jun 22, 2013 at 4:21 PM, blah blah tmp5...@gmail.com wrote: Hi all Disclaimer I am creating a prototype Application Master. I am using old Yarn development version. Revision 1437315, from 2013-01-23 (SNAPSHOT 3.0.0). I can not update to current trunk version, as prototype

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Prashant Kommireddi
Following-up on this. Please let me know if this is expected/bug and if you would like me to file a JIRA On Thu, Jun 20, 2013 at 9:45 PM, Prashant Kommireddi prash1...@gmail.comwrote: Hello, I came across an issue that occurs with the job notification callbacks in MR2. It works fine if the

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Ravi Prakash
Hi Prashant, I would tend to agree with you. Although job-end notification is only a best-effort mechanism (i.e. we cannot always guarantee notification for example when the AM OOMs), I agree with you that we can do more. If you feel strongly about this, please create a JIRA and possibly

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Prashant Kommireddi
Thanks Ravi. Well, in this case its a no-effort :) A failure of AM init should be considered as failure of the job? I looked at the code and best-effort makes sense with respect to retry logic etc. You make a good point that there would be no notification in case AM OOMs, but I do feel AM init

Re: Inputformat

2013-06-22 Thread Azuryy Yu
you had to write a JSONInputFormat, or google first to find it. --Send from my Sony mobile. On Jun 23, 2013 7:06 AM, jamal sasha jamalsha...@gmail.com wrote: Then how should I approach this issue? On Fri, Jun 21, 2013 at 4:25 PM, Niels Basjes ni...@basjes.nl wrote: If you try to hammer in