java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase

2012-05-04 Thread 唐方爽
Hi, I try to run a Hadoop reduce-side join, then I get the following: java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791)

java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase

2012-05-04 Thread 唐方爽
Hi, I try to run a Hadoop reduce-side join, then I get the following: java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791)

Re: Splitting data input to Distcp

2012-05-04 Thread Pedro Figueiredo
On 3 May 2012, at 23:47, Himanshu Vijay wrote: Pedro, Thanks for the response. Unfortunately I am running it on in-house cluster and from there I need to upload to S3. Hi, Last night I was thinking about this... what happens if you copy

java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase

2012-05-04 Thread 唐方爽
Hi, I try to run a Hadoop reduce-side join, then I get the following: java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at

Re: java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase

2012-05-04 Thread JunYong Li
is any other error log, check 1. JoinHadoop.jar collectly submit to hadoop 2. DataJoinMapperBase really in the JoinHadoop.jar 2012/5/4 唐方爽 fstang...@gmail.com Hi, I try to run a Hadoop reduce-side join, then I get the following: java.lang.NoClassDefFoundError:

Re: java.lang.NoClassDefFoundError: org/apache/hadoop/contrib/utils/join/DataJoinMapperBase

2012-05-04 Thread 唐方爽
DataJoinMapperBase is not in the JoinHadoop.jar. When I add it and related classes to JoinHadoop.jar, it works! (although I got an IOException at reduce stage... maybe I should check the code or input files) thanks! 2012/5/4 JunYong Li lij...@gmail.com is any other error log, check 1.

How to create an archive-file in Java to distribute a MapFile via Distributed Cache

2012-05-04 Thread i...@christianherta.de
Hello, I have written a chain of map-reduce jobs which creates a Mapfile. I want to use the Mapfile in a proximate map-reduce job via distributed cache. Therefore I have to create an archive file of the folder with holds the /data and /index files. In the documentation and in the Book Hadoop the

Re: How to create an archive-file in Java to distribute a MapFile via Distributed Cache

2012-05-04 Thread Harsh J
Hi, The Java API offers a DistributedCache class which lets you do this. The usage is detailed at http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/filecache/DistributedCache.html On Fri, May 4, 2012 at 5:11 PM, i...@christianherta.de i...@christianherta.de wrote: Hello, I

Re: How to create an archive-file in Java to distribute a MapFile via Distributed Cache

2012-05-04 Thread Shi Yu
My humble experience: I would prefer specifying the files in command line using -files option, then treat them explicitly in the Mapper configure or setup function using File f1 = new File(file1name); File f2 = new File(file2name); Cause I am not 100% sure how does distributed cached

Bad connect ack with firstBadLink

2012-05-04 Thread madhu phatak
Hi, We are running a three node cluster . From two days whenever we copy file to hdfs , it is throwing java.IO.Exception Bad connect ack with firstBadLink . I searched in net, but not able to resolve the issue. The following is the stack trace from datanode log 2012-05-04 18:08:08,868 INFO

Re: Bad connect ack with firstBadLink

2012-05-04 Thread Mohit Anchlia
Please see: http://hbase.apache.org/book.html#dfs.datanode.max.xcievers On Fri, May 4, 2012 at 5:46 AM, madhu phatak phatak@gmail.com wrote: Hi, We are running a three node cluster . From two days whenever we copy file to hdfs , it is throwing java.IO.Exception Bad connect ack with

Re: Reduce Hangs at 66%

2012-05-04 Thread Michael Segel
Well That was one of the things I had asked. ulimit -a says it all. But you have to do this for the users... hdfs, mapred, and hadoop (Which is why I asked about which flavor.) On May 3, 2012, at 7:03 PM, Raj Vishwanathan wrote: Keith What is the the output for ulimit -n? Your value for

Re: Bad connect ack with firstBadLink

2012-05-04 Thread Mapred Learn
Check your number of blocks in the cluster. This also indicates that your datanodes are more full than they should be. Try deleting unnecessary blocks. On Fri, May 4, 2012 at 7:40 AM, Mohit Anchlia mohitanch...@gmail.comwrote: Please see:

Re: Reduce Hangs at 66%

2012-05-04 Thread Keith Thompson
Thanks everyone for your help. It is running fine now. On Fri, May 4, 2012 at 11:22 AM, Michael Segel michael_se...@hotmail.comwrote: Well That was one of the things I had asked. ulimit -a says it all. But you have to do this for the users... hdfs, mapred, and hadoop (Which is why I

Fwd: May 21 talk at Pasadena JUG

2012-05-04 Thread Mattmann, Chris A (388J)
(apologies for cross posting) Hey Folks in the SoCal area -- if you're around on May 21st, I'll be speaking at the Pasadena JUG on Apache OODT, Big Data and likely Apache Hadoop (in prep for my Hadoop Summit coming talk). Info is below thanks to David Noble for setting this up! Cheers, Chris

Need to improve documentation for v 0.23.x ( v 2.x)

2012-05-04 Thread Jagat
Hello All, As Apache Hadoop community is ready to release the next 2.0 alpha version of Hadoop , i would like to bring attention towards need to make better documentation of the tutorials and examples for the same. Just one short example See the Single Node Setup tutorials for v

Re: How to add debugging to map- red code

2012-05-04 Thread Mapred Learn
Hi Harsh, Could you show one sample of how to do this ? I have not seen/written any mapper code where people use log4j logger or log4j file to set the log level. Thanks in advance -JJ On Thu, May 3, 2012 at 4:32 PM, Harsh J ha...@cloudera.com wrote: Doing (ii) would be an isolated app-level

Re: How to add debugging to map- red code

2012-05-04 Thread Nitin Pawar
here is a sample code from log4j documentation if you want to specify a specific file where you want to write the log .. you can have a log4j properties file and add it to the classpath import com.foo.Bar; // Import log4j classes. *import org.apache.log4j.Logger; import

Re: How to add debugging to map- red code

2012-05-04 Thread Mapred Learn
Thanks Nitin but I was asking in context to mapper code.. Sent from my iPhone On May 4, 2012, at 9:06 PM, Nitin Pawar nitinpawar...@gmail.com wrote: here is a sample code from log4j documentation if you want to specify a specific file where you want to write the log .. you can have a log4j