Re: Re: Hadoop NoClassDefFoundError

2014-04-16 Thread laozh...@sina.cn 






This is the result ./home/laozhao0 $ unzip -l myjob.jarArchive: 
/home/laozhao0/myjob.jar
Length Date Time Name
- -- - 
0 04-15-2014 14:54 META-INF/
71 04-15-2014 14:54 META-INF/MANIFEST.MF
1300 04-15-2014 14:54 MyJob$MapClass.class
1628 04-15-2014 14:54 MyJob$Reduce.class
2160 04-15-2014 14:54 MyJob.class
- ---
5159 5 files



laozh...@sina.cn
 From: Stanley ShiDate: 2014-04-16 10:25To: user@hadoop.apache.orgSubject: Re: 
Re: Hadoop NoClassDefFoundErrorcan do you an unzip -l myjob.jar to see if 
your jar file has the correct hierarchy?
Regards,Stanley Shi,




On Tue, Apr 15, 2014 at 6:53 PM, laozh...@sina.cn laozh...@sina.cn wrote:


Thank you for your advice . When i user your command , i get the below error 
info .
$ hadoop jar myjob.jar myjob.MyJob input output
Exception in thread main java.lang.ClassNotFoundException: myjob.MyJob
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)




 
From: Azuryy YuDate: 2014-04-15 16:14To: user@hadoop.apache.org
Subject: Re: Hadoop NoClassDefFoundErrorPlease use: hadoop jar myjob.jar 
myjob.MyJob input output



On Tue, Apr 15, 2014 at 3:06 PM, laozh...@sina.cn laozh...@sina.cn wrote:


Hello EveryOne:    I am new to hadoop,and i am reading Hadoop in action.When i 
tried to run a demo from this book,I got a problem and could not find answer 
from the net. Can you help me on this ?


below is the error info :




 $ hadoop jar myjob.jar MyJob input output

Exception in thread main java.lang.NoClassDefFoundError: MyJob (wrong name: 
myjob/MyJob)
at java.lang.ClassLoader.defineClass1(Native Method)


at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)


at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)


at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)


at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)


at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)



and this is the command that i compile the .java , I compiled in Win7 and ran 
on ubuntu .



below is MyJob.java
package myjob;



import java.io.IOException;

import java.util.Iterator;



import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.conf.Configured;

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapred.FileInputFormat;

import org.apache.hadoop.mapred.FileOutputFormat;

import org.apache.hadoop.mapred.JobClient;

import org.apache.hadoop.mapred.JobConf;

import org.apache.hadoop.mapred.KeyValueTextInputFormat;

import org.apache.hadoop.mapred.MapReduceBase;

import org.apache.hadoop.mapred.Mapper;

import org.apache.hadoop.mapred.OutputCollector;

import org.apache.hadoop.mapred.Reducer;

import org.apache.hadoop.mapred.Reporter;

import org.apache.hadoop.mapred.TextOutputFormat;

import org.apache.hadoop.util.Tool;

import org.apache.hadoop.util.ToolRunner;


public class MyJob extends Configured implements Tool{



@Override

public int run(String[] args) throws Exception {

Configuration conf = getConf();

JobConf job = new JobConf(conf,MyJob.class);

Path in = new Path(args[0]);

Path out = new Path(args[1]);

FileInputFormat.setInputPaths(job, in);

FileOutputFormat.setOutputPath(job, out);

job.setJobName(MyJob);

job.setJarByClass(MyJob.class);

job.setMapperClass(MapClass.class);

job.setReducerClass(Reduce.class);



job.setInputFormat(KeyValueTextInputFormat.class);

job.setOutputFormat(TextOutputFormat.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(Text.class);

job.set(key.value.separator.in.input.line,,);

JobClient.runJob(job);

return 0;

}



public static class MapClass extends MapReduceBase implements 
MapperText,Text,Text,Text{



@Override

public void map(Text key,

Re: Re: Hadoop NoClassDefFoundError

2014-04-16 Thread Rahul Singh 
When i unzip my jar.. i get the class files inside the package hirarchy
while in your case all files are lying outside? are you  sure your jar was
properly created?


On Wed, Apr 16, 2014 at 1:21 PM, laozh...@sina.cn laozh...@sina.cn wrote:

 This is the result .
 /home/laozhao0 $ unzip -l myjob.jar
 Archive: /home/laozhao0/myjob.jar
 Length Date Time Name
 - -- - 
 0 04-15-2014 14:54 META-INF/
 71 04-15-2014 14:54 META-INF/MANIFEST.MF
 1300 04-15-2014 14:54 MyJob$MapClass.class
 1628 04-15-2014 14:54 MyJob$Reduce.class
 2160 04-15-2014 14:54 MyJob.class
 - ---
 5159 5 files

 --
 laozh...@sina.cn


 *From:* Stanley Shi s...@gopivotal.com
 *Date:* 2014-04-16 10:25
 *To:* user@hadoop.apache.org
 *Subject:* Re: Re: Hadoop NoClassDefFoundError
 can do you an unzip -l myjob.jar to see if your jar file has the correct
 hierarchy?

 Regards,
 *Stanley Shi,*



 On Tue, Apr 15, 2014 at 6:53 PM, laozh...@sina.cn laozh...@sina.cnwrote:

 Thank you for your advice . When i user your command , i get the below
 error info .
 $ hadoop jar myjob.jar myjob.MyJob input output
 Exception in thread main java.lang.ClassNotFoundException: myjob.MyJob
 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

 --


 *From:* Azuryy Yu azury...@gmail.com
 *Date:* 2014-04-15 16:14
 *To:* user@hadoop.apache.org
 *Subject:* Re: Hadoop NoClassDefFoundError
 Please use: hadoop jar myjob.jar myjob.MyJob input output


 On Tue, Apr 15, 2014 at 3:06 PM, laozh...@sina.cn laozh...@sina.cnwrote:

 Hello EveryOne:
 I am new to hadoop,and i am reading Hadoop in action.
 When i tried to run a demo from this book,I got a problem and could not
 find answer from the net. Can you help me on this ?

 below is the error info :

   $ hadoop jar myjob.jar MyJob input output
 Exception in thread main java.lang.NoClassDefFoundError: MyJob (wrong
 name: myjob/MyJob)
 at java.lang.ClassLoader.defineClass1(Native Method)
 at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
 at
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

 and this is the command that i compile the .java , I compiled in Win7
 and ran on ubuntu .


 below is MyJob.java

 package myjob;

 import java.io.IOException;
 import java.util.Iterator;

 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.conf.Configured;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.io.Text;
 import org.apache.hadoop.mapred.FileInputFormat;
 import org.apache.hadoop.mapred.FileOutputFormat;
 import org.apache.hadoop.mapred.JobClient;
 import org.apache.hadoop.mapred.JobConf;
 import org.apache.hadoop.mapred.KeyValueTextInputFormat;
 import org.apache.hadoop.mapred.MapReduceBase;
 import org.apache.hadoop.mapred.Mapper;
 import org.apache.hadoop.mapred.OutputCollector;
 import org.apache.hadoop.mapred.Reducer;
 import org.apache.hadoop.mapred.Reporter;
 import org.apache.hadoop.mapred.TextOutputFormat;
 import org.apache.hadoop.util.Tool;
 import org.apache.hadoop.util.ToolRunner;

 public class MyJob extends Configured implements Tool{

 @Override
 public int run(String[] args) throws Exception {
 Configuration conf = getConf();
 JobConf job = new JobConf(conf,MyJob.class);
 Path in = new Path(args[0]);
 Path out = new Path(args[1]);
 FileInputFormat.setInputPaths(job, in);
 FileOutputFormat.setOutputPath(job, out);
 job.setJobName(MyJob);
 job.setJarByClass(MyJob.class);
 job.setMapperClass(MapClass.class);
 job.setReducerClass(Reduce.class);

 job.setInputFormat(KeyValueTextInputFormat.class);
 job.setOutputFormat(TextOutputFormat.class);
 job.setOutputKeyClass(Text.class);
 job.setOutputValueClass(Text.class);
 job.set(key.value.separator.in.input.line,,);
 JobClient.runJob(job);
 return 0;
 }

 public static class MapClass extends MapReduceBase implements
 MapperText,Text,Text,Text

Hadoop NoClassDefFoundError

2014-04-15 Thread laozh...@sina.cn 






Hello EveryOne:    I am new to hadoop,and i am reading Hadoop in action.When i 
tried to run a demo from this book,I got a problem and could not find answer 
from the net. Can you help me on this ?
below is the error info :
 $ hadoop jar myjob.jar MyJob input outputException in thread main 
java.lang.NoClassDefFoundError: MyJob (wrong name: myjob/MyJob)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

and this is the command that i compile the .java , I compiled in Win7 and ran 
on ubuntu .

below is MyJob.java
package myjob;



import java.io.IOException;

import java.util.Iterator;



import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.conf.Configured;

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapred.FileInputFormat;

import org.apache.hadoop.mapred.FileOutputFormat;

import org.apache.hadoop.mapred.JobClient;

import org.apache.hadoop.mapred.JobConf;

import org.apache.hadoop.mapred.KeyValueTextInputFormat;

import org.apache.hadoop.mapred.MapReduceBase;

import org.apache.hadoop.mapred.Mapper;

import org.apache.hadoop.mapred.OutputCollector;

import org.apache.hadoop.mapred.Reducer;

import org.apache.hadoop.mapred.Reporter;

import org.apache.hadoop.mapred.TextOutputFormat;

import org.apache.hadoop.util.Tool;

import org.apache.hadoop.util.ToolRunner;


public class MyJob extends Configured implements Tool{



@Override

public int run(String[] args) throws Exception {

Configuration conf = getConf();

JobConf job = new JobConf(conf,MyJob.class);

Path in = new Path(args[0]);

Path out = new Path(args[1]);

FileInputFormat.setInputPaths(job, in);

FileOutputFormat.setOutputPath(job, out);

job.setJobName(MyJob);

job.setJarByClass(MyJob.class);

job.setMapperClass(MapClass.class);

job.setReducerClass(Reduce.class);



job.setInputFormat(KeyValueTextInputFormat.class);

job.setOutputFormat(TextOutputFormat.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(Text.class);

job.set(key.value.separator.in.input.line,,);

JobClient.runJob(job);

return 0;

}



public static class MapClass extends MapReduceBase implements 
MapperText,Text,Text,Text{



@Override

public void map(Text key, Text value, OutputCollectorText, 
Text output,

Reporter reporter) throws IOException {

output.collect(value, key);

}

}



public static class Reduce extends MapReduceBase implements 
ReducerText,Text,Text,Text{



@Override

public void reduce(Text key, IteratorText values,

OutputCollectorText, Text output, Reporter 
reporter)

throws IOException {

String csv = ;

while(values.hasNext()){

if(csv.length()  0)

csv += ,;

csv += values.next().toString();

}

}

}



public static void main(String[] args) throws Exception {

int res = ToolRunner.run(new Configuration(), new MyJob(), 
args);

System.exit(res);

}

} 

Thank you for your kindly help !

2014-04-15_150135.png

Re: Hadoop NoClassDefFoundError

2014-04-15 Thread Azuryy Yu 
Please use: hadoop jar myjob.jar myjob.MyJob input output


On Tue, Apr 15, 2014 at 3:06 PM, laozh...@sina.cn laozh...@sina.cn wrote:

 Hello EveryOne:
 I am new to hadoop,and i am reading Hadoop in action.
 When i tried to run a demo from this book,I got a problem and could not
 find answer from the net. Can you help me on this ?

 below is the error info :

  $ hadoop jar myjob.jar MyJob input output
 Exception in thread main java.lang.NoClassDefFoundError: MyJob (wrong
 name: myjob/MyJob)
 at java.lang.ClassLoader.defineClass1(Native Method)
 at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

 and this is the command that i compile the .java , I compiled in Win7 and
 ran on ubuntu .


 below is MyJob.java

 package myjob;

 import java.io.IOException;
 import java.util.Iterator;

 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.conf.Configured;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.io.Text;
 import org.apache.hadoop.mapred.FileInputFormat;
 import org.apache.hadoop.mapred.FileOutputFormat;
 import org.apache.hadoop.mapred.JobClient;
 import org.apache.hadoop.mapred.JobConf;
 import org.apache.hadoop.mapred.KeyValueTextInputFormat;
 import org.apache.hadoop.mapred.MapReduceBase;
 import org.apache.hadoop.mapred.Mapper;
 import org.apache.hadoop.mapred.OutputCollector;
 import org.apache.hadoop.mapred.Reducer;
 import org.apache.hadoop.mapred.Reporter;
 import org.apache.hadoop.mapred.TextOutputFormat;
 import org.apache.hadoop.util.Tool;
 import org.apache.hadoop.util.ToolRunner;

 public class MyJob extends Configured implements Tool{

 @Override
 public int run(String[] args) throws Exception {
 Configuration conf = getConf();
 JobConf job = new JobConf(conf,MyJob.class);
 Path in = new Path(args[0]);
 Path out = new Path(args[1]);
 FileInputFormat.setInputPaths(job, in);
 FileOutputFormat.setOutputPath(job, out);
 job.setJobName(MyJob);
 job.setJarByClass(MyJob.class);
 job.setMapperClass(MapClass.class);
 job.setReducerClass(Reduce.class);

 job.setInputFormat(KeyValueTextInputFormat.class);
 job.setOutputFormat(TextOutputFormat.class);
 job.setOutputKeyClass(Text.class);
 job.setOutputValueClass(Text.class);
 job.set(key.value.separator.in.input.line,,);
 JobClient.runJob(job);
 return 0;
 }

 public static class MapClass extends MapReduceBase implements
 MapperText,Text,Text,Text{

 @Override
 public void map(Text key, Text value, OutputCollectorText, Text output,
 Reporter reporter) throws IOException {
 output.collect(value, key);
 }
 }

 public static class Reduce extends MapReduceBase implements
 ReducerText,Text,Text,Text{

 @Override
 public void reduce(Text key, IteratorText values,
 OutputCollectorText, Text output, Reporter reporter)
 throws IOException {
 String csv = ;
 while(values.hasNext()){
 if(csv.length()  0)
 csv += ,;
 csv += values.next().toString();
 }
 }
 }

 public static void main(String[] args) throws Exception {
 int res = ToolRunner.run(new Configuration(), new MyJob(), args);
 System.exit(res);
 }
 }
 --
 Thank you for your kindly help !

inline: 2014-04-15_150135.png

Re: Hadoop NoClassDefFoundError

2014-04-15 Thread Azuryy Yu 
Please use: hadoop jar myjob.jar myjob.MyJob input output


On Tue, Apr 15, 2014 at 3:06 PM, laozh...@sina.cn laozh...@sina.cn wrote:

 Hello EveryOne:
 I am new to hadoop,and i am reading Hadoop in action.
 When i tried to run a demo from this book,I got a problem and could not
 find answer from the net. Can you help me on this ?

 below is the error info :

  $ hadoop jar myjob.jar MyJob input output
 Exception in thread main java.lang.NoClassDefFoundError: MyJob (wrong
 name: myjob/MyJob)
 at java.lang.ClassLoader.defineClass1(Native Method)
 at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

 and this is the command that i compile the .java , I compiled in Win7 and
 ran on ubuntu .


 below is MyJob.java

 package myjob;

 import java.io.IOException;
 import java.util.Iterator;

 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.conf.Configured;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.io.Text;
 import org.apache.hadoop.mapred.FileInputFormat;
 import org.apache.hadoop.mapred.FileOutputFormat;
 import org.apache.hadoop.mapred.JobClient;
 import org.apache.hadoop.mapred.JobConf;
 import org.apache.hadoop.mapred.KeyValueTextInputFormat;
 import org.apache.hadoop.mapred.MapReduceBase;
 import org.apache.hadoop.mapred.Mapper;
 import org.apache.hadoop.mapred.OutputCollector;
 import org.apache.hadoop.mapred.Reducer;
 import org.apache.hadoop.mapred.Reporter;
 import org.apache.hadoop.mapred.TextOutputFormat;
 import org.apache.hadoop.util.Tool;
 import org.apache.hadoop.util.ToolRunner;

 public class MyJob extends Configured implements Tool{

 @Override
 public int run(String[] args) throws Exception {
 Configuration conf = getConf();
 JobConf job = new JobConf(conf,MyJob.class);
 Path in = new Path(args[0]);
 Path out = new Path(args[1]);
 FileInputFormat.setInputPaths(job, in);
 FileOutputFormat.setOutputPath(job, out);
 job.setJobName(MyJob);
 job.setJarByClass(MyJob.class);
 job.setMapperClass(MapClass.class);
 job.setReducerClass(Reduce.class);

 job.setInputFormat(KeyValueTextInputFormat.class);
 job.setOutputFormat(TextOutputFormat.class);
 job.setOutputKeyClass(Text.class);
 job.setOutputValueClass(Text.class);
 job.set(key.value.separator.in.input.line,,);
 JobClient.runJob(job);
 return 0;
 }

 public static class MapClass extends MapReduceBase implements
 MapperText,Text,Text,Text{

 @Override
 public void map(Text key, Text value, OutputCollectorText, Text output,
 Reporter reporter) throws IOException {
 output.collect(value, key);
 }
 }

 public static class Reduce extends MapReduceBase implements
 ReducerText,Text,Text,Text{

 @Override
 public void reduce(Text key, IteratorText values,
 OutputCollectorText, Text output, Reporter reporter)
 throws IOException {
 String csv = ;
 while(values.hasNext()){
 if(csv.length()  0)
 csv += ,;
 csv += values.next().toString();
 }
 }
 }

 public static void main(String[] args) throws Exception {
 int res = ToolRunner.run(new Configuration(), new MyJob(), args);
 System.exit(res);
 }
 }
 --
 Thank you for your kindly help !

inline: 2014-04-15_150135.png

Re: Re: Hadoop NoClassDefFoundError

2014-04-15 Thread laozh...@sina.cn 






Thank you for your advice . When i user your command , i get the below error 
info .$ hadoop jar myjob.jar myjob.MyJob input outputException in thread main 
java.lang.ClassNotFoundException: myjob.MyJob
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)




 From: Azuryy YuDate: 2014-04-15 16:14To: user@hadoop.apache.orgSubject: Re: 
Hadoop NoClassDefFoundErrorPlease use: hadoop jar myjob.jar myjob.MyJob input 
output


On Tue, Apr 15, 2014 at 3:06 PM, laozh...@sina.cn laozh...@sina.cn wrote:


Hello EveryOne:    I am new to hadoop,and i am reading Hadoop in action.When i 
tried to run a demo from this book,I got a problem and could not find answer 
from the net. Can you help me on this ?

below is the error info :


 $ hadoop jar myjob.jar MyJob input output
Exception in thread main java.lang.NoClassDefFoundError: MyJob (wrong name: 
myjob/MyJob)
at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)

at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)


and this is the command that i compile the .java , I compiled in Win7 and ran 
on ubuntu .


below is MyJob.java
package myjob;



import java.io.IOException;

import java.util.Iterator;



import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.conf.Configured;

import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapred.FileInputFormat;

import org.apache.hadoop.mapred.FileOutputFormat;

import org.apache.hadoop.mapred.JobClient;

import org.apache.hadoop.mapred.JobConf;

import org.apache.hadoop.mapred.KeyValueTextInputFormat;

import org.apache.hadoop.mapred.MapReduceBase;

import org.apache.hadoop.mapred.Mapper;

import org.apache.hadoop.mapred.OutputCollector;

import org.apache.hadoop.mapred.Reducer;

import org.apache.hadoop.mapred.Reporter;

import org.apache.hadoop.mapred.TextOutputFormat;

import org.apache.hadoop.util.Tool;

import org.apache.hadoop.util.ToolRunner;


public class MyJob extends Configured implements Tool{



@Override

public int run(String[] args) throws Exception {

Configuration conf = getConf();

JobConf job = new JobConf(conf,MyJob.class);

Path in = new Path(args[0]);

Path out = new Path(args[1]);

FileInputFormat.setInputPaths(job, in);

FileOutputFormat.setOutputPath(job, out);

job.setJobName(MyJob);

job.setJarByClass(MyJob.class);

job.setMapperClass(MapClass.class);

job.setReducerClass(Reduce.class);



job.setInputFormat(KeyValueTextInputFormat.class);

job.setOutputFormat(TextOutputFormat.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(Text.class);

job.set(key.value.separator.in.input.line,,);

JobClient.runJob(job);

return 0;

}



public static class MapClass extends MapReduceBase implements 
MapperText,Text,Text,Text{



@Override

public void map(Text key, Text value, OutputCollectorText, 
Text output,

Reporter reporter) throws IOException {

output.collect(value, key);

}

}



public static class Reduce extends MapReduceBase implements 
ReducerText,Text,Text,Text{



@Override

public void reduce(Text key, IteratorText values,

OutputCollectorText, Text output, Reporter 
reporter)

throws IOException {

String csv = ;

while(values.hasNext()){

if(csv.length()  0)

Re: Re: Hadoop NoClassDefFoundError

2014-04-15 Thread Stanley Shi 
can do you an unzip -l myjob.jar to see if your jar file has the correct
hierarchy?

Regards,
*Stanley Shi,*



On Tue, Apr 15, 2014 at 6:53 PM, laozh...@sina.cn laozh...@sina.cn wrote:

 Thank you for your advice . When i user your command , i get the below
 error info .
 $ hadoop jar myjob.jar myjob.MyJob input output
 Exception in thread main java.lang.ClassNotFoundException: myjob.MyJob
 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

 --


 *From:* Azuryy Yu azury...@gmail.com
 *Date:* 2014-04-15 16:14
 *To:* user@hadoop.apache.org
 *Subject:* Re: Hadoop NoClassDefFoundError
 Please use: hadoop jar myjob.jar myjob.MyJob input output


 On Tue, Apr 15, 2014 at 3:06 PM, laozh...@sina.cn laozh...@sina.cnwrote:

 Hello EveryOne:
 I am new to hadoop,and i am reading Hadoop in action.
 When i tried to run a demo from this book,I got a problem and could not
 find answer from the net. Can you help me on this ?

 below is the error info :

   $ hadoop jar myjob.jar MyJob input output
 Exception in thread main java.lang.NoClassDefFoundError: MyJob (wrong
 name: myjob/MyJob)
 at java.lang.ClassLoader.defineClass1(Native Method)
 at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

 and this is the command that i compile the .java , I compiled in Win7 and
 ran on ubuntu .


 below is MyJob.java

 package myjob;

 import java.io.IOException;
 import java.util.Iterator;

 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.conf.Configured;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.io.Text;
 import org.apache.hadoop.mapred.FileInputFormat;
 import org.apache.hadoop.mapred.FileOutputFormat;
 import org.apache.hadoop.mapred.JobClient;
 import org.apache.hadoop.mapred.JobConf;
 import org.apache.hadoop.mapred.KeyValueTextInputFormat;
 import org.apache.hadoop.mapred.MapReduceBase;
 import org.apache.hadoop.mapred.Mapper;
 import org.apache.hadoop.mapred.OutputCollector;
 import org.apache.hadoop.mapred.Reducer;
 import org.apache.hadoop.mapred.Reporter;
 import org.apache.hadoop.mapred.TextOutputFormat;
 import org.apache.hadoop.util.Tool;
 import org.apache.hadoop.util.ToolRunner;

 public class MyJob extends Configured implements Tool{

 @Override
 public int run(String[] args) throws Exception {
 Configuration conf = getConf();
 JobConf job = new JobConf(conf,MyJob.class);
 Path in = new Path(args[0]);
 Path out = new Path(args[1]);
 FileInputFormat.setInputPaths(job, in);
 FileOutputFormat.setOutputPath(job, out);
 job.setJobName(MyJob);
 job.setJarByClass(MyJob.class);
 job.setMapperClass(MapClass.class);
 job.setReducerClass(Reduce.class);

 job.setInputFormat(KeyValueTextInputFormat.class);
 job.setOutputFormat(TextOutputFormat.class);
 job.setOutputKeyClass(Text.class);
 job.setOutputValueClass(Text.class);
 job.set(key.value.separator.in.input.line,,);
 JobClient.runJob(job);
 return 0;
 }

 public static class MapClass extends MapReduceBase implements
 MapperText,Text,Text,Text{

 @Override
 public void map(Text key, Text value, OutputCollectorText, Text output,
 Reporter reporter) throws IOException {
 output.collect(value, key);
 }
 }

 public static class Reduce extends MapReduceBase implements
 ReducerText,Text,Text,Text{

 @Override
 public void reduce(Text key, IteratorText values,
 OutputCollectorText, Text output, Reporter reporter)
 throws IOException {
 String csv = ;
 while(values.hasNext()){
 if(csv.length()  0)
 csv += ,;
 csv += values.next().toString();
 }
 }
 }

 public static void main(String[] args) throws Exception {
 int res = ToolRunner.run(new Configuration(), new MyJob(), args);
 System.exit(res);
 }
 }
 --
 Thank you for your kindly help !



inline: 2014-04-15_15013(04-15-18-51-38).png