Re: Re: How to use combiner in hadoop streaming

[xieyeshan]

Hi,Harsh J 

    I use a Java Class as combiner and get following error infomation:
Exception in thread "main" java.lang.NoClassDefFoundError: 
org/apache/hadoop/streaming/PlsaCombine (wrong name: PlsaCombine)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at 
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:761)
        at 
org.apache.hadoop.streaming.StreamUtil.goodClassOrNull(StreamUtil.java:56)
        at org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:709)
        at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:117)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at 
org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:32)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
      


Thanks!


2011-01-18 



xieyeshan 



发件人： Harsh J 
发送时间： 2011-01-18  16:07:05 
收件人： general 
抄送： 
主题： Re: How to use combiner in hadoop streaming 
 
Right now, the Combiner has to be a Java Class. A command is
unacceptable, as noted here:
http://wiki.apache.org/hadoop/HadoopStreaming
On Tue, Jan 18, 2011 at 1:30 PM, xieyeshan <xieyes...@software.ict.ac.cn> wrote:
> hi,all
>
>     I want to use combiner in hadoop streaming, but combiner is not the same 
> as mapper or reducer in streaming.How could I do this?
>     Thanks!
>
>
> Yours,
>
> Yeshan Xie
>
> 2011-01-18
>
>
>
> xieyeshan
>
-- 
Harsh J
www.harshj.com