Hi Costin, I had downloaded the elasticsearch-hadoop-2.1.0.Beta1.zip file and used all the jars from that for the program. Later I even tried replacing all the jars in my program with jars from with elasticsearch-hadoop-2.0.0.zip file, but still facing the same error.
On Tue, Sep 9, 2014 at 6:52 PM, Costin Leau <costin.l...@gmail.com> wrote: > Most likely you have a classpath conflict caused by multiple versions of > es-hadoop. Can you double check you only have one version (2.1.0.Beta1) > available? > Based on the error, I'm guessing you have some 1.3 Mx or the RC somewhere > in there... > > On 9/9/14 4:06 PM, gaurav redkar wrote: > >> Hi Costin, >> >> Thanks for the heads up regarding gist. I will try to follow the >> guidelines in the future. As for my program, I am using >> Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed >> the output value class to LinkedMapWritable. but >> now I am getting the following error. >> >> https://gist.github.com/gauravub/7d55bc6b10cb63935eb8 >> >> Any idea why is this happening ? I even tried using the v2.0.0 of >> es-hadoop but am still getting the same error. >> >> On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.l...@gmail.com >> <mailto:costin.l...@gmail.com>> wrote: >> >> Hi, >> >> What version of es-hadoop are you using? The problem stems from the >> difference in the types mentioned on your >> Mapper, namely the output value class: >> >> conf.setMapOutputValueClass(__MapWritable.class); >> >> >> to MapWritable while LinkedMapWritable is returned. The latest >> versions automatically detect this and use the proper >> type so I recommend upgrading. >> If that's not an option, use LinkedMapWritable. >> >> Cheers, >> >> P.S. Please don't post code and stracktraces on the mailing list >> since it highly reduces the readability of your >> email. Instead use gist or any other service >> to post the code as indicated in the docs [1]. Thanks >> >> [1] >> http://www.elasticsearch.org/__guide/en/elasticsearch/ >> hadoop/__2.1.Beta/troubleshooting.html#___where_ >> do_i_post_my___information >> <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/ >> 2.1.Beta/troubleshooting.html#_where_do_i_post_my_information> >> >> >> >> >> On 9/9/14 11:59 AM, gaurav redkar wrote: >> >> Hi, I was following the example given on official elasticsearch >> documentation to read data from elasticsearch using >> hadoop but i am getting the following error. >> >> java.lang.Exception: java.io.IOException: Type mismatch in value >> from map: expected org.apache.hadoop.io >> <http://org.apache.hadoop.io>.__MapWritable, >> received org.elasticsearch.hadoop.mr <http://org.elasticsearch. >> hadoop.mr>.__LinkedMapWritable >> >> at org.apache.hadoop.mapred.__LocalJobRunner$Job.runTasks(__ >> LocalJobRunner.java:462) >> >> at org.apache.hadoop.mapred.__LocalJobRunner$Job.run(__ >> LocalJobRunner.java:522) >> >> Caused by: java.io.IOException: Type mismatch in value from map: >> expected org.apache.hadoop.io >> <http://org.apache.hadoop.io>.__MapWritable, received >> org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr >> >.__LinkedMapWritable >> >> at org.apache.hadoop.mapred.__MapTask$MapOutputBuffer.__ >> collect(MapTask.java:1054) >> >> at org.apache.hadoop.mapred.__MapTask$OldOutputCollector.__ >> collect(MapTask.java:587) >> >> at es2.mapper1.map(mapper1.java:__29) >> >> at org.apache.hadoop.mapred.__MapRunner.run(MapRunner.java:__54) >> >> at org.apache.hadoop.mapred.__MapTask.runOldMapper(MapTask._ >> _java:430) >> >> at org.apache.hadoop.mapred.__MapTask.run(MapTask.java:342) >> >> at org.apache.hadoop.mapred.__LocalJobRunner$Job$__ >> MapTaskRunnable.run(__LocalJobRunner.java:243) >> >> at java.util.concurrent.__Executors$RunnableAdapter.__ >> call(Executors.java:471) >> >> at java.util.concurrent.__FutureTask$Sync.innerRun(__ >> FutureTask.java:334) >> >> at java.util.concurrent.__FutureTask.run(FutureTask.__java:166) >> >> at java.util.concurrent.__ThreadPoolExecutor.runWorker(_ >> _ThreadPoolExecutor.java:1145) >> >> at java.util.concurrent.__ThreadPoolExecutor$Worker.run( >> __ThreadPoolExecutor.java:615) >> >> at java.lang.Thread.run(Thread.__java:724) >> >> 14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 >> failed with state FAILED due to: NA >> >> 14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0 >> >> Exception in thread "main" java.io.IOException: Job failed! >> >> at org.apache.hadoop.mapred.__JobClient.runJob(JobClient.__ >> java:836) >> >> at es2.Es2.main(Es2.java:48) >> >> at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native >> Method) >> >> at sun.reflect.__NativeMethodAccessorImpl.__invoke(__ >> NativeMethodAccessorImpl.java:__57) >> >> at sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__ >> DelegatingMethodAccessorImpl.__java:43) >> >> at java.lang.reflect.Method.__invoke(Method.java:606) >> >> at org.apache.hadoop.util.RunJar.__main(RunJar.java:212) >> >> 14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit >> [node=[QMYbfSRMTsyXVVgfFweMkA/__Conceptnet|192.168.1.202:9200 < >> http://192.168.1.202:9200>]__,shard=0] > map >> >> >> The code for my main runner class is as follows:- >> >> >> public class Es2 { >> >> static private final Path TMP_DIR = new >> Path(Es2.class.getSimpleName() >> >> + "_TMP_1"); >> >> >> /** >> >> * @param args the command line arguments >> >> */ >> >> public static void main(String[] args) throws IOException{ >> >> //#String node="192.168.1.202"; >> >> JobConf conf = new JobConf(); >> >> conf.set("es.resource", "data/content"); >> >> conf.set("es.nodes", "192.168.1.202:9200 < >> http://192.168.1.202:9200>"); >> >> conf.set("es.query", "?q=s_start:java"); >> >> conf.setInputFormat(__EsInputFormat.class); >> >> conf.setMapOutputKeyClass(__Text.class); >> >> conf.setMapOutputValueClass(__MapWritable.class); >> >> conf.setOutputKeyClass(Text.__class); >> >> conf.setOutputValueClass(__MapWritable.class); >> >> conf.setOutputFormat(__TextOutputFormat.class); >> >> conf.setMapperClass(mapper1.__class); >> >> final Path outDir = new Path(TMP_DIR, "out"); >> >> FileOutputFormat.__setOutputPath(conf, outDir); >> >> JobClient.runJob(conf); >> >> } >> >> } >> >> The code for my mapper class is as follows:- >> >> public class mapper1 extends MapReduceBase implements >> >> Mapper{ >> >> @Override >> >> public void map(Object key, Object value, OutputCollector >> output, >> >> Reporter reporter) throws IOException { >> >> Text docId = (Text) key; >> >> MapWritable doc = (MapWritable) value; >> >> output.collect(docId,doc); >> >> } >> >> } >> >> Kindly guide me on this issue. >> >> Thanks >> >> -- >> You received this message because you are subscribed to the >> Google Groups "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, >> send an email to >> elasticsearch+unsubscribe@__googlegroups.com <mailto: >> elasticsearch%2bunsubscr...@googlegroups.com> >> <mailto:elasticsearch+__unsubscr...@googlegroups.com <mailto: >> elasticsearch%2bunsubscr...@googlegroups.com>>. >> To view this discussion on the web visit >> https://groups.google.com/d/__msgid/elasticsearch/3b110c43-_ >> _faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/3b110c43- >> faa8-4a08-acfa-0fac5fd89585%40googlegroups.com> >> <https://groups.google.com/d/__msgid/elasticsearch/3b110c43- >> __faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_ >> medium=__email&utm_source=footer >> <https://groups.google.com/d/msgid/elasticsearch/3b110c43- >> faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium= >> email&utm_source=footer>>. >> For more options, visit https://groups.google.com/d/__optout < >> https://groups.google.com/d/optout>. >> >> >> -- >> Costin >> >> -- >> You received this message because you are subscribed to a topic in >> the Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/__ >> topic/elasticsearch/HS1A___psM7fw/unsubscribe >> <https://groups.google.com/d/topic/elasticsearch/HS1A_ >> psM7fw/unsubscribe>. >> To unsubscribe from this group and all its topics, send an email to >> elasticsearch+unsubscribe@__googlegroups.com >> <mailto:elasticsearch%2bunsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/__msgid/elasticsearch/540ED720._ >> _5040501%40gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/540ED720. >> 5040501%40gmail.com>. >> >> For more options, visit https://groups.google.com/d/__optout < >> https://groups.google.com/d/optout>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to >> elasticsearch+unsubscr...@googlegroups.com <mailto:elasticsearch+ >> unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ% >> 2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ% >> 2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA% >> 40mail.gmail.com?utm_medium=email&utm_source=footer>. >> For more options, visit https://groups.google.com/d/optout. >> > -- > Costin > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/elasticsearch/HS1A_psM7fw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/elasticsearch/540EFF05.6040508%40gmail.com. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
