Hi Costin, Thanks for the heads up regarding gist. I will try to follow the guidelines in the future. As for my program, I am using Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed the output value class to LinkedMapWritable. but now I am getting the following error.
https://gist.github.com/gauravub/7d55bc6b10cb63935eb8 Any idea why is this happening ? I even tried using the v2.0.0 of es-hadoop but am still getting the same error. On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.l...@gmail.com> wrote: > Hi, > > What version of es-hadoop are you using? The problem stems from the > difference in the types mentioned on your Mapper, namely the output value > class: > > conf.setMapOutputValueClass(MapWritable.class); >> > > to MapWritable while LinkedMapWritable is returned. The latest versions > automatically detect this and use the proper type so I recommend upgrading. > If that's not an option, use LinkedMapWritable. > > Cheers, > > P.S. Please don't post code and stracktraces on the mailing list since it > highly reduces the readability of your email. Instead use gist or any other > service > to post the code as indicated in the docs [1]. Thanks > > [1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/ > 2.1.Beta/troubleshooting.html#_where_do_i_post_my_information > > > > > On 9/9/14 11:59 AM, gaurav redkar wrote: > >> Hi, I was following the example given on official elasticsearch >> documentation to read data from elasticsearch using >> hadoop but i am getting the following error. >> >> java.lang.Exception: java.io.IOException: Type mismatch in value from >> map: expected org.apache.hadoop.io.MapWritable, >> received org.elasticsearch.hadoop.mr.LinkedMapWritable >> >> at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks( >> LocalJobRunner.java:462) >> >> at org.apache.hadoop.mapred.LocalJobRunner$Job.run( >> LocalJobRunner.java:522) >> >> Caused by: java.io.IOException: Type mismatch in value from map: expected >> org.apache.hadoop.io.MapWritable, received >> org.elasticsearch.hadoop.mr.LinkedMapWritable >> >> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer. >> collect(MapTask.java:1054) >> >> at org.apache.hadoop.mapred.MapTask$OldOutputCollector. >> collect(MapTask.java:587) >> >> at es2.mapper1.map(mapper1.java:29) >> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) >> >> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) >> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) >> >> at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run( >> LocalJobRunner.java:243) >> >> at java.util.concurrent.Executors$RunnableAdapter. >> call(Executors.java:471) >> >> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >> >> at java.util.concurrent.ThreadPoolExecutor.runWorker( >> ThreadPoolExecutor.java:1145) >> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >> ThreadPoolExecutor.java:615) >> >> at java.lang.Thread.run(Thread.java:724) >> >> 14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed >> with state FAILED due to: NA >> >> 14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0 >> >> Exception in thread "main" java.io.IOException: Job failed! >> >> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836) >> >> at es2.Es2.main(Es2.java:48) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:57) >> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:606) >> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >> >> 14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit >> [node=[QMYbfSRMTsyXVVgfFweMkA/Conceptnet|192.168.1.202:9200],shard=0] > >> map >> >> >> The code for my main runner class is as follows:- >> >> >> public class Es2 { >> >> static private final Path TMP_DIR = new >> Path(Es2.class.getSimpleName() >> >> + "_TMP_1"); >> >> >> /** >> >> * @param args the command line arguments >> >> */ >> >> public static void main(String[] args) throws IOException{ >> >> //#String node="192.168.1.202"; >> >> JobConf conf = new JobConf(); >> >> conf.set("es.resource", "data/content"); >> >> conf.set("es.nodes", "192.168.1.202:9200"); >> >> conf.set("es.query", "?q=s_start:java"); >> >> conf.setInputFormat(EsInputFormat.class); >> >> conf.setMapOutputKeyClass(Text.class); >> >> conf.setMapOutputValueClass(MapWritable.class); >> >> conf.setOutputKeyClass(Text.class); >> >> conf.setOutputValueClass(MapWritable.class); >> >> conf.setOutputFormat(TextOutputFormat.class); >> >> conf.setMapperClass(mapper1.class); >> >> final Path outDir = new Path(TMP_DIR, "out"); >> >> FileOutputFormat.setOutputPath(conf, outDir); >> >> JobClient.runJob(conf); >> >> } >> >> } >> >> The code for my mapper class is as follows:- >> >> public class mapper1 extends MapReduceBase implements >> >> Mapper{ >> >> @Override >> >> public void map(Object key, Object value, OutputCollector output, >> >> Reporter reporter) throws IOException { >> >> Text docId = (Text) key; >> >> MapWritable doc = (MapWritable) value; >> >> output.collect(docId,doc); >> >> } >> >> } >> >> Kindly guide me on this issue. >> >> Thanks >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to >> elasticsearch+unsubscr...@googlegroups.com <mailto:elasticsearch+ >> unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/3b110c43- >> faa8-4a08-acfa-0fac5fd89585%40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/3b110c43- >> faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium= >> email&utm_source=footer>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > Costin > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/elasticsearch/HS1A_psM7fw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/elasticsearch/540ED720.5040501%40gmail.com. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
