Most likely you have a classpath conflict caused by multiple versions of es-hadoop. Can you double check you only have
one version (2.1.0.Beta1) available?
Based on the error, I'm guessing you have some 1.3 Mx or the RC somewhere in
there...
On 9/9/14 4:06 PM, gaurav redkar wrote:
Hi Costin,
Thanks for the heads up regarding gist. I will try to follow the guidelines in
the future. As for my program, I am using
Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed the
output value class to LinkedMapWritable. but
now I am getting the following error.
https://gist.github.com/gauravub/7d55bc6b10cb63935eb8
Any idea why is this happening ? I even tried using the v2.0.0 of es-hadoop but
am still getting the same error.
On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.l...@gmail.com
<mailto:costin.l...@gmail.com>> wrote:
Hi,
What version of es-hadoop are you using? The problem stems from the
difference in the types mentioned on your
Mapper, namely the output value class:
conf.setMapOutputValueClass(__MapWritable.class);
to MapWritable while LinkedMapWritable is returned. The latest versions
automatically detect this and use the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.
Cheers,
P.S. Please don't post code and stracktraces on the mailing list since it
highly reduces the readability of your
email. Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks
[1]
http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>
On 9/9/14 11:59 AM, gaurav redkar wrote:
Hi, I was following the example given on official elasticsearch
documentation to read data from elasticsearch using
hadoop but i am getting the following error.
java.lang.Exception: java.io.IOException: Type mismatch in value from
map: expected org.apache.hadoop.io
<http://org.apache.hadoop.io>.__MapWritable,
received org.elasticsearch.hadoop.mr
<http://org.elasticsearch.hadoop.mr>.__LinkedMapWritable
at
org.apache.hadoop.mapred.__LocalJobRunner$Job.runTasks(__LocalJobRunner.java:462)
at
org.apache.hadoop.mapred.__LocalJobRunner$Job.run(__LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in value from map:
expected org.apache.hadoop.io
<http://org.apache.hadoop.io>.__MapWritable, received
org.elasticsearch.hadoop.mr
<http://org.elasticsearch.hadoop.mr>.__LinkedMapWritable
at
org.apache.hadoop.mapred.__MapTask$MapOutputBuffer.__collect(MapTask.java:1054)
at
org.apache.hadoop.mapred.__MapTask$OldOutputCollector.__collect(MapTask.java:587)
at es2.mapper1.map(mapper1.java:__29)
at org.apache.hadoop.mapred.__MapRunner.run(MapRunner.java:__54)
at org.apache.hadoop.mapred.__MapTask.runOldMapper(MapTask.__java:430)
at org.apache.hadoop.mapred.__MapTask.run(MapTask.java:342)
at
org.apache.hadoop.mapred.__LocalJobRunner$Job$__MapTaskRunnable.run(__LocalJobRunner.java:243)
at
java.util.concurrent.__Executors$RunnableAdapter.__call(Executors.java:471)
at
java.util.concurrent.__FutureTask$Sync.innerRun(__FutureTask.java:334)
at java.util.concurrent.__FutureTask.run(FutureTask.__java:166)
at
java.util.concurrent.__ThreadPoolExecutor.runWorker(__ThreadPoolExecutor.java:1145)
at
java.util.concurrent.__ThreadPoolExecutor$Worker.run(__ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.__java:724)
14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001
failed with state FAILED due to: NA
14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.__JobClient.runJob(JobClient.__java:836)
at es2.Es2.main(Es2.java:48)
at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)
at
sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__57)
at
sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
at java.lang.reflect.Method.__invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.__main(RunJar.java:212)
14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/__Conceptnet|192.168.1.202:9200
<http://192.168.1.202:9200>]__,shard=0] > map
The code for my main runner class is as follows:-
public class Es2 {
static private final Path TMP_DIR = new
Path(Es2.class.getSimpleName()
+ "_TMP_1");
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws IOException{
//#String node="192.168.1.202";
JobConf conf = new JobConf();
conf.set("es.resource", "data/content");
conf.set("es.nodes", "192.168.1.202:9200
<http://192.168.1.202:9200>");
conf.set("es.query", "?q=s_start:java");
conf.setInputFormat(__EsInputFormat.class);
conf.setMapOutputKeyClass(__Text.class);
conf.setMapOutputValueClass(__MapWritable.class);
conf.setOutputKeyClass(Text.__class);
conf.setOutputValueClass(__MapWritable.class);
conf.setOutputFormat(__TextOutputFormat.class);
conf.setMapperClass(mapper1.__class);
final Path outDir = new Path(TMP_DIR, "out");
FileOutputFormat.__setOutputPath(conf, outDir);
JobClient.runJob(conf);
}
}
The code for my mapper class is as follows:-
public class mapper1 extends MapReduceBase implements
Mapper{
@Override
public void map(Object key, Object value, OutputCollector output,
Reporter reporter) throws IOException {
Text docId = (Text) key;
MapWritable doc = (MapWritable) value;
output.collect(docId,doc);
}
}
Kindly guide me on this issue.
Thanks
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2bunsubscr...@googlegroups.com>
<mailto:elasticsearch+__unsubscr...@googlegroups.com
<mailto:elasticsearch%2bunsubscr...@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/__optout
<https://groups.google.com/d/optout>.
--
Costin
--
You received this message because you are subscribed to a topic in the Google Groups
"elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2bunsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540ED720.5040501%40gmail.com>.
For more options, visit https://groups.google.com/d/__optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
elasticsearch+unsubscr...@googlegroups.com
<mailto:elasticsearch+unsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/540EFF05.6040508%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
