Hi,
Nothing jumps out from your configuration. The error indicates that the values passed to es-hadoop cannot be processed
for some reason. Which is more surpsing considering your Mapper writes some constants to the output.
I've pushed some improvements to the 2.x branch which explain better conditions in which the error appears - you can
either build the jar yourself [1] and test it out or wait for the nightly build to publish the artifact [2].
Cheers,
[1] https://github.com/elasticsearch/elasticsearch-hadoop/tree/2.x
[2] http://build.elasticsearch.com/view/Hadoop/job/es-hadoop-nightly-2x/
On 7/14/14 1:19 PM, Aurélien wrote:
Hi,
I can't sort that ! I'm using hadoop CDH3u6, and trying to get ES index my
data. I tried with raw json and MapWritable,
I always get the same kind of errors :
|
java.lang.Exception:org.elasticsearch.hadoop.EsHadoopIllegalArgumentException:[org.elasticsearch.hadoop.serialization.field.MapWritableFieldExtractor@35b5f7bd]cannot
extract value fromobject[org.apache.hadoop.io.MapWritable@11c757a1]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:349)
Causedby:org.elasticsearch.hadoop.EsHadoopIllegalArgumentException:[org.elasticsearch.hadoop.serialization.field.MapWritableFieldExtractor@35b5f7bd]cannot
extract value fromobject[org.apache.hadoop.io.MapWritable@11c757a1]
at
org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk$FieldWriter.write(TemplatedBulk.java:49)
at
org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.writeTemplate(TemplatedBulk.java:101)
at
org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:77)
at
org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:130)
at
org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:161)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:531)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at my.jobs.index.IndexMapper.map(IndexMapper.java:27)
at my.jobs.index.IndexMapper.map(IndexMapper.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:648)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:218)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
|
Seems to me that all is right, here the configuration of the index mapper :
|
Jobjob =newJob(getConf(),"Indexing into Elastic search.");
job.setJarByClass(getClass());
DomainRankDriver.loadLibrariesToDistributedCache(job);
Pathinput =newPath(args[0]);
FileInputFormat.addInputPath(job,input);
FileOutputFormat.setOutputPath(job,newPath(args[1]));
// Used by ES-hadoop to take Text as Json
job.setOutputFormatClass(EsOutputFormat.class);
// job.setMapOutputValueClass(Text.class);
job.setMapOutputValueClass(MapWritable.class);
job.setMapperClass(IndexMapper.class);
job.setNumReduceTasks(0);
|
And my simple mapper :
|
@Override
publicvoidmap(LongWritablekey,Textvalue,Contextcontext)
throwsIOException,InterruptedException{
MapWritablemap =newMapWritable();
map.put(newText("test"),newText("value"));
context.write(newLongWritable(),map);
}
|
Any clue to search for more ? I'm stuck.
Thanks,
Aurelien
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7f6545ab-d6d9-4fdf-8923-0b60e0ea5297%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/7f6545ab-d6d9-4fdf-8923-0b60e0ea5297%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53C3FBCE.9040302%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
