I am using elasticsearch-hadoop plugin (
https://github.com/elasticsearch/elasticsearch-hadoop) and trying to index
some documents. Iam using elasticsearch version 0.90.2 and Hadoop
Hortonworks 2.2.0. The search functionality works fine, but while
indexing application hangs
https://github.com/elasticsearch/elasticsearch-hadoop
The json file location is passed as a command line arguemnt and below is
the indexing code snippet
Configuration conf = new Configuration();
conf.setBoolean("mapred.map.tasks.speculative.execution", false);
conf.setBoolean("mapred.reduce.tasks.speculative.execution", false);
conf.setInt("mapred.min.split.size",40);
conf.set("es.resource", "test/test");
conf.set("es.nodes", "localhost");
conf.set("es.port", "9200");
conf.set("es.input.json", "yes");
conf.set("es.nodes", "localhost");
conf.set("es.port", "9200");
Job job = Job.getInstance(conf);
job.setMapperClass(Mapper.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(EsOutputFormat.class);
job.setMapOutputKeyClass(LongWritable.class);
job.setMapOutputValueClass(Text.class);
Path jarPath = new Path(args[0]);
FileSystem fs = FileSystem.get(conf);
Path dst = new Path(fs.getHomeDirectory(), jarPath.getName());
fs.copyFromLocalFile(false, true, jarPath, dst);
FileInputFormat.setInputPaths(job, dst);
job.waitForCompletion(true);
Am I missing anything, pls help
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/15b0d1e9-e258-4fd4-b7bc-d7b81596fc81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
