Re: Elasticsearch Hadoop WRITE operation not using Reducer

Hi, es-hadoop does not use either mapper or reducers; the Map/Reduce integration relies on the Input/OutputFormat which can be invoked either from a Mapper or from a Reducer. Your Reducer might not be invoked for a variety of reasons; typically the map and reduce phases have different output

Re: Elasticsearch Hadoop WRITE operation not using Reducer

Hi Costin, Thanks for the response. You are right. Map/Reduce integration relies on the Input/OutputFormat. Even after removing EsOutputFormat my custom reducer is not invoked. Should be some issue with hadoop configuration. Thanks, Sarath -- You received this message because you are

Re: Elasticsearch Hadoop WRITE operation not using Reducer

Hi, Doesn't look like you've set the number of reduce tasks in your job config. i.e. 'job.setNumReduceTasks(10);' -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email

Re: Elasticsearch Hadoop WRITE operation not using Reducer

Hi Telax, Even though i don't set number of reduce tasks, Hadoop takes care of starting reducers if needed. In my case 1 reducer is running. Issue here custom reducer defined as part of Job configuration is not invoked by Hadoop Thanks, Sarath -- You received this message because you are

Elasticsearch Hadoop WRITE operation not using Reducer

Hi All, Will Elasticsearch Hadoop WRITE operation doesn't use our custom reducer? I tried with following code and observed that our customer reducer is not invoked. job.setOutputFormatClass(EsOutputFormat.class); job.setMapOutputKeyClass(NullWritable.class);