HI. When I use elasticsearch-hadoop, I encounter this error: Could not write all entries [1576/10485504 <https://github.com/elasticsearch/elasticsearch-hadoop/commit/maybe%20ES%20was%20overloaded?>]. Bailing out... My task execution schedule is as follows: 14/11/05 16:06:06 INFO mapred.JobClient: map 100% reduce 69% 14/11/05 16:06:12 INFO mapred.JobClient: map 100% reduce 66% 14/11/05 16:06:14 INFO mapred.JobClient: map 100% reduce 65% 14/11/05 16:06:16 INFO mapred.JobClient: map 100% reduce 66% 14/11/05 16:06:20 INFO mapred.JobClient: map 100% reduce 67% 14/11/05 16:06:49 INFO mapred.JobClient: map 100% reduce 68% 14/11/05 16:06:57 INFO mapred.JobClient: map 100% reduce 69% 14/11/05 16:07:09 INFO mapred.JobClient: map 100% reduce 70% 14/11/05 16:08:21 INFO mapred.JobClient: map 100% reduce 69% 14/11/05 16:08:25 INFO mapred.JobClient: map 100% reduce 68% 14/11/05 16:08:33 INFO mapred.JobClient: map 100% reduce 69% 14/11/05 16:09:08 INFO mapred.JobClient: map 100% reduce 70% 14/11/05 16:10:02 INFO mapred.JobClient: map 100% reduce 69% 14/11/05 16:10:04 INFO mapred.JobClient: map 100% reduce 68% 14/11/05 16:10:06 INFO mapred.JobClient: map 100% reduce 65% 14/11/05 16:10:09 INFO mapred.JobClient: map 100% reduce 62% 14/11/05 16:10:11 INFO mapred.JobClient: map 100% reduce 61% 14/11/05 16:10:13 INFO mapred.JobClient: map 100% reduce 62% 14/11/05 16:10:15 INFO mapred.JobClient: map 100% reduce 63% 14/11/05 16:10:17 INFO mapred.JobClient: map 100% reduce 64% 14/11/05 16:10:19 INFO mapred.JobClient: map 100% reduce 65% 14/11/05 16:10:25 INFO mapred.JobClient: map 100% reduce 64% 14/11/05 16:10:35 INFO mapred.JobClient: map 100% reduce 65% 14/11/05 16:10:45 INFO mapred.JobClient: map 100% reduce 66% 14/11/05 16:10:49 INFO mapred.JobClient: map 100% reduce 67% 14/11/05 16:10:51 INFO mapred.JobClient: map 100% reduce 68% 14/11/05 16:10:59 INFO mapred.JobClient: map 100% reduce 69%
My parameters are set as follows: conf.set("es.batch.size.bytes", "1048576"); conf.set("es.batch.size.entries", "1000"); conf.set("mapred.reduce.tasks", "200"); conf.setBoolean("mapred.map.tasks.speculative.execution", false); conf.setBoolean("mapred.reduce.tasks.speculative.execution", false); The jar I used is: elasticsearch-hadoop-2.1.0.Beta2.jar My hardware configuration is as follows: 100GB RAM , 4TB hard disk, 10 nodes, 1 balancer, 1 master Can anyone help me find out the reason why this error appear? Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/80420872-9b3e-42ac-afe7-386bc37f6664%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.