On my testing (5 virtual nodes on a single box), I've been observing pretty much the same. I've tried pointing logstash to different nodes in the cluster and getting approx. the same performance. My main objective was to determine if by inserting the data into the cluster on different nodes whether the data would be distributed differently. Surprisingly, I've fond so far there is little difference at the cluster level (distribution of shards and data across nodes) although the details (actual shard locations on which nodes) would be different. In other words, the overall "uneven-ness" of data was surprisingly almost identical with each try although <where> the uneven-ness was typically different. And, I also found that inserting data into more than one node at once didn't seem to make a diff. One possibility is that your two EC2 instances might be running on the same hardware which could explain our similar results? I remember sitting a presentation years ago about this and how that person "encouraged" EC2 to deploy nodes on different hardware. I don't remember the details, I only remember that person determined he couldn't make it a certainty but could tilt the odds so much in his favor (6:1?) that his VMs would usually be on different hardware. Tony
On Tuesday, February 18, 2014 12:45:32 PM UTC-8, Bastien Chong wrote: > Even with 2 logstashs instance writing to the same cluster, it's not > faster. > > On Friday, February 14, 2014 2:45:47 PM UTC-5, Binh Ly wrote: >> >> It's hard to diagnose things offline, but is it possible for you to run >> another logstash somewhere else (like maybe on the second box) and both of >> them in parallel and see what your combined ES throughput is. So they would >> be both writing to the same single ES cluster. >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e85d8071-3424-4c0c-a35f-a87a52bad20a%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
