There are many reasons that may cause this, just to name a few - benchmarking tool setup ( do they show correct numbers?) - network bandwidth limits - cluster setup (e.g. complex mapping, high latency between nodes) - pattern of the data input - method of data input (bulk vs. index, HTTP vs. Java API) - concurrency of server/client connections - thread pooling resource limits - SSD setup (e.g. RAID 0) - resource limits due to hardware setup (host OS often put hard restrictions on VM guest OS resources) - etc.
First I recommend testing a single VM for basic performance, especially SSD raw write throughput. Then, testing document processing, without and with a single node ES, for measuring source processing overhead on client side, to eliminate issues cause by clients. Then, indexing to single node, and repeating this by doubling nodes each step, for measuring the performance gain. Do you see suspicious measurements even on a single node? Also noteworthy is that VMs are generally quite slowish, compared to bare metal. Performance depends on VM setup of the host OS. But do not expect miracles. There are many ES monitoring tools/plugins that can assist you in getting metrics. Jörg On Mon, Jun 9, 2014 at 5:40 PM, pranav amin <parulpate...@gmail.com> wrote: > Hi all, > > While doing some prototyping in ES using SSD's we got some good Write TPS. > But the Write TPS saturated after adding some more nodes! > > > Here are the details i used for prototyping - > > Requirement: To read data as soon as possible since the read is followed > by write. > Version of ES:1.0.0 > Document Size:144 KB > Use of SSD for Storage: Yes > Benchmarking Tool: Soap UI or Jmeter > VM: Ubuntu, 64 Bit OS > Total Nodes: 12 > Total Shards: 60 > Threads: 200 > Replica: 2 > Index Shards: 20 > Total Index:1 > Hardware configuration: 4 CPU, 6 GB RAM, 3 GB Heap > > Using the above setup we got Write TPS ~= 500. > > We wanted to know by adding more node if we can increase our Write TPS. > But we couldn't. > * By adding 3 more nodes (i..e Total Nodes = 15) the TPS just increase by > 10 i.e. ~= 510. > * Adding more Hardware like CPU, RAM and increasing Heap didn't help as > well [8 CPU, 12 GB RAM, 5 GB Heap]. > > Can someone help out or point ideas what will be wrong? Conceptually ES > should scale in terms of Write & Read TPS by adding more nodes. However we > aren't able to get that. > > Much appreciated if someone can point us in the right direction. Let me > know if more information is needed. > > Thanks > Pranav. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1e34d7c7-d3da-40c7-8fca-16281494065b%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/1e34d7c7-d3da-40c7-8fca-16281494065b%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHnCgkUE0ZG9eprGTPzipP_H2wd7yCRX_go0ukZfW_o6w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
