Hi Stack, I would like to summarize all results that we have in this email.
At start of this discussion I had following config: http://pastebin.com/m6c7358e6 Which allowed me to have 10-15 Mb per second of throughput (with WAL on) for a serial client which inserts 100K of 25Kb records and exits. Now I've removed block.multiplier option (default is used now) and set region size to 1GB. In both case I had hbase.hregion.memstore.flush.size = 67108864 (i.e. by default). Changing of regionsize makes sense as you said and for 25Kb test and throughput with WAL on is about 25-27 Mb per second now when I have about 200 regions per server. As for your questions: Understood, but is it because the regionservers+datanodes load is going up > if you add more clients? Or are the timeouts because of something else? > (Clients are running on the machine that has NN/Master/ZK? If so, could > the clients be sucking resources from these servers in a way that slows > down whole cluster? Is load on machine high when clients are running? > That > kinda thing). > Yes, clients are running on namenode machine, and that's may be a problem when running several of them in parallel. I think I need to move running out of cluster to see, but I don't see any reason to do that on performance graphs. Also I'm running tests on LZO-enabled tables and will provide my results soon. Dmitriy.
