really? maybe that's the reason for my bad performance results too. However, how could logging makes so huge effect to the performance. I mean, usually it only leads to about 20% performance loss when you adding more log in the program. This phenomenon is beyond my understanding. Could any one give me some explanation?
thanks 2011/3/21 Flavio Junqueira <f...@yahoo-inc.com> > Ah, right! Good catch. > > -Flavio > > On Mar 21, 2011, at 6:35 AM, lei he wrote: > > Sorry for the report, I made a mistake. When I turn off the debug level log > in zookeeper servers, the performance is like you described. > Thanks for you guys. > 在 2011年3月10日 上午10:31，lei he <hl0...@gmail.com>写道： > > thanks for reply. I use scp to check my ethernet again, the transmission > > rate is about 60MB/s. I send about 100000 requests in each process(60 > > processes in 2 clients), and it costs about 550 seconds to finish all the > > requests. And I am sure that all the processes had been spread around all > > the servers. > > > Thanks > > Lei, He > > > 在 2011年3月9日 上午9:22，Benjamin Reed <br...@apache.org>写道： > > > are you using 1 gbs ethernet or 100 mbs? those numbers seem to correspond > > to > > 100 mbs. you may want to do a simple bandwidth test just to make sure. > > > how long are you running the test for? > > > also, you might check that the clients are being spread around the zk > > servers. using the 4 letter works or phunts zktop. > > > ben > > > 2011/3/9 lei he <hl0...@gmail.com> > > > Hi Flavio and Qian, I did some tests in my environment, too. And it > > seems > > it is not as good as it supposed to be. I used async > > interface(zoo_aget), > > send about 10000 requests to3 servers at the same time. But it turn out > > to > > be on the same level with Qian's test. Is there anything I should pay > > attentio to? > > > thanks a lot > > He, Lei > > > 2011/3/9 Flavio Junqueira <f...@yahoo-inc.com> > > > How many outstanding requests do you have at a time, Qian? Only one? If > > so, you should instead have multiple outstanding. That is, you don't > > wait > > for the callback to issue a new request. > > > -Flavio > > > On Mar 9, 2011, at 3:44 AM, Qian Ye wrote: > > > Hi Flavio, asynchronous calls doesn't perform better, here is some > > results > > we've got, > > > § 1 client server，1 process per client server，connect 1 zookeeper > > server，all reads：cpu：14%~15%，qps：3833，latency：0.000261 > > § 1 client server，1 process per client server，connect all 3 zookeeper > > server，all reads：cpu：14%~15%，qps：3832，latency：0.000261 > > § 1 client server，10 process per client server，connect all 3 zookeeper > > server，all reads，cpu：13%~20%，qps：14000->12000，latency：0.000469 > > *§ 1 client server，30 process per client server，connect all 3 zookeeper > > server，all reads，cpu：15%~20%，qps：14000->10000，,latency： > > § 2 client server，30 process per client server，connect all 3 zookeeper > > server，all reads，cpu：15%~20%，qps：about 11000，latency：* > > > It seems that the asynchronous calls perform even worse than the > > synchronous calls. > > > > On Wed, Mar 9, 2011 at 12:29 AM, Flavio Junqueira <f...@yahoo-inc.com > > wrote: > > > Hi Qian, If I understand your description correctly, you are using > > synchronous calls. To get high throughput values, you need multiple > > outstanding requests, so you will need to use asynchronous calls. > > > -Flavio > > > On Mar 8, 2011, at 5:16 PM, Qian Ye wrote: > > > P.S. 1 we use zookeeper 3.3.2 > > P.S. 2 all our testing process get data from the same znode. The size > > of > > data on the znode is less than 1K. > > > On Wed, Mar 9, 2011 at 12:08 AM, Qian Ye <yeqian....@gmail.com> > > wrote: > > > Hi all: > > > > These days my friend and I did some performance tests on zookeeper. We > > > found the performance of zookeeper is not as good as it is described > > in > > the > > > Zookeeper Overview ( > > > http://hadoop.apache.org/zookeeper/docs/r3.3.2/zookeeperOver.html) . > > In > > > the Zookeeper Overview, the "ZooKeeper Throughput as the Read-Write > > Ratio > > > Varies" shows that in a ensemble of 3 Zookeeper server, the throughput > > can > > > reach about 80000, if the requests are all reads. However, we cannot > > get > > > results like that in our performance test with the synchronized > > interface, > > > zkpython. > > > > Here is some of our test results: > > > (3 zookeeper ensemble, 8 core CPU, 2.4GHZ, 16 RAM, Linux 2.6.9) > > > > § 1 client server，1 process per client server，connect 1 zookeeper > > > server，all reads：cpu：8%~9%，qps：2208，latency：0.000453s > > > § 1 client server，1 process per client server，connect all 3 zookeeper > > > server，all reads：cpu：8%~9%，qps：2376.241573 ，latency:0.000421s > > > § 1 client server，1 process per client server，connect all 3 zookeeper > > > server，all reads，cpu：10%~20%，qps：15600，latency：0.000764s > > > *§ 1 client server，30 process per client server，connect all 3 > > zookeeper > > > server，all reads，cpu：10%~20%，qps：15200，latency：* > > > *§ 2 client server，30 process **per client server**，connect all 3 > > > zookeeper server，all reads，cpu：10%~20%，qps：15800，latency：0.003487* > > > > qps means "query per second", that is throughput. The result shows > > that > > > when adding more client server, the utilization rate of CPU don't > > increase, > > > and the throughput don't increase much. It seems that the throughput > > won't > > > reach 80000, even if we add 28 more client servers to reach the number > > you > > > mentioned in the Zookeeper Overview. > > > > Maybe I've done the tests wrong. Is there any particular thing I > > should > > pay > > > attention to in this case? We set the max java heap size to 12GB in > > our > > > test. > > > > *Could you tell me the details about how you do the performance test, > > from > > > which you get the results showed in the Zookeeper Overview?* > > > > -- > > > With Regards! > > > > Ye, Qian > > > > > > > -- > > With Regards! > > > Ye, Qian > > > > *flavio* > > *junqueira* > > > research scientist > > > f...@yahoo-inc.com > > direct +34 93-183-8828 > > > avinguda diagonal 177, 8th floor, barcelona, 08018, es > > phone (408) 349 3300 fax (408) 349 3301 > > > > > > > -- > > With Regards! > > > Ye, Qian > > > > *flavio* > > *junqueira* > > > research scientist > > > f...@yahoo-inc.com > > direct +34 93-183-8828 > > > avinguda diagonal 177, 8th floor, barcelona, 08018, es > > phone (408) 349 3300 fax (408) 349 3301 > > > > > > > > > > > -- > Best Regards > Lei, He > > > *flavio* > *junqueira* > > research scientist > > f...@yahoo-inc.com > direct +34 93-183-8828 > > avinguda diagonal 177, 8th floor, barcelona, 08018, es > phone (408) 349 3300 fax (408) 349 3301 > > > -- With Regards! Ye, Qian
