Hi Flavio and Qian, I did some tests in my environment, too. And it seems it is not as good as it supposed to be. I used async interface(zoo_aget), send about 10000 requests to3 servers at the same time. But it turn out to be on the same level with Qian's test. Is there anything I should pay attentio to?
thanks a lot He, Lei 2011/3/9 Flavio Junqueira <f...@yahoo-inc.com> > How many outstanding requests do you have at a time, Qian? Only one? If so, > you should instead have multiple outstanding. That is, you don't wait for > the callback to issue a new request. > > -Flavio > > On Mar 9, 2011, at 3:44 AM, Qian Ye wrote: > > Hi Flavio, asynchronous calls doesn't perform better, here is some results > we've got, > > § 1 client server，1 process per client server，connect 1 zookeeper > server，all reads：cpu：14%~15%，qps：3833，latency：0.000261 > § 1 client server，1 process per client server，connect all 3 zookeeper > server，all reads：cpu：14%~15%，qps：3832，latency：0.000261 > § 1 client server，10 process per client server，connect all 3 zookeeper > server，all reads，cpu：13%~20%，qps：14000->12000，latency：0.000469 > *§ 1 client server，30 process per client server，connect all 3 zookeeper > server，all reads，cpu：15%~20%，qps：14000->10000，,latency： > § 2 client server，30 process per client server，connect all 3 zookeeper > server，all reads，cpu：15%~20%，qps：about 11000，latency：* > > It seems that the asynchronous calls perform even worse than the > synchronous calls. > > > On Wed, Mar 9, 2011 at 12:29 AM, Flavio Junqueira <f...@yahoo-inc.com>wrote: > >> Hi Qian, If I understand your description correctly, you are using >> synchronous calls. To get high throughput values, you need multiple >> outstanding requests, so you will need to use asynchronous calls. >> >> -Flavio >> >> On Mar 8, 2011, at 5:16 PM, Qian Ye wrote: >> >> P.S. 1 we use zookeeper 3.3.2 >> P.S. 2 all our testing process get data from the same znode. The size of >> data on the znode is less than 1K. >> >> On Wed, Mar 9, 2011 at 12:08 AM, Qian Ye <yeqian....@gmail.com> wrote: >> >> Hi all: >> >> >> These days my friend and I did some performance tests on zookeeper. We >> >> found the performance of zookeeper is not as good as it is described in >> the >> >> Zookeeper Overview ( >> >> http://hadoop.apache.org/zookeeper/docs/r3.3.2/zookeeperOver.html) . In >> >> the Zookeeper Overview, the "ZooKeeper Throughput as the Read-Write Ratio >> >> Varies" shows that in a ensemble of 3 Zookeeper server, the throughput can >> >> reach about 80000, if the requests are all reads. However, we cannot get >> >> results like that in our performance test with the synchronized interface, >> >> zkpython. >> >> >> Here is some of our test results: >> >> (3 zookeeper ensemble, 8 core CPU, 2.4GHZ, 16 RAM, Linux 2.6.9) >> >> >> § 1 client server，1 process per client server，connect 1 zookeeper >> >> server，all reads：cpu：8%~9%，qps：2208，latency：0.000453s >> >> § 1 client server，1 process per client server，connect all 3 zookeeper >> >> server，all reads：cpu：8%~9%，qps：2376.241573 ，latency:0.000421s >> >> § 1 client server，1 process per client server，connect all 3 zookeeper >> >> server，all reads，cpu：10%~20%，qps：15600，latency：0.000764s >> >> *§ 1 client server，30 process per client server，connect all 3 zookeeper >> >> server，all reads，cpu：10%~20%，qps：15200，latency：* >> >> *§ 2 client server，30 process **per client server**，connect all 3 >> >> zookeeper server，all reads，cpu：10%~20%，qps：15800，latency：0.003487* >> >> >> qps means "query per second", that is throughput. The result shows that >> >> when adding more client server, the utilization rate of CPU don't >> increase, >> >> and the throughput don't increase much. It seems that the throughput won't >> >> reach 80000, even if we add 28 more client servers to reach the number you >> >> mentioned in the Zookeeper Overview. >> >> >> Maybe I've done the tests wrong. Is there any particular thing I should >> pay >> >> attention to in this case? We set the max java heap size to 12GB in our >> >> test. >> >> >> *Could you tell me the details about how you do the performance test, from >> >> which you get the results showed in the Zookeeper Overview?* >> >> >> -- >> >> With Regards! >> >> >> Ye, Qian >> >> >> >> >> >> -- >> With Regards! >> >> Ye, Qian >> >> >> *flavio* >> *junqueira* >> >> research scientist >> >> f...@yahoo-inc.com >> direct +34 93-183-8828 >> >> avinguda diagonal 177, 8th floor, barcelona, 08018, es >> phone (408) 349 3300 fax (408) 349 3301 >> >> >> > > > -- > With Regards! > > Ye, Qian > > > *flavio* > *junqueira* > > research scientist > > f...@yahoo-inc.com > direct +34 93-183-8828 > > avinguda diagonal 177, 8th floor, barcelona, 08018, es > phone (408) 349 3300 fax (408) 349 3301 > > >
