Sorry for the report, I made a mistake. When I turn off the debug level log in zookeeper servers, the performance is like you described. Thanks for you guys. 在 2011年3月10日 上午10:31，lei he <hl0...@gmail.com>写道：
> thanks for reply. I use scp to check my ethernet again, the transmission > rate is about 60MB/s. I send about 100000 requests in each process(60 > processes in 2 clients), and it costs about 550 seconds to finish all the > requests. And I am sure that all the processes had been spread around all > the servers. > > Thanks > Lei, He > > 在 2011年3月9日 上午9:22，Benjamin Reed <br...@apache.org>写道： > > are you using 1 gbs ethernet or 100 mbs? those numbers seem to correspond >> to >> 100 mbs. you may want to do a simple bandwidth test just to make sure. >> >> how long are you running the test for? >> >> also, you might check that the clients are being spread around the zk >> servers. using the 4 letter works or phunts zktop. >> >> ben >> >> 2011/3/9 lei he <hl0...@gmail.com> >> >> > Hi Flavio and Qian, I did some tests in my environment, too. And it >> seems >> > it is not as good as it supposed to be. I used async >> interface(zoo_aget), >> > send about 10000 requests to3 servers at the same time. But it turn out >> to >> > be on the same level with Qian's test. Is there anything I should pay >> > attentio to? >> > >> > thanks a lot >> > He, Lei >> > >> > 2011/3/9 Flavio Junqueira <f...@yahoo-inc.com> >> > >> >> How many outstanding requests do you have at a time, Qian? Only one? If >> >> so, you should instead have multiple outstanding. That is, you don't >> wait >> >> for the callback to issue a new request. >> >> >> >> -Flavio >> >> >> >> On Mar 9, 2011, at 3:44 AM, Qian Ye wrote: >> >> >> >> Hi Flavio, asynchronous calls doesn't perform better, here is some >> results >> >> we've got, >> >> >> >> § 1 client server，1 process per client server，connect 1 zookeeper >> >> server，all reads：cpu：14%~15%，qps：3833，latency：0.000261 >> >> § 1 client server，1 process per client server，connect all 3 zookeeper >> >> server，all reads：cpu：14%~15%，qps：3832，latency：0.000261 >> >> § 1 client server，10 process per client server，connect all 3 zookeeper >> >> server，all reads，cpu：13%~20%，qps：14000->12000，latency：0.000469 >> >> *§ 1 client server，30 process per client server，connect all 3 zookeeper >> >> server，all reads，cpu：15%~20%，qps：14000->10000，,latency： >> >> § 2 client server，30 process per client server，connect all 3 zookeeper >> >> server，all reads，cpu：15%~20%，qps：about 11000，latency：* >> >> >> >> It seems that the asynchronous calls perform even worse than the >> >> synchronous calls. >> >> >> >> >> >> On Wed, Mar 9, 2011 at 12:29 AM, Flavio Junqueira <f...@yahoo-inc.com >> >wrote: >> >> >> >>> Hi Qian, If I understand your description correctly, you are using >> >>> synchronous calls. To get high throughput values, you need multiple >> >>> outstanding requests, so you will need to use asynchronous calls. >> >>> >> >>> -Flavio >> >>> >> >>> On Mar 8, 2011, at 5:16 PM, Qian Ye wrote: >> >>> >> >>> P.S. 1 we use zookeeper 3.3.2 >> >>> P.S. 2 all our testing process get data from the same znode. The size >> of >> >>> data on the znode is less than 1K. >> >>> >> >>> On Wed, Mar 9, 2011 at 12:08 AM, Qian Ye <yeqian....@gmail.com> >> wrote: >> >>> >> >>> Hi all: >> >>> >> >>> >> >>> These days my friend and I did some performance tests on zookeeper. We >> >>> >> >>> found the performance of zookeeper is not as good as it is described >> in >> >>> the >> >>> >> >>> Zookeeper Overview ( >> >>> >> >>> http://hadoop.apache.org/zookeeper/docs/r3.3.2/zookeeperOver.html) . >> In >> >>> >> >>> the Zookeeper Overview, the "ZooKeeper Throughput as the Read-Write >> Ratio >> >>> >> >>> Varies" shows that in a ensemble of 3 Zookeeper server, the throughput >> >>> can >> >>> >> >>> reach about 80000, if the requests are all reads. However, we cannot >> get >> >>> >> >>> results like that in our performance test with the synchronized >> >>> interface, >> >>> >> >>> zkpython. >> >>> >> >>> >> >>> Here is some of our test results: >> >>> >> >>> (3 zookeeper ensemble, 8 core CPU, 2.4GHZ, 16 RAM, Linux 2.6.9) >> >>> >> >>> >> >>> § 1 client server，1 process per client server，connect 1 zookeeper >> >>> >> >>> server，all reads：cpu：8%~9%，qps：2208，latency：0.000453s >> >>> >> >>> § 1 client server，1 process per client server，connect all 3 zookeeper >> >>> >> >>> server，all reads：cpu：8%~9%，qps：2376.241573 ，latency:0.000421s >> >>> >> >>> § 1 client server，1 process per client server，connect all 3 zookeeper >> >>> >> >>> server，all reads，cpu：10%~20%，qps：15600，latency：0.000764s >> >>> >> >>> *§ 1 client server，30 process per client server，connect all 3 >> zookeeper >> >>> >> >>> server，all reads，cpu：10%~20%，qps：15200，latency：* >> >>> >> >>> *§ 2 client server，30 process **per client server**，connect all 3 >> >>> >> >>> zookeeper server，all reads，cpu：10%~20%，qps：15800，latency：0.003487* >> >>> >> >>> >> >>> qps means "query per second", that is throughput. The result shows >> that >> >>> >> >>> when adding more client server, the utilization rate of CPU don't >> >>> increase, >> >>> >> >>> and the throughput don't increase much. It seems that the throughput >> >>> won't >> >>> >> >>> reach 80000, even if we add 28 more client servers to reach the number >> >>> you >> >>> >> >>> mentioned in the Zookeeper Overview. >> >>> >> >>> >> >>> Maybe I've done the tests wrong. Is there any particular thing I >> should >> >>> pay >> >>> >> >>> attention to in this case? We set the max java heap size to 12GB in >> our >> >>> >> >>> test. >> >>> >> >>> >> >>> *Could you tell me the details about how you do the performance test, >> >>> from >> >>> >> >>> which you get the results showed in the Zookeeper Overview?* >> >>> >> >>> >> >>> -- >> >>> >> >>> With Regards! >> >>> >> >>> >> >>> Ye, Qian >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> With Regards! >> >>> >> >>> Ye, Qian >> >>> >> >>> >> >>> *flavio* >> >>> *junqueira* >> >>> >> >>> research scientist >> >>> >> >>> f...@yahoo-inc.com >> >>> direct +34 93-183-8828 >> >>> >> >>> avinguda diagonal 177, 8th floor, barcelona, 08018, es >> >>> phone (408) 349 3300 fax (408) 349 3301 >> >>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> With Regards! >> >> >> >> Ye, Qian >> >> >> >> >> >> *flavio* >> >> *junqueira* >> >> >> >> research scientist >> >> >> >> f...@yahoo-inc.com >> >> direct +34 93-183-8828 >> >> >> >> avinguda diagonal 177, 8th floor, barcelona, 08018, es >> >> phone (408) 349 3300 fax (408) 349 3301 >> >> >> >> >> >> >> > >> > > -- Best Regards Lei, He
