thanks for reply. I use scp to check my ethernet again, the transmission rate is about 60MB/s. I send about 100000 requests in each process(60 processes in 2 clients), and it costs about 550 seconds to finish all the requests. And I am sure that all the processes had been spread around all the servers.
Thanks Lei, He 在 2011年3月9日 上午9:22，Benjamin Reed <[email protected]>写道： > are you using 1 gbs ethernet or 100 mbs? those numbers seem to correspond > to > 100 mbs. you may want to do a simple bandwidth test just to make sure. > > how long are you running the test for? > > also, you might check that the clients are being spread around the zk > servers. using the 4 letter works or phunts zktop. > > ben > > 2011/3/9 lei he <[email protected]> > > > Hi Flavio and Qian, I did some tests in my environment, too. And it seems > > it is not as good as it supposed to be. I used async interface(zoo_aget), > > send about 10000 requests to3 servers at the same time. But it turn out > to > > be on the same level with Qian's test. Is there anything I should pay > > attentio to? > > > > thanks a lot > > He, Lei > > > > 2011/3/9 Flavio Junqueira <[email protected]> > > > >> How many outstanding requests do you have at a time, Qian? Only one? If > >> so, you should instead have multiple outstanding. That is, you don't > wait > >> for the callback to issue a new request. > >> > >> -Flavio > >> > >> On Mar 9, 2011, at 3:44 AM, Qian Ye wrote: > >> > >> Hi Flavio, asynchronous calls doesn't perform better, here is some > results > >> we've got, > >> > >> § 1 client server，1 process per client server，connect 1 zookeeper > >> server，all reads：cpu：14%~15%，qps：3833，latency：0.000261 > >> § 1 client server，1 process per client server，connect all 3 zookeeper > >> server，all reads：cpu：14%~15%，qps：3832，latency：0.000261 > >> § 1 client server，10 process per client server，connect all 3 zookeeper > >> server，all reads，cpu：13%~20%，qps：14000->12000，latency：0.000469 > >> *§ 1 client server，30 process per client server，connect all 3 zookeeper > >> server，all reads，cpu：15%~20%，qps：14000->10000，,latency： > >> § 2 client server，30 process per client server，connect all 3 zookeeper > >> server，all reads，cpu：15%~20%，qps：about 11000，latency：* > >> > >> It seems that the asynchronous calls perform even worse than the > >> synchronous calls. > >> > >> > >> On Wed, Mar 9, 2011 at 12:29 AM, Flavio Junqueira <[email protected] > >wrote: > >> > >>> Hi Qian, If I understand your description correctly, you are using > >>> synchronous calls. To get high throughput values, you need multiple > >>> outstanding requests, so you will need to use asynchronous calls. > >>> > >>> -Flavio > >>> > >>> On Mar 8, 2011, at 5:16 PM, Qian Ye wrote: > >>> > >>> P.S. 1 we use zookeeper 3.3.2 > >>> P.S. 2 all our testing process get data from the same znode. The size > of > >>> data on the znode is less than 1K. > >>> > >>> On Wed, Mar 9, 2011 at 12:08 AM, Qian Ye <[email protected]> wrote: > >>> > >>> Hi all: > >>> > >>> > >>> These days my friend and I did some performance tests on zookeeper. We > >>> > >>> found the performance of zookeeper is not as good as it is described in > >>> the > >>> > >>> Zookeeper Overview ( > >>> > >>> http://hadoop.apache.org/zookeeper/docs/r3.3.2/zookeeperOver.html) . > In > >>> > >>> the Zookeeper Overview, the "ZooKeeper Throughput as the Read-Write > Ratio > >>> > >>> Varies" shows that in a ensemble of 3 Zookeeper server, the throughput > >>> can > >>> > >>> reach about 80000, if the requests are all reads. However, we cannot > get > >>> > >>> results like that in our performance test with the synchronized > >>> interface, > >>> > >>> zkpython. > >>> > >>> > >>> Here is some of our test results: > >>> > >>> (3 zookeeper ensemble, 8 core CPU, 2.4GHZ, 16 RAM, Linux 2.6.9) > >>> > >>> > >>> § 1 client server，1 process per client server，connect 1 zookeeper > >>> > >>> server，all reads：cpu：8%~9%，qps：2208，latency：0.000453s > >>> > >>> § 1 client server，1 process per client server，connect all 3 zookeeper > >>> > >>> server，all reads：cpu：8%~9%，qps：2376.241573 ，latency:0.000421s > >>> > >>> § 1 client server，1 process per client server，connect all 3 zookeeper > >>> > >>> server，all reads，cpu：10%~20%，qps：15600，latency：0.000764s > >>> > >>> *§ 1 client server，30 process per client server，connect all 3 zookeeper > >>> > >>> server，all reads，cpu：10%~20%，qps：15200，latency：* > >>> > >>> *§ 2 client server，30 process **per client server**，connect all 3 > >>> > >>> zookeeper server，all reads，cpu：10%~20%，qps：15800，latency：0.003487* > >>> > >>> > >>> qps means "query per second", that is throughput. The result shows that > >>> > >>> when adding more client server, the utilization rate of CPU don't > >>> increase, > >>> > >>> and the throughput don't increase much. It seems that the throughput > >>> won't > >>> > >>> reach 80000, even if we add 28 more client servers to reach the number > >>> you > >>> > >>> mentioned in the Zookeeper Overview. > >>> > >>> > >>> Maybe I've done the tests wrong. Is there any particular thing I should > >>> pay > >>> > >>> attention to in this case? We set the max java heap size to 12GB in our > >>> > >>> test. > >>> > >>> > >>> *Could you tell me the details about how you do the performance test, > >>> from > >>> > >>> which you get the results showed in the Zookeeper Overview?* > >>> > >>> > >>> -- > >>> > >>> With Regards! > >>> > >>> > >>> Ye, Qian > >>> > >>> > >>> > >>> > >>> > >>> -- > >>> With Regards! > >>> > >>> Ye, Qian > >>> > >>> > >>> *flavio* > >>> *junqueira* > >>> > >>> research scientist > >>> > >>> [email protected] > >>> direct +34 93-183-8828 > >>> > >>> avinguda diagonal 177, 8th floor, barcelona, 08018, es > >>> phone (408) 349 3300 fax (408) 349 3301 > >>> > >>> > >>> > >> > >> > >> -- > >> With Regards! > >> > >> Ye, Qian > >> > >> > >> *flavio* > >> *junqueira* > >> > >> research scientist > >> > >> [email protected] > >> direct +34 93-183-8828 > >> > >> avinguda diagonal 177, 8th floor, barcelona, 08018, es > >> phone (408) 349 3300 fax (408) 349 3301 > >> > >> > >> > > >
