Thanks Sam for the detailed reply. I'll try to comeup with a way partition the counters. i could find a library written for Zookeeper based id generation. [1] After using it with some randomization to minimize the contention i could get much higher throughput. (~400id/s). So looks like this bottleneck is purely due to the approach i m following(as you pointed out).
[1]http://sourceforge.net/projects/zkidgen/ thanks again for the help. cheers, Charith On Wed, Jun 13, 2012 at 7:23 AM, Sam Tunnicliffe <[email protected]> wrote: > Yes, the contention occurs in a multithreaded context when two or more > clients attempt to increment the same counter concurrently. The first > in wins and the following updates are rejected as the znode version no > longer matches the one obtained from the getData call. A clients who's > increment is rejected just goes around the loop again, we don't have > any more sophisticated logic in place to ensure fairness. > > We don't do any special tuning on the server side and we're running in > AWS on standard m1.large instances. It could be the case that we're > only seeing 10s of updates/s per unique counter like you, but because > our counters are partitioned and we have multiple concurrent clients, > total throughput is increased. > > Cheers, > Sam > > On 13 June 2012 11:52, Charith Wickramarachchi > <[email protected]> wrote: > > Thanks Sam for the reply. > > > > The contention happens when we to setData from muliple clients right ? > (due > > to locking). But here even when i try using a single client the > throughput > > is in 20-30 id/s range. > > Did you do any tuning in the server side. ? > > > > regards, > > Charith > > > > On Wed, Jun 13, 2012 at 6:17 AM, Sam Tunnicliffe <[email protected]> wrote: > > > >> Hi Charith, > >> > >> can you partition your counters somehow, rather than having a single > >> global counter? This would let you use multiple znodes, reducing > >> contention during the setData call. > >> We use almost exactly the same approach as you in our zookeeper backed > >> sequence server[1] but with sharded counters and haven't had any > >> issues with throughput (sorry, I don't have perf numbers to hand, but > >> peak load pushes a few hundred r/s through) > >> > >> Cheers, > >> Sam > >> > >> [1] https://github.com/talis/H1 > >> > >> On 13 June 2012 05:20, Charith Wickramarachchi > >> <[email protected]> wrote: > >> > Hi All , > >> > > >> > I'm evaluating Apache Zookeepers to use in my project where i have a > >> > requirement to generate unique and sequential message ids(long) > across a > >> > cluster. > >> > After some web search i found[1] and some other different > suggestions. > >> I > >> > started with the solution suggested in [1] but even with single > client i > >> > only got a throughput around 20-30 ids/sec. But according to the > >> performance > >> > numbers pointed in the website i think i should be able get better > >> results ( > >> > i m expecting to get at least 1000 id/s as i m using this for a MoM > where > >> > for each message i'll be calling zk id generator.) > >> > > >> > In this case I m using ZK version 3.3.4 running in my local machine > with > >> > default settings and client also running in the same machine. I m > >> attaching > >> > my id generator class. > >> > > >> > Any tips to improve the performance of this is highly appreciated. Am > i > >> > doing some thing wrong ? > >> > > >> > > >> > [1] > >> > http://zookeeper-user.578899.n2.nabble.com/Unique-Id-Generation-td2688494.html > >> > > >> > cheers, > >> > Charith > >> > -- > >> > Charith Dhanushka Wickramarachchi > >> > http://charithwiki.blogspot.com/ > >> > > >> > > > > > > > > -- > > Charith Dhanushka Wickramarachchi > > http://charithwiki.blogspot.com/ > -- Charith Dhanushka Wickramarachchi http://charithwiki.blogspot.com/
