Hi guys, I fixed the stats opaque issue pointed out by Dustin's test case at the hackathon and just pushed it to github:
http://github.com/tmaesaka/memcached/commit/86159eb7b5c611d25a33eb1cb6c750c806350912 I also refactored 'server_stats()' which is a function that returns server specific stats. In the previous implementation, a string in the usual stats format was created and returned if server_stats() was working for the ascii protocol. Otherwise, server_stats() would parse the string and create binary protocol packets based on that data. The above approach is not nice at all and I didn't like it from the beginning. It was implemented like that because it was written before the callback idea was adopted. Now there are no conditional statements to determine whether server_stats() is working for the binary protocol or not (because the appropriate callback is now passed to server_stats). Oh, and I noticed that Dustin had merged my binary tree and there is something I feel really guilty about... Some pushes I made to my tree were done manually to bring over 1.2.6/1.2.7 stable patches into the binary tree. This is probably going to hit us badly when merging other trees even with the same ancestor (e.g. brad's http://github.com/bradfitz/memcached/commits/incr_fix). Theres probably a way to fix this but I'm not quite a git ninja yet... Does anyone have any thoughts/solution for this? Cheers, Toru On Fri, Oct 17, 2008 at 5:08 AM, Ben Manes <[EMAIL PROTECTED]> wrote: > "Trond also mentioned interesting lock reduction ideas, like moving from a > single cache lock to lock per hash region (perhaps like java concurrent hash > map). I think Marc, in a different conversation, suggested the idea of a > lock-free hash table (ala Cliff Click)." > > You may also want to consider the hopscotch algorithm: > http://groups.google.com/group/hopscotch-hashing?pli=1 > > ----- Original Message ---- > From: steve.yen <[EMAIL PROTECTED]> > To: memcached <memcached@googlegroups.com> > Sent: Thursday, October 16, 2008 9:08:26 AM > Subject: Re: Memcached Hackathon Report(s) > > > A fun hackathon, folks were there until 2am. Other conversations that > I recall... > > - Paul Saab & Marc Kwiatkowski described a bunch of the Facebook > changes that helped them get towards 300K reqs/sec on 8 core server. > I think they worked a bit towards putting up an svn/git repo so their > changes > could be more easily folded back by the community into the mainline > (putting > out something with change history rather than just a huge hard-to- > digest patch). > Appreciate that! > > - In particular, they described one performance blocker which they > fixed > of the single stats lock and how they moved towards a stats-per- > thread > model that tracks info in thread local data. These TLD stats are > aggregated lazily, on-demand in response to a stats request. > > - Trond and I talked a little about making the upcoming storage engine > interface > async rather than synchronous flavored. Trond also mentioned > interesting lock > reduction ideas, like moving from a single cache lock to lock per hash > region > (perhaps like java concurrent hash map). I think Marc, in a different > conversation, > suggested the idea of a lock-free hash table (ala Cliff Click). > > - For those of you who thought a hash table per thread might be > interesting (like me), > Trond says it's been tried and is not an improvement. The contention > just moves > towards the request queueing in front of each thread. > > - There was talk and progress about adding replication into > libmemcached client lib. > Senthil (apologies if I misremembered your name) from Yahoo did the > work, and we > talked with Brian Aker about replication issues, such as handling > flapping nodes. > > steve > > On Oct 15, 1:48 am, Dustin <[EMAIL PROTECTED]> wrote: >> The hackathon went well again. Admittedly, I didn't talk to everyone >> there, but from the corner with the food, there was some additional >> protocol work and tree merging going on. Please fill in interesting >> things that happened that I didn't know about or forgot. >> >> In particular, Toru implemented stats for the binary protocol. It's >> pretty good stuff. I wrote java client support for it, and helped >> iron out some minor issues that arose. >> >> I think the only change that came out of this was that the opaque >> behavior was kind of undefined. All opaque values coming out of stats >> were set to 0 (and the requested opaque was ignored). Stats values >> will have the magic opaque value of 0x746f7275 and the terminating >> stat will have the requested opaque value. >> >> *INCOMPATIBILITY NOTES* >> >> We have one change coming up that has been affecting me a lot re: >> compatibility and that's the removal of hold values from delete. If >> you rely on this behavior, please let someone know -- it's going away. >> >> Brad did some work on making the long parser more strict in incr/ >> decr. If you rely on arbitrary objects being treated as 0, you have a >> silent bug that will turn into a loud bug in the future. > >
