oops, clicked send before I finished the email :( ASFAIK, Trond and Matt has been experimenting quite a bit with what I've mentioned above.
Thanks, Toru On Sat, Dec 20, 2008 at 9:42 PM, Toru Maesaka <tmaes...@gmail.com> wrote: > Hi! > > I've quickly glanced through the repo and even though I think the idea > of the flat allocator is neat and how it uses mmap (/me loves mmap), I > think the flat allocator itself should belong outside the codebase. > So, what I'm trying to say is, shouldn't this be a separate storage > engine? > > Trond and I are currently experimenting on reshaping the codebase by > changing the item structure to be a minimal and generic 32/64bit > aligned structure, which the storage engine will handle. If you're > interested I can write more about this. > > So what I'd like to know is, what exactly is it that you guys are > concerned about dynamic linking at startup? If the performance loss > (if we could even call it that) is going to become a huge issue for > the memcached userbase, we could discuss/plan static linking too. > > Personally, I think working on things like improving memcached's space > efficiency (more bang for buck) seem more important than the dynamic > linking issue. > > Cheers, > Toru > > Opinions? > ASFAIK, > > On Sat, Dec 20, 2008 at 4:28 AM, dormando <dorma...@rydia.net> wrote: >> >> As-is it doesn't seem to work very well. The slab must be empty, have no >> locked items, etc. I think brad also mentioned crashes in the code? I >> haven't tested it heavily myself, but the feature seems incomplete right >> now. >> >>> Do tell! :) >>> >>> On the protocol side: It'd be great to get it added to >>> http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt >>> and eventually to the various client libraries. >>> >>> On the implementation side: Does there need to be a completely stale >>> slab available? Does it / can it shuffle items around between slabs to >>> free one up (I presume not, since that wouldn't be O(1))? Does it / >>> can it LRU-out a bunch of items to free up a slab (also not O(1))? >>> Without some amount of item reshuffling / forced expiring, it's not >>> immediately clear to me how effective this would be. With the internal >>> reshuffling, I'd be worried about in-band performance on a heavily >>> used instance. Or perhaps I just don't know what I'm talking about. >>> >>> Seriously, though, it'd be super helpful to make this more visible. >>> Just to illustrate, my initial Googling on this topic brings up: >>> >>> (1) Complaints about the problem: http://www.linuxjournal.com/article/7451 >>> (2) Your client code: >>> http://github.com/dustin/java-memcached-client/tree/slab-reassign >>> (3) A hint that slab rebalancing is coming: >>> >>> http://code.sixapart.com/svn/memcached/trunk/server/doc/memory_management.txt >>> (4) Advice to bounce memcache: >>> http://www.socialtext.net/memcached/index.cgi?managing_a_memcached_cluster >>> >>> I'd be happy to help write up / clean up documentation...once I know >>> how it works. If you're swamped, I can just go read the code, but that >>> probably won't happen soon. :) >>> >>> Regards, >>> Josh >>> >> >
