Re: facebook memcached on github
On Jan 9, 4:52 am, Arjen van der Meijden wrote: > On 9-1-2009 5:22 Du Song wrote: > > > It's already there. > > in PECL-memcache 3.x > >http://pecl.php.net/package/memcache > > > Regards, > > Du Song > > That code is still beta, which afaik it should be, since at least we > repeatedly run into segfaults in php, which dissappear when using the > stable 2.2-version. > Unfortunately we can't produce a reliable test-case that also segfaults > on the developer's systems. > > So if you have complex object-structures in your memcached, especially > when they do some work in the __wakeup(), you should be very careful > with choosing the 3.x-versions. Or hope you find another test case for > the same bug...http://pecl.php.net/bugs/bug.php?id=13623 > > Best regards, > > Arjen I see; I'll check this 3.0.2b and see how it goes. On another hand, by any chance, would it be possible for you Facebook guys to explain better the changes you guys made on the kernel to optimize it for UDP ? Diffs would be perfect =) I'm in the process of getting some machines up to use exclusively with memcached (8x2.5GHz w/ 64GB RAM) and it would be great if I could poke around with the optimizations you guys made (based on what I read on http://www.facebook.com/note.php?note_id=39391378919) Thanks!
Re: facebook memcached on github
On 9-1-2009 5:22 Du Song wrote: It's already there. in PECL-memcache 3.x http://pecl.php.net/package/memcache Regards, Du Song That code is still beta, which afaik it should be, since at least we repeatedly run into segfaults in php, which dissappear when using the stable 2.2-version. Unfortunately we can't produce a reliable test-case that also segfaults on the developer's systems. So if you have complex object-structures in your memcached, especially when they do some work in the __wakeup(), you should be very careful with choosing the 3.x-versions. Or hope you find another test case for the same bug... http://pecl.php.net/bugs/bug.php?id=13623 Best regards, Arjen
Re: facebook memcached on github
On Sun, Jan 4, 2009 at 23:43, Advt wrote: > > > > On Dec 13 2008, 11:24 am, marc kwiatkowski > wrote: >> As some of you may have noticed we've putfacebook'sversion of >> memcached up on github >> >> http://github.com/fbmarc/facebook-memcached >> >> We've done this for a couple of reasons. >> >> First, as far as we know, we run the largest installation of memcached >> in the world. We think this is true in terms of ops/secs, number of >> objects, amount of RAM, and number of servers. To get memcached to >> support these levels we've had to make a lot of customizations. We >> believe the memcache community would appreciate seeing what we've >> done. >> >> Second, a less complete repository of our version of memcached was >> already up on github. We thought it would be better to have a >> repository with our complete revision history both for our production >> version and our attempt at integration with the 1.2.5 release. >> >> As far as licensing goes. memcached was released under the BSD >> license. We cannot change that even if we wanted to. It goes without >> saying that we don't want to. >> >> Finally, I produced these branches by running git-svn and git-filter- >> branch against our internal SVN repositories. The addition of the >> root "src" directory and the lack of a clear branch from 1.2.3 make >> comparison with existing branches unnecessarily difficult. I >> apologize for the inconvenience. I will redo the git svn extraction >> to correct those issues later this week. >> >> In the mean time I hope you will let us know what you think and what >> else we can do to make memcached even better. We atfacebookwould be >> happy to discuss questions and issues about our release in this group. >> >> Thanks > > > Hi Marc, > > Thanks for making this public and congratz for the wonderful work ! > Any idea on when will the PHP client that uses UDP be available, if > ever ? > > Thanks, > Daniel It's already there. in PECL-memcache 3.x http://pecl.php.net/package/memcache Regards, Du Song
Re: facebook memcached on github
On Dec 13 2008, 11:24 am, marc kwiatkowski wrote: > As some of you may have noticed we've putfacebook'sversion of > memcached up on github > > http://github.com/fbmarc/facebook-memcached > > We've done this for a couple of reasons. > > First, as far as we know, we run the largest installation of memcached > in the world. We think this is true in terms of ops/secs, number of > objects, amount of RAM, and number of servers. To get memcached to > support these levels we've had to make a lot of customizations. We > believe the memcache community would appreciate seeing what we've > done. > > Second, a less complete repository of our version of memcached was > already up on github. We thought it would be better to have a > repository with our complete revision history both for our production > version and our attempt at integration with the 1.2.5 release. > > As far as licensing goes. memcached was released under the BSD > license. We cannot change that even if we wanted to. It goes without > saying that we don't want to. > > Finally, I produced these branches by running git-svn and git-filter- > branch against our internal SVN repositories. The addition of the > root "src" directory and the lack of a clear branch from 1.2.3 make > comparison with existing branches unnecessarily difficult. I > apologize for the inconvenience. I will redo the git svn extraction > to correct those issues later this week. > > In the mean time I hope you will let us know what you think and what > else we can do to make memcached even better. We atfacebookwould be > happy to discuss questions and issues about our release in this group. > > Thanks Hi Marc, Thanks for making this public and congratz for the wonderful work ! Any idea on when will the PHP client that uses UDP be available, if ever ? Thanks, Daniel
Re: facebook memcached on github (redux)
On Dec 21, 12:45 am, Tony Tung wrote: > It could be a separate storage engine, but not as spec'ed. The external > storage engine specification does not allow for scatter-gather storage of the > key and data. Can you be more specific as to how the current interface fails you? What changes would be required for it to do what you need?
Re: facebook memcached on github (redux)
On 12/20/08 4:42 AM, "Toru Maesaka" wrote: Hi! I've quickly glanced through the repo and even though I think the idea of the flat allocator is neat and how it uses mmap (/me loves mmap), I think the flat allocator itself should belong outside the codebase. So, what I'm trying to say is, shouldn't this be a separate storage engine? Trond and I are currently experimenting on reshaping the codebase by changing the item structure to be a minimal and generic 32/64bit aligned structure, which the storage engine will handle. If you're interested I can write more about this. So what I'd like to know is, what exactly is it that you guys are concerned about dynamic linking at startup? If the performance loss (if we could even call it that) is going to become a huge issue for the memcached userbase, we could discuss/plan static linking too. Personally, I think working on things like improving memcached's space efficiency (more bang for buck) seem more important than the dynamic linking issue. Cheers, Toru Opinions? ASFAIK, It could be a separate storage engine, but not as spec'ed. The external storage engine specification does not allow for scatter-gather storage of the key and data. Thanks, Tony
Re: facebook memcached on github (redux)
On Dec 20, 12:56 pm, Matt Ingenthron wrote: > True, more Trond than I. I've been working on a method of testing, and > have a good workload generator. We're actually not using mmap() at the > moment but we'd talked about it before. I don't think it would be a > complex change. > > Also, Trond had some thoughts on how to do this with less locking which > he'd sketched out, but hasn't told me about yet. I think he was working > on resynching everything with the latest engine code. Do any of these require things other than the engine abstraction? One of the main reasons for the engine abstraction was to allow for such experimentation, wasn't it?
Re: facebook memcached on github (redux)
Toru Maesaka wrote: oops, clicked send before I finished the email :( ASFAIK, Trond and Matt has been experimenting quite a bit with what I've mentioned above. True, more Trond than I. I've been working on a method of testing, and have a good workload generator. We're actually not using mmap() at the moment but we'd talked about it before. I don't think it would be a complex change. Also, Trond had some thoughts on how to do this with less locking which he'd sketched out, but hasn't told me about yet. I think he was working on resynching everything with the latest engine code. - Matt On Sat, Dec 20, 2008 at 9:42 PM, Toru Maesaka wrote: Hi! I've quickly glanced through the repo and even though I think the idea of the flat allocator is neat and how it uses mmap (/me loves mmap), I think the flat allocator itself should belong outside the codebase. So, what I'm trying to say is, shouldn't this be a separate storage engine? Trond and I are currently experimenting on reshaping the codebase by changing the item structure to be a minimal and generic 32/64bit aligned structure, which the storage engine will handle. If you're interested I can write more about this. So what I'd like to know is, what exactly is it that you guys are concerned about dynamic linking at startup? If the performance loss (if we could even call it that) is going to become a huge issue for the memcached userbase, we could discuss/plan static linking too. Personally, I think working on things like improving memcached's space efficiency (more bang for buck) seem more important than the dynamic linking issue. Cheers, Toru Opinions? ASFAIK,
Re: facebook memcached on github (redux)
oops, clicked send before I finished the email :( ASFAIK, Trond and Matt has been experimenting quite a bit with what I've mentioned above. Thanks, Toru On Sat, Dec 20, 2008 at 9:42 PM, Toru Maesaka wrote: > Hi! > > I've quickly glanced through the repo and even though I think the idea > of the flat allocator is neat and how it uses mmap (/me loves mmap), I > think the flat allocator itself should belong outside the codebase. > So, what I'm trying to say is, shouldn't this be a separate storage > engine? > > Trond and I are currently experimenting on reshaping the codebase by > changing the item structure to be a minimal and generic 32/64bit > aligned structure, which the storage engine will handle. If you're > interested I can write more about this. > > So what I'd like to know is, what exactly is it that you guys are > concerned about dynamic linking at startup? If the performance loss > (if we could even call it that) is going to become a huge issue for > the memcached userbase, we could discuss/plan static linking too. > > Personally, I think working on things like improving memcached's space > efficiency (more bang for buck) seem more important than the dynamic > linking issue. > > Cheers, > Toru > > Opinions? > ASFAIK, > > On Sat, Dec 20, 2008 at 4:28 AM, dormando wrote: >> >> As-is it doesn't seem to work very well. The slab must be empty, have no >> locked items, etc. I think brad also mentioned crashes in the code? I >> haven't tested it heavily myself, but the feature seems incomplete right >> now. >> >>> Do tell! :) >>> >>> On the protocol side: It'd be great to get it added to >>> http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt >>> and eventually to the various client libraries. >>> >>> On the implementation side: Does there need to be a completely stale >>> slab available? Does it / can it shuffle items around between slabs to >>> free one up (I presume not, since that wouldn't be O(1))? Does it / >>> can it LRU-out a bunch of items to free up a slab (also not O(1))? >>> Without some amount of item reshuffling / forced expiring, it's not >>> immediately clear to me how effective this would be. With the internal >>> reshuffling, I'd be worried about in-band performance on a heavily >>> used instance. Or perhaps I just don't know what I'm talking about. >>> >>> Seriously, though, it'd be super helpful to make this more visible. >>> Just to illustrate, my initial Googling on this topic brings up: >>> >>> (1) Complaints about the problem: http://www.linuxjournal.com/article/7451 >>> (2) Your client code: >>> http://github.com/dustin/java-memcached-client/tree/slab-reassign >>> (3) A hint that slab rebalancing is coming: >>> >>> http://code.sixapart.com/svn/memcached/trunk/server/doc/memory_management.txt >>> (4) Advice to bounce memcache: >>> http://www.socialtext.net/memcached/index.cgi?managing_a_memcached_cluster >>> >>> I'd be happy to help write up / clean up documentation...once I know >>> how it works. If you're swamped, I can just go read the code, but that >>> probably won't happen soon. :) >>> >>> Regards, >>> Josh >>> >> >
Re: facebook memcached on github (redux)
Hi! I've quickly glanced through the repo and even though I think the idea of the flat allocator is neat and how it uses mmap (/me loves mmap), I think the flat allocator itself should belong outside the codebase. So, what I'm trying to say is, shouldn't this be a separate storage engine? Trond and I are currently experimenting on reshaping the codebase by changing the item structure to be a minimal and generic 32/64bit aligned structure, which the storage engine will handle. If you're interested I can write more about this. So what I'd like to know is, what exactly is it that you guys are concerned about dynamic linking at startup? If the performance loss (if we could even call it that) is going to become a huge issue for the memcached userbase, we could discuss/plan static linking too. Personally, I think working on things like improving memcached's space efficiency (more bang for buck) seem more important than the dynamic linking issue. Cheers, Toru Opinions? ASFAIK, On Sat, Dec 20, 2008 at 4:28 AM, dormando wrote: > > As-is it doesn't seem to work very well. The slab must be empty, have no > locked items, etc. I think brad also mentioned crashes in the code? I > haven't tested it heavily myself, but the feature seems incomplete right > now. > >> Do tell! :) >> >> On the protocol side: It'd be great to get it added to >> http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt >> and eventually to the various client libraries. >> >> On the implementation side: Does there need to be a completely stale >> slab available? Does it / can it shuffle items around between slabs to >> free one up (I presume not, since that wouldn't be O(1))? Does it / >> can it LRU-out a bunch of items to free up a slab (also not O(1))? >> Without some amount of item reshuffling / forced expiring, it's not >> immediately clear to me how effective this would be. With the internal >> reshuffling, I'd be worried about in-band performance on a heavily >> used instance. Or perhaps I just don't know what I'm talking about. >> >> Seriously, though, it'd be super helpful to make this more visible. >> Just to illustrate, my initial Googling on this topic brings up: >> >> (1) Complaints about the problem: http://www.linuxjournal.com/article/7451 >> (2) Your client code: >> http://github.com/dustin/java-memcached-client/tree/slab-reassign >> (3) A hint that slab rebalancing is coming: >> >> http://code.sixapart.com/svn/memcached/trunk/server/doc/memory_management.txt >> (4) Advice to bounce memcache: >> http://www.socialtext.net/memcached/index.cgi?managing_a_memcached_cluster >> >> I'd be happy to help write up / clean up documentation...once I know >> how it works. If you're swamped, I can just go read the code, but that >> probably won't happen soon. :) >> >> Regards, >> Josh >> >
Re: facebook memcached on github (redux)
As-is it doesn't seem to work very well. The slab must be empty, have no locked items, etc. I think brad also mentioned crashes in the code? I haven't tested it heavily myself, but the feature seems incomplete right now. Do tell! :) On the protocol side: It'd be great to get it added to http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt and eventually to the various client libraries. On the implementation side: Does there need to be a completely stale slab available? Does it / can it shuffle items around between slabs to free one up (I presume not, since that wouldn't be O(1))? Does it / can it LRU-out a bunch of items to free up a slab (also not O(1))? Without some amount of item reshuffling / forced expiring, it's not immediately clear to me how effective this would be. With the internal reshuffling, I'd be worried about in-band performance on a heavily used instance. Or perhaps I just don't know what I'm talking about. Seriously, though, it'd be super helpful to make this more visible. Just to illustrate, my initial Googling on this topic brings up: (1) Complaints about the problem: http://www.linuxjournal.com/article/7451 (2) Your client code: http://github.com/dustin/java-memcached-client/tree/slab-reassign (3) A hint that slab rebalancing is coming: http://code.sixapart.com/svn/memcached/trunk/server/doc/memory_management.txt (4) Advice to bounce memcache: http://www.socialtext.net/memcached/index.cgi?managing_a_memcached_cluster I'd be happy to help write up / clean up documentation...once I know how it works. If you're swamped, I can just go read the code, but that probably won't happen soon. :) Regards, Josh
Re: facebook memcached on github (redux)
> Well, there's #3 - rebalance the existing slabs using the slab > reassignment command that nobody knows about. :) Do tell! :) On the protocol side: It'd be great to get it added to http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt and eventually to the various client libraries. On the implementation side: Does there need to be a completely stale slab available? Does it / can it shuffle items around between slabs to free one up (I presume not, since that wouldn't be O(1))? Does it / can it LRU-out a bunch of items to free up a slab (also not O(1))? Without some amount of item reshuffling / forced expiring, it's not immediately clear to me how effective this would be. With the internal reshuffling, I'd be worried about in-band performance on a heavily used instance. Or perhaps I just don't know what I'm talking about. Seriously, though, it'd be super helpful to make this more visible. Just to illustrate, my initial Googling on this topic brings up: (1) Complaints about the problem: http://www.linuxjournal.com/article/7451 (2) Your client code: http://github.com/dustin/java-memcached-client/tree/slab-reassign (3) A hint that slab rebalancing is coming: http://code.sixapart.com/svn/memcached/trunk/server/doc/memory_management.txt (4) Advice to bounce memcache: http://www.socialtext.net/memcached/index.cgi?managing_a_memcached_cluster I'd be happy to help write up / clean up documentation...once I know how it works. If you're swamped, I can just go read the code, but that probably won't happen soon. :) Regards, Josh
Re: facebook memcached on github (redux)
On Dec 19, 9:17 am, "Josh Snyder" wrote: > > First, we wanted to solve the slab-ghetto problem. That is the > > situation where one slab is forcing out hot keys for lack of memory > > while another slab holds onto stale objects. > > Just wanted to give a +1 that this is an issue for others as well. At > AdMob, this has been a pretty serious inconvenience in one of our > subsystems. We've used two workarounds to date: > > (1) Bounce memcache every so often to get a clean set of slabs. > (2) Run multiple memcache instances and segregate objects > appropriately. Half our cached data is generally bounded in total > size, critical, and expensive to generate; the other half is generally > unbounded (with slow growth), not quite critical, and fairly cheap to > generate. They also inconveniently tend to live in separate slabs. > Together in one instance, this can generate very painful cache churn. > > Neither of these workarounds is ideal. Bouncing caches is unpleasant > and can require manual attention, and having different memcache > instances has a small but noticeable perf impact in the critical path. > > It would be awesome to have a proper solution to this problem. Well, there's #3 - rebalance the existing slabs using the slab reassignment command that nobody knows about. :)
Re: facebook memcached on github (redux)
On Dec 19, 3:27 am, marc kwiatkowski wrote: > I re-did facebook-memcached to make it consistent with dustin's github > repository. (No more "src" subdirectory, and I redid the commit > history so that it appears to be a branch from the original 1.2.3 > release at > http://github.com/dustin/memcached/commit/437beab948aec088dac361f27c8... > ) > > It is at > > http://github.com/fbmarc/facebook-memcached Hey, this is a great start. I wrote a tool yesterday to do content- based tree comparisons so we can see where trees diverged regardless of recorded ancestry (and messages, and authors, etc...). Yesterday, I couldn't get any variation of a facebook tree to ever match, but today I get this: http://public.west.spy.net/memcached/fbmarc-rwbin.html You can see a rather large divergence, but at the very least we can see the individual diffs that diverge. > I left the original at > > http://github.com/fbmarc/facebook-memcached-old > > I hope this will make it easier to for anyone interested to compare, > merge, and cherry-pick updates. I am a complete git n00b, so if you > have suggestions on how I can repackage this repository to make life > easier, feel free to enlighten me. Well, you can do it all in a single repo. :) My memcached repo has a number of unrelated trees (including this one now). > In the next couple of weeks I'm going to review all of the updates in > dustin's master branch and pull as much as I can into ours. I'm also > cleaning up our C UDP memcache client, php extension and proxy and > will get them up on github as soon as I get everything in presentable > shape. One thing that might make this easier for you is to just fork the repo on github and use their fork queue. I've got your master as of 689aafc9d4420e0193368cd149c94ac9524e51ce and it's good. It's a minor detail at this point, but I'd like to get the authors rewritten before merging anything back. That is, I'd like to avoid this kinds of thing: Author: ps I can clean all those up for you if you'd like, I just need a map from these: 43 ttung 11 ps 9 hzhao 1 marc 1 sky 1 nneul to real names and addresses.
Re: facebook memcached on github (redux)
> First, we wanted to solve the slab-ghetto problem. That is the > situation where one slab is forcing out hot keys for lack of memory > while another slab holds onto stale objects. Just wanted to give a +1 that this is an issue for others as well. At AdMob, this has been a pretty serious inconvenience in one of our subsystems. We've used two workarounds to date: (1) Bounce memcache every so often to get a clean set of slabs. (2) Run multiple memcache instances and segregate objects appropriately. Half our cached data is generally bounded in total size, critical, and expensive to generate; the other half is generally unbounded (with slow growth), not quite critical, and fairly cheap to generate. They also inconveniently tend to live in separate slabs. Together in one instance, this can generate very painful cache churn. Neither of these workarounds is ideal. Bouncing caches is unpleasant and can require manual attention, and having different memcache instances has a small but noticeable perf impact in the critical path. It would be awesome to have a proper solution to this problem. -josh
Re: facebook memcached on github
On Mon, Dec 15, 2008 at 9:14 PM, Tony Tung wrote: > > On 12/14/08 3:40 AM, "Anatoly Vorobey" wrote: > > Tony, > > >1. As you mentioned, we don't see any benefit to runtime selection of >the storage engine; thus the indirect call is unnecessary in our >environment. > > I'm curious about this choice. Have you tried to benchmark the benefit of > using a compile-time choice of a storage engine vs. indirect calls at > runtime? Do you happen to have any statistics on hand regarding whether that > change consistently influences latency/throughput? > > > We haven't done any benchmarks. However, as I understand Toru and Trond's > design, I suspect it would not be particularly expensive. Because the key > is at a fixed location no matter the storage engine, the assoc_find call > would not need to call into the storage engine. > Thanks. I would be surprised if using direct vs indirect calls to the storage engine registered in any meaningful way at all, even if it were done inside assoc_find. -- Anatoly Vorobey, avoro...@gmail.com http://avva.livejournal.com (Russian) http://www.lovestwell.org (English)
Re: facebook memcached on github
On 12/14/08 3:40 AM, "Anatoly Vorobey" wrote: Tony, 1. As you mentioned, we don't see any benefit to runtime selection of the storage engine; thus the indirect call is unnecessary in our environment. I'm curious about this choice. Have you tried to benchmark the benefit of using a compile-time choice of a storage engine vs. indirect calls at runtime? Do you happen to have any statistics on hand regarding whether that change consistently influences latency/throughput? We haven't done any benchmarks. However, as I understand Toru and Trond's design, I suspect it would not be particularly expensive. Because the key is at a fixed location no matter the storage engine, the assoc_find call would not need to call into the storage engine. Thanks, Tony
Re: facebook memcached on github
On Sun, Dec 14, 2008 at 3:40 AM, Anatoly Vorobey wrote: > Tony, >> >> As you mentioned, we don't see any benefit to runtime selection of the >> storage engine; thus the indirect call is unnecessary in our environment. > > I'm curious about this choice. Have you tried to benchmark the benefit of > using a compile-time choice of a storage engine vs. indirect calls at > runtime? Do you happen to have any statistics on hand regarding whether that > change consistently influences latency/throughput? If it turns out in benchmarks to really honestly be painful enough to warrant compiling with direct function calls, maybe a macro could be used that maps at compile-time either directly to one storage engine or to the run-time selected function pointer. Aaron > -- > Anatoly Vorobey, avoro...@gmail.com > http://avva.livejournal.com (Russian) > http://www.lovestwell.org (English) > >
Re: facebook memcached on github
Tony, > >1. As you mentioned, we don't see any benefit to runtime selection of >the storage engine; thus the indirect call is unnecessary in our >environment. > > I'm curious about this choice. Have you tried to benchmark the benefit of using a compile-time choice of a storage engine vs. indirect calls at runtime? Do you happen to have any statistics on hand regarding whether that change consistently influences latency/throughput? -- Anatoly Vorobey, avoro...@gmail.com http://avva.livejournal.com (Russian) http://www.lovestwell.org (English)
Re: facebook memcached on github
On 12/14/08 3:21 AM, "Tony Tung" wrote: In any case, we started our work on supporting multiple storage engines before we heard of Toru and Trond's work. When we did hear about it, we examined their proposal but found that it did not fit our model for two reasons: 1. As you mentioned, we don't see any benefit to runtime selection of the storage engine; thus the indirect call is unnecessary in our environment. 2. It does not support spreading the data out to multiple blocks as we ended up doing with our flat allocator. While it is true that it could be managed by copying the data to a contiguous buffer, that's very expensive. One more thing -- I am certainly not saying that the way we implemented multiple storage engines is The Right Way. I'm just saying that this was right for us; YMMV. :) Thanks, Tony
Re: facebook memcached on github
Comments inline. Thanks, Tony On 12/13/08 3:45 PM, "dormando" wrote: - The FB tree contains a binary protocol that was proprietary up until the release. The memcached binary protocol we have was designed and written by a consortium of users, and has existing clients. I don't think it would be difficult to bring the FB tree's binary protocol to a compatible state. It would also be pretty easy to remove it altogether, as it is pretty modular. - The FB tree contains a weird mishmash of storage engines. Flat allocator? Ok. Memory pools, ok... #ifdef's for the old engine. Fair enough. The memcached community (mainly Toru and Trond) have been working towards a pluggable storage engine interface for memcached. With the intention of pulling in a lot of these forks, and allowing us to ship multiple storage engines which can be runtime pluggable. I recall this was brought up during the hackathon, and FB had serious concerns about the usage of indirect function calls for a storage engine interface. You'd prefer to #ifdef and compile it all out in order to save a few cycles from some primary calls. This goes against what the community has been working towards. (there're even handy public drafts of the interface: http://code.google.com/p/memcached/wiki/EngineInterface). Flat allocator is a storage engine. Memory pools are for finding memory hogs/leaks. In any case, we started our work on supporting multiple storage engines before we heard of Toru and Trond's work. When we did hear about it, we examined their proposal but found that it did not fit our model for two reasons: 1. As you mentioned, we don't see any benefit to runtime selection of the storage engine; thus the indirect call is unnecessary in our environment. 2. It does not support spreading the data out to multiple blocks as we ended up doing with our flat allocator. While it is true that it could be managed by copying the data to a contiguous buffer, that's very expensive. - The FB tree contains dead ends, debug code, and dangerous features. We would never release 'flush_regex' on the main release. Not on purpose, anyway. Besides flush_regex, could you be more specific? - All of the stats output code was rewritten into long, ugly 'append_to_buffer' lines. This doesn't seem necessary, and fills in the gap of what little code wasn't rewritten elsewhere... It's actually quite relevant. If someone were to add to the stats output without adjusting the size of the output buffers, we could easily overrun them. Tracking down memory corruption is such a pain and append_to_buffer guarantees that will not happen (at least not for stats).
Re: facebook memcached on github
Hey, > Finally, I produced these branches by running git-svn and git-filter- > branch against our internal SVN repositories. The addition of the > root "src" directory and the lack of a clear branch from 1.2.3 make > comparison with existing branches unnecessarily difficult. I > apologize for the inconvenience. I will redo the git svn extraction > to correct those issues later this week. Thank you for the clarification. Though that's similar to what our own git repo was until dustin rewrote it, and what mine presently is. What we have right now are actually three trees with unrelated history. Mine - the original cleaned up git-svn import. Dustin's, the rewritten binary tree, and now FB's, which is a weird rewrite based off of 1.2.3. So right now I have all the benefits of a nice distributed RCS, but none of the benefits of being able to manage these branches, since we're all touting to have the same thing with unrelated histories. My work in the short term is to get the binary and stable trees related again, since even if the binary protocol goes out, we will have to maintain the 1.2 series for a while. > In the mean time I hope you will let us know what you think and what > else we can do to make memcached even better. We at facebook would be > happy to discuss questions and issues about our release in this group. Sounds good. I'll reiterate the things we've gone over privately already and we'll work from there. First, for clarity, a few small bits that were discernable from the last merge request were committed and credited in the stable tree. I don't think that's been in a release yet, however. Apologies on my part. As Dustin said, the facebook tree, in its current state, is basically unmergable. For the handful of us who know the codebase well enough can readily tell that this tree is actually a rewrite. There's no items.c at all, there're new files spread about, #ifdef's everywhere. Also as Dustin said the original diffs were longer than the original codebase. During our last discussion at the hackathon in october, we talked about our inability to work with the original code dump. I had directly asked for "at least a tree with history", so we could get a *hint* of insight into what the changes were, and get smaller diffs to examine. The github repository is exactly this. I even okay'ed the usage of github instead of an SVN branch. Easier for all of us to work against. The hope was that we'd be able to work together on messaging and figuring out /what parts/ of the tree were mergable. Instead I've been reading a flood from users chattering about the amazingly performant facebook fork of memcached. - while it's an implementation of memcached provided by facebook, I'm hard pressed to consider it a fork at this point. As another poster has pointed out, FB isn't obligated to do anything, even a code dump is nice. However, most people don't put up code dumps and claim them as merge requests. Even on other projects that I effectively own now (see: MogileFS) I actually bother to rewrite topic branches into clean mail-able, documented patchsets, which go up for review. This is a project where I really don't have to do that at all, and the changes are all internal performance/feature fixes for livejournal and typepad. A lot of things stand between us and mergability: - The FB tree is not compatible with either tree, as CAS was removed. Counters are also very different, but that's minor. Trond has a patch making the CAS requirement a runtime option... I haven't examined it yet, but I recall the FB complaint was not wanting to waste 8 bytes per item storing the CAS identifier. - The FB tree contains a binary protocol that was proprietary up until the release. The memcached binary protocol we have was designed and written by a consortium of users, and has existing clients. - The FB tree contains a weird mishmash of storage engines. Flat allocator? Ok. Memory pools, ok... #ifdef's for the old engine. Fair enough. The memcached community (mainly Toru and Trond) have been working towards a pluggable storage engine interface for memcached. With the intention of pulling in a lot of these forks, and allowing us to ship multiple storage engines which can be runtime pluggable. I recall this was brought up during the hackathon, and FB had serious concerns about the usage of indirect function calls for a storage engine interface. You'd prefer to #ifdef and compile it all out in order to save a few cycles from some primary calls. This goes against what the community has been working towards. (there're even handy public drafts of the interface: http://code.google.com/p/memcached/wiki/EngineInterface). - The FB tree contains dead ends, debug code, and dangerous features. We would never release 'flush_regex' on the main release. Not on purpose, anyway. - All of the stats output code was rewritten into long, ugly 'append_to_buffer' lines. This doesn't seem necessary, and fills in the gap of what little cod
