> On Fri, Jul 23, 2010 at 8:47 AM, dormando <dorma...@rydia.net> wrote: > I tried. > > Try the engine branch? > > I guess, I'll have to at some point. > > Just wanted to say, that LRU was designed as an algorithm for a uniform cost > model, where all elements are almost equally important (have the same cost of > miss) and the only thing that distinguishes them is > the pattern of accesses. This is clearly not a good model for memcache, > where: some elements are totally unimportant as they have already expired, > some elements are larger than the others, some are always > processed in batches (multigets), and so on. In my opinion GC moves the > reality closer to the model, by removing unimportant elements, so if you want > LRU to work correctly you should at least perform GC. > You could also try to modify LRU to model that one large item actually > occupies space that could be better utilies by several small elements (this > is also a simple change). If you fill comfortable without > GC, I am OK with that, just do not suggest, that GC is against LRU.
Alright, I'm sorry. I've been unfair to you (and a few others recently). I've been unnecessarily grumpy. I tried to explain myself as fairly as possible, and Dustin added the words that I apparently forgot already, in that these things are better pressed through via SE's. I get annoyed by these threads because: - I really don't care for arguments on this level. When I said "GC goes against the LRU" I mean that the LRU we have doesn't require GC. The whole point of adding the LRU was so we could skip that part. I'm describing *intent*, I'm just too tired to keep arguing these things. - The thread hijacking is seriously annoying. If you want to ping us about an ignored patch, start a new thread or necro your own old thread. :( - Your original e-mail opened with "We run this in single threaded mode and the performance is good enough for us so please merge it". I'm pretty dumbfounded that people can take a project which is supposed to be the performant underpinnings of the entire bloody internet and not do any sort of performance testing. I try to test things and I do have some hardware on hand but I'm still trying to find the motivation in myself to do a thorough performance run through of the engine branch. There's a lot of stuff going on in there. This is time consuming and often frustrating work. You did make a good attempt at building an efficient implementation, and it's a very clever way to go about the business, but best case: - You're adding logic to the most central global lock - You're adding 16 bytes per object - Plus some misc memory overhead (minor). If they're not causing the locks to be problems, the memory efficiency drop is an issue for many more people. If we make changes to the memory requirements of the default engine, I really only want to entertain ideas that make it *drop* requirements (we have some, need to start testing them as the engine stuff gets out there). The big picture is many users have small items, and if we push this change many people will suffer. Yes it's true that once those metrics "expose" an issue you technically already have an issue, but it's not an instant dropoff. Easily calculable with graphs and things like the "evicted_time" stats. Items dropping off the end that haven't been touched in 365,000+ seconds aren't likely to cause you a problem tomorrow or even next week, but watch for that number to fall. This is also why the evicted and evicted_nonzero stats were split. Eviction of an item with a 0 expiration is nearly meaningless. However, I can't seem to get this through without being rude to people, and I apologize for that. I should've responded to your original message with these *technical* problems instead of just harping on the idea that it looks like you weren't using all of the available statistics properly. I'm trying to chillax and get back to being a fun (albeit grumpy) productive hacker dude. Sorry, all. -Dormando
