Hi Dormando, > Can you explain what you mean by "expire" ? You've gone into great detail > about needing it, but I don't see how explicitly expiring items is doing > you get good over letting them be evicted? Or by setting more sane > expiration times?
well, lets say I have 2 G RAM in total for memcached. if I just let memcached be running, it will allocate (over time) the total 2 G RAM and put those 1MB pages to slabs. the page-slab-assignment is fixed, so if 1.8 G of RAM would be assigned to slab 18, there are only 200 MB left for all other slabs. for some slabs there might as well be only 1 page assigned (see last mail). If by then too many items for that 1-page-slab should be stored, they cannot. contrary, if I do a memcache-restart, and then regularly expire the memcache, it won't allocate the total of 2 G RAM but only as much as really needed. Because when the entry is REALLY expired (= deleted from cache), that ram in that slab will be reused for the next entries. So instead of getting 300.000 items into slab 18/19 (with 290.000 of them already expired) I will only have lets say 10.000 to 20.000 Items there. So memcached will only allocate 50-100 MB RAM for that slab and not 1.8 GB. If in that case there would come too many items for a 1-page-slab, then memcached is able to allocate just as many 1-MB-pages to that slab, as there are enough 1-MB pages (still) free and not yet allocated. doing this, memcached does not need to evict any items. > There is expire in the protocol already? Just not.... mass expire? I did not find a thing, I only know about flush_all, possible with timestamp, for doing the flush later. but that is not what I would need. > Automatic (and possibly manual) slab reassignment should happen in the > coming months I don't know the details here, so I cannot see if this could help, but it sounds good. But there should be some clever algorithm behind to reassign the "empty" pages to other slabs, or at least maybe reorder some items to other pages to get one empty. If it would just evict all items on a page to make that page empty for reassignment, then it would be no help, as then still valid items would be evicted. We are storing (among lots of other things) sessions into memcached and expiriencing some lost sessions and it seems to be the above thing. I see that this excactly is the problem and the difference between a cache and a database, and that memcached is not a database (and does not intent to be one). But we have choosen it for performance at this task, and the expire above could help, as long as there would be enough not-yet-assigned-1-MB-pages left. our big memcache-servers have 4 GB RAM assigned to memcached, but if I do my expire, I can strip that down to 300-700 MB RAM, so there would be over 3 GB left to be freely assigned when needed. > For now you should use 1.4.0, that must be through our testing environment first and then be approved by others in our departement. At the moment I can live with my "client-side"-Expire, but it has just been written as a "proof-of-concept" first, and I still feel, it is just the wrong for a expire. (client-side php script instead of some lines C-code serverside). > and if it becomes a problem, slow roll a restart of your > memcached's occasionally. that's what we have to do now, but normally our memcacheds have uptimes as high as 200+ days. > Most folks almost never have to do this. It might just depend on the use, the amount of stores/retrievals, and so on. The problem just comes up, when the cache is running too long (months) and the behaviour of the application changes (as e.g. the typical session is some bytes larger and thus does not fit into the same slab as before). > Are you perhaps talking about something like needing namespace-level > expiration? > http://code.google.com/p/memcached/wiki/FAQ#Namespaces no, sorry. I just need a deletion of old (expired) objects, so that the memcached would find a new slot in a already assigned page in the same slab instead of allocation more RAM when it is told to store a new item :) so back to my question: >> So, now the question to the maintainer folks: >> c) I could imagine to try and implement on my own, (my knowledge in C >> is a bit dusty but at least I could read the code till now). >> but this clearly only makes sense if that patch would make its way >> into the sources and stay there. regards Werner.
