Status: Accepted Owner: dorma...@rydia.net Labels: Type-Defect Priority-Medium
New issue 95 by dorma...@rydia.net: Memory allocation default change (-m < 40 doesn't work)
http://code.google.com/p/memcached/issues/detail?id=95 Yo, The default way memcached allocates memory is a little broken right now. It also stands for debate that there can be improvements to the defaults. However since this is a potential default adjustment, I want to address the community before we do anything. The issue I'm working off of: - '-m n' where n < number_of_slab_classes does not presently work. When memory is requested for a slab class and there is none available (either there is no memory at all in the slab class or it is out of space), memcached allocates 1 page if there is free space available in the main pool, *OR* will *always* allocate 1 page if the slab class has *0* pages allocated to it. This latter part ignores the memory limit. This means if you set -m 18, fill up one slab class, then put one item in all the other slab classes, you will be using over 60 megs of ram. This gets worse if you adjust the page size. An option for dealing with this is to enable forced memory preallocation (1 page per slab) at start time and whine if there isn't enough memory. The other side of this issue is that by default we allocate memory 1 megabyte at a time as pages are requested. So if you set '-m 8000' and insert items very slowly with low expire times, you won't ever get to 8G. This leads to users telling memcached to use more RAM than is available on the box, and then grinding into swap after several days or weeks. The -L option will preallocate the memory if you have hugetlbs/etc available. If a mass allocation (one giant block) doesn't work, it falls back to allocating one page at a time. I've thought about this before, and Dustin just reiterated, that the default should probably be to allocate all of the memory requested at start time. It should attempt to do a mass allocation, and if that fails it should spin on 1 meg allocations until it's full. From there, the slabs should be preallocated with at least 1 page, since the existing behavior will not work anymore. However this is a pretty big change to the defaults. It should have a large effect on the usability of memcached, so I want to propose that we bother doing this in some form. Thoughts? Please respond for discussion on the mailing list or in the issue here. (this won't be queued for or block 1.4.2, but could show in 1.4.3) -- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings
