Read carefully:
item_size_ok(const size_t nkey, const int flags, const int nbytes) {
passes:
size_t ntotal = item_make_header(nkey + 1, flags, nbytes,
prefix, &nsuffix);
Then conditionally:
if (settings.use_cas) {
ntotal += sizeof(uint64_t);
}
item_make_header is doing:
nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags, nbytes - 2);
Then:
return sizeof(item) + nkey + *nsuffix + nbytes;
It's convoluted but shirt.
the lengths are:
key +
1 +
data_bytes +
2b appended to data_bytes for an extra "\r\n" +
stringified rep of the flags + data length
+ 4 bytes for spaces and \r\n (these are carriage returns, one byte each)
+ 8b for CAS if enabled
CAS can be turned off via the -C starttime arg. it takes up 8 bytes.
Example:
* Key : 2c
* Val : 28b
* Flg : 0 (1b)
turns into:
* Key : 3b
* Val : 30b
* Hdr : 4b + 3b == 7b
* Itm : 56b
=> 96b. which is the cap for slab 1 in a default setup.
It's tough to get it exact for small chunks due to the way the header is
added. You should ballpark or tune the -f value to align with your
observed data.
On Sun, 15 Nov 2015, Nicolas Martinez wrote:
> Hi,
> Is CAS always used?
> If yes, we have to always add 56 bytes to the KEY and VALUE ?
> you don't count FLAGS characters??
>
> I've found that Flags's size (number of characters) impact the storage.
>
> Example:
> * Key : 2 characters = 2 bytes
> * Value : 28 characters = 28 bytes
> * FLAGS : 1 characters = 1 bytes
> => 31 bytes
>
> seems to take the same storage as
> * Key : 1 characters = 1 bytes
> * Value : 28 characters = 28 bytes
> * FLAGS : 2 characters = 2 bytes
> => 31 bytes ... wich is the limit to be stored in Slab1
>
> ok for the /r/n ... should take 4 bytes no?
>
> So, if we count 56 bytes for CAS : 56(cas)+31(key+value+flags)+4(/r/n)= 91
>
> Not good... :(
>
> where I'm wrong ??
>
> Le samedi 14 novembre 2015 23:55:12 UTC+1, Dormando a écrit :
> The mysql docs don't speak for the main tree... that's their own thing.
>
> the "sizes" binary that comes with the source tree tells you how many
> bytes an item will use (though I intend to add this output to the
> 'stats'
> output somewhere). With CAS this is 56 bytes.
>
> 56 + 2 + 30 == 88. Class 1 by default (in 1.4.24) is 96 bytes, but the
> item still ends up in class 2.
>
> Why is this? (unfortunately?) because memcached pre-renders part of the
> text protocol into the item header:
>
> *nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags, nbytes -
> 2);
> return sizeof(item) + nkey + *nsuffix + nbytes;
>
> so the flags + length are getting flattened + \r\n added to the end.
> Together that's just enough to push it over the edge. It'd also be nice
> to
> add a highly optimized numerics printf so I could twiddle options to
> save
> a few bytes of memory in objects, but don't get your hopes up for that
> happening soon :)
>
> On Sat, 14 Nov 2015, Nicolas Martinez wrote:
>
> > Add: Memcached version : 1.4.4 (RedHat)
> >
> > Le samedi 14 novembre 2015 14:49:37 UTC+1, Nicolas Martinez a écrit :
> > Hi, few days i'm reading Memcached documentation and blogs...
> and i don't understand how objects are stored.
> >
> > My test
> >
> > 3 slabs :
> >
> > * 96.0 Bytes
> > * 120.0 Bytes
> > * 240.0 Bytes
> > Everywhere, it's told :
> > * if data is < 96 Bytes, it will be stored in Slabs1 (96B)
> > * if data > 96B and < 120B, it will be stored in Slabs2 (120B)
> > * if data > 120B, it will be stored in Slabs3 (240B)
> > * etc.
> > BUT, for example, when i'm creating an 30B object, it's stored in
> Slab2 (120B), and NOT in Slab1 (96B).
> >
> > External sources:
> > For example, the default size for the smallest block is 88
> bytes (40 bytes of value, and the default 48 bytes for the key and flag
> data). If the size of the first item you store into the cache is less than 40
> bytes, then a slab with a block size of 88 bytes is created and the value
> stored.
> > =>
> https://dev.mysql.com/doc/mysql-ha-scalability/en/ha-memcached-using-memory.html
> >
> >
> > WRONG
> >
> > A slab class is a collection of pages divided into same sized
> chunks. Each slab class is referenced to by its chunk size. So we’ll have
> Slab class 80kb, Slab class 100kb and so on. When an object needs to be
> stored, its size determines where it gets stored. So if the object is larger
> than 80kb but less than 100kb, it gets stored into
> Slab
> > class 100kb.
> > =>
> http://returnfoo.com/2012/02/memcached-memory-allocation-and-optimization-2/
> >
> >
> > WRONG
> >
> > How i create an object:
> >
> > data=$(pwgen 30 -c 1)
> > echo -e 'set 30 0 3600 30\r\n'$data'\r'| nc ${memcached_server}
> 11211
> >
> >
> > So, when 30B object is creating :
> > * key name : 30 = 2 bytes
> > * value: 30 characters = 30 bytes
> > * tags : 0 = 1 bytes
> > => All = 33 bytes
> > If i add the default 48b as explained on Mysql website : 33 + 48 =
> 81B ... so < Slab1 (91B)... but always stored in Slab2 (120B)
> >
> > So, the size used to store object in the good Slab is not:
> > * object value size
> > * sum of KEY, VALUE and TAGS in bytes
> > KEY size : 1 character = 1 B
> > VALUE size : 1 character = 1 B
> > TAGS size : 1 character = 1 B
> >
> > ... as read everywhere
> >
> > So, It seems that: (SUM of KEY+VALUE+TAGS )
> > * For slab1 96.0 Bytes, data stored if <= 31 B (SUM of 2+28+1 )
> > * For slab2 120.0 Bytes, data stored if <= 55 B (SUM of 2+52+1 )
> > * For slab3 152.0 Bytes, data stored if <= 87 B (SUM of 2+84+1 )
> > * For slab4 192.0 Bytes, data stored if <= 126 B (SUM of 3+122+1 )
> > * For slab5 240.0 Bytes, data stored if <= 174 B (SUM of 3+170+1 )
> > * etc.
> >
> > My configuration :
> > * Chunk Size : 48
> > * Chunk Growth Factore: 1,25
> > * Max Bytes: 64.0 MBytes
> >
> > So, someone could explain me how the data is stored in the right
> Slabs???
> > How calculate it???
> >
> > Thank you
> >
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "memcached" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send an email to memcached+...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
> >
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to memcached+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
