Nice, thank you very much. Are you ok with :
Example : •Key : test •Data : 90 bytes •Flags : 0 Size : (4+1) + (90+2) + (1+2+4) + 48 + 8 = 160 Bytes => Slab 4 (192 Bytes) Key (characters count) + 1 + Data (characters count) + 2 bytes ( \r\n ) + Header + Chunk Size + CAS Size Header = Flags (characters count) + nData (characters number of DATA count => 90 = 2 characters) + 4 bytes (2 spaces and \r\n) Chunk Size = 48 bytes (default) CAS Size = 8 bytes (platform 64 bits) If CAS disabled: Size : (4+1) + (90+2) + (1+2+4) + 48 = 152 Bytes => Slab 3 (152 Bytes) Sounds good :) Le lundi 16 novembre 2015 23:54:34 UTC+1, Dormando a écrit : > > key + 1b > val + 2b > item struct 48b > [optional CAS] 8b > then the suffix trailer printf: please read the manpage for snprintf: > > snprintf(suffix, 40, " %d %d\r\n", flags, nbytes - 2); > > there are four characters in there. two spaces, and then \r and \n. the > two %d's change into: "0" for flags, and "90" for bytes (ignore the -2 in > there). that's 7b total. > > In your no-cas test: 5 + 92 + 48 + 7 == 152. that's the exact size of slab > 3. (152b). > > On Mon, 16 Nov 2015, Nicolas Martinez wrote: > > > The test was : > > •Key : test > > •Data : 90 bytes > > •Flags : 0 > > > > :( > > > > Le lundi 16 novembre 2015 19:01:50 UTC+1, Nicolas Martinez a écrit : > > Ok... i think i have understood what you say: > > > > Key (Characters Number) + 1 + Data (Characters Number) + Header + > Chunk Size + CAS Size > > > > Header = Flags (Characters Number) + Key (Characters Numbers) > > > > + 2 bytes ( \r\n ) > > > > + 4 bytes (2 spaces and 1 \r) > > > > Chunk Size = 48 bytes (default) > > > > CAS Size = 8 bytes (64 bits platform) > > > > > > Seems to be good: > > > > 4 + 1 + 90 + (1+4+2+4) + 48 + 8 = 162 Bytes => Slab 4 (192 Bytes) > > > > > > But, if i start Memcached with -C, it's wrong. > > > > 4 + 1 + 90 + (1+4+2+4) + 48 = 154 Bytes => Slab 3 (152 Bytes) > > > > > > It must be in Slab4 (192 Bytes) No? > > > > > > Le lundi 16 novembre 2015 15:37:59 UTC+1, Nicolas Martinez a > écrit : > > Thank you very much for yours answers.Ok for CAS... i don't > use -C so i have to add 8 bytes > > > > i still don"t understand these lines : > > >> 2b appended to data_bytes for an extra "\r\n" + > > So, \r\n == 2b ? > > > > >> + 4 bytes for spaces and \r\n > > which spaces? > > What is this \r\n ? Isn't already counted before? > > There is 1 "\r\n" and 1 "\" > > > > echo -e 'set 30 0 3600 30\r\n'$data'\r'| nc ${memcached_server} > 11211 > > > > > > With your example: > > * Key : 2c > > * Val : 28 bytes > > * Flg : 0 (1bytes) > > > > turns into: > > * Key : 3b > > > > => key number characters + 1 > > * Val : 30b > > > > => 28 bytes + 2 bytes ("\r\n") > > * Hdr : 4b + 3b == 7b > > > > => What are 4b? > > => 3b are: flags (1b) + ?? > > * Itm : 56b > > > > => 48b + 8b (CAS) > > => 96b. which is the cap for slab 1 in a default setup. > > > > > > > > Thank you again. > > > > Le lundi 16 novembre 2015 00:31:07 UTC+1, Dormando a écrit : > > Read carefully: > > > > item_size_ok(const size_t nkey, const int flags, const int nbytes) > { > > > > passes: > > > > size_t ntotal = item_make_header(nkey + 1, flags, nbytes, > > prefix, &nsuffix); > > > > Then conditionally: > > > > if (settings.use_cas) { > > ntotal += sizeof(uint64_t); > > } > > > > item_make_header is doing: > > nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags, > nbytes - 2); > > > > Then: > > > > return sizeof(item) + nkey + *nsuffix + nbytes; > > > > It's convoluted but shirt. > > > > the lengths are: > > key + > > 1 + > > data_bytes + > > 2b appended to data_bytes for an extra "\r\n" + > > stringified rep of the flags + data length > > + 4 bytes for spaces and \r\n (these are carriage returns, one > byte each) > > + 8b for CAS if enabled > > > > CAS can be turned off via the -C starttime arg. it takes up 8 > bytes. > > > > Example: > > * Key : 2c > > * Val : 28b > > * Flg : 0 (1b) > > > > turns into: > > * Key : 3b > > * Val : 30b > > * Hdr : 4b + 3b == 7b > > * Itm : 56b > > => 96b. which is the cap for slab 1 in a default setup. > > > > It's tough to get it exact for small chunks due to the way the > header is > > added. You should ballpark or tune the -f value to align with your > > observed data. > > > > On Sun, 15 Nov 2015, Nicolas Martinez wrote: > > > > > Hi, > > > Is CAS always used? > > > If yes, we have to always add 56 bytes to the KEY and VALUE ? > > > you don't count FLAGS characters?? > > > > > > I've found that Flags's size (number of characters) impact the > storage. > > > > > > Example: > > > * Key : 2 characters = 2 bytes > > > * Value : 28 characters = 28 bytes > > > * FLAGS : 1 characters = 1 bytes > > > => 31 bytes > > > > > > seems to take the same storage as > > > * Key : 1 characters = 1 bytes > > > * Value : 28 characters = 28 bytes > > > * FLAGS : 2 characters = 2 bytes > > > => 31 bytes ... wich is the limit to be stored in Slab1 > > > > > > ok for the /r/n ... should take 4 bytes no? > > > > > > So, if we count 56 bytes for CAS : > 56(cas)+31(key+value+flags)+4(/r/n)= 91 > > > > > > Not good... :( > > > > > > where I'm wrong ?? > > > > > > Le samedi 14 novembre 2015 23:55:12 UTC+1, Dormando a écrit : > > > The mysql docs don't speak for the main tree... that's > their own thing. > > > > > > the "sizes" binary that comes with the source tree tells > you how many > > > bytes an item will use (though I intend to add this output > to the 'stats' > > > output somewhere). With CAS this is 56 bytes. > > > > > > 56 + 2 + 30 == 88. Class 1 by default (in 1.4.24) is 96 > bytes, but the > > > item still ends up in class 2. > > > > > > Why is this? (unfortunately?) because memcached > pre-renders part of the > > > text protocol into the item header: > > > > > > *nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", > flags, nbytes - > > > 2); > > > return sizeof(item) + nkey + *nsuffix + nbytes; > > > > > > so the flags + length are getting flattened + \r\n added > to the end. > > > Together that's just enough to push it over the edge. It'd > also be nice to > > > add a highly optimized numerics printf so I could twiddle > options to save > > > a few bytes of memory in objects, but don't get your hopes > up for that > > > happening soon :) > > > > > > On Sat, 14 Nov 2015, Nicolas Martinez wrote: > > > > > > > Add: Memcached version : 1.4.4 (RedHat) > > > > > > > > Le samedi 14 novembre 2015 14:49:37 UTC+1, Nicolas > Martinez a écrit : > > > > Hi, few days i'm reading Memcached documentation > and blogs... and i don't understand how objects are stored. > > > > > > > > My test > > > > > > > > 3 slabs : > > > > > > > > * 96.0 Bytes > > > > * 120.0 Bytes > > > > * 240.0 Bytes > > > > Everywhere, it's told : > > > > * if data is < 96 Bytes, it will be stored in Slabs1 > (96B) > > > > * if data > 96B and < 120B, it will be stored in > Slabs2 (120B) > > > > * if data > 120B, it will be stored in Slabs3 (240B) > > > > * etc. > > > > BUT, for example, when i'm creating an 30B object, it's > stored in Slab2 (120B), and NOT in Slab1 (96B). > > > > > > > > External sources: > > > > For example, the default size for the smallest > block is 88 bytes (40 bytes of value, and the default 48 bytes for the key > and flag data). If the size of the first item you store into the cache is > less than 40 bytes, then a slab with a block size of 88 bytes is created > and the value stored. > > > > => > https://dev.mysql.com/doc/mysql-ha-scalability/en/ha-memcached-using-memory.html > > > > > > > > > > > > > WRONG > > > > > > > > A slab class is a collection of pages divided into > same sized chunks. Each slab class is referenced to by its chunk size. So > we’ll have Slab class 80kb, Slab class 100kb and so on. When an object > needs to be stored, its size determines where it gets stored. So if the > object is larger than 80kb but less than 100kb, it > > gets stored into > > > Slab > > > > class 100kb. > > > > => > http://returnfoo.com/2012/02/memcached-memory-allocation-and-optimization-2/ > > > > > > > > > > > > WRONG > > > > > > > > How i create an object: > > > > > > > > data=$(pwgen 30 -c 1) > > > > echo -e 'set 30 0 3600 30\r\n'$data'\r'| nc > ${memcached_server} 11211 > > > > > > > > > > > > So, when 30B object is creating : > > > > * key name : 30 = 2 bytes > > > > * value: 30 characters = 30 bytes > > > > * tags : 0 = 1 bytes > > > > => All = 33 bytes > > > > If i add the default 48b as explained on Mysql website : > 33 + 48 = 81B ... so < Slab1 (91B)... but always stored in Slab2 (120B) > > > > > > > > So, the size used to store object in the good Slab is > not: > > > > * object value size > > > > * sum of KEY, VALUE and TAGS in bytes > > > > KEY size : 1 character = 1 B > > > > VALUE size : 1 character = 1 B > > > > TAGS size : 1 character = 1 B > > > > > > > > ... as read everywhere > > > > > > > > So, It seems that: (SUM of KEY+VALUE+TAGS ) > > > > * For slab1 96.0 Bytes, data stored if <= 31 B (SUM of > 2+28+1 ) > > > > * For slab2 120.0 Bytes, data stored if <= 55 B (SUM > of 2+52+1 ) > > > > * For slab3 152.0 Bytes, data stored if <= 87 B (SUM > of 2+84+1 ) > > > > * For slab4 192.0 Bytes, data stored if <= 126 B (SUM > of 3+122+1 ) > > > > * For slab5 240.0 Bytes, data stored if <= 174 B (SUM > of 3+170+1 ) > > > > * etc. > > > > > > > > My configuration : > > > > * Chunk Size : 48 > > > > * Chunk Growth Factore: 1,25 > > > > * Max Bytes: 64.0 MBytes > > > > > > > > So, someone could explain me how the data is stored in > the right Slabs??? > > > > How calculate it??? > > > > > > > > Thank you > > > > > > > > > > > > -- > > > > > > > > --- > > > > You received this message because you are subscribed to > the Google Groups "memcached" group. > > > > To unsubscribe from this group and stop receiving emails > from it, send an email to memcached+...@googlegroups.com. > > > > For more options, visit > https://groups.google.com/d/optout. > > > > > > > > > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the > Google Groups "memcached" group. > > > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+...@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+...@googlegroups.com <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > > > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.