Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread Shweta Agrawal
>Your evictions are literally zero, in these stats. You saw them before, when the instances were smaller? Yes we have seen it and it impacted the business as well. >There're a maximum of 63 classes, so making the number smaller has a limited effect. The more slab classes you have, the harder th

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread dormando
> >Also your instance hasn't even malloc'ed half of its memory limit. You  > have over 6 gigabytes unused. There aren't any evictions despite the  > uptime being over two months.  > Was eviction of active items expeted as well? We have eviction of unsed and > unfetched items.  Your evictions are

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread Shweta Agrawal
>you said you were seeing evictions? Was this on a different instance? Yes we had this issue before and therefore we provisioned larger instance to fix that temporarily. But now we want to reduce cost by using instance with less memory. >I don't really have any control or influence over what am

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread dormando
you said you were seeing evictions? Was this on a different instance? I don't really have any control or influence over what amazon deploys for elasticache. They've also changed the daemon. Some of your settings are different from the defaults that 1.5.10 has (automove should default to 1 and hash

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread Shweta Agrawal
yes On Wednesday, July 8, 2020 at 9:35:19 AM UTC+5:30, Dormando wrote: > > Oh, so this is amazon elasticache? > > On Tue, 7 Jul 2020, Shweta Agrawal wrote: > > > We use aws for deployment and don't have that information. What > particularly looks odd in settings? > > > > On Wednesday, July 8

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread dormando
Oh, so this is amazon elasticache? On Tue, 7 Jul 2020, Shweta Agrawal wrote: > We use aws for deployment and don't have that information. What particularly > looks odd in settings?  > > On Wednesday, July 8, 2020 at 8:10:04 AM UTC+5:30, Dormando wrote: > what're your start arguments? the s

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread Shweta Agrawal
We use aws for deployment and don't have that information. What particularly looks odd in settings? On Wednesday, July 8, 2020 at 8:10:04 AM UTC+5:30, Dormando wrote: > > what're your start arguments? the settings look a little odd. ie; the full > commandline (censoring anything important) that

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread dormando
what're your start arguments? the settings look a little odd. ie; the full commandline (censoring anything important) that you used to start memcached On Tue, 7 Jul 2020, Shweta Agrawal wrote: > Sorry. Here it is. > > On Wednesday, July 8, 2020 at 12:38:38 AM UTC+5:30, Dormando wrote: > 'st

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread Shweta Agrawal
Sorry. Here it is. On Wednesday, July 8, 2020 at 12:38:38 AM UTC+5:30, Dormando wrote: > > 'stats settings' file is empty > > On Tue, 7 Jul 2020, Shweta Agrawal wrote: > > > Hi Dormando, > > Got the stats for production. Please find attached files for stats > settings. stats items, stats, stat

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-07 Thread dormando
'stats settings' file is empty On Tue, 7 Jul 2020, Shweta Agrawal wrote: > Hi Dormando, > Got the stats for production. Please find attached files for stats settings. > stats items, stats, stats slabs. Summary for all slabs. > > Other details that might help: > * TTL is two days or more.  > *

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-05 Thread Shweta Agrawal
Got it. Thank you for the explanation :) On Sunday, July 5, 2020 at 9:10:23 AM UTC+5:30, Dormando wrote: > > On Sat, 4 Jul 2020, Shweta Agrawal wrote: > > > > the rest is free memory, which should be measured separately. > > free memory for a class will be : (free_chunks * chunk_size) > > And

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-04 Thread dormando
On Sat, 4 Jul 2020, Shweta Agrawal wrote: > > the rest is free memory, which should be measured separately. > free memory for a class will be : (free_chunks * chunk_size)  > And total memory reserved by a class will be : (total_pages*page_size) > > > If you're getting evictions in class A but ther

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-04 Thread dormando
(memory_requested / (chunk_size * chunk_used)) * 100 is roughly the storage overhead of memory used in the system. the rest is free memory, which should be measured separately. If you're getting evictions in class A but there's too much free memory in classes C, D, etc then you have a balance issu

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-04 Thread Shweta Agrawal
> I'll need the raw output from "stats items" and "stats slabs". I don't > think that efficiency column is very helpful. ohkay no worries. I can get by Tuesday and will share. Efficiency for each slab is calcuated as (("stats slabs" -> memory_requested) / (("stats slabs" -> total_pages) * pa

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-04 Thread dormando
ah okay. I'll need the raw output from "stats items" and "stats slabs". I don't think that efficiency column is very helpful. On Fri, 3 Jul 2020, Shweta Agrawal wrote: > > > On Saturday, July 4, 2020 at 9:41:49 AM UTC+5:30, Dormando wrote: > No attachment > > On Fri, 3 Jul 2020, Shwe

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-03 Thread Shweta Agrawal
On Saturday, July 4, 2020 at 9:41:49 AM UTC+5:30, Dormando wrote: > > No attachment > > On Fri, 3 Jul 2020, Shweta Agrawal wrote: > > > > > Wooo...so quick. :):) > > > Correct, close. It actually uses more like 3 512k chunks and then one > > > smaller chunk from a different class to fit exa

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-03 Thread dormando
No attachment On Fri, 3 Jul 2020, Shweta Agrawal wrote: > > Wooo...so quick. :):) > > Correct, close. It actually uses more like 3 512k chunks and then one  > > smaller chunk from a different class to fit exactly 1.6MB.  > I see.Got it. > > >Can you share snapshots from "stats items" and "stats s

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-03 Thread Shweta Agrawal
Sorry forgot to mention. summary is from one instance. Instance has 13 GB of RAM On Saturday, July 4, 2020 at 9:22:13 AM UTC+5:30, Shweta Agrawal wrote: > > > Wooo...so quick. :):) > > > Correct, close. It actually uses more like 3 512k chunks and then one > > smaller chunk from a different clas

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-03 Thread dormando
Hey, > Putting my understanding to re-confirm: > 1) Page size will always be 1MB and we cannot change it.Moreover, it's not > required to be changed. Correct. > 2) We can store items larger than 1MB and it is done by combining chunks > together. (example: let's say item size: ~1.6MB --> 4 slab

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-03 Thread Shweta Agrawal
Hi Dormando, Thanks a lot for the quick and promt reply. *Putting my understanding to re-confirm:* 1) Page size will always be 1MB and we cannot change it.Moreover, it's not required to be changed. 2) We can store items larger than 1MB and it is done by combining chunks together. (example: let

Re: Total memory allocated with -m NOT EQUAL to (total pages * max_item_size). Request to provide clarification on how page size works.

2020-07-03 Thread dormando
Hey, Looks like I never updated the manpage. In the past the item size max was achieved by changing the slab page size, but that hasn't been true for a long time. >From ./memcached -h: -m, --memory-limit= item memory in megabytes (default: 64) ... -m just means the memory limit in megabytes, ab