Re: Curiosity killed the `stats cachedump`

2011-08-19 Thread Neil Mckee
On Aug 18, 2011, at 10:48 PM, dormando wrote: 1) Sampling useful data out of a cluster. 2) Providing something useful for application developers The second case is an OS X user who fires up memcached locally, writes some rails code, then wonders what's going on under the hood. 1-in-1000

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
On a positive note, it does seem like there is some consensus on the value of random-transaction-sampling here. But do we have agreement that this feed should be made available for external consumption (i.e. the whole cluster sends to one place that is not itself a memcached node),

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
Although there are already 30+ companies and open-source projects with sFlow collectors I fully expect most memcached users will write their own collection-and-analysis tools once they can get this data! Don't you agree? So it's not about any one collector, it's about defining a

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread Neil Mckee
On Aug 18, 2011, at 5:19 PM, dormando wrote: On a positive note, it does seem like there is some consensus on the value of random-transaction-sampling here. But do we have agreement that this feed should be made available for external consumption (i.e. the whole cluster sends to one

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
Not critical at all. The duration_uS field can be set to -1 in the XDR output to indicate that it is not implemented. I added this measurement when porting to the 1.6 branch, where it makes more sense. I left it in when I updated the 1.4 branch because, well, the overhead seemed negligible

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread Neil Mckee
On Aug 18, 2011, at 5:25 PM, dormando wrote: Although there are already 30+ companies and open-source projects with sFlow collectors I fully expect most memcached users will write their own collection-and-analysis tools once they can get this data! Don't you agree? So it's not about

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
1) Sampling useful data out of a cluster. 2) Providing something useful for application developers The second case is an OS X user who fires up memcached locally, writes some rails code, then wonders what's going on under the hood. 1-in-1000 sampling there is counterproductive.

Re: Curiosity killed the `stats cachedump`

2011-08-09 Thread Dustin
On Aug 8, 10:00 pm, neilmckee neil.mckee...@gmail.com wrote:   Well, all the memcached operations are built on top of it... do you mean specifically multiget might call into the engine multiple times for a single request? Yes.  That's one example.  I think there were others where the

Re: Curiosity killed the `stats cachedump`

2011-08-09 Thread Neil Mckee
On Aug 9, 2011, at 1:50 PM, Dustin wrote: On Aug 8, 10:00 pm, neilmckee neil.mckee...@gmail.com wrote: Well, all the memcached operations are built on top of it... do you mean specifically multiget might call into the engine multiple times for a single request? Yes. That's one

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread Neil Mckee
I think it's clearer if we separate the requirements: (A) continuous monitoring of the whole cluster, in production. (B) troubleshooting a specific node, key, operation or client -- without impacting (A) For (A) you want the most robust, scale-out measurement you can find that will not impact

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread Neil Mckee
I looked pretty hard at the shim idea back in May, but the engine protocol is really a different protocol. There was not a 1:1 correspondence with the standard memcached operations. If we define a standard sFlow-MEMCACHE measurement then it should be something that any memcache daemon can

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread Dustin
On Aug 8, 11:47 am, Neil Mckee neil.mckee...@gmail.com wrote: I looked pretty hard at the shim idea back in May,  but the engine protocol is really a different protocol.  There was not a 1:1 correspondence with the standard memcached operations. Well, all the memcached operations are

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread neilmckee
On Aug 8, 5:52 pm, Dustin dsalli...@gmail.com wrote: On Aug 8, 11:47 am, Neil Mckee neil.mckee...@gmail.com wrote: I looked pretty hard at the shim idea back in May,  but the engine protocol is really a different protocol.  There was not a 1:1 correspondence with the standard memcached

Re: Curiosity killed the `stats cachedump`

2011-08-07 Thread Peter Portante
How 'bout random sample request profiling? The Alpha processor used to do this (still does if you are using EV6 or later), called ProfileMe: Alpha 21264A processors (and later) use a different method called instruction sampling. PC sampling on out-of-order execution engines like the Alpha 21264

RE: Curiosity killed the `stats cachedump`

2011-08-07 Thread Andrew O'Brien
From: memcached@googlegroups.com [mailto:memcached@googlegroups.com] On Behalf Of Peter Portante Sent: Monday, 8 August 2011 10:49 AM How 'bout random sample request profiling? Profiling for monitoring and activity estimation purposes - isn't that the point of the sFlow set of patches

RE: Curiosity killed the `stats cachedump`

2011-08-07 Thread dormando
From: memcached@googlegroups.com [mailto:memcached@googlegroups.com] On Behalf Of Peter Portante Sent: Monday, 8 August 2011 10:49 AM How 'bout random sample request profiling? Profiling for monitoring and activity estimation purposes - isn't that the point of the sFlow set of

Re: Curiosity killed the `stats cachedump`

2011-08-07 Thread Dustin
On Aug 7, 6:01 pm, Andrew O'Brien andr...@oriel.com.au wrote: Profiling for monitoring and activity estimation purposes - isn't that the point of the sFlow set of patches mentioned a few times on list? My opinion on the sFlow patches has been that it shouldn't be a change to memcached.

Re: Curiosity killed the `stats cachedump`

2011-07-31 Thread dormando
I owe all of you better tap documentation (the last couple of weeks have really killed me). It does some pretty great stuff in this area and has many practical uses. Now would be a great time to sell us on it, then :)