Curiosity killed the `stats cachedump`

2011-07-31 Thread dormando
Yo, We've threatened to kill the `stats cachedump` command for probably five years. I've daydreamed about randomizing the command name on every minor release, every git push, ensuring that it stays around as a last ditch debugging tool. A lot of you continue to build programs which rely on stats

Re: Curiosity killed the `stats cachedump`

2011-07-31 Thread Dustin
On Jul 31, 12:03 pm, dormando wrote: > Yo, > > We've threatened to kill the `stats cachedump` command for probably five > years. I've daydreamed about randomizing the command name on every minor > release, every git push, ensuring that it stays around as a last ditch > debugging tool. I remain

Re: Curiosity killed the `stats cachedump`

2011-07-31 Thread dormando
> I owe all of you better tap documentation (the last couple of weeks > have really killed me). It does some pretty great stuff in this area > and has many practical uses. Now would be a great time to sell us on it, then :)

Re: Curiosity killed the `stats cachedump`

2011-08-07 Thread Peter Portante
How 'bout random sample request profiling? The Alpha processor used to do this (still does if you are using EV6 or later), called ProfileMe: Alpha 21264A processors (and later) use a different method called "instruction sampling." PC sampling on out-of-order execution engines like the Alpha 21264

RE: Curiosity killed the `stats cachedump`

2011-08-07 Thread Andrew O'Brien
> From: memcached@googlegroups.com [mailto:memcached@googlegroups.com] On > Behalf Of Peter Portante > Sent: Monday, 8 August 2011 10:49 AM > > How 'bout random sample request profiling? Profiling for monitoring and activity estimation purposes - isn't that the point of the sFlow set of patches

RE: Curiosity killed the `stats cachedump`

2011-08-07 Thread dormando
> > From: memcached@googlegroups.com [mailto:memcached@googlegroups.com] On > > Behalf Of Peter Portante > > Sent: Monday, 8 August 2011 10:49 AM > > > > How 'bout random sample request profiling? > > Profiling for monitoring and activity estimation purposes - isn't that the > point of the sFlow s

Re: Curiosity killed the `stats cachedump`

2011-08-07 Thread Dustin
On Aug 7, 6:01 pm, Andrew O'Brien wrote: > Profiling for monitoring and activity estimation purposes - isn't that the > point of the sFlow set of patches mentioned a few times on list? My opinion on the sFlow patches has been that it shouldn't be a change to memcached. What can we do to m

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread Neil Mckee
I think it's clearer if we separate the requirements: (A) continuous monitoring of the whole cluster, in production. (B) troubleshooting a specific node, key, operation or client -- without impacting (A) For (A) you want the most robust, scale-out measurement you can find that will not impact

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread Neil Mckee
I looked pretty hard at the shim idea back in May, but the engine protocol is really a different protocol. There was not a 1:1 correspondence with the standard memcached operations. If we define a standard sFlow-MEMCACHE measurement then it should be something that any memcache daemon can ex

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread Dustin
On Aug 8, 11:47 am, Neil Mckee wrote: > I looked pretty hard at the shim idea back in May,  but the engine protocol > is really a different protocol.  There was not a 1:1 correspondence with the > standard memcached operations. Well, all the memcached operations are built on top of it... do

Re: Curiosity killed the `stats cachedump`

2011-08-08 Thread neilmckee
On Aug 8, 5:52 pm, Dustin wrote: > On Aug 8, 11:47 am, Neil Mckee wrote: > > > I looked pretty hard at the shim idea back in May,  but the engine protocol > > is really a different protocol.  There was not a 1:1 correspondence with > > the standard memcached operations. > >   Well, all the me

Re: Curiosity killed the `stats cachedump`

2011-08-09 Thread Dustin
On Aug 8, 10:00 pm, neilmckee wrote: > >   Well, all the memcached operations are built on top of it... do you > > mean specifically multiget might call into the engine multiple times > > for a single "request"? > > Yes.  That's one example.  I think there were others where the > memcache operat

Re: Curiosity killed the `stats cachedump`

2011-08-09 Thread Neil Mckee
On Aug 9, 2011, at 1:50 PM, Dustin wrote: > > On Aug 8, 10:00 pm, neilmckee wrote: > >>> Well, all the memcached operations are built on top of it... do you >>> mean specifically multiget might call into the engine multiple times >>> for a single "request"? >> >> Yes. That's one example. I

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
> > > >> On a positive note, it does seem like there is some consensus on the > >> value of random-transaction-sampling here. But do we have agreement > >> that this feed should be made available for external consumption (i.e. > >> the whole cluster sends to one place that is not itself a memcac

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
> > Although there are already 30+ companies and open-source projects with > sFlow collectors I fully expect most memcached users will write their > own collection-and-analysis tools once they can get this data! Don't > you agree? So it's not about any one collector, it's about > defining a u

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread Neil Mckee
On Aug 18, 2011, at 5:19 PM, dormando wrote: >>> On a positive note, it does seem like there is some consensus on the value of random-transaction-sampling here. But do we have agreement that this feed should be made available for external consumption (i.e. the whole cluste

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
> Not critical at all. The duration_uS field can be set to -1 in the XDR > output to indicate that it is not implemented. I added this measurement > when porting to the 1.6 branch, where it makes more sense. I left it in > when I updated the 1.4 branch because, well, the overhead seemed > neglig

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread Neil Mckee
On Aug 18, 2011, at 5:25 PM, dormando wrote: >> >> Although there are already 30+ companies and open-source projects with >> sFlow collectors I fully expect most memcached users will write their >> own collection-and-analysis tools once they can get this data! Don't >> you agree? So it's not

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread dormando
> > 1) Sampling useful data out of a cluster. > > > > 2) Providing something useful for application developers > > > > The second case is an OS X user who fires up memcached locally, writes > > some rails code, then wonders what's going on under the hood. 1-in-1000 > > sampling there is counterprod

Re: Curiosity killed the `stats cachedump`

2011-08-18 Thread Neil Mckee
On Aug 18, 2011, at 10:48 PM, dormando wrote: >>> 1) Sampling useful data out of a cluster. >>> >>> 2) Providing something useful for application developers >>> >>> The second case is an OS X user who fires up memcached locally, writes >>> some rails code, then wonders what's going on under the

Re: Curiosity killed the `stats cachedump`

2011-08-19 Thread dormando
> >> So there's no need to hesitate if you can already do (1) today. Let's > >> face it, you have been very successful and there are rather a lot of > >> users who have already gotten past (2) :) > > > > Okay, I'm kinda tired of that argument. Just beacuse you say something > > isn't possible, doe

Re: Curiosity killed the `stats cachedump`

2011-08-19 Thread Neil Mckee
On Aug 19, 2011, at 12:56 AM, dormando wrote: So there's no need to hesitate if you can already do (1) today. Let's face it, you have been very successful and there are rather a lot of users who have already gotten past (2) :) >>> >>> Okay, I'm kinda tired of that argument. Just