Re: Multiple consumer connectors in process - lag metrics?

2013-11-16 Thread Jason Rosenberg
Ah, that makes sense Jun. By default, the clientid is the same as the groupid? But we can override that for individual connectors sharing the same groupid, I see. Thanks, Jason On Sat, Nov 16, 2013 at 2:18 AM, Jun Rao wrote: > You could use different client id in those consumers. Our metric

Re: Multiple consumer connectors in process - lag metrics?

2013-11-15 Thread Jun Rao
You could use different client id in those consumers. Our metrics name for maxlag includes the client id. Thanks, Jun On Thu, Nov 14, 2013 at 7:47 PM, Jason Rosenberg wrote: > Hi, > > We are experimenting with having multiple consumer connectors running in > the same process, under the same g

Re: Multiple consumer connectors in process - lag metrics?

2013-11-15 Thread Joel Koshy
Yes I think you are right (i.e., I believe what you're seeing is the value for the last mbean registered) - we should probably allow prefixing metric names with a configurable metric-id to prevent collisions. On Fri, Nov 15, 2013 at 12:52:39PM -0500, Jason Rosenberg wrote: > Hmmm.I'm not sure

Re: Multiple consumer connectors in process - lag metrics?

2013-11-15 Thread Jason Rosenberg
Hmmm.I'm not sure I see how that will work? If using multiple connectors (and thus multiple ConsumerFetcherManagers?). We were seeing the MaxLag at 0, with multiple connectors/same groupId, even though we were quite sure there was significant lag in some of the connectors Jason On Fri,

Re: Multiple consumer connectors in process - lag metrics?

2013-11-15 Thread Neha Narkhede
I think ConsumerFetcherManager metric will report data for all of the connectors with the same group.id transparently. Thanks, Neha On Nov 14, 2013 7:48 PM, "Jason Rosenberg" wrote: > Hi, > > We are experimenting with having multiple consumer connectors running in > the same process, under the s

Multiple consumer connectors in process - lag metrics?

2013-11-14 Thread Jason Rosenberg
Hi, We are experimenting with having multiple consumer connectors running in the same process, under the same groupId (but with different topic filters). I'm wondering what the expected effect of this is with metrics, like ConsumerFetcherManager.-MaxLag It looks like in AbstractFetcherManager, t