Re: Multiple consumer connectors in process - lag metrics?

I think ConsumerFetcherManager metric will report data for all of the connectors with the same group.id transparently. Thanks, Neha On Nov 14, 2013 7:48 PM, "Jason Rosenberg" wrote: > Hi, > > We are experimenting with having multiple consumer connectors running in > the same process, under the s

List of topics with JMX?

Hello, Is it possible to see the list of topics in Kafka using JMX like it is possible using the kafka-list-topic.sh command? In addition is it possible to see the number of messages stored for a topic in kafka using JMX? I couldn't find anything about that in the documentation: http://kafka.apa

Re: List of topics with JMX?

Yes you can see them in jconsole but you can even see throughput of partition. But no total throughput I think 发自我的 iPhone > 在 Nov 15, 2013，8:08 AM，"Maier, Dr. Andreas" 写道： > > Hello, > > Is it possible to see the list of topics in Kafka using JMX > like it is possible using the kafka-list-topi

Producer SSL?

Hi, While searching the user group messages I found the following thread - http://grokbase.com/t/kafka/users/138vqq1x07/getting-leadernotavailableexception-in-console-producer-after-increasing-partitions-from-4-to-16. It shows the following stack trace with 0.8. [2013-08-27 08:29:30,372] INFO Fe

Re: Multiple consumer connectors in process - lag metrics?

Hmmm.I'm not sure I see how that will work? If using multiple connectors (and thus multiple ConsumerFetcherManagers?). We were seeing the MaxLag at 0, with multiple connectors/same groupId, even though we were quite sure there was significant lag in some of the connectors Jason On Fri,

Re: Producer SSL?

Hi Jonathan We forked kafka to add SSL feature. It not part of kafka official release Sent from my iPhone On Nov 15, 2013, at 12:32 PM, Jonathan Hodges wrote: > Hi, > > While searching the user group messages I found the following thread - > http://grokbase.com/t/kafka/users/138vqq1x07/getting

Re: Producer SSL?

Is this something you can make available/open source? On Fri, Nov 15, 2013 at 1:03 PM, Rajasekar Elango wrote: > Hi Jonathan > > We forked kafka to add SSL feature. It not part of kafka official release > > Sent from my iPhone > > On Nov 15, 2013, at 12:32 PM, Jonathan Hodges wrote: > > > Hi, >

NodeExists Keeper exception

Hi team, Still this is from beta1. I notice this exception occurred frequently in our broker logs. [2013-11-14 21:09:58,714] INFO Got user-level KeeperException when proce ssing sessionid:0x24250e816b000df type:create cxid:0x26b zxid:0x fffe txntype:unknown reqpath:n/a Error Path:/con

Re: NodeExists Keeper exception

Hello Libo, This error typically indicates failures in consumer rebalances due to multiple consumers try to claim the same partition and gets conflicts. Could you give some more context where you see this exception? Guozhang On Fri, Nov 15, 2013 at 1:23 PM, Yu, Libo wrote: > Hi team, > > Sti

Re: Multiple consumer connectors in process - lag metrics?

Yes I think you are right (i.e., I believe what you're seeing is the value for the last mbean registered) - we should probably allow prefixing metric names with a configurable metric-id to prevent collisions. On Fri, Nov 15, 2013 at 12:52:39PM -0500, Jason Rosenberg wrote: > Hmmm.I'm not sure

Re: Producer SSL?

I echo Jason's question. That would be much easier than using something like stunnel. On Fri, Nov 15, 2013 at 11:41 AM, Jason Rosenberg wrote: > Is this something you can make available/open source? > > > On Fri, Nov 15, 2013 at 1:03 PM, Rajasekar Elango >wrote: > > > Hi Jonathan > > > > We f

Re: Multiple consumer connectors in process - lag metrics?

You could use different client id in those consumers. Our metrics name for maxlag includes the client id. Thanks, Jun On Thu, Nov 14, 2013 at 7:47 PM, Jason Rosenberg wrote: > Hi, > > We are experimenting with having multiple consumer connectors running in > the same process, under the same g