Hi again, few more questions from me: *1.*
What I see in JMX: kafka.consumer:type="ZookeeperConsumerConnector",name="af_servers-af_servers-spm_new_cluster_topic-af_servers_wawanawna-Dell-1401353748289-fcaaea29-0-FetchQueueSize" >From code: newGauge( config.clientId + "-" + config.groupId + "-" + topicThreadId._1 + "-" + topicThreadId._2 + "-FetchQueueSize", new Gauge[Int] { def value = q.size } ) I've tried to parse part as I've understood they. config.clientId >> af_servers topicThreadId._1 >> af_servers-spm_new_cluster_topic topicThreadId._2 >> af_servers_wawanawna-Dell-1401353748289-fcaaea29-0 Yes I can suppose that this topicThreadId._1 will always looks like GROUP_ID+TOPIC and topicThreadId._2 will contain CONSUMER HOST, but will it always true? *2.* >From code I see that sometimes Kafka uses "_" as separator, not only "-": val consumerIdString = { var consumerUuid : String = null config.consumerId match { case Some(consumerId) // for testing only => consumerUuid = consumerId case None // generate unique consumerId automatically => val uuid = UUID.randomUUID() consumerUuid = "%s-%d-%s".format( InetAddress.getLocalHost.getHostName, System.currentTimeMillis, uuid.getMostSignificantBits().toHexString.substring(0,8)) } config.groupId + "_" + consumerUuid } That means if user will use "_" as part of his host/topic/groupId name it maybe be a problem to parse string like: kafka.consumer:type="ZookeeperConsumerConnector",name="af_servers-af_servers-spm_new_cluster_topic-af_servers_wawanawna-Dell-1401353748289-fcaaea29-0-FetchQueueSize" Look at part: "spm_new_cluster_topic-af_servers_wawanawna-Dell", what is host name here "servers_wawanawna-Dell" or "wawanawna-Dell" ? So from one side if we want to be able parse name without any problems we have to avoid using "-" and "_" in host/topic/groupId/clientId, but at the same time I see (from http://grokbase.com/t/kafka/users/133xfsnpdh/cant-use-in-client-name): *"Client id is used for registering jmx beans for monitoring. Because of the* *restrictions in bean names, we limit the client id to be only alpha-numeric* *plus "-" and "_"."* Does that mean user can use only camelCase in his host/topic/groupId/clientId for distinguish one part of name from another? Is this a problem? Or I didn't understand something? Best regards from Sematext. On Tue, Jun 3, 2014 at 3:24 AM, Otis Gospodnetic <otis.gospodne...@gmail.com > wrote: > Hi Guozhang, > > On Mon, Jun 2, 2014 at 7:18 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > That is indeed a problem, for now, we recommend group name and topic > names > > to use "_" when there is a need for "-", but this should be fixed > > systematically. > > > > Right! > > For you use case, could you change your topic/group name using "_"? > > > Our own Kafka doesn't use topics with "-" characters, so we don't have a > problem. > > The problem, in our case, is that we have a general (Kafka) monitoring tool > that other people use to monitor Kafka - see http://sematext.com/spm/ . > So > we can't really tell people "hey, our tool will work but only if you don't > have a dash in your topic names and hosts and ... because if you use dashes > we won't know how to parse your Kafka's MBean names" :) > > > > Also, do you mind to file a JIRA ticket to keep track of this issue? > > > Here it is: https://issues.apache.org/jira/browse/KAFKA-1481 > > Otis > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > On Mon, Jun 2, 2014 at 5:18 AM, Vladimir Tretyakov < > > vladimir.tretya...@sematext.com> wrote: > > > > > Hello everyone, > > > > > > We are adding Kafka 0.8.x monitoring support to SPM > > > <http://sematext.com/spm/> here at Sematext. Unfortunately, we quickly > > hit > > > an issue caused by the new bean naming convention that embeds things > like > > > topic and host names in the beans along with metrics, separated by > > dashes, > > > making it hard to parse these beans. > > > > > > To put it simply: it is hard/impossible to automatically figure out > which > > > part of the bean name is e.g. consumer group, which is the topic, which > > is > > > the host name, and which is the name of the metric. > > > > > > Let me show you what I mean: > > > > > > kafka.consumer:type="ConsumerTopicMetrics", > > > > > > name="af_servers-spm_topic-BytesPerSec" > > > > > > Here we actually CAN extract: > > > > > > * consumer group ('af_servers') > > > > > > * topic ('spm_topic') > > > > > > * metric (‘BytesPerSec’) > > > > > > BUT what if the consumer group id and/or topic name contain '-'? > > > > > > Then how would we extract consumer group and topic? > > > > > > Here is a concrete example of this problem: > > > > > > kafka.consumer:type="ConsumerTopicMetrics", > > > > > > name="af-servers-spm-topic-BytesPerSec" > > > > > > How can we know what is group id or topic name here? > > > > > > This looks like a problem to me, but maybe I’m missing something? > > > > > > Is it possible to have all these values (group id, topic name) as > > separate > > > attributes inside JMX bean? > > > > > > Or maybe the problem could be solved if a different delimiter was used, > > > such as the pipe (“I”)? > > > > > > It is really needed things and will be nice to have it to build good > tool > > > for monitoring. > > > > > > Thx and best regards from Sematext. > > > > > > > > > > > -- > > -- Guozhang > > >