+1 as well for getting rid of the quotes. Isn't it an inheritance of metrics-core? Anyway, hope Kafka devs will find a way to get rid of these!
Cordialement / Best Regards, Alexandre VERMEERBERGEN R&D Cloud Service Supervision Development Director ––––––––––––––––––––––– Office: +33 1 6162 4992 alexandre.vermeerber...@3ds.com http://www.3ds.com ––––––––––––––––––––––– Dassault Systèmes | 10 rue Marcel Dassault, CS 40501 | 78946 Vélizy-Villacoublay Cedex | France -----Original Message----- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Friday, October 17, 2014 22:49 To: users@kafka.apache.org Subject: Re: MBeans, dashes, underscores, and KAFKA-1481 +1 on getting rid of the quotes. On Fri, Oct 17, 2014 at 12:31 PM, Magnus Spångdal < magnus.spang...@deltaprojects.com> wrote: > +1 to get rid of quotes, thanks! > > > > > > > — > Sent from Mailbox > > On Fri, Oct 17, 2014 at 8:54 PM, Jun Rao <jun...@gmail.com> wrote: > > > Hi, everyone, > > We are fixing the mbean names in kafka-1482, by adding separate > > explicit tags in the name for things like clientId and topic. > > Another thing that some people have complained before is that we use quotes > > in the jmx name. > > Should we also just get rid of the quotes as part of kafka-1482? So, > > instead of > > "kafka.server":type="BrokerTopicMetrics",name="topic-1-BytesInPerSec" > > we will have > > > > kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=topic- > > 1 > > Thanks, > > Jun > > On Thu, Oct 9, 2014 at 11:12 AM, Neha Narkhede > > <neha.narkh...@gmail.com> > > wrote: > >> I am going to vote for 1482 to be included in 0.8.2, if we have a > >> patch submitted in a week. I think we've had this JIRA opened for > >> too long > and we > >> held people back so it's only fair to release this. > >> > >> On Wed, Oct 8, 2014 at 9:40 PM, Jun Rao <jun...@gmail.com> wrote: > >> > >> > Otis, > >> > > >> > Just have the patch ready asap. We can make a call then. > >> > > >> > Thanks, > >> > > >> > Jun > >> > > >> > On Wed, Oct 8, 2014 at 6:13 AM, Otis Gospodnetic < > >> > otis.gospodne...@gmail.com > >> > > wrote: > >> > > >> > > Hi Jun, > >> > > > >> > > Would by the end of next week be acceptable for 0.8.2? > >> > > > >> > > Thanks, > >> > > Otis > >> > > -- > >> > > Monitoring * Alerting * Anomaly Detection * Centralized Log > Management > >> > > Solr & Elasticsearch Support * http://sematext.com/ > >> > > > >> > > > >> > > On Tue, Oct 7, 2014 at 4:04 PM, Jun Rao <jun...@gmail.com> wrote: > >> > > > >> > > > Otis, > >> > > > > >> > > > Yes, if you guys can help provide a patch in a few days, we > >> > > > can > >> > probably > >> > > > get it to the 0.8.2 release. > >> > > > > >> > > > Thanks, > >> > > > > >> > > > Jun > >> > > > > >> > > > On Tue, Oct 7, 2014 at 12:10 PM, Otis Gospodnetic < > >> > > > otis.gospodne...@gmail.com> wrote: > >> > > > > >> > > > > Hi Jun, > >> > > > > > >> > > > > I think your MBean renaming approach will work. I see > >> > > > > https://issues.apache.org/jira/browse/KAFKA-1481 has Fix > Version > >> > > 0.8.2, > >> > > > > but > >> > > > > is not marked as a Blocker. We'd love to get the MBeans > >> > > > > fixed > so > >> > this > >> > > > > makes it in 0.8.2 release. Do you know if this is on > >> > > > > anyone's > >> plate > >> > > (the > >> > > > > issue is currently Unassigned)? If not, should we provide > >> > > > > a new > >> > patch > >> > > > that > >> > > > > uses your approach? > >> > > > > > >> > > > > Thanks, > >> > > > > Otis > >> > > > > -- > >> > > > > Monitoring * Alerting * Anomaly Detection * Centralized Log > >> > Management > >> > > > > Solr & Elasticsearch Support * http://sematext.com/ > >> > > > > > >> > > > > > >> > > > > On Thu, Sep 18, 2014 at 4:49 PM, Jun Rao <jun...@gmail.com> > wrote: > >> > > > > > >> > > > > > Otis, > >> > > > > > > >> > > > > > In kafka-1481, we will have to change the mbean names (at > least > >> the > >> > > > ones > >> > > > > > with clientid and topic) anyway. Using the name/value > >> > > > > > pair in > the > >> > > mbean > >> > > > > > name allows us to do this in a cleaner way. Yes, "," is > >> > > > > > not > >> allowed > >> > > in > >> > > > > > clientid or topic. > >> > > > > > > >> > > > > > Bhavesh, > >> > > > > > > >> > > > > > Yes, I was thinking of making changes in the new metrics > package. > >> > > > > Something > >> > > > > > like allowing the sensor names to have name/value pairs. > >> > > > > > The > jmx > >> > > names > >> > > > > will > >> > > > > > just follow accordingly. This is probably cleaner than > >> > > > > > doing > the > >> > > > > escaping. > >> > > > > > Also, the metric names are more intuitive (otherwise, you > have to > >> > > know > >> > > > > > which part is the clientid and which part is the topic). > >> > > > > > > >> > > > > > Thanks, > >> > > > > > > >> > > > > > Jun > >> > > > > > > >> > > > > > On Wed, Sep 17, 2014 at 2:32 PM, Otis Gospodnetic < > >> > > > > > otis.gospodne...@gmail.com> wrote: > >> > > > > > > >> > > > > > > Hi Jun, > >> > > > > > > > >> > > > > > > On Wed, Sep 17, 2014 at 12:35 PM, Jun Rao > >> > > > > > > <jun...@gmail.com > > > >> > > wrote: > >> > > > > > > > >> > > > > > > > Bhavesh, > >> > > > > > > > > >> > > > > > > > Yes, allowing dot in clientId and topic makes it a > >> > > > > > > > bit > harder > >> > to > >> > > > > define > >> > > > > > > the > >> > > > > > > > JMX bean names. I see a couple of solutions here. > >> > > > > > > > > >> > > > > > > > 1. Disable dot in clientId and topic names. The issue > >> > > > > > > > is > that > >> > dot > >> > > > may > >> > > > > > > > already be used in existing deployment. > >> > > > > > > > > >> > > > > > > > 2. We can represent the JMX bean name differently in > >> > > > > > > > the > new > >> > > > > producer. > >> > > > > > > > Instead of > >> > > > > > > > kafka.producer.myclientid:type=mytopic > >> > > > > > > > we could change it to > >> > > > > > > > kafka.producer:clientId=myclientid,topic=mytopic > >> > > > > > > > > >> > > > > > > > I felt that option 2 is probably better since it > >> > > > > > > > doesn't > >> affect > >> > > > > > existing > >> > > > > > > > users. > >> > > > > > > > > >> > > > > > > > >> > > > > > > If it doesn't affect existing users, great. > >> > > > > > > > >> > > > > > > If you are saying that each "piece" of MBean name could > >> > > > > > > be > >> > > expressed > >> > > > as > >> > > > > > > name=value pair, with something like "," (forbidden in > >> > > > > > > host > >> > names, > >> > > > > topic > >> > > > > > > names, client IDs, etc. I assume?) then yes, I think > >> > > > > > > this > would > >> > be > >> > > > > easier > >> > > > > > > to parse and it would be easier for people to > >> > > > > > > understand > what > >> is > >> > > > what. > >> > > > > > > > >> > > > > > > Otis > >> > > > > > > -- > >> > > > > > > Monitoring * Alerting * Anomaly Detection * Centralized > >> > > > > > > Log > >> > > > Management > >> > > > > > > Solr & Elasticsearch Support * http://sematext.com/ > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > Otis, > >> > > > > > > > > >> > > > > > > > We probably can also use option 2 to address KAFKA-1481. > For > >> > > > > > > topic/clientid > >> > > > > > > > specific metrics, we could explicitly specify the > >> > > > > > > > metric > name > >> > so > >> > > > that > >> > > > > > it > >> > > > > > > > contains "topic=mytopic,clientid=myclientid". That > >> > > > > > > > seems > to > >> be > >> > a > >> > > > much > >> > > > > > > > cleaner way than having all parts included in a > >> > > > > > > > single > string > >> > > > > separated > >> > > > > > > by > >> > > > > > > > '|'. > >> > > > > > > > > >> > > > > > > > Thanks, > >> > > > > > > > > >> > > > > > > > Jun > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > On Tue, Sep 16, 2014 at 5:15 PM, Bhavesh Mistry < > >> > > > > > > > mistry.p.bhav...@gmail.com> > >> > > > > > > > wrote: > >> > > > > > > > > >> > > > > > > > > HI Otis, > >> > > > > > > > > > >> > > > > > > > > What is migration path ? If topic with special > >> > > > > > > > > chars > >> exists > >> > > > > already( > >> > > > > > > > > ".","-","|" etc) in previous version of > producer/consumer > >> of > >> > > > > Kafka, > >> > > > > > > what > >> > > > > > > > > happens after the upgrade new producer or consumer > (kafka > >> > > > version) > >> > > > > ? > >> > > > > > > > Also, > >> > > > > > > > > in new producer API (Kafka Trunk), does this > >> > > > > > > > > enforce the > >> rule > >> > > > about > >> > > > > > > > client > >> > > > > > > > > id as well ? > >> > > > > > > > > > >> > > > > > > > > Thanks, > >> > > > > > > > > > >> > > > > > > > > Bhavesh > >> > > > > > > > > > >> > > > > > > > > On Tue, Sep 16, 2014 at 2:09 PM, Otis Gospodnetic < > >> > > > > > > > > otis.gospodne...@gmail.com> wrote: > >> > > > > > > > > > >> > > > > > > > > > Hi, > >> > > > > > > > > > > >> > > > > > > > > > So maybe I should I should have asked the Q > explicitly: > >> > > > > > > > > > Could we commit the patch from > >> > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-1481 > >> > > > > > > > > > now > >> > that, I > >> > > > > hope, > >> > > > > > > > it's > >> > > > > > > > > > clear what problems the current MBean names can cause? > >> > > > > > > > > > > >> > > > > > > > > > Thanks, > >> > > > > > > > > > Otis > >> > > > > > > > > > -- > >> > > > > > > > > > Monitoring * Alerting * Anomaly Detection * > Centralized > >> Log > >> > > > > > > Management > >> > > > > > > > > > Solr & Elasticsearch Support * > >> > > > > > > > > > http://sematext.com/ > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > On Mon, Sep 15, 2014 at 10:40 PM, Otis > >> > > > > > > > > > Gospodnetic < otis.gospodne...@gmail.com> wrote: > >> > > > > > > > > > > >> > > > > > > > > > > Hi, > >> > > > > > > > > > > > >> > > > > > > > > > > *Problem:* > >> > > > > > > > > > > Some Kafka 0.8.x MBeans have names composed of > things > >> > like > >> > > > > > > <consumer > >> > > > > > > > > > > group>-<topic>-<metric name>. Note how dashes > >> > > > > > > > > > > group>are > used > >> > as > >> > > > > > > > delimiters. > >> > > > > > > > > > > When <consumer group> and <topic> don't > >> > > > > > > > > > > contain the > >> > > > delimiter > >> > > > > > > > > character > >> > > > > > > > > > > all is good if you want to extract parts of > >> > > > > > > > > > > this > MBean > >> > name > >> > > > by > >> > > > > > > simply > >> > > > > > > > > > > splitting on the delimiter character. The > >> > > > > > > > > > > problem > is > >> > that > >> > > > > dashes > >> > > > > > > are > >> > > > > > > > > > > allowed in topic and group names, so this > >> > > > > > > > > > > splitting > >> > doesn't > >> > > > > work. > >> > > > > > > > > > > Moreover, underscores are also used as > >> > > > > > > > > > > delimiters, > and > >> > they > >> > > > can > >> > > > > > > also > >> > > > > > > > be > >> > > > > > > > > > > used in things like topic names. > >> > > > > > > > > > > > >> > > > > > > > > > > *Example*: > >> > > > > > > > > > > This MBean's name is composed of <consumer > >> > > > > > > > group>-<topic>-BytesPerSec: > >> > > > > > > > > > > > >> > > > > > > > > > > kafka.consumer:type="ConsumerTopicMetrics", > >> > > > > > > > > name="*myGroup**-myTopic**-* > >> > > > > > > > > > > BytesPerSec" > >> > > > > > > > > > > > >> > > > > > > > > > > Here we can actually split on "-" and extract > >> > > > > > > > > > > all 3 > >> parts > >> > > > from > >> > > > > > the > >> > > > > > > > > MBean > >> > > > > > > > > > > name:: > >> > > > > > > > > > > * consumer group ('*myGroup*') > >> > > > > > > > > > > * topic ('*myTopic*') > >> > > > > > > > > > > * metric (‘BytesPerSec’) > >> > > > > > > > > > > > >> > > > > > > > > > > All good! > >> > > > > > > > > > > > >> > > > > > > > > > > But imagine if I named the group: *my-Group* > >> > > > > > > > > > > And if I named the topic: *my-Topic* > >> > > > > > > > > > > > >> > > > > > > > > > > Then we'd have: > >> > > > > > > > > > > kafka.consumer:type="ConsumerTopicMetrics", > >> > > > > > > > > > name="*my-Group**-my-Topic**-* > >> > > > > > > > > > > BytesPerSec" > >> > > > > > > > > > > > >> > > > > > > > > > > Now splitting on "-" would no longer work! To > extract > >> > > > > "my-Group" > >> > > > > > > and > >> > > > > > > > > > > "my-Topic" and "BytesPerSec" parts I would have > >> > > > > > > > > > > to > know > >> > the > >> > > > > > > specific > >> > > > > > > > > > group > >> > > > > > > > > > > name and topic name to look for and could not > >> > > > > > > > > > > use > >> generic > >> > > > > > approach > >> > > > > > > of > >> > > > > > > > > > just > >> > > > > > > > > > > splitting the MBean name on the delimiter. > >> > > > > > > > > > > > >> > > > > > > > > > > *Solution*: > >> > > > > > > > > > > The patch in > >> > > > https://issues.apache.org/jira/browse/KAFKA-1481 > >> > > > > > > > replaces > >> > > > > > > > > > > all _ and - characters where they are used as > >> delimiters > >> > in > >> > > > > MBean > >> > > > > > > > names > >> > > > > > > > > > > with a "|" character. Because the "I" > >> > > > > > > > > > > character is > not > >> > > > allowed > >> > > > > > in > >> > > > > > > > > topic > >> > > > > > > > > > > names, consumer groups, host names, splitting > >> > > > > > > > > > > on > this > >> new > >> > > and > >> > > > > > > unified > >> > > > > > > > > > > delimiter works. > >> > > > > > > > > > > > >> > > > > > > > > > > I hope this explains the problem, the solution, > >> > > > > > > > > > > and > >> that > >> > > this > >> > > > > can > >> > > > > > > > make > >> > > > > > > > > it > >> > > > > > > > > > > in the next 0.8.x. > >> > > > > > > > > > > > >> > > > > > > > > > > Otis > >> > > > > > > > > > > -- > >> > > > > > > > > > > Monitoring * Alerting * Anomaly Detection * > Centralized > >> > Log > >> > > > > > > > Management > >> > > > > > > > > > > Solr & Elasticsearch Support * > >> > > > > > > > > > > http://sematext.com/ > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email. For other languages, go to http://www.3ds.com/terms/email-disclaimer
