Thanks Xavier for the KIP, I think it is a great idea to add to AK. A couple of questions / comments:
1. I do feel there're similar needs for clients make JMX configurable. Some context: in modules like Connect and Streams we have added / refactored a large number of metrics so far [0, 1], and although we've added a reporting level config [2] to clients, this is statically defined at code and cannot be dynamically changed either. [0] https://cwiki.apache.org/confluence/display/KAFKA/KIP-196%3A+Add+metrics+to+Kafka+Connect+framework [1] https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams [2] https://cwiki.apache.org/confluence/display/KAFKA/KIP-105%3A+Addition+of+Recording+Level+for+Sensors 2. This may be out of the scope of this KIP, but have you thought about how to make the metrics collection to be configurable (i.e. basically for those metrics which we know would not be exposed, we do not collect them either) dynamically? Again this is related to our previous approach in [2] but it is static and also quite coarse-grained: you can only turn on ALL debug-level sensors or none of them, for example. For some sensors the collection itself can actually be quite expensive since it is either on critical code path, or it is relying on third-party library calls. If we can dynamically change what metrics we would collect at runtime that would be great. Guozhang On Wed, Nov 6, 2019 at 3:42 PM Xavier Léauté <xav...@confluent.io> wrote: > > > > Since these configs will work with Kafka's own metrics library, will the > > configs be part of the clients' configurations? It would be good to point > > that out explicitly in the KIP. > > > > Those configs are currently only at the broker level. If we feel this is > useful on the client as well, we could submit a similar KIP for the client > side. > I am not sure if the number of metrics is as problematic on the client side > as on the server though. > > > > Would the regex apply to the whole string? i.e would we be able to match > > parts of the string like `type=`, `name=`, `topic=`, or would it only > apply > > to the values? > > > > Yes, to keep things simple I decided to apply the regex to the entire JMX > mbean string, since that's typically how a user would refer to those > metrics, regardless of whether it came from KafkaMetrics or YammerMetrics > > I've updated the KIP to add some example filters > -- -- Guozhang