Hi, James. I was mistaken about how the Kafka metrics are converted to
MBeans and attributes. The MBean is constructed from the group and tags,
and the metrics show up as attributes on the MBean. I'll update the KIP to
reflect this.
On Tue, Sep 12, 2017 at 1:43 AM, James Cheng <wushuja...@gmail.com> wrote:
> Thanks for the KIP, Randall.
>
> The KIP has one MBean per metric name. Can I suggest an alternate grouping?
>
> kafka.connect:type=connector-metrics,connector=([-.\w]+)
> connector-type
> connector-class
> connector-version
> status
>
> kafka.connect:type=task-metrics,connector=([-.\w]+),task=([\d]+)
> status
> pause-ratio
> offset-commit-success-percentage
> offset-commit-failure-percentage
> offset-commit-max-time
> offset-commit-99p-time
> offset-commit-95p-time
> offset-commit-90p-time
> offset-commit-75p-time
> offset-commit-50p-time
> batch-size-max
> batch-size-avg
>
> kafka.connect:type=source-task-metrics,connector=([-.\w]+),task=([\d]+)
> source-record-poll-rate
> source-record-write-rate
>
> kafka.connect:type=sink-task-metrics,connector=([-.\w]+),task=([\d]+)
> sink-record-read-rate
> sink-record-send-rate
> sink-record-lag-max
> partition-count
> offset-commit-95p-time
> offset-commit-90p-time
> offset-commit-75p-time
> offset-commit-50p-time
> batch-size-max
> batch-size-avg
>
> kafka.connect:type=sink-task-metrics,connector=([-.\w]+),
> task=([\d]+),topic=([-.\w]+),partition=([\d]+)
> sink-record-lag
> sink-record-lag-avg
> sink-record-lag-max
>
> kafka.connect:type=connect-coordinator-metrics
> task-count
> connector-count
> leader-name
> state
> rest-request-rate
>
> kafka.connect:type=connect-coordinator-metrics,name=assigned-tasks
> assigned-tasks (existing metric, so can't merge in above without
> breaking compatibility)
> kafka.connect:type=connect-coordinator-metrics,name=assigned-connectors
> (existing metric, so can't merge in above without breaking compatibility)
> assigned-connectors (existing metric, so can't merge in above
> without breaking compatibility)
>
> kafka.connect:type=connect-worker-rebalance-metrics
> rebalance-success-total
> rebalance-success-percentage
> rebalance-failure-total
> rebalance-failure-percentage
> rebalance-max-time
> rebalance-99p-time
> rebalance-95p-time
> rebalance-90p-time
> rebalance-75p-time
> rebalance-50p-time
> time-since-last-rebalance
> task-failure-rate
>
> This lets you use a single MBean selector to select multiple related
> attributes all at once. You can use JMX's wildcards to target which
> connectors or tasks or topics or partitions you care about.
>
> Also notice that for the topic and partition level metrics, the attributes
> are named identically ("sink-record-lag-avg" instead of
> "sink-record-{topic}-{partition}.records-lag-avg"), so monitoring systems
> have a consistent string they can use, instead of needing to
> prefix-and-suffix matching against the attribute name. And TBH, it
> integrates better with the work I'm doing in https://issues.apache.org/
> jira/browse/KAFKA-3480
>
> -James
>
> > On Sep 7, 2017, at 4:50 PM, Randall Hauch <rha...@gmail.com> wrote:
> >
> > Hi everyone.
> >
> > I've created a new KIP to add metrics to the Kafka Connect framework:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 196%3A+Add+metrics+to+Kafka+Connect+framework
> >
> > The KIP approval deadline is looming, so if you're interested in Kafka
> > Connect metrics please review and provide feedback as soon as possible.
> I'm
> > interested not only in whether the metrics are sufficient and
> appropriate,
> > but also in whether the MBean naming conventions are okay.
> >
> > Best regards,
> >
> > Randall
>
>
