Re: Where can I find the document for consumer metrics
On Thursday, February 11, 2016 at 19:53, Jason Gustafson wrote: > We have them in the Confluent docs: > http://docs.confluent.io/2.0.0/kafka/monitoring.html#new-consumer-metrics. Thanks Jason, that’s very helpful! Any chance this could be copied over to the canonical Kafka docs?
Re: Where can I find the document for consumer metrics
We have them in the Confluent docs: http://docs.confluent.io/2.0.0/kafka/monitoring.html#new-consumer-metrics. -Jason On Thu, Feb 11, 2016 at 4:40 AM, Avi Flax wrote: > On Thursday, December 17, 2015 at 18:08, Guozhang Wang wrote: > > We should add a section for that. Siyuan can you file a JIRA? > > Did this ever happen? This documentation would be very helpful. > >
Re: Where can I find the document for consumer metrics
On Thursday, December 17, 2015 at 18:08, Guozhang Wang wrote: > We should add a section for that. Siyuan can you file a JIRA? Did this ever happen? This documentation would be very helpful.
Re: Where can I find the document for consumer metrics
We should add a section for that. Siyuan can you file a JIRA?
Guozhang
On Thu, Dec 17, 2015 at 11:08 AM, hsy...@gmail.com wrote:
> I can find some broker/producer metrics here
> http://kafka.apache.org/documentation.html#monitoring
>
> but where can I find consumer metrics docs
>
> Everytime I have to log this to find out what metrics I want
>
> MetricName [name=join-rate, group=consumer-coordinator-metrics,
> description=The number of group joins per second,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@ebea716
> ;MetricName
> [name=fetch-size-avg, group=consumer-fetch-manager-metrics, description=The
> average number of bytes fetched per request,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@6cb9cea
> ;MetricName
> [name=commit-latency-avg, group=consumer-coordinator-metrics,
> description=The average time taken for a commit request,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@21aaca22
> ;MetricName
> [name=join-time-avg, group=consumer-coordinator-metrics, description=The
> average time taken for a group rejoin,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@53bc8f72
> ;MetricName
> [name=incoming-byte-rate, group=consumer-metrics, description=Bytes/second
> read off all sockets,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@34f9c3e0
> ;MetricName
> [name=bytes-consumed-rate, group=consumer-fetch-manager-metrics,
> description=The average number of bytes consumed per second,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@36c7401a
> ;MetricName
> [name=response-rate, group=consumer-metrics, description=Responses received
> sent per second.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@5341870e
> ;MetricName
> [name=connection-creation-rate, group=consumer-metrics, description=New
> connections established per second in the window.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@9f0d8f4
> ;MetricName
> [name=fetch-rate, group=consumer-fetch-manager-metrics, description=The
> number of fetch requests per second.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@23338045
> ;MetricName
> [name=join-time-max, group=consumer-coordinator-metrics, description=The
> max time taken for a group rejoin,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@5cdabd4d
> ;MetricName
> [name=io-wait-ratio, group=consumer-metrics, description=The fraction of
> time the I/O thread spent waiting.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@659b7186
> ;MetricName
> [name=fetch-size-max, group=consumer-fetch-manager-metrics, description=The
> maximum number of bytes fetched per request,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@403a4887
> ;MetricName
> [name=assigned-partitions, group=consumer-coordinator-metrics,
> description=The number of partitions currently assigned to this consumer,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@326fb802
> ;MetricName
> [name=io-time-ns-avg, group=consumer-metrics, description=The average
> length of time for I/O per select call in nanoseconds.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@432b0ee3
> ;MetricName
> [name=records-consumed-rate, group=consumer-fetch-manager-metrics,
> description=The average number of records consumed per second,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@3fde7b88
> ;MetricName
> [name=io-wait-time-ns-avg, group=consumer-metrics, description=The average
> length of time the I/O thread spent waiting for a socket ready for reads or
> writes in nanoseconds.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@5485cfd8
> ;MetricName
> [name=select-rate, group=consumer-metrics, description=Number of times the
> I/O layer checked for new I/O to perform per second,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@2cbdcaf6
> ;MetricName
> [name=fetch-throttle-time-max, group=consumer-fetch-manager-metrics,
> description=The maximum throttle time in ms,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@6057f36c
> ;MetricName
> [name=heartbeat-response-time-max, group=consumer-coordinator-metrics,
> description=The max time taken to receive a response to a heartbeat
> request,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@2e2e68de
> ;MetricName
> [name=network-io-rate, group=consumer-metrics, description=The average
> number of network operations (reads or writes) on all connections per
> second.,
>
> tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@68e6de81
> ;MetricName
> [name=fetch-latency-max, group=consumer-fetch-manager-metrics,
> description=The max time taken for an
Where can I find the document for consumer metrics
I can find some broker/producer metrics here
http://kafka.apache.org/documentation.html#monitoring
but where can I find consumer metrics docs
Everytime I have to log this to find out what metrics I want
MetricName [name=join-rate, group=consumer-coordinator-metrics,
description=The number of group joins per second,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@ebea716;MetricName
[name=fetch-size-avg, group=consumer-fetch-manager-metrics, description=The
average number of bytes fetched per request,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@6cb9cea;MetricName
[name=commit-latency-avg, group=consumer-coordinator-metrics,
description=The average time taken for a commit request,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@21aaca22;MetricName
[name=join-time-avg, group=consumer-coordinator-metrics, description=The
average time taken for a group rejoin,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@53bc8f72;MetricName
[name=incoming-byte-rate, group=consumer-metrics, description=Bytes/second
read off all sockets,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@34f9c3e0;MetricName
[name=bytes-consumed-rate, group=consumer-fetch-manager-metrics,
description=The average number of bytes consumed per second,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@36c7401a;MetricName
[name=response-rate, group=consumer-metrics, description=Responses received
sent per second.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@5341870e;MetricName
[name=connection-creation-rate, group=consumer-metrics, description=New
connections established per second in the window.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@9f0d8f4;MetricName
[name=fetch-rate, group=consumer-fetch-manager-metrics, description=The
number of fetch requests per second.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@23338045;MetricName
[name=join-time-max, group=consumer-coordinator-metrics, description=The
max time taken for a group rejoin,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@5cdabd4d;MetricName
[name=io-wait-ratio, group=consumer-metrics, description=The fraction of
time the I/O thread spent waiting.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@659b7186;MetricName
[name=fetch-size-max, group=consumer-fetch-manager-metrics, description=The
maximum number of bytes fetched per request,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@403a4887;MetricName
[name=assigned-partitions, group=consumer-coordinator-metrics,
description=The number of partitions currently assigned to this consumer,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@326fb802;MetricName
[name=io-time-ns-avg, group=consumer-metrics, description=The average
length of time for I/O per select call in nanoseconds.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@432b0ee3;MetricName
[name=records-consumed-rate, group=consumer-fetch-manager-metrics,
description=The average number of records consumed per second,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@3fde7b88;MetricName
[name=io-wait-time-ns-avg, group=consumer-metrics, description=The average
length of time the I/O thread spent waiting for a socket ready for reads or
writes in nanoseconds.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@5485cfd8;MetricName
[name=select-rate, group=consumer-metrics, description=Number of times the
I/O layer checked for new I/O to perform per second,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@2cbdcaf6;MetricName
[name=fetch-throttle-time-max, group=consumer-fetch-manager-metrics,
description=The maximum throttle time in ms,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@6057f36c;MetricName
[name=heartbeat-response-time-max, group=consumer-coordinator-metrics,
description=The max time taken to receive a response to a heartbeat
request,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@2e2e68de;MetricName
[name=network-io-rate, group=consumer-metrics, description=The average
number of network operations (reads or writes) on all connections per
second.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@68e6de81;MetricName
[name=fetch-latency-max, group=consumer-fetch-manager-metrics,
description=The max time taken for any fetch request.,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@d1a1cf5;MetricName
[name=request-size-avg, group=consumer-metrics, description=The average
size of all requests in the window..,
tags={client-id=consumer-4}]=org.apache.kafka.common.metrics.KafkaMetric@2d631f8b;MetricName
[name=commit-rate, group=consumer-coordinator-metrics, descri
