[jira] [Updated] (KAFKA-14672) Producer queue time does not reflect batches expired in the accumulator
[
https://issues.apache.org/jira/browse/KAFKA-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirk True updated KAFKA-14672:
--
Description:
The producer exposes two metrics for the time a record has spent in the
accumulator waiting to be drained:
* {{record-queue-time-avg}}
* {{record-queue-time-max}}
The metric is only updated when a batch is ready to send to a broker. It is
also possible for a batch to be expired before it can be sent, but in this
case, the metric is not updated. This seems surprising and makes the queue time
misleading. The only metric I could find that does reflect batch expirations in
the accumulator is the generic {{{}record-error-rate{}}}. It would make sense
to let the queue-time metrics record the time spent in the queue regardless of
the outcome of the record send attempt.
was:
The producer exposes two metrics for the time a record has spent in the
accumulator waiting to be drained:
* `record-queue-time-avg`
* `record-queue-time-max`
The metric is only updated when a batch is ready to send to a broker. It is
also possible for a batch to be expired before it can be sent, but in this
case, the metric is not updated. This seems surprising and makes the queue time
misleading. The only metric I could find that does reflect batch expirations in
the accumulator is the generic `record-error-rate`. It would make sense to let
the queue-time metrics record the time spent in the queue regardless of the
outcome of the record send attempt.
> Producer queue time does not reflect batches expired in the accumulator
> ---
>
> Key: KAFKA-14672
> URL: https://issues.apache.org/jira/browse/KAFKA-14672
> Project: Kafka
> Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: Kirk True
>Priority: Major
>
> The producer exposes two metrics for the time a record has spent in the
> accumulator waiting to be drained:
> * {{record-queue-time-avg}}
> * {{record-queue-time-max}}
> The metric is only updated when a batch is ready to send to a broker. It is
> also possible for a batch to be expired before it can be sent, but in this
> case, the metric is not updated. This seems surprising and makes the queue
> time misleading. The only metric I could find that does reflect batch
> expirations in the accumulator is the generic {{{}record-error-rate{}}}. It
> would make sense to let the queue-time metrics record the time spent in the
> queue regardless of the outcome of the record send attempt.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KAFKA-14672) Producer queue time does not reflect batches expired in the accumulator
[ https://issues.apache.org/jira/browse/KAFKA-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-14672: Description: The producer exposes two metrics for the time a record has spent in the accumulator waiting to be drained: * `record-queue-time-avg` * `record-queue-time-max` The metric is only updated when a batch is ready to send to a broker. It is also possible for a batch to be expired before it can be sent, but in this case, the metric is not updated. This seems surprising and makes the queue time misleading. The only metric I could find that does reflect batch expirations in the accumulator is the generic `record-error-rate`. It would make sense to let the queue-time metrics record the time spent in the queue regardless of the outcome of the record send attempt. was: The producer exposes two metrics for the time a record has spent in the accumulator waiting to be drained: * `record-queue-time-avg` * `record-queue-time-max` The metric is only updated when a batch is drained for sending to the broker. It is also possible for a batch to be expired before it can be drained, but in this case, the metric is not updated. This seems surprising and makes the queue time misleading. The only metric I could find that does reflect batch expirations in the accumulator is the generic `record-error-rate`. It would make sense to let the queue-time metrics record the time spent in the queue regardless of the outcome of the record send attempt. > Producer queue time does not reflect batches expired in the accumulator > --- > > Key: KAFKA-14672 > URL: https://issues.apache.org/jira/browse/KAFKA-14672 > Project: Kafka > Issue Type: Bug >Reporter: Jason Gustafson >Priority: Major > > The producer exposes two metrics for the time a record has spent in the > accumulator waiting to be drained: > * `record-queue-time-avg` > * `record-queue-time-max` > The metric is only updated when a batch is ready to send to a broker. It is > also possible for a batch to be expired before it can be sent, but in this > case, the metric is not updated. This seems surprising and makes the queue > time misleading. The only metric I could find that does reflect batch > expirations in the accumulator is the generic `record-error-rate`. It would > make sense to let the queue-time metrics record the time spent in the queue > regardless of the outcome of the record send attempt. -- This message was sent by Atlassian Jira (v8.20.10#820010)
