[
https://issues.apache.org/jira/browse/KAFKA-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-16033:
-----------------------------------
Description:
The retry logic for OffsetFetch and OffsetCommit requests lives in the
CommitRequestManager, and applies to requests issued from multiple components
(AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for
the regular auto-commits, MembershipManager for auto-commits before rebalance,
auto-commit before closing consumer). While this approach helps to avoid having
the retry logic in each caller, currently the CommitManager has it in different
places and it ends up being rather hard to follow.
This task aims at reviewing the retry logic from a high level perspective
(multiple callers, with retry needs that have similarities and differences at
the same time). So the review should asses the similarities vs differences, and
then consider two options:
1. Keep retry logic centralized in the CommitManager, but fixed in a more
consistent way, applied the same way for all requests, depending on the
intention expressed by the caller. Advantages of this approach (current
approach + improvement) is that callers that require the same retry logic could
reuse if, keeping it in a single place (ex. commitSync from the consumer
retries in the same way as the auto-commit before rebalance).
2. move retry logic to the caller. This aligns with the way it was done on the
legacy coordinator, but the main challenge seems to be not duplicating the
retry logic in callers that require the same.
This task will also review what exceptions are indeed retried on the
OffsetCommit and OffsetFetch, considering that the legacy implementation only
retries on some expected Retriable errors (not all)
was:
The retry logic for OffsetFetch and OffsetCommit requests lives in the
CommitRequestManager, and applies to requests issued from multiple components
(AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for
the regular auto-commits, MembershipManager for auto-commits before rebalance,
auto-commit before closing consumer). While this approach helps to avoid having
the retry logic in each caller, currently the CommitManager has it in different
places and it ends up being rather hard to follow.
This task aims at reviewing the retry logic from a high level perspective
(multiple callers, with retry needs that have similarities and differences at
the same time). So the review should asses the similarities vs differences, and
then consider two options:
1. Keep retry logic centralized in the CommitManager, but fixed in a more
consistent way, applied the same way for all requests, depending on the
intention expressed by the caller. Advantages of this approach (current
approach + improvement) is that callers that require the same retry logic could
reuse if, keeping it in a single place (ex. commitSync from the consumer
retries in the same way as the auto-commit before rebalance).
2. move retry logic to the caller. This aligns with the way it was done on the
legacy coordinator, but the main challenge seems to be not duplicating the
retry logic in callers that require the same.
> Review client retry logic of OffsetFetch and OffsetCommit responses
> -------------------------------------------------------------------
>
> Key: KAFKA-16033
> URL: https://issues.apache.org/jira/browse/KAFKA-16033
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Major
> Labels: kip-848-client-support
> Fix For: 3.8.0
>
>
> The retry logic for OffsetFetch and OffsetCommit requests lives in the
> CommitRequestManager, and applies to requests issued from multiple components
> (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for
> the regular auto-commits, MembershipManager for auto-commits before
> rebalance, auto-commit before closing consumer). While this approach helps to
> avoid having the retry logic in each caller, currently the CommitManager has
> it in different places and it ends up being rather hard to follow.
> This task aims at reviewing the retry logic from a high level perspective
> (multiple callers, with retry needs that have similarities and differences at
> the same time). So the review should asses the similarities vs differences,
> and then consider two options:
> 1. Keep retry logic centralized in the CommitManager, but fixed in a more
> consistent way, applied the same way for all requests, depending on the
> intention expressed by the caller. Advantages of this approach (current
> approach + improvement) is that callers that require the same retry logic
> could reuse if, keeping it in a single place (ex. commitSync from the
> consumer retries in the same way as the auto-commit before rebalance).
> 2. move retry logic to the caller. This aligns with the way it was done on
> the legacy coordinator, but the main challenge seems to be not duplicating
> the retry logic in callers that require the same.
> This task will also review what exceptions are indeed retried on the
> OffsetCommit and OffsetFetch, considering that the legacy implementation only
> retries on some expected Retriable errors (not all)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
