[jira] [Updated] (KAFKA-16033) Review client retry logic of OffsetFetch and OffsetCommit responses
[ https://issues.apache.org/jira/browse/KAFKA-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16033: -- Priority: Critical (was: Major) > Review client retry logic of OffsetFetch and OffsetCommit responses > --- > > Key: KAFKA-16033 > URL: https://issues.apache.org/jira/browse/KAFKA-16033 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Lianet Magrans >Assignee: Lianet Magrans >Priority: Critical > Labels: kip-848-client-support, offset > Fix For: 3.8.0 > > > The retry logic for OffsetFetch and OffsetCommit requests lives in the > CommitRequestManager, and applies to requests issued from multiple components > (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for > the regular auto-commits, MembershipManager for auto-commits before > rebalance, auto-commit before closing consumer). While this approach helps to > avoid having the retry logic in each caller, currently the CommitManager has > it in different places and it ends up being rather hard to follow. > This task aims at reviewing the retry logic from a high level perspective > (multiple callers, with retry needs that have similarities and differences at > the same time). So the review should asses the similarities vs differences, > and then consider two options: > 1. Keep retry logic centralized in the CommitManager, but fixed in a more > consistent way, applied the same way for all requests, depending on the > intention expressed by the caller. Advantages of this approach (current > approach + improvement) is that callers that require the same retry logic > could reuse if, keeping it in a single place (ex. commitSync from the > consumer retries in the same way as the auto-commit before rebalance). > 2. move retry logic to the caller. This aligns with the way it was done on > the legacy coordinator, but the main challenge seems to be not duplicating > the retry logic in callers that require the same. > This task will also review what exceptions are indeed retried on the > OffsetCommit and OffsetFetch, considering that the legacy implementation only > retries on some expected Retriable errors (not all) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-16033) Review client retry logic of OffsetFetch and OffsetCommit responses
[ https://issues.apache.org/jira/browse/KAFKA-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16033: -- Labels: kip-848-client-support offset (was: kip-848-client-support) > Review client retry logic of OffsetFetch and OffsetCommit responses > --- > > Key: KAFKA-16033 > URL: https://issues.apache.org/jira/browse/KAFKA-16033 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Lianet Magrans >Assignee: Lianet Magrans >Priority: Major > Labels: kip-848-client-support, offset > Fix For: 3.8.0 > > > The retry logic for OffsetFetch and OffsetCommit requests lives in the > CommitRequestManager, and applies to requests issued from multiple components > (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for > the regular auto-commits, MembershipManager for auto-commits before > rebalance, auto-commit before closing consumer). While this approach helps to > avoid having the retry logic in each caller, currently the CommitManager has > it in different places and it ends up being rather hard to follow. > This task aims at reviewing the retry logic from a high level perspective > (multiple callers, with retry needs that have similarities and differences at > the same time). So the review should asses the similarities vs differences, > and then consider two options: > 1. Keep retry logic centralized in the CommitManager, but fixed in a more > consistent way, applied the same way for all requests, depending on the > intention expressed by the caller. Advantages of this approach (current > approach + improvement) is that callers that require the same retry logic > could reuse if, keeping it in a single place (ex. commitSync from the > consumer retries in the same way as the auto-commit before rebalance). > 2. move retry logic to the caller. This aligns with the way it was done on > the legacy coordinator, but the main challenge seems to be not duplicating > the retry logic in callers that require the same. > This task will also review what exceptions are indeed retried on the > OffsetCommit and OffsetFetch, considering that the legacy implementation only > retries on some expected Retriable errors (not all) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-16033) Review client retry logic of OffsetFetch and OffsetCommit responses
[ https://issues.apache.org/jira/browse/KAFKA-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16033: --- Description: The retry logic for OffsetFetch and OffsetCommit requests lives in the CommitRequestManager, and applies to requests issued from multiple components (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for the regular auto-commits, MembershipManager for auto-commits before rebalance, auto-commit before closing consumer). While this approach helps to avoid having the retry logic in each caller, currently the CommitManager has it in different places and it ends up being rather hard to follow. This task aims at reviewing the retry logic from a high level perspective (multiple callers, with retry needs that have similarities and differences at the same time). So the review should asses the similarities vs differences, and then consider two options: 1. Keep retry logic centralized in the CommitManager, but fixed in a more consistent way, applied the same way for all requests, depending on the intention expressed by the caller. Advantages of this approach (current approach + improvement) is that callers that require the same retry logic could reuse if, keeping it in a single place (ex. commitSync from the consumer retries in the same way as the auto-commit before rebalance). 2. move retry logic to the caller. This aligns with the way it was done on the legacy coordinator, but the main challenge seems to be not duplicating the retry logic in callers that require the same. This task will also review what exceptions are indeed retried on the OffsetCommit and OffsetFetch, considering that the legacy implementation only retries on some expected Retriable errors (not all) was: The retry logic for OffsetFetch and OffsetCommit requests lives in the CommitRequestManager, and applies to requests issued from multiple components (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for the regular auto-commits, MembershipManager for auto-commits before rebalance, auto-commit before closing consumer). While this approach helps to avoid having the retry logic in each caller, currently the CommitManager has it in different places and it ends up being rather hard to follow. This task aims at reviewing the retry logic from a high level perspective (multiple callers, with retry needs that have similarities and differences at the same time). So the review should asses the similarities vs differences, and then consider two options: 1. Keep retry logic centralized in the CommitManager, but fixed in a more consistent way, applied the same way for all requests, depending on the intention expressed by the caller. Advantages of this approach (current approach + improvement) is that callers that require the same retry logic could reuse if, keeping it in a single place (ex. commitSync from the consumer retries in the same way as the auto-commit before rebalance). 2. move retry logic to the caller. This aligns with the way it was done on the legacy coordinator, but the main challenge seems to be not duplicating the retry logic in callers that require the same. > Review client retry logic of OffsetFetch and OffsetCommit responses > --- > > Key: KAFKA-16033 > URL: https://issues.apache.org/jira/browse/KAFKA-16033 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Lianet Magrans >Assignee: Lianet Magrans >Priority: Major > Labels: kip-848-client-support > Fix For: 3.8.0 > > > The retry logic for OffsetFetch and OffsetCommit requests lives in the > CommitRequestManager, and applies to requests issued from multiple components > (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for > the regular auto-commits, MembershipManager for auto-commits before > rebalance, auto-commit before closing consumer). While this approach helps to > avoid having the retry logic in each caller, currently the CommitManager has > it in different places and it ends up being rather hard to follow. > This task aims at reviewing the retry logic from a high level perspective > (multiple callers, with retry needs that have similarities and differences at > the same time). So the review should asses the similarities vs differences, > and then consider two options: > 1. Keep retry logic centralized in the CommitManager, but fixed in a more > consistent way, applied the same way for all requests, depending on the > intention expressed by the caller. Advantages of this approach (current > approach + improvement) is that callers that require the same retry logic > could reuse if, keeping it in a single place (ex. commitSync from th
[jira] [Updated] (KAFKA-16033) Review client retry logic of OffsetFetch and OffsetCommit responses
[ https://issues.apache.org/jira/browse/KAFKA-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16033: --- Parent: KAFKA-14048 Issue Type: Sub-task (was: Improvement) > Review client retry logic of OffsetFetch and OffsetCommit responses > --- > > Key: KAFKA-16033 > URL: https://issues.apache.org/jira/browse/KAFKA-16033 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Lianet Magrans >Assignee: Lianet Magrans >Priority: Major > Labels: kip-848-client-support > Fix For: 3.8.0 > > > The retry logic for OffsetFetch and OffsetCommit requests lives in the > CommitRequestManager, and applies to requests issued from multiple components > (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for > the regular auto-commits, MembershipManager for auto-commits before > rebalance, auto-commit before closing consumer). While this approach helps to > avoid having the retry logic in each caller, currently the CommitManager has > it in different places and it ends up being rather hard to follow. > This task aims at reviewing the retry logic from a high level perspective > (multiple callers, with retry needs that have similarities and differences at > the same time). So the review should asses the similarities vs differences, > and then consider two options: > 1. Keep retry logic centralized in the CommitManager, but fixed in a more > consistent way, applied the same way for all requests, depending on the > intention expressed by the caller. Advantages of this approach (current > approach + improvement) is that callers that require the same retry logic > could reuse if, keeping it in a single place (ex. commitSync from the > consumer retries in the same way as the auto-commit before rebalance). > 2. move retry logic to the caller. This aligns with the way it was done on > the legacy coordinator, but the main challenge seems to be not duplicating > the retry logic in callers that require the same. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-16033) Review client retry logic of OffsetFetch and OffsetCommit responses
[ https://issues.apache.org/jira/browse/KAFKA-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16033: --- Summary: Review client retry logic of OffsetFetch and OffsetCommit responses (was: Review retry logic of OffsetFetch and OffsetCommit responses) > Review client retry logic of OffsetFetch and OffsetCommit responses > --- > > Key: KAFKA-16033 > URL: https://issues.apache.org/jira/browse/KAFKA-16033 > Project: Kafka > Issue Type: Improvement > Components: clients, consumer >Reporter: Lianet Magrans >Assignee: Lianet Magrans >Priority: Major > Labels: kip-848-client-support > Fix For: 3.8.0 > > > The retry logic for OffsetFetch and OffsetCommit requests lives in the > CommitRequestManager, and applies to requests issued from multiple components > (AsyncKakfaConsumer for commitSync and commitAsync, CommitRequestManager for > the regular auto-commits, MembershipManager for auto-commits before > rebalance, auto-commit before closing consumer). While this approach helps to > avoid having the retry logic in each caller, currently the CommitManager has > it in different places and it ends up being rather hard to follow. > This task aims at reviewing the retry logic from a high level perspective > (multiple callers, with retry needs that have similarities and differences at > the same time). So the review should asses the similarities vs differences, > and then consider two options: > 1. Keep retry logic centralized in the CommitManager, but fixed in a more > consistent way, applied the same way for all requests, depending on the > intention expressed by the caller. Advantages of this approach (current > approach + improvement) is that callers that require the same retry logic > could reuse if, keeping it in a single place (ex. commitSync from the > consumer retries in the same way as the auto-commit before rebalance). > 2. move retry logic to the caller. This aligns with the way it was done on > the legacy coordinator, but the main challenge seems to be not duplicating > the retry logic in callers that require the same. -- This message was sent by Atlassian Jira (v8.20.10#820010)
