[
https://issues.apache.org/jira/browse/KAFKA-15438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-15438:
---
Description:
The refactored async consumer reuses part of the core logic required for
resetting and validating positions. That currently works on the principle of
async requests, that reset/validate positions when responses are received. If
the responses include errors, or if a validation verification fails (ex. log
truncation detected), exceptions are saved in-memory, to be thrown on the next
call to the reset/validate. Note that these functionalities are periodically
called as part of the poll loop to update fetch positions before fetching
records.
As an initial implementation, the async consumer reuses this same caching
logic, as it has the asyn nature required. Keeping this caching logic ensure
that we maintaint the timing of the exceptions thrown for reset/validate (they
are currently not thrown when discovered, instead they are thrown on the next
call to reset/validate). This task aims at reviewing the implications of
changing this behaviour, and rely on the completion of the Reset and Validate
events instead, to propagate the errors found. Note that this would happen
closely inter-wined with the continued poll loop, that may have already issued
a new reset/validate.
was:
The refactored async consumer reuses part of the core logic required for
resetting and validating positions. That currently works on the principle of
async requests, that reset/validate positions when responses are received. If
the responses include errors, or if a validation verification fails (ex. log
truncation detected), exceptions are saved in-memory, to be thrown on the next
call to the reset/validate. Note that these functionalities are periodically
called as part of the poll loop to update fetch positions before fetching
records.
As an initial implementation, the async consumer reuses this same caching
logic, as it has the asyn nature required. This task aims at reviewing the
processing of `ResetApplicationEvent `and `ValidatePositionsApplicationEvent`
to evaluate if they should rely on event completion instead, to propagate the
errors found. It would align with how other application events manage async
requests and responses/errors for the new async consumer (based on
CompletableFutures), but with the trade-off of heavily changing a caching logic
that is currently reused by the legacy and the new consumer in the
OffsetFetcherUtils.
> Review exception caching logic used for reset/validate positions in async
> consumer
> --
>
> Key: KAFKA-15438
> URL: https://issues.apache.org/jira/browse/KAFKA-15438
> Project: Kafka
> Issue Type: Task
> Components: clients, consumer
>Reporter: Lianet Magrans
>Priority: Minor
> Labels: consumer-threading-refactor
>
> The refactored async consumer reuses part of the core logic required for
> resetting and validating positions. That currently works on the principle of
> async requests, that reset/validate positions when responses are received. If
> the responses include errors, or if a validation verification fails (ex. log
> truncation detected), exceptions are saved in-memory, to be thrown on the
> next call to the reset/validate. Note that these functionalities are
> periodically called as part of the poll loop to update fetch positions before
> fetching records.
>
> As an initial implementation, the async consumer reuses this same caching
> logic, as it has the asyn nature required. Keeping this caching logic ensure
> that we maintaint the timing of the exceptions thrown for reset/validate
> (they are currently not thrown when discovered, instead they are thrown on
> the next call to reset/validate). This task aims at reviewing the
> implications of changing this behaviour, and rely on the completion of the
> Reset and Validate events instead, to propagate the errors found. Note that
> this would happen closely inter-wined with the continued poll loop, that may
> have already issued a new reset/validate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
