[
https://issues.apache.org/jira/browse/KAFKA-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743662#comment-17743662
]
Jorge Esteban Quilcate Otoya commented on KAFKA-15181:
------------------------------------------------------
Sure. The TBRLMM is not the one not recovering, but the Replica Fetcher.
My understanding is that this issue happens when a Replica is recovering its
state after being offline. TBRLMM receives partitions assigned, starts managed,
and is marked as initialized and open to receive requests; however the
consumers may have not catch up yet and cache is not ready.
As this calls to the cache happen on the replica fetcher; not the consumer
fetch; exceptions/retries may not be covering this.
The suggested approach is to respond to partitions assigned the same way
writing events is handled, by acknowledging the consumer has catch up
(therefore cache is ready).
> Race condition on partition assigned to TopicBasedRemoteLogMetadataManager
> ---------------------------------------------------------------------------
>
> Key: KAFKA-15181
> URL: https://issues.apache.org/jira/browse/KAFKA-15181
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Reporter: Jorge Esteban Quilcate Otoya
> Assignee: Jorge Esteban Quilcate Otoya
> Priority: Major
> Labels: tiered-storage
>
> TopicBasedRemoteLogMetadataManager (TBRLMM) uses a cache to be prepared
> whever partitions are assigned.
> When partitions are assigned to the TBRLMM instance, a consumer is started to
> keep the cache up to date.
> If the cache hasn't finalized to build, TBRLMM fails to return remote
> metadata about partitions that are store on the backing topic. TBRLMM may not
> recover from this failing state.
> A proposal to fix this issue would be wait after a partition is assigned for
> the consumer to catch up. A similar logic is used at the moment when TBRLMM
> writes to the topic, and uses send callback to wait for consumer to catch up.
> This logic can be reused whever a partition is assigned, so when TBRLMM is
> marked as initialized, cache is ready to serve requests.
> Reference: https://github.com/aiven/kafka/issues/33
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
