Kamal Chandraprakash created KAFKA-15632:
--------------------------------------------
Summary: Drop the invalid remote log metadata events
Key: KAFKA-15632
URL: https://issues.apache.org/jira/browse/KAFKA-15632
Project: Kafka
Issue Type: Task
Reporter: Kamal Chandraprakash
Assignee: Kamal Chandraprakash
{{__remote_log_metadata}} topic cleanup policy is set to {{DELETE}} and default
retention is set to unlimited.
The expectation is that the user will configure the maximum retention time for
this internal topic compared to all the other user created topics in the
cluster. We cannot keep it to unlimited as the contents of this internal topic
need to be in the local storage.
RemoteLogMetadata cache expect the events to be in the order of
[RemoteLogSegmentState#isValidTransition|https://github.com/apache/kafka/blob/trunk/storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogSegmentState.java#L93]
Once the retention time got expired for this topic say after 30 days due to
breach by retention size/time, then there can be partial metadata events and
the
[cache|https://github.com/apache/kafka/blob/trunk/storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataCache.java#L160]
start to throw RemoteResourceNotFoundError.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
