satishd commented on code in PR #13561:
URL: https://github.com/apache/kafka/pull/13561#discussion_r1181520975
##########
core/src/main/java/kafka/log/remote/RemoteLogManager.java:
##########
@@ -581,11 +588,18 @@ public void run() {
if (isLeader()) {
// Copy log segments to remote storage
copyLogSegmentsToRemote();
+ // Cleanup/delete expired remote log segments
+ cleanupExpiredRemoteLogSegments();
Review Comment:
This is different from local log deletion. It requires the deletion of
segments from local storage which need to really delete the files.
But incase of remote storages, it does not wait for the data to be deleted
but it marks the file or object for deletion in their respective metadata
stores. Respective garbage collectors in those storages will take care of
deleting the data asynchronously. There is no perf impact for these delete
calls as they take a much shorter time than copying segments. It is very
unlikely that copying segments get affected because of the deletion of
segments. Deletion checks are happening in every iteration so there will not be
many segments that need to be deleted.
Anyways, we can discuss this separately in a separate JIRA. On another note,
all this logic will go to UnifiedLog in future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
