satishd commented on a change in pull request #11058:
URL: https://github.com/apache/kafka/pull/11058#discussion_r711985629
##########
File path:
storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java
##########
@@ -120,6 +174,29 @@ public void run() {
}
}
+ private void syncCommittedDataAndOffsets(boolean forceSync) {
+ boolean noOffsetUpdates =
committedPartitionToConsumedOffsets.equals(partitionToConsumedOffsets);
+ if (noOffsetUpdates || !forceSync && time.milliseconds() -
lastSyncedTimeMs < committedOffsetSyncIntervalMs) {
+ log.debug("Skip syncing committed offsets, noOffsetUpdates: {},
forceSync: {}", noOffsetUpdates, forceSync);
+ return;
+ }
+
+ try {
+ // todo sync the snapshot file
+ for (TopicIdPartition topicIdPartition : assignedTopicPartitions) {
+ int metadataPartition =
topicPartitioner.metadataPartition(topicIdPartition);
+
remotePartitionMetadataEventHandler.syncLogMetadataDataFile(topicIdPartition,
metadataPartition,
+
partitionToConsumedOffsets.get(metadataPartition));
+ }
+
+ committedOffsetsFile.writeEntries(partitionToConsumedOffsets);
+ committedPartitionToConsumedOffsets = new
HashMap<>(partitionToConsumedOffsets);
Review comment:
Right, it can avoid writing if it is already synced and offset did not
move further. It is addressed in the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
