klsince commented on code in PR #14094:
URL: https://github.com/apache/pinot/pull/14094#discussion_r1779274145
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java:
##########
@@ -989,6 +1016,12 @@ protected void doTakeSnapshot() {
}
}
_updatedSegmentsSinceLastSnapshot.clear();
+ // Persist TTL watermark after taking snapshots if TTL is enabled, so that
segments out of TTL can be loaded with
+ // updated validDocIds bitmaps. If the TTL watermark is persisted first,
segments out of TTL may get loaded with
+ // stale bitmaps or even no bitmap snapshots to use.
+ if (isTTLEnabled()) {
+ persistWatermark(_largestSeenComparisonValue.get());
+ }
Review Comment:
as discussed offline, there is no harm to persist/restore watermark for
table just using deletedKeysTTL, and considering when deletedKeysTTL and
metadataTTL are both enabled for a table, deletedKeysTTL uses the restored
watermark anyway, so I kept this logic to make tracking watermark consistent
for both TTL types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
