tarun11Mavani commented on code in PR #19776:
URL: https://github.com/apache/pinot/pull/19776#discussion_r4385275823
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -369,6 +370,33 @@ private void moveSegmentsToDeletedDir(String segmentId,
Long deletedSegmentsRete
LOGGER.warn("Could not move segment {} from {} to {}", segmentId,
fileToDeleteURI.toString(),
deletedSegmentMoveDestURI.toString(), e);
}
+ // Independent of whether the segment object existed, so no orphan sidecar
is left in the live directory.
+ moveSegmentMetadataToDeletedDir(pinotFS, fileToDeleteURI, segmentId,
rawTableName, retentionSuffix);
+ }
+
+ private void moveSegmentMetadataToDeletedDir(PinotFS pinotFS, URI
segmentFileUri, String segmentId,
+ String rawTableName, String retentionSuffix) {
+ // Live sidecar is <segment>.metadata.tar.gz; in Deleted_Segments it is
renamed to <segment>.metadata<suffix> so
+ // that, like the segment object, it carries no .tar.gz extension and the
retention suffix stays last.
+ try {
+ URI segmentMetadataUri =
SegmentPushUtils.generateSegmentMetadataURI(segmentFileUri.toString(),
segmentId);
+ if (!pinotFS.exists(segmentMetadataUri)) {
+ return;
+ }
+ URI deletedMetadataUri = URIUtils.getUri(_dataDir, DELETED_SEGMENTS,
rawTableName,
+ URIUtils.encode(segmentId) + DELETED_METADATA_FILE_SUFFIX +
retentionSuffix);
+ if (pinotFS.move(segmentMetadataUri, deletedMetadataUri, true)) {
Review Comment:
Fixed: retained sidecars now go to Deleted_Segments/<table>/metadata/ with
the segment's archive name, so they can't take a segment's archive name.
testSidecarNeverOverwritesArchivedSegment covers both deletion orders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
