Jackie-Jiang commented on code in PR #15048:
URL: https://github.com/apache/pinot/pull/15048#discussion_r1955188242
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -235,7 +236,11 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
long retentionMs = deletedSegmentsRetentionMs == null
? _defaultDeletedSegmentsRetentionMs : deletedSegmentsRetentionMs;
String rawTableName =
TableNameBuilder.extractRawTableName(tableNameWithType);
- URI fileToDeleteURI = URIUtils.getUri(_dataDir, rawTableName,
URIUtils.encode(segmentId));
+ URI fileToDeleteURI = getFileToDeleteURI(rawTableName, segmentId);
+ if (fileToDeleteURI == null) {
+ LOGGER.warn("No segment file found for segment: {} in deep store",
segmentId);
Review Comment:
(minor)
```suggestion
LOGGER.warn("No segment file found for segment: {} in deep store,
skipping deletion", segmentId);
```
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -282,6 +287,42 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
}
}
+ /**
+ * Gets URI for segment deletion by attempting to locate the segment file in
the following order:
+ * 1. Plain segment file (without extension)
+ * 2. Compressed segment file (with .tar.gz extension)
+ *
+ * @param rawTableName The name of the table
+ * @param segmentId The ID of the segment to delete
+ * @return URI of the existing segment file, or null if neither variant
exists or in case of filesystem errors
+ */
+ private URI getFileToDeleteURI(String rawTableName, String segmentId) {
+ try {
+ // Create both URIs upfront
+ URI plainFileUri = URIUtils.getUri(_dataDir, rawTableName,
URIUtils.encode(segmentId));
+ URI compressedFileUri = URIUtils.getUri(_dataDir, rawTableName,
Review Comment:
(minor) Move this URI creation after checking the plain uri
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -282,6 +287,42 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
}
}
+ /**
+ * Gets URI for segment deletion by attempting to locate the segment file in
the following order:
+ * 1. Plain segment file (without extension)
+ * 2. Compressed segment file (with .tar.gz extension)
+ *
+ * @param rawTableName The name of the table
+ * @param segmentId The ID of the segment to delete
+ * @return URI of the existing segment file, or null if neither variant
exists or in case of filesystem errors
+ */
+ private URI getFileToDeleteURI(String rawTableName, String segmentId) {
Review Comment:
(minor) Annotate it as `@Nullable`
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java:
##########
@@ -282,6 +287,42 @@ protected void removeSegmentFromStore(String
tableNameWithType, String segmentId
}
}
+ /**
+ * Gets URI for segment deletion by attempting to locate the segment file in
the following order:
+ * 1. Plain segment file (without extension)
+ * 2. Compressed segment file (with .tar.gz extension)
+ *
+ * @param rawTableName The name of the table
+ * @param segmentId The ID of the segment to delete
+ * @return URI of the existing segment file, or null if neither variant
exists or in case of filesystem errors
+ */
+ private URI getFileToDeleteURI(String rawTableName, String segmentId) {
+ try {
+ // Create both URIs upfront
+ URI plainFileUri = URIUtils.getUri(_dataDir, rawTableName,
URIUtils.encode(segmentId));
+ URI compressedFileUri = URIUtils.getUri(_dataDir, rawTableName,
+ URIUtils.encode(segmentId +
TarCompressionUtils.TAR_GZ_FILE_EXTENSION));
+
+ // Get filesystem instance once
+ PinotFS pinotFS = PinotFSFactory.create(plainFileUri.getScheme());
+
+ // Check for plain segment file first
+ if (pinotFS.exists(plainFileUri)) {
+ return plainFileUri;
+ }
+
+ // Check for compressed segment file
+ if (pinotFS.exists(compressedFileUri)) {
+ return compressedFileUri;
+ }
+ LOGGER.error("No file found for segment: {} in deepstore", segmentId);
+ return null;
+ } catch (Exception e) {
+ LOGGER.error("Caught exception while trying to find file for segment: {}
in deepstore", segmentId);
Review Comment:
(minor)
```suggestion
LOGGER.error("No file found for segment: {} in deep store", segmentId);
return null;
} catch (Exception e) {
LOGGER.error("Caught exception while trying to find file for segment:
{} in deep store", segmentId);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
