Jackie-Jiang commented on code in PR #17512:
URL: https://github.com/apache/pinot/pull/17512#discussion_r2739562561
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentUploadDownloadRestletResource.java:
##########
@@ -627,8 +627,14 @@ private SuccessResponse uploadSegments(String tableName,
TableType tableType, Fo
SegmentValidationUtils.validateTimeInterval(segmentMetadata,
tableConfig);
}
// TODO: Include the un-tarred segment size when using the METADATA
push rest API. Currently we can only use the
- // tarred segment size as an approximation. Additionally, add the
storage quota check for batch upload mode.
+ // tarred segment size as an approximation.
long segmentSizeInBytes = getSegmentSizeFromFile(sourceDownloadURIStr);
+ if (segmentSizeInBytes < 0) {
+ segmentSizeInBytes = FileUtils.sizeOfDirectory(tempSegmentDir);
Review Comment:
When segment size is unavailable, we should skip the storage quota check.
This size is just the metadata size
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentUploadDownloadRestletResource.java:
##########
@@ -664,9 +670,13 @@ private SuccessResponse uploadSegments(String tableName,
TableType tableType, Fo
allowRefresh, headers, segmentUploadMetadataList);
}
}
+ } catch (WebApplicationException e) {
+ throw e;
Review Comment:
Do we also want to bump up metrics when storage quota check fails? If so, do
a `instanceof` check in the general `catch (Exception e)` instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
