Gargi-jais11 commented on code in PR #10166:
URL: https://github.com/apache/ozone/pull/10166#discussion_r3179555611
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java:
##########
@@ -647,9 +647,47 @@ private OmKeyInfo readKeyInfo(OmKeyArgs args, BucketLayout
bucketLayout)
.sum());
}
}
+ if (args.hasByteRange()) {
+ filterKeyLocationsByByteRange(value, args.getByteRangeStart(),
+ args.getByteRangeEnd());
+ }
return value;
}
+ private void filterKeyLocationsByByteRange(OmKeyInfo keyInfo,
+ long byteRangeStart, long byteRangeEnd) {
+ OmKeyLocationInfoGroup latestLocationVersion =
+ keyInfo.getLatestVersionLocations();
+ if (latestLocationVersion == null) {
+ return;
+ }
+
+ List<OmKeyLocationInfo> currentLocations =
+ latestLocationVersion.getBlocksLatestVersionOnly();
+ List<OmKeyLocationInfo> filteredLocations = new ArrayList<>();
+ long blockStart = 0;
+ long firstBlockStart = 0;
+ for (OmKeyLocationInfo locationInfo : currentLocations) {
+ long blockEnd = blockStart + locationInfo.getLength() - 1;
+ if (blockStart > byteRangeEnd) {
+ break;
+ }
+ if (blockEnd >= byteRangeStart) {
+ if (filteredLocations.isEmpty()) {
+ firstBlockStart = blockStart;
+ }
+ filteredLocations.add(locationInfo);
+ }
+ blockStart += locationInfo.getLength();
+ }
+
+ keyInfo.updateLocationInfoList(filteredLocations,
+ latestLocationVersion.isMultipartKey(), true);
+ if (!filteredLocations.isEmpty()) {
+ keyInfo.setByteRangeStartOffset(byteRangeStart - firstBlockStart);
+ }
Review Comment:
I think we should also set `keyInfo.setDataSize (byteRangeEnd -
byteRangeStart + 1)` as `OzoneKeyDetails` uses
it`RpcClient.getOzoneKeyDetails(...)` does something like new
`OzoneKeyDetails(..., keyInfo.getDataSize(), ...)`.
If you don’t update `dataSize` after filtering, `getDataSize()` still looks
like the whole object / whole part, while `getContent() `only covers some
blocks — confusing and wrong for anything that uses getDataSize() for buffers,
metrics etc.
```suggestion
if (!filteredLocations.isEmpty()) {
keyInfo.setByteRangeStartOffset(byteRangeStart - firstBlockStart);
}
keyInfo.setDataSize (byteRangeEnd - byteRangeStart + 1)
```
##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java:
##########
@@ -1880,6 +1880,37 @@ public OzoneKeyDetails getS3KeyDetails(String
bucketName, String keyName,
return getOzoneKeyDetails(keyInfo);
}
+ @Override
+ public OzoneKeyDetails getS3KeyDetails(String bucketName, String keyName,
+ int partNumber, long startOffset,
+ long endOffset) throws IOException {
Review Comment:
Nit: Please use 4 indent space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
