henry3260 opened a new pull request, #11174:
URL: https://github.com/apache/ozone/pull/11174
## What changes were proposed in this pull request?
Recon stores file counts in a histogram keyed by `(volume, bucket,
fileSizeUpperBound)`, where the
upper bound is a power of two computed by
`ReconUtils.getFileSizeUpperBound()`. The write path applies
this normalization in `FileSizeCountTaskHelper#getFileSizeCountKey()`, but
the read path in
`UtilizationEndpoint#getFileCounts()` built the lookup key from the raw
`fileSize` query parameter.
As a result, `GET /api/v1/utilization/fileCount` returned an empty list for
any size that is not
an exact bin boundary:
```
# bucket contains two 100000-byte keys, stored in bin 131072
GET /api/v1/utilization/fileCount?volume=vol1&bucket=bucket1&fileSize=100000
-> []
GET /api/v1/utilization/fileCount?volume=vol1&bucket=bucket1&fileSize=131072
-> [{"fileSize":131072,"count":2}]
```
The sibling endpoint `/utilization/containerCount` already normalizes its
parameter through
`ReconUtils.getContainerSizeUpperBound()`, so the two APIs behaved
inconsistently.
This PR normalizes the parameter before the lookup, and sets the normalized
value on the response.
The latter matters because the unfiltered branch of the same endpoint returns
`key.getFileSizeUpperBound()`, so without it the meaning of the `fileSize`
response field would
depend on whether the query parameter was supplied.
Changed files:
*
`hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/UtilizationEndpoint.java`
-
normalize `fileSize` via `ReconUtils.getFileSizeUpperBound()` and return
the bin upper bound.
*
`hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestEndpoints.java`
-
add a non-boundary query assertion. The existing test only queried exact
bin boundaries, which is
why this was not caught.
The write path and the on-disk layout are unchanged, so no existing Recon
data is affected.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16357
## How was this patch tested?
Unit tests: `mvn -pl :ozone-recon test -Dtest=TestEndpoints` passes (15/15).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
