[
https://issues.apache.org/jira/browse/HDDS-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy updated HDDS-16760:
----------------------------------
Status: Patch Available (was: In Progress)
> Remove the per-call InfoBucket from rooted-ofs listStatus
> ---------------------------------------------------------
>
> Key: HDDS-16760
> URL: https://issues.apache.org/jira/browse/HDDS-16760
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
> Labels: pull-request-available
>
> `BasicRootedOzoneClientAdapterImpl.listStatus` issues a `getBucketDetails`
> (`InfoBucket`) RPC on every
> call, so a file system client doing N listings sends 2N OM requests. On
> metadata-heavy `ofs` workloads
> this makes `InfoBucket` one of the highest-volume RPCs OM serves — compare
> OM's `NumBucketInfos` counter
> against `NumListStatus`.
> The fetch does two jobs, and both have to be addressed before it can be
> dropped.
> *Layout check.* `OzoneFSUtils.validateBucketLayout` rejects `OBJECT_STORE`.
> Move it into OM, following
> HDDS-15925: `OmMetadataReader.listStatus` and `listStatusLight` lack the
> check today, and
> `ResolvedBucket` already carries the link-resolved layout, so it costs no
> extra lookup. `listStatus` has
> no object-store caller — nothing in `s3gateway` calls it, and
> `DeleteKeyHandler`'s call is gated on
> `isFileSystemOptimized()` — so the rejection can be unconditional, with no
> protobuf change.
> *Orphan-link pre-check.* `isSourcePathExist()` makes `listStatus` return an
> empty list when a link
> bucket's source has been deleted. OM already fails that case with
> `BUCKET_NOT_FOUND` via
> `resolveBucketLink(..., allowDanglingBuckets=false)`. Move the disambiguation
> to the error path: list
> optimistically with a locally built handle, and only on `BUCKET_NOT_FOUND` do
> the `getBucket()` plus link
> resolve that picks between the empty list and `FileNotFoundException`.
> The client then drops the fetch behind a new `OzoneManagerVersion` constant,
> retaining the current path
> as the fallback for an older OM.
>
> {{listStatus}} _does_ have object-store callers. {{OzoneBucket.listKeys(...,
> shallow=true)}} on OBS/LEGACY buckets uses ListStatusLight, and S3 Gateway
> uses it for {{ListObjects}} with {{{}delimiter=/{}}}. OM trash also lists OBS
> buckets. So the rejection is opt-in: {{ListStatusRequest}} gets {{{}optional
> bool requireFileSystemLayout = 7{}}}, set only by the ofs client. OM rejects
> OBJECT_STORE only when it is set. This is one new optional proto field, with
> no new result code or configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
