[
https://issues.apache.org/jira/browse/HDDS-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy reassigned HDDS-16948:
-------------------------------------
Assignee: Andrey Yarovoy
> Remove the per-call InfoBucket from rooted-ofs open
> ---------------------------------------------------
>
> Key: HDDS-16948
> URL: https://issues.apache.org/jira/browse/HDDS-16948
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> `BasicRootedOzoneClientAdapterImpl.readFile` (behind `FileSystem.open`) calls
> `getBucket()`, which is an
> `InfoBucket` RPC. For a link bucket it adds an `InfoVolume` and an
> `InfoBucket` per hop. It then calls
> `OzoneBucket.readFile`. Every `open` therefore costs at least 2 OM requests.
> On read-heavy file system
> workloads (Hive, Impala, Spark scans) `open` is one of the most frequent
> metadata operations, so this is
> the largest single `InfoBucket` source left after HDDS-15925 and HDDS-16760.
> The fetch does one job: `OzoneFSUtils.validateBucketLayout`, which rejects
> OBJECT_STORE. `readFile`
> reads nothing else from the bucket.
> ### Why HDDS-15951's lookupFile check does not cover this
> `RpcClient.readFile` uses `lookupFile` only for an OM older than
> `OPTIMIZED_GET_KEY_INFO` (3). For every
> current OM it sends **`GetKeyInfo`**, then checks `isFile()` on the client.
> Its read-retry path
> (`RpcClient.getKeyInfo`) also sends `GetKeyInfo`. So the OBJECT_STORE
> rejection that HDDS-15951 added to
> `lookupFile` is never reached from `open`.
> ### Why the rejection is opt-in
> `GetKeyInfo` is the object store read RPC. It sits behind
> `OzoneBucket.readKey`, `getKeyDetails`,
> `headObject` and the S3 key lookups (`assumeS3Context=true`), which the S3
> Gateway uses for GetObject and
> HeadObject. `cli-shell`, `freon` and `cli-debug` call it as well. An
> unconditional OBJECT_STORE rejection
> would break all of them. As in HDDS-16760, the client asks for the check:
> {code}
> message GetKeyInfoRequest {
> required KeyArgs keyArgs = 1;
> optional bool assumeS3Context = 2;
> // Set by file system clients: OM rejects the request with
> // NOT_SUPPORTED_OPERATION if the resolved bucket layout is OBJECT_STORE.
> optional bool requireFileSystemLayout = 3;
> }
> {code}
> ### Proposed change
> *OM.*
> * `OzoneManagerRequestHandler.getKeyInfo` copies the field into `OmKeyArgs`.
> The `requireFileSystemLayout`
> field added there by HDDS-16760 is reused, and the builder already survives
> `ResolvedBucket.update`.
> * `OmMetadataReader.getKeyInfo` already holds the link-resolved
> `ResolvedBucket`. When the field is set,
> it calls the existing `validateFileSystemLayout(bucket)` before the ACL
> check. This is the same
> placement as `listStatus` in HDDS-16760, so it adds no lookup.
> *Client.*
> * `RpcClient.readFile` sets the field on the `OmKeyArgs` it builds for
> `GetKeyInfo`. The read-retry
> `getKeyInfo` call also sets it, so a retried read is checked too.
> * `OzoneManagerProtocolClientSideTranslatorPB.getKeyInfo` builds `KeyArgs`
> through
> `OmKeyArgs.toProtobuf()` but sets the request-level field explicitly. Do not
> set it unconditionally:
> `readKey` / `getKeyDetails` share this translator method.
> * The adapter needs a way to ask for it, mirroring HDDS-16760's
> `ListStatusLightOptions`. Options are a
> `readFile` overload or an options object on `ClientProtocol`, or
> `OzoneBucket.readFile` setting it
> itself. `OzoneBucket.readFile` is a file-oriented API, so it could always set
> it; decide on review.
> * The adapter skips `getBucket()` when `proxy.getOmVersion() >=` a new
> `OzoneManagerVersion` constant.
> Place it after `LIST_STATUS_REJECTS_OBS(101)` and never below `ZDU(100)`; the
> number is assigned at
> merge. It maps `NOT_SUPPORTED_OPERATION` to `IllegalArgumentException`.
> Against any older OM, including
> one that reports ZDU, it keeps the current path.
> * The `lookupFile` branch for OMs older than version 3 is unchanged, and such
> OMs take the fallback anyway.
> ### Rejected alternative
> `GetKeyInfoResponse` already returns `bucketLayout` (HDDS-16362), so the
> client could check that instead
> and need no new request field. Two problems:
> * It only works when the key exists. A missing key on an OBJECT_STORE bucket
> would then report
> `FileNotFoundException` instead of today's `IllegalArgumentException`.
> * The field was added without a version gate, so an older OM leaves it unset,
> and the client cannot
> tell "not OBJECT_STORE" from "not reported".
> ### Compatibility
> | | Behavior |
> |---|---|
> | New client, new OM | 1 RPC (`GetKeyInfo`); OM rejects OBJECT_STORE |
> | New client, older OM (incl. ZDU) | `getBucket()` + `GetKeyInfo`, as today |
> | Old client / S3 Gateway / `readKey` / `getKeyDetails`, new OM | field
> absent ⇒ unchanged |
> There is one new optional proto field, no new result code and no new
> configuration.
> ## Testing
> * Adapter (Mockito):
> *`open` makes exactly one OM call, carrying the field.
> *`NOT_SUPPORTED_OPERATION` maps to `IllegalArgumentException`;
> `KEY_NOT_FOUND` / `FILE_NOT_FOUND` /
> `NOT_A_FILE` map to `FileNotFoundException`, as today.
> * An older OM, including `ZDU`, falls back to `getBucket()`.
> * `TestOMMetadataReader`:
> * With the field set, `getKeyInfo` rejects OBJECT_STORE without reaching
> `KeyManager` and serves FSO
> and **LEGACY**.
> * Without the field, it still serves OBJECT_STORE: the S3 Gateway case, and
> the guard against anyone
> making the check unconditional.
> * The same holds with `assumeS3Context=true`.
> * `TestOzoneManagerRequestHandler`: the field is forwarded into `OmKeyArgs`.
> * Existing S3 and object-store read suites pass unchanged.
> * `AbstractRootedOzoneFileSystemTest`: `open` on an OBJECT_STORE bucket fails
> with
> `IllegalArgumentException`. Do RPC counting in the adapter test, not with
> exact OMMetrics deltas
> (HDDS-16513).
> * Benchmark `TestOfsOpenRpcBenchmark`, same shape as
> `TestOfsListStatusRpcBenchmark`:
> * 200 FSO buckets, open + close of a small file, single thread and 10 threads.
> * n=3 per arm, built and run strictly one after the other.
> * Invariant: exactly 1 OM RPC per `open` instead of 2. `open` alone, with no
> data read, so DN reads do
> not mask the OM saving.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
