[
https://issues.apache.org/jira/browse/HDDS-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy updated HDDS-16951:
----------------------------------
Description:
`BasicRootedOzoneFileSystem.rename` calls `adapterImpl.getBucket(ofsSrc,
false)`. That is an `InfoBucket`,
plus an `InfoVolume` and an `InfoBucket` per link hop. It then *{*}branches on
the layout{*}*:
* *{*}FSO{*}*: `renameFSO` → one `RenameKey`. OM renames a file or a whole
directory atomically. Total:
2 OM RPCs.
* *{*}LEGACY{*}*: two or three `getFileStatus` calls, a possible `listStatus`,
then `RenameIterator`, which
issues another `getBucket()` and one `RenameKey` per key under the source.
* *{*}OBJECT_STORE{*}*: `getBucket()` throws `IllegalArgumentException` from
`validateBucketLayout`.
So for the common case, an FSO bucket, half the RPCs of every `rename` exist
only to learn the layout. Hive, Spark and MapReduce commit by renaming, so
`rename` is one of the hottest file system write
operations.
This differs from the other Jiras under HDDS-16761. The fetch is not just a
rejection check. Its result chooses *which algorithm* the client runs. So
moving a check into OM is not enough. OM has to say "this is not FSO", and the
client then has to fall back.
*Why it has to be opt-in*
`RenameKey` is dual-use. `OMKeyRenameRequest` is registered for
`BucketLayout.OBJECT_STORE` (that
registration also serves LEGACY), and these callers send it on any layout:
* `cli-shell`'s `RenameKeyHandler`;
* `TrashOzoneFileSystem`;
* `KeyLifecycleService` (non-OBJECT_STORE);
* the public `OzoneBucket.renameKey`.
On LEGACY a single `RenameKey` renames one key, not a directory tree. So OM
cannot reject anything by default, and the client needs a request field:
{code:java}
message RenameKeyRequest {
required KeyArgs keyArgs = 1;
required string toKeyName = 2;
optional uint64 updateID = 3;
// Set by file system clients that rely on atomic directory rename: OM
// rejects the request with NOT_SUPPORTED_OPERATION unless the resolved
// bucket layout is FILE_SYSTEM_OPTIMIZED.
optional bool requireFileSystemOptimized = 4;
}
{code}
This is deliberately *{*}not{*}* `requireFileSystemLayout`, which only rejects
OBJECT_STORE.
Here LEGACY must also be rejected, because a single `RenameKey` there does not
implement the file system `rename`. The name says what it requires.
*Proposed change*
*OM.* Check on the leader before Ratis, in `OMKeyRenameRequest.preExecute` /
`OMKeyRenameRequestWithFSO.preExecute`
or right after `createClientRequest`. Use the link-resolved layout
`BucketLayoutAwareOMKeyRequestFactory`
already read. When the field is set and the layout is not
`FILE_SYSTEM_OPTIMIZED`, throw
`OMException(NOT_SUPPORTED_OPERATION)`. Neither rename class uses that code
today, so the client can treat it as unambiguous. Do not put the check in
`createRequest` itself, because that also runs at apply time.
*Client.* When `proxy.getOmVersion() >=` a new `OzoneManagerVersion` constant,
`BasicRootedOzoneFileSystem.rename`
first sends `RenameKey` with the field set, with no `getBucket()` before it.
Place the constant after
`LIST_STATUS_REJECTS_OBS(101)`, never below `ZDU(100)`; the number is assigned
at merge.
* Success, or `KEY_ALREADY_EXISTS` / `KEY_RENAME_ERROR` / `KEY_NOT_FOUND`:
same handling as today's
`renameFSO`.
* `NOT_SUPPORTED_OPERATION`: run today's code from `getBucket()` onward.
OBJECT_STORE raises
`IllegalArgumentException` and LEGACY takes the iterating path, both exactly as
today.
* `BUCKET_NOT_FOUND` / `VOLUME_NOT_FOUND`: same fallback, so the error a
caller sees for a missing bucket
or an orphan link is the one `getBucket()` gives today.
* Against any older OM, including one that reports ZDU, keep today's path.
Cost per `rename`:
* FSO: 2 → 1 RPC.
* LEGACY: one extra failed `RenameKey`. LEGACY is rare for ofs, and its rename
already costs many RPCs.
* OBJECT_STORE: one extra failed RPC before the same exception.
`adapterImpl.rename(OzoneBucket, …)` needs a variant that does not take a
fetched bucket. It can build
the handle locally with `OzoneBucket.newBuilder(...)`, or call
`ClientProtocol.renameKey` with the new
options.
*Compatibility*
| |Behavior|
|—|—|
|New client, new OM, FSO|1 RPC|
|New client, new OM, LEGACY / OBJECT_STORE|failed `RenameKey`, then today's
path|
|New client, older OM (incl. ZDU)|as today|
|Old client / shell / trash / lifecycle / `OzoneBucket.renameKey`, new OM|field
absent ⇒ unchanged|
There is one new optional proto field, no new result code and no new
configuration.
*Testing*
* `BasicRootedOzoneFileSystem` / adapter (Mockito):
* FSO: exactly one `RenameKey` with the field set and no `getBucket()`.
*`NOT_SUPPORTED_OPERATION` on a LEGACY bucket takes the iterating path.
*`NOT_SUPPORTED_OPERATION` on an OBJECT_STORE bucket raises
`IllegalArgumentException`.
* The `KEY_*` codes give `false` as today.
* An older OM, including one at `ZDU`, falls back.
* OM request tests:
* With the field set, FSO succeeds, and *{*}both LEGACY and OBJECT_STORE{*}*
get `NOT_SUPPORTED_OPERATION`
before Ratis.
* Without the field, LEGACY and OBJECT_STORE succeed, guarding the shell,
trash and lifecycle callers.
* A link to an FSO bucket succeeds.
* `AbstractRootedOzoneFileSystemTest` (run as `TestOFS` and `TestOFSWithFSO`):
the existing rename suite
passes unchanged on both layouts. That includes rename into an existing
directory, rename onto itself,
and rename across buckets (rejected client-side before any RPC). Count RPCs in
the adapter test, not
with OMMetrics deltas (HDDS-16513).
* Benchmark `TestOfsRenameRpcBenchmark`, same shape as
`TestOfsListStatusRpcBenchmark`:
* 200 FSO buckets, renaming one file back and forth, single-threaded and 10
threads.
* n=3 per arm, built and run strictly one after the other.
* Invariant: exactly 1 OM RPC per FSO `rename`, from 2.
was:
`BasicRootedOzoneFileSystem.rename` calls `adapterImpl.getBucket(ofsSrc,
false)`. That is an `InfoBucket`,
plus an `InfoVolume` and an `InfoBucket` per link hop. It then **branches on
the layout**:
* **FSO**: `renameFSO` → one `RenameKey`. OM renames a file or a whole
directory atomically. Total:
2 OM RPCs.
* **LEGACY**: two or three `getFileStatus` calls, a possible `listStatus`, then
`RenameIterator`, which
issues another `getBucket()` and one `RenameKey` per key under the source.
* **OBJECT_STORE**: `getBucket()` throws `IllegalArgumentException` from
`validateBucketLayout`.
So for the common case, an FSO bucket, half the RPCs of every `rename` exist
only to learn the layout. Hive, Spark and MapReduce commit by renaming, so
`rename` is one of the hottest file system write
operations.
This differs from the other Jiras under HDDS-16761. The fetch is not just a
rejection check. Its result chooses *which algorithm* the client runs. So
moving a check into OM is not enough. OM has to say "this is not FSO", and the
client then has to fall back.
### Why it has to be opt-in
`RenameKey` is dual-use. `OMKeyRenameRequest` is registered for
`BucketLayout.OBJECT_STORE` (that
registration also serves LEGACY), and these callers send it on any layout:
* `cli-shell`'s `RenameKeyHandler`;
* `TrashOzoneFileSystem`;
* `KeyLifecycleService` (non-OBJECT_STORE);
* the public `OzoneBucket.renameKey`.
On LEGACY a single `RenameKey` renames one key, not a directory tree. So OM
cannot reject anything by default, and the client needs a request field:
{code}
message RenameKeyRequest {
required KeyArgs keyArgs = 1;
required string toKeyName = 2;
optional uint64 updateID = 3;
// Set by file system clients that rely on atomic directory rename: OM
// rejects the request with NOT_SUPPORTED_OPERATION unless the resolved
// bucket layout is FILE_SYSTEM_OPTIMIZED.
optional bool requireFileSystemOptimized = 4;
}
{code}
This is deliberately **not** `requireFileSystemLayout`, which only rejects
OBJECT_STORE.
Here LEGACY must also be rejected, because a single `RenameKey` there does not
implement the file system `rename`. The name says what it requires.
### Proposed change
*OM.* Check on the leader before Ratis, in `OMKeyRenameRequest.preExecute` /
`OMKeyRenameRequestWithFSO.preExecute`
or right after `createClientRequest`. Use the link-resolved layout
`BucketLayoutAwareOMKeyRequestFactory`
already read. When the field is set and the layout is not
`FILE_SYSTEM_OPTIMIZED`, throw
`OMException(NOT_SUPPORTED_OPERATION)`. Neither rename class uses that code
today, so the client can treat it as unambiguous. Do not put the check in
`createRequest` itself, because that also runs at apply time.
*Client.* When `proxy.getOmVersion() >=` a new `OzoneManagerVersion` constant,
`BasicRootedOzoneFileSystem.rename`
first sends `RenameKey` with the field set, with no `getBucket()` before it.
Place the constant after
`LIST_STATUS_REJECTS_OBS(101)`, never below `ZDU(100)`; the number is assigned
at merge.
* Success, or `KEY_ALREADY_EXISTS` / `KEY_RENAME_ERROR` / `KEY_NOT_FOUND`: same
handling as today's
`renameFSO`.
* `NOT_SUPPORTED_OPERATION`: run today's code from `getBucket()` onward.
OBJECT_STORE raises
`IllegalArgumentException` and LEGACY takes the iterating path, both exactly as
today.
* `BUCKET_NOT_FOUND` / `VOLUME_NOT_FOUND`: same fallback, so the error a caller
sees for a missing bucket
or an orphan link is the one `getBucket()` gives today.
* Against any older OM, including one that reports ZDU, keep today's path.
Cost per `rename`:
* FSO: 2 → 1 RPC.
* LEGACY: one extra failed `RenameKey`. LEGACY is rare for ofs, and its rename
already costs many RPCs.
* OBJECT_STORE: one extra failed RPC before the same exception.
`adapterImpl.rename(OzoneBucket, …)` needs a variant that does not take a
fetched bucket. It can build
the handle locally with `OzoneBucket.newBuilder(...)`, or call
`ClientProtocol.renameKey` with the new
options.
### Compatibility
| | Behavior |
|---|---|
| New client, new OM, FSO | 1 RPC |
| New client, new OM, LEGACY / OBJECT_STORE | failed `RenameKey`, then today's
path |
| New client, older OM (incl. ZDU) | as today |
| Old client / shell / trash / lifecycle / `OzoneBucket.renameKey`, new OM |
field absent ⇒ unchanged |
There is one new optional proto field, no new result code and no new
configuration.
Points for review:
* The field's semantics are FSO-only, and they differ from the other HDDS-16761
fields. If reviewers
prefer one shape across Jiras, an enum `requiredBucketLayout` (FILE_SYSTEM |
FILE_SYSTEM_OPTIMIZED) is
the alternative. Decide together with the delete Jira.
* The BUCKET/READ ACL check from `getBucketDetails` is dropped on the FSO path.
`RenameKey` keeps its own
key ACL checks.
* The audit log shows a failed `RENAME_KEY` before `INFO_BUCKET` on LEGACY /
OBJECT_STORE.
## Testing
* `BasicRootedOzoneFileSystem` / adapter (Mockito):
* FSO: exactly one `RenameKey` with the field set and no `getBucket()`.
*`NOT_SUPPORTED_OPERATION` on a LEGACY bucket takes the iterating path.
*`NOT_SUPPORTED_OPERATION` on an OBJECT_STORE bucket raises
`IllegalArgumentException`.
* The `KEY_*` codes give `false` as today.
* An older OM, including one at `ZDU`, falls back.
* OM request tests:
* With the field set, FSO succeeds, and **both LEGACY and OBJECT_STORE** get
`NOT_SUPPORTED_OPERATION`
before Ratis.
* Without the field, LEGACY and OBJECT_STORE succeed, guarding the shell, trash
and lifecycle callers.
* A link to an FSO bucket succeeds.
* `AbstractRootedOzoneFileSystemTest` (run as `TestOFS` and `TestOFSWithFSO`):
the existing rename suite
passes unchanged on both layouts. That includes rename into an existing
directory, rename onto itself,
and rename across buckets (rejected client-side before any RPC). Count RPCs in
the adapter test, not
with OMMetrics deltas (HDDS-16513).
* Benchmark `TestOfsRenameRpcBenchmark`, same shape as
`TestOfsListStatusRpcBenchmark`:
* 200 FSO buckets, renaming one file back and forth, single-threaded and 10
threads.
* n=3 per arm, built and run strictly one after the other.
* Invariant: exactly 1 OM RPC per FSO `rename`, from 2.
> Remove the per-call InfoBucket from rooted-ofs rename on FSO buckets
> --------------------------------------------------------------------
>
> Key: HDDS-16951
> URL: https://issues.apache.org/jira/browse/HDDS-16951
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> `BasicRootedOzoneFileSystem.rename` calls `adapterImpl.getBucket(ofsSrc,
> false)`. That is an `InfoBucket`,
> plus an `InfoVolume` and an `InfoBucket` per link hop. It then *{*}branches
> on the layout{*}*:
> * *{*}FSO{*}*: `renameFSO` → one `RenameKey`. OM renames a file or a whole
> directory atomically. Total:
> 2 OM RPCs.
> * *{*}LEGACY{*}*: two or three `getFileStatus` calls, a possible
> `listStatus`, then `RenameIterator`, which
> issues another `getBucket()` and one `RenameKey` per key under the source.
> * *{*}OBJECT_STORE{*}*: `getBucket()` throws `IllegalArgumentException` from
> `validateBucketLayout`.
> So for the common case, an FSO bucket, half the RPCs of every `rename` exist
> only to learn the layout. Hive, Spark and MapReduce commit by renaming, so
> `rename` is one of the hottest file system write
> operations.
> This differs from the other Jiras under HDDS-16761. The fetch is not just a
> rejection check. Its result chooses *which algorithm* the client runs. So
> moving a check into OM is not enough. OM has to say "this is not FSO", and
> the client then has to fall back.
> *Why it has to be opt-in*
> `RenameKey` is dual-use. `OMKeyRenameRequest` is registered for
> `BucketLayout.OBJECT_STORE` (that
> registration also serves LEGACY), and these callers send it on any layout:
> * `cli-shell`'s `RenameKeyHandler`;
> * `TrashOzoneFileSystem`;
> * `KeyLifecycleService` (non-OBJECT_STORE);
> * the public `OzoneBucket.renameKey`.
> On LEGACY a single `RenameKey` renames one key, not a directory tree. So OM
> cannot reject anything by default, and the client needs a request field:
> {code:java}
> message RenameKeyRequest {
> required KeyArgs keyArgs = 1;
> required string toKeyName = 2;
> optional uint64 updateID = 3;
> // Set by file system clients that rely on atomic directory rename: OM
> // rejects the request with NOT_SUPPORTED_OPERATION unless the resolved
> // bucket layout is FILE_SYSTEM_OPTIMIZED.
> optional bool requireFileSystemOptimized = 4;
> }
> {code}
> This is deliberately *{*}not{*}* `requireFileSystemLayout`, which only
> rejects OBJECT_STORE.
> Here LEGACY must also be rejected, because a single `RenameKey` there does
> not implement the file system `rename`. The name says what it requires.
> *Proposed change*
> *OM.* Check on the leader before Ratis, in `OMKeyRenameRequest.preExecute` /
> `OMKeyRenameRequestWithFSO.preExecute`
> or right after `createClientRequest`. Use the link-resolved layout
> `BucketLayoutAwareOMKeyRequestFactory`
> already read. When the field is set and the layout is not
> `FILE_SYSTEM_OPTIMIZED`, throw
> `OMException(NOT_SUPPORTED_OPERATION)`. Neither rename class uses that code
> today, so the client can treat it as unambiguous. Do not put the check in
> `createRequest` itself, because that also runs at apply time.
> *Client.* When `proxy.getOmVersion() >=` a new `OzoneManagerVersion`
> constant, `BasicRootedOzoneFileSystem.rename`
> first sends `RenameKey` with the field set, with no `getBucket()` before it.
> Place the constant after
> `LIST_STATUS_REJECTS_OBS(101)`, never below `ZDU(100)`; the number is
> assigned at merge.
> * Success, or `KEY_ALREADY_EXISTS` / `KEY_RENAME_ERROR` / `KEY_NOT_FOUND`:
> same handling as today's
> `renameFSO`.
> * `NOT_SUPPORTED_OPERATION`: run today's code from `getBucket()` onward.
> OBJECT_STORE raises
> `IllegalArgumentException` and LEGACY takes the iterating path, both exactly
> as today.
> * `BUCKET_NOT_FOUND` / `VOLUME_NOT_FOUND`: same fallback, so the error a
> caller sees for a missing bucket
> or an orphan link is the one `getBucket()` gives today.
> * Against any older OM, including one that reports ZDU, keep today's path.
> Cost per `rename`:
> * FSO: 2 → 1 RPC.
> * LEGACY: one extra failed `RenameKey`. LEGACY is rare for ofs, and its
> rename already costs many RPCs.
> * OBJECT_STORE: one extra failed RPC before the same exception.
> `adapterImpl.rename(OzoneBucket, …)` needs a variant that does not take a
> fetched bucket. It can build
> the handle locally with `OzoneBucket.newBuilder(...)`, or call
> `ClientProtocol.renameKey` with the new
> options.
> *Compatibility*
> | |Behavior|
> |—|—|
> |New client, new OM, FSO|1 RPC|
> |New client, new OM, LEGACY / OBJECT_STORE|failed `RenameKey`, then today's
> path|
> |New client, older OM (incl. ZDU)|as today|
> |Old client / shell / trash / lifecycle / `OzoneBucket.renameKey`, new
> OM|field absent ⇒ unchanged|
> There is one new optional proto field, no new result code and no new
> configuration.
> *Testing*
> * `BasicRootedOzoneFileSystem` / adapter (Mockito):
> * FSO: exactly one `RenameKey` with the field set and no `getBucket()`.
> *`NOT_SUPPORTED_OPERATION` on a LEGACY bucket takes the iterating path.
> *`NOT_SUPPORTED_OPERATION` on an OBJECT_STORE bucket raises
> `IllegalArgumentException`.
> * The `KEY_*` codes give `false` as today.
> * An older OM, including one at `ZDU`, falls back.
> * OM request tests:
> * With the field set, FSO succeeds, and *{*}both LEGACY and OBJECT_STORE{*}*
> get `NOT_SUPPORTED_OPERATION`
> before Ratis.
> * Without the field, LEGACY and OBJECT_STORE succeed, guarding the shell,
> trash and lifecycle callers.
> * A link to an FSO bucket succeeds.
> * `AbstractRootedOzoneFileSystemTest` (run as `TestOFS` and
> `TestOFSWithFSO`): the existing rename suite
> passes unchanged on both layouts. That includes rename into an existing
> directory, rename onto itself,
> and rename across buckets (rejected client-side before any RPC). Count RPCs
> in the adapter test, not
> with OMMetrics deltas (HDDS-16513).
> * Benchmark `TestOfsRenameRpcBenchmark`, same shape as
> `TestOfsListStatusRpcBenchmark`:
> * 200 FSO buckets, renaming one file back and forth, single-threaded and 10
> threads.
> * n=3 per arm, built and run strictly one after the other.
> * Invariant: exactly 1 OM RPC per FSO `rename`, from 2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
