Andrey Yarovoy created HDDS-16950:
-------------------------------------
Summary: Remove the per-call InfoBucket from rooted-ofs mkdirs
Key: HDDS-16950
URL: https://issues.apache.org/jira/browse/HDDS-16950
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Client
Reporter: Andrey Yarovoy
`BasicRootedOzoneClientAdapterImpl.createDirectory` (behind
`FileSystem.mkdirs`) calls
`getBucket(ofsPath, createIfNotExist=true)` and then
`OzoneBucket.createDirectory`. So every `mkdirs` below bucket level sends at
least 2 OM requests: `InfoBucket` + `CreateDirectory`, plus `InfoVolume` +
`InfoBucket` per link hop.
The fetch does two jobs:
1. **Layout check.** `OzoneFSUtils.validateBucketLayout` rejects OBJECT_STORE.
2. **Auto-create.** When the volume or bucket is missing, `getBucket` creates
it (`InfoVolume` /
`CreateVolume` / `CreateBucket`), so `mkdirs /vol/bucket/dir` works on an empty
namespace. When the path
is only `/vol/bucket`, this is the whole operation.
Both must be handled before the fetch can be dropped, the same way HDDS-16760
handled listStatus's two
jobs.
### Why the rejection is opt-in
`OMDirectoryCreateRequest` is registered for `BucketLayout.OBJECT_STORE` (that
registration also serves
LEGACY), so `CreateDirectory` succeeds on an OBJECT_STORE bucket today. Its
callers:
* The S3 Gateway's `ObjectEndpoint` creates directories for a zero-byte key
ending in `/`, but only when
`bucket.getBucketLayout() == FILE_SYSTEM_OPTIMIZED`.
* `KeyLifecycleService`, on non-OBJECT_STORE buckets.
* `cli-shell`'s `DeleteKeyHandler`, on FSO buckets.
* Direct Java API callers of `OzoneBucket.createDirectory`, which may target
OBJECT_STORE.
the client asks for the check:
{code}
message CreateDirectoryRequest {
required KeyArgs keyArgs = 1;
// Set by file system clients: OM rejects the request with
// NOT_SUPPORTED_OPERATION if the resolved bucket layout is OBJECT_STORE.
optional bool requireFileSystemLayout = 2;
}
{code}
### Proposed change
*OM.* Check on the leader before Ratis, in
`OMDirectoryCreateRequest.preExecute` /
`OMDirectoryCreateRequestWithFSO.preExecute`
or right after `createClientRequest` in
`OMExecutionFlow.submitExecutionToRatis`. Use the link-resolved layout that
`BucketLayoutAwareOMKeyRequestFactory.createRequest` already read, so there is
no new lookup. Use the
`validateBucketLayout` message, wrapped in
`OMException(NOT_SUPPORTED_OPERATION)`. Do not put the check
in `createRequest` itself, because that also runs at apply time on every OM.
See the setTimes Jira for
the reasoning.
*Client.*
* For a path below bucket level, when `proxy.getOmVersion() >=` a new
`OzoneManagerVersion` constant
(placed after `LIST_STATUS_REJECTS_OBS(101)`, never below `ZDU(100)`), the
adapter sends `CreateDirectory`
with the field set, using a locally built handle or a `ClientProtocol` call. No
`getBucket()` first.
* **Auto-create on the error path:** on `VOLUME_NOT_FOUND` /
`BUCKET_NOT_FOUND`, call today's
`getBucket(ofsPath, true)`. It creates what is missing and applies the
client-side layout check. Then
retry `CreateDirectory` once. Only an `mkdirs` that creates a bucket pays the
extra round trip, and it
already pays several.
* A link whose source is missing must keep its current behaviour. `getBucket(…,
true)` on an orphan link
must not create a bucket under the link's name. The error path calls
`getBucket()` as a whole, so it
behaves exactly as today. The HDDS-16760 orphan-link handling has the same
property.
* `NOT_SUPPORTED_OPERATION` maps to `IllegalArgumentException`.
`FILE_ALREADY_EXISTS` keeps mapping to
`FileAlreadyExistsException`.
* `mkdirs /vol/bucket` (no key) keeps calling `getBucket(…, true)`, because
that call is the operation.
* Against any older OM, including one that reports ZDU, keep the current path.
### Compatibility
| | Behavior |
|---|---|
| New client, new OM, bucket exists | 1 RPC; OM rejects OBJECT_STORE |
| New client, new OM, bucket missing | failed `CreateDirectory` → `getBucket(…,
true)` (creates) → `CreateDirectory` |
| New client, older OM (incl. ZDU) | as today |
| Old client / S3 Gateway / lifecycle / direct `createDirectory`, new OM |
field absent ⇒ unchanged |
There is one new optional proto field, no new result code and no new
configuration.
Points for review:
* The BUCKET/READ ACL check from `getBucketDetails` is dropped.
`CreateDirectory` keeps its own ACL
checks (bucket WRITE / key CREATE).
* The audit log shows a failed `CREATE_DIRECTORY` before the `CREATE_BUCKET` in
the auto-create case.
## Testing
* Adapter (Mockito):
* Existing bucket: exactly one `CreateDirectory`, carrying the field, and no
`getBucket()`.
* Missing bucket: `CreateDirectory` fails, then `getBucket(…, true)`, then one
retry.
* Orphan link: today's result.
*`NOT_SUPPORTED_OPERATION` maps to `IllegalArgumentException`, and
`FILE_ALREADY_EXISTS` to
`FileAlreadyExistsException`.
* An older OM, including one reporting `ZDU`, falls back.
* OM request tests (both the FSO and non-FSO classes):
* With the field set, OBJECT_STORE is rejected before Ratis, and FSO and
**LEGACY** succeed.
* Without the field, OBJECT_STORE succeeds.
* A link to an OBJECT_STORE bucket is rejected.
* `AbstractRootedOzoneFileSystemTest`: `mkdirs` into a missing volume/bucket
still auto-creates both, and
`mkdirs` on an OBJECT_STORE bucket still fails with `IllegalArgumentException`.
Count RPCs in the
adapter test, not with OMMetrics deltas (HDDS-16513).
* Benchmark `TestOfsMkdirsRpcBenchmark`, same shape as
`TestOfsListStatusRpcBenchmark`:
* 200 FSO buckets, unique directory names, single-threaded and 10 threads.
* n=3 per arm, built and run strictly one after the other.
* Invariant: exactly 1 OM RPC per `mkdirs` into an existing bucket, instead of
2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
