[
https://issues.apache.org/jira/browse/HDDS-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy reassigned HDDS-17714:
-------------------------------------
Assignee: Andrey Yarovoy
> OFS getFileStatus should not refresh pipelines or return block locations
> ------------------------------------------------------------------------
>
> Key: HDDS-17714
> URL: https://issues.apache.org/jira/browse/HDDS-17714
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Filesystem
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> *Description*
> `BasicRootedOzoneFileSystem.getFileStatus` sends `GetFileStatus` with
> `headOp=false`. For a file, OM then:
> - calls `refresh(fileKeyInfo)`, which looks up every block's container
> pipeline (`KeyManagerImpl.getOzoneFileStatusFSO`);
> - optionally sorts the datanodes;
> - returns the full `KeyLocationList`.
> The client parses it (`OmKeyInfo.getFromProtobuf` →
> `OmKeyLocationInfo.getFromProtobuf` streams), builds a `BlockLocation[]`, and
> returns a
> `LocatedFileStatus`.
> HDFS `getFileStatus` returns no block locations; callers that need them call
> `getFileBlockLocations` or `listLocatedStatus`. Most
> `getFileStatus` callers use only length, type, mtime and owner. HBase is the
> extreme case. Every `StoreScanner.trySwitchToStreamRead`
> reopens each store file, and `StoreFileInfo.createReaderContext` calls
> `fs.getFileStatus(path)` only for `getLen()`.
> *Follow-up to HDDS-15678 / HDDS-15877*
> - **HDDS-15678** built the server side. With `headOp=true`, OM skips
> `refresh`/`sortDatanodes`, and
> `OzoneManagerRequestHandler.getOzoneFileStatus` clears `KeyLocationList` from
> the response. On the client it switched ofs
> `isFile`/`isDirectory` to head ops.
> - **HDDS-15877** did the same for o3fs and the remaining *type-only* callers
> (trash roots, shell `DeleteKeyHandler`).
> - Both deliberately left the public `FileSystem.getFileStatus(Path)` on
> `headOp=false`, because its result is a `LocatedFileStatus` that
> `getFileBlockLocations(FileStatus)` reads. That is the call HBase makes on
> every store-file reopen, and still the call that pays the refresh and
> returns all block locations.
> This Jira finishes the job for `getFileStatus(Path)`. It returns a plain
> `FileStatus` (like HDFS), and `getFileBlockLocations` fetches the
> locations only when they are actually requested.
> *Evidence (HBase YCSB on EC, RegionServer async-profiler, 6×60s per arm)*
> - `getFileStatus` is 12.5–14.6% of busy RS handler time, all spent waiting on
> OM.
> - Client-side status conversion is 1.2–1.6% of RS CPU (`toFileStatusAdapter`,
> `getBlockLocations`, `OmKeyInfo.getFromProtobuf` for the status).
> Parsing the `GetFileStatusResponse` `KeyLocationList` is ~3% of the reopen
> subtree.
> - OM-side cost (`refresh` + response encode) is not visible from RS profiles.
> **Measure on OM before and after.**
> *Proposed fix*
> 1. `BasicRootedOzoneFileSystem.getFileStatus(Path)` (and
> `BasicOzoneFileSystem`) call the adapter with `headOp=true`. They return a
> plain
> `FileStatus` (no `BlockLocation[]`). Length, mtime, owner, replication, EC
> policy and encryption flags come from the same `KeyInfo` fields as
> today.
> 2. `getFileBlockLocations(FileStatus, start, len)`: when the status is not a
> `LocatedFileStatus` and is a file, fetch a full status (`headOp=false`)
> and compute the range from it, instead of `super.getFileBlockLocations`
> (which returns a fake `localhost` location).
> 3. Override `getFileBlockLocations(Path, start, len)` to fetch the full
> status directly, so a path-based call stays at **1** RPC.
> 4. `listStatus` / `listLocatedStatus` / `listFiles` are unchanged; they keep
> their own location handling.
> 5. Optional escape hatch: `ozone.fs.getfilestatus.include.block.locations`
> (default `false`) restores today's `LocatedFileStatus` from
> `getFileStatus`.
> {code:java}
> // BasicRootedOzoneFileSystem
> public FileStatus getFileStatus(Path f) throws IOException {
> return TracingUtil.executeInNewSpan("ofs getFileStatus",
> () -> convertFileStatus(getFileStatusAdapter(f, !includeBlockLocations)));
> }
> {code}
> *Old OM compatibility*
> An OM without HDDS-15678's response stripping still returns locations on a
> head op, but they are unrefreshed (no pipeline). The client must
> ignore `KeyLocationList` whenever it sent `headOp=true`, whatever the OM
> version. With that rule no version gate is needed. Old OMs still
> skip `refresh` on head ops (that check predates HDDS-15678).
> *Compatibility / side effects*
> - **Behavior change:** `getFileStatus` no longer returns a
> `LocatedFileStatus`.
> - A caller that casts the result to `LocatedFileStatus` breaks; in-tree,
> check `ozonefs` tests and the `BasicOzoneFileSystem` copy.
> - A caller that does `getFileStatus` then `getFileBlockLocations(status)` now
> pays 2 OM RPCs instead of 1. MapReduce/Spark/Hive split
> planning uses `listLocatedStatus`, so it is unaffected; one-off single-file
> planners pay +1 RPC.
> - No proto change. No new OM version.
> *Tests*
> - Unit (adapter, Mockito): `getFileStatus` sends `headOp=true`. The result
> has the correct length/mtime/owner/EC fields and is not a
> `LocatedFileStatus`. `getFileBlockLocations(status)` sends exactly one
> `GetFileStatus` with `headOp=false` and returns real locations.
> `getFileBlockLocations(path)` sends exactly one RPC.
> - Integration: `TestRootedOzoneFileSystem` / `TestOzoneFileSystem` locality
> tests still return real datanode hosts. EC and Ratis keys,
> encrypted keys.
> - Benchmark: `getFileStatus` latency/throughput A/B on a multi-block EC key
> (n=3 per arm, sequential), with OM CPU per call.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
