smengcl commented on code in PR #11434: URL: https://github.com/apache/ozone/pull/11434#discussion_r4279814640
########## hadoop-hdds/docs/content/design/append.md: ########## @@ -0,0 +1,1223 @@ +--- +title: Append support in Ozone +summary: Reopen an existing file for a single appender while preserving its committed data and snapshot references. +date: 2026-10-02 +jira: HDDS-4333 +status: draft +author: Siyao Meng +--- +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +## Status and tracking + +This is a proposal for discussion, not an implemented or accepted feature. The code baseline is Apache Ozone master +`29b9853270342eea4d065b1f14c332ad384f83e5`. The behavioral choices are specified below; remaining implementation details +and activation checks are listed in the [implementation plan](#implementation-plan-and-activation). + +[HDDS-4333](https://issues.apache.org/jira/browse/HDDS-4333), "Ozone supports append operation", is the existing append +umbrella. Its design subtask is [HDDS-4373](https://issues.apache.org/jira/browse/HDDS-4373). The older combined append/truncate +issue is [HDDS-3714](https://issues.apache.org/jira/browse/HDDS-3714); its append subtask, +[HDDS-4240](https://issues.apache.org/jira/browse/HDDS-4240), was resolved as a duplicate with a pointer to HDDS-4333. +The earlier [design PR #1515](https://github.com/apache/ozone/pull/1515) was closed without merging. + +This draft incorporates the useful requirements and review discussion from PR #1515, the new-block approach from +Wei-Chiu Chuang's later append proposal, and the review questions summarized below. It does not imply that the earlier +authors or reviewers endorse these new decisions. + +Draft prepared with assistance from Codex (GPT-6 Astra) and Claude Code (Fable 5.1). + +## Problem statement + +OzoneFS currently rejects `FileSystem.append()`. Applications that reopen transaction logs or resume existing output files +cannot use that operation through `ofs://` or `o3fs://`. Applications may have additional requirements such as `truncate()`; +adding append alone does not establish compatibility with every HDFS application. + +Reopen a closed file for one appender, keep its existing blocks immutable, and allocate new blocks for appended bytes. +Successful close publishes the extended file. Supported replicated streams also provide their existing `hflush`/`hsync` +behavior. EC append publishes on close and does not require implementing EC `hsync`. + +```text +Before append: file -> [A, B] +During append: committed prefix [A, B]; session writes new block C +After close: file -> [A, B, C] +``` + +The hard parts are session ownership, preserving the prefix through failure and cleanup, and retaining blocks referenced +by snapshots. These are prerequisites, not follow-up optimizations. + +## Goals and non-goals + +Goals: + +- Support the ordinary Hadoop filesystem append API and append builder through both OzoneFS URI schemes. +- Preserve the existing prefix, file identity, attributes, replication configuration, and encryption information. +- Admit one appender, fence stale sessions, and make retry and recovery behavior explicit. +- Support RATIS, RATIS Streaming, and EC using existing writers and SCM allocation. +- Support snapshots, quotas, deletion, and recovery without reclaiming referenced data. +- Reuse layout-aware code for FSO and LEGACY rather than imposing an FSO-only restriction on the shared write path. + +Non-goals: + +- Reopening old blocks or closed containers, filling the previous partial block, or changing existing prefix bytes. +- `truncate`, `concat`, random writes, concurrent appender merging, or automatic small-block compaction. The choices that + keep these possible later are listed under [compatibility with future file editing](#compatibility-with-future-file-editing). +- Native object or S3 append in OBJECT_STORE buckets. +- EC `hflush`/`hsync`, or reconstruction of an arbitrary uncommitted EC suffix after writer failure. +- A throughput guarantee for repeated small append/close cycles. Such cycles remain correct but may be expensive. + +## Filesystem contract + +Implement `append(Path, int, Progressable)` so the ordinary overloads and `appendFile(...).build()` use the same path. +Return an `FSDataOutputStream` positioned at the existing EOF. Append does not create a missing file or directory. +Reject directories, snapshot paths, unsupported layouts or options, and requests without write permission. +Preserve existing ACL checks, authentication, and block-token issuance. + +An independently issued append cannot acquire ownership while another writer holds it, including in the same client. +Only renewable append sessions participate in automatic recovery within a bounded `FileSystem.append()` call. Ordinary +create/overwrite conflicts, including hsync-create streams, fail immediately and require application-directed recovery or +cleanup. [Session RPCs and retries](#session-rpcs-and-retries) specifies early conflicts, deadlines, and idempotent admission. +An explicit `recoverLease()` call remains available; no OM RPC waits indefinitely for a writer. + +The committed prefix remains readable throughout append. Before any suffix publication, a reader opening the file sees +the original contents. A reader opening after successful close sees the extended contents. After a replicated append +hsync, a newly opened live reader can read at least the published contents and may discover additional datanode-visible +bytes in an already published under-construction suffix block. File status may therefore lag what a live reader can read. +Observing those extra bytes does not add a recovery guarantee before successful sync or close. This does not expose private +allocations absent from the published block list. An already-open reader is not required to discover later appends +automatically. EC remains close-only; snapshot reads remain bounded by their captured lengths. + +An empty append preserves contents and length and does not retain a zero-length block. Allocate lazily when practical; +otherwise reclaim unused allocations through the existing cleanup mechanism after excluding the committed prefix. +Opening a session does not change the data modification time; successful publication of nonempty appended data does. + +`CommonPathCapabilities.FS_APPEND` must reflect the supported path, layout, server version, and feature state. Append does +not imply `hflush` or `hsync`; those remain stream capabilities. Validate the Hadoop shell `-appendToFile` and HTTPFS path +where they delegate to the filesystem API. Unsupported append flags must fail explicitly. Review Comment: The main use cases are reopening existing transaction logs during recovery and resuming existing output files. In the case of Solr 8.11.4, append reopens old tlogs during startup/recovery. Normal ingestion keeps its stream open rather than calling append per record. Frequent writes or hsync calls within that stream therefore do not each create a new block. Repeated reopen/close workloads will need separate measurements because they do create small blocks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
