peterxcli opened a new pull request, #11140:
URL: https://github.com/apache/ozone/pull/11140
## What changes were proposed in this pull request?
S3 `CopyObject` in Ozone streams every byte of the source key from the
datanodes
through the S3 Gateway and writes it back as a new key, so copy latency is
proportional to object size. There is no server side copy RPC in OM, which is
why `ozone sh key cp` is a client side stream copy for the same reason.
This adds an OM `CopyKey` request that creates the destination key from the
source key's committed block list, so a copy moves no data at all.
### Why this is not just a new request
Committed blocks are owned by exactly one key and there is no reference
counting in OM or SCM. Deleting or overwriting either key would move the
shared
blocks to `deletedTable`, `KeyDeletingService` would hand them to SCM, and
the
surviving key would silently stop being readable, because a datanode asked to
delete an already deleted block succeeds quietly. The one existing
protection,
`OMKeyRequest.filterOutBlocksStillInUse`, only compares against the new
version
of the *same* key.
### Approach
* an optional `sharedBlockGroupId` on `KeyInfo` (field 23). Zero, the default
for every existing key, means the key owns its blocks exclusively, so no
database needs backfilling. A non-zero value is the `objectID` of the
lineage
root, and copies of copies join the group their source already belongs to.
* a `sharedBlockGroupTable` column family holding **one sharer count per copy
lineage, not one row per block**: copying a key with ten thousand blocks
costs
a single row.
* `KeyDeletingService` consults that count at the single point where
`deletedTable` entries become SCM block deletions. While a sharer outside
the
batch is alive, the blocks are withheld from the SCM call and the entry
gets a
synthetic success, exactly as empty keys already do, so the metadata is
purged
and the blocks are not. When the whole group dies together, one member
releases the blocks and the rest are withheld so SCM is told once.
* the count drop rides the same Ratis transaction as the purge, carried by
the
final `PurgeKeysRequest` batch on purpose: losing that batch after the rows
were purged leaks blocks, which an audit can reclaim, whereas decrementing
first and then failing to purge would let a retry decrement twice and
release
blocks a live key still uses.
**No block release producer changes**, because the tag travels with
`OmKeyInfo`
into `deletedTable` through `OmUtils.prepareKeyForDelete`. Every failure
direction degrades to a leak rather than to data loss, which is the main
reason
for preferring this over a per-block reference count that plain delete, bulk
delete, commit overwrite, multipart complete, FSO directory purge and open
key
cleanup would each have to maintain correctly.
A copy still falls back to reading and rewriting the data when the source is
encrypted (block data is ciphertext under the source key's own DEK and IV),
GDPR
enforced (erasure destroys the key's own secret), or hsync active, when the
replication config would change (the read path picks the EC or Ratis stream
from
the key's own config), and for `UploadPartCopy`, which cannot express a byte
range as whole object sharing.
Design doc: `hadoop-hdds/docs/content/design/server-side-copy-key.md`
### Scope, and what is deliberately missing
This is a **proof of concept**, opened as a draft for design feedback rather
than for merge. It covers `OBJECT_STORE` buckets, same bucket, no destination
overwrite.
Required before it could merge, and not done here:
* **`OMLayoutFeature` gating.** This is the load-bearing guard: an older OM
treats the tag as an unknown proto field and would reclaim shared blocks
normally, which is silent data loss. Finalization is the only thing that
can
prevent a downgrade from doing that.
* cross-bucket copy (needs ordered multi-bucket locking), destination
overwrite,
FSO buckets, `OzoneManagerVersion` negotiation, an `ozone repair om` audit
that
rebuilds counts by scanning for tags, Recon awareness, and S3 Gateway
wiring
with fallback.
### Relation to HDDS-569
[HDDS-569](https://issues.apache.org/jira/browse/HDDS-569) proposed the same
thing in 2018 and states the copy "in ozone world, this is just a metadata
change", but its two attached patches define proto messages only, and the
block
reclaim problem above was never addressed. Its proto predates
Ratis-replicated
OM requests, FSO and snapshots, so nothing there is reusable today.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16319
## How was this patch tested?
* New integration test in the existing `TestKeyPurging` suite,
`testCopiedKeyKeepsBlocksUntilLastSharerIsDeleted`: copies a key, asserts
both
keys name the same blocks with different `objectID`s and a shared group id
with a sharer count of two, deletes the source, waits for the count row to
disappear, and asserts the copy still reads back the original bytes before
deleting it too. The existing test in that class still passes.
* `BenchmarkCopyKey` compares both copy paths on a three-datanode
`MiniOzoneCluster` and separately verifies that source and copy have
identical
MD5 digests. It is named `Benchmark` rather than `Test` so surefire does
not
pick it up in CI.
| object size | read-and-rewrite copy | CopyKey | ratio |
|-------------|----------------------|---------|-------|
| 1 MiB | 117 ms | 7.5 ms | 16x |
| 16 MiB | 889 ms | 8.2 ms | 108x |
| 64 MiB | 2 983 ms | 6.7 ms | 446x |
| 256 MiB | 11 100 ms | 9.2 ms | 1204x |
The shape is the result rather than the ratios: `CopyKey` shows no trend
across a 256-fold size range because it is one OM Ratis transaction
whatever
the key holds. The ratios overstate a real cluster, where the byte copy
beats
the 18 to 23 MiB/s a single host manages; projecting from a realistic
100 MiB/s gives roughly 320x at 256 MiB and 1280x at 1 GiB.
* `TestOmMetadataManager`, `TestBucketLayoutAwareOMKeyFactory`,
`TestOMKeyPurgeRequestAndResponse` and `TestSnapshotDiffValueParser` pass
with
the updated expectations.
* `checkstyle.sh` clean; `rat.sh` reports nothing for the new files.
Generated-by: Claude Code (claude-fable-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
