[
https://issues.apache.org/jira/browse/HDDS-15860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-15860:
------------------------------
Description:
h2. Problem
Incremental snapshot defragmentation computes RocksDB SST files that differ
between two snapshot versions and applies the resulting changes to a temporary
checkpoint database.
When a table has exactly one candidate SST and the snapshot version is greater
than zero, {{SnapshotDefragService}} bypasses the normal spill-and-rewrite path
and passes the live SST directly to {{{}Table.loadFromFile{}}}.
The candidate is a live RocksDB SST selected at file granularity, not an
external SST containing an exact logical bucket delta. This causes two observed
failure modes on current master with RocksDB 10.10.1.1 (or with previous 7.7.3):
* A DB-generated flush SST is rejected by external-file ingestion with
{{External file version not found}} (because DB-generated SST lacks
rocksdb.external_sst_file.versionrocksdb.external_sst_file.version and
rocksdb.external_sst_file.global_seqno properties)
* An SST previously ingested into a live database can be accepted, but it
installs the value stored in that live SST instead of the value from the
current snapshot.
The direct path also bypasses bucket-prefix filtering, current-versus-previous
snapshot comparison, and tombstone generation.
h2. Trigger condition
* The snapshot version is greater than zero.
* Exactly one candidate SST is returned for an incremental table.
Once triggered, snapshot defragmentation cannot correctly advance the affected
snapshot to its next defragmented version.
h2. Proposed fix
Remove the single-file direct-ingestion shortcut. All candidate SST sets,
including singleton sets, should pass through
{{{}spillTableDiffIntoSstFile{}}}. This existing path:
* Filters keys by the target bucket prefix.
* Compares values in the current and previous snapshots.
* Emits only actual updates and insertions.
* Emits tombstones for deletions.
* Creates a fresh external SST suitable for RocksDB ingestion.
h2. Test coverage
* Update the existing incremental-defragmentation unit test to expect
rewriting for every snapshot version and candidate-file count.
* Add a real RocksDB regression test covering both DB-generated live SSTs and
SSTs previously ingested into a live database.
was:
h2. Problem
Incremental snapshot defragmentation computes RocksDB SST files that differ
between two snapshot versions and applies the resulting changes to a temporary
checkpoint database.
When a table has exactly one candidate SST and the snapshot version is greater
than zero, {{SnapshotDefragService}} bypasses the normal spill-and-rewrite path
and passes the live SST directly to {{Table.loadFromFile}}.
The candidate is a live RocksDB SST selected at file granularity, not an
external SST containing an exact logical bucket delta. This causes two observed
failure modes on current master with RocksDB 10.10.1.1:
* A DB-generated flush SST is rejected by external-file ingestion with
{{External file version not found}}.
* An SST previously ingested into a live database can be accepted, but it
installs the value stored in that live SST instead of the value from the
current snapshot.
The direct path also bypasses bucket-prefix filtering, current-versus-previous
snapshot comparison, and tombstone generation.
h2. Trigger condition
* The snapshot version is greater than zero.
* Exactly one candidate SST is returned for an incremental table.
Once triggered, snapshot defragmentation cannot correctly advance the affected
snapshot to its next defragmented version.
h2. Proposed fix
Remove the single-file direct-ingestion shortcut. All candidate SST sets,
including singleton sets, should pass through {{spillTableDiffIntoSstFile}}.
This existing path:
* Filters keys by the target bucket prefix.
* Compares values in the current and previous snapshots.
* Emits only actual updates and insertions.
* Emits tombstones for deletions.
* Creates a fresh external SST suitable for RocksDB ingestion.
h2. Test coverage
* Update the existing incremental-defragmentation unit test to expect rewriting
for every snapshot version and candidate-file count.
* Add a real RocksDB regression test covering both DB-generated live SSTs and
SSTs previously ingested into a live database.
> Snapshot defrag fails when directly ingesting a single live RocksDB SST
> -----------------------------------------------------------------------
>
> Key: HDDS-15860
> URL: https://issues.apache.org/jira/browse/HDDS-15860
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> h2. Problem
> Incremental snapshot defragmentation computes RocksDB SST files that differ
> between two snapshot versions and applies the resulting changes to a
> temporary checkpoint database.
> When a table has exactly one candidate SST and the snapshot version is
> greater than zero, {{SnapshotDefragService}} bypasses the normal
> spill-and-rewrite path and passes the live SST directly to
> {{{}Table.loadFromFile{}}}.
> The candidate is a live RocksDB SST selected at file granularity, not an
> external SST containing an exact logical bucket delta. This causes two
> observed failure modes on current master with RocksDB 10.10.1.1 (or with
> previous 7.7.3):
> * A DB-generated flush SST is rejected by external-file ingestion with
> {{External file version not found}} (because DB-generated SST lacks
> rocksdb.external_sst_file.versionrocksdb.external_sst_file.version and
> rocksdb.external_sst_file.global_seqno properties)
> * An SST previously ingested into a live database can be accepted, but it
> installs the value stored in that live SST instead of the value from the
> current snapshot.
> The direct path also bypasses bucket-prefix filtering,
> current-versus-previous snapshot comparison, and tombstone generation.
> h2. Trigger condition
> * The snapshot version is greater than zero.
> * Exactly one candidate SST is returned for an incremental table.
> Once triggered, snapshot defragmentation cannot correctly advance the
> affected snapshot to its next defragmented version.
> h2. Proposed fix
> Remove the single-file direct-ingestion shortcut. All candidate SST sets,
> including singleton sets, should pass through
> {{{}spillTableDiffIntoSstFile{}}}. This existing path:
> * Filters keys by the target bucket prefix.
> * Compares values in the current and previous snapshots.
> * Emits only actual updates and insertions.
> * Emits tombstones for deletions.
> * Creates a fresh external SST suitable for RocksDB ingestion.
> h2. Test coverage
> * Update the existing incremental-defragmentation unit test to expect
> rewriting for every snapshot version and candidate-file count.
> * Add a real RocksDB regression test covering both DB-generated live SSTs
> and SSTs previously ingested into a live database.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
