sezruby opened a new issue, #18020: URL: https://github.com/apache/iceberg/issues/18020
This proposes making concurrent UPDATE/MERGE on V3 deletion vectors fail less often when the operations touched disjoint rows, to gauge appetite before writing code. The core step builds on existing DV primitives; the follow-ups below are included so the direction reads as a whole design rather than a single case. Happy to move discussion to [email protected] if preferred. ## Background With one DV per data file, two concurrent operations touching the same data file conflict at the file level. The DV design discussion already noted this tradeoff — a single DV per file makes concurrent conflicts easier because the conflict becomes file-level rather than row-level: - https://lists.apache.org/thread/plsqgnlb0f84y5l5wf09qqtfmzdxbsxv - https://lists.apache.org/thread/l5f5tkogpzcyk9sptbbj7nxby9so9fjg Today `validateAddedDVs` fails the commit when a concurrent op added a DV for a data file this op also DV'd — purely at file granularity, regardless of whether the deleted positions actually overlap. It throws a `ValidationException`, which is not retryable (the commit retry loop is `onlyRetryOn CommitFailedException`), so the whole operation has to be recomputed and re-run. For two operations that modified **disjoint** rows in the same file, that is wasted work on a spurious conflict. Prior art: Delta Lake's row-level concurrency reduces concurrent-write conflicts on deletion-vector-enabled tables from file granularity to row granularity, so operations that modify disjoint rows in the same file do not conflict. This proposal pursues the same outcome for Iceberg V3 DVs. (https://docs.delta.io/latest/concurrency-control.html) ## Core proposal Refine `validateAddedDVs` from file granularity to position granularity. When a concurrent DV exists for a file this op also DV'd, load the three DVs' positions and test overlap of the newly-deleted positions: (dv_concurrent intersect dv_self) minus dv_base == empty where `dv_base` is the file's DV at this op's read snapshot (both written DVs already contain it). If the new positions are disjoint, there is no lost update: union the two into a single DV (`dv_self union dv_concurrent`) for the file and proceed, instead of failing. If they overlap, fail as today. What this reuses vs. what is new: - **Reused:** DV positions load from just the Puffin blob (`DVUtil.readDV` → `PositionDeleteIndex`, bitmap only, never the data file); bitmap union exists (`PositionDeleteIndex.merge`); the single-DV write path exists (`BaseDVFileWriter`, which records the superseded DV as a rewritten delete). - **New:** the position-overlap test (intersect/andNot over the bitmaps is not currently exposed — a small helper), plus wiring a **cross-operation** union into the validation path. The existing write-time DV merge only unions an operation's **own** duplicate DVs for a file; it never reads a concurrent or base DV, so unioning with the concurrent DV is new logic. This touches only the physical one-DV-per-file check; it does not change isolation semantics, which are enforced separately by `validateNoConflictingDeletes` / `validateNoConflictingData`. ## Conditions for soundness - **One DV per file preserved:** disjoint results union into a single DV, never a second parallel DV for the same file. - **Position stability:** DV positions are comparable only if the data file is unchanged. A concurrent compaction / copy-on-write that rewrote or removed the file makes positions meaningless → fail (`validateDataFilesExist` / `validateDeletedFiles` already guard this). - **DV-only:** bitmap overlap requires both deletes to be DVs on the same file. An equality delete or legacy position-delete file falls back to a conservative failure. - **Added rows are not ignored:** an UPDATE is delete-old + insert-new; the insert/phantom dimension stays with `validateAddedDataFiles`, so serializable isolation is unaffected. ## Planned refinements (sequenced; not in the core) A concurrent UPDATE/MERGE can conflict on three row-set axes: rows the winner deleted that I also deleted (core), rows the winner removed or rewrote out from under what I read, and rows the winner added that match what I read. The core covers the first at bitmap granularity. The rest, in order: **A. Row-granular read-scope delete check.** `validateNoConflictingDeletes` matches concurrently-added delete files against the conflict-detection filter at partition/metadata granularity, so it can also fail a disjoint UPDATE/MERGE. Making it row-granular raises a design question: compare against this op's own DV (write set, bitmap-only, snapshot-isolation lost-update), or against its exact read positions (which requires evaluating the read predicate against the file — a data read). The latter is what a serializable read-scope guarantee needs. The removed-file variant of this axis splits by how the row was removed: - a DV added to a still-present data file → the removed rows are exactly the winner's new DV positions, so overlap is answerable bitmap-only (same approach as the core); - a physical rewrite (compaction / copy-on-write) → positions are invalidated, so establishing which removed rows matched the read requires reading the old file contents (a data read; see B). **B. Value-exact checks that require a commit-time data read.** Two checks are metadata/metrics-only today and false-positive on predicates bounds cannot model: - `validateAddedDataFiles` uses inclusive min/max metrics, so it fails on e.g. `col % 7 = 3` even when no added row matches; - the physical-rewrite removed-file case above, where the un-deleted rows must be read to know if any matched the predicate. Evaluating the real predicate against file rows removes these false failures, but injects a data scan into a commit path that is metadata + (one) DV-bitmap only today — so this is a separate, heavier effort, naturally bounded by a byte budget. Net: the bitmap tier (core; A write-set; DV-on-live-file removal) reads no data. The value-exact tier (A read-set; B) reads data-file contents at commit and is foreign to the current commit path. ## Scope / non-goals - MoR (deletion-vector) tables only; copy-on-write is out of scope. - The win concentrates on expensive UPDATE/MERGE under contention; DELETE already tolerates concurrent deletes (`validateNoConflictingDeleteFiles` isn't required for DELETE). - No change to failing on genuine (row-overlap) conflicts. - **Concurrent compaction vs. DML is out of scope.** Today these do not reconcile: whichever commits second fails — a compaction that races a DML adding position deletes to files it rewrites fails with "found new position delete for replaced data file", and a DML whose referenced data file was rewritten fails with "Cannot commit, missing data files". Bitmap overlap cannot help, because positions do not survive a rewrite (position N in the old file is not position N in the compacted file). The mechanism that could reconcile it is V3 row lineage (`_row_id`), which is preserved when a row moves to a new file during a rewrite but is not consulted by any conflict validator today — so that is a separate, larger effort, not an extension of the DV-bitmap work here. ## Configuration An opt-in refinement plugs into the existing isolation infrastructure — `write.{delete,update,merge}.isolation-level` and `IsolationLevel {SERIALIZABLE, SNAPSHOT}` — rather than adding a parallel switch. ## Questions 1. Is the file-level false-failure in `validateAddedDVs` worth refining to position granularity? 2. Where should the disjoint-case union live: in the validation path (union in place, one commit attempt), or by turning the conflict into a retryable commit that re-reads and unions the base DV on each refreshed apply? The in-place option is the smaller change — the retry path today only reuses already-written files; re-reading and unioning the concurrent DV on retry would be additional plumbing on top of the same union logic. 3. For refinement A (the read-scope delete check), there are two ways to make it row-granular — different cost, different guarantee. Which is worth doing? - **Cheap (bitmap, no data read):** compare the winner's deleted positions only against the rows this op itself modified (its own DV). This catches a concurrent delete of a row I also updated — it prevents lost updates and gives snapshot isolation. It does **not** protect rows I only read. - **Exact (data read at commit):** compare against the rows this op actually read (its predicate evaluated against the file). This also protects read-only rows and preserves serializable isolation, but scans data-file contents in the commit path. Is the cheap snapshot-isolation variant useful on its own, is only the serializable variant worthwhile, or should both be offered and selected by the isolation-level config above? 4. Any prior art or earlier design decision I've missed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
