smaheshwar-pltr opened a new pull request, #17986:
URL: https://github.com/apache/iceberg/pull/17986
Relates to apache/iceberg#9521.
## Before this PR
`BaseCommitService.offer` is called by each file-group worker when its
rewrite finishes. It extracts a batch of completed groups under `synchronized
(completedRewrites)`, then commits that batch after releasing the monitor:
```java
synchronized (completedRewrites) {
if (canCreateCommitGroup()) {
batch = ...;
}
}
if (batch != null) {
commitOrClean(batch);
}
```
The queue is protected, but the table commit is not. With
`max-concurrent-file-group-rewrites > 1`, workers from the same
`rewrite_data_files` action can issue concurrent commits to the same table.
Iceberg's optimistic commit protocol makes those commits conflict, retry, and
back off even when there is no external writer. If a worker exhausts
`commit.retry.num-retries`, `commitOrClean` deletes its newly written files and
drops the completed group.
There is also a bookkeeping window after a batch leaves `completedRewrites`
and before its token enters `inProgressCommits`. During that window, close-time
accounting can observe neither queued nor active work.
The performance workload uses Spark 4.1 on `local[*]`, a local Hadoop
catalog, JDK 21, and 32 cores. Each JMH single-shot iteration rebuilds a table
with 100,000 rows in 20 files, rewrites 10 two-file groups with partial
progress enabled and one group per commit, and verifies that all groups
committed and the table has 11 snapshots. The timing run uses 20 commit retries
so conflicts appear as latency rather than lost work.
## After this PR
One offering thread takes non-blocking drain ownership with
`ReentrantLock.tryLock()` and commits every ready batch serially. A competing
offer leaves its group queued and returns instead of waiting for the active
commit. The owner rechecks eligibility after releasing the lock so a group
offered during handoff is not stranded.
Batch extraction and in-progress publication now happen in the same queue
critical section, while commit and cleanup remain outside it. This keeps
close-time accounting live and allows timeout cleanup to abort queued groups
during a slow commit. Non-positive batch sizes are rejected so the drain loop
always makes progress. External writers are unchanged and still use Iceberg's
existing optimistic retry behavior.
On the workload above, serializing commits removes the retry backoff caused
by self-conflict:
| `max-concurrent-file-group-rewrites` | Before | After | Change |
| ---: | ---: | ---: | ---: |
| 1 | 1.643 s ± 0.105 | 1.605 s ± 0.120 | no measurable difference |
| 4 | 5.541 s ± 2.148 | 1.180 s ± 0.105 | 4.7× faster |
| 8 | 6.524 s ± 0.675 | 1.141 s ± 0.054 | 5.7× faster |
The values are mean ± 99.9% confidence interval in seconds per operation.
The one-worker control is unchanged. These are end-to-end results for this
local Hadoop catalog workload; its lock-manager retry contributes to the
measured before latency, so the multiplier is not catalog-independent.
A second run sets `commit.retry.num-retries=0`, making every self-conflict
an immediately observable failed commit. Across 10 trials with 4 rewrite
workers:
| | Before | After |
| --- | ---: | ---: |
| Groups committed per run | 3–4 of 10 | 10 of 10 |
| Runs that lost groups | 10 of 10 | 0 of 10 |
| Mean groups lost per run | 6.8 | 0 |
No self-conflict was observed after serialization in those trials.
## AI Tooling
This PR was written by Codex but reviewed by me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
