Rangsh commented on issue #12058:
URL: https://github.com/apache/seatunnel/issues/12058#issuecomment-5569177382
@nzw921rx thank you for clarifying the expectations on this issue — and
sorry for my earlier note that “this issue has been resolved.” That was
premature and does not match the issue goals.
### How I now understand #12058 vs #12081
Per the issue description and your comments here / on #12081:
1. **#12058 is primarily an investigation task.** The immediate goal is to
explain the within-run latency variance on `checkpointIdAtomicIncrement` /
`checkpointOverviewIncrementalUpdate` with reproducible evidence (same JDK,
machine, JMH settings, state-store config). There is no expectation to close
this issue via a PR until that evidence chain is in place. I’ll follow the
methodology you expanded in #12086.
2. **#12081 is a related correctness PR, not a CV resolution.** After
discussion with you, @DanielLeens, and @SEZ9, the PR is now framed as **Related
to #12058** (not `Fixes`). It lands independent WAL wait-path correctness fixes
(`RequestFuture` / `WALWorkHandler` / shared batch deadline) plus collapsing
redundant `HdfsWriter.flush()` sync calls. Those stand on their own merits, but:
- merging #12081 must **not** close this issue;
- the sync-collapse remains a **working hypothesis** for the CV symptom,
not a confirmed same-machine isolated root cause;
- call-count tests are method-call-count parity only, not disk-sync-count
parity or proof that CV is fixed.
So please keep #12058 open for the variance investigation, independent of
whether #12081 merges.
### Next step on this issue (investigation-first)
As you suggested, I’ll start with **one** method
(`checkpointOverviewIncrementalUpdate`), keep the existing measurement
settings, and post here:
- local reproduction (Score / Error / CV) on a consistent machine
- initial observations comparing normal vs slower iterations
- what we can already rule in/out (including `file:///` LocalFileSystem
`hsync`/`hflush` semantics vs real HDFS)
Existing fork-Actions evidence from #12081 remains useful input, but I will
not treat it as closing evidence for this issue. Happy to discuss findings here
before any further production change aimed at CV.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
