SEZ9 commented on PR #12313: URL: https://github.com/apache/seatunnel/pull/12313#issuecomment-5843196629
Thanks both — this is exactly the scrutiny the fixture needed. @SEPURI-SAI-KRISHNA, you're right on the frequency: the free window occurs once per split, not once per second. With `row.num = 2000000` / `split.num = 500` each 4000-row split is emitted while the queue is full, so at ~500 rows/sec (`write_delay_ms = 2`) a split takes ~8 s to drain before the 1 s `Thread.sleep(1000L)` fires. The window recurs roughly every 9 s, and against `checkpoint.interval = 15000` the headroom is ~1.7x, not the 15x the current wording implies. I'll rewrite the class Javadoc to state the per-split cadence, the ~9 s recurrence and the real margin, and note that `row.num`, `split.num`, `write_delay_ms` and `checkpoint.interval` are coupled so the next edit is made against the actual budget. Once that's pushed, could you take a quick look at the new wording? @goutamadwant, thanks for the loop runs across JDK 8 and 11 against `deb16a3c3`. Agreed on how to read them: 0 failures shows the fixture change doesn't break the test, and nothing more — I'm not claiming local evidence that it fixes the flake. On coverage, I agree. With 4000-row splits the reader releases the lock once per split, so this test no longer exercises barrier injection losing the lock to back-to-back polls. Keeping a variant on the original `split.num = 1` workload is the right way to keep that engine-side behaviour covered. Two questions before I add it: 1. Since that workload is the one that starves on `dev` today, would you prefer the `split.num = 1` variant be added in this PR but kept disabled until the engine-side change lands, or added directly alongside that change so it's enabled together with the fix it covers? 2. If it goes here, do you want it as a separate test method in `BackpressureSlowSinkIT`, or a parameterised variant of the existing method? Plan on my side: push the Javadoc correction, then add the `split.num = 1` variant per whichever option you pick. <!-- streview-comment:1331 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
