voonhous commented on code in PR #19202:
URL: https://github.com/apache/hudi/pull/19202#discussion_r3585927231
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java:
##########
@@ -4246,11 +4281,37 @@ private static boolean
isAcceptableTerminalFailure(Throwable e) {
if (isNullPointerException(cur) && containsReadNextRowGroupFrame(cur)) {
return true;
}
+ // Cause #4 (see the call site): the CDC-iterator teardown twin. A
close() on the split-fetcher
+ // thread races the task-thread drain of a queued batch. The production
read path is synchronized
+ // (CdcFileSplitsIterator), so this surfaces as a NoSuchElementException
- or, at adjacent timing,
+ // an NPE - from a CdcIterators nested iterator, not a torn read. Scoped
to those exact frames so
+ // genuine CDC read bugs (which are deterministic, not teardown-timed)
still fail the test.
+ if ((isNullPointerException(cur) || isNoSuchElementException(cur))
+ && containsCdcIteratorTeardownFrame(cur)) {
+ return true;
+ }
cur = cur.getCause();
}
return false;
}
+ /**
+ * Whether {@code e} (or any of its causes) is a {@link TimeoutException}
from
+ * {@link org.apache.flink.table.api.TableResult#await(long, TimeUnit)} -
i.e. the await window elapsed
+ * before the sink reached its expected row count and threw
+ * {@link CollectSinkTableFactory.SuccessException}. Unlike the
teardown-race causes in
+ * {@link #isAcceptableTerminalFailure} the job is still running (it never
terminated), so the caller
+ * cancels it and retries the read rather than swallowing it.
+ */
+ private static boolean isAwaitTimeout(Throwable e) {
+ for (Throwable cur = e; cur != null; cur = cur.getCause()) {
Review Comment:
`TableResult.await()` throws its timeout **bare**; anything wrapped in an
`ExecutionException` is a job failure. Walking the whole cause chain
misclassifies real failures that embed a `TimeoutException` (checkpoint expiry,
RPC timeout) as a slow shard - cancelled, retried, and finally reported as a
row-count mismatch with the real cause discarded. Suggest top-level `e
instanceof TimeoutException` only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
