voonhous commented on code in PR #19485:
URL: https://github.com/apache/hudi/pull/19485#discussion_r3956279167
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamerTestBase.java:
##########
@@ -763,34 +776,110 @@ static HoodieInstant
assertCommitMetadataForIncrSource(String expected, String t
return lastInstant;
}
+ /**
+ * Polls {@code condition} until it holds, the deltastreamer future
finishes, or the timeout expires.
+ *
+ * <p>On timeout the last error the condition threw is attached to the
failure, so the report names the
+ * assertion that never held rather than only this method.
+ */
static void waitTillCondition(Function<Boolean, Boolean> condition, Future
dsFuture, long timeoutInSecs) throws Exception {
- Future<Boolean> res = Executors.newSingleThreadExecutor().submit(() -> {
- boolean ret = false;
- while (!ret && !dsFuture.isDone()) {
- try {
- Thread.sleep(2000);
- ret = condition.apply(true);
- log.info("Condition completed successfully");
- } catch (Throwable error) {
- log.debug("Got error waiting for condition", error);
- ret = false;
+ AtomicReference<Throwable> lastError = new AtomicReference<>();
+ AtomicInteger completedEvaluations = new AtomicInteger();
+ ExecutorService executor = Executors.newSingleThreadExecutor();
+ try {
+ Future<Boolean> res = executor.submit(() -> {
+ boolean ret = false;
+ // The executor check matters as well as the interrupt flag: the
interrupt from shutdownNow is
+ // delivered once, and a condition that swallows it would otherwise
leave the flag clear and keep
+ // this thread polling for the lifetime of the JVM.
+ while (!ret && !dsFuture.isDone() &&
!Thread.currentThread().isInterrupted() && !executor.isShutdown()) {
+ try {
+ Thread.sleep(POLL_INTERVAL_MS);
+ ret = condition.apply(true);
+ completedEvaluations.incrementAndGet();
+ if (ret) {
+ log.info("Condition completed successfully");
+ }
+ } catch (InterruptedException interrupted) {
+ // Thread.sleep clears the interrupt flag when it throws, so
catching this with everything
+ // else would re-enter the loop. Restore the flag and stop; this
is not a condition failure,
+ // so it is deliberately not recorded as one.
+ Thread.currentThread().interrupt();
+ break;
+ } catch (Throwable error) {
+ log.debug("Got error waiting for condition", error);
+ lastError.set(error);
+ completedEvaluations.incrementAndGet();
+ ret = false;
+ }
+ }
+ return ret;
+ });
+ try {
+ res.get(timeoutInSecs, TimeUnit.SECONDS);
+ } catch (TimeoutException e) {
+ Throwable last = lastError.get();
+ Throwable cause = last == null ? e : last;
+ AssertionError failure = new AssertionError(
+ describeTimeout(last, completedEvaluations.get(),
timeoutInSecs), cause);
+ if (cause != e) {
+ // The condition's own error is the more useful cause, but the
fact that this was a timeout is
+ // still part of the diagnosis, so it is carried along rather than
dropped.
+ failure.addSuppressed(e);
}
+ throw failure;
}
- return ret;
- });
- res.get(timeoutInSecs, TimeUnit.SECONDS);
+ } finally {
+ // stop the polling thread: this method runs once per continuous-mode
test, so a leak accumulates
+ executor.shutdownNow();
+ }
+ }
+
+ /**
+ * Builds the timeout report. Which of the three shapes it takes is the
whole diagnostic: an error the
+ * condition threw, a condition that never completed an evaluation, or one
that kept returning false.
+ */
+ static String describeTimeout(Throwable last, int completed, long
timeoutInSecs) {
Review Comment:
Fixed in `9f3cd75`.
`describeTimeoutReportsAnErrorEvenWithNoCompletedEvaluation` calls it directly
with `(new AssertionError("boom"), 0, ...)` and asserts the error is reported
and the "No evaluation of the condition completed" text is not.
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/JavaTestUtils.java:
##########
@@ -28,7 +28,10 @@ public static boolean checkNestedExceptionContains(Throwable
t, String errorMsg)
Throwable throwable = t;
boolean res = false;
while (throwable != null) {
- if (throwable.getMessage().contains(errorMsg)) {
+ // String.valueOf rather than getMessage().contains: a null message
anywhere in the chain would
+ // otherwise NPE here and lose the failure the caller was trying to
assert on. A TimeoutException
+ // raised before its condition ever threw is one such case, and NPEs in
a chain are another.
+ if (String.valueOf(throwable.getMessage()).contains(errorMsg)) {
Review Comment:
Fixed in `9f3cd75`, and the helper itself changed: `String.valueOf` also
made an `errorMsg` of `"null"` match a message-less throwable, so it is a plain
null check now. New `TestJavaTestUtils` in hudi-common covers null head, null
link mid-chain, the `"null"` non-match, a head match and no match at all. 5/5
in 0.02s.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
