voonhous commented on code in PR #19485:
URL: https://github.com/apache/hudi/pull/19485#discussion_r3955536813
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -1744,22 +1744,43 @@ static void deltaStreamerTestRunner(HoodieDeltaStreamer
ds, HoodieDeltaStreamer.
static void deltaStreamerTestRunner(HoodieDeltaStreamer ds,
HoodieDeltaStreamer.Config cfg, Function<Boolean, Boolean> condition, String
jobId) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
- Future dsFuture = executor.submit(() -> {
+ try {
+ Future dsFuture = executor.submit(() -> {
+ try {
+ ds.sync();
+ } catch (Exception ex) {
+ log.warn("DS continuous job failed, hence not proceeding with
condition check for {}", jobId);
+ throw new RuntimeException(ex.getMessage(), ex);
+ }
+ });
try {
- ds.sync();
- } catch (Exception ex) {
- log.warn("DS continuous job failed, hence not proceeding with
condition check for {}", jobId);
- throw new RuntimeException(ex.getMessage(), ex);
+ TestHelpers.waitTillCondition(condition, dsFuture, 360);
+ } catch (Throwable failure) {
+ // Surefire runs this module with forkCount=1 and reuseForks=true, so
a continuous streamer left
+ // running here reads on into the next test, whose setup deletes
basePath and whose teardown closes
+ // the data generators underneath it. Stop it before letting the
failure out.
+ try {
+ ds.shutdownGracefully();
Review Comment:
Agreed, and fixed in `ae6065c` (this was raised against `8e9e54f`). The stop
now runs on its own thread with a 60s bound and cancels the ingest task if it
does not return, so the diagnostic `AssertionError` is thrown either way rather
than waiting on the 24h `awaitTermination`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
