junaiddshaukat opened a new issue, #39452: URL: https://github.com/apache/beam/issues/39452
### What `ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch` is sickbayed in the Kafka Streams runner's `validatesRunner` task (added in the ParDoTest sweep PR). This issue tracks un-sickbaying it. ### Why it's sickbayed The test runs a DoFn whose `@StartBundle` throws, and asserts `run()` surfaces the user's error. Instead the run blocks for the SDK data service's three-minute timeout and then fails with a `TimeoutException` rather than the user's exception. The DoFn fails during bundle setup, so the SDK harness never opens its data stream. Meanwhile `SdkHarnessClient.newBundle` has already sent the `ProcessBundleRequest` and then blocks in `GrpcDataService.createOutboundAggregator`, which waits up to three minutes (`connectedClient.get(3, TimeUnit.MINUTES)`) for that data stream. So the error on the control channel never gets a chance to surface — the run just waits out the timeout. This is in shared `java-fn-execution` code, not runner-specific: - The Flink runner sickbays all of `LifecycleTests` (`runners/flink/flink_runner.gradle`), with the note that its errors are non-deterministic and the real cause is only in the logs. - The Prism runner sickbays all three of the `LifecycleTests` error tests (`runners/prism/java/build.gradle`). The `@ProcessElement` and `@FinishBundle` variants pass in the Kafka Streams runner, because by the time those run the data stream is already established. ### Possible directions - Detect a bundle that failed on the control channel and stop waiting on the outbound aggregator, so the control-channel error surfaces instead of the data-stream timeout. - Or shorten the wait for this test setup so the run fails fast with the real cause. Either likely belongs in `java-fn-execution` and would benefit the Flink and Prism runners too, so worth checking with those owners before changing shared code. Part of #18479. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
