geserdugarov commented on PR #10362: URL: https://github.com/apache/hudi/pull/10362#issuecomment-1868303781
I don't understand what is happening with CI. I've changed 2 unit tests: - `TestJavaHoodieBackedMetadata`, from `hudi-client/hudi-java-client`, - `TestHoodieDeltaStreamer`, from `hudi-utilities`. Both are Java tests. ### Azure CI `hudi-client/hudi-java-client` is not included in the Azure CI. `hudi-utilities` is included in the Azure CI in `UT FT other modules` job at `UT other modules` stage. So, `TestHoodieDeltaStreamer` test is the only one, which could brake the Azure CI. But the last log from `UT other modules` stage is > [INFO] Running org.apache.hudi.utilities.sources.TestSqlSource before > This job was abandoned. We have detected that logs from the agent may have not finished uploading. We have included our in-memory record of all log lines uploaded before we lost contact with the agent: My change in this test couldn't brake it this way, only test failure is possible. Maybe with my MR test ordering is changed and the unit tests running is hung at `@AfterAll/Each` of some test class or at `@BeforeAll/Each` of another one. But I couldn't reproduce the problem locally. This part of CI job is passing without any problem locally. ### GitHub Actions My change in `TestJavaHoodieBackedMetadata` from `hudi-client/hudi-java-client` should affect only `test-hudi-hadoop-mr-and-hudi-java-client` job, but not `test-spark`. And I see that `test-hudi-hadoop-mr-and-hudi-java-client` is ok, but there are hungs in `test-spark` and failure at `TestDataSourceForBootstrap` scala test after > 2023-12-23T04:01:07.0996155Z 4017081 [Executor task launch worker for task 372] ERROR org.apache.spark.executor.Executor [] - Exception in task 0.0 in stage 133.0 (TID 372) 2023-12-23T04:01:07.0997116Z java.lang.OutOfMemoryError: GC overhead limit exceeded @danny0405 , @yihua Could you, please, give me any suggestions what else can I try? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
