[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raymond Xu closed HUDI-3054. ---------------------------- > Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter > ----------------------------------------------------------------------- > > Key: HUDI-3054 > URL: https://issues.apache.org/jira/browse/HUDI-3054 > Project: Apache Hudi > Issue Type: Sub-task > Components: Testing > Reporter: sivabalan narayanan > Assignee: Manoj Govindassamy > Priority: Major > Labels: pull-request-available > Fix For: 0.11.0 > > > Ref: > [https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_apis/build/builds/4428/logs/21] > > {code:java} > 2021-12-17T11:39:57.1645757Z [INFO] Running > org.apache.hudi.client.TestHoodieClientMultiWriter > 2021-12-17T11:39:57.3453991Z 339506 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit8865530218583640556/dataset/.hoodie/metadata > 2021-12-17T11:39:57.3984328Z 339559 [dispatcher-event-loop-5] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:39:57.5278608Z 339689 [dispatcher-event-loop-2] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:39:57.9783107Z 340139 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit8865530218583640556/dataset/.hoodie/metadata > 2021-12-17T11:39:57.9927490Z 340154 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit8865530218583640556/dataset/.hoodie/metadata > 2021-12-17T11:40:10.1428665Z 352304 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:40:10.9930023Z 353149 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:40:11.4294603Z 353590 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit3262960667280061850/dataset/.hoodie/metadata > 2021-12-17T11:40:11.4763085Z 353637 [dispatcher-event-loop-5] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:40:11.6014876Z 353762 [dispatcher-event-loop-2] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:40:12.0892513Z 354250 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit3262960667280061850/dataset/.hoodie/metadata > 2021-12-17T11:40:12.1061317Z 354267 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit3262960667280061850/dataset/.hoodie/metadata > 2021-12-17T11:40:23.1499732Z 365311 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:40:24.1626167Z 366323 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit294667857867877904/dataset/.hoodie/metadata > 2021-12-17T11:40:24.1945944Z 366355 [dispatcher-event-loop-5] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:40:24.3084730Z 366469 [dispatcher-event-loop-2] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:40:24.7350862Z 366896 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit294667857867877904/dataset/.hoodie/metadata > 2021-12-17T11:40:24.7482727Z 366909 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit294667857867877904/dataset/.hoodie/metadata > 2021-12-17T11:40:43.1530857Z 385314 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:40:44.0641298Z 386225 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:40:44.5065216Z 386667 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit4425942033067835021/dataset/.hoodie/metadata > 2021-12-17T11:40:44.5408012Z 386702 [dispatcher-event-loop-5] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:40:44.6779567Z 386839 [dispatcher-event-loop-1] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:40:45.1410815Z 387301 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit4425942033067835021/dataset/.hoodie/metadata > 2021-12-17T11:40:45.1543539Z 387315 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit4425942033067835021/dataset/.hoodie/metadata > 2021-12-17T11:41:03.1384862Z 405299 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:41:04.0150577Z 406176 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit5137952068799441319/dataset/.hoodie/metadata > 2021-12-17T11:41:04.0514934Z 406212 [dispatcher-event-loop-5] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:41:04.1677826Z 406329 [dispatcher-event-loop-2] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:41:04.6164436Z 406777 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit5137952068799441319/dataset/.hoodie/metadata > 2021-12-17T11:41:04.6343307Z 406791 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit5137952068799441319/dataset/.hoodie/metadata > 2021-12-17T11:41:13.0144546Z 415165 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:41:13.8545883Z 416010 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:41:14.2624248Z 416423 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit1733672335771853967/dataset/.hoodie/metadata > 2021-12-17T11:41:14.2957800Z 416457 [dispatcher-event-loop-5] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:41:14.3989543Z 416560 [dispatcher-event-loop-3] WARN > org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very > large size (101 KB). The maximum recommended task size is 100 KB. > 2021-12-17T11:41:14.7647136Z 416926 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit1733672335771853967/dataset/.hoodie/metadata > 2021-12-17T11:41:14.7772128Z 416938 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit1733672335771853967/dataset/.hoodie/metadata > 2021-12-17T11:41:22.6703021Z 424828 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:41:23.5771003Z 425738 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit4023714709478834450/dataset/.hoodie/metadata > 2021-12-17T11:41:24.3341519Z 426495 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit4023714709478834450/dataset/.hoodie/metadata > 2021-12-17T11:41:24.3488610Z 426510 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit4023714709478834450/dataset/.hoodie/metadata > 2021-12-17T11:41:45.2669117Z 447428 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:41:46.1117672Z 448273 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:41:46.5285885Z 448689 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit6437731807948505371/dataset/.hoodie/metadata > 2021-12-17T11:41:47.2040819Z 449365 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit6437731807948505371/dataset/.hoodie/metadata > 2021-12-17T11:41:47.2190197Z 449380 [main] WARN > org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not > found at path /tmp/junit6437731807948505371/dataset/.hoodie/metadata > 2021-12-17T11:42:12.0807715Z 474241 [main] WARN > org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system > instance used in previous test-run > 2021-12-17T11:42:12.8434589Z [ERROR] Tests run: 8, Failures: 0, Errors: 1, > Skipped: 0, Time elapsed: 135.656 s <<< FAILURE! - in > org.apache.hudi.client.TestHoodieClientMultiWriter > 2021-12-17T11:42:12.8435532Z [ERROR] > org.apache.hudi.client.TestHoodieClientMultiWriter.testHoodieClientBasicMultiWriter(HoodieTableType)[1] > Time elapsed: 13.652 s <<< ERROR! > 2021-12-17T11:42:12.8436481Z java.util.concurrent.ExecutionException: > java.lang.RuntimeException: > org.apache.hudi.exception.HoodieWriteConflictException: > java.util.ConcurrentModificationException: Cannot resolve conflicts for > overlapping writes > 2021-12-17T11:42:12.8437368Z at > org.apache.hudi.client.TestHoodieClientMultiWriter.testHoodieClientBasicMultiWriter(TestHoodieClientMultiWriter.java:129) > 2021-12-17T11:42:12.8438208Z Caused by: java.lang.RuntimeException: > org.apache.hudi.exception.HoodieWriteConflictException: > java.util.ConcurrentModificationException: Cannot resolve conflicts for > overlapping writes > 2021-12-17T11:42:12.8439052Z at > org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testHoodieClientBasicMultiWriter$1(TestHoodieClientMultiWriter.java:125) > 2021-12-17T11:42:12.8439840Z Caused by: > org.apache.hudi.exception.HoodieWriteConflictException: > java.util.ConcurrentModificationException: Cannot resolve conflicts for > overlapping writes > 2021-12-17T11:42:12.8440785Z at > org.apache.hudi.client.TestHoodieClientMultiWriter.createCommitWithUpserts(TestHoodieClientMultiWriter.java:432) > 2021-12-17T11:42:12.8441570Z at > org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testHoodieClientBasicMultiWriter$1(TestHoodieClientMultiWriter.java:122) > 2021-12-17T11:42:12.8442342Z Caused by: > java.util.ConcurrentModificationException: Cannot resolve conflicts for > overlapping writes > 2021-12-17T11:42:12.8443007Z at > org.apache.hudi.client.TestHoodieClientMultiWriter.createCommitWithUpserts(TestHoodieClientMultiWriter.java:432) > 2021-12-17T11:42:12.8443792Z at > org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testHoodieClientBasicMultiWriter$1(TestHoodieClientMultiWriter.java:122) > 2021-12-17T11:42:12.8444188Z {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)