[
https://issues.apache.org/jira/browse/HUDI-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914227#comment-17914227
]
Davis Zhang commented on HUDI-7483:
-----------------------------------
PR to repro, check it out and follow intellij config to get a repro:
https://github.com/Davis-Zhang-Onehouse/hudi-oss/pull/new/HUDI-7483
What the test does is:
- Delta commit sequentially for 3 times.
- Fire 3 threads and ideal execution order is
Compaction being scheduled. (compaction.requested)
client2.scheduleTableService(compactionTimeStamp, Option.empty(),
TableServiceType.COMPACT);
Cleaning being scheduled (I did not see clean.requested as there is no
compaction, so no old file slice to clean up)
client3.scheduleTableService(cleanCommitTime, Option.empty(),
TableServiceType.CLEAN);
Upsert kicks in for MOR. We expect concurrent modification exception.
Happy case timeline:
- compaction.requested
- deltacommit.requested
- deltacommit.inflight
exception is thrown in future 1 object
Unhappy case timeline:
- compaction.requested
- deltacommit.requested
- deltacommit.inflight
- deltacommit.completed <-- no exception, upsert is completed
I have attached the table folder for both pass/error cases. I need guidance on
how to verify if the error case upsert does not conflict on file slice that
compaction works on.
I compared the compaction and delta commit plan between the 2, nothing
suspicious.
> TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict
> -----------------------------------------------------------------------------
>
> Key: HUDI-7483
> URL: https://issues.apache.org/jira/browse/HUDI-7483
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Lin Liu
> Priority: Major
> Attachments: 7483.zip
>
>
> https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=22801&view=logs&j=600e7de6-e133-5e69-e615-50ee129b3c08&t=bbbd7bcc-ae73-56b8-887a-cd2d6deaafc7&s=859b8d9a-8fd6-5a5c-6f5e-f84f1990894e
> {code:java}
> [ERROR] Tests run: 29, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 307.331 s <<< FAILURE! - in org.apache.hudi.client.TestHoodieClientMultiWriter
> [ERROR] testMultiWriterWithAsyncTableServicesWithConflict{HoodieTableType,
> Class, ConflictResolutionStrategy}[6] Time elapsed: 16.083 s <<< ERROR!
> java.util.concurrent.ExecutionException: org.opentest4j.AssertionFailedError:
> Expected org.apache.hudi.exception.HoodieWriteConflictException to be thrown,
> but nothing was thrown.
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.hudi.client.TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict(TestHoodieClientMultiWriter.java:565)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
