Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-16 Thread via GitHub
danny0405 merged PR #9850: URL: https://github.com/apache/hudi/pull/9850 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-15 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1763821127 ## CI report: * d827ee5a0f974aa38b4298f45d6a7d4f7e465b24 UNKNOWN * 1297401002c4712836c5c09f56868add98f6f847 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-15 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1763680880 ## CI report: * d827ee5a0f974aa38b4298f45d6a7d4f7e465b24 UNKNOWN * f73c1b694ae0cb5557a109f28cfe25f87a264825 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-15 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1763675402 ## CI report: * d827ee5a0f974aa38b4298f45d6a7d4f7e465b24 UNKNOWN * f73c1b694ae0cb5557a109f28cfe25f87a264825 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-15 Thread via GitHub
danny0405 commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1763620590 [6480.patch.zip](https://github.com/apache/hudi/files/12911422/6480.patch.zip) Thanks for the contribution, I have reviewed and created a patch, you need to rebase with latest master th

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1359177852 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -524,9 +524,15 @@ private void checkInstantState(HoodieInstant.Sta

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761799261 ## CI report: * d827ee5a0f974aa38b4298f45d6a7d4f7e465b24 UNKNOWN * f73c1b694ae0cb5557a109f28cfe25f87a264825 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761786081 ## CI report: * d827ee5a0f974aa38b4298f45d6a7d4f7e465b24 UNKNOWN * a247acb83fce64f8f9b8e9f696a53a629534b4f6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761621340 ## CI report: * d6a5091c80ac89d8cfe4527e1eb63c962bfb17df Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2032

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761538586 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761523198 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761462595 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761447384 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
beyond1920 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358191937 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -524,9 +524,15 @@ private void checkInstantState(HoodieInstant.St

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
beyond1920 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358145916 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -538,11 +539,20 @@ public void testWriteMultiWriterInvolved() th

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358087824 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -524,9 +524,15 @@ private void checkInstantState(HoodieInstant.Sta

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358086955 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteMergeOnReadWithCompact.java: ## @@ -70,6 +86,80 @@ protected Map getMiniBatchExpected() {

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358070940 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/utils/TransactionUtils.java: ## @@ -67,7 +67,8 @@ public static Option resolveWriteConflictIfAny(

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358068254 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -538,11 +539,20 @@ public void testWriteMultiWriterInvolved() thr

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358061351 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -360,6 +367,13 @@ public static boolean allowCommitOnEmptyBa

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358061351 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java: ## @@ -360,6 +367,13 @@ public static boolean allowCommitOnEmptyBa

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358057951 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -2609,6 +2614,12 @@ public Integer getWritesFileIdEncoding() {

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1358056689 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -540,9 +541,7 @@ public void testWriteMultiWriterInvolved() throw

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761235101 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1761221614 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
beyond1920 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1357983985 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -540,9 +541,7 @@ public void testWriteMultiWriterInvolved() thro

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-13 Thread via GitHub
beyond1920 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1357981634 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -524,9 +528,7 @@ private void checkInstantState(HoodieInstant.Sta

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
beyond1920 commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1760672959 @danny0405 Thanks for review. I would add more tests soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1357634569 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java: ## @@ -540,9 +541,7 @@ public void testWriteMultiWriterInvolved() throw

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1357634270 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteMergeOnRead.java: ## @@ -213,6 +213,14 @@ protected Map getMiniBatchExpected() { ret

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
danny0405 commented on code in PR #9850: URL: https://github.com/apache/hudi/pull/9850#discussion_r1357633319 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/utils/TestWriteBase.java: ## @@ -524,9 +528,7 @@ private void checkInstantState(HoodieInstant.Stat

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1759954394 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1759544663 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2030

Re: [PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
hudi-bot commented on PR #9850: URL: https://github.com/apache/hudi/pull/9850#issuecomment-1759527066 ## CI report: * 72aebcc59f5ebebc64402dc8d1d9a491474b1dd0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[PR] [HUDI-6480] Flink support non-blocking concurrency control [hudi]

2023-10-12 Thread via GitHub
beyond1920 opened a new pull request, #9850: URL: https://github.com/apache/hudi/pull/9850 ### Change Logs Since [#9776](https://github.com/apache/hudi/pull/9776) is merged, this pr aims to support multiple streaming writers into the same MOR table with bucket index. Set the f