Re: [PR] [HUDI-6382] support hoodie-table-type changing in hudi-cli [hudi]

2023-10-31 Thread via GitHub
waitingF commented on code in PR #9937: URL: https://github.com/apache/hudi/pull/9937#discussion_r1378444938 ## hudi-cli/src/main/java/org/apache/hudi/cli/commands/TableCommand.java: ## @@ -233,10 +237,28 @@ public String deleteTableConfig( return renderOldNewProps(newProps

Re: [PR] [HUDI-6949] Spark support non-blocking concurrency control [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9921: URL: https://github.com/apache/hudi/pull/9921#issuecomment-1788488521 ## CI report: * 00152b4450f2453c6b37f26dde9cfc19fe865425 UNKNOWN * 8401cb3b04bea4ac0388d33ed40ad5853a5b7090 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-7017] Prevent full schema evolution from wrongly falling back t… [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9966: URL: https://github.com/apache/hudi/pull/9966#issuecomment-1788488812 ## CI report: * c7a463feafa866e145bdff853fb969e38d22d2e5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2060

Re: [PR] [HUDI-7017] Prevent full schema evolution from wrongly falling back t… [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9966: URL: https://github.com/apache/hudi/pull/9966#issuecomment-1788480951 ## CI report: * c7a463feafa866e145bdff853fb969e38d22d2e5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2060

Re: [PR] [HUDI-6949] Spark support non-blocking concurrency control [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9921: URL: https://github.com/apache/hudi/pull/9921#issuecomment-1788480592 ## CI report: * 00152b4450f2453c6b37f26dde9cfc19fe865425 UNKNOWN * 8401cb3b04bea4ac0388d33ed40ad5853a5b7090 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6990] Configurable clustering task parallelism [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9925: URL: https://github.com/apache/hudi/pull/9925#issuecomment-1788473264 ## CI report: * ff4b7fc25d2f80b59998b2034771118d4e4da09c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2058

Re: [PR] [HUDI-6991] Fix hoodie.parquet.max.file.size conf reset error [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9924: URL: https://github.com/apache/hudi/pull/9924#issuecomment-1788473214 ## CI report: * b94dc476b85d357b804730f879c2d7751c20ea27 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2058

Re: [I] [SUPPORT] Hudi Spark WEB UI [SQL / DataFrame] Details for Query, It does not display detailed indicator information [hudi]

2023-10-31 Thread via GitHub
chestnutqiang commented on issue #9944: URL: https://github.com/apache/hudi/issues/9944#issuecomment-1788455223 @boneanxs Thank you very much for your answer. SparkPlan -> DataSet What do I need to do? -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788438089 ## CI report: * 8090fb8887411f5e188211b70214452da428d5fd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-6806] Support Spark 3.5.0 [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9717: URL: https://github.com/apache/hudi/pull/9717#issuecomment-1788437616 ## CI report: * 9b8fdd2d1b69da528069e364790b53af1d6150af UNKNOWN * 95a05c09af41cd1f570b796f25e753982e571e97 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788431279 ## CI report: * 8090fb8887411f5e188211b70214452da428d5fd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-5210] Implement functional indexes [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9872: URL: https://github.com/apache/hudi/pull/9872#issuecomment-1788430760 ## CI report: * 475e0edf5b41a2ec0aec5b23b9852f814e0b4aa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2057

Re: [PR] [HUDI-7017] Prevent full schema evolution from wrongly falling back t… [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9966: URL: https://github.com/apache/hudi/pull/9966#issuecomment-1788423154 ## CI report: * c7a463feafa866e145bdff853fb969e38d22d2e5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2060

Re: [PR] [HUDI-5210] Implement functional indexes [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9872: URL: https://github.com/apache/hudi/pull/9872#issuecomment-1788422635 ## CI report: * 475e0edf5b41a2ec0aec5b23b9852f814e0b4aa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2057

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788423070 ## CI report: * 8090fb8887411f5e188211b70214452da428d5fd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-6992] IncrementalInputSplits incorrectly set the latestCommit attr [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9923: URL: https://github.com/apache/hudi/pull/9923#issuecomment-1788422820 ## CI report: * 4ac599030452361a0b29376d3c2646ceec742dd8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2054

[jira] [Updated] (HUDI-7012) The BootstrapOperator reduces the memory.

2023-10-31 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-7012: - Description: the stream job and hoodieTable never release memory. !image-2023-11-01-12-32-19-627.png|width=987,he

[jira] [Updated] (HUDI-7012) The BootstrapOperator reduces the memory.

2023-10-31 Thread Bo Cui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HUDI-7012: - Attachment: image-2023-11-01-12-32-19-627.png > The BootstrapOperator reduces the memory. > --

Re: [PR] [HUDI-7017] Prevent full schema evolution from wrongly falling back t… [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9966: URL: https://github.com/apache/hudi/pull/9966#issuecomment-1788392010 ## CI report: * c7a463feafa866e145bdff853fb969e38d22d2e5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788391945 ## CI report: * 8090fb8887411f5e188211b70214452da428d5fd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-6992] IncrementalInputSplits incorrectly set the latestCommit attr [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9923: URL: https://github.com/apache/hudi/pull/9923#issuecomment-1788391744 ## CI report: * 4ac599030452361a0b29376d3c2646ceec742dd8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2054

Re: [PR] [HUDI-5210] Implement functional indexes [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9872: URL: https://github.com/apache/hudi/pull/9872#issuecomment-1788391577 ## CI report: * 475e0edf5b41a2ec0aec5b23b9852f814e0b4aa4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2057

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
cuibo01 commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788389638 ![1698812175252](https://github.com/apache/hudi/assets/36881649/e329f735-5008-4bac-803a-5bdebe86a37d) -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] [HUDI-5823][DNM]Poc code for Partition ttl management [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9723: URL: https://github.com/apache/hudi/pull/9723#issuecomment-1788385742 ## CI report: * 8081b2974abb643283b189da5ec5658726f7edf9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2057

Re: [PR] [HUDI-6992] IncrementalInputSplits incorrectly set the latestCommit attr [hudi]

2023-10-31 Thread via GitHub
zhuanshenbsj1 commented on PR #9923: URL: https://github.com/apache/hudi/pull/9923#issuecomment-1788378744 > Please fix the test failures first. Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
cuibo01 commented on code in PR #9959: URL: https://github.com/apache/hudi/pull/9959#discussion_r1378373111 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapOperator.java: ## @@ -155,6 +155,7 @@ protected void preLoadIndexRecords() throws

[PR] [HUDI-7017] Prevent full schema evolution from wrongly falling back t… [hudi]

2023-10-31 Thread via GitHub
voonhous opened a new pull request, #9966: URL: https://github.com/apache/hudi/pull/9966 …o OOB schema evolution Please refer to [HUDI-7017](https://issues.apache.org/jira/browse/HUDI-7017) for details. ### Change Logs Minor fix to prevent full schema evolution from wron

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7017: - Labels: pull-request-available (was: ) > Prevent full schema evolution from wrongly falling back

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon updated HUDI-7017: --- Description: For MOR tables that have these 2 configurations enabled:   {code:java} hoodie.schema.on.read.enable=true

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon updated HUDI-7017: --- Attachment: (was: image-2023-11-01-11-43-02-166.png) > Prevent full schema evolution from wrongly falling back to

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon updated HUDI-7017: --- Attachment: image-2023-11-01-11-43-02-166.png > Prevent full schema evolution from wrongly falling back to OOB > -

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon updated HUDI-7017: --- Attachment: image-2023-11-01-11-43-14-149.png > Prevent full schema evolution from wrongly falling back to OOB > -

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon updated HUDI-7017: --- Attachment: image-2023-11-01-11-41-25-604.png > Prevent full schema evolution from wrongly falling back to OOB > -

Re: [PR] [HUDI-5823][DNM]Poc code for Partition ttl management [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9723: URL: https://github.com/apache/hudi/pull/9723#issuecomment-1788356879 ## CI report: * b99492dd527ca230935ed480d8290b40912d3fee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1994

[jira] [Created] (HUDI-7018) Test case for spark catalog refresh table

2023-10-31 Thread HunterXHunter (Jira)
HunterXHunter created HUDI-7018: --- Summary: Test case for spark catalog refresh table Key: HUDI-7018 URL: https://issues.apache.org/jira/browse/HUDI-7018 Project: Apache Hudi Issue Type: Test

Re: [PR] [HUDI-6949] Spark support non-blocking concurrency control [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9921: URL: https://github.com/apache/hudi/pull/9921#issuecomment-1788351067 ## CI report: * 00152b4450f2453c6b37f26dde9cfc19fe865425 UNKNOWN * 8401cb3b04bea4ac0388d33ed40ad5853a5b7090 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-5823][DNM]Poc code for Partition ttl management [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9723: URL: https://github.com/apache/hudi/pull/9723#issuecomment-1788350770 ## CI report: * b99492dd527ca230935ed480d8290b40912d3fee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1994

Re: [PR] [HUDI-0000] DO NOT MERGE Fix incr errors new reader [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9954: URL: https://github.com/apache/hudi/pull/9954#issuecomment-1788344459 ## CI report: * 18ae9b1b08a6e3c7fd5ea4daf6935a10ed3e0fff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2058

Re: [PR] [HUDI-6949] Spark support non-blocking concurrency control [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9921: URL: https://github.com/apache/hudi/pull/9921#issuecomment-1788344223 ## CI report: * 00152b4450f2453c6b37f26dde9cfc19fe865425 UNKNOWN * 8401cb3b04bea4ac0388d33ed40ad5853a5b7090 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider … [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1788341844 ## CI report: * 4864528010313c4bcf59866ff67ef11b8f1e14d0 UNKNOWN * 96f0e0ab21f32e06b0fdd7a552f428cb2acee7f5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[I] [SUPPORT] Solution for synchronizing the entire database table in flink [hudi]

2023-10-31 Thread via GitHub
bajiaolong opened a new issue, #9965: URL: https://github.com/apache/hudi/issues/9965 At present, I am sending the data of the whole database to a topic of kafka. I want to use flink to read kafka data and write the data into multiple tables of hudi. Do you have any idea -- This is an au

Re: [PR] [HUDI-7016] Fix bundling of RoaringBitmap dependency [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9963: URL: https://github.com/apache/hudi/pull/9963#issuecomment-1788313932 ## CI report: * 3e06c1122a408c5fe5b7c0a2af50798863607060 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788313851 ## CI report: * 8090fb8887411f5e188211b70214452da428d5fd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-7013] Drop table command cannot delete dir even when purge is enable [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9960: URL: https://github.com/apache/hudi/pull/9960#issuecomment-1788313872 ## CI report: * 0d690d42416e63d40ebd8cf88d5f640e488adf6e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-7014] Optimize the code of BoundedPartitionAwareCompactionStrategy [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9961: URL: https://github.com/apache/hudi/pull/9961#issuecomment-1788313898 ## CI report: * 75c27611b7a2ceacf43aa903f5b542fffc7b27cf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-7010] Build clustering group reduces redundant traversals [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9957: URL: https://github.com/apache/hudi/pull/9957#issuecomment-1788313832 ## CI report: * 0c34110238584b4ec7862d8849e5ca5353769051 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-7009] Filtering out null values from avro kafka source [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9955: URL: https://github.com/apache/hudi/pull/9955#issuecomment-1788313779 ## CI report: * 8809ad5187203de0326cca32a3e59a4b1e1b9ca0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2058

Re: [PR] [HUDI-6993] Support Flink 1.18 [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9949: URL: https://github.com/apache/hudi/pull/9949#issuecomment-1788313742 ## CI report: * de7e253bee1055bf5f74f31dd8f5ed2c2db01872 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2056

Re: [PR] [HUDI-7005] Fix hudi-aws-bundle relocation issue with avro [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9946: URL: https://github.com/apache/hudi/pull/9946#issuecomment-1788313725 ## CI report: * 7f9424da664e341e19f38d11f1db10868c72284b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2056

Re: [PR] [HUDI-6382] support hoodie-table-type changing in hudi-cli [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9937: URL: https://github.com/apache/hudi/pull/9937#issuecomment-1788313623 ## CI report: * db4cacd5465f0086d1743a5a276af76f64f13c72 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2054

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9943: URL: https://github.com/apache/hudi/pull/9943#issuecomment-1788313676 ## CI report: * cf2a16031d1e2048d3ae75cc9ecf35ae409eac17 UNKNOWN * 0d581beb02b4918d7418e00f05029b9e84a2da40 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-6991] Fix hoodie.parquet.max.file.size conf reset error [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9924: URL: https://github.com/apache/hudi/pull/9924#issuecomment-1788313503 ## CI report: * 328aa0b4454248266a14923e5eb45c068da17020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2050

Re: [PR] [MINOR] Delete the duplicated getMetadata method in HoodieTable. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9964: URL: https://github.com/apache/hudi/pull/9964#issuecomment-1788313971 ## CI report: * 65f4f941f1d14cabf9f5b48cc219bcbf4de38de4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

[jira] [Updated] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon updated HUDI-7017: --- Description: For MOR tables that have these 2 configurations enabled:   {code:java} hoodie.schema.on.read.enable=true

[jira] [Assigned] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] voon reassigned HUDI-7017: -- Assignee: voon > Prevent full schema evolution from wrongly falling back to OOB > -

Re: [PR] [HUDI-7011] a metric to indicate whether rollback has occurred in final compaction state [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9956: URL: https://github.com/apache/hudi/pull/9956#issuecomment-1788313808 ## CI report: * eb46f914afe0193ababb1320b8515bd74bdfe104 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2059

Re: [PR] [HUDI-0000] DO NOT MERGE Fix incr errors new reader [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9954: URL: https://github.com/apache/hudi/pull/9954#issuecomment-1788313762 ## CI report: * 18ae9b1b08a6e3c7fd5ea4daf6935a10ed3e0fff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2058

Re: [PR] [HUDI-7006] reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9945: URL: https://github.com/apache/hudi/pull/9945#issuecomment-1788313700 ## CI report: * 902e5fc608a6cea89bae36f415e4dc1f9a04425d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2055

Re: [PR] [HUDI-6990] Configurable clustering task parallelism [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9925: URL: https://github.com/apache/hudi/pull/9925#issuecomment-1788313532 ## CI report: * f4666e6772c618f8b9d54bb381e174a894eddcda Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2051

Re: [PR] [HUDI-6989] Stop handling more data if task is aborted & clean partial files if possible in task side [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9922: URL: https://github.com/apache/hudi/pull/9922#issuecomment-1788313446 ## CI report: * a9b3bd0f7aa15a9fbef3caaae798aa34790e027a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2049

[jira] [Created] (HUDI-7017) Prevent full schema evolution from wrongly falling back to OOB

2023-10-31 Thread voon (Jira)
voon created HUDI-7017: -- Summary: Prevent full schema evolution from wrongly falling back to OOB Key: HUDI-7017 URL: https://issues.apache.org/jira/browse/HUDI-7017 Project: Apache Hudi Issue Type: Bug

Re: [PR] [HUDI-5210] Implement functional indexes [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9872: URL: https://github.com/apache/hudi/pull/9872#issuecomment-1788313054 ## CI report: * 65d4f829b812cc7c76c0efb5cc03658ce2c88f29 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=205

Re: [PR] [HUDI-5823][DNM]Poc code for Partition ttl management [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9723: URL: https://github.com/apache/hudi/pull/9723#issuecomment-1788312393 ## CI report: * b99492dd527ca230935ed480d8290b40912d3fee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1994

Re: [PR] [HUDI-6806] Support Spark 3.5.0 [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9717: URL: https://github.com/apache/hudi/pull/9717#issuecomment-1788312196 ## CI report: * 9b8fdd2d1b69da528069e364790b53af1d6150af UNKNOWN * ef1785580edd0f4514576ec261aa0d9a17f71aa4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-2461] Support out of order commits in MDT with completion time view [hudi]

2023-10-31 Thread via GitHub
danny0405 commented on code in PR #9871: URL: https://github.com/apache/hudi/pull/9871#discussion_r1378336573 ## hudi-client/hudi-java-client/src/test/java/org/apache/hudi/client/TestJavaHoodieBackedMetadata.java: ## @@ -525,7 +523,6 @@ public void testVirtualKeysInBaseFiles() t

Re: [PR] [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider … [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1788308615 ## CI report: * 4864528010313c4bcf59866ff67ef11b8f1e14d0 UNKNOWN * Unknown: [CANCELED](TBD) * 96f0e0ab21f32e06b0fdd7a552f428cb2acee7f5 UNKNOWN Bot commands

Re: [PR] [HUDI-6949] Spark support non-blocking concurrency control [hudi]

2023-10-31 Thread via GitHub
beyond1920 commented on code in PR #9921: URL: https://github.com/apache/hudi/pull/9921#discussion_r1378334970 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/RDDSimpleBucketBulkInsertPartitioner.java: ## @@ -93,7 +96,9 @@ Map> getPartitionMa

Re: [PR] [HUDI-6949] Spark support non-blocking concurrency control [hudi]

2023-10-31 Thread via GitHub
beyond1920 commented on code in PR #9921: URL: https://github.com/apache/hudi/pull/9921#discussion_r1378334583 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/functional/TestHoodieSparkNBCC.java: ## @@ -0,0 +1,437 @@ +/* + * Licensed to the Apache Software F

Re: [PR] [MINOR] Delete the duplicated getMetadata method in HoodieTable. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9964: URL: https://github.com/apache/hudi/pull/9964#issuecomment-1788302307 ## CI report: * 65f4f941f1d14cabf9f5b48cc219bcbf4de38de4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7013] Drop table command cannot delete dir even when purge is enable [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9960: URL: https://github.com/apache/hudi/pull/9960#issuecomment-1788302016 ## CI report: * 0d690d42416e63d40ebd8cf88d5f640e488adf6e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7016] Fix bundling of RoaringBitmap dependency [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9963: URL: https://github.com/apache/hudi/pull/9963#issuecomment-1788302193 ## CI report: * 3e06c1122a408c5fe5b7c0a2af50798863607060 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7014] Optimize the code of BoundedPartitionAwareCompactionStrategy [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9961: URL: https://github.com/apache/hudi/pull/9961#issuecomment-1788302074 ## CI report: * 75c27611b7a2ceacf43aa903f5b542fffc7b27cf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7012] The BootstrapOperator reduces the memory. [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9959: URL: https://github.com/apache/hudi/pull/9959#issuecomment-1788301872 ## CI report: * 8090fb8887411f5e188211b70214452da428d5fd UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7010] Build clustering group reduces redundant traversals [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9957: URL: https://github.com/apache/hudi/pull/9957#issuecomment-1788301467 ## CI report: * 0c34110238584b4ec7862d8849e5ca5353769051 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7011] a metric to indicate whether rollback has occurred in final compaction state [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9956: URL: https://github.com/apache/hudi/pull/9956#issuecomment-1788301251 ## CI report: * eb46f914afe0193ababb1320b8515bd74bdfe104 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7009] Filtering out null values from avro kafka source [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9955: URL: https://github.com/apache/hudi/pull/9955#issuecomment-1788301128 ## CI report: * 8809ad5187203de0326cca32a3e59a4b1e1b9ca0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-6993] Support Flink 1.18 [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9949: URL: https://github.com/apache/hudi/pull/9949#issuecomment-1788300880 ## CI report: * de7e253bee1055bf5f74f31dd8f5ed2c2db01872 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2056

Re: [PR] [HUDI-0000] DO NOT MERGE Fix incr errors new reader [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9954: URL: https://github.com/apache/hudi/pull/9954#issuecomment-1788300997 ## CI report: * 18ae9b1b08a6e3c7fd5ea4daf6935a10ed3e0fff UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

Re: [PR] [HUDI-7004] Add support of snapshotLoadQuerySplitter in s3/gcs sources [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9943: URL: https://github.com/apache/hudi/pull/9943#issuecomment-1788300504 ## CI report: * cf2a16031d1e2048d3ae75cc9ecf35ae409eac17 UNKNOWN * 0d581beb02b4918d7418e00f05029b9e84a2da40 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-7006] reduce unnecessary is_empty rdd calls in StreamSync [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9945: URL: https://github.com/apache/hudi/pull/9945#issuecomment-1788300651 ## CI report: * 902e5fc608a6cea89bae36f415e4dc1f9a04425d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2055

Re: [PR] [HUDI-7005] Fix hudi-aws-bundle relocation issue with avro [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9946: URL: https://github.com/apache/hudi/pull/9946#issuecomment-1788300770 ## CI report: * 7f9424da664e341e19f38d11f1db10868c72284b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2056

Re: [PR] [HUDI-6382] support hoodie-table-type changing in hudi-cli [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9937: URL: https://github.com/apache/hudi/pull/9937#issuecomment-1788300371 ## CI report: * db4cacd5465f0086d1743a5a276af76f64f13c72 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2054

Re: [PR] [HUDI-6990] Configurable clustering task parallelism [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9925: URL: https://github.com/apache/hudi/pull/9925#issuecomment-1788300164 ## CI report: * f4666e6772c618f8b9d54bb381e174a894eddcda Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2051

Re: [PR] [HUDI-6991] Fix hoodie.parquet.max.file.size conf reset error [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9924: URL: https://github.com/apache/hudi/pull/9924#issuecomment-1788300021 ## CI report: * 328aa0b4454248266a14923e5eb45c068da17020 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2050

Re: [PR] [HUDI-6989] Stop handling more data if task is aborted & clean partial files if possible in task side [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9922: URL: https://github.com/apache/hudi/pull/9922#issuecomment-1788299905 ## CI report: * a9b3bd0f7aa15a9fbef3caaae798aa34790e027a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2049

Re: [PR] [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider … [hudi]

2023-10-31 Thread via GitHub
LinMingQiang commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1788299725 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [HUDI-6790] Support incremental/CDC queries using HadoopFsRelation [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9888: URL: https://github.com/apache/hudi/pull/9888#issuecomment-1788299657 ## CI report: * 2501f4ca40591cd9b2d94b5c4daa360aa6454cef UNKNOWN * 18f548a7df24f79a7abe8f1065db719562ba14c2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-5210] Implement functional indexes [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9872: URL: https://github.com/apache/hudi/pull/9872#issuecomment-1788299564 ## CI report: * 65d4f829b812cc7c76c0efb5cc03658ce2c88f29 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=205

Re: [PR] [HUDI-2461] Support out of order commits in MDT with completion time view [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9871: URL: https://github.com/apache/hudi/pull/9871#issuecomment-1788299517 ## CI report: * 975e7bcd435002af8902a17bf3500accb437d959 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2056

Re: [PR] [HUDI-5823][DNM]Poc code for Partition ttl management [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9723: URL: https://github.com/apache/hudi/pull/9723#issuecomment-1788299309 ## CI report: * b99492dd527ca230935ed480d8290b40912d3fee Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1994

Re: [PR] [HUDI-6806] Support Spark 3.5.0 [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #9717: URL: https://github.com/apache/hudi/pull/9717#issuecomment-1788299219 ## CI report: * 9b8fdd2d1b69da528069e364790b53af1d6150af UNKNOWN * ef1785580edd0f4514576ec261aa0d9a17f71aa4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

Re: [PR] [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider … [hudi]

2023-10-31 Thread via GitHub
LinMingQiang commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1788298960 > @LinMingQiang : Can you resolve conflicts and add tests. Thanks. done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [HUDI-5591] HoodieSparkSqlWriter#getHiveTableNames needs to consider … [hudi]

2023-10-31 Thread via GitHub
hudi-bot commented on PR #7718: URL: https://github.com/apache/hudi/pull/7718#issuecomment-1788296811 ## CI report: * 0da7f5ff6c70c773c035aae60d84ec460252aae6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1449

[PR] [MINOR] Delete the duplicated getMetadata method in HoodieTable. [hudi]

2023-10-31 Thread via GitHub
majian1998 opened a new pull request, #9964: URL: https://github.com/apache/hudi/pull/9964 ### Change Logs In `HoodieTable.java`, the code at line 173 is: `public HoodieTableMetadata getMetadata() { return metadata; }` While the code at line 1036 is: `public HoodieTa

Re: [PR] [HUDI-7013] Drop table command cannot delete dir even when purge is enable [hudi]

2023-10-31 Thread via GitHub
xuzifu666 commented on code in PR #9960: URL: https://github.com/apache/hudi/pull/9960#discussion_r1378314872 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -699,7 +699,7 @@ public static boolean deleteDir( pairOfSubPathAndConf -> deleteS

Re: [PR] [HUDI-7013] Drop table command cannot delete dir even when purge is enable [hudi]

2023-10-31 Thread via GitHub
xuzifu666 commented on code in PR #9960: URL: https://github.com/apache/hudi/pull/9960#discussion_r1378314872 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -699,7 +699,7 @@ public static boolean deleteDir( pairOfSubPathAndConf -> deleteS

Re: [PR] [HUDI-2461] Support out of order commits in MDT with completion time view [hudi]

2023-10-31 Thread via GitHub
codope commented on code in PR #9871: URL: https://github.com/apache/hudi/pull/9871#discussion_r1378302314 ## hudi-client/hudi-java-client/src/test/java/org/apache/hudi/client/TestJavaHoodieBackedMetadata.java: ## @@ -525,7 +523,6 @@ public void testVirtualKeysInBaseFiles() thro

Re: [PR] [HUDI-7005] Fix hudi-aws-bundle relocation issue with avro [hudi]

2023-10-31 Thread via GitHub
danny0405 commented on code in PR #9946: URL: https://github.com/apache/hudi/pull/9946#discussion_r1378287961 ## packaging/hudi-flink-bundle/pom.xml: ## @@ -84,7 +84,6 @@ org.apache.hudi:hudi-sync-common org.apache.hudi:hudi-hadoop-mr

Re: [I] [SUPPORT] Facing java.util.NoSuchElementException on EMR 6.12 (Hudi 0.13) with inline compaction and cleaning on MoR tables [hudi]

2023-10-31 Thread via GitHub
rahil-c commented on issue #9861: URL: https://github.com/apache/hudi/issues/9861#issuecomment-1788226880 cc @yihua -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [HUDI-7011] a metric to indicate whether rollback has occurred in final compaction state [hudi]

2023-10-31 Thread via GitHub
danny0405 commented on code in PR #9956: URL: https://github.com/apache/hudi/pull/9956#discussion_r1378286087 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/metrics/FlinkCompactionMetrics.java: ## @@ -103,4 +110,16 @@ public void endCompaction() { this.co

  1   2   >