[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090275991 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090269465 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,43 @@ private List g

[GitHub] [hudi] duc-dn opened a new issue, #7791: [SUPPORT] Don't see metadata folder in .hoodie folder when ingesting data use hudi kafka connector

2023-01-29 Thread via GitHub
duc-dn opened a new issue, #7791: URL: https://github.com/apache/hudi/issues/7791 **Describe the problem you faced** - I use hudi kafka connector to ingest data from kafka into minio but I don't see metadata folder in .hoodie folder - This is my .hoodie folder ![image](https://user

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090266167 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] nsivabalan commented on pull request #7787: [HUDI-5646] Guard dropping columns by a config, do not allow by default

2023-01-29 Thread via GitHub
nsivabalan commented on PR #7787: URL: https://github.com/apache/hudi/pull/7787#issuecomment-1408128144 Addressed my comments and pushed an update -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090261765 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090251349 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090250986 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090249282 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090246803 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090245158 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090241285 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090241285 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090241285 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090237751 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] trushev commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
trushev commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090236238 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/RecordIterators.java: ## @@ -50,8 +50,7 @@ public static ClosableIterator getParquetRecordI

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090231178 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] hudi-bot commented on pull request #7790: [HUDI-5632] Fix failure launching Spark jobs from `hudi-cli-bundle`

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7790: URL: https://github.com/apache/hudi/pull/7790#issuecomment-1408084890 ## CI report: * 8e01aae6f0dfb7b11e73cdff30fcbd17536ac8cf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7761: URL: https://github.com/apache/hudi/pull/7761#issuecomment-1408084801 ## CI report: * 93e4876eb944c4b3976ac5422586ff35e8b7aef5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1466

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090231178 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] voonhous commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090228713 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/RecordIterators.java: ## @@ -50,8 +50,7 @@ public static ClosableIterator getParquetRecord

[GitHub] [hudi] hudi-bot commented on pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7669: URL: https://github.com/apache/hudi/pull/7669#issuecomment-1408079359 ## CI report: * dae2ca6c5ab37f7865789823dae7ec3033c7b452 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1435

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090228224 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090227913 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] yihua commented on pull request #7790: [HUDI-5632] Fix failure launching Spark jobs from `hudi-cli-bundle`

2023-01-29 Thread via GitHub
yihua commented on PR #7790: URL: https://github.com/apache/hudi/pull/7790#issuecomment-1408078217 @rahil-c could you review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[jira] [Updated] (HUDI-5632) CLI commands using Spark fails to execute with hudi-cli-bundle

2023-01-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5632: - Labels: pull-request-available (was: ) > CLI commands using Spark fails to execute with hudi-cli-

[GitHub] [hudi] yihua opened a new pull request, #7790: [HUDI-5632] Fix failure launching Spark jobs from `hudi-cli-bundle`

2023-01-29 Thread via GitHub
yihua opened a new pull request, #7790: URL: https://github.com/apache/hudi/pull/7790 ### Change Logs The Hudi CLI commands which require launching Spark cannot be executed in Hudi CLI shell with hudi-cli-bundle: ``` savepoint create --commit --sparkMaster local savepoint de

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090226772 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090226467 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] codope commented on pull request #7781: [HUDI-5637] Add Kryo for hive sync

2023-01-29 Thread via GitHub
codope commented on PR #7781: URL: https://github.com/apache/hudi/pull/7781#issuecomment-1408076378 > @alexeykudinkin @codope don't we want to include kryo in aws, gcp, datahub-sync, and timeline-server bundles too? these all could run as standalone app. In #7702, we mainly want to exclude

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090225408 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] hudi-bot commented on pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7669: URL: https://github.com/apache/hudi/pull/7669#issuecomment-1408074928 ## CI report: * dae2ca6c5ab37f7865789823dae7ec3033c7b452 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1435

[jira] [Closed] (HUDI-5637) Hive sync using run_sync_tool fails due to NoClassDefFoundError

2023-01-29 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-5637. - Resolution: Fixed > Hive sync using run_sync_tool fails due to NoClassDefFoundError >

[GitHub] [hudi] hudi-bot commented on pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7761: URL: https://github.com/apache/hudi/pull/7761#issuecomment-1408070065 ## CI report: * 93e4876eb944c4b3976ac5422586ff35e8b7aef5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1466

[hudi] branch master updated (53b813a565b -> e6c0bd63b2f)

2023-01-29 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 53b813a565b [HUDI-5629] Clean CDC log files for enable/disable scenario (#7786) add e6c0bd63b2f [HUDI-5637] Add Kr

[GitHub] [hudi] codope merged pull request #7781: [HUDI-5637] Add Kryo for hive sync

2023-01-29 Thread via GitHub
codope merged PR #7781: URL: https://github.com/apache/hudi/pull/7781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[GitHub] [hudi] hudi-bot commented on pull request #7680: [HUDI-5548] spark sql show|update hudi's table properties

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1408069870 ## CI report: * 0970573f82ef1a49184d1875975463f76f7d791d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1468

[GitHub] [hudi] hudi-bot commented on pull request #7672: [HUDI-5557]Avoid converting columns that are not indexed in CSI

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7672: URL: https://github.com/apache/hudi/pull/7672#issuecomment-1408069749 ## CI report: * 46e27a0438ecf51a0faee5dbee9d710b44cf0572 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1475

[GitHub] [hudi] hudi-bot commented on pull request #7410: [HUDI-5634] Rename CDC related classes

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7410: URL: https://github.com/apache/hudi/pull/7410#issuecomment-1408069270 ## CI report: * 3ce7c48363aae540d7d00c96edd2dae649e82fc2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1475

[GitHub] [hudi] voonhous commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090219771 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/RecordIterators.java: ## @@ -50,8 +50,7 @@ public static ClosableIterator getParquetRecord

[GitHub] [hudi] YannByron merged pull request #7786: [HUDI-5629] Clean CDC log files for enable/disable scenario

2023-01-29 Thread via GitHub
YannByron merged PR #7786: URL: https://github.com/apache/hudi/pull/7786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[hudi] branch master updated (88d8e5e96d5 -> 53b813a565b)

2023-01-29 Thread biyan
This is an automated email from the ASF dual-hosted git repository. biyan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 88d8e5e96d5 [MINOR] Cleaning up recently introduced configs (#7772) add 53b813a565b [HUDI-5629] Clean CDC log files

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090218337 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] voonhous commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090217706 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List getA

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090216637 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -98,4 +103,42 @@ private List g

[GitHub] [hudi] YannByron commented on pull request #7410: [HUDI-5634] Rename CDC related classes

2023-01-29 Thread via GitHub
YannByron commented on PR #7410: URL: https://github.com/apache/hudi/pull/7410#issuecomment-1408044938 the failed UT is not related to this, will land this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] trushev commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
trushev commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090203187 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/RecordIterators.java: ## @@ -50,8 +50,7 @@ public static ClosableIterator getParquetRecordI

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090198785 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkDeletePartitionCommitActionExecutor.java: ## @@ -18,13 +18,17 @@ package org

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
SteNicholas commented on code in PR #7669: URL: https://github.com/apache/hudi/pull/7669#discussion_r1090198502 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/table/action/commit/FlinkDeletePartitionCommitActionExecutor.java: ## @@ -18,6 +18,8 @@ package org.a

[GitHub] [hudi] SteNicholas commented on pull request #7716: [HUDI-5558] Serializable interface implementation don't explicitly declare serialVersionUID

2023-01-29 Thread via GitHub
SteNicholas commented on PR #7716: URL: https://github.com/apache/hudi/pull/7716#issuecomment-1408035488 @alexeykudinkin, the `serialVersionUID` should only be set to 1 because different JVM generates different `serialVersionUID` and the compatibility problem of `serialVersionUID` is inevit

[GitHub] [hudi] codope commented on pull request #7781: [HUDI-5637] Add Kryo for hive sync

2023-01-29 Thread via GitHub
codope commented on PR #7781: URL: https://github.com/apache/hudi/pull/7781#issuecomment-1408034900 Again it is deltastreamer multiwriter `testUpsertsContinuousModeWithMultipleWritersForConflicts` which is failing. It's known to be flaky and under investigation. I am going to land this PR

[GitHub] [hudi] lokeshj1703 opened a new pull request, #7789: Bundle validation 0.13.0-rc1

2023-01-29 Thread via GitHub
lokeshj1703 opened a new pull request, #7789: URL: https://github.com/apache/hudi/pull/7789 ### Change Logs Bundle validation for 0.13.0-rc1 ### Impact NA ### Risk level (write none, low medium or high below) Low ### Documentation Update _Descr

[GitHub] [hudi] XuQianJin-Stars commented on pull request #7680: [HUDI-5548] spark sql show|update hudi's table properties

2023-01-29 Thread via GitHub
XuQianJin-Stars commented on PR #7680: URL: https://github.com/apache/hudi/pull/7680#issuecomment-1408027326 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] maheshguptags commented on issue #7613: Not able to Delete record

2023-01-29 Thread via GitHub
maheshguptags commented on issue #7613: URL: https://github.com/apache/hudi/issues/7613#issuecomment-1408021672 @danny0405 can you please post the status? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] hudi-bot commented on pull request #7684: [HUDI-5567] Modified to make bootstrapping exception message clearer

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7684: URL: https://github.com/apache/hudi/pull/7684#issuecomment-1408000736 ## CI report: * 5aca03f24eac492f0181657fddfc79e40fd32208 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1475

[hudi] 02/19: [HUDI-5534] Optimizing Bloom Index lookup when using Bloom Filters from Metadata Table (#7642)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 9ac8765a3a79d2cffe8a43ac6f33e3986f91baf7 Author: Alexey Kudinkin AuthorDate: Fri Jan 27 15:47:12 2023 -0800 [H

[hudi] 05/19: [HUDI-5023] Switching default Write Executor type to `SIMPLE` (#7476)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit ee7764b9749d929917a4e9641f4a32f269674100 Author: Alexey Kudinkin AuthorDate: Fri Jan 27 18:56:32 2023 -0800 [H

[hudi] 13/19: [HUDI-5628] Fixing log record reader scan V2 config name (#7764)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 0d7d4c4624b6d0eae6f23326b787e76f4b15dd35 Author: Sivabalan Narayanan AuthorDate: Sat Jan 28 16:37:22 2023 -0800

[hudi] 19/19: [MINOR] Cleaning up recently introduced configs (#7772)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 5765e1b9b9663dfac08b874ca42f02245e0ce914 Author: Alexey Kudinkin AuthorDate: Sun Jan 29 14:57:05 2023 -0800 [M

[hudi] 14/19: [HUDI-5638] Fix metaserver test setup (#7785)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit be1aa615a3d816b0e00f7ec4cae2c8d13c19d2ea Author: Shiyan Xu <2701446+xushi...@users.noreply.github.com> AuthorDate: S

[hudi] 15/19: [HUDI-5640] Add missing profiles in `deploy_staging_jars.sh` (#7784)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 53c1d5dd6a1750ef77b3d435f83b6ef3613a9529 Author: Y Ethan Guo AuthorDate: Sat Jan 28 22:33:59 2023 -0800 [HUDI-

[hudi] 08/19: [HUDI-5626] Rename CDC logging mode options (#7760)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit cf52dc23db830aa23f966ca473ef3c4df85039f0 Author: Shiyan Xu <2701446+xushi...@users.noreply.github.com> AuthorDate: S

[hudi] 17/19: [MINOR] Fix `HoodieCDCRDD` setting flag `usesVirtualKeys` (#7777)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 2ce0869ef341395b286cfcc1192f055d03b0633c Author: Shiyan Xu <2701446+xushi...@users.noreply.github.com> AuthorDate: S

[hudi] 12/19: [HUDI-5624] Fix HoodieAvroRecordMerger to use new precombine API (#7759)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 01d56a6f3f9d4b87645a2d2a2aa0fced8c79857e Author: bschell AuthorDate: Sat Jan 28 12:29:31 2023 -0800 [HUDI-5624

[hudi] 16/19: [HUDI-5639] Fixing stream identifier for single writer with spark streaming ingest (#7783)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 8ccb80cb0521ee217d6cb7ae9869c3bf975be8b1 Author: Sivabalan Narayanan AuthorDate: Sat Jan 28 22:50:23 2023 -0800

[hudi] 06/19: [HUDI-5630] Fixing flaky parquet projection tests (#7768)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 01acf99e27967cec22199e2718eff30fb2a6a80c Author: Sivabalan Narayanan AuthorDate: Fri Jan 27 22:09:46 2023 -0800

[hudi] 09/19: [MINOR] Skip docs generation for table service manager (#7773)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 44cd68bdbd651fb07725d45c92a13e9d83b80764 Author: Shiyan Xu <2701446+xushi...@users.noreply.github.com> AuthorDate: S

[hudi] 18/19: [HUDI-5503] Optimize flink table factory option check (#7608)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 48977a61e26e93603085bfc6f349bce8dd72b940 Author: Bingeng Huang <304979...@qq.com> AuthorDate: Sun Jan 29 21:13:33 20

[hudi] 11/19: [HUDI-5631] Improve defaults of early conflict detection configs (#7770)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 9495918c6330a69116570d92e17e181ef7f4e2fe Author: Y Ethan Guo AuthorDate: Sat Jan 28 12:03:02 2023 -0800 [HUDI-

[hudi] 10/19: Revert "[MINOR] Add `hudi-platform-service` and `hudi-metaserver-server-bundle` to root pom (#7774)" (#7782)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 1b592c2be81c1f3438645c791a0e41ffeda89cad Author: Shiyan Xu <2701446+xushi...@users.noreply.github.com> AuthorDate: S

[hudi] 07/19: [HUDI-5629] Clean CDC log files for enable/disable scenario (#7767)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 281b29eb7b3e6a48a0598f6b01b5148fe4afc541 Author: Yann Byron AuthorDate: Sat Jan 28 14:24:33 2023 +0800 [HUDI-5

[hudi] 01/19: [HUDI-5592] Fixing some of the flaky tests in CI (#7720)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit 7f67e6abd82fb532163cde33c62ba642f3d0b65a Author: Sivabalan Narayanan AuthorDate: Thu Jan 26 16:40:04 2023 -0800

[hudi] 04/19: [HUDI-5623] Increase default time to wait between retries by lock provider client (#7758)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit df05628334cce2e4b276782a663219afe5753986 Author: Y Ethan Guo AuthorDate: Fri Jan 27 15:56:13 2023 -0800 [HUDI-

[hudi] branch release-0.13.0 updated (ca72c96bd2e -> 5765e1b9b96)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git from ca72c96bd2e [MINOR] Fix validate_staged_release.sh (#7780) new 7f67e6abd82 [HUDI-5592] Fixing some of the fl

[hudi] 03/19: [HUDI-5627] Improve the usability of Hudi CLI bundle (#7762)

2023-01-29 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch release-0.13.0 in repository https://gitbox.apache.org/repos/asf/hudi.git commit c570e6bd2711c23aa9aa898bd66d94cbdc5aa1a0 Author: Y Ethan Guo AuthorDate: Fri Jan 27 15:55:36 2023 -0800 [HUDI-

[GitHub] [hudi] voonhous commented on pull request #7669: [HUDI-5553] Prevent partition(s) from being dropped if there are pending…

2023-01-29 Thread via GitHub
voonhous commented on PR #7669: URL: https://github.com/apache/hudi/pull/7669#issuecomment-1407957877 @danny0405 Could you please to review this PR again? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] xushiyan commented on a diff in pull request #7511: [WIP][DNM][MINOR] Test bundle jars from apache repo

2023-01-29 Thread via GitHub
xushiyan commented on code in PR #7511: URL: https://github.com/apache/hudi/pull/7511#discussion_r1090152941 ## .github/workflows/bot.yml: ## @@ -13,94 +13,23 @@ env: MVN_ARGS: -ntp -B -V -Pwarn-log -Dorg.slf4j.simpleLogger.log.org.apache.maven.plugins.shade=warn -Dorg.slf4

[GitHub] [hudi] hudi-bot commented on pull request #7672: [HUDI-5557]Avoid converting columns that are not indexed in CSI

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7672: URL: https://github.com/apache/hudi/pull/7672#issuecomment-1407953961 ## CI report: * d4dbecfbbbda61fa665e242fbae644306bdfccc9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1433

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7729: [UBER] Enhance rollback logic in AbstractHoodieLogRecordReader

2023-01-29 Thread via GitHub
nsivabalan commented on code in PR #7729: URL: https://github.com/apache/hudi/pull/7729#discussion_r1090150345 ## hudi-common/src/test/java/org/apache/hudi/common/functional/TestHoodieLogFormat.java: ## @@ -1391,14 +1404,128 @@ public void testAvroLogRecordReaderWithDeleteAndRo

[GitHub] [hudi] hudi-bot commented on pull request #7672: [HUDI-5557]Avoid converting columns that are not indexed in CSI

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7672: URL: https://github.com/apache/hudi/pull/7672#issuecomment-1407950984 ## CI report: * d4dbecfbbbda61fa665e242fbae644306bdfccc9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1433

[GitHub] [hudi] hudi-bot commented on pull request #7410: [HUDI-5634] Rename CDC related classes

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7410: URL: https://github.com/apache/hudi/pull/7410#issuecomment-1407950753 ## CI report: * 3ce7c48363aae540d7d00c96edd2dae649e82fc2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1475

[GitHub] [hudi] voonhous closed pull request #6915: [HUDI-5004] Allow nested field as primary key and preCombineField in flink sql

2023-01-29 Thread via GitHub
voonhous closed pull request #6915: [HUDI-5004] Allow nested field as primary key and preCombineField in flink sql URL: https://github.com/apache/hudi/pull/6915 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [hudi] voonhous commented on pull request #6915: [HUDI-5004] Allow nested field as primary key and preCombineField in flink sql

2023-01-29 Thread via GitHub
voonhous commented on PR #6915: URL: https://github.com/apache/hudi/pull/6915#issuecomment-1407950553 Closing this PR as @hbgstc123 has already fixed this issue by disabling schema sanity checks prior to creating the source in this [PR](https://github.com/apache/hudi/pull/7608) -- This i

[GitHub] [hudi] BruceKellan commented on pull request #7719: [HUDI-5584] When the table to be synchronized already exists in hive,…

2023-01-29 Thread via GitHub
BruceKellan commented on PR #7719: URL: https://github.com/apache/hudi/pull/7719#issuecomment-1407950480 This modification adds an API method, maybe we can do it in another way. We can place storage descriptor modifications within the scope of the org.apache.hudi.hive.HoodieHiveSyncClient

[GitHub] [hudi] KnightChess opened a new issue, #7788: [SUPPORT] Insert Overwrite Partition can not use bulk insert

2023-01-29 Thread via GitHub
KnightChess opened a new issue, #7788: URL: https://github.com/apache/hudi/issues/7788 **Describe the problem you faced** hi Excuse me, why insert overwrite `partition table` can not use `bulk insert`(L144)? what problem will cause if support it?thanks https://github.com/apach

[GitHub] [hudi] hudi-bot commented on pull request #7786: [HUDI-5629] Clean CDC log files for enable/disable scenario

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7786: URL: https://github.com/apache/hudi/pull/7786#issuecomment-1407947353 ## CI report: * e95ec5e220dec18c75a11d180653224c57cfc4a9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1474

[GitHub] [hudi] hudi-bot commented on pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7761: URL: https://github.com/apache/hudi/pull/7761#issuecomment-1407947263 ## CI report: * 93e4876eb944c4b3976ac5422586ff35e8b7aef5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1466

[GitHub] [hudi] hudi-bot commented on pull request #7787: [HUDI-5646] Guard dropping columns by a config, do not allow by default

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7787: URL: https://github.com/apache/hudi/pull/7787#issuecomment-1407947389 ## CI report: * 819dd49cb6060031b722c193826797e488fcbf7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1475

[GitHub] [hudi] hudi-bot commented on pull request #7410: [HUDI-5634] Rename CDC related classes

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7410: URL: https://github.com/apache/hudi/pull/7410#issuecomment-1407946706 ## CI report: * 3ce7c48363aae540d7d00c96edd2dae649e82fc2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1474

[GitHub] [hudi] xiarixiaoyao commented on pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
xiarixiaoyao commented on PR #7761: URL: https://github.com/apache/hudi/pull/7761#issuecomment-1407917143 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] voonhous commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090132321 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/InternalSchemaManager.java: ## @@ -93,60 +93,62 @@ public InternalSchema getQuerySchema() {

[GitHub] [hudi] voonhous commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
voonhous commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090132321 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/InternalSchemaManager.java: ## @@ -93,60 +93,62 @@ public InternalSchema getQuerySchema() {

[jira] [Commented] (HUDI-5648) Deltastreamer Transformer output cannot be used in partitioning

2023-01-29 Thread David Palmer (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681851#comment-17681851 ] David Palmer commented on HUDI-5648: Related to this, is there some other way to confi

[GitHub] [hudi] xiarixiaoyao commented on pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
xiarixiaoyao commented on PR #7761: URL: https://github.com/apache/hudi/pull/7761#issuecomment-1407903228 @voonhous Thank you for your contribution With this pr our codes is more readable -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (HUDI-5648) Deltastreamer Transformer output cannot be used in partitioning

2023-01-29 Thread David Palmer (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Palmer updated HUDI-5648: --- Description: When using a Deltastreamer Transformer, the output of the Transformer cannot be used as

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #7761: [MINOR] Standardise schema concepts on Flink Engine

2023-01-29 Thread via GitHub
xiarixiaoyao commented on code in PR #7761: URL: https://github.com/apache/hudi/pull/7761#discussion_r1090126531 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/InternalSchemaManager.java: ## @@ -93,60 +93,62 @@ public InternalSchema getQuerySchema

[jira] [Created] (HUDI-5648) Deltastreamer Transformer output cannot be used in partitioning

2023-01-29 Thread David Palmer (Jira)
David Palmer created HUDI-5648: -- Summary: Deltastreamer Transformer output cannot be used in partitioning Key: HUDI-5648 URL: https://issues.apache.org/jira/browse/HUDI-5648 Project: Apache Hudi

[GitHub] [hudi] YannByron commented on pull request #7786: [HUDI-5629] Clean CDC log files for enable/disable scenario

2023-01-29 Thread via GitHub
YannByron commented on PR #7786: URL: https://github.com/apache/hudi/pull/7786#issuecomment-1407895206 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #7410: [HUDI-5634] Rename CDC related classes

2023-01-29 Thread via GitHub
hudi-bot commented on PR #7410: URL: https://github.com/apache/hudi/pull/7410#issuecomment-1407894219 ## CI report: * 3ce7c48363aae540d7d00c96edd2dae649e82fc2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] YannByron commented on pull request #7410: [HUDI-5634] Rename CDC related classes

2023-01-29 Thread via GitHub
YannByron commented on PR #7410: URL: https://github.com/apache/hudi/pull/7410#issuecomment-1407893319 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

  1   2   3   >