[GitHub] [hudi] bkosuru commented on issue #8800: GCP: Hudi stopped working in Dataproc Serverless 1.1.4

2023-06-01 Thread via GitHub
bkosuru commented on issue #8800: URL: https://github.com/apache/hudi/issues/8800#issuecomment-1572227348 It started working with hudi 0.13.1, did you fix anything in 0.13.1 to make it work? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] machadoluiz commented on issue #8824: [SUPPORT] Performance and Data Integrity Issues with Hudi for Long-Term Data Retention

2023-06-01 Thread via GitHub
machadoluiz commented on issue #8824: URL: https://github.com/apache/hudi/issues/8824#issuecomment-1572195137 @ad1happy2go, the runtime increment happens gradually. In a specific example, it reached 2 minutes and 30 seconds around 300 commits (or 10 months). This poses a challenge for us, g

[GitHub] [hudi] hudi-bot commented on pull request #8866: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8866: URL: https://github.com/apache/hudi/pull/8866#issuecomment-1572191506 ## CI report: * badb098e6bd6b0ee8b317514f08eb460659a8d93 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] hudi-bot commented on pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8745: URL: https://github.com/apache/hudi/pull/8745#issuecomment-1572190692 ## CI report: * 62377696531fc1d4ee2b7c0c86897d1cfb6b5de9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] hudi-bot commented on pull request #8865: [HUDI-6306] dynamic catalog parameter

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8865: URL: https://github.com/apache/hudi/pull/8865#issuecomment-1572082961 ## CI report: * 821e287f35e93974ae28f1e1e7a513c68749c281 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] stream2000 commented on pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

2023-06-01 Thread via GitHub
stream2000 commented on PR #8745: URL: https://github.com/apache/hudi/pull/8745#issuecomment-1572050050 @danny0405 Hi danny, could you help to review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
xiarixiaoyao commented on code in PR #8851: URL: https://github.com/apache/hudi/pull/8851#discussion_r1211598407 ## hudi-spark-datasource/hudi-spark3.0.x/src/main/java/org/apache/spark/sql/execution/datasources/parquet/Spark30HoodieVectorizedParquetRecordReader.java: ## @@ -184,

[GitHub] [hudi] hudi-bot commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1572012983 ## CI report: * 23a574b64681c95c17db47d4c63c86d7e0215ba9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1752

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1572011208 ## CI report: * dacba722974aa32f506626c106d90fa86d22cd23 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1737

[GitHub] [hudi] hudi-bot commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1571998458 ## CI report: * 23a574b64681c95c17db47d4c63c86d7e0215ba9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1752

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1571996984 ## CI report: * dacba722974aa32f506626c106d90fa86d22cd23 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1737

[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571996647 ## CI report: * 280515ea1c939f0afa7a4cd8a5593e55bd394648 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] KnightChess commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

2023-06-01 Thread via GitHub
KnightChess commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1571974348 before: https://github.com/apache/hudi/assets/20125927/9dc94884-347f-4284-8c9c-58d38a6c936a";> after: https://github.com/apache/hudi/assets/20125927/12f6bc24-42fe-4ee4-83ef-92

[GitHub] [hudi] hudi-bot commented on pull request #8868: [HUDI-6278] Fixed the use of DynamoDBLockConfig class

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8868: URL: https://github.com/apache/hudi/pull/8868#issuecomment-1571922230 ## CI report: * b6f86c770f7e35d7488cff0066d2d760453eb931 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1756

[GitHub] [hudi] hudi-bot commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1571921986 ## CI report: * 2db6852dd391973eab275dc7ef70c02bfbc5f652 UNKNOWN * 60c1399ac012bc61421f3bb1feb208decbcb6b6a UNKNOWN * e712d534d9c0a16b3027706ed394de88ff2b293d Azure: [CANCEL

[GitHub] [hudi] hudi-bot commented on pull request #8863: [HUDI-6305] s3a parameters cannot be filtered

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8863: URL: https://github.com/apache/hudi/pull/8863#issuecomment-1571909098 ## CI report: * c23542faa1651ed5fc6e6e6dadd60087d2183eee Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] hudi-bot commented on pull request #8868: [HUDI-6278] Fixed the use of DynamoDBLockConfig class

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8868: URL: https://github.com/apache/hudi/pull/8868#issuecomment-1571909212 ## CI report: * b6f86c770f7e35d7488cff0066d2d760453eb931 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1571908993 ## CI report: * 2db6852dd391973eab275dc7ef70c02bfbc5f652 UNKNOWN * 60c1399ac012bc61421f3bb1feb208decbcb6b6a UNKNOWN * be4650d94edd3eca1a15a956b6a60e2b8ec5daca Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #8759: Add metrics counters for compaction requested/completed events.

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8759: URL: https://github.com/apache/hudi/pull/8759#issuecomment-1571897075 ## CI report: * c8b0280ae3bc74278bf20f376b7a329db6580a62 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[jira] [Updated] (HUDI-6278) Dynamodb config hoodie.write.lock.dynamodb.partition_key not setting automatically

2023-06-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6278: - Labels: pull-request-available (was: ) > Dynamodb config hoodie.write.lock.dynamodb.partition_key

[GitHub] [hudi] ad1happy2go opened a new pull request, #8868: [HUDI-6278] Fixed the use of DynamoDBLockConfig class

2023-06-01 Thread via GitHub
ad1happy2go opened a new pull request, #8868: URL: https://github.com/apache/hudi/pull/8868 Change Logs Fixed use of DynamoDBLockConfig class Impact none Risk level (write none, low medium or high below) low Documentation Update none Contributor's c

[GitHub] [hudi] hudi-bot commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1571833174 ## CI report: * 6a8fa73c9e31a90f6249772b5b840acf42ae1df5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1756

[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1571831003 ## CI report: * 8082df232089396b2a9f9be2b915e51b3645f172 UNKNOWN * 55ece2d5ee0d752f48d7c33cd6b8235b5ffacac5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] reswqa commented on pull request #8861: [HUDI-6303] Bump flink version to 1.16.2 and 1.17.1

2023-06-01 Thread via GitHub
reswqa commented on PR #8861: URL: https://github.com/apache/hudi/pull/8861#issuecomment-1571822573 Thanks @danny0405 for the quick review! 🙇 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[hudi] branch master updated: [HUDI-6303] Bump flink version to 1.16.2 and 1.17.1 (#8861)

2023-06-01 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new e8ca0d4121a [HUDI-6303] Bump flink version to 1

[GitHub] [hudi] danny0405 merged pull request #8861: [HUDI-6303] Bump flink version to 1.16.2 and 1.17.1

2023-06-01 Thread via GitHub
danny0405 merged PR #8861: URL: https://github.com/apache/hudi/pull/8861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] hudi-bot commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1571817445 ## CI report: * 6a8fa73c9e31a90f6249772b5b840acf42ae1df5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8792: [HUDI-6256] Fix the data table archiving and MDT cleaning config conf…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8792: URL: https://github.com/apache/hudi/pull/8792#issuecomment-1571816982 ## CI report: * 683dc368e714ace1c44d741d642f1fe64b7910b2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754

[GitHub] [hudi] danny0405 commented on pull request #8792: [HUDI-6256] Fix the data table archiving and MDT cleaning config conf…

2023-06-01 Thread via GitHub
danny0405 commented on PR #8792: URL: https://github.com/apache/hudi/pull/8792#issuecomment-1571816278 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1571815820 ## CI report: * 8082df232089396b2a9f9be2b915e51b3645f172 UNKNOWN * 55ece2d5ee0d752f48d7c33cd6b8235b5ffacac5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8795: URL: https://github.com/apache/hudi/pull/8795#issuecomment-1571807209 ## CI report: * 130523be1324218f56ce15ddc6ac3255e7cfcd9a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[jira] [Updated] (HUDI-6307) Sync TIMESTAMP_MILLIS to hive

2023-06-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6307: - Labels: pull-request-available (was: ) > Sync TIMESTAMP_MILLIS to hive >

[GitHub] [hudi] xicm opened a new pull request, #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

2023-06-01 Thread via GitHub
xicm opened a new pull request, #8867: URL: https://github.com/apache/hudi/pull/8867 ### Change Logs Sync TIMESTAMP_MILLIS to hive https://github.com/apache/hudi/issues/8844#issuecomment-1571299404 ### Impact none ### Risk level (write none, low medium or hi

[jira] [Updated] (HUDI-6307) Sync TIMESTAMP_MILLIS to hive

2023-06-01 Thread xi chaomin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xi chaomin updated HUDI-6307: - Description: https://github.com/apache/hudi/issues/8844#issuecomment-1571299404 > Sync TIMESTAMP_MILLIS t

[jira] [Created] (HUDI-6307) Sync TIMESTAMP_MILLIS to hive

2023-06-01 Thread xi chaomin (Jira)
xi chaomin created HUDI-6307: Summary: Sync TIMESTAMP_MILLIS to hive Key: HUDI-6307 URL: https://issues.apache.org/jira/browse/HUDI-6307 Project: Apache Hudi Issue Type: Improvement R

[GitHub] [hudi] hudi-bot commented on pull request #8866: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8866: URL: https://github.com/apache/hudi/pull/8866#issuecomment-1571737105 ## CI report: * badb098e6bd6b0ee8b317514f08eb460659a8d93 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] hudi-bot commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1571736923 ## CI report: * 2db6852dd391973eab275dc7ef70c02bfbc5f652 UNKNOWN * 60c1399ac012bc61421f3bb1feb208decbcb6b6a UNKNOWN * be4650d94edd3eca1a15a956b6a60e2b8ec5daca Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8745: URL: https://github.com/apache/hudi/pull/8745#issuecomment-1571736084 ## CI report: * 5b623fd433f9e52d0464f82fda115a08353da6da Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1740

[GitHub] [hudi] hudi-bot commented on pull request #8866: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8866: URL: https://github.com/apache/hudi/pull/8866#issuecomment-1571724456 ## CI report: * badb098e6bd6b0ee8b317514f08eb460659a8d93 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1571724297 ## CI report: * 2db6852dd391973eab275dc7ef70c02bfbc5f652 UNKNOWN * 60c1399ac012bc61421f3bb1feb208decbcb6b6a UNKNOWN * be4650d94edd3eca1a15a956b6a60e2b8ec5daca Azure: [FAILUR

[GitHub] [hudi] hudi-bot commented on pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8745: URL: https://github.com/apache/hudi/pull/8745#issuecomment-1571723780 ## CI report: * 5b623fd433f9e52d0464f82fda115a08353da6da Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1740

[GitHub] [hudi] voonhous opened a new pull request, #8866: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task…

2023-06-01 Thread via GitHub
voonhous opened a new pull request, #8866: URL: https://github.com/apache/hudi/pull/8866 … more reasonable Synchronizing the changes made in !8854 with HoodieFlinkClusteringJob. ### Change Logs When the `clustering_tasks` was set, take the minimum of `inputGroupSize` an

[GitHub] [hudi] SteNicholas commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
SteNicholas commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1571658635 @xiarixiaoyao, could you help to review this pull request? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hudi] hudi-bot commented on pull request #8865: [HUDI-6306] dynamic catalog parameter

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8865: URL: https://github.com/apache/hudi/pull/8865#issuecomment-1571640945 ## CI report: * 821e287f35e93974ae28f1e1e7a513c68749c281 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] hudi-bot commented on pull request #8859: set operation to bulk insert on fresh table

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8859: URL: https://github.com/apache/hudi/pull/8859#issuecomment-1571640814 ## CI report: * 8f4d284eef389ff420b473b9a721212de46302da Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754

[GitHub] [hudi] SteNicholas closed pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
SteNicholas closed pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value URL: https://github.com/apache/hudi/pull/8851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hudi] xuzifu666 closed pull request #8632: [HUDI-6171] Fix Testcase not run on windows system for SparkSQL TestCase

2023-06-01 Thread via GitHub
xuzifu666 closed pull request #8632: [HUDI-6171] Fix Testcase not run on windows system for SparkSQL TestCase URL: https://github.com/apache/hudi/pull/8632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] hudi-bot commented on pull request #8865: [HUDI-6306] dynamic catalog parameter

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8865: URL: https://github.com/apache/hudi/pull/8865#issuecomment-1571627328 ## CI report: * 821e287f35e93974ae28f1e1e7a513c68749c281 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] SteNicholas closed pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

2023-06-01 Thread via GitHub
SteNicholas closed pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value URL: https://github.com/apache/hudi/pull/8851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571625646 ## CI report: * 195a90f292a61ec35698ab1d2472c56316c4a611 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1731

[GitHub] [hudi] hudi-bot commented on pull request #8861: [HUDI-6303] Bump flink version to 1.16.2 and 1.17.1

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8861: URL: https://github.com/apache/hudi/pull/8861#issuecomment-1571616216 ## CI report: * 611580e4214c5a3cc1922f06fb820b5fc3a7355e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754

[GitHub] [hudi] hudi-bot commented on pull request #8792: [HUDI-6256] Fix the data table archiving and MDT cleaning config conf…

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8792: URL: https://github.com/apache/hudi/pull/8792#issuecomment-1571615821 ## CI report: * 683dc368e714ace1c44d741d642f1fe64b7910b2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1754

[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571614677 ## CI report: * 195a90f292a61ec35698ab1d2472c56316c4a611 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1731

[GitHub] [hudi] xushiyan commented on a diff in pull request #8860: Fix spark sql core flow

2023-06-01 Thread via GitHub
xushiyan commented on code in PR #8860: URL: https://github.com/apache/hudi/pull/8860#discussion_r1212806986 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestSparkSqlCoreFlow.scala: ## @@ -46,18 +47,18 @@ class TestSparkSqlCoreFlow extends Hoodie

[GitHub] [hudi] pratyakshsharma commented on pull request #8836: [MINOR] Remove unnecessary code in DeltaSync#syncOnce

2023-06-01 Thread via GitHub
pratyakshsharma commented on PR #8836: URL: https://github.com/apache/hudi/pull/8836#issuecomment-1571608525 Good catch @voonhous -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[jira] [Updated] (HUDI-6306) Underlying multi-storage support

2023-06-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6306: - Labels: pull-request-available (was: ) > Underlying multi-storage support > -

[GitHub] [hudi] linfey90 opened a new pull request, #8865: [HUDI-6306] dynamic catalog parameter

2023-06-01 Thread via GitHub
linfey90 opened a new pull request, #8865: URL: https://github.com/apache/hudi/pull/8865 ### Change Logs When the underlying storage is multiple, such as hdfs, minio, and obs, parameters need to be dynamically transmitted to the underlying file system through catalog. Therefore, para

[jira] [Created] (HUDI-6306) Underlying multi-storage support

2023-06-01 Thread linfey.nie (Jira)
linfey.nie created HUDI-6306: Summary: Underlying multi-storage support Key: HUDI-6306 URL: https://issues.apache.org/jira/browse/HUDI-6306 Project: Apache Hudi Issue Type: Bug Report

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-01 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1212748606 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java: ## @@ -338,14 +338,20 @@ protected void initMetadataTable(Option instan

[GitHub] [hudi] ChangjunZhang closed issue #8864: [SUPPORT]NoSuchMethodException: org.apache.avro.generic.GenericRecord.(org.apache.hudi.org.apache.avro.generic.GenericRecord, java.lang.Comparab

2023-06-01 Thread via GitHub
ChangjunZhang closed issue #8864: [SUPPORT]NoSuchMethodException: org.apache.avro.generic.GenericRecord.(org.apache.hudi.org.apache.avro.generic.GenericRecord, java.lang.Comparable) URL: https://github.com/apache/hudi/issues/8864 -- This is an automated message from the Apache Git Service.

[GitHub] [hudi] ChangjunZhang commented on issue #8864: [SUPPORT]NoSuchMethodException: org.apache.avro.generic.GenericRecord.(org.apache.hudi.org.apache.avro.generic.GenericRecord, java.lang.Co

2023-06-01 Thread via GitHub
ChangjunZhang commented on issue #8864: URL: https://github.com/apache/hudi/issues/8864#issuecomment-1571557398 resoved: remove hudi-aws-bundle-0.13.1.jar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Closed] (HUDI-6152) Check for valid commit during tagLocation does not handle mixed timestamps in the dataset

2023-06-01 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason closed HUDI-6152. Resolution: Fixed > Check for valid commit during tagLocation does not handle mixed timestamps in >

[jira] [Closed] (HUDI-6060) Add config to backup instants before deletion during rollbacks

2023-06-01 Thread Prashant Wason (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason closed HUDI-6060. Resolution: Fixed > Add config to backup instants before deletion during rollbacks > ---

[GitHub] [hudi] hudi-bot commented on pull request #8497: [HUDI-6102] HoodieAvroUtils refactor with remove unused methods

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8497: URL: https://github.com/apache/hudi/pull/8497#issuecomment-1571536185 ## CI report: * 5d05d9d3dec138710416cdea58bcc02d3745f8df Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1645

[GitHub] [hudi] ChangjunZhang commented on issue #8864: [SUPPORT]NoSuchMethodException: org.apache.avro.generic.GenericRecord.(org.apache.hudi.org.apache.avro.generic.GenericRecord, java.lang.Co

2023-06-01 Thread via GitHub
ChangjunZhang commented on issue #8864: URL: https://github.com/apache/hudi/issues/8864#issuecomment-1571531663 why to find org.apache.hudi.org.apache.avro.generic.GenericRecord ? right class is :org.apache.avro.generic.GenericRecord -- This is an automated message from the Apache Git Ser

[GitHub] [hudi] danny0405 commented on a diff in pull request #8830: [MINOR] auto generate init client id

2023-06-01 Thread via GitHub
danny0405 commented on code in PR #8830: URL: https://github.com/apache/hudi/pull/8830#discussion_r1212738284 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/configuration/TestOptionsInference.java: ## @@ -69,6 +70,12 @@ void testSetupClientId() throws Exceptio

[GitHub] [hudi] ChangjunZhang opened a new issue, #8864: [SUPPORT]NoSuchMethodException: org.apache.avro.generic.GenericRecord.(org.apache.hudi.org.apache.avro.generic.GenericRecord, java.lang.C

2023-06-01 Thread via GitHub
ChangjunZhang opened a new issue, #8864: URL: https://github.com/apache/hudi/issues/8864 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at dev-su

[GitHub] [hudi] hudi-bot commented on pull request #8863: [HUDI-6305] s3a parameters cannot be filtered

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8863: URL: https://github.com/apache/hudi/pull/8863#issuecomment-1571526779 ## CI report: * c23542faa1651ed5fc6e6e6dadd60087d2183eee Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1755

[GitHub] [hudi] danny0405 commented on a diff in pull request #8526: [HUDI-6116] Optimize log block reading by removing seeks to check corrupted blocks.

2023-06-01 Thread via GitHub
danny0405 commented on code in PR #8526: URL: https://github.com/apache/hudi/pull/8526#discussion_r1212732181 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java: ## @@ -152,98 +153,107 @@ private void addShutDownHook() { // TODO : convert c

[GitHub] [hudi] danny0405 commented on a diff in pull request #8526: [HUDI-6116] Optimize log block reading by removing seeks to check corrupted blocks.

2023-06-01 Thread via GitHub
danny0405 commented on code in PR #8526: URL: https://github.com/apache/hudi/pull/8526#discussion_r1212724688 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java: ## @@ -152,98 +153,107 @@ private void addShutDownHook() { // TODO : convert c

[GitHub] [hudi] c-f-cooper commented on a diff in pull request #8830: [MINOR] auto generate init client id

2023-06-01 Thread via GitHub
c-f-cooper commented on code in PR #8830: URL: https://github.com/apache/hudi/pull/8830#discussion_r1212716258 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/configuration/TestOptionsInference.java: ## @@ -69,6 +70,12 @@ void testSetupClientId() throws Excepti

[GitHub] [hudi] danny0405 commented on a diff in pull request #8830: [MINOR] auto generate init client id

2023-06-01 Thread via GitHub
danny0405 commented on code in PR #8830: URL: https://github.com/apache/hudi/pull/8830#discussion_r1212707111 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/configuration/TestOptionsInference.java: ## @@ -69,6 +70,12 @@ void testSetupClientId() throws Exceptio

[GitHub] [hudi] hudi-bot commented on pull request #8863: [HUDI-6305] s3a parameters cannot be filtered

2023-06-01 Thread via GitHub
hudi-bot commented on PR #8863: URL: https://github.com/apache/hudi/pull/8863#issuecomment-1571470413 ## CI report: * c23542faa1651ed5fc6e6e6dadd60087d2183eee UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

<    1   2