[GitHub] [hudi] ad1happy2go commented on issue #8253: [SUPPORT]HoodieJavaWriteClientExample Process finished with exit code 137 (interrupted by signal 9: SIGKILL) with jol-core 0.16

2023-07-04 Thread via GitHub
ad1happy2go commented on issue #8253: URL: https://github.com/apache/hudi/issues/8253#issuecomment-1621063805 @Mulavar Sorry for delay on this, But I am able to successfully run the HoodieJavaWriteClientExample with the this jdk version. Looks to be laptop issue only , so closing the

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1621056203 ## CI report: * ef7585ba8d32d772500f31f95f3c04bfcac046e7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9122: URL: https://github.com/apache/hudi/pull/9122#issuecomment-1621023436 ## CI report: * 2f44e3cd97dbc108faabcdd5da0d805b1680e211 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1621023467 ## CI report: * 7708ff75ba467e2156b6396ee2886ec645b7b44f Azure:

[GitHub] [hudi] danny0405 commented on a diff in pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #9122: URL: https://github.com/apache/hudi/pull/9122#discussion_r1252517410 ## hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java: ## @@ -144,18 +144,7 @@ public BaseHoodieTableFileIndex(HoodieEngineContext engineContext,

[GitHub] [hudi] danny0405 commented on a diff in pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #9122: URL: https://github.com/apache/hudi/pull/9122#discussion_r1252517112 ## hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java: ## @@ -144,18 +144,7 @@ public BaseHoodieTableFileIndex(HoodieEngineContext engineContext,

[jira] [Closed] (HUDI-6476) Improve the performance of getAllPartitionPaths

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6476. Resolution: Fixed Fixed via master branch: 72f047715fe8f2ad9ff19a31728fbfb761fbe0d9 > Improve the

[jira] [Updated] (HUDI-6476) Improve the performance of getAllPartitionPaths

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6476: - Fix Version/s: 0.14.0 > Improve the performance of getAllPartitionPaths >

[hudi] branch master updated: [HUDI-6476] Improve the performance of getAllPartitionPaths (#9121)

2023-07-04 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 72f047715fe [HUDI-6476] Improve the

[GitHub] [hudi] danny0405 merged pull request #9121: [HUDI-6476] Improve the performance of getAllPartitionPaths

2023-07-04 Thread via GitHub
danny0405 merged PR #9121: URL: https://github.com/apache/hudi/pull/9121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on a diff in pull request #9121: [HUDI-6476] Improve the performance of getAllPartitionPaths

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #9121: URL: https://github.com/apache/hudi/pull/9121#discussion_r1252515100 ## hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java: ## @@ -106,42 +107,33 @@ private List getPartitionPathWithPathPrefix(String

[GitHub] [hudi] flashJd commented on pull request #9113: [HUDI-6466] Fix spark insert overwrite partitioned table with dynamic partition

2023-07-04 Thread via GitHub
flashJd commented on PR #9113: URL: https://github.com/apache/hudi/pull/9113#issuecomment-1620990951 I'm confused why `insert overwrite hudi_cow_pt_tbl select 13, 'a13', 1100, '2021-12-09', '12' is a not dynamic partition writing?` the semantics should can be controled by config. we

[GitHub] [hudi] hudi-bot commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620980724 ## CI report: * 5b52b7900c734adba70ac16da20bdc23f21b01d0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620976354 ## CI report: * 5b52b7900c734adba70ac16da20bdc23f21b01d0 Azure:

[GitHub] [hudi] danny0405 commented on a diff in pull request #9106: [HUDI-6118] Some fixes to improve the MDT and record index code base.

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #9106: URL: https://github.com/apache/hudi/pull/9106#discussion_r1252501322 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java: ## @@ -91,8 +108,9 @@ public static HoodieWriteConfig

[GitHub] [hudi] danny0405 commented on a diff in pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #8837: URL: https://github.com/apache/hudi/pull/8837#discussion_r1252500090 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -851,26 +919,49 @@ public void

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620970058 ## CI report: * 984c3d691c3e7915fb1333ee823a641098774270 Azure:

[GitHub] [hudi] danny0405 commented on issue #9119: [SUPPORT] ERROR BaseSparkCommitActionExecutor: Error upserting bucketType UPDATE for partition :13

2023-07-04 Thread via GitHub
danny0405 commented on issue #9119: URL: https://github.com/apache/hudi/issues/9119#issuecomment-1620968741 Sorry for the unstability, we will be more conservative about code reviewing and merging in the future. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] danny0405 commented on a diff in pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #9123: URL: https://github.com/apache/hudi/pull/9123#discussion_r1252497920 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -112,6 +113,36 @@ trait ProvidesHoodieConfig

[GitHub] [hudi] danny0405 commented on a diff in pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

2023-07-04 Thread via GitHub
danny0405 commented on code in PR #9123: URL: https://github.com/apache/hudi/pull/9123#discussion_r1252496143 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala: ## @@ -1094,6 +1094,11 @@ object HoodieSparkSqlWriter { if

[jira] [Updated] (HUDI-6475) Optimize TableNotFoundException message

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6475: - Fix Version/s: 0.14.0 > Optimize TableNotFoundException message > ---

[jira] [Closed] (HUDI-6475) Optimize TableNotFoundException message

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6475. Resolution: Fixed Fixed via master branch: 2322ac9d22784df2ccebcbdf898286c16fe0c211 > Optimize

[hudi] branch master updated: [HUDI-6475] Optimize TableNotFoundException message (#9120)

2023-07-04 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 2322ac9d227 [HUDI-6475] Optimize

[GitHub] [hudi] danny0405 merged pull request #9120: [HUDI-6475] Optimize TableNotFoundException message

2023-07-04 Thread via GitHub
danny0405 merged PR #9120: URL: https://github.com/apache/hudi/pull/9120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
danny0405 commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620954248 Thanks for the contribution, it is greate if we can have details to explain that can help the reviewers to get the context more quickly. -- This is an automated message from the Apache

[jira] [Closed] (HUDI-6329) Introduce UpdateStrategy for Flink to handle conflict between clustering/resize with update

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6329. Resolution: Fixed Fixed via master branch: e8b1ddd708bc2ba99144f92d7533c7200f12509f > Introduce

[jira] [Updated] (HUDI-6329) Introduce UpdateStrategy for Flink to handle conflict between clustering/resize with update

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6329: - Fix Version/s: 0.14.0 > Introduce UpdateStrategy for Flink to handle conflict between >

[hudi] branch master updated: [HUDI-6329] Adjust the partitioner automatically for flink consistent hashing index (#9087)

2023-07-04 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new e8b1ddd708b [HUDI-6329] Adjust the partitioner

[GitHub] [hudi] danny0405 merged pull request #9087: [HUDI-6329] Adjust the partitioner automatically for flink consistent hashing index

2023-07-04 Thread via GitHub
danny0405 merged PR #9087: URL: https://github.com/apache/hudi/pull/9087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on pull request #9087: [HUDI-6329] Adjust the partitioner automatically for flink consistent hashing index

2023-07-04 Thread via GitHub
danny0405 commented on PR #9087: URL: https://github.com/apache/hudi/pull/9087#issuecomment-1620951822 Tests have passed: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=18287=results -- This is an automated message from the Apache Git Service. To respond

[jira] [Closed] (HUDI-6423) Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6423. Resolution: Fixed Fixed via master branch: 07164406c44b4092eee810710a242d092c97bd58 > Incremental cleaning

[hudi] branch master updated: [HUDI-6423] Incremental cleaning should consider inflight compaction instant (#9038)

2023-07-04 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 07164406c44 [HUDI-6423] Incremental cleaning

[GitHub] [hudi] danny0405 merged pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
danny0405 merged PR #9038: URL: https://github.com/apache/hudi/pull/9038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9122: URL: https://github.com/apache/hudi/pull/9122#issuecomment-1620942995 ## CI report: * 2f44e3cd97dbc108faabcdd5da0d805b1680e211 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1620943023 ## CI report: * 7708ff75ba467e2156b6396ee2886ec645b7b44f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620942945 ## CI report: * 984c3d691c3e7915fb1333ee823a641098774270 Azure:

[GitHub] [hudi] Zouxxyy closed pull request #9051: [HUDI-6436] Make the function of AlterHoodieTableChangeColumnCommand …

2023-07-04 Thread via GitHub
Zouxxyy closed pull request #9051: [HUDI-6436] Make the function of AlterHoodieTableChangeColumnCommand … URL: https://github.com/apache/hudi/pull/9051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] hudi-bot commented on pull request #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9122: URL: https://github.com/apache/hudi/pull/9122#issuecomment-1620937689 ## CI report: * 2f44e3cd97dbc108faabcdd5da0d805b1680e211 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9123: URL: https://github.com/apache/hudi/pull/9123#issuecomment-1620937715 ## CI report: * 7708ff75ba467e2156b6396ee2886ec645b7b44f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Created] (HUDI-6479) Update release docs and quick start guide around INSERT_INTO default behavior change

2023-07-04 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-6479: - Summary: Update release docs and quick start guide around INSERT_INTO default behavior change Key: HUDI-6479 URL: https://issues.apache.org/jira/browse/HUDI-6479

[jira] [Updated] (HUDI-6478) Simplify INSERT_INTO configs

2023-07-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6478: - Labels: pull-request-available (was: ) > Simplify INSERT_INTO configs >

[GitHub] [hudi] nsivabalan opened a new pull request, #9123: [HUDI-6478] Simplifying INSERT_INTO configs for spark-sql

2023-07-04 Thread via GitHub
nsivabalan opened a new pull request, #9123: URL: https://github.com/apache/hudi/pull/9123 ### Change Logs With the intent to simplify different config options with INSERT_INTO spark-sql, we are doing a overhaul. We have 3 to 4 configs with INSERT_INTO like Operation type, insert

[jira] [Created] (HUDI-6478) Simplify INSERT_INTO configs

2023-07-04 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-6478: - Summary: Simplify INSERT_INTO configs Key: HUDI-6478 URL: https://issues.apache.org/jira/browse/HUDI-6478 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-6477) Lazy fetching partition path & file slice when refresh in HoodieFileIndex

2023-07-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6477: - Labels: pull-request-available (was: ) > Lazy fetching partition path & file slice when refresh

[GitHub] [hudi] Zouxxyy opened a new pull request, #9122: [HUDI-6477] Lazy fetching partition path & file slice when refresh in…

2023-07-04 Thread via GitHub
Zouxxyy opened a new pull request, #9122: URL: https://github.com/apache/hudi/pull/9122 … HoodieFileIndex ### Change Logs Currently there is a lazy list mechanism in `hoodieFileIndex`, but it only takes effect during initialization. We can make it take effect when refresh. At

[jira] [Created] (HUDI-6477) Lazy fetching partition path & file slice when refresh in HoodieFileIndex

2023-07-04 Thread zouxxyy (Jira)
zouxxyy created HUDI-6477: - Summary: Lazy fetching partition path & file slice when refresh in HoodieFileIndex Key: HUDI-6477 URL: https://issues.apache.org/jira/browse/HUDI-6477 Project: Apache Hudi

[GitHub] [hudi] boneanxs commented on pull request #9113: [HUDI-6466] Fix spark insert overwrite partitioned table with dynamic partition

2023-07-04 Thread via GitHub
boneanxs commented on PR #9113: URL: https://github.com/apache/hudi/pull/9113#issuecomment-1620905284 You can still use dynamic partition, in this way: ```sql insert overwrite hudi_cow_pt_tbl partition(dt, hh) select 13, 'a13', 1100, '2021-12-09', '12' ``` the main

[GitHub] [hudi] hudi-bot commented on pull request #9083: [HUDI-6464] Spark SQL Merge Into for pkless tables

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9083: URL: https://github.com/apache/hudi/pull/9083#issuecomment-1620904971 ## CI report: * 3a0bfb88049cf2c0f8afe5c925dbd76fa6f7cd89 UNKNOWN * 1d32092354e9065499631ed860a09a9c918c088d Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9006: [HUDI-6404] Implement ParquetToolsExecutionStrategy for clustering

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9006: URL: https://github.com/apache/hudi/pull/9006#issuecomment-1620904718 ## CI report: * b385ea4a4d4b7986ba27f5df352686652dc53c36 Azure:

[GitHub] [hudi] flashJd commented on pull request #9113: [HUDI-6466] Fix spark insert overwrite partitioned table with dynamic partition

2023-07-04 Thread via GitHub
flashJd commented on PR #9113: URL: https://github.com/apache/hudi/pull/9113#issuecomment-1620903128 As we need the capacity to insert overwrite the whole partitioned table, why not use the config to enable it and make semantics forward compatible, meanwhile not lose the dynamic partition

[GitHub] [hudi] flashJd commented on pull request #9113: [HUDI-6466] Fix spark insert overwrite partitioned table with dynamic partition

2023-07-04 Thread via GitHub
flashJd commented on PR #9113: URL: https://github.com/apache/hudi/pull/9113#issuecomment-1620899951 > @flashJd I noticed this issue before. Yes, this is a behavior change for `INSERT_OVERWRITE` without partition columns after #7365, but I think it's the right modification? if users don't

[GitHub] [hudi] Zouxxyy commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
Zouxxyy commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620893285 > It's greate if we can add a simple test case. done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620864476 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * cb6fd2a6af75b79129b86a56f02a4566e2fe4e4f Azure:

[GitHub] [hudi] yihua commented on a diff in pull request #9066: [HUDI-6452] Add MOR snapshot reader to integrate with query engines without using Hadoop APIs

2023-07-04 Thread via GitHub
yihua commented on code in PR #9066: URL: https://github.com/apache/hudi/pull/9066#discussion_r1252417782 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieMergeOnReadSnapshotReader.java: ## @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620837762 ## CI report: * 984c3d691c3e7915fb1333ee823a641098774270 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9083: [HUDI-6464] Spark SQL Merge Into for pkless tables

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9083: URL: https://github.com/apache/hudi/pull/9083#issuecomment-1620837706 ## CI report: * 3a0bfb88049cf2c0f8afe5c925dbd76fa6f7cd89 UNKNOWN * f156c1694aca3a9e2ca4ed26959c6a5a1b773354 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9083: [HUDI-6464] Spark SQL Merge Into for pkless tables

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9083: URL: https://github.com/apache/hudi/pull/9083#issuecomment-1620833117 ## CI report: * 3a0bfb88049cf2c0f8afe5c925dbd76fa6f7cd89 UNKNOWN * f156c1694aca3a9e2ca4ed26959c6a5a1b773354 Azure:

[GitHub] [hudi] nsivabalan commented on a diff in pull request #9106: [HUDI-6118] Some fixes to improve the MDT and record index code base.

2023-07-04 Thread via GitHub
nsivabalan commented on code in PR #9106: URL: https://github.com/apache/hudi/pull/9106#discussion_r1252401000 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java: ## @@ -116,11 +134,10 @@ public static HoodieWriteConfig

[GitHub] [hudi] hudi-bot commented on pull request #9006: [HUDI-6404] Implement ParquetToolsExecutionStrategy for clustering

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9006: URL: https://github.com/apache/hudi/pull/9006#issuecomment-1620806187 ## CI report: * 775343a4b7c9d72e3476ddee84078883af27f01e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9006: [HUDI-6404] Implement ParquetToolsExecutionStrategy for clustering

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9006: URL: https://github.com/apache/hudi/pull/9006#issuecomment-1620801120 ## CI report: * 775343a4b7c9d72e3476ddee84078883af27f01e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9105: [HUDI-6459] Add Rollback and multi-writer tests for Record Level Index

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9105: URL: https://github.com/apache/hudi/pull/9105#issuecomment-1620798243 ## CI report: * fad064d3590670a75b8f68c5eca91e059d235241 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9038: URL: https://github.com/apache/hudi/pull/9038#issuecomment-1620798158 ## CI report: * a65a29c0cf1c8feb9f39e168ba80c99ebcae1c5d UNKNOWN * 43c37c8a48763d8fdf71937fab4ccb900b313385 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9121: [HUDI-6476] Improve the performance of getAllPartitionPaths

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9121: URL: https://github.com/apache/hudi/pull/9121#issuecomment-1620766916 ## CI report: * 8555b51e9fa8f7ec9096df39d11e81d8b5177015 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

2023-07-04 Thread via GitHub
hudi-bot commented on PR #8837: URL: https://github.com/apache/hudi/pull/8837#issuecomment-1620722497 ## CI report: * e6568126aab0b098ccaac59e137e902d7a1070c3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620714045 ## CI report: * 5b52b7900c734adba70ac16da20bdc23f21b01d0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620665909 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * 7898678550ef22db9e564d5a4bef2b7845e6b5e0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620658010 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * 035aa770c2fdeb9dcd9e91097f41904d39bca70f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620652006 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * 035aa770c2fdeb9dcd9e91097f41904d39bca70f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620622515 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * c37cc8fa71f68c1088ac1d06fbe34635776f1e14 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620617639 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * 7f04db759666f31a92888564d16216943674ac5b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9064: [HUDI-6450] Fix null strings handling in convertRowToJsonString

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9064: URL: https://github.com/apache/hudi/pull/9064#issuecomment-1620617516 ## CI report: * b8418b74febf4551c0f79c7ebe71cf24916124e6 UNKNOWN * 3e0876320ac294a7da6c81a8b26630ed518606cd Azure:

[GitHub] [hudi] GallonREX commented on issue #7925: [SUPPORT]hudi 0.8 upgrade to hudi 0.12 report java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes

2023-07-04 Thread via GitHub
GallonREX commented on issue #7925: URL: https://github.com/apache/hudi/issues/7925#issuecomment-1620580148 这是自动回复。谢谢您的邮件,您的邮件我已收到,我将尽快回复您。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] ad1happy2go commented on issue #7925: [SUPPORT]hudi 0.8 upgrade to hudi 0.12 report java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes

2023-07-04 Thread via GitHub
ad1happy2go commented on issue #7925: URL: https://github.com/apache/hudi/issues/7925#issuecomment-1620579931 @GallonREX The error what you getting `Cannot resolve conflicts for overlapping writes` is normally comes when you try to update the same file group concurrently. This should not

[GitHub] [hudi] hudi-bot commented on pull request #8796: [HUDI-6129] Support rate limit for Spark streaming source

2023-07-04 Thread via GitHub
hudi-bot commented on PR #8796: URL: https://github.com/apache/hudi/pull/8796#issuecomment-1620576704 ## CI report: * 6c568f15e26e072d07cdb5de7e7a39fa2b9fbc6f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9120: [HUDI-6475] Optimize TableNotFoundException message

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9120: URL: https://github.com/apache/hudi/pull/9120#issuecomment-1620571410 ## CI report: * ac6f163af4a9ab33b78a9304b25babc7caa90714 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9118: [HUDI-2141] Support flink write metrics

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9118: URL: https://github.com/apache/hudi/pull/9118#issuecomment-1620520120 ## CI report: * f6d7dd97c73898206da91b17144326a7dbbffae8 UNKNOWN * 6127808e39fcbf9e2acae98666887a455e0e926e Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620477542 ## CI report: * df41145f4bfa32fbd1f705cd6d04b74a93a0747a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9038: URL: https://github.com/apache/hudi/pull/9038#issuecomment-1620476985 ## CI report: * a65a29c0cf1c8feb9f39e168ba80c99ebcae1c5d UNKNOWN * 5b354dd07b4381c270e17001a1010141bf7086e8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9113: [HUDI-6466] Fix spark insert overwrite partitioned table with dynamic partition

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9113: URL: https://github.com/apache/hudi/pull/9113#issuecomment-1620477463 ## CI report: * 72e9fc345a516c34387ba34d5fde2f8ea631b404 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9105: [HUDI-6459] Add Rollback and multi-writer tests for Record Level Index

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9105: URL: https://github.com/apache/hudi/pull/9105#issuecomment-1620477346 ## CI report: * fad064d3590670a75b8f68c5eca91e059d235241 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620477273 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * 7f04db759666f31a92888564d16216943674ac5b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620467909 ## CI report: * df41145f4bfa32fbd1f705cd6d04b74a93a0747a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9105: [HUDI-6459] Add Rollback and multi-writer tests for Record Level Index

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9105: URL: https://github.com/apache/hudi/pull/9105#issuecomment-1620467764 ## CI report: * fad064d3590670a75b8f68c5eca91e059d235241 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620467693 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * ec568a0c309690a1b0931249aae1e4aab9eddc9b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9038: URL: https://github.com/apache/hudi/pull/9038#issuecomment-1620467397 ## CI report: * a65a29c0cf1c8feb9f39e168ba80c99ebcae1c5d UNKNOWN * 59d464a6e1f7a69ba0d0ab331ad01e3ed66f8e62 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9121: [HUDI-6476] Improve the performance of getAllPartitionPaths

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9121: URL: https://github.com/apache/hudi/pull/9121#issuecomment-1620456769 ## CI report: * 8555b51e9fa8f7ec9096df39d11e81d8b5177015 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620456654 ## CI report: * 2a046240c1e7c0a18f9b57c0845298ea65b72951 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620456458 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * ec568a0c309690a1b0931249aae1e4aab9eddc9b Azure:

[GitHub] [hudi] BBency commented on issue #9094: Async Clustering failing with errors for MOR table

2023-07-04 Thread via GitHub
BBency commented on issue #9094: URL: https://github.com/apache/hudi/issues/9094#issuecomment-1620436262 Approach 1: ![image](https://github.com/apache/hudi/assets/118782050/ddd0627a-3909-4237-bbca-89965860ebb0) Approach 2:

[GitHub] [hudi] Alowator commented on pull request #9112: [HUDI-6465] Fix data skipping support BIGINT

2023-07-04 Thread via GitHub
Alowator commented on PR #9112: URL: https://github.com/apache/hudi/pull/9112#issuecomment-1620428039 If there is no any suggestions or questions, it could be merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] zhuanshenbsj1 commented on a diff in pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
zhuanshenbsj1 commented on code in PR #9038: URL: https://github.com/apache/hudi/pull/9038#discussion_r1252072216 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java: ## @@ -111,14 +111,15 @@ HoodieCleanerPlan

[GitHub] [hudi] hudi-bot commented on pull request #9121: [HUDI-6476] Improve the performance of getAllPartitionPaths

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9121: URL: https://github.com/apache/hudi/pull/9121#issuecomment-1620397104 ## CI report: * 8555b51e9fa8f7ec9096df39d11e81d8b5177015 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620396976 ## CI report: * 2a046240c1e7c0a18f9b57c0845298ea65b72951 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9095: Test ci

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9095: URL: https://github.com/apache/hudi/pull/9095#issuecomment-1620396794 ## CI report: * 99475ffc62972ee49905fca98ea70f2096cfb135 UNKNOWN * ec568a0c309690a1b0931249aae1e4aab9eddc9b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9038: URL: https://github.com/apache/hudi/pull/9038#issuecomment-1620396492 ## CI report: * a65a29c0cf1c8feb9f39e168ba80c99ebcae1c5d UNKNOWN * 59d464a6e1f7a69ba0d0ab331ad01e3ed66f8e62 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #9038: [HUDI-6423] Incremental cleaning should consider inflight compaction instant

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9038: URL: https://github.com/apache/hudi/pull/9038#issuecomment-1620380794 ## CI report: * a65a29c0cf1c8feb9f39e168ba80c99ebcae1c5d UNKNOWN * Unknown: [CANCELED](TBD) * 59d464a6e1f7a69ba0d0ab331ad01e3ed66f8e62 Azure:

[GitHub] [hudi] codope commented on pull request #9115: [HUDI-6469] Revert HUDI-6311

2023-07-04 Thread via GitHub
codope commented on PR #9115: URL: https://github.com/apache/hudi/pull/9115#issuecomment-1620378064 > Hi @jonvex Can you elaborate a little more why to revert the changes? @danny0405 This reverts part of #8875 i.e. revert the behavior change of spark-sql insert into using bulk

[jira] [Updated] (HUDI-6476) Improve the performance of getAllPartitionPaths

2023-07-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6476: - Labels: pull-request-available (was: ) > Improve the performance of getAllPartitionPaths >

[GitHub] [hudi] wecharyu opened a new pull request, #9121: [HUDI-6476] Improve the performance of getAllPartitionPaths

2023-07-04 Thread via GitHub
wecharyu opened a new pull request, #9121: URL: https://github.com/apache/hudi/pull/9121 ### Change Logs Currently Hudi will list all status of files in hudi table directory, which can be avoid to improve the performance of getAllPartitionPaths, especially for the non-partitioned

[GitHub] [hudi] hudi-bot commented on pull request #9116: [HUDI-6470] Add spark sql conf in AlterTableCommand

2023-07-04 Thread via GitHub
hudi-bot commented on PR #9116: URL: https://github.com/apache/hudi/pull/9116#issuecomment-1620367087 ## CI report: * df41145f4bfa32fbd1f705cd6d04b74a93a0747a Azure:

  1   2   >