[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057490946 ## hudi-common/src/main/java/org/apache/hudi/common/util/CleanerUtils.java: ## @@ -161,4 +163,42 @@ public static void

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365670055 ## CI report: * 4a650f35f480a719a12e9c8367bab0e45ee1e33c Azure:

[jira] [Closed] (HUDI-5478) The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread YangXuan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangXuan closed HUDI-5478. -- Resolution: Not A Bug > The output of the show compaction command is empty when compaction is > complete. >

[GitHub] [hudi] watermelon12138 closed pull request #7563: [HUDI-5478] The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread GitBox
watermelon12138 closed pull request #7563: [HUDI-5478] The output of the show compaction command is empty when compaction is complete. URL: https://github.com/apache/hudi/pull/7563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365666836 ## CI report: * 50deef28be0792ac2b9c6a0d6979beeac4c49d4b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7563: [HUDI-5478] The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread GitBox
hudi-bot commented on PR #7563: URL: https://github.com/apache/hudi/pull/7563#issuecomment-1365663490 ## CI report: * d199a3cf4f0f6480a221209ef38653d51ac712e3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7403: [HUDI-5343] HoodieFlinkStreamer supports async clustering for append mode

2022-12-26 Thread GitBox
hudi-bot commented on PR #7403: URL: https://github.com/apache/hudi/pull/7403#issuecomment-1365663072 ## CI report: * bcbc7116f14c66b3497b290914680cf3d280caef Azure:

[GitHub] [hudi] TengHuo commented on a diff in pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-12-26 Thread GitBox
TengHuo commented on code in PR #5677: URL: https://github.com/apache/hudi/pull/5677#discussion_r1057480300 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/HoodieFlinkCompactor.java: ## @@ -218,74 +228,112 @@ private void compact() throws

[GitHub] [hudi] TengHuo commented on a diff in pull request #5677: [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once

2022-12-26 Thread GitBox
TengHuo commented on code in PR #5677: URL: https://github.com/apache/hudi/pull/5677#discussion_r1057479092 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/HoodieFlinkCompactor.java: ## @@ -218,74 +228,112 @@ private void compact() throws

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057474518 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java: ## @@ -188,8 +188,12 @@ private List

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057424247 ## hudi-common/src/main/java/org/apache/hudi/common/util/CleanerUtils.java: ## @@ -161,4 +163,42 @@ public static void

[GitHub] [hudi] SteNicholas commented on pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on PR #7405: URL: https://github.com/apache/hudi/pull/7405#issuecomment-1365650078 @danny0405, I have reviewed the above patch and tested whether the data of the replaced files is duplicated. The test result is that the data of the replaced files isn't duplicated for

[GitHub] [hudi] masthanmca123 commented on issue #7344: [SUPPORT] - Hoodi Delta Streamer - Do not allowing to perform upsert/insert in same partition when there is new column added in source data

2022-12-26 Thread GitBox
masthanmca123 commented on issue #7344: URL: https://github.com/apache/hudi/issues/7344#issuecomment-1365634969 As per my understanding, below reported bundle jar is optional in case of delta streamer. Did you tested delta streamer with schema changes ? On Tue, 13 Dec, 2022,

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365629403 ## CI report: * 50deef28be0792ac2b9c6a0d6979beeac4c49d4b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365627578 ## CI report: * 50deef28be0792ac2b9c6a0d6979beeac4c49d4b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7445: [HUDI-5380] Fixing change table path but table location in metastore …

2022-12-26 Thread GitBox
hudi-bot commented on PR #7445: URL: https://github.com/apache/hudi/pull/7445#issuecomment-1365625489 ## CI report: * 8e4c06f9406b74a190655e38bf113b0312beca89 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365625317 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 2fe0d6a4dd0fe655a6c0b7f9c7bd3889e91a84f2 Azure:

[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-12-26 Thread GitBox
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1365612404 > * FAILURE @alexeykudinkin would you help review the CI failure once more? I have rebased to the latest and it seems not the problem of this PR's specific changes ?

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365603228 ## CI report: * 50deef28be0792ac2b9c6a0d6979beeac4c49d4b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7563: [HUDI-5478] The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread GitBox
hudi-bot commented on PR #7563: URL: https://github.com/apache/hudi/pull/7563#issuecomment-1365601370 ## CI report: * d199a3cf4f0f6480a221209ef38653d51ac712e3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7403: [HUDI-5343] HoodieFlinkStreamer supports async clustering for append mode

2022-12-26 Thread GitBox
hudi-bot commented on PR #7403: URL: https://github.com/apache/hudi/pull/7403#issuecomment-1365601169 ## CI report: * 3cc324162077962217e8daab42b5655c808e9c7a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7563: [HUDI-5478] The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread GitBox
hudi-bot commented on PR #7563: URL: https://github.com/apache/hudi/pull/7563#issuecomment-1365598679 ## CI report: * d199a3cf4f0f6480a221209ef38653d51ac712e3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7403: [HUDI-5343] HoodieFlinkStreamer supports async clustering for append mode

2022-12-26 Thread GitBox
hudi-bot commented on PR #7403: URL: https://github.com/apache/hudi/pull/7403#issuecomment-1365598404 ## CI report: * 3cc324162077962217e8daab42b5655c808e9c7a Azure:

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057424247 ## hudi-common/src/main/java/org/apache/hudi/common/util/CleanerUtils.java: ## @@ -161,4 +163,42 @@ public static void

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057414214 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -429,6 +430,8 @@ private Stream getCommitInstantsToArchive()

[GitHub] [hudi] weimingdiit commented on a diff in pull request #7281: [HUDI-5270] Duplicate key error when insert_overwrite same partition …

2022-12-26 Thread GitBox
weimingdiit commented on code in PR #7281: URL: https://github.com/apache/hudi/pull/7281#discussion_r1057423556 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -243,7 +244,14 @@ private void

[GitHub] [hudi] weimingdiit commented on a diff in pull request #7281: [HUDI-5270] Duplicate key error when insert_overwrite same partition …

2022-12-26 Thread GitBox
weimingdiit commented on code in PR #7281: URL: https://github.com/apache/hudi/pull/7281#discussion_r1057423354 ## hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java: ## @@ -243,7 +244,14 @@ private void

[GitHub] [hudi] danny0405 commented on pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
danny0405 commented on PR #7405: URL: https://github.com/apache/hudi/pull/7405#issuecomment-1365587493 [5341.patch.zip](https://github.com/apache/hudi/files/10305723/5341.patch.zip) Hi, @zhuanshenbsj1 , i have tried to review and applied a patch, add a weak guarantee on the archiver pre

[jira] [Updated] (HUDI-5478) The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5478: - Labels: pull-request-available (was: ) > The output of the show compaction command is empty when

[GitHub] [hudi] watermelon12138 opened a new pull request, #7563: [HUDI-5478] The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread GitBox
watermelon12138 opened a new pull request, #7563: URL: https://github.com/apache/hudi/pull/7563 ### Change Logs 1. need to filter out complete compaction instant. ### Impact The result of show compaction command is valid. ### Risk level (write none, low medium or

[jira] [Created] (HUDI-5478) The output of the show compaction command is empty when compaction is complete.

2022-12-26 Thread YangXuan (Jira)
YangXuan created HUDI-5478: -- Summary: The output of the show compaction command is empty when compaction is complete. Key: HUDI-5478 URL: https://issues.apache.org/jira/browse/HUDI-5478 Project: Apache Hudi

[GitHub] [hudi] LinMingQiang closed pull request #6520: [HUDI-4726] Incremental input splits result is not as expected when f…

2022-12-26 Thread GitBox
LinMingQiang closed pull request #6520: [HUDI-4726] Incremental input splits result is not as expected when f… URL: https://github.com/apache/hudi/pull/6520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057414214 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -429,6 +430,8 @@ private Stream getCommitInstantsToArchive()

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057413233 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstantTimeGenerator.java: ## @@ -81,6 +81,15 @@ public static String

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057412716 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstant.java: ## @@ -82,6 +85,7 @@ public HoodieInstant(FileStatus fileStatus) { String

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365570797 ## CI report: * 0bbe20835010f2ca26bf5ca4a2900da23b9fad0f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7445: [HUDI-5380] Fixing change table path but table location in metastore …

2022-12-26 Thread GitBox
hudi-bot commented on PR #7445: URL: https://github.com/apache/hudi/pull/7445#issuecomment-1365570684 ## CI report: * 0263e5d4cd94c2c152f4d9307a5fd336a18ddeb8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365570551 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 0183d2d329f289bfd939ca3158c5d64419977c31 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365568988 ## CI report: * 0bbe20835010f2ca26bf5ca4a2900da23b9fad0f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7445: [HUDI-5380] Fixing change table path but table location in metastore …

2022-12-26 Thread GitBox
hudi-bot commented on PR #7445: URL: https://github.com/apache/hudi/pull/7445#issuecomment-1365568885 ## CI report: * 0263e5d4cd94c2c152f4d9307a5fd336a18ddeb8 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365568737 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 0183d2d329f289bfd939ca3158c5d64419977c31 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365565820 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 0183d2d329f289bfd939ca3158c5d64419977c31 Azure:

[GitHub] [hudi] BruceKellan commented on a diff in pull request #7445: [HUDI-5380] Fixing change table path but table location in metastore …

2022-12-26 Thread GitBox
BruceKellan commented on code in PR #7445: URL: https://github.com/apache/hudi/pull/7445#discussion_r1057409205 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieMetaSyncOperations.java: ## @@ -89,6 +89,15 @@ default void dropPartitions(String

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057409183 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstant.java: ## @@ -104,12 +108,14 @@ public HoodieInstant(boolean isInflight, String

[GitHub] [hudi] BruceKellan commented on a diff in pull request #7445: [HUDI-5380] Fixing change table path but table location in metastore …

2022-12-26 Thread GitBox
BruceKellan commented on code in PR #7445: URL: https://github.com/apache/hudi/pull/7445#discussion_r1057409141 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/SparkDataSourceTableUtils.java: ## @@ -98,8 +98,15 @@ public static Map

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
SteNicholas commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057409132 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstant.java: ## @@ -73,6 +75,7 @@ public enum State { private State state =

[GitHub] [hudi] zhuanshenbsj1 commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
zhuanshenbsj1 commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365554687 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #7561: [HUDI-5477][DO NOTE MERGE] Optimize timeline loading in Hudi sync client

2022-12-26 Thread GitBox
hudi-bot commented on PR #7561: URL: https://github.com/apache/hudi/pull/7561#issuecomment-1365538403 ## CI report: * df28b5141ea2b920a55149668c12ebda1416194a Azure:

[GitHub] [hudi] soumilshah1995 opened a new issue, #7562: [SUPPORT] How to Fire Async Compaction on Pyspark

2022-12-26 Thread GitBox
soumilshah1995 opened a new issue, #7562: URL: https://github.com/apache/hudi/issues/7562 Hello There # Sample Script ``` """ Install https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop2.tgz hadoop2.7

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365493622 ## CI report: * 0bbe20835010f2ca26bf5ca4a2900da23b9fad0f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7561: [HUDI-5477][DO NOTE MERGE] Optimize timeline loading in Hudi sync client

2022-12-26 Thread GitBox
hudi-bot commented on PR #7561: URL: https://github.com/apache/hudi/pull/7561#issuecomment-1365492500 ## CI report: * df28b5141ea2b920a55149668c12ebda1416194a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7561: [HUDI-5477][DO NOTE MERGE] Optimize timeline loading in Hudi sync client

2022-12-26 Thread GitBox
hudi-bot commented on PR #7561: URL: https://github.com/apache/hudi/pull/7561#issuecomment-1365491089 ## CI report: * df28b5141ea2b920a55149668c12ebda1416194a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7559: [HUDI-5447] Adding read support for record level index in metadata table

2022-12-26 Thread GitBox
hudi-bot commented on PR #7559: URL: https://github.com/apache/hudi/pull/7559#issuecomment-1365489452 ## CI report: * 011baa31e9fa6cf6e27f39c3b42f5eb71c7a9e8b Azure:

[jira] [Updated] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5477: - Labels: pull-request-available (was: ) > Optimize timeline loading in Hudi sync client >

[GitHub] [hudi] yihua opened a new pull request, #7561: [HUDI-5477][DO NOTE MERGE] Optimize timeline loading in Hudi sync client

2022-12-26 Thread GitBox
yihua opened a new pull request, #7561: URL: https://github.com/apache/hudi/pull/7561 ### Change Logs Before this change, the Hudi archived timeline is always loaded during the metastore sync process if the last sync time is given. Besides, the archived timeline is not cached inside

[jira] [Updated] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5477: Component/s: meta-sync > Optimize timeline loading in Hudi sync client >

[jira] [Updated] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5477: Component/s: archiving > Optimize timeline loading in Hudi sync client >

[jira] [Updated] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5477: Description: The Hudi archived timeline is always loaded during the metastore sync process if the last sync

[jira] [Updated] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5477: Priority: Blocker (was: Major) > Optimize timeline loading in Hudi sync client >

[jira] [Updated] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5477: Fix Version/s: 0.13.0 > Optimize timeline loading in Hudi sync client >

[jira] [Created] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-5477: --- Summary: Optimize timeline loading in Hudi sync client Key: HUDI-5477 URL: https://issues.apache.org/jira/browse/HUDI-5477 Project: Apache Hudi Issue Type:

[jira] [Assigned] (HUDI-5477) Optimize timeline loading in Hudi sync client

2022-12-26 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-5477: --- Assignee: Ethan Guo > Optimize timeline loading in Hudi sync client >

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365454657 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 0183d2d329f289bfd939ca3158c5d64419977c31 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365404332 ## CI report: * 0bbe20835010f2ca26bf5ca4a2900da23b9fad0f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7559: [HUDI-5447] Adding read support for record level index in metadata table

2022-12-26 Thread GitBox
hudi-bot commented on PR #7559: URL: https://github.com/apache/hudi/pull/7559#issuecomment-1365404329 ## CI report: * 011baa31e9fa6cf6e27f39c3b42f5eb71c7a9e8b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
hudi-bot commented on PR #7560: URL: https://github.com/apache/hudi/pull/7560#issuecomment-1365402912 ## CI report: * 0bbe20835010f2ca26bf5ca4a2900da23b9fad0f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7559: [HUDI-5447] Adding read support for record level index in metadata table

2022-12-26 Thread GitBox
hudi-bot commented on PR #7559: URL: https://github.com/apache/hudi/pull/7559#issuecomment-1365402893 ## CI report: * 011baa31e9fa6cf6e27f39c3b42f5eb71c7a9e8b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
hudi-bot commented on PR #7405: URL: https://github.com/apache/hudi/pull/7405#issuecomment-1365401249 ## CI report: * 677991282ce2a2e034dfdce41a8aecfb456e6c0c Azure:

svn commit: r58994 - in /release/hudi/0.12.2: ./ hudi-0.12.2.src.tgz hudi-0.12.2.src.tgz.asc hudi-0.12.2.src.tgz.sha512

2022-12-26 Thread sivabalan
Author: sivabalan Date: Mon Dec 26 19:23:05 2022 New Revision: 58994 Log: Adding 0.12.2 binaries Added: release/hudi/0.12.2/ release/hudi/0.12.2/hudi-0.12.2.src.tgz (with props) release/hudi/0.12.2/hudi-0.12.2.src.tgz.asc release/hudi/0.12.2/hudi-0.12.2.src.tgz.sha512 Added:

[jira] [Updated] (HUDI-5476) Flaky test in hudi-utilities

2022-12-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5476: - Labels: pull-request-available (was: ) > Flaky test in hudi-utilities >

[jira] [Updated] (HUDI-5447) Add support for Record level index read from MDT

2022-12-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5447: - Labels: pull-request-available (was: ) > Add support for Record level index read from MDT >

[GitHub] [hudi] xushiyan opened a new pull request, #7560: [HUDI-5476] Fix utilities test failure with zookeeper

2022-12-26 Thread GitBox
xushiyan opened a new pull request, #7560: URL: https://github.com/apache/hudi/pull/7560 ### Change Logs Fix test failures with zookeeper ``` 2022-12-26T15:57:30.3678486Z 688759 [Listener at localhost/44497] WARN org.apache.spark.sql.SparkSession$Builder [] - Using an

[GitHub] [hudi] nsivabalan opened a new pull request, #7559: [HUDI-5447] Adding read support for record level index in metadata table

2022-12-26 Thread GitBox
nsivabalan opened a new pull request, #7559: URL: https://github.com/apache/hudi/pull/7559 ### Change Logs Adding record level index support in hudi write path. ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk

[GitHub] [hudi] hudi-bot commented on pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
hudi-bot commented on PR #7405: URL: https://github.com/apache/hudi/pull/7405#issuecomment-1365341656 ## CI report: * b7a1071aa728627a1457eafb26f323b3e1b44e5b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365341054 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 45cf7f3c242e20f49b95242a06efe1e24649edc7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
hudi-bot commented on PR #7405: URL: https://github.com/apache/hudi/pull/7405#issuecomment-1365330813 ## CI report: * b7a1071aa728627a1457eafb26f323b3e1b44e5b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7159: [HUDI-5173]Skip if there is only one file in clusteringGroup

2022-12-26 Thread GitBox
hudi-bot commented on PR #7159: URL: https://github.com/apache/hudi/pull/7159#issuecomment-1365330019 ## CI report: * 15ecd91180d32c7fa1905c11408f4bc23347e682 UNKNOWN * 45cf7f3c242e20f49b95242a06efe1e24649edc7 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7553: [HUDI-5408] Rollback partially failed commits in MDT in all cases

2022-12-26 Thread GitBox
hudi-bot commented on PR #7553: URL: https://github.com/apache/hudi/pull/7553#issuecomment-1365318879 ## CI report: * 7be0a3637dfdb66268f3a84d60cb062f03bae960 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7543: [HUDI-5401] Ensure user-provided hive metastore uri is set in HiveConf if not already set

2022-12-26 Thread GitBox
hudi-bot commented on PR #7543: URL: https://github.com/apache/hudi/pull/7543#issuecomment-1365318709 ## CI report: * 7111dcc7f5d772f79a5927da583c20fe447c2d71 Azure:

[GitHub] [hudi] zhuanshenbsj1 commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
zhuanshenbsj1 commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057306015 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java: ## @@ -455,6 +458,10 @@ private Stream

[GitHub] [hudi] zhuanshenbsj1 commented on a diff in pull request #7405: [HUDI-5341] IncrementalCleaning consider clustering completed later

2022-12-26 Thread GitBox
zhuanshenbsj1 commented on code in PR #7405: URL: https://github.com/apache/hudi/pull/7405#discussion_r1057302648 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java: ## @@ -188,8 +188,12 @@ private List

[GitHub] [hudi] hudi-bot commented on pull request #7403: [HUDI-5343] HoodieFlinkStreamer supports async clustering for append mode

2022-12-26 Thread GitBox
hudi-bot commented on PR #7403: URL: https://github.com/apache/hudi/pull/7403#issuecomment-1365279329 ## CI report: * 3cc324162077962217e8daab42b5655c808e9c7a Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7553: [HUDI-5408] Rollback partially failed commits in MDT in all cases

2022-12-26 Thread GitBox
hudi-bot commented on PR #7553: URL: https://github.com/apache/hudi/pull/7553#issuecomment-1365250732 ## CI report: * 8f5b05486bad7c6e9f70d560d6b5560094973f50 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7543: [HUDI-5401] Ensure user-provided hive metastore uri is set in HiveConf if not already set

2022-12-26 Thread GitBox
hudi-bot commented on PR #7543: URL: https://github.com/apache/hudi/pull/7543#issuecomment-1365250703 ## CI report: * 5882d457327b1a4eef10dc1932a6f8a71be0b344 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7553: [HUDI-5408] Rollback partially failed commits in MDT in all cases

2022-12-26 Thread GitBox
hudi-bot commented on PR #7553: URL: https://github.com/apache/hudi/pull/7553#issuecomment-1365248440 ## CI report: * 8f5b05486bad7c6e9f70d560d6b5560094973f50 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7543: [HUDI-5401] Ensure user-provided hive metastore uri is set in HiveConf if not already set

2022-12-26 Thread GitBox
hudi-bot commented on PR #7543: URL: https://github.com/apache/hudi/pull/7543#issuecomment-1365248390 ## CI report: * 5882d457327b1a4eef10dc1932a6f8a71be0b344 Azure:

[GitHub] [hudi] complone commented on a diff in pull request #7306: [HUDI-5280] Fix taskmanager concurrent registry

2022-12-26 Thread GitBox
complone commented on code in PR #7306: URL: https://github.com/apache/hudi/pull/7306#discussion_r1057250527 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/metrics/HoodieLockMetrics.java: ## @@ -70,11 +71,13 @@ private String

[GitHub] [hudi] hudi-bot commented on pull request #7403: [HUDI-5343] HoodieFlinkStreamer supports async clustering for append mode

2022-12-26 Thread GitBox
hudi-bot commented on PR #7403: URL: https://github.com/apache/hudi/pull/7403#issuecomment-136520 ## CI report: * 3cc324162077962217e8daab42b5655c808e9c7a Azure:

[GitHub] [hudi] leesf commented on pull request #7403: [HUDI-5343] HoodieFlinkStreamer supports async clustering for append mode

2022-12-26 Thread GitBox
leesf commented on PR #7403: URL: https://github.com/apache/hudi/pull/7403#issuecomment-1365150030 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] hudi-bot commented on pull request #7558: [MINOR] Update DOAP with 0.12.2 release

2022-12-26 Thread GitBox
hudi-bot commented on PR #7558: URL: https://github.com/apache/hudi/pull/7558#issuecomment-1365135377 ## CI report: * 97944fbce804bda91aefcf982587b733c8c0f901 Azure:

[GitHub] [hudi] kepplertreet commented on issue #7453: [SUPPORT] Hudi Upsert fails for

2022-12-26 Thread GitBox
kepplertreet commented on issue #7453: URL: https://github.com/apache/hudi/issues/7453#issuecomment-1365068469 Hi @yihua, we have managed to solve this. We inferred it to be a problem that corresponds to our .hoodie/metadata being corrupted on being tampered with multiple different

[GitHub] [hudi] AdarshKadameriTR commented on issue #7487: [SUPPORT] S3 Buckets reached quota limit when reading from hudi tables

2022-12-26 Thread GitBox
AdarshKadameriTR commented on issue #7487: URL: https://github.com/apache/hudi/issues/7487#issuecomment-1365052246 Hi @yihua we discussed with AWS support person and find out 503 errors for ‘REST.GET.BUCKET’ which refer to LIST requests made to list the bucket. and was seeing a very

[GitHub] [hudi] hudi-bot commented on pull request #7558: [MINOR] Update DOAP with 0.12.2 release

2022-12-26 Thread GitBox
hudi-bot commented on PR #7558: URL: https://github.com/apache/hudi/pull/7558#issuecomment-1365042542 ## CI report: * 97944fbce804bda91aefcf982587b733c8c0f901 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7558: [MINOR] Update DOAP with 0.12.2 release

2022-12-26 Thread GitBox
hudi-bot commented on PR #7558: URL: https://github.com/apache/hudi/pull/7558#issuecomment-1365038974 ## CI report: * 97944fbce804bda91aefcf982587b733c8c0f901 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #4966: [HUDI-3572]support DAY_ROLLING strategy in ClusteringPlanPartitionFilterMode

2022-12-26 Thread GitBox
hudi-bot commented on PR #4966: URL: https://github.com/apache/hudi/pull/4966#issuecomment-1365037563 ## CI report: * 2f04d4f01d2898541e9f089f33385de0c78ee66b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-12-26 Thread GitBox
hudi-bot commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1365034273 ## CI report: * fbb9ed33252880509289e8385fa8615e7f46b7fe Azure:

[jira] [Updated] (HUDI-5353) Close file reader wherever missing

2022-12-26 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5353: - Fix Version/s: 0.13.0 (was: 0.12.2) > Close file reader wherever missing >

[jira] [Closed] (HUDI-5353) Close file reader wherever missing

2022-12-26 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish closed HUDI-5353. Resolution: Fixed > Close file reader wherever missing > -- > >

[jira] [Resolved] (HUDI-5353) Close file reader wherever missing

2022-12-26 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish resolved HUDI-5353. -- > Close file reader wherever missing > -- > > Key: HUDI-5353 >

[jira] [Updated] (HUDI-5091) MergeInto syntax merge_condition does not support Non-Equal

2022-12-26 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5091: - Fix Version/s: (was: 0.12.2) > MergeInto syntax merge_condition does not support Non-Equal >

  1   2   >