[GitHub] [hudi] linfey90 commented on a diff in pull request #8865: [HUDI-6306] dynamic catalog parameter

linfey90 commented on code in PR #8865: URL: https://github.com/apache/hudi/pull/8865#discussion_r1213956512 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/HadoopConfigurations.java: ## @@ -63,6 +63,7 @@ public static

[jira] [Closed] (HUDI-6308) add num_commits_after_last_request to flink

[ https://issues.apache.org/jira/browse/HUDI-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric closed HUDI-6308. -- Resolution: Not A Problem > add num_commits_after_last_request to flink > --- >

[GitHub] [hudi] hudi-bot commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

hudi-bot commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1573191026 ## CI report: * 7e24575b30fc34d8174a74a98431c6e1f42bef7c Azure:

[GitHub] [hudi] eric9204 commented on a diff in pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

eric9204 commented on code in PR #8871: URL: https://github.com/apache/hudi/pull/8871#discussion_r1213950388 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -642,6 +642,7 @@ private FlinkOptions() { public static final

[GitHub] [hudi] eric9204 closed pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

eric9204 closed pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink URL: https://github.com/apache/hudi/pull/8871 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] danny0405 commented on a diff in pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

danny0405 commented on code in PR #8871: URL: https://github.com/apache/hudi/pull/8871#discussion_r1213944708 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -642,6 +642,7 @@ private FlinkOptions() { public static

[GitHub] [hudi] eric9204 commented on a diff in pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

eric9204 commented on code in PR #8871: URL: https://github.com/apache/hudi/pull/8871#discussion_r1213939610 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -642,6 +642,7 @@ private FlinkOptions() { public static final

[GitHub] [hudi] SteNicholas commented on a diff in pull request #8759: Add metrics counters for compaction requested/completed events.

SteNicholas commented on code in PR #8759: URL: https://github.com/apache/hudi/pull/8759#discussion_r1213933746 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/compact/TestHoodieCompactor.java: ## @@ -129,6 +152,10 @@ public void

[GitHub] [hudi] SteNicholas commented on pull request #8759: Add metrics counters for compaction requested/completed events.

SteNicholas commented on PR #8759: URL: https://github.com/apache/hudi/pull/8759#issuecomment-1573162218 @amrishlal, could you create a JIRA ticket or issue for this metric introduction? Meanwhile, you should update the title of this pull request, otherwise this pull request fails to

[jira] [Updated] (HUDI-6241) HIVE_SYNC_TABLE_STRATEGY in HiveSyncConfigHolder Documentation fix

[ https://issues.apache.org/jira/browse/HUDI-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xy updated HUDI-6241: - Fix Version/s: 0.14.0 > HIVE_SYNC_TABLE_STRATEGY in HiveSyncConfigHolder Documentation fix >

[GitHub] [hudi] stream2000 commented on a diff in pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

stream2000 commented on code in PR #8745: URL: https://github.com/apache/hudi/pull/8745#discussion_r1213873676 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java: ## @@ -298,6 +298,22 @@ protected void syncHoodieTable(String tableName, boolean

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213863164 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java: ## @@ -351,6 +347,7 @@ public void run() {

[GitHub] [hudi] danny0405 commented on a diff in pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

danny0405 commented on code in PR #8745: URL: https://github.com/apache/hudi/pull/8745#discussion_r1213871041 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java: ## @@ -298,6 +298,22 @@ protected void syncHoodieTable(String tableName, boolean

[GitHub] [hudi] hudi-bot commented on pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

hudi-bot commented on PR #8871: URL: https://github.com/apache/hudi/pull/8871#issuecomment-1573073017 ## CI report: * 88a2f49c4c02ce1ade13549ac56d6bf396411289 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

hudi-bot commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1573072987 ## CI report: * 6a8fa73c9e31a90f6249772b5b840acf42ae1df5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

hudi-bot commented on PR #8871: URL: https://github.com/apache/hudi/pull/8871#issuecomment-1573067586 ## CI report: * 88a2f49c4c02ce1ade13549ac56d6bf396411289 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

hudi-bot commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1573067541 ## CI report: * 6a8fa73c9e31a90f6249772b5b840acf42ae1df5 Azure:

[GitHub] [hudi] danny0405 commented on a diff in pull request #8871: [HUDI-6308]add num_commits_after_last_request to flink

danny0405 commented on code in PR #8871: URL: https://github.com/apache/hudi/pull/8871#discussion_r1213865107 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -642,6 +642,7 @@ private FlinkOptions() { public static

[GitHub] [hudi] danny0405 commented on issue #8855: [SUPPORT][FLINK SQL] Can not insert join result into hudi table

danny0405 commented on issue #8855: URL: https://github.com/apache/hudi/issues/8855#issuecomment-1573064043 Intreasting, can you share us the flink checkpoint configuration params? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Commented] (HUDI-6293) Make HoodieFlinkCompactor's parallelism of compact_task more reasonable.

[ https://issues.apache.org/jira/browse/HUDI-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728544#comment-17728544 ] Danny Chen commented on HUDI-6293: -- Another fix for clustering: b36e7c459904860b0be086c144ba0b175961e805

[GitHub] [hudi] danny0405 merged pull request #8866: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task…

danny0405 merged PR #8866: URL: https://github.com/apache/hudi/pull/8866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task more reasonable (#8866)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b36e7c45990 [HUDI-6293] Make

[jira] [Updated] (HUDI-6258) support olap engine query mor table in table name without ro/rt suffix

[ https://issues.apache.org/jira/browse/HUDI-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xy updated HUDI-6258: - Description: when we query mor table with olap engine such as starrocks、doris、presto，we can get data only from rt/ro

[jira] [Updated] (HUDI-6258) support olap engine query mor table in table name without ro/rt suffix

[ https://issues.apache.org/jira/browse/HUDI-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xy updated HUDI-6258: - Description: when we query mor table with olap engine such as starrocks、doris、presto，we can get data only from rt/ro

[GitHub] [hudi] danny0405 commented on a diff in pull request #8830: [MINOR] auto generate init client id

danny0405 commented on code in PR #8830: URL: https://github.com/apache/hudi/pull/8830#discussion_r1213860936 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/configuration/TestOptionsInference.java: ## @@ -69,6 +70,12 @@ void testSetupClientId() throws

[GitHub] [hudi] danny0405 commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

danny0405 commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1573057917 Hi @satishkotha can you help double check this change? I need some help for the background why in https://github.com/apache/hudi/pull/2129 only timestamp(6) is synced as timestamp in

[GitHub] [hudi] garyli1019 commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

garyli1019 commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1213858505 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review +

[GitHub] [hudi] danny0405 commented on a diff in pull request #8865: [HUDI-6306] dynamic catalog parameter

danny0405 commented on code in PR #8865: URL: https://github.com/apache/hudi/pull/8865#discussion_r1213858024 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/HadoopConfigurations.java: ## @@ -63,6 +63,7 @@ public static

[GitHub] [hudi] c-f-cooper commented on a diff in pull request #8830: [MINOR] auto generate init client id

c-f-cooper commented on code in PR #8830: URL: https://github.com/apache/hudi/pull/8830#discussion_r1213854825 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/configuration/TestOptionsInference.java: ## @@ -69,6 +70,12 @@ void testSetupClientId() throws

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213814148 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -111,18 +111,27 @@ public abstract class

[GitHub] [hudi] danny0405 commented on a diff in pull request #8863: [HUDI-6305] s3a parameters cannot be filtered

danny0405 commented on code in PR #8863: URL: https://github.com/apache/hudi/pull/8863#discussion_r1213851720 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/HadoopConfigurations.java: ## @@ -49,8 +48,7 @@ public static

[jira] [Updated] (HUDI-6308) add num_commits_after_last_request to flink

[ https://issues.apache.org/jira/browse/HUDI-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6308: - Labels: pull-request-available (was: ) > add num_commits_after_last_request to flink >

[GitHub] [hudi] eric9204 opened a new pull request, #8871: [HUDI-6308]add num_commits_after_last_request to flink

eric9204 opened a new pull request, #8871: URL: https://github.com/apache/hudi/pull/8871 ### Change Logs None ### Impact None ### Risk level (write none, low medium or high below) None ### Documentation Update None ### Contributor's

[jira] [Comment Edited] (HUDI-6258) support olap engine query mor table in table name without ro/rt suffix

[ https://issues.apache.org/jira/browse/HUDI-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728532#comment-17728532 ] xy edited comment on HUDI-6258 at 6/2/23 2:19 AM: -- Fixed via master

[jira] [Resolved] (HUDI-6258) support olap engine query mor table in table name without ro/rt suffix

[ https://issues.apache.org/jira/browse/HUDI-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xy resolved HUDI-6258. -- > support olap engine query mor table in table name without ro/rt suffix >

[jira] [Commented] (HUDI-6258) support olap engine query mor table in table name without ro/rt suffix

[ https://issues.apache.org/jira/browse/HUDI-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728532#comment-17728532 ] xy commented on HUDI-6258: -- master commit：3f9c45fdfa9b03e8092db07188b76c5931475733 > support olap engine query

[GitHub] [hudi] zhangyue19921010 commented on pull request #6868: [Hudi-4882] Multiple ordering fields and null value update for partial update to handle out-of-order events

zhangyue19921010 commented on PR #6868: URL: https://github.com/apache/hudi/pull/6868#issuecomment-1573024411 @fengjian428 would u mind to rebase master ? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] stream2000 commented on a diff in pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

stream2000 commented on code in PR #8745: URL: https://github.com/apache/hudi/pull/8745#discussion_r1213840107 ## hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java: ## @@ -298,6 +298,22 @@ protected void syncHoodieTable(String tableName, boolean

[jira] [Created] (HUDI-6308) add num_commits_after_last_request to flink

eric created HUDI-6308: -- Summary: add num_commits_after_last_request to flink Key: HUDI-6308 URL: https://issues.apache.org/jira/browse/HUDI-6308 Project: Apache Hudi Issue Type: Improvement

[GitHub] [hudi] Riddle4045 commented on issue #8870: [SUPPORT] Trino returns 0 rows when reading Hudi tables written by Flink 1.16

Riddle4045 commented on issue #8870: URL: https://github.com/apache/hudi/issues/8870#issuecomment-1573003923 > The compaction is executed async by default every 5 delta_commit on the table, did you have any chance to see the Parquet files already? @danny0405 no, there were total 6

[GitHub] [hudi] danny0405 commented on a diff in pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

danny0405 commented on code in PR #8867: URL: https://github.com/apache/hudi/pull/8867#discussion_r1213826815 ## hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java: ## @@ -641,6 +643,26 @@ public void testSyncWithSchema(String syncMode, String

[GitHub] [hudi] danny0405 commented on issue #8870: [SUPPORT] Trino returns 0 rows when reading Hudi tables written by Flink 1.16

danny0405 commented on issue #8870: URL: https://github.com/apache/hudi/issues/8870#issuecomment-1572999177 The compaction is executed async by default every 5 delta_commit on the table, did you have any chance to see the Parquet files already? -- This is an automated message from the

[jira] [Closed] (HUDI-6256) fix the data table archiving and MDT cleaning config conflict

[ https://issues.apache.org/jira/browse/HUDI-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6256. Fix Version/s: 0.14.0 Resolution: Fixed Fixed via master branch:

[GitHub] [hudi] danny0405 merged pull request #8792: [HUDI-6256] Fix the data table archiving and MDT cleaning config conf…

danny0405 merged PR #8792: URL: https://github.com/apache/hudi/pull/8792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[hudi] branch master updated: [HUDI-6256] Fix the data table archiving and MDT cleaning config conf… (#8792)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 32adbe4dfb2 [HUDI-6256] Fix the data table

[GitHub] [hudi] XuQianJin-Stars commented on pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

XuQianJin-Stars commented on PR #8795: URL: https://github.com/apache/hudi/pull/8795#issuecomment-1572974659 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[hudi] branch master updated (e8ca0d4121a -> 3f9c45fdfa9)

This is an automated email from the ASF dual-hosted git repository. forwardxu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from e8ca0d4121a [HUDI-6303] Bump flink version to 1.16.2 and 1.17.1 (#8861) add 3f9c45fdfa9 [HUDI-6258] support

[GitHub] [hudi] XuQianJin-Stars merged pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

XuQianJin-Stars merged PR #8795: URL: https://github.com/apache/hudi/pull/8795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] hudi-bot commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

hudi-bot commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1572836228 ## CI report: * 2d4e285ba5ef3c5b07ec91af6ab3a2669d2b485d Azure:

[GitHub] [hudi] Riddle4045 commented on issue #8870: [SUPPORT] Trino returns 0 rows when reading Hudi tables written by Flink 1.16

Riddle4045 commented on issue #8870: URL: https://github.com/apache/hudi/issues/8870#issuecomment-1572789850 possibly related to https://github.com/apache/hudi/issues/8038 @codope could you help me understand how to configure the table for read optimized queries? or is it something that

[GitHub] [hudi] hudi-bot commented on pull request #8869: Added logic to correctly verify partition keys for CustomAvroKeyGen

hudi-bot commented on PR #8869: URL: https://github.com/apache/hudi/pull/8869#issuecomment-1572785544 ## CI report: * 54977785e91e2ee46baddd399a0d1889a323c612 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

hudi-bot commented on PR #8795: URL: https://github.com/apache/hudi/pull/8795#issuecomment-1572772807 ## CI report: * 130523be1324218f56ce15ddc6ac3255e7cfcd9a Azure:

[GitHub] [hudi] Riddle4045 opened a new issue, #8870: [SUPPORT] Trino returns 0 rows when reading Hudi tables written by Flink 1.16

Riddle4045 opened a new issue, #8870: URL: https://github.com/apache/hudi/issues/8870 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213649019 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataWriter.java: ## @@ -41,35 +44,23 @@ public interface

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1572681066 ## CI report: * f71ca7ad4339c60719c97f3d54339b6a7bd5205f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8868: [HUDI-6278] Fixed the use of DynamoDBLockConfig class

hudi-bot commented on PR #8868: URL: https://github.com/apache/hudi/pull/8868#issuecomment-1572673768 ## CI report: * b6f86c770f7e35d7488cff0066d2d760453eb931 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

hudi-bot commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1572673644 ## CI report: * 2db6852dd391973eab275dc7ef70c02bfbc5f652 UNKNOWN * 60c1399ac012bc61421f3bb1feb208decbcb6b6a UNKNOWN * 0328e76358dd170d62b94fd286a9ffb728516429 Azure:

[GitHub] [hudi] parisni commented on pull request #8740: [HUDI-6231] Handle glue comments

parisni commented on PR #8740: URL: https://github.com/apache/hudi/pull/8740#issuecomment-1572624810 > @parisni Hi, do we have plan to push-forward this feature? Yes I do. I am currently in vacations -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Assigned] (HUDI-6253) Treat full bootstrap table as regular table

[ https://issues.apache.org/jira/browse/HUDI-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-6253: - Assignee: (was: Jonathan Vexler) > Treat full bootstrap table as regular table >

[GitHub] [hudi] codope commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

codope commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213269335 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataWriter.java: ## @@ -99,7 +90,25 @@ public interface HoodieTableMetadataWriter

[GitHub] [hudi] hudi-bot commented on pull request #8526: [HUDI-6116] Optimize log block reading by removing seeks to check corrupted blocks.

hudi-bot commented on PR #8526: URL: https://github.com/apache/hudi/pull/8526#issuecomment-1572527934 ## CI report: * 0f2f4ddd192879cdc6a9c91aa2b2c5c6813ab490 Azure:

[GitHub] [hudi] prashantwason commented on a diff in pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

prashantwason commented on code in PR #8837: URL: https://github.com/apache/hudi/pull/8837#discussion_r1213476749 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -851,26 +919,49 @@ public void

[GitHub] [hudi] prashantwason commented on a diff in pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

prashantwason commented on code in PR #8837: URL: https://github.com/apache/hudi/pull/8837#discussion_r1213473799 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -837,10 +840,75 @@ public void

[GitHub] [hudi] prashantwason commented on a diff in pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

prashantwason commented on code in PR #8837: URL: https://github.com/apache/hudi/pull/8837#discussion_r1213473799 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -837,10 +840,75 @@ public void

[GitHub] [hudi] prashantwason commented on a diff in pull request #8837: [HUDI-6153] Changed the rollback mechanism for MDT to actual rollbacks rather than appending revert blocks.

prashantwason commented on code in PR #8837: URL: https://github.com/apache/hudi/pull/8837#discussion_r1213470892 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -669,32 +669,51 @@ public void restoreToSavepoint() { *

[GitHub] [hudi] prashantwason commented on a diff in pull request #8604: [HUDI-6151] Rollback previously applied commits to MDT when operations are retried.

prashantwason commented on code in PR #8604: URL: https://github.com/apache/hudi/pull/8604#discussion_r1213467835 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieBackedTableMetadataWriter.java: ## @@ -159,6 +162,13 @@ protected void

[GitHub] [hudi] prashantwason commented on pull request #8526: [HUDI-6116] Optimize log block reading by removing seeks to check corrupted blocks.

prashantwason commented on PR #8526: URL: https://github.com/apache/hudi/pull/8526#issuecomment-1572480457 @danny0405 PTAL again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] prashantwason commented on a diff in pull request #8526: [HUDI-6116] Optimize log block reading by removing seeks to check corrupted blocks.

prashantwason commented on code in PR #8526: URL: https://github.com/apache/hudi/pull/8526#discussion_r1213452295 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java: ## @@ -152,98 +153,107 @@ private void addShutDownHook() { // TODO :

[GitHub] [hudi] prashantwason commented on a diff in pull request #8526: [HUDI-6116] Optimize log block reading by removing seeks to check corrupted blocks.

prashantwason commented on code in PR #8526: URL: https://github.com/apache/hudi/pull/8526#discussion_r1213451923 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java: ## @@ -152,98 +153,107 @@ private void addShutDownHook() { // TODO :

[GitHub] [hudi] prashantwason commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

prashantwason commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1572447665 @nsivabalan I fixed the conflict and all tests are passing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] hudi-bot commented on pull request #8792: [HUDI-6256] Fix the data table archiving and MDT cleaning config conf…

hudi-bot commented on PR #8792: URL: https://github.com/apache/hudi/pull/8792#issuecomment-1572436343 ## CI report: * 683dc368e714ace1c44d741d642f1fe64b7910b2 Azure:

[jira] [Assigned] (HUDI-6253) Treat full bootstrap table as regular table

[ https://issues.apache.org/jira/browse/HUDI-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-6253: - Assignee: Jonathan Vexler > Treat full bootstrap table as regular table >

[jira] [Closed] (HUDI-5987) Clustering on bootstrap table fails when row writer is disabled

[ https://issues.apache.org/jira/browse/HUDI-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5987. - Resolution: Fixed > Clustering on bootstrap table fails when row writer is disabled >

[GitHub] [hudi] jonvex commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

jonvex commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1213427044 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review + +##

[GitHub] [hudi] hudi-bot commented on pull request #8867: [HUDI-6307] Sync TIMESTAMP_MILLIS to hive

hudi-bot commented on PR #8867: URL: https://github.com/apache/hudi/pull/8867#issuecomment-1572398800 ## CI report: * 6a8fa73c9e31a90f6249772b5b840acf42ae1df5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

hudi-bot commented on PR #8795: URL: https://github.com/apache/hudi/pull/8795#issuecomment-1572398057 ## CI report: * 130523be1324218f56ce15ddc6ac3255e7cfcd9a Azure:

[GitHub] [hudi] bvaradar commented on pull request #8847: [HUDI-2071] Support Reading Bootstrap MOR RT Table In Spark DataSource Table

bvaradar commented on PR #8847: URL: https://github.com/apache/hudi/pull/8847#issuecomment-1572371115 Sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hudi] jonvex commented on pull request #8847: [HUDI-2071] Support Reading Bootstrap MOR RT Table In Spark DataSource Table

jonvex commented on PR #8847: URL: https://github.com/apache/hudi/pull/8847#issuecomment-1572364211 @bvaradar do you think you would be able to review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] gamblewin commented on issue #8855: [SUPPORT][FLINK SQL] Can not insert join result into hudi table

gamblewin commented on issue #8855: URL: https://github.com/apache/hudi/issues/8855#issuecomment-1572347014 https://github.com/apache/hudi/assets/39117591/2def2c5a-39bc-4bfc-9d8b-0575d3fc3119;> it seems like it doesn't trigger a checkpoint. -- This is an automated message from the

[GitHub] [hudi] hudi-bot commented on pull request #8795: [HUDI-6258] support olap engine query mor table in table name without ro/rt suffix

hudi-bot commented on PR #8795: URL: https://github.com/apache/hudi/pull/8795#issuecomment-1572324712 ## CI report: * 130523be1324218f56ce15ddc6ac3255e7cfcd9a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8869: Added logic to correctly verify partition keys for CustomAvroKeyGen

hudi-bot commented on PR #8869: URL: https://github.com/apache/hudi/pull/8869#issuecomment-1572325298 ## CI report: * 54977785e91e2ee46baddd399a0d1889a323c612 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8869: Added logic to correctly verify partition keys for CustomAvroKeyGen

hudi-bot commented on PR #8869: URL: https://github.com/apache/hudi/pull/8869#issuecomment-1572312116 ## CI report: * 54977785e91e2ee46baddd399a0d1889a323c612 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8452: [HUDI-6077] Add more partition push down filters

hudi-bot commented on PR #8452: URL: https://github.com/apache/hudi/pull/8452#issuecomment-1572310179 ## CI report: * 8082df232089396b2a9f9be2b915e51b3645f172 UNKNOWN * 9e5504e078b93d1997cf901868234e36c69dd97e Azure:

[GitHub] [hudi] cbts-alec-johnson commented on issue #8857: [SUPPORT] Column comments not syncing to AWS Glue Catalog

cbts-alec-johnson commented on issue #8857: URL: https://github.com/apache/hudi/issues/8857#issuecomment-1572291524 > Guess this is what you needed: https://github.com/apache/hudi/pull/8740/files Yes this is what I need. Also, I think you may have tagged this gcp-support instead of

[GitHub] [hudi] ad1happy2go opened a new pull request, #8869: Added logic to correctly verify partition keys for CustomAvroKeyGen

ad1happy2go opened a new pull request, #8869: URL: https://github.com/apache/hudi/pull/8869 ### Change Logs Added logic to correctly verify partition keys for CustomAvroKeyGenerator It will fix for Github issue - https://github.com/apache/hudi/issues/8372 ### Impact

[GitHub] [hudi] bkosuru commented on issue #8800: GCP: Hudi stopped working in Dataproc Serverless 1.1.4

bkosuru commented on issue #8800: URL: https://github.com/apache/hudi/issues/8800#issuecomment-1572227348 It started working with hudi 0.13.1, did you fix anything in 0.13.1 to make it work? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] machadoluiz commented on issue #8824: [SUPPORT] Performance and Data Integrity Issues with Hudi for Long-Term Data Retention

machadoluiz commented on issue #8824: URL: https://github.com/apache/hudi/issues/8824#issuecomment-1572195137 @ad1happy2go, the runtime increment happens gradually. In a specific example, it reached 2 minutes and 30 seconds around 300 commits (or 10 months). This poses a challenge for us,

[GitHub] [hudi] hudi-bot commented on pull request #8866: [HUDI-6293] Make HoodieClusteringJob's parallelism of clustering_task…

hudi-bot commented on PR #8866: URL: https://github.com/apache/hudi/pull/8866#issuecomment-1572191506 ## CI report: * badb098e6bd6b0ee8b317514f08eb460659a8d93 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

hudi-bot commented on PR #8745: URL: https://github.com/apache/hudi/pull/8745#issuecomment-1572190692 ## CI report: * 62377696531fc1d4ee2b7c0c86897d1cfb6b5de9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8865: [HUDI-6306] dynamic catalog parameter

hudi-bot commented on PR #8865: URL: https://github.com/apache/hudi/pull/8865#issuecomment-1572082961 ## CI report: * 821e287f35e93974ae28f1e1e7a513c68749c281 Azure:

[GitHub] [hudi] stream2000 commented on pull request #8745: [HUDI-6182] Hive sync use state transient time to avoid losing partit…

stream2000 commented on PR #8745: URL: https://github.com/apache/hudi/pull/8745#issuecomment-1572050050 @danny0405 Hi danny, could you help to review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

xiarixiaoyao commented on code in PR #8851: URL: https://github.com/apache/hudi/pull/8851#discussion_r1211598407 ## hudi-spark-datasource/hudi-spark3.0.x/src/main/java/org/apache/spark/sql/execution/datasources/parquet/Spark30HoodieVectorizedParquetRecordReader.java: ## @@

[GitHub] [hudi] hudi-bot commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

hudi-bot commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1572012983 ## CI report: * 23a574b64681c95c17db47d4c63c86d7e0215ba9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1572011208 ## CI report: * dacba722974aa32f506626c106d90fa86d22cd23 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

hudi-bot commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1571998458 ## CI report: * 23a574b64681c95c17db47d4c63c86d7e0215ba9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1571996984 ## CI report: * dacba722974aa32f506626c106d90fa86d22cd23 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.

hudi-bot commented on PR #8487: URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571996647 ## CI report: * 280515ea1c939f0afa7a4cd8a5593e55bd394648 Azure:

[GitHub] [hudi] KnightChess commented on pull request #8856: [HUDI-6300] fix file size parallelism not work when init metadata table

KnightChess commented on PR #8856: URL: https://github.com/apache/hudi/pull/8856#issuecomment-1571974348 before: https://github.com/apache/hudi/assets/20125927/9dc94884-347f-4284-8c9c-58d38a6c936a;> after:

[GitHub] [hudi] hudi-bot commented on pull request #8868: [HUDI-6278] Fixed the use of DynamoDBLockConfig class

hudi-bot commented on PR #8868: URL: https://github.com/apache/hudi/pull/8868#issuecomment-1571922230 ## CI report: * b6f86c770f7e35d7488cff0066d2d760453eb931 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #8851: [HUDI-6281] Comprehensive schema evolution supports column change with a default value

hudi-bot commented on PR #8851: URL: https://github.com/apache/hudi/pull/8851#issuecomment-1571921986 ## CI report: * 2db6852dd391973eab275dc7ef70c02bfbc5f652 UNKNOWN * 60c1399ac012bc61421f3bb1feb208decbcb6b6a UNKNOWN * e712d534d9c0a16b3027706ed394de88ff2b293d Azure:

  1   2   >