[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028678659 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028676981 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028676981 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028584032 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot removed a comment on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1028335503 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * d78e61c34acac7e23477f196388076cdd822dd69 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4352: [HUDI-1295] Metadata Index - Bloom filter and Column stats index to speed up index lookups

hudi-bot commented on pull request #4352: URL: https://github.com/apache/hudi/pull/4352#issuecomment-1028676669 ## CI report: * 235981abd20a498a3e29e98ce0eda9de35018f99 UNKNOWN * d78e61c34acac7e23477f196388076cdd822dd69 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot commented on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028601646 ## CI report: * 071c6180b5023f782da229552a6d3f63d1e4a67b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot removed a comment on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028572870 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

alexeykudinkin commented on a change in pull request #4556: URL: https://github.com/apache/hudi/pull/4556#discussion_r798204981 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -65,11 +65,70 @@ import java.util.

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

alexeykudinkin commented on a change in pull request #4556: URL: https://github.com/apache/hudi/pull/4556#discussion_r798204981 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -65,11 +65,70 @@ import java.util.

[jira] [Closed] (HUDI-3337) ParquetUtils fails extracting Parquet Column Range Metadata

[ https://issues.apache.org/jira/browse/HUDI-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-3337. - Resolution: Fixed > ParquetUtils fails extracting Parquet Column Range Metadata >

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028584032 ## CI report: * 86b335a77fb92cf34cb5e653694be626f6c7eba4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028556982 ## CI report: * 891d9658daa099eb50560741086aac23924e3600 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] yihua commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

yihua commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798194069 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the Apache Sof

[GitHub] [hudi] hudi-bot commented on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot commented on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028572870 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot removed a comment on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028571985 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot removed a comment on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028486554 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot commented on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028571985 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[jira] [Commented] (HUDI-3362) Hudi 0.8.0 cannot roleback MoR table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486197#comment-17486197 ] sivabalan narayanan commented on HUDI-3362: --- [~vinish_jail97] : we might need to

[jira] [Assigned] (HUDI-3362) Hudi 0.8.0 cannot roleback MoR table

[ https://issues.apache.org/jira/browse/HUDI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-3362: - Assignee: sivabalan narayanan > Hudi 0.8.0 cannot roleback MoR table > --

[GitHub] [hudi] zhangyue19921010 commented on pull request #4346: [HUDI-3045] New clustering regex match config to choose partitions when building clustering plan

zhangyue19921010 commented on pull request #4346: URL: https://github.com/apache/hudi/pull/4346#issuecomment-1028557537 Sure, pick it up -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028556982 ## CI report: * 891d9658daa099eb50560741086aac23924e3600 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028555830 ## CI report: * 891d9658daa099eb50560741086aac23924e3600 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

zhangyue19921010 commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798177169 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot commented on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1028555830 ## CI report: * 891d9658daa099eb50560741086aac23924e3600 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4721: [HUDI-3320] Hoodie metadata table validator

hudi-bot removed a comment on pull request #4721: URL: https://github.com/apache/hudi/pull/4721#issuecomment-1027541190 ## CI report: * 891d9658daa099eb50560741086aac23924e3600 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

zhangyue19921010 commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798176659 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

zhangyue19921010 commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798176560 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the

[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

zhangyue19921010 commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798176463 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the

[GitHub] [hudi] hudi-bot commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028547091 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f2627512c UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot removed a comment on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028522181 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f26275

[hudi] branch master updated (819e801 -> d681824)

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 819e801 [HUDI-3322][HUDI-3343] Fixing Metadata Table Records Duplication Issues (#4716) add d681824 [HUDI-33

[GitHub] [hudi] nsivabalan merged pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

nsivabalan merged pull request #4705: URL: https://github.com/apache/hudi/pull/4705 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan commented on a change in pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

nsivabalan commented on a change in pull request #4556: URL: https://github.com/apache/hudi/pull/4556#discussion_r798155468 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java ## @@ -65,11 +65,70 @@ import java.util.stre

[GitHub] [hudi] hudi-bot commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028522181 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f2627512c UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot removed a comment on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028513791 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f26275

[GitHub] [hudi] alexeykudinkin commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

alexeykudinkin commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028521365 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [hudi] hudi-bot commented on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot commented on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028521054 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * e6c57d02768a5561537546c4380ed141a4a497e0 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot removed a comment on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028469893 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5

[GitHub] [hudi] hudi-bot removed a comment on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot removed a comment on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028462344 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f26275

[GitHub] [hudi] hudi-bot commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028513791 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f2627512c UNKN

[jira] [Created] (HUDI-3366) Remove unnecessary hardcoded logic of disabling metadata table in tests

Ethan Guo created HUDI-3366: --- Summary: Remove unnecessary hardcoded logic of disabling metadata table in tests Key: HUDI-3366 URL: https://issues.apache.org/jira/browse/HUDI-3366 Project: Apache Hudi

[jira] [Updated] (HUDI-3366) Remove unnecessary hardcoded logic of disabling metadata table in tests

[ https://issues.apache.org/jira/browse/HUDI-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3366: Fix Version/s: 0.11.0 > Remove unnecessary hardcoded logic of disabling metadata table in tests > --

[jira] [Updated] (HUDI-3366) Remove unnecessary hardcoded logic of disabling metadata table in tests

[ https://issues.apache.org/jira/browse/HUDI-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-3366: Priority: Blocker (was: Major) > Remove unnecessary hardcoded logic of disabling metadata table in tests >

[jira] [Assigned] (HUDI-3366) Remove unnecessary hardcoded logic of disabling metadata table in tests

[ https://issues.apache.org/jira/browse/HUDI-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-3366: --- Assignee: Ethan Guo > Remove unnecessary hardcoded logic of disabling metadata table in tests > -

[GitHub] [hudi] hudi-bot commented on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot commented on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028504428 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot removed a comment on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028459357 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] yihua commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

yihua commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798125748 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the Apache Sof

[GitHub] [hudi] yihua commented on issue #4666: [SUPPORT] Table downgrade fails to delete non-existing file

yihua commented on issue #4666: URL: https://github.com/apache/hudi/issues/4666#issuecomment-1028487024 > hey @ganczarek : looks like there is a bug https://issues.apache.org/jira/browse/HUDI-3346. we will work on the fix. should be straight forward. Atleast in this case, manually deleting

[GitHub] [hudi] hudi-bot removed a comment on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot removed a comment on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028432638 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot commented on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028486554 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

alexeykudinkin commented on a change in pull request #4739: URL: https://github.com/apache/hudi/pull/4739#discussion_r798117376 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java ## @@ -87,40 +89,58 @@ public static void deleteMetadat

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

alexeykudinkin commented on a change in pull request #4739: URL: https://github.com/apache/hudi/pull/4739#discussion_r798116440 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java ## @@ -215,19 +215,16 @@ public HoodieMetadataPayload pr

[GitHub] [hudi] yihua commented on pull request #4679: [HUDI-3315] RFC-35 Make Flink writer stream friendly

yihua commented on pull request #4679: URL: https://github.com/apache/hudi/pull/4679#issuecomment-1028475241 > cc @yihua could you also please review this from the angle of making the write client abstractions more friendly I'll review this. -- This is an automated message from th

[GitHub] [hudi] alexeykudinkin closed pull request #4560: [WIP] Fixing generic usages for `HoodieRecordPayload`

alexeykudinkin closed pull request #4560: URL: https://github.com/apache/hudi/pull/4560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-un

[GitHub] [hudi] alexeykudinkin commented on pull request #4560: [WIP] Fixing generic usages for `HoodieRecordPayload`

alexeykudinkin commented on pull request #4560: URL: https://github.com/apache/hudi/pull/4560#issuecomment-1028472992 Given that we're on a path to eventually deprecate `HoodieRecordPayload` this is unnecessary -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [hudi] hudi-bot removed a comment on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot removed a comment on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028457685 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5

[GitHub] [hudi] hudi-bot commented on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot commented on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028469893 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5297 Azur

[jira] [Updated] (HUDI-1296) Implement Spark DataSource using range metadata for file/partition pruning

[ https://issues.apache.org/jira/browse/HUDI-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-1296: -- Story Points: 6 (was: 4) > Implement Spark DataSource using range metadata for file/partition p

[jira] [Commented] (HUDI-1296) Implement Spark DataSource using range metadata for file/partition pruning

[ https://issues.apache.org/jira/browse/HUDI-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486166#comment-17486166 ] Alexey Kudinkin commented on HUDI-1296: --- I've done high-level scoping of this effort

[GitHub] [hudi] yihua commented on issue #4600: [SUPPORT]When hive queries Hudi data, the query path is wrong

yihua commented on issue #4600: URL: https://github.com/apache/hudi/issues/4600#issuecomment-1028469418 As @xiarixiaoyao mentions, compaction should compact log files into base file formats like parquet, which can be then read by Hive. There are different ways to trigger compaction, e.g.,

[GitHub] [hudi] hudi-bot commented on pull request #4559: [HUDI-3206][Stacked on 4556] Unify Hive's MOR implementations to avoid duplication

hudi-bot commented on pull request #4559: URL: https://github.com/apache/hudi/pull/4559#issuecomment-1028468345 ## CI report: * 47970bd3a9cbbf2eb85b0a87f899256487efdffa UNKNOWN * 6d402f17d668a5a2d8bec5b8094fad6e997407b8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4559: [HUDI-3206][Stacked on 4556] Unify Hive's MOR implementations to avoid duplication

hudi-bot removed a comment on pull request #4559: URL: https://github.com/apache/hudi/pull/4559#issuecomment-1028380183 ## CI report: * 47970bd3a9cbbf2eb85b0a87f899256487efdffa UNKNOWN * c18bd337517c5842f2db1ee0075df19c05fafe91 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] nsivabalan commented on a change in pull request #4721: [HUDI-3320] Hoodie metadata table validator

nsivabalan commented on a change in pull request #4721: URL: https://github.com/apache/hudi/pull/4721#discussion_r798108001 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java ## @@ -0,0 +1,458 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] nsivabalan closed pull request #4734: [DO NOT MERGE] Testing CI 1

nsivabalan closed pull request #4734: URL: https://github.com/apache/hudi/pull/4734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan closed pull request #4732: [DO_NOT_MERGE][WIP] debugging some tests for metadata restore

nsivabalan closed pull request #4732: URL: https://github.com/apache/hudi/pull/4732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan closed pull request #4736: [DO NOT MERGE] Testing CI 3

nsivabalan closed pull request #4736: URL: https://github.com/apache/hudi/pull/4736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan closed pull request #4735: [DO NOT MERGE] Testing CI 2

nsivabalan closed pull request #4735: URL: https://github.com/apache/hudi/pull/4735 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan closed pull request #4737: [DO NOT MERGE] Testing CI 4

nsivabalan closed pull request #4737: URL: https://github.com/apache/hudi/pull/4737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan closed pull request #4738: [DO NOT MERGE] Testing CI 5

nsivabalan closed pull request #4738: URL: https://github.com/apache/hudi/pull/4738 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] hudi-bot removed a comment on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot removed a comment on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028457463 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f26275

[GitHub] [hudi] hudi-bot commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028462344 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f2627512c UNKN

[GitHub] [hudi] nsivabalan commented on a change in pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

nsivabalan commented on a change in pull request #4739: URL: https://github.com/apache/hudi/pull/4739#discussion_r798098988 ## File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java ## @@ -215,19 +215,16 @@ public HoodieMetadataPayload preCom

[GitHub] [hudi] hudi-bot commented on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot commented on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028459357 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot removed a comment on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028457602 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot commented on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028457685 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5297 Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot removed a comment on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028455703 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5

[GitHub] [hudi] hudi-bot commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028457463 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f2627512c UNKN

[GitHub] [hudi] hudi-bot commented on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot commented on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1028457602 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot removed a comment on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028455516 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f26275

[GitHub] [hudi] hudi-bot removed a comment on pull request #4659: [HUDI-3091] Making SIMPLE index as the default index type

hudi-bot removed a comment on pull request #4659: URL: https://github.com/apache/hudi/pull/4659#issuecomment-1019743003 ## CI report: * 647c6a4d9dad7e517c48d70857f9ebd2faf5c57c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] yihua commented on pull request #3866: [HUDI-1430] SparkDataFrameWriteClient

yihua commented on pull request #3866: URL: https://github.com/apache/hudi/pull/3866#issuecomment-1028456039 @xushiyan when this is mostly in shape, we can go through the code again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [hudi] hudi-bot removed a comment on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot removed a comment on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028430624 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5

[GitHub] [hudi] hudi-bot commented on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot commented on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028455703 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5297 Azur

[GitHub] [hudi] hudi-bot commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028455516 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f2627512c UNKN

[GitHub] [hudi] hudi-bot removed a comment on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

hudi-bot removed a comment on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028382543 ## CI report: * 77d11131baabd1c4e3cc2050337daca4df5f6427 UNKNOWN * 3d9c2ae28da858d1e8476052c99391015effb7db UNKNOWN * 31b0669d7b638bd65a17b22a2ceb772f26275

[GitHub] [hudi] alexeykudinkin commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

alexeykudinkin commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028453307 @yihua yeah, it's rebased on master now and ready for another pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

alexeykudinkin commented on a change in pull request #4556: URL: https://github.com/apache/hudi/pull/4556#discussion_r798092122 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieRealtimeFileSplit.java ## @@ -44,9 +44,7 @@ private Option hood

[GitHub] [hudi] yihua commented on pull request #4556: [HUDI-3191] Removing duplicating file-listing process w/in Hive's MOR `FileInputFormat`s

yihua commented on pull request #4556: URL: https://github.com/apache/hudi/pull/4556#issuecomment-1028442601 @alexeykudinkin is this PR ready for another look or you're still addressing comments -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [hudi] hudi-bot commented on pull request #4667: [HUDI-3276][Stacked on 4559] Rebased Parquet-based `FileInputFormat` impls to inherit from `MapredParquetInputFormat`

hudi-bot commented on pull request #4667: URL: https://github.com/apache/hudi/pull/4667#issuecomment-1028440018 ## CI report: * ed1df9c2c6a5c79a5b450cf37e783fddfe861d35 UNKNOWN * 29733a0d437997485b21327a6c256233e35c4d3b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4667: [HUDI-3276][Stacked on 4559] Rebased Parquet-based `FileInputFormat` impls to inherit from `MapredParquetInputFormat`

hudi-bot removed a comment on pull request #4667: URL: https://github.com/apache/hudi/pull/4667#issuecomment-1028372936 ## CI report: * ed1df9c2c6a5c79a5b450cf37e783fddfe861d35 UNKNOWN * b504aa798fa399e7b162203627490f9090656a32 Azure: [FAILURE](https://dev.azure.com/apache-hud

[GitHub] [hudi] yihua commented on pull request #4078: [HUDI-2833][Design] Merge small archive files instead of expanding indefinitely.

yihua commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-1028439247 cc @vinothchandar this PR adds new functionality in archived timeline with a feature flag and a piece of error handling logic which cannot be feature flagged. You may want to take

[GitHub] [hudi] hudi-bot commented on pull request #4669: [HUDI-3239][Stacked on 4667] Convert `BaseHoodieTableFileIndex` to Java

hudi-bot commented on pull request #4669: URL: https://github.com/apache/hudi/pull/4669#issuecomment-1028436169 ## CI report: * 54e68ec73aa7292556d5f00c90859a776acc109f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4669: [HUDI-3239][Stacked on 4667] Convert `BaseHoodieTableFileIndex` to Java

hudi-bot removed a comment on pull request #4669: URL: https://github.com/apache/hudi/pull/4669#issuecomment-1028372962 ## CI report: * 851e61cdb31748501d28d638bc192ccc4955d665 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] yihua commented on pull request #4346: [HUDI-3045] New clustering regex match config to choose partitions when building clustering plan

yihua commented on pull request #4346: URL: https://github.com/apache/hudi/pull/4346#issuecomment-1028433650 > @yihua I feel it would be better to add a new option in `ClusteringPlanPartitionFilterMode` rather than doing regex in place. Yes, that could allow more flexible filtering.

[GitHub] [hudi] hudi-bot removed a comment on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot removed a comment on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028430746 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot commented on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028432638 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4739: [HUDI-3365] Make sure Metadata Table records are updated appropriately on HDFS

hudi-bot commented on pull request #4739: URL: https://github.com/apache/hudi/pull/4739#issuecomment-1028430746 ## CI report: * 5967651c87dc1a020e82a9a92de0f20ebeefb785 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot removed a comment on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028399211 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5

[GitHub] [hudi] hudi-bot commented on pull request #4705: [HUDI-3337] Fixing Parquet Column Range metadata extraction

hudi-bot commented on pull request #4705: URL: https://github.com/apache/hudi/pull/4705#issuecomment-1028430624 ## CI report: * fc6f1f4af2201fb5541aeae70c745e7b6dc3981e UNKNOWN * 2fa66c4290ee2555973e29934ca7ecb8e4a0e709 UNKNOWN * 2ad2dcddf911dea6b73d6fb7a394dbbb5297 Azur

  1   2   >