[GitHub] [hudi] hudi-bot commented on pull request #8701: Update docker-compose_hadoop284_hive233_spark244_mac_aarch64.yml

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8701: URL: https://github.com/apache/hudi/pull/8701#issuecomment-1546534125 ## CI report: * a52a8f5096a050f7cf934d4a253e518bcbb91cd3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] hudi-bot commented on pull request #8702: Update docker-compose_hadoop284_hive233_spark244.yml

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8702: URL: https://github.com/apache/hudi/pull/8702#issuecomment-1546534135 ## CI report: * f1adb748936a24ff160b5b05f9f126793f25def3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192914114 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -824,25 +863,22 @@ private interface ConvertMetadata

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192914114 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -824,25 +863,22 @@ private interface ConvertMetadata

[jira] [Assigned] (HUDI-6207) Files pruning for bucket index table pk filtering queries using Spark SQL

2023-05-12 Thread Jing Zhang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhang reassigned HUDI-6207: Assignee: Jing Zhang > Files pruning for bucket index table pk filtering queries using Spark SQL >

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192913868 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieBackedTableMetadataWriter.java: ## @@ -118,46 +123,32 @@ protected void initRegistry()

[jira] [Created] (HUDI-6207) Files pruning for bucket index table pk filtering queries using Spark SQL

2023-05-12 Thread Jing Zhang (Jira)
Jing Zhang created HUDI-6207: Summary: Files pruning for bucket index table pk filtering queries using Spark SQL Key: HUDI-6207 URL: https://issues.apache.org/jira/browse/HUDI-6207 Project: Apache Hudi

[GitHub] [hudi] danny0405 merged pull request #8698: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvol…

2023-05-12 Thread via GitHub
danny0405 merged PR #8698: URL: https://github.com/apache/hudi/pull/8698 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[hudi] branch master updated: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvolution (#8698)

2023-05-12 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 3df303eb770 [MINOR] Prevent timeline server fro

[jira] [Assigned] (HUDI-6070) Files pruning for bucket index table pk filtering queries

2023-05-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-6070: Assignee: Jing Zhang > Files pruning for bucket index table pk filtering queries >

[GitHub] [hudi] danny0405 commented on a diff in pull request #8698: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvol…

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8698: URL: https://github.com/apache/hudi/pull/8698#discussion_r1192911458 ## hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestSchemaEvolution.java: ## @@ -272,6 +272,7 @@ private TableOptions defaultTableOptions(Strin

[GitHub] [hudi] hudi-bot commented on pull request #8505: [HUDI-6106] Spark offline compaction/Clustering Job will do clean like Flink job

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8505: URL: https://github.com/apache/hudi/pull/8505#issuecomment-1546515757 ## CI report: * f7c73e83812258b53b979afbd6d465e9066b801f UNKNOWN * 269fad02a5346121e823a15c9804e2e63eb16c30 UNKNOWN * 442430f680316bdfefc27c4aca9f7cd94e95373c UNKNOWN * e6

[GitHub] [hudi] danny0405 commented on a diff in pull request #8683: [HUDI-5533] Support spark columns comments

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8683: URL: https://github.com/apache/hudi/pull/8683#discussion_r1192910419 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala: ## @@ -58,33 +58,37 @@ private[sql] object SchemaConverters {

[GitHub] [hudi] danny0405 commented on pull request #8505: [HUDI-6106] Spark offline compaction/Clustering Job will do clean like Flink job

2023-05-12 Thread via GitHub
danny0405 commented on PR #8505: URL: https://github.com/apache/hudi/pull/8505#issuecomment-1546513645 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [hudi] hudi-bot commented on pull request #8702: Update docker-compose_hadoop284_hive233_spark244.yml

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8702: URL: https://github.com/apache/hudi/pull/8702#issuecomment-1546503183 ## CI report: * f1adb748936a24ff160b5b05f9f126793f25def3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] hudi-bot commented on pull request #8701: Update docker-compose_hadoop284_hive233_spark244_mac_aarch64.yml

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8701: URL: https://github.com/apache/hudi/pull/8701#issuecomment-1546503176 ## CI report: * a52a8f5096a050f7cf934d4a253e518bcbb91cd3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] hudi-bot commented on pull request #8702: Update docker-compose_hadoop284_hive233_spark244.yml

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8702: URL: https://github.com/apache/hudi/pull/8702#issuecomment-1546501676 ## CI report: * f1adb748936a24ff160b5b05f9f126793f25def3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8701: Update docker-compose_hadoop284_hive233_spark244_mac_aarch64.yml

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8701: URL: https://github.com/apache/hudi/pull/8701#issuecomment-1546501663 ## CI report: * a52a8f5096a050f7cf934d4a253e518bcbb91cd3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1546501423 ## CI report: * 27d61f01fb6709e3aaa08de9ace7738dbedffb24 UNKNOWN * b572d737ef10724f71642084c0edf9a9a26540cc UNKNOWN * a44c71610c2efd1ebdb1a19c5195f8b1b5e59df7 UNKNOWN * df

[GitHub] [hudi] alberttwong opened a new pull request, #8702: Update docker-compose_hadoop284_hive233_spark244.yml

2023-05-12 Thread via GitHub
alberttwong opened a new pull request, #8702: URL: https://github.com/apache/hudi/pull/8702 Updates to fix issue https://github.com/apache/hudi/issues/8700 ### Change Logs Fixes to the docker compose yaml so that you don't get URI Exception error when remotely connecting to doc

[GitHub] [hudi] alberttwong opened a new pull request, #8701: Update docker-compose_hadoop284_hive233_spark244_mac_aarch64.yml

2023-05-12 Thread via GitHub
alberttwong opened a new pull request, #8701: URL: https://github.com/apache/hudi/pull/8701 Fix for https://github.com/apache/hudi/issues/8700 ### Change Logs Fixes URI exception error when connecting remotely using a java client ### Impact none ### Risk lev

[GitHub] [hudi] hudi-bot commented on pull request #8645: [HUDI-6193] Add support to standalone utility tool to fetch file size stats for a given table w/ optional partition filters

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8645: URL: https://github.com/apache/hudi/pull/8645#issuecomment-1546485772 ## CI report: * 3f4a740c6e9df40b04416e8c9632eec06487f76c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1702

[GitHub] [hudi] Sam-Serpoosh commented on issue #8519: [SUPPORT] Deltastreamer AvroDeserializer failing with java.lang.NullPointerException

2023-05-12 Thread via GitHub
Sam-Serpoosh commented on issue #8519: URL: https://github.com/apache/hudi/issues/8519#issuecomment-1546481863 @the-other-tim-brown According to this [SO question/thread](https://stackoverflow.com/questions/76239689/non-nested-avro-schema-for-postgres-change-log-events-debezium-confluent-sch

[GitHub] [hudi] yihua commented on a diff in pull request #8629: [HUDI-6168] Add ability to parse partition value into row for S3 and GCS sources

2023-05-12 Thread via GitHub
yihua commented on code in PR #8629: URL: https://github.com/apache/hudi/pull/8629#discussion_r1192889237 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/CloudStoreIngestionConfig.java: ## @@ -103,4 +103,8 @@ public class CloudStoreIngestionConfig {

[GitHub] [hudi] yihua commented on a diff in pull request #8699: [DOCS] Add new Blogs to Hudi website

2023-05-12 Thread via GitHub
yihua commented on code in PR #8699: URL: https://github.com/apache/hudi/pull/8699#discussion_r1192887917 ## website/blog/2023-01-27-Introducing-native-support-for-Apache-Hudi-Delta-Lake-Apache-Iceberg-on-AWS-Glue-for-Apache-Spark.mdx: ## @@ -5,8 +5,8 @@ authors: category: blog

[GitHub] [hudi] bhasudha merged pull request #8699: [DOCS] Add new Blogs to Hudi website

2023-05-12 Thread via GitHub
bhasudha merged PR #8699: URL: https://github.com/apache/hudi/pull/8699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[hudi] branch asf-site updated: [DOCS] Add new Blogs to Hudi website (#8699)

2023-05-12 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 32420b46698 [DOCS] Add new Blogs to Hudi

[GitHub] [hudi] bhasudha commented on a diff in pull request #8699: [DOCS] Add new Blogs to Hudi website

2023-05-12 Thread via GitHub
bhasudha commented on code in PR #8699: URL: https://github.com/apache/hudi/pull/8699#discussion_r1192885012 ## website/blog/2023-01-27-Introducing-native-support-for-Apache-Hudi-Delta-Lake-Apache-Iceberg-on-AWS-Glue-for-Apache-Spark.mdx: ## @@ -5,8 +5,8 @@ authors: category: b

[GitHub] [hudi] hudi-bot commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1546465206 ## CI report: * 27d61f01fb6709e3aaa08de9ace7738dbedffb24 UNKNOWN * b572d737ef10724f71642084c0edf9a9a26540cc UNKNOWN * ce89b12639ebe78146afcd2f9c95d646226f1127 Azure: [SUCCES

[hudi] branch master updated: [HUDI-6204] Add bundle validation on Spark 3.3.2 (#8692)

2023-05-12 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 90c40d48b06 [HUDI-6204] Add bundle validation on Sp

[GitHub] [hudi] yihua merged pull request #8692: [HUDI-6204] Add bundle validation on Spark 3.3.2

2023-05-12 Thread via GitHub
yihua merged PR #8692: URL: https://github.com/apache/hudi/pull/8692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] yihua commented on a diff in pull request #8699: [DOCS] Add new Blogs to Hudi website

2023-05-12 Thread via GitHub
yihua commented on code in PR #8699: URL: https://github.com/apache/hudi/pull/8699#discussion_r1192871749 ## website/blog/2023-01-27-Introducing-native-support-for-Apache-Hudi-Delta-Lake-Apache-Iceberg-on-AWS-Glue-for-Apache-Spark.mdx: ## @@ -5,8 +5,8 @@ authors: category: blog

[GitHub] [hudi] blrnw3 commented on pull request #7146: [HUDI-5165][WIP] Adding sorting option during insert/upsert

2023-05-12 Thread via GitHub
blrnw3 commented on PR #7146: URL: https://github.com/apache/hudi/pull/7146#issuecomment-1546440212 We'd love to have this feature, very valuable for us -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] hudi-bot commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1546433657 ## CI report: * 27d61f01fb6709e3aaa08de9ace7738dbedffb24 UNKNOWN * b572d737ef10724f71642084c0edf9a9a26540cc UNKNOWN * ce89b12639ebe78146afcd2f9c95d646226f1127 Azure: [SUCCES

[GitHub] [hudi] bhasudha opened a new pull request, #8699: [DOCS] Add new Blogs to Hudi website

2023-05-12 Thread via GitHub
bhasudha opened a new pull request, #8699: URL: https://github.com/apache/hudi/pull/8699 - Add new blogs - Fix tags in older ones ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public A

[GitHub] [hudi] hudi-bot commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1546429197 ## CI report: * 27d61f01fb6709e3aaa08de9ace7738dbedffb24 UNKNOWN * b572d737ef10724f71642084c0edf9a9a26540cc UNKNOWN * ce89b12639ebe78146afcd2f9c95d646226f1127 Azure: [SUCCES

[GitHub] [hudi] kazdy commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
kazdy commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1546422024 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [hudi] hudi-bot commented on pull request #8682: [DO NOT MERGE] [HUDI-6198] Run gh actions with Spark 3.4.0

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8682: URL: https://github.com/apache/hudi/pull/8682#issuecomment-1546331279 ## CI report: * c23f6ed02a81dfac0d218cee75d18fee3a9b31df UNKNOWN * d3756a68d846716a0ebfc6ae546249fe362e7d6f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1546330086 ## CI report: * 27d61f01fb6709e3aaa08de9ace7738dbedffb24 UNKNOWN * b572d737ef10724f71642084c0edf9a9a26540cc UNKNOWN * ce89b12639ebe78146afcd2f9c95d646226f1127 Azure: [SUCCES

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8690: [WIP][HUDI-6199] Fix deletes with custom payload implementation

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8690: URL: https://github.com/apache/hudi/pull/8690#discussion_r1192800728 ## hudi-common/src/main/java/org/apache/hudi/common/model/debezium/AbstractDebeziumAvroPayload.java: ## @@ -91,4 +90,14 @@ private Option handleDeleteOperation(Indexed

[GitHub] [hudi] yihua commented on a diff in pull request #8690: [WIP][HUDI-6199] Fix deletes with custom payload implementation

2023-05-12 Thread via GitHub
yihua commented on code in PR #8690: URL: https://github.com/apache/hudi/pull/8690#discussion_r1192793403 ## hudi-common/src/main/java/org/apache/hudi/common/model/debezium/AbstractDebeziumAvroPayload.java: ## @@ -91,4 +90,14 @@ private Option handleDeleteOperation(IndexedRecor

[GitHub] [hudi] hudi-bot commented on pull request #8698: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvol…

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8698: URL: https://github.com/apache/hudi/pull/8698#issuecomment-1546273950 ## CI report: * f9b07d31865ec1d31885b9be4542b82f4590abb3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] hudi-bot commented on pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8697: URL: https://github.com/apache/hudi/pull/8697#issuecomment-1546188410 ## CI report: * 6250cd0f2bfe2ba9c3b3053940f2a75be78c2f98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192729597 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieMetadataBulkInsertPartitioner.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the Apac

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192728731 ## hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java: ## @@ -670,17 +670,75 @@ private Long getTableChecksum() { return getLong(TABLE_C

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192728311 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java: ## @@ -2482,6 +2483,14 @@ public void testMetadataMetric

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192726306 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -1097,87 +1165,76 @@ protected void cleanIfNecessar

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192725066 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -562,53 +532,144 @@ private boolean isCommitRever

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192724508 ## hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java: ## @@ -670,17 +670,75 @@ private Long getTableChecksum() { return getLong(TABLE_C

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192722934 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieMetadataBulkInsertPartitioner.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apac

[jira] [Updated] (HUDI-6206) Enhance Metadata tests to validate col stats

2023-05-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-6206: -- Description: I happened to check for col stats tests in TestHoodieBackedMetadata and did

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192717879 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -562,53 +532,144 @@ private boolean isCommitRever

[jira] [Updated] (HUDI-6206) Enhance Metadata tests to validate col stats

2023-05-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-6206: -- Description: I happened to check for col stats tests in TestHoodieBackedMetadata and did

[jira] [Created] (HUDI-6206) Enhance Metadata tests to validate col stats

2023-05-12 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-6206: - Summary: Enhance Metadata tests to validate col stats Key: HUDI-6206 URL: https://issues.apache.org/jira/browse/HUDI-6206 Project: Apache Hudi Issu

[jira] [Updated] (HUDI-6206) Enhance Metadata tests to validate col stats

2023-05-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-6206: -- Description: I happened to check for col stats tests in TestHoodieBackedMetadata and did

[jira] [Updated] (HUDI-6206) Enhance Metadata tests to validate col stats

2023-05-12 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-6206: -- Labels: release-0.14.0-blocker (was: ) > Enhance Metadata tests to validate col stats >

[GitHub] [hudi] xushiyan commented on a diff in pull request #7143: [HUDI-5175] Improving FileIndex load performance in PARALLELISM mode

2023-05-12 Thread via GitHub
xushiyan commented on code in PR #7143: URL: https://github.com/apache/hudi/pull/7143#discussion_r1192695414 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieCommonConfig.java: ## @@ -41,6 +41,13 @@ public class HoodieCommonConfig extends HoodieConfig { .n

[GitHub] [hudi] hudi-bot commented on pull request #8698: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvol…

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8698: URL: https://github.com/apache/hudi/pull/8698#issuecomment-1546140933 ## CI report: * f9b07d31865ec1d31885b9be4542b82f4590abb3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192689257 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -373,105 +356,92 @@ public List getEnabledPartitio

[jira] [Created] (HUDI-6205) Users should be able to dictate which partition in MDT needs to be built async and which one is inline

2023-05-12 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-6205: - Summary: Users should be able to dictate which partition in MDT needs to be built async and which one is inline Key: HUDI-6205 URL: https://issues.apache.org/jira/browse

[GitHub] [hudi] hudi-bot commented on pull request #8698: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvol…

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8698: URL: https://github.com/apache/hudi/pull/8698#issuecomment-1546132819 ## CI report: * f9b07d31865ec1d31885b9be4542b82f4590abb3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192675584 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/ScheduleIndexActionExecutor.java: ## @@ -100,15 +99,6 @@ public Option execute() {

[GitHub] [hudi] hudi-bot commented on pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8684: URL: https://github.com/apache/hudi/pull/8684#issuecomment-1546124497 ## CI report: * 364b1fe0dde303750ca0f8eebe5cd530c2a66a3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] ad1happy2go commented on issue #5451: [SUPPORT] Hudi 0.10.1 raises exception java.lang.NoClassDefFoundError: com/amazonaws/services/dynamodbv2/model/LockNotGrantedException

2023-05-12 Thread via GitHub
ad1happy2go commented on issue #5451: URL: https://github.com/apache/hudi/issues/5451#issuecomment-1546096342 Also I have tried with both Glue 3.0 and Glue 4.0 with default "--datalake-formats hudi" and both of them are working fine with dynamo db concurrency. -- This is an automated mes

[GitHub] [hudi] voonhous opened a new pull request, #8698: [MINOR] Prevent timeline server from being reused in ITTestSchemaEvol…

2023-05-12 Thread via GitHub
voonhous opened a new pull request, #8698: URL: https://github.com/apache/hudi/pull/8698 …ution ### Change Logs Fix `org.apache.hudi.table.ITTestSchemaEvolution` tests that were failing with the following error: ```log 22793 [main] INFO org.apache.hudi.common.table.

[GitHub] [hudi] hudi-bot commented on pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8684: URL: https://github.com/apache/hudi/pull/8684#issuecomment-1546079839 ## CI report: * e2785f4675ddf74582ff34590608a5d71c5e9a2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1700

[GitHub] [hudi] hudi-bot commented on pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8697: URL: https://github.com/apache/hudi/pull/8697#issuecomment-1546072324 ## CI report: * 6250cd0f2bfe2ba9c3b3053940f2a75be78c2f98 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] hudi-bot commented on pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8684: URL: https://github.com/apache/hudi/pull/8684#issuecomment-1546072246 ## CI report: * e2785f4675ddf74582ff34590608a5d71c5e9a2d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1700

[GitHub] [hudi] hudi-bot commented on pull request #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8697: URL: https://github.com/apache/hudi/pull/8697#issuecomment-1546064642 ## CI report: * 6250cd0f2bfe2ba9c3b3053940f2a75be78c2f98 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8666: URL: https://github.com/apache/hudi/pull/8666#issuecomment-1546064468 ## CI report: * bb7e42e3225e97863cb3e96953784385e19ec638 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1704

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8690: [WIP][HUDI-6199] Fix deletes with custom payload implementation

2023-05-12 Thread via GitHub
nsivabalan commented on code in PR #8690: URL: https://github.com/apache/hudi/pull/8690#discussion_r1192602586 ## hudi-common/src/main/java/org/apache/hudi/common/model/debezium/AbstractDebeziumAvroPayload.java: ## @@ -91,4 +90,14 @@ private Option handleDeleteOperation(Indexed

[GitHub] [hudi] nsivabalan commented on pull request #8389: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

2023-05-12 Thread via GitHub
nsivabalan commented on PR #8389: URL: https://github.com/apache/hudi/pull/8389#issuecomment-1546013839 Closing in favor of https://github.com/apache/hudi/pull/8697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] nsivabalan opened a new pull request, #8697: [HUDI-5514] Improving usability/performance with out of box default for append only use-cases

2023-05-12 Thread via GitHub
nsivabalan opened a new pull request, #8697: URL: https://github.com/apache/hudi/pull/8697 ### Change Logs We are adding [auto record key gen](https://github.com/apache/hudi/pull/8107) mainly to cater to append only use-cases. So, as part of the work stream, this is a follow up patch

[GitHub] [hudi] codope closed issue #7663: [SUPPORT] ALTER TABLE DROP PARTITION DDL may cause data inconsistencies when table service actions are performed

2023-05-12 Thread via GitHub
codope closed issue #7663: [SUPPORT] ALTER TABLE DROP PARTITION DDL may cause data inconsistencies when table service actions are performed URL: https://github.com/apache/hudi/issues/7663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] ad1happy2go commented on issue #7663: [SUPPORT] ALTER TABLE DROP PARTITION DDL may cause data inconsistencies when table service actions are performed

2023-05-12 Thread via GitHub
ad1happy2go commented on issue #7663: URL: https://github.com/apache/hudi/issues/7663#issuecomment-1545983576 @voonhous Confirmed with the master code , that fix is working fine. It is now restricting the drop partition to happen only if any remaining compaction/clustering is pending.

[GitHub] [hudi] ad1happy2go commented on issue #5451: [SUPPORT] Hudi 0.10.1 raises exception java.lang.NoClassDefFoundError: com/amazonaws/services/dynamodbv2/model/LockNotGrantedException

2023-05-12 Thread via GitHub
ad1happy2go commented on issue #5451: URL: https://github.com/apache/hudi/issues/5451#issuecomment-1545975371 @jtmzheng I was not able to reproduce the bug. I tried with both versions 0.12.2 and master code. I was able to successfully use DynamoDB without any issues with spark bundle and aw

[GitHub] [hudi] hudi-bot commented on pull request #8638: added new exception types

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8638: URL: https://github.com/apache/hudi/pull/8638#issuecomment-1545937630 ## CI report: * cffdd5242a58a50e4ce11a5d098d4ab7f4da1ec8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1684

[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #8638: added new exception types

2023-05-12 Thread via GitHub
the-other-tim-brown commented on code in PR #8638: URL: https://github.com/apache/hudi/pull/8638#discussion_r1192524362 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/exception/HoodieDeltaStreamerMetaSyncException.java: ## @@ -0,0 +1,25 @@ +/* + * Licensed to the Apac

[GitHub] [hudi] hudi-bot commented on pull request #7865: [HUDI-5710] Load all partitions in advance for clean when MDT is enabled

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7865: URL: https://github.com/apache/hudi/pull/7865#issuecomment-1545925359 ## CI report: * 59c457e89bef1b404627f9b3700d65235044387c UNKNOWN * bb88f6ebfc9ac76a3789073190c7cc5c21fc1d80 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] PhantomHunt commented on issue #8678: [SUPPORT] Hudi MOR table not getting cleaned/compacted and filling up S3 bucket

2023-05-12 Thread via GitHub
PhantomHunt commented on issue #8678: URL: https://github.com/apache/hudi/issues/8678#issuecomment-1545918096 Hi @ad1happy2go, We added this configuration to the table (with 999+ objects) `'hoodie.compact.inline' : "true", # 'hoodie.compact.inline.max.delta.commits': compaction_

[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8666: URL: https://github.com/apache/hudi/pull/8666#issuecomment-1545860703 ## CI report: * 1b2f28447ac507b35f82a0534ebd958a8fd8980d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1702

[GitHub] [hudi] yesemsanthoshkumar commented on a diff in pull request #8399: [HUDI-4630] Add transformer capability to individual feeds in MultiTableDeltaStreamer

2023-05-12 Thread via GitHub
yesemsanthoshkumar commented on code in PR #8399: URL: https://github.com/apache/hudi/pull/8399#discussion_r1192470009 ## hudi-utilities/src/test/resources/delta-streamer-config/short_trip_uber_config.properties: ## @@ -25,3 +25,4 @@ hoodie.datasource.hive_sync.table=short_trip

[GitHub] [hudi] hudi-bot commented on pull request #8666: [HUDI-915] Add missing partititonpath to records COW

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8666: URL: https://github.com/apache/hudi/pull/8666#issuecomment-1545849363 ## CI report: * 1b2f28447ac507b35f82a0534ebd958a8fd8980d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1702

[GitHub] [hudi] kazdy commented on pull request #7998: [HUDI-5824] Fix: do not combine if write operation is Upsert and COMBINE_BEFORE_UPSERT is false

2023-05-12 Thread via GitHub
kazdy commented on PR #7998: URL: https://github.com/apache/hudi/pull/7998#issuecomment-1545726848 @bvaradar CI is green, could you please take a look at it again? thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] zhuanshenbsj1 opened a new pull request, #8505: [HUDI-6106] Spark offline compaction/Clustering Job will do clean like Flink job

2023-05-12 Thread via GitHub
zhuanshenbsj1 opened a new pull request, #8505: URL: https://github.com/apache/hudi/pull/8505 ### Change Logs Adjust the cleaning operation in Spark offline compact/cluster, when ASYNC_CLEAN is true will start asynchronous cleaning in prewrite and wait for the async-clean completion,

[GitHub] [hudi] danny0405 closed pull request #8505: [HUDI-6106] Spark offline compaction/Clustering Job will do clean like Flink job

2023-05-12 Thread via GitHub
danny0405 closed pull request #8505: [HUDI-6106] Spark offline compaction/Clustering Job will do clean like Flink job URL: https://github.com/apache/hudi/pull/8505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] vinothchandar commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

2023-05-12 Thread via GitHub
vinothchandar commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1192338773 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review

[GitHub] [hudi] vinothchandar commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

2023-05-12 Thread via GitHub
vinothchandar commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1192335443 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review

[GitHub] [hudi] vinothchandar commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

2023-05-12 Thread via GitHub
vinothchandar commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1192332216 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review

[GitHub] [hudi] hudi-bot commented on pull request #7865: [HUDI-5710] Load all partitions in advance for clean when MDT is enabled

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7865: URL: https://github.com/apache/hudi/pull/7865#issuecomment-1545679619 ## CI report: * 59c457e89bef1b404627f9b3700d65235044387c UNKNOWN * d58c1e20faba3f484e8fa38c474a955b1b1dd0f0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8574: [HUDI-6139] Add support for Transformer schema validation in DeltaStreamer

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8574: URL: https://github.com/apache/hudi/pull/8574#issuecomment-1545670822 ## CI report: * 4550fea4dfa7a73ae3face52bfc66d4b46adac37 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1703

[GitHub] [hudi] hudi-bot commented on pull request #7865: [HUDI-5710] Load all partitions in advance for clean when MDT is enabled

2023-05-12 Thread via GitHub
hudi-bot commented on PR #7865: URL: https://github.com/apache/hudi/pull/7865#issuecomment-1545669303 ## CI report: * 59c457e89bef1b404627f9b3700d65235044387c UNKNOWN * d58c1e20faba3f484e8fa38c474a955b1b1dd0f0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] kazdy commented on a diff in pull request #8679: [DOCS] [RFC-69] Hudi 1.X

2023-05-12 Thread via GitHub
kazdy commented on code in PR #8679: URL: https://github.com/apache/hudi/pull/8679#discussion_r1192298242 ## rfc/rfc-69/rfc-69.md: ## @@ -0,0 +1,159 @@ + +# RFC-69: Hudi 1.X + +## Proposers + +* Vinoth Chandar + +## Approvers + +* Hudi PMC + +## Status + +Under Review + +## Ab

[GitHub] [hudi] slfan1989 commented on a diff in pull request #8478: [HUDI-6086] Improve HiveSchemaUtil#generateCreateDDL With StringBuilder

2023-05-12 Thread via GitHub
slfan1989 commented on code in PR #8478: URL: https://github.com/apache/hudi/pull/8478#discussion_r1192283013 ## hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/util/TestHiveSchemaUtil.java: ## @@ -145,4 +145,11 @@ public void testSchemaDiffForTimestampMicros() {

[GitHub] [hudi] chinmay-032 commented on issue #8625: [SUPPORT] Hudi partial updates not working with JSON inferred dataframe

2023-05-12 Thread via GitHub
chinmay-032 commented on issue #8625: URL: https://github.com/apache/hudi/issues/8625#issuecomment-1545603221 New insights after discussing witn @ad1happy2go: The problem arises when using a dynamically created StructType schema. When a statically declared schema is used, the updates

[GitHub] [hudi] PhantomHunt commented on issue #8676: [SUPPORT] Cannot install Hoodie CLI on EC2

2023-05-12 Thread via GitHub
PhantomHunt commented on issue #8676: URL: https://github.com/apache/hudi/issues/8676#issuecomment-1545600681 Thanks @ad1happy2go! I have now installed hudi cli successfully but while running it, got this error - `hudi->create --path s3://gn-video-richmedia-nonprod-hudi-tables/nonprod_h

[GitHub] [hudi] hudi-bot commented on pull request #8669: [HUDI-5362] Rebase IncrementalRelation over HoodieBaseRelation

2023-05-12 Thread via GitHub
hudi-bot commented on PR #8669: URL: https://github.com/apache/hudi/pull/8669#issuecomment-1545524592 ## CI report: * 0eacefd8bc063e0c574068f09670014804f10dc2 UNKNOWN * d2f1d265394f8a6fa33f96577704af6f8422e996 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] prashantwason commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
prashantwason commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192180200 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1378,6 +1339,206 @@ public static Set getInflightAndCompletedMetadataPart

[GitHub] [hudi] prashantwason commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
prashantwason commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192169835 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1378,6 +1339,206 @@ public static Set getInflightAndCompletedMetadataPart

[GitHub] [hudi] prashantwason commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
prashantwason commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192166826 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -873,17 +908,7 @@ public void buildMetadataParti

  1   2   >