[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-13 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1227895765 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieMetadataBulkInsertPartitioner.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-10 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1225702442 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieMetadataBulkInsertPartitioner.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-10 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1225695785 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieMetadataBulkInsertPartitioner.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-08 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1223719557 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -1052,51 +1091,81 @@ protected HoodieData prepRecor

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-08 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1222772904 ## hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/metadata/FlinkHoodieBackedTableMetadataWriter.java: ## @@ -104,40 +106,19 @@ protected void initRegistry()

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-08 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1222548105 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1453,7 +1453,11 @@ public static String deleteMetadataTablePartition(HoodieTa

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-07 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1222533305 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1453,7 +1453,11 @@ public static String deleteMetadataTablePartition(HoodieTa

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-06-06 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1220774838 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -1453,7 +1453,11 @@ public static String deleteMetadataTablePartition(HoodieTa

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-24 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1203914496 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java: ## @@ -338,14 +338,20 @@ protected void initMetadataTable(Option instant

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-24 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1203913275 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java: ## @@ -338,14 +338,20 @@ protected void initMetadataTable(Option instant

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-24 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1203785303 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -564,53 +532,147 @@ private boolean isCommitRevert

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-23 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1194736640 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java: ## @@ -337,15 +337,33 @@ protected void initMetadataTable(Option instant

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-16 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1194731645 ## hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java: ## @@ -694,17 +695,75 @@ private Long getTableChecksum() { return getLong(TABLE_CH

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192914114 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -824,25 +863,22 @@ private interface ConvertMetadata

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192914114 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -824,25 +863,22 @@ private interface ConvertMetadata

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-12 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1192913868 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieBackedTableMetadataWriter.java: ## @@ -118,46 +123,32 @@ protected void initRegistry()

[GitHub] [hudi] danny0405 commented on a diff in pull request #8684: [HUDI-6200] Enhancements to the MDT for improving performance of larger indexes.

2023-05-11 Thread via GitHub
danny0405 commented on code in PR #8684: URL: https://github.com/apache/hudi/pull/8684#discussion_r1190914223 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -619,41 +680,34 @@ private boolean anyPendingDataIn