[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1228452051 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java: ## @@ -234,6 +235,55 @@ public final class HoodieMetadataConfig extends

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225644689 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java: ## @@ -193,121 +190,126 @@ public HoodieData> getRecordsByKeyPrefixes(L

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225642806 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java: ## @@ -163,16 +160,16 @@ public List getPartitionPathWithPathPrefixes(List

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225642468 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java: ## @@ -163,16 +160,16 @@ public List getPartitionPathWithPathPrefixes(List

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225634500 ## hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java: ## @@ -143,46 +126,33 @@ public List getAllPartitionPaths() throws IOException { *

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225625662 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/SparkMetadataTableRecordIndex.java: ## @@ -0,0 +1,253 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225598968 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/SparkMetadataTableRecordIndex.java: ## @@ -0,0 +1,253 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225594921 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java: ## @@ -123,6 +124,9 @@ public static HoodieWriteConfig

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225585604 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -432,6 +468,7 @@ private boolean

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225584303 ## hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieAvroHFileReader.java: ## @@ -545,6 +545,33 @@ public void close() { } } + @Override + public

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225582669 ## hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java: ## @@ -234,6 +235,55 @@ public final class HoodieMetadataConfig extends

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225581002 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java: ## @@ -390,6 +403,18 @@ public HoodieMetadataPayload

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225580315 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/SparkMetadataTableRecordIndex.java: ## @@ -0,0 +1,227 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225571336 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java: ## @@ -170,6 +171,17 @@ public static HoodieWriteConfig

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225568266 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -430,53 +395,212 @@ private boolean

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225553185 ## hudi-client/hudi-client-common/src/test/java/org/apache/hudi/common/testutils/HoodieMetadataTestTable.java: ## @@ -77,7 +79,8 @@ public HoodieCommitMetadata

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225549928 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -147,84 +156,55 @@ protected

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1225438663 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -432,6 +468,7 @@ private boolean

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213939512 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieMetadataBulkInsertPartitioner.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213863164 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java: ## @@ -351,6 +347,7 @@ public void run() {

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213814148 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -111,18 +111,27 @@ public abstract class

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1213649019 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataWriter.java: ## @@ -41,35 +44,23 @@ public interface

[GitHub] [hudi] nsivabalan commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

nsivabalan commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1198316621 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/SparkMetadataTableRecordIndex.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Software