[
https://issues.apache.org/jira/browse/HUDI-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo updated HUDI-9370:
------------------------------
Description:
We should unify the logic of fetching files and file slices in the metadata
table writer so the index initialization is only based on two types of
information of the file system view:
- (1) all the files in a table used by `FILES`, `BLOOM_FILTERS`, and
`COLUMN_STATS` partitions;
- (2) latest merged file slices used by `RECORD_INDEX`, `EXPRESSION_INDEX`,
`PARTITION_STATS`, and `SECONDARY_INDEX` partitions.
> Unify logic of fetching files and file slices in the metadata table writer
> --------------------------------------------------------------------------
>
> Key: HUDI-9370
> URL: https://issues.apache.org/jira/browse/HUDI-9370
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Y Ethan Guo
> Assignee: Y Ethan Guo
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> We should unify the logic of fetching files and file slices in the metadata
> table writer so the index initialization is only based on two types of
> information of the file system view:
> - (1) all the files in a table used by `FILES`, `BLOOM_FILTERS`, and
> `COLUMN_STATS` partitions;
> - (2) latest merged file slices used by `RECORD_INDEX`, `EXPRESSION_INDEX`,
> `PARTITION_STATS`, and `SECONDARY_INDEX` partitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
