[jira] [Updated] (HUDI-1072) Reader changes to support clustering and insert overwrite
[ https://issues.apache.org/jira/browse/HUDI-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-1072: - Fix Version/s: 0.7.0 > Reader changes to support clustering and insert overwrite > - > > Key: HUDI-1072 > URL: https://issues.apache.org/jira/browse/HUDI-1072 > Project: Apache Hudi > Issue Type: Sub-task >Reporter: satish >Assignee: satish >Priority: Major > Labels: pull-request-available > Fix For: 0.7.0 > > > * Add metadata to track ‘replaced’ files. Replaced files are essentially file > groups to be ignored. For ‘insert overwrite’ this is all existing files in > the partition overwritten. For ‘clustering’, this is all file groups that are > merged into a new set of file groups. > * Change Views to ignore replaced files (AbstractTableFileSystemView and all > subclasses) > * Change cleaner to delete data files that have been replaced (Introduce a > new policy?) > * Change archival to not delete active commits that have this special > metadata if corresponding data files are not deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HUDI-1072) Reader changes to support clustering and insert overwrite
[ https://issues.apache.org/jira/browse/HUDI-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1072: - Status: In Progress (was: Open) > Reader changes to support clustering and insert overwrite > - > > Key: HUDI-1072 > URL: https://issues.apache.org/jira/browse/HUDI-1072 > Project: Apache Hudi > Issue Type: Sub-task >Reporter: satish >Assignee: satish >Priority: Major > Labels: pull-request-available > > * Add metadata to track ‘replaced’ files. Replaced files are essentially file > groups to be ignored. For ‘insert overwrite’ this is all existing files in > the partition overwritten. For ‘clustering’, this is all file groups that are > merged into a new set of file groups. > * Change Views to ignore replaced files (AbstractTableFileSystemView and all > subclasses) > * Change cleaner to delete data files that have been replaced (Introduce a > new policy?) > * Change archival to not delete active commits that have this special > metadata if corresponding data files are not deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HUDI-1072) Reader changes to support clustering and insert overwrite
[ https://issues.apache.org/jira/browse/HUDI-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1072: - Status: Open (was: New) > Reader changes to support clustering and insert overwrite > - > > Key: HUDI-1072 > URL: https://issues.apache.org/jira/browse/HUDI-1072 > Project: Apache Hudi > Issue Type: Sub-task >Reporter: satish >Assignee: satish >Priority: Major > Labels: pull-request-available > > * Add metadata to track ‘replaced’ files. Replaced files are essentially file > groups to be ignored. For ‘insert overwrite’ this is all existing files in > the partition overwritten. For ‘clustering’, this is all file groups that are > merged into a new set of file groups. > * Change Views to ignore replaced files (AbstractTableFileSystemView and all > subclasses) > * Change cleaner to delete data files that have been replaced (Introduce a > new policy?) > * Change archival to not delete active commits that have this special > metadata if corresponding data files are not deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HUDI-1072) Reader changes to support clustering and insert overwrite
[ https://issues.apache.org/jira/browse/HUDI-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1072: - Labels: pull-request-available (was: ) > Reader changes to support clustering and insert overwrite > - > > Key: HUDI-1072 > URL: https://issues.apache.org/jira/browse/HUDI-1072 > Project: Apache Hudi > Issue Type: Sub-task >Reporter: satish >Assignee: satish >Priority: Major > Labels: pull-request-available > > * Add metadata to track ‘replaced’ files. Replaced files are essentially file > groups to be ignored. For ‘insert overwrite’ this is all existing files in > the partition overwritten. For ‘clustering’, this is all file groups that are > merged into a new set of file groups. > * Change Views to ignore replaced files (AbstractTableFileSystemView and all > subclasses) > * Change cleaner to delete data files that have been replaced (Introduce a > new policy?) > * Change archival to not delete active commits that have this special > metadata if corresponding data files are not deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
