[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Epic Link: HUDI-1292 > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3343: -- Sprint: Hudi-Sprint-Jan-24, Hudi-Sprint-Jan-31 (was: Hudi-Sprint-Jan-24) > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Story Points: 0 (was: 2) > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3343: -- Component/s: metadata > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Status: Patch Available (was: In Progress) > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Reviewers: sivabalan narayanan, Y Ethan Guo > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Status: In Progress (was: Open) > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Story Points: 2 > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Sprint: Hudi-Sprint-Jan-24 > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Assignee: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Fix Version/s: 0.11.0 > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3343) Metadata Table includes Uncommitted Log Files during Bootstrap
[ https://issues.apache.org/jira/browse/HUDI-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3343: -- Priority: Blocker (was: Major) > Metadata Table includes Uncommitted Log Files during Bootstrap > -- > > Key: HUDI-3343 > URL: https://issues.apache.org/jira/browse/HUDI-3343 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Priority: Blocker > > While working on a fix for HUDI-3322, discovered a following issue: > If we're bootstrapping the MT during pending Rollback operation (this could > happen when previous writer had MT *disabled* when writing the data), since > bootstrapping is done _after_ Rollback is executed (with its side-effects > already being reflected on FS) bootstrapping would incorrectly include > intermediary files created by the Rollback (like log-files being created with > Rollback Command Block appended). > > Filtering of the files is performed here: > https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java#L752 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
