[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5434: -- Fix Version/s: 0.12.3 > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0, 0.12.3 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. > > Impact: > could result in huge no of entries in active timeline in MDT. might hamper > perf or throttling in cloud stores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5434: -- Reviewers: sivabalan narayanan > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. > > Impact: > could result in huge no of entries in active timeline in MDT. might hamper > perf or throttling in cloud stores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5434: - Sprint: 0.13.0 Final Sprint, 0.13.0 Final Sprint 2 (was: 0.13.0 Final Sprint) > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. > > Impact: > could result in huge no of entries in active timeline in MDT. might hamper > perf or throttling in cloud stores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5434: Status: Patch Available (was: In Progress) > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. > > Impact: > could result in huge no of entries in active timeline in MDT. might hamper > perf or throttling in cloud stores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5434: - Labels: pull-request-available (was: ) > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Labels: pull-request-available > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. > > Impact: > could result in huge no of entries in active timeline in MDT. might hamper > perf or throttling in cloud stores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5434: -- Description: as of now, archival in MDT is guarded until first entry in DT's active timeline. but DT could contain rollback that could date back few days or even weeks. So, we need to fix that to check for first write action in DT (commit, delta commit, replace commit) and then guard MDT archival based on that. Impact: could result in huge no of entries in active timeline in MDT. might hamper perf or throttling in cloud stores. was:as of now, archival in MDT is guarded until first entry in DT's active timeline. but DT could contain rollback that could date back few days or even weeks. So, we need to fix that to check for first write action in DT (commit, delta commit, replace commit) and then guard MDT archival based on that. > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. > > Impact: > could result in huge no of entries in active timeline in MDT. might hamper > perf or throttling in cloud stores. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5434: Status: In Progress (was: Open) > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Assignee: Ethan Guo >Priority: Blocker > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5434: -- Sprint: 0.13.0 Final Sprint > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5434: -- Fix Version/s: 0.13.0 > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5434) Fix archival in MDT to not rely on rollbacks/clean in DT
[ https://issues.apache.org/jira/browse/HUDI-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5434: -- Epic Link: HUDI-1292 Story Points: 2 > Fix archival in MDT to not rely on rollbacks/clean in DT > > > Key: HUDI-5434 > URL: https://issues.apache.org/jira/browse/HUDI-5434 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: sivabalan narayanan >Priority: Blocker > Fix For: 0.13.0 > > > as of now, archival in MDT is guarded until first entry in DT's active > timeline. but DT could contain rollback that could date back few days or even > weeks. So, we need to fix that to check for first write action in DT (commit, > delta commit, replace commit) and then guard MDT archival based on that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
