[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Wason updated HUDI-3342: - Fix Version/s: 0.14.1 (was: 0.14.0) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Raymond Xu >Priority: Critical > Fix For: 0.14.1 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yue Zhang updated HUDI-3342: Fix Version/s: 0.14.0 (was: 0.13.1) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Raymond Xu >Priority: Critical > Fix For: 0.14.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaojing Yu updated HUDI-3342: -- Fix Version/s: 0.13.0 (was: 0.12.1) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Raymond Xu >Priority: Critical > Fix For: 0.13.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Sprint: (was: 2022/09/05) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Raymond Xu >Priority: Critical > Fix For: 0.12.1 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Sprint: 2022/09/05 (was: 2022/08/22) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Priority: Critical > Fix For: 0.12.1 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3342: -- Sprint: Hudi-Sprint-Mar-07, Hudi-Sprint-Mar-14, Hudi-Sprint-Mar-21, 2022/08/22 (was: Hudi-Sprint-Mar-07, Hudi-Sprint-Mar-14, Hudi-Sprint-Mar-21) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Critical > Fix For: 0.12.1 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-3342: -- Fix Version/s: 0.12.1 (was: 0.12.0) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Critical > Fix For: 0.12.1 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Fix Version/s: 0.12.0 (was: 0.11.0) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Blocker > Fix For: 0.12.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Priority: Critical (was: Blocker) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Critical > Fix For: 0.12.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Sprint: Hudi-Sprint-Mar-07, Hudi-Sprint-Mar-14, Hudi-Sprint-Mar-21 (was: Hudi-Sprint-Mar-07, Hudi-Sprint-Mar-14) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Sprint: Hudi-Sprint-Mar-07, Hudi-Sprint-Mar-14 (was: Hudi-Sprint-Mar-07) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Story Points: 2 > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Sprint: Hudi-Sprint-Mar-07 > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Assignee: Forward Xu >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3342: - Reviewers: Raymond Xu > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-3342: -- Component/s: metadata > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug > Components: metadata >Reporter: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3342: -- Description: While working on HUDI-3322, i've spotted following contraption: When we are rolling back Delta Commits, we add corresponding {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we restore, we issue a sequence of Rollbacks, which means that stack if such Rollback Blocks could be of size > 1. However, when reading that MOR table if the reader does not specify `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants increment) therefore essentially rendering such Rollback Blocks useless since they can't "unmerge" previously merged records, resurrecting the data that was supposed to be rolled back. was: While working on HUDI-3322, i've spotted following contraption: When we are rolling back Delta Commits, we add corresponding {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". However, when reading that MOR table if the reader does not specify `readBlocksLazily=true`, we'd be merging Blocks eagerly therefore essentially rendering such Rollback Blocks useless, resurrecting the data that was previously rolled back. > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we > restore, we issue a sequence of Rollbacks, which means that stack if such > Rollback Blocks could be of size > 1. > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants > increment) therefore essentially rendering such Rollback Blocks useless since > they can't "unmerge" previously merged records, resurrecting the data that > was supposed to be rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3342) MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
[ https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-3342: -- Summary: MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled (was: MOR Delta Block rollbacks is not working correctly if lazy Block reading is disabled) > MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled > --- > > Key: HUDI-3342 > URL: https://issues.apache.org/jira/browse/HUDI-3342 > Project: Apache Hudi > Issue Type: Bug >Reporter: Alexey Kudinkin >Priority: Blocker > Fix For: 0.11.0 > > > While working on HUDI-3322, i've spotted following contraption: > When we are rolling back Delta Commits, we add corresponding > {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". > However, when reading that MOR table if the reader does not specify > `readBlocksLazily=true`, we'd be merging Blocks eagerly therefore essentially > rendering such Rollback Blocks useless, resurrecting the data that was > previously rolled back. -- This message was sent by Atlassian Jira (v8.20.1#820001)