[jira] [Commented] (HDDS-15125) MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
[
https://issues.apache.org/jira/browse/HDDS-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18087188#comment-18087188
]
Hui Fei commented on HDDS-15125:
<>
Apache Ozone 2.2.0 release is in progress. I'm updating all unresolved jiras
targeting 2.2.0 to retarget 2.3.0.
> MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
> --
>
> Key: HDDS-15125
> URL: https://issues.apache.org/jira/browse/HDDS-15125
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM, Snapshot
>Reporter: Chu Cheng Li
>Assignee: Chu Cheng Li
>Priority: Major
>
> h2. Summary
> Refactor snapshot key reclamation so newly written deleted-key versions can
> be evaluated by MVCC-style visibility intervals instead of opening previous
> snapshot RocksDB instances and comparing key/block state.
> A key version is visible to a snapshot when:
> {code:java}
> seqNumMin <= snapshotCreateSeqNum < seqNumMax
> {code}
> A deleted key version is reclaimable when no active snapshot for that bucket
> has a create sequence number inside that interval.
> This is a phase-1 optimization for {{deletedTable}} key versions only.
> Existing snapshot deletion movement remains unchanged, and legacy entries
> without sequence interval metadata continue using the current
> {{ReclaimableKeyFilter}} fallback.
> h2. Schema And Data Model
> ||Table||Value Type||Change||
> |{{snapshotInfoTable}}|{{SnapshotInfo}}|No new field. Use existing
> {{createTransactionInfo}} as the snapshot create sequence source. Do not
> reuse deprecated {{{}dbTxSequenceNumber{}}}.|
> |{{keyTable}} / {{fileTable}}|{{KeyInfo}} via {{OmKeyInfo}}|Add optional
> {{{}uint64 seqNumMin = 24{}}}. Active key versions do not set
> {{{}seqNumMax{}}}.|
> |{{deletedTable}}|{{RepeatedKeyInfo}} containing {{KeyInfo}}|Store
> {{seqNumMin}} and {{seqNumMax}} on each contained {{{}KeyInfo{}}}. Add
> optional {{{}uint64 seqNumMax = 25{}}}.|
> |{{deletedDirTable}}|{{OmKeyInfo}}|No v1 schema behavior change beyond fields
> naturally existing on {{{}KeyInfo{}}}; directory interval optimization is out
> of v1.|
> |{{snapshotRenamedTable}}|{{String}}|No change in v1. Keep current
> previous-snapshot lookup logic for rename entries.|
> Add nullable boxed fields to {{OmKeyInfo}} and its builder:
> {code:java}
> private Long seqNumMin;
> private Long seqNumMax;
> {code}
> Serialize only when non-null. Missing fields mean “not eligible for interval
> optimization”.
> h2. Fill Rules
> Use OM/Ratis transaction index as the sequence number.
> For active key versions:
> * On new committed key creation, set {{{}seqNumMin =
> currentTransactionIndex{}}}.
> * On overwrite or data-changing commit, set the new key version’s
> {{{}seqNumMin = currentTransactionIndex{}}}.
> * On metadata-only operations such as ACL changes, mtime changes, and
> rename, preserve existing {{{}seqNumMin{}}}.
> * If an existing legacy key has no {{{}seqNumMin{}}}, do not infer one from
> {{{}updateID{}}}; keep it missing so deletion falls back to the old logic.
> For deleted key versions:
> * Centralize deleted-key preparation in
> {{{}OmUtils.prepareKeyForDelete(...){}}}.
> * Preserve the source key’s {{{}seqNumMin{}}}.
> * Set {{{}seqNumMax = currentTransactionIndex{}}}, where the current
> transaction is the delete or overwrite transaction that makes this version no
> longer visible.
> * For pseudo/uncommitted block reclaim records that were never visible to
> snapshots, set {{seqNumMin == seqNumMax == currentTransactionIndex}} or keep
> the existing unconditional reclaim marker path. The interval must be empty.
> * When {{SnapshotDeletingService}} moves deleted entries from a deleted
> snapshot to the next active snapshot or AOS, preserve {{seqNumMin}} and
> {{seqNumMax}} unchanged.
> For snapshots:
> * Continue setting {{SnapshotInfo.createTransactionInfo}} in snapshot create.
> * Extract the transaction index from {{createTransactionInfo}} for active
> snapshot visibility checks.
> * If any active snapshot for a bucket lacks {{{}createTransactionInfo{}}},
> disable interval optimization for that bucket and use the current fallback.
> h2. Reclaim Algorithm
> Build an in-memory sorted {{long[]}} of active snapshot create sequence
> numbers per snapshot path:
> {code:java}
> /volume/bucket -> sorted active snapshot create seq nums
> {code}
> Include only {{SNAPSHOT_ACTIVE}} snapshots. {{SNAPSHOT_DELETED}} snapshots no
> longer protect user visibility.
> For each deleted key version with both fields present:
> {code:java}
> int pos = lowerBound(activeSnapshotSeqNums, seqNumMin);
> boolean referenced = pos < activeSnapshotSeqNums.length
> && activeSnapshotSeqNums[pos] < seqNumMax;
> boolean reclaimable = !referenced;
> {code}
> If {{seqNumM
[jira] [Commented] (HDDS-15125) MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
[
https://issues.apache.org/jira/browse/HDDS-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18076672#comment-18076672
]
Wei-Chiu Chuang commented on HDDS-15125:
I think this proposal has its own merit: opening/closing RocksDB instances has
a visible overhead, usually on the order of hundred milliseconds to 0.5
seconds. Though we cache Snapshot RocksDB instances so the impact shouldn't be
that terrible.
But IMO the thing is the key deletion happens in the background, so unless it
becomes so terrible it blocks foreground traffic, it's unlikely to cause
visible impact. Right now, we are prioritizing the efficient Snapshot Diff
project which has a great impact to cluster replication SLAs.
> MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
> --
>
> Key: HDDS-15125
> URL: https://issues.apache.org/jira/browse/HDDS-15125
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM, Snapshot
>Reporter: Chu Cheng Li
>Assignee: Chu Cheng Li
>Priority: Major
>
> h2. Summary
> Refactor snapshot key reclamation so newly written deleted-key versions can
> be evaluated by MVCC-style visibility intervals instead of opening previous
> snapshot RocksDB instances and comparing key/block state.
> A key version is visible to a snapshot when:
> {code:java}
> seqNumMin <= snapshotCreateSeqNum < seqNumMax
> {code}
> A deleted key version is reclaimable when no active snapshot for that bucket
> has a create sequence number inside that interval.
> This is a phase-1 optimization for {{deletedTable}} key versions only.
> Existing snapshot deletion movement remains unchanged, and legacy entries
> without sequence interval metadata continue using the current
> {{ReclaimableKeyFilter}} fallback.
> h2. Schema And Data Model
> ||Table||Value Type||Change||
> |{{snapshotInfoTable}}|{{SnapshotInfo}}|No new field. Use existing
> {{createTransactionInfo}} as the snapshot create sequence source. Do not
> reuse deprecated {{{}dbTxSequenceNumber{}}}.|
> |{{keyTable}} / {{fileTable}}|{{KeyInfo}} via {{OmKeyInfo}}|Add optional
> {{{}uint64 seqNumMin = 24{}}}. Active key versions do not set
> {{{}seqNumMax{}}}.|
> |{{deletedTable}}|{{RepeatedKeyInfo}} containing {{KeyInfo}}|Store
> {{seqNumMin}} and {{seqNumMax}} on each contained {{{}KeyInfo{}}}. Add
> optional {{{}uint64 seqNumMax = 25{}}}.|
> |{{deletedDirTable}}|{{OmKeyInfo}}|No v1 schema behavior change beyond fields
> naturally existing on {{{}KeyInfo{}}}; directory interval optimization is out
> of v1.|
> |{{snapshotRenamedTable}}|{{String}}|No change in v1. Keep current
> previous-snapshot lookup logic for rename entries.|
> Add nullable boxed fields to {{OmKeyInfo}} and its builder:
> {code:java}
> private Long seqNumMin;
> private Long seqNumMax;
> {code}
> Serialize only when non-null. Missing fields mean “not eligible for interval
> optimization”.
> h2. Fill Rules
> Use OM/Ratis transaction index as the sequence number.
> For active key versions:
> * On new committed key creation, set {{{}seqNumMin =
> currentTransactionIndex{}}}.
> * On overwrite or data-changing commit, set the new key version’s
> {{{}seqNumMin = currentTransactionIndex{}}}.
> * On metadata-only operations such as ACL changes, mtime changes, and
> rename, preserve existing {{{}seqNumMin{}}}.
> * If an existing legacy key has no {{{}seqNumMin{}}}, do not infer one from
> {{{}updateID{}}}; keep it missing so deletion falls back to the old logic.
> For deleted key versions:
> * Centralize deleted-key preparation in
> {{{}OmUtils.prepareKeyForDelete(...){}}}.
> * Preserve the source key’s {{{}seqNumMin{}}}.
> * Set {{{}seqNumMax = currentTransactionIndex{}}}, where the current
> transaction is the delete or overwrite transaction that makes this version no
> longer visible.
> * For pseudo/uncommitted block reclaim records that were never visible to
> snapshots, set {{seqNumMin == seqNumMax == currentTransactionIndex}} or keep
> the existing unconditional reclaim marker path. The interval must be empty.
> * When {{SnapshotDeletingService}} moves deleted entries from a deleted
> snapshot to the next active snapshot or AOS, preserve {{seqNumMin}} and
> {{seqNumMax}} unchanged.
> For snapshots:
> * Continue setting {{SnapshotInfo.createTransactionInfo}} in snapshot create.
> * Extract the transaction index from {{createTransactionInfo}} for active
> snapshot visibility checks.
> * If any active snapshot for a bucket lacks {{{}createTransactionInfo{}}},
> disable interval optimization for that bucket and use the current fallback.
> h2. Reclaim Algorithm
> Build an in-memory sorted {{long[]}} of active snapshot create sequence
> numbers per snapshot path:
> {code:java}
> /volume/bucket -> sorted active snapsh
[jira] [Commented] (HDDS-15125) MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
[
https://issues.apache.org/jira/browse/HDDS-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18076669#comment-18076669
]
Wei-Chiu Chuang commented on HDDS-15125:
the gist of it is: make "can we reclaim this deleted key version’s blocks?”
decidable from OM transaction sequence intervals + a sorted list of snapshot
create seqs, instead of repeatedly consulting prior snapshot storage, while
falling back to today’s behavior when metadata is missing or optimization is
turned off."
> MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
> --
>
> Key: HDDS-15125
> URL: https://issues.apache.org/jira/browse/HDDS-15125
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM, Snapshot
>Reporter: Chu Cheng Li
>Assignee: Chu Cheng Li
>Priority: Major
>
> h2. Summary
> Refactor snapshot key reclamation so newly written deleted-key versions can
> be evaluated by MVCC-style visibility intervals instead of opening previous
> snapshot RocksDB instances and comparing key/block state.
> A key version is visible to a snapshot when:
> {code:java}
> seqNumMin <= snapshotCreateSeqNum < seqNumMax
> {code}
> A deleted key version is reclaimable when no active snapshot for that bucket
> has a create sequence number inside that interval.
> This is a phase-1 optimization for {{deletedTable}} key versions only.
> Existing snapshot deletion movement remains unchanged, and legacy entries
> without sequence interval metadata continue using the current
> {{ReclaimableKeyFilter}} fallback.
> h2. Schema And Data Model
> ||Table||Value Type||Change||
> |{{snapshotInfoTable}}|{{SnapshotInfo}}|No new field. Use existing
> {{createTransactionInfo}} as the snapshot create sequence source. Do not
> reuse deprecated {{{}dbTxSequenceNumber{}}}.|
> |{{keyTable}} / {{fileTable}}|{{KeyInfo}} via {{OmKeyInfo}}|Add optional
> {{{}uint64 seqNumMin = 24{}}}. Active key versions do not set
> {{{}seqNumMax{}}}.|
> |{{deletedTable}}|{{RepeatedKeyInfo}} containing {{KeyInfo}}|Store
> {{seqNumMin}} and {{seqNumMax}} on each contained {{{}KeyInfo{}}}. Add
> optional {{{}uint64 seqNumMax = 25{}}}.|
> |{{deletedDirTable}}|{{OmKeyInfo}}|No v1 schema behavior change beyond fields
> naturally existing on {{{}KeyInfo{}}}; directory interval optimization is out
> of v1.|
> |{{snapshotRenamedTable}}|{{String}}|No change in v1. Keep current
> previous-snapshot lookup logic for rename entries.|
> Add nullable boxed fields to {{OmKeyInfo}} and its builder:
> {code:java}
> private Long seqNumMin;
> private Long seqNumMax;
> {code}
> Serialize only when non-null. Missing fields mean “not eligible for interval
> optimization”.
> h2. Fill Rules
> Use OM/Ratis transaction index as the sequence number.
> For active key versions:
> * On new committed key creation, set {{{}seqNumMin =
> currentTransactionIndex{}}}.
> * On overwrite or data-changing commit, set the new key version’s
> {{{}seqNumMin = currentTransactionIndex{}}}.
> * On metadata-only operations such as ACL changes, mtime changes, and
> rename, preserve existing {{{}seqNumMin{}}}.
> * If an existing legacy key has no {{{}seqNumMin{}}}, do not infer one from
> {{{}updateID{}}}; keep it missing so deletion falls back to the old logic.
> For deleted key versions:
> * Centralize deleted-key preparation in
> {{{}OmUtils.prepareKeyForDelete(...){}}}.
> * Preserve the source key’s {{{}seqNumMin{}}}.
> * Set {{{}seqNumMax = currentTransactionIndex{}}}, where the current
> transaction is the delete or overwrite transaction that makes this version no
> longer visible.
> * For pseudo/uncommitted block reclaim records that were never visible to
> snapshots, set {{seqNumMin == seqNumMax == currentTransactionIndex}} or keep
> the existing unconditional reclaim marker path. The interval must be empty.
> * When {{SnapshotDeletingService}} moves deleted entries from a deleted
> snapshot to the next active snapshot or AOS, preserve {{seqNumMin}} and
> {{seqNumMax}} unchanged.
> For snapshots:
> * Continue setting {{SnapshotInfo.createTransactionInfo}} in snapshot create.
> * Extract the transaction index from {{createTransactionInfo}} for active
> snapshot visibility checks.
> * If any active snapshot for a bucket lacks {{{}createTransactionInfo{}}},
> disable interval optimization for that bucket and use the current fallback.
> h2. Reclaim Algorithm
> Build an in-memory sorted {{long[]}} of active snapshot create sequence
> numbers per snapshot path:
> {code:java}
> /volume/bucket -> sorted active snapshot create seq nums
> {code}
> Include only {{SNAPSHOT_ACTIVE}} snapshots. {{SNAPSHOT_DELETED}} snapshots no
> longer protect user visibility.
> For each deleted key version with both fields present:
> {code:java}
> i
[jira] [Commented] (HDDS-15125) MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
[
https://issues.apache.org/jira/browse/HDDS-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18076600#comment-18076600
]
Wei-Chiu Chuang commented on HDDS-15125:
cc [~smeng]
> MVCC-Style Snapshot Reclaimability Using seqNumMin / seqNumMax
> --
>
> Key: HDDS-15125
> URL: https://issues.apache.org/jira/browse/HDDS-15125
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM, Snapshot
>Reporter: Chu Cheng Li
>Assignee: Chu Cheng Li
>Priority: Major
>
> h2. Summary
> Refactor snapshot key reclamation so newly written deleted-key versions can
> be evaluated by MVCC-style visibility intervals instead of opening previous
> snapshot RocksDB instances and comparing key/block state.
> A key version is visible to a snapshot when:
> {code:java}
> seqNumMin <= snapshotCreateSeqNum < seqNumMax
> {code}
> A deleted key version is reclaimable when no active snapshot for that bucket
> has a create sequence number inside that interval.
> This is a phase-1 optimization for {{deletedTable}} key versions only.
> Existing snapshot deletion movement remains unchanged, and legacy entries
> without sequence interval metadata continue using the current
> {{ReclaimableKeyFilter}} fallback.
> h2. Schema And Data Model
> ||Table||Value Type||Change||
> |{{snapshotInfoTable}}|{{SnapshotInfo}}|No new field. Use existing
> {{createTransactionInfo}} as the snapshot create sequence source. Do not
> reuse deprecated {{{}dbTxSequenceNumber{}}}.|
> |{{keyTable}} / {{fileTable}}|{{KeyInfo}} via {{OmKeyInfo}}|Add optional
> {{{}uint64 seqNumMin = 24{}}}. Active key versions do not set
> {{{}seqNumMax{}}}.|
> |{{deletedTable}}|{{RepeatedKeyInfo}} containing {{KeyInfo}}|Store
> {{seqNumMin}} and {{seqNumMax}} on each contained {{{}KeyInfo{}}}. Add
> optional {{{}uint64 seqNumMax = 25{}}}.|
> |{{deletedDirTable}}|{{OmKeyInfo}}|No v1 schema behavior change beyond fields
> naturally existing on {{{}KeyInfo{}}}; directory interval optimization is out
> of v1.|
> |{{snapshotRenamedTable}}|{{String}}|No change in v1. Keep current
> previous-snapshot lookup logic for rename entries.|
> Add nullable boxed fields to {{OmKeyInfo}} and its builder:
> {code:java}
> private Long seqNumMin;
> private Long seqNumMax;
> {code}
> Serialize only when non-null. Missing fields mean “not eligible for interval
> optimization”.
> h2. Fill Rules
> Use OM/Ratis transaction index as the sequence number.
> For active key versions:
> * On new committed key creation, set {{{}seqNumMin =
> currentTransactionIndex{}}}.
> * On overwrite or data-changing commit, set the new key version’s
> {{{}seqNumMin = currentTransactionIndex{}}}.
> * On metadata-only operations such as ACL changes, mtime changes, and
> rename, preserve existing {{{}seqNumMin{}}}.
> * If an existing legacy key has no {{{}seqNumMin{}}}, do not infer one from
> {{{}updateID{}}}; keep it missing so deletion falls back to the old logic.
> For deleted key versions:
> * Centralize deleted-key preparation in
> {{{}OmUtils.prepareKeyForDelete(...){}}}.
> * Preserve the source key’s {{{}seqNumMin{}}}.
> * Set {{{}seqNumMax = currentTransactionIndex{}}}, where the current
> transaction is the delete or overwrite transaction that makes this version no
> longer visible.
> * For pseudo/uncommitted block reclaim records that were never visible to
> snapshots, set {{seqNumMin == seqNumMax == currentTransactionIndex}} or keep
> the existing unconditional reclaim marker path. The interval must be empty.
> * When {{SnapshotDeletingService}} moves deleted entries from a deleted
> snapshot to the next active snapshot or AOS, preserve {{seqNumMin}} and
> {{seqNumMax}} unchanged.
> For snapshots:
> * Continue setting {{SnapshotInfo.createTransactionInfo}} in snapshot create.
> * Extract the transaction index from {{createTransactionInfo}} for active
> snapshot visibility checks.
> * If any active snapshot for a bucket lacks {{{}createTransactionInfo{}}},
> disable interval optimization for that bucket and use the current fallback.
> h2. Reclaim Algorithm
> Build an in-memory sorted {{long[]}} of active snapshot create sequence
> numbers per snapshot path:
> {code:java}
> /volume/bucket -> sorted active snapshot create seq nums
> {code}
> Include only {{SNAPSHOT_ACTIVE}} snapshots. {{SNAPSHOT_DELETED}} snapshots no
> longer protect user visibility.
> For each deleted key version with both fields present:
> {code:java}
> int pos = lowerBound(activeSnapshotSeqNums, seqNumMin);
> boolean referenced = pos < activeSnapshotSeqNums.length
> && activeSnapshotSeqNums[pos] < seqNumMax;
> boolean reclaimable = !referenced;
> {code}
> If {{seqNumMin}} or {{seqNumMax}} is missing, or the bucket’s active snapshot
> sequence array cann
