[
https://issues.apache.org/jira/browse/HDDS-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-9154:
----------------------------------
Epic Name: Optimize Ozone Snapdiff
Release Note:
HDDS-9154 introduces an optimized Snapshot Diff (SnapDiff) implementation that
reduces random RocksDB lookups and disk I/O when comparing bucket snapshots.
- Configuration: Set ozone.om.snapshot.diff.optimized.enabled=true on OM to
enable the optimization. It defaults to false.
- CLI: Adds --ordered to ozone sh snapshot diff to request dependency-ordered
reports for FSO buckets when the optimization is enabled. Completed reports
indicate whether dependency ordering was applied.
- Progress: Provides more detailed stage and percentage reporting during
optimized diff computation.
> Ozone Snapshot Phase 5.1: Optimize Snapdiff
> -------------------------------------------
>
> Key: HDDS-9154
> URL: https://issues.apache.org/jira/browse/HDDS-9154
> Project: Apache Ozone
> Issue Type: Epic
> Reporter: Swaminathan Balachandran
> Assignee: Saketa Chalamchala
> Priority: Critical
> Labels: pull-request-available
> Fix For: 2.3.0
>
>
> The current Snapshot Diff (SnapDiff) mechanism identifies changes between
> snapshots by constructing a reverse object ID mapping, a process that
> requires intensive I/O to read Delta SST files and perform cross-database
> lookups between the source and target snapshot RocksDB instances.
> To improve efficiency, the diff engine can be redesigned to compute
> differences by directly scanning the internal contents of the SST files. By
> evaluating the RocksDB sequence numbers and values embedded within the SST
> entries, the system can determine key mutations without the need for
> exhaustive mapping or external lookups.
> This transition from a mapping-centric approach to a direct metadata-driven
> analysis significantly minimizes IOPS, drastically reducing the latency and
> resource consumption of the SnapDiff computation.
>
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
