[
https://issues.apache.org/jira/browse/HDDS-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-15474:
------------------------------------
Fix Version/s: 2.2.0
Resolution: Implemented
Status: Resolved (was: Patch Available)
> Further speed up TestOmSnapshot by reducing config-matrix redundancy
> --------------------------------------------------------------------
>
> Key: HDDS-15474
> URL: https://issues.apache.org/jira/browse/HDDS-15474
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: test
> Reporter: Chi-Hsuan Huang
> Assignee: Chi-Hsuan Huang
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.2.0
>
>
> Follow-up to HDDS-10308. {{TestOmSnapshot}} is an abstract base with ~73
> tests; each of the 8 concrete subclasses starts its own MiniOzoneCluster and
> re-runs the whole suite under one config combination {{(bucketLayout,
> disableNativeDiff, createLinkedBucket, ...)}}. The subclasses contain no test
> methods of their own, only a constructor passing the config.
> Code analysis showed that of the three varied dimensions, only
> {{bucketLayout}} changes observable output ({{SnapshotDiffManager}} branches
> on {{isFileSystemOptimized()}}). {{disableNativeDiff}} changes only the diff
> mechanism (identical reports), and {{createLinkedBucket}} changes only input
> resolution: every snapshot operation resolves the bucket link up-front
> (verified at all entry points: create/delete/rename snapshot, listSnapshot,
> getSnapshotInfo, snapshotDiff/submit/cancel/listJobs, and key reads via
> {{.snapshot/}}), before any layout-specific logic runs. The four
> {{...WithLinkedBuckets}} subclasses, added by HDDS-11705 to verify that
> resolution, therefore re-prove the same behaviour once per layout/native
> setting.
> Change (PR #10432): keep one full linked run and drop the redundant copies.
> * Deleted: {{TestOmSnapshotObjectStoreWithLinkedBuckets}},
> {{TestOmSnapshotFsoWithNativeLibWithLinkedBuckets}},
> {{TestOmSnapshotWithBucketLinkingLegacy}}.
> * Kept: {{TestOmSnapshotFsoWithoutNativeLibWithLinkedBuckets}} still runs the
> entire suite through linked buckets, so the HDDS-11705 contract stays fully
> exercised. Native-off is chosen so this coverage also runs locally without
> the RocksDB native lib.
> * No test method is removed; the matrix shrinks from 8 to 5 combinations,
> saving ~3 full MiniOzoneCluster suite runs. Measured on the PR's CI: the
> {{integration (snapshot)}} job dropped from 2861s (same-day master run) to
> 2146s, ~12 min / ~25% faster; the {{TestOmSnapshot}} matrix dropped from
> ~2046.8s (8 classes) to 1309.7s (5 classes), all green with the expected skip
> pattern.
> * Tradeoff: OBJECT_STORE and LEGACY no longer run through linked buckets; FSO
> carries linked coverage alone. Per-layout diff output remains covered by the
> four plain subclasses.
> Original proposal (superseded): classify each of the ~70 tests as
> config-sensitive vs config-independent and pin the config-independent ones to
> a single subclass via {{assumeTrue}}. The dimensional collapse above captures
> most of the win with a much smaller, reviewable change. Per-test pinning, and
> dropping the never-varied constructor params ({{enableFileSystemPaths}} and
> {{forceFullSnapshotDiff}} are {{false}} in all subclasses), remain possible
> follow-ups if profiling justifies them.
> Relates to HDDS-10308.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
