[ 
https://issues.apache.org/jira/browse/HDDS-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siyao Meng updated HDDS-16135:
------------------------------
    Attachment: HDDS-16135.001.patch

> Intermittent failure in 
> TestKeyLifecycleService$Normal#testNestedFSODirectoryScanResume
> ---------------------------------------------------------------------------------------
>
>                 Key: HDDS-16135
>                 URL: https://issues.apache.org/jira/browse/HDDS-16135
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: Ozone Manager, test
>            Reporter: Siyao Meng
>            Priority: Major
>         Attachments: HDDS-16135.001.patch
>
>
> {code}
> [ERROR] Tests run: 148, Failures: 1, Errors: 1, Skipped: 6, Time elapsed: 
> 99.18 s <<< FAILURE! -- in 
> org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal
> [ERROR] 
> org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.testNestedFSODirectoryScanResume(BucketLayout,
>  int)[2][2] -- Time elapsed: 0.336 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
>       at 
> org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.testNestedFSODirectoryScanResume(TestKeyLifecycleService.java:797)
> [ERROR] 
> org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.testNestedFSODirectoryScanResume(BucketLayout,
>  int)[2][3] -- Time elapsed: 5.580 s <<< ERROR!
> java.util.concurrent.TimeoutException:
> {code}
> The FSO directory-scan resume mechanism itself works, but the test races the 
> shared periodic {{KeyLifecycleService}} (fires every ~300ms) against a 
> one-shot fault injector. Only the first scan aborts at the intended resume 
> point; the next tick resumes and overwrites the persisted 
> {{OmLifecycleScanState}}, so the exact-value assertions at lines 785-808 
> depend on reading inside the ~300ms window (the {{[2][2]}} {{lastScannedKey}} 
> failure). {{getNumKeyIterated()}} is a service-wide cumulative counter that 
> concurrent re-scans inflate, so the exact-equality wait can never settle, 
> giving the {{[2][3]}} timeout. Same fragility family as HDDS-16033 (different 
> method). Suggested hardening: park the scan at the resume point (blocking 
> injector) for deterministic inspection, and make the metric wait monotonic 
> ({{>=}}) instead of exact.
> Seen in:
>  - 
> https://github.com/adoroszlai/ozone-build-results/blob/master/2026/08/05/51974/integration-om/hadoop-ozone/ozone-manager/org.apache.hadoop.ozone.om.service.TestKeyLifecycleService$Normal.txt



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to