[
https://issues.apache.org/jira/browse/HDDS-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Rose updated HDDS-14686:
------------------------------
Summary: Open Key tests in TestOzoneShellHA depend on state from other
tests (was: HSync tests in TestOzoneShellHA depend on state from other tests)
> Open Key tests in TestOzoneShellHA depend on state from other tests
> -------------------------------------------------------------------
>
> Key: HDDS-14686
> URL: https://issues.apache.org/jira/browse/HDDS-14686
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ethan Rose
> Assignee: Ashish Kumar
> Priority: Major
>
> In HDDS-14580, a new test {{testPrepareCommandNoOp}} was added to
> {{{}TestOzoneShellHA{}}}. JUnit runs these tests in the following order:
> # {{testPrepareCommandNoOp}}
> # (other tests)
> # {{testAdminCmdListOpenFilesWithDeletedKeys}}
> # (other tests)
> # {{testAdminCmdListOpenFiles}}
> When this happens, the two open file tests start failing. In particular,
> {{testAdminCmdListOpenFilesWithDeletedKeys}} fails with the following error:
> {code:java}
> 2026-02-20 13:42:56,595 [ForkJoinPool-1-worker-9] INFO
> shell.TestOzoneShellHA (TestOzoneShellHA.java:execute(235)) - Executing
> OzoneShell command with args [om, lof, --service-id, om-service-test1,
> --show-deleted, -p, /volume-list-del/buck1]
> org.opentest4j.AssertionFailedError: key should be hsync'ed and not deleted
> ==>
> Expected :true
> Actual :false
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63)
> at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36)
> at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:214)
> at
> org.apache.hadoop.ozone.shell.TestOzoneShellHA.testAdminCmdListOpenFilesWithDeletedKeys(TestOzoneShellHA.java:748)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:289)
> {code}
> When the test was passing previously, the assert read the following output:
> {code:java}
> 5 total open files. Showing 5 open files (limit 100) under path prefix:
> /volume-list-del/buck1
> Client ID Creation time Hsync'ed Deleted
> Open File Path
> 116099515204632578 2026-02-19T21:53:32.721Z No No
> /volume-list-del/buck1/-9223372036854772991/key0
> 116099515213545475 2026-02-19T21:53:32.902Z No No
> /volume-list-del/buck1/-9223372036854772991/key1
> 116099515214462980 2026-02-19T21:53:32.916Z No No
> /volume-list-del/buck1/-9223372036854772991/key2
> 116099515214856197 2026-02-19T21:53:32.922Z No No
> /volume-list-del/buck1/-9223372036854772991/key3
> 116099515217346566 2026-02-19T21:53:32.960Z Yes No
> /volume-list-del/buck1/-9223372036854772991/key4
> Reached the end of the list.
> {code}
> When the test failed, the assert read the following output:
> {code:java}
> 6 total open files. Showing 5 open files (limit 100) under path prefix:
> /volume-list-del/buck1
> Client ID Creation time Hsync'ed Deleted
> Open File Path
> 116099528644428109 2026-02-19T21:56:48.999Z No No
> /volume-list-del/buck1/-9223372036854716927/key0
> 116099528644690254 2026-02-19T21:56:49.099Z No No
> /volume-list-del/buck1/-9223372036854716927/key1
> 116099528644952399 2026-02-19T21:56:49.105Z No No
> /volume-list-del/buck1/-9223372036854716927/key2
> 116099528645214544 2026-02-19T21:56:49.111Z No No
> /volume-list-del/buck1/-9223372036854716927/key3
> 116099528645411153 2026-02-19T21:56:49.115Z No No
> /volume-list-del/buck1/-9223372036854716927/key4
> Reached the end of the list.
> {code}
> That 6th key showing up has somehow been overwritten within the test in the
> failure case, because adding {{--show-overwritten}} to the command when the
> test is failing shows
> {code:java}
> 6 total open files. Showing 6 open files (limit 100) under path prefix:
> /
> Client ID Creation time Hsync'ed Deleted
> Overwritten Open File Path
> 116099583048352077 2026-02-19T22:10:37.365Z No No
> No /volume-list-del/buck1/-9223372036854716927/key0
> 116099583048810830 2026-02-19T22:10:37.481Z No No
> No /volume-list-del/buck1/-9223372036854716927/key1
> 116099583049400655 2026-02-19T22:10:37.487Z No No
> No /volume-list-del/buck1/-9223372036854716927/key2
> 116099583050121552 2026-02-19T22:10:37.492Z No No
> No /volume-list-del/buck1/-9223372036854716927/key3
> 116099582361469038 2026-02-19T22:10:37.498Z Yes No
> Yes /volume-list-del/buck1/-9223372036854716927/key4
> 116099583050514769 2026-02-19T22:10:37.498Z No No
> No /volume-list-del/buck1/-9223372036854716927/key4
> Reached the end of the list.
> {code}
> Based on the volume name we can see that this is not state introduced by the
> prepare test. If only those three tests are run in the same order without the
> other tests in the class, they all pass. This also seems to indicate that the
> issue is not with the prepare test, but with assumptions that the open key
> tests are making about state from previous tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
