Siyao Meng created HDDS-16165:
---------------------------------
Summary: Intermittent failure in
TestOzoneClientRetriesOnExceptions#testMaxRetriesByOzoneClient
Key: HDDS-16165
URL: https://issues.apache.org/jira/browse/HDDS-16165
Project: Apache Ozone
Issue Type: Sub-task
Components: Ozone Client, test
Reporter: Siyao Meng
{code}
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 54.04 s
<<< FAILURE! -- in
org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions
[ERROR]
org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.testMaxRetriesByOzoneClient
-- Time elapsed: 16.51 s <<< ERROR!
java.io.IOException: Failed to execute command cmdType: CreateContainer
containerID: 1 ... createContainer { state: OPEN }
at
org.apache.hadoop.hdds.scm.XceiverClientSpi.getIOExceptionForSendCommand(XceiverClientSpi.java:178)
at
org.apache.hadoop.hdds.scm.XceiverClientSpi.sendCommand(XceiverClientSpi.java:131)
at
org.apache.hadoop.ozone.client.rpc.TestOzoneClientRetriesOnExceptions.testMaxRetriesByOzoneClient(TestOzoneClientRetriesOnExceptions.java:206)
{code}
The datanode log for the same run shows the underlying cause:
{code}
Operation: CreateContainer , Message: Container creation failed because key
value container already exists , Result: CONTAINER_ALREADY_EXISTS
{code}
h3. Root cause
{{testMaxRetriesByOzoneClient}} writes a key spanning {{MAX_RETRIES + 1}} block
entries, expects each entry to land on a distinct container, and issues a
CreateContainer for each so the write can be retried across N+1 different
blocks. It guards the distinct-container premise, but that guard was neutered:
{code}
//
hadoop-ozone/integration-test/.../client/rpc/TestOzoneClientRetriesOnExceptions.java:204
assertThat(containerList.contains(containerID)); // AssertJ boolean assert
with no terminal, a no-op
{code}
{{assertThat(boolean)}} without a terminal (such as {{isFalse()}}) checks
nothing. When block allocation reuses a container the loop proceeds to
CreateContainer on an already created container, which fails with
{{CONTAINER_ALREADY_EXISTS}} and surfaces as the IOException above. The line
was changed from
{{Assumptions.assumeFalse(containerList.contains(containerID))}} to this no-op
form in HDDS-11020, which also introduced RoundRobinPipelineChoosePolicy
(making container reuse more likely). The test is not tagged {{@Flaky}}, so it
runs in the gating {{integration (client)}} split.
h3. Fix
Restore the assumption so the test skips (aborts) the inconclusive case instead
of double-creating a container:
{code}
Assumptions.assumeFalse(containerList.contains(containerID));
{code}
- https://github.com/apache/ozone/actions/runs/31581614789/job/94114425679
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
