shuan1026 opened a new pull request, #11029:
URL: https://github.com/apache/ozone/pull/11029
## What changes were proposed in this pull request?
`TestXceiverClientMetrics#testMetrics` was tagged `@Flaky("HDDS-11646")`
because of an
intermittent `waitFor` `TimeoutException`.
Root cause: with the default `STAND_ALONE` pipeline, writes go through
`XceiverClientGrpc`, whose `sendCommandAsync` already blocks the caller
until the
response arrives (`shouldBlockAndWaitAsyncReply` returns `true` for
non-read-only
requests). The test used a single background thread that fired 10 "async"
writes
serially and then slept 1s, so at most 1 request was ever in flight at a
time,
`PendingOps` floated back to 0 for most of each cycle, and the main thread's
100ms
poll frequently missed the narrow non-zero window, causing the observed
timeout.
This PR fixes the root cause by:
* Replaces the single serialized sender thread with 10 concurrent sender
threads,
each continuously sending blocking writes until the pending spike is
observed.
With genuine concurrent in-flight requests, `PendingOps` stays non-zero
for a
sustained window instead of spiking for microseconds, so the poll reliably
catches it.
* `computeResults` is wrapped in `Collections.synchronizedList` for
thread-safe
concurrent writes, and each sender thread now `countDown()` the shared
`CountDownLatch` in a `finally` block so `latch.await()` can't hang if a
thread
exits via an exception.
* The pending count increased poll now
reads`XceiverClientManager.getXceiverClientMetrics().getPendingContainerOpCountMetrics(...)`
directly instead of going through `MetricsAsserts.getMetrics(SOURCE_NAME)` on
every tick; the poll interval is also tightened from 100ms to 10ms.
* Adds a short retry before the final `PendingOps == 0` /
`numPendingPutSmallFile
== 0` assertions, absorbing the small window between a response future
completing and its pending-metric decrement (two separate, non-atomic
steps in
`XceiverClientGrpc`'s gRPC callback).
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11646
## How was this patch tested?
* Confirmed with `.github/workflows/intermittent-test-check.yml` all green
https://github.com/shuan1026/ozone/actions/runs/31959022505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
