anuragrai16 opened a new pull request, #19064:
URL: https://github.com/apache/pinot/pull/19064
## Summary
`skipUnavailableServers=true` only degraded to partial results when a server
was unreachable at request **send** time (Path 1). If a server's Netty channel
went inactive **after** the request was dispatched (Path 2), or a write to it
failed, the broker force-failed the whole query with `BROKER_REQUEST_SEND`
(425) instead of returning partial results from the healthy servers.
## Changes
- Split `markServerDown` into:
- `markServerUnavailable` — skippable: under the flag it degrades to
partial results and records the down server so the failure detector can
quarantine it from routing.
- `markServerCancelled` — direct-memory OOM: always fails the query (all
channels are closing, so there is no partial data to return).
- Made the latch decrement **idempotent** — a server can be reported down
through more than one path (a write failure closes the channel, which also
fires `channelInactive`), so a repeat report must not release a healthy
server's latch slot.
- `getFailedServer()` may now be set on a partial success (it records the
quarantined server), so a non-null value no longer implies the query failed —
check `getException()` / `getStatus()`.
- Added broker meters:
- `SERVER_MARKED_DOWN_SKIPPED` (global)
- `BROKER_RESPONSES_WITH_SEND_EXCEPTIONS` (per table)
- `NETTY_CONNECTION_CHANNEL_ACTIVE` / `NETTY_CONNECTION_CHANNEL_INACTIVE`
— per-server, tagged with the server short name, for visibility into channel
flaps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
