anjy7 opened a new pull request, #22669:
URL: https://github.com/apache/kafka/pull/22669
## Background
KafkaRaftClient implements KRaft, Kafka's Raft consensus protocol. Today a
single KafkaRaftClientDriver thread repeatedly calls KafkaRaftClient#poll() and
each poll() iteration is an "all-in-one" event handler: it updates state,
sends/handles RPCs, appends records (on the leader), and updates listeners to
the log. The Kafka metadata team plans to refactor this programming model so
that each of those responsibilities becomes a distinct event scheduled on a
single-threaded event queue, the model already used elsewhere in Kafka (e.g.
the controller's event queue).
The raft module has strong correctness protection (unit tests and simulation
tests), but it has no performance benchmarks.A refactor of this size to a layer
this critical can silently regress performance
This PR adds a protocol-level JMH benchmark suite.
## What changed
Each benchmark follows the shape state X -> event -> state Y and reuses the
existing RaftClientTestContext fixture.
### This first PR covers:
ElectionBenchmark drives a full Unattached -> Leader election. It runs in
SingleShotTime mode with a fresh context per invocation, since electing a
leader consumes the context's state.
LeaderBenchmark measures steady-state leader FETCH handling from a fully
caught-up follower (a FETCH that does not advance the high watermark). It runs
in AverageTime mode on one prepared leader reused across invocations.
Because the mocks are in-memory, elapsed time alone is blind to regressions
like an extra flush per operation, so each benchmark also reports
machine-independent work counters via @AuxCounters: log flushes, reads, and
truncations; RPC requests and responses sent; and quorum-state-file writes.
These are reported as per-iteration totals plus an operations count, so the
per-operation value is total / operations; the counts are integer-exact and
should be stable across a correct refactor (a flush count going from 1 to 2 per
op is a behavioral diff, not noise). Pairing with -prof gc gives allocations
per operation (gc.alloc.rate.norm).
A thin public adapter RaftClientBenchmarkContext is added to the raft test
fixtures. It exposes factories, scenario primitives, and the drainable work
counters. The counters themselves are small additive fields on the existing
mocks (MockLog, MockNetworkChannel, MockQuorumStateStore, MockMessageQueue).
RaftClientTestContext.pollUntil(...) is given a configurable poll interval.
It defaults to 100ms (unchanged for existing tests); the benchmark contexts set
it to 0 to busy-poll, mirroring KafkaRaftClientDriver, so the harness's
sleep-between-polls does not dominate benchmark timing.
## Run with:
Run individually:
./jmh-benchmarks/jmh.sh BenchmarkClassName
Run all:
./jmh-benchmarks/jmh.sh ".*jmh.raft.*"
## Testing
Ran the suite locally and the work counters match the expected per-operation
profiles.
## Reviewers:
Kevin Wu [email protected]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
