nzw921rx opened a new pull request, #11986:
URL: https://github.com/apache/seatunnel/pull/11986
### Purpose
Part of #11598.
Add a focused JMH benchmark for comparing the production intermediate
BlockingQueue and Disruptor record-handoff paths.
### Changes
- Add two separately reported JMH benchmarks:
- `blockingQueueRecordHandoff`
- `disruptorRecordHandoff`
- Use independent JMH states and consumers so the two implementations do not
interfere with each other.
- Exercise the production `IntermediateBlockingQueue`,
`IntermediateDisruptor`, and `IntermediateQueueFlowLifeCycle` paths.
- Reuse 4,096 varied `SeaTunnelRow` records containing an ID, string field,
256-byte payload, and partition value.
- Read multiple row fields and calculate a checksum on the consumer side.
- Add a deterministic test that transfers 10,000 records through both queue
implementations and verifies:
- published record count
- consumed record count
- consumed checksum
- Add `IntermediateQueueBenchmark` to the manual GitHub Actions selector.
- Scheduled benchmark runs already use `.*`, so both methods will also run
daily on Java 8 and Java 11.
- Declare Disruptor as a direct dependency because the benchmark constructs
the production RingBuffer configuration directly.
### Benchmark scope
The measured operation is one record handoff from the producer into the
selected production intermediate queue.
A dedicated consumer reads the row and updates the verification checksum.
Record construction and queue startup happen during trial setup and are
excluded from the measured operation.
Default parameters:
- Queue capacity: `1024`
- Record pool size: `4096`
- Producer threads: `1`
A short local smoke run produced approximately:
- BlockingQueue: 3.08 million ops/s
- Disruptor: 6.60 million ops/s
These numbers only verify that the two benchmarks run and report
independently. Scheduled multi-fork results should be used for performance
analysis.
### Verification
```bash
./mvnw -Pbenchmark -pl seatunnel-benchmarks spotless:apply
./mvnw -Pbenchmark -pl seatunnel-benchmarks \
-DskipIT -DskipUT=false \
-Dtest=IntermediateQueueBenchmarkTest test
./mvnw -Pbenchmark -pl seatunnel-benchmarks -DskipTests package
java -jar seatunnel-benchmarks/target/benchmarks.jar \
-l 'IntermediateQueueBenchmark'
java -jar seatunnel-benchmarks/target/benchmarks.jar \
'IntermediateQueueBenchmark.*RecordHandoff$' \
-p capacity=1024 \
-p recordPoolSize=4096 \
-f 1 -wi 1 -i 2 -w 1s -r 1s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
