xiangfu0 opened a new pull request, #19538:
URL: https://github.com/apache/pinot/pull/19538
## Summary
The benchmark module contains duplicate workloads, retired implementation
experiments, and standalone runners that no longer produce useful measurements.
Normal unit-test runs also include an ad hoc interner benchmark with about 737
million calls and a 400,000-segment retention test that only checks elapsed
time.
Remove 10 obsolete benchmark classes, the unused `LazyDataList` helper,
three unregistered off-heap ID-map methods, and five obsolete launcher entries.
Add a README guide pointing to current workload benchmarks. Production behavior
is unchanged.
## Benchmark removals
| Removed benchmark | Reason |
| --- | --- |
| `BenchmarkSumQuery` | Exact SUM workload duplicate of retained
`SumIntAggregationFunctionBenchmark`, which also compares SUMINT. |
| `StringDictionaryPerfTest` | Deletes its segment after lookup, then
attempts to reload it for the next measurement; retained JMH dictionary
benchmarks cover lookup and reads. |
| `RawIndexBenchmark` | Reads STRING columns through numeric conversion and
measures v1-style files although new segments default to v3; retain current raw
reader/writer benchmarks. |
| `ForwardIndexWriterBenchmark` | Unfinished file-based size experiment with
an unpopulated offsets array and no timing. |
| `BenchmarkOffHeapDictionaryMemory` | Standalone runner never initializes
its memory manager; retain the working dictionary capacity/overflow benchmark. |
| `BenchmarkQueryEngine` | Hard-coded data-directory/table template with
only COUNT and no teardown; use the configurable driver or current SSQE/MSQE
workloads. |
| `BenchmarkRoaringBitmapCreation` | Measures benchmark-local copies of
retired SoftReference cache implementations rather than the current index
reader. |
| `BenchmarkRoaringBitmapMapping` | Requires external, unprovided fixtures
and compares algorithms that discard different trailing results. |
| `BenchmarkJsonKeyMap` | Retired map/concatenation experiment; one variant
fills a different map from the one it clears and returns. |
| `BenchmarkGroovyExpressionEvaluation` | Generic Java/Groovy comparison
bypasses Pinot's evaluator, times input generation, and discards results. |
Keep benchmarks for current query, index, dictionary, aggregation, JSON/MAP,
ingestion, resource-accounting, and vector paths, including meaningful
implementation and algorithm baselines. The removed experiments do not all have
one-to-one replacements; current workload coverage remains.
## Unit-test cleanup
- Remove those two timing-only tests and their unused helper code, plus the
three disabled methods in `VectorSearchBenchmark`.
- Preserve the HDFS batch-deletion assertions while removing its
elapsed-time threshold.
- Rename the ten-value `ANY_VALUE` test to describe its correctness
coverage, and make the interner helper always assert equality.
The remaining correctness-test datasets and assertions are unchanged.
## How to reproduce
On the base revision, run the interner benchmark through the normal
unit-test runner:
```sh
./mvnw -pl pinot-common -am \
'-Dtest=FALFInternerTest#benchmarkingTest' \
-Dsurefire.failIfNoSpecifiedTests=false test
```
The selected method repeatedly interns values and prints timings without
checking correctness. The cleanup removes that method while retaining
`testInterningByteBuffers` and its assertions. The retention benchmark
similarly checks only a 30-second threshold; existing retention tests continue
to verify tracked-segment exclusion and deletion candidates.
## Validation
Using JDK 25, 136 focused tests passed with no failures, errors, or skips:
```sh
./mvnw -pl
pinot-common,pinot-core,pinot-controller,pinot-segment-local,pinot-plugins/pinot-file-system/pinot-hdfs
-am \
-Dtest=FALFInternerTest,RetentionManagerTest,AnyValueAggregationFunctionTest,HadoopPinotFSTest,IvfFlatVectorIndexTest,IvfFlatFilterAwareTest,ScalarQuantizerTest
\
-Dsurefire.failIfNoSpecifiedTests=false test
```
Spotless, Checkstyle, license formatting, and license validation passed for
all five affected modules. Isolated `javac -Xlint:all` checks passed for all
four modified test classes with no warnings on added lines. The additional
reactor build with expanded deprecation checking hit the existing missing
JetBrains `NotNull` annotation dependency in unchanged `ZstandardDecompressor`;
the normal reactor build and tests passed.
For the benchmark module, all four style/license checks passed. The complete
64-module reactor package build passed:
```sh
./mvnw -pl pinot-perf -am -Ppinot-fastdev -DskipTests package
```
JMH discovery from the generated package lists 302 retained benchmark
methods; removed benchmarks are absent. All 16 generated launcher scripts
reference existing classes, and the five retired launchers are absent.
Forked smoke runs passed for the retained SUM and dictionary-lookup
workloads. These runs validate execution only:
```sh
java @.mvn/jvm.config -cp 'pinot-perf/target/pinot-perf-pkg/lib/*'
org.openjdk.jmh.Main \
'SumIntAggregationFunctionBenchmark.testSumAggregation$' \
-p _nullHandling=false -p _nullPeriod=16 -f 1 -wi 0 -i 1 -r 100ms -foe true
java @.mvn/jvm.config -cp 'pinot-perf/target/pinot-perf-pkg/lib/*'
org.openjdk.jmh.Main \
'BenchmarkDictionaryLookup.benchmarkPlainBinarySearch$' \
-p _cardinality=100 -p _lookupPercentage=100 -f 1 -wi 0 -i 1 -r 100ms -foe
true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
