xiangfu0 opened a new pull request, #19514:
URL: https://github.com/apache/pinot/pull/19514
Timestamp range predicates such as `event_time >= CAST('2024-01-01 00:00:00'
AS TIMESTAMP)` compile a generated Java reducer for each constant cast during
Calcite field trimming. Numeric epoch strings also pay this cost even though
Calcite cannot parse them and leaves the casts for later Pinot planning.
Add a narrow executor for ordinary CHAR/VARCHAR literal-to-TIMESTAMP casts.
It calls the same Avatica parser and RexBuilder literal construction as
Calcite's generated code. Unsupported expression batches still delegate to
Calcite; a failed supported batch remains entirely unchanged. This preserves
nulls, precision, invalid-input behavior and batch ordering.
Install the executor only during SQL-to-rel conversion, restoring the prior
executor before optimization and preserving explicitly supplied executors.
Installing it only in a RelBuilder context is insufficient because
`RelBuilder.transform` discards that context.
### Benchmark
Independently packaged baseline `c6116095f7d3834613649c770d75fed7dd5a874a`
and this change, with identical benchmark source and bytecode. The benchmark
calls `QueryEnvironment.compile` and closes each compiled query, covering SQL
parsing, validation, SQL-to-rel conversion and logical optimization. It cycles
through 1,024 prebuilt synthetic account IDs. The measured window query filters
a timestamp range and selects the latest event version using `ROW_NUMBER()`.
Warmed results on a shared Apple M4 Pro host with 24 GiB RAM, OpenJDK
25.0.4, JMH 1.37, one benchmark thread, 1 GiB heap and `ActiveProcessorCount=4`:
| Window-query constant form | Baseline µs/op | PR µs/op | Latency change |
Baseline B/op | PR B/op | Allocation change |
|---|---:|---:|---:|---:|---:|---:|
| Timestamp-string CAST | 1,327.98 | 483.56 | **−63.59%** | 1,332,499 |
860,004 | **−35.46%** |
| Epoch-string CAST | 1,314.98 | 506.25 | **−61.50%** | 1,335,453 | 864,977
| **−35.23%** |
| Typed TIMESTAMP literal control | 469.26 | 478.46 | +1.96% | 846,269 |
846,059 | −0.02% |
Each revision has two independent JVM forks, run baseline/PR/PR/baseline.
Each fork uses five 3-second warmups and five 2-second measurements with the GC
profiler. Means include every measured iteration. The control's direction
reverses between pairs; one baseline control iteration is elevated and
retained. These shared-host measurements establish a compilation benefit for
these queries, not end-to-end query throughput or precise bounds on small
regressions. Shorter pilot runs were excluded because they were still warming
up.
<details>
<summary>Independent fork means and reproduction</summary>
| Constant form | Baseline A/B µs/op | PR A/B µs/op |
|---|---|---|
| Timestamp-string CAST | 1,312.65 / 1,343.30 | 486.42 / 480.70 |
| Epoch-string CAST | 1,329.88 / 1,300.09 | 496.79 / 515.71 |
| Typed literal control | 452.94 / 485.58 | 486.35 / 470.58 |
Copy `BenchmarkConstantCastPlanning.java` to a worktree at the baseline
commit. Build each worktree separately with JDK 25:
```sh
./mvnw -pl pinot-perf -am package -Ppinot-fastdev -DskipTests \
-Dskip.npm -Dskip.installnodenpm -Dcheckstyle.skip -Dspotless.check.skip \
-Dlicense.skip -Denforcer.skip -Dshade.phase.prop=none \
-Dassembly.skipAssembly=true -Dmaven.javadoc.skip=true
```
Run the following once in each worktree in baseline/PR/PR/baseline order,
without concurrent builds. Use distinct result filenames for each run. Verify
the packaged planner and benchmark jars match their module target jars before
running.
```sh
"$JAVA_HOME/bin/java" -Xmx1g \
-cp 'pinot-perf/target/pinot-perf-pkg/lib/*' org.openjdk.jmh.Main \
BenchmarkConstantCastPlanning \
-p _queryShape=WINDOW \
-p _literalForm=CAST,EPOCH_STRING_CAST,TIMESTAMP_LITERAL \
-wi 5 -w 3s -i 5 -r 2s -f 1 -t 1 -foe true -prof gc \
-jvmArgs '-Xms1g -Xmx1g -XX:ActiveProcessorCount=4' \
-rf json -rff result.json
```
The harness also includes a simple FILTER query and a LONG epoch-millis
column control for additional experiments.
</details>
Separate matched JFR runs corroborate the mechanism: Janino occurs in
142/471 baseline JMH-worker execution samples and 0/750 PR samples. Sampling
alone does not establish absence; the integration tests independently count
generated reductions exactly.
### Validation
- **900 planner tests passed:** 50 executor cases, 8 full-compilation cases,
240 query-compilation cases, 10 type-coercion cases and 592 resource-based
query plans.
- The new full-compilation tests compare plans, row types and table names
with typed timestamp literals, assert zero generated reducers, and verify the
planner executor is restored. The baseline produces two generated reducers for
the same queries while preserving plan equivalence.
- Differential executor tests cover nulls, fractional precision/rounding,
pre-epoch values, whitespace, invalid input, unsupported/mixed batches, custom
executors and concurrent calls.
- Spotless, Checkstyle and license checks pass for both changed modules.
Direct JDK 25 lint compilation of all five changed sources has zero added-line
warnings.
- The stricter deprecation-enabled reactor check encounters an existing
issue in unchanged `ZstandardDecompressor.java:51`: Zstd's provided JetBrains
annotation dependency is absent from that module's compile classpath. This is
reproduced on the identical baseline source; adding the annotation jar resolves
that check. No unrelated dependency change is included here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
