[
https://issues.apache.org/jira/browse/GROOVY-12263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18104999#comment-18104999
]
ASF GitHub Bot commented on GROOVY-12263:
-----------------------------------------
daniellansun opened a new pull request, #2790:
URL: https://github.com/apache/groovy/pull/2790
https://issues.apache.org/jira/browse/GROOVY-12263
# Performance Verification Report
**Subject.** `1c3820bff71419b5c40e73142bbf3e82210ea317` — *Invoke cached
Closure doCall targets via MethodHandle*
**Baseline.** `9bb195dee52e82518bb5f4e5cda9ddbf08c16d39` — immediate parent
of the subject. The only production delta is
`src/main/java/groovy/lang/Closure.java`. The accompanying unit-test file is
not on the JMH hot path.
**Verdict.** The change delivers a statistically significant, host-stable,
and *path-specific* improvement on the Java/GDK `Closure.call` entry that the
commit claims to accelerate.
On the four GDK iteration benches that actually execute
`DefaultGroovyMethods` → `Closure.call(Object)` → `call(Object...)`, wall-clock
time falls by **13–21%** (geometric mean **1.168×**, about **4–6 ns per
callback**). Groovy `invokedynamic` call sites that already bind straight to
`doCall` are unchanged (geomean **0.990×**). A `MethodClosure` negative control
is unchanged (**0.994×**). Host-calibration rulers sit at **0.998×**, so the
GDK movement is not a host-speed artifact.
---
## 1. What was compared
`git diff --stat 9bb195dee5 1c3820bff7`:
| File | Role |
|---|---|
| `src/main/java/groovy/lang/Closure.java` | Production: cached `doCall` /
`call` targets are invoked via `MethodHandle.invokeExact` instead of
`Method.invoke`. |
| `src/test/groovy/groovy/lang/ClosureCallHandleTest.groovy` | Tests only.
Not loaded by the JMH measurement loops. |
No other module, Gradle flag, or benchmark source differs between the two
worktrees.
**Isolated bytecode check** (same class extracted from each JMH fat JAR):
| Artifact | `Closure.class` SHA-256 | `invokeExact` count | `Method.invoke`
count |
|---|---|---|---|
| Baseline JAR | `7426d0a0…5461d609` | **0** | **2** |
| Target JAR | `2475a3f1…3e76a198` | **6** | **1** |
The six `invokeExact` sites are the specialized 0–4-arity cases plus the
defensive spreader. The remaining `Method.invoke` is the documented fallback
when `unreflect` fails. The two JARs therefore implement the two dispatch
strategies under test and nothing else.
---
## 2. Measurement protocol
The experiment is a *paired, same-host, same-JVM, sequential* comparison of
two isolated builds. It is deliberately *not* a comparison against the 90-day
gh-pages history: that history is dominated by runner-hardware noise (see
`subprojects/performance/README.adoc`).
### 2.1 Isolation
- Separate git worktrees at the two SHAs (`/tmp/groovy-base-9bb195d`,
`/tmp/groovy-mh-1c3820b`).
- Each worktree built with `./gradlew :performance:jmhJar --offline`.
- JMH executed against that worktree’s own fat JAR
(`performance-6.0.0-SNAPSHOT-jmh.jar`).
- Runs were **serial**, not concurrent, so they did not contend for the six
vCPUs.
### 2.2 JMH configuration
These flags override the `@Fork(2)` annotations on the benchmark classes.
| Parameter | Value | Rationale |
|---|---|---|
| Benchmarks | `org.apache.groovy.perf.ClosureBench` (19 methods) +
`org.apache.groovy.perf.HostCalibrationBench` (3 rulers) | The project’s own
closure suite plus the core-hz hardware rulers |
| Mode / unit | `AverageTime`, as declared by the benches (`ms/op` for
`ClosureBench`, `us/op` for rulers) | Lower is better |
| Forks | **4** independent JVMs | Between-fork variance is visible; one
noisy fork cannot dominate |
| Warmup | 4 × 2 s | Past the C1/C2 transition on these loops |
| Measurement | 5 × 2 s | 20 samples per bench (4 × 5) |
| Heap | `-Xms2g -Xmx2g -XX:+AlwaysPreTouch` | Removes heap-resize and
first-touch noise |
| Confidence | JMH default **99.9%** CI | Primary significance criterion:
non-overlapping CIs |
| Order | Baseline first (23:49–00:19), target second (00:19–00:48), same
host | Calibration rulers quantify any thermal or neighbor drift |
### 2.3 Host
| Item | Value |
|---|---|
| Host | `hera` |
| CPU | AMD EPYC 7763, 6 vCPUs, 1 thread/core |
| Memory | 23 GiB |
| OS | Linux 6.15.5 x86_64 |
| JDK | Amazon Corretto **25.0.2+10-LTS** (`25.0.2-amzn`) |
| Frequency governor | not exposed (`cpufreq` n/a) |
### 2.4 How “faster” is defined
For `AverageTime`, **speedup = baseline / target**. Values greater than 1
mean the target is faster. A result is labeled **faster** or **slower** only
when the two 99.9% CIs do not overlap; otherwise **inconclusive**.
A Welch two-sample *t* on the 20 raw iteration samples is reported as a
secondary check (`t`, Welch–Satterthwaite `df`). Two-sided *p*-values are not
tabulated: SciPy is not installed on this host, and every GDK *t* exceeds 11 on
`df > 22`, which is `p ≪ 0.001` under any reasonable tail model.
---
## 3. Path analysis: which benches *must* move
The production change lives **only** in `Closure.call(Object...)`. A bench
can improve only if its steady-state work actually enters that method.
Generated closures declare `doCall(...)` and do **not** override
`call(Object)`. Two distinct caller shapes then arise:
1. **Java / GDK entry.** `DefaultGroovyMethods.each` / `collect` / `findAll`
/ `inject` compile as Java `closure.call(item)` (or `call(acc, val)`). That
resolves to `Closure.call(Object)` / `call(Object, Object)`, which wrap into
`call(Object...)`. This is the path named in the commit comment (the `each` /
`collect` hot path). **Primary treatment.**
2. **Groovy `invokedynamic` entry.** `c(i)` and `c.call(i)` in
`ClosureBench` compile to the same indy site,
`invoke:(Lgroovy/lang/Closure;I)`. After warmup the site binds **directly to
`doCall(Object)`** and never enters `Closure.call(Object...)`. **Must not
move.** Confirmed by disassembly of `ClosureBench.closureCallMethod` and
`closureReuse` in the target JAR, and by the generated class
`ClosureBench$_closureCallMethod_closure14` exposing only `doCall` / `doCall()`.
3. **Adapters that re-enter a generated closure.** `CurriedClosure` and
`MethodClosure` are explicitly `CallOverride.NONE`.
`ComposedClosure.doCall(Object[])` is array-typed and likewise uncached. Their
*outer* call stays on the metaclass. The *inner* generated bodies, however, are
invoked via `call(...)` after uncurry or composition, so they **can** pick up
the `MethodHandle` path. These are **secondary / indirect**, not clean
negatives.
4. **True negative control.** `list.&size` is a `MethodClosure`:
`CallOverride.lookup` returns `NONE`, and there is no generated `doCall` body
to re-enter. **Must not move.**
5. **Hardware rulers.** `HostCalibrationBench.{cpuIntegerOps,
memoryPointerChase, allocationChurn}` are pure Java and Groovy-independent.
Their geometric mean is the **calibration factor**. A factor near 1 means the
two 29-minute windows ran at equivalent host speed.
```
Groovy indy ──► doCall
(ClosureBench.closureReuse / .closureCallMethod)
▲
Java/GDK call(Object) │
└─► Closure.call(Object...) ──► Method.invoke [baseline]
└─► invokeExact [target] ◄── this
commit
│
Curried / Composed outer ──► metaclass ──┘ (re-enters inner call(...))
MethodClosure ──► metaclass, no generated doCall
```
---
## 4. Results
### 4.1 Hardware calibration (must be ~1.0×)
| Ruler | Baseline | Target | Speedup | 99.9% CIs | Verdict |
|---|---:|---:|---:|---|---|
| `cpuIntegerOps` | 407.502 ± 1.914 µs/op | 405.945 ± 1.898 µs/op | 1.004× |
overlap | inconclusive |
| `memoryPointerChase` | 1419.758 ± 52.048 µs/op | 1451.855 ± 19.503 µs/op |
0.978× | overlap | inconclusive |
| `allocationChurn` | 96.799 ± 4.399 µs/op | 95.504 ± 3.902 µs/op | 1.014× |
overlap | inconclusive |
| **Geomean** | | | **0.998×** | | **no host drift** |
The second window is 0.2% slower on the geometric mean of the rulers — well
inside JMH noise, and in the *opposite* direction of the GDK result. A 17% GDK
movement cannot be attributed to the machine speeding up.
### 4.2 Primary treatment — GDK Java callbacks (must improve if the claim is
true)
Each of these methods performs **1 000 000** closure invocations per JMH op
(`ITERATIONS/10` outer loops × a 10-element list, or 2-arg `inject` over the
same list). Scores are therefore also **nanoseconds per callback**, including
iterator and DGM overhead.
| Benchmark | Baseline (ms/op) | Target (ms/op) | Speedup | Δ per call |
99.9% CIs | Welch *t* (df) | Verdict |
|---|---:|---:|---:|---:|---|---:|---|
| `eachWithClosure` | 34.438 ± 0.666 | 29.404 ± 0.385 | **1.171×** | **−5.03
ns** | disjoint | 25.41 (30.4) | **faster** |
| `collectWithClosure` | 36.576 ± 1.598 | 31.454 ± 0.481 | **1.163×** |
**−5.12 ns** | disjoint | 11.92 (22.4) | **faster** |
| `findAllWithClosure` | 34.202 ± 0.986 | 30.313 ± 0.906 | **1.128×** |
**−3.89 ns** | disjoint | 11.28 (37.7) | **faster** |
| `injectWithClosure` | 34.052 ± 1.019 | 28.095 ± 0.677 | **1.212×** |
**−5.96 ns** | disjoint | 18.90 (33.0) | **faster** |
| **Geomean** | | | **1.168×** | **≈ −5.0 ns** | | | **all four faster**
|
Per-fork means (ms/op) — every fork of every GDK bench moves in the same
direction; this is not a single lucky fork:
| Bench | Baseline forks | Target forks |
|---|---|---|
| `each` | 34.28, 34.05, 34.51, 34.92 | 29.12, 29.54, 29.54, 29.42 |
| `collect` | 36.04, 37.11, 35.90, 37.25 | 31.67, 31.37, 31.44, 31.34 |
| `findAll` | 34.46, 34.72, 33.66, 33.97 | 30.34, 30.65, 30.24, 30.02 |
| `inject` | 34.41, 34.46, 33.62, 33.73 | 27.47, 28.61, 28.18, 28.12 |
Relative CI half-width stays in the 1.3–4.4% band on both sides; the target
is if anything tighter. `findAll` saves a little less (~3.9 ns) than `each` /
`collect` / `inject` (~5–6 ns), which is consistent with
`BooleanClosureWrapper` adding a fixed cost that the `MethodHandle` change does
not touch.
**Reading the 5 ns.** A Groovy-indy `doCall` of `{ it * 2 }` is ~2.2 ns in
the same process (`closureCallMethod`). The GDK benches spend ~34 ns per
callback, of which iterator + DGM + `call(Object)` array wrap + `doCall` body
account for the rest. Removing `Method.invoke`’s reflective wrapper from that
mix and replacing it with `invokeExact` is expected to save a handful of
nanoseconds, not tens. The measured −5 ns/call matches that model. It is
**not** a 17% reduction inside `doCall` itself; it is a 17% reduction in the
*GDK callback round-trip*, which is exactly the surface the commit optimizes.
### 4.3 Groovy indy sites (must *not* improve)
| Benchmark | Baseline | Target | Speedup | Verdict |
|---|---:|---:|---:|---|
| `closureCallMethod` | 2.202 ± 0.094 | 2.200 ± 0.073 | 1.001× |
inconclusive |
| `closureReuse` | 2.220 ± 0.105 | 2.182 ± 0.059 | 1.017× | inconclusive |
| `closureWithCapture` | 2.601 ± 0.062 | 2.580 ± 0.067 | 1.008× |
inconclusive |
| `closureModifyCapture` | 4.813 ± 0.168 | 4.862 ± 0.184 | 0.990× |
inconclusive |
| `closureMultiParams` | 14.942 ± 0.316 | 14.855 ± 0.443 | 1.006× |
inconclusive |
| `simpleClosureCreation` | 29.528 ± 0.691 | 29.288 ± 0.807 | 1.008× |
inconclusive |
| `closureAsParameter` | 28.606 ± 2.925 | 29.563 ± 3.470 | 0.968× |
inconclusive |
| `closureDelegation` | 50.361 ± 1.603 | 46.799 ± 2.949 | 1.076× |
inconclusive |
| `nestedClosures` | 33.478 ± 5.876 | 39.171 ± 7.665 | 0.855× | inconclusive
|
| **Geomean** | | | **0.990×** | **flat** |
`nestedClosures` and `closureDelegation` have wide CIs (inner allocation and
property-dispatch work dominate) and still overlap. The near-2.2 ns
`closureCallMethod` / `closureReuse` pair is identical to 0.1%. This group is
the **specificity check**: if the whole JVM had simply gotten faster, these
would have moved with the GDK benches. They did not.
### 4.4 Adapters and the true negative control
| Benchmark | Path | Baseline | Target | Speedup | Verdict |
|---|---|---:|---:|---:|---|
| `methodReference` | `MethodClosure` → `NONE` | 87.927 ± 3.022 | 88.473 ±
3.352 | **0.994×** | inconclusive |
| `curriedClosure` | outer `NONE`, inner re-enters `call` | 24.920 ± 0.757 |
20.975 ± 0.936 | 1.188× | faster |
| `rightCurriedClosure` | same | 25.006 ± 0.564 | 20.072 ± 0.690 | 1.246× |
faster |
| `closureComposition` | `doCall(Object[])` uncached; inners via `call` |
39.456 ± 0.952 | 33.935 ± 1.119 | 1.163× | faster |
`methodReference` is the clean negative and does not move (*t* = −0.47).
Curry, rcurry, and compose **do** move, in the same 16–25% band as the GDK
group. That is expected once the inner generated closures are taken into
account: `CurriedClosure` is excluded from the cache *so that* `MetaClassImpl`
can uncurry and re-enter, and that re-entry is a `call(...)` on a generated
closure. Treating them as proof of a general, unspecified speedup would be
wrong; treating them as a contradiction would also be wrong. They are a
**consistent secondary effect**.
### 4.5 Context benches (not a test of this commit)
| Benchmark | Baseline | Target | Speedup | Verdict |
|---|---:|---:|---:|---|
| `closureTrampoline` | 44.738 ± 2.324 | 44.550 ± 2.327 | 1.004× |
inconclusive |
| `closureSpread` | 1786.532 ± 64.833 | 1832.367 ± 94.159 | 0.975× |
inconclusive |
Trampoline is dominated by `TrampolineClosure` machinery. Spread
(`sum3(*args)`) is dominated by argument packing. CIs overlap.
---
## 5. Why this is the MethodHandle change, not a confound
| Alternative explanation | Why it is rejected |
|---|---|
| Host sped up between the two 29-minute windows | Calibration geomean
**0.998×**; integer-ruler CIs overlap; the 0.4% `cpuIntegerOps` tick is smaller
than, and opposite in direction to, a 17% GDK claim. |
| Different sources, flags, or dependency versions | `git diff` is two
files; both JARs built `--offline` from worktrees pinned at the two SHAs;
`Closure.class` hashes and `invokeExact` counts match the intended
implementations. |
| JIT / fork noise | 4 forks; every GDK fork moves the same way; 20 samples;
99.9% CIs disjoint; Welch *t* ∈ [11, 25]. |
| “Everything that uses a closure got faster” | Indy `doCall` sites and
`MethodClosure` did **not** move. The improvement is confined to callers that
enter `Closure.call(Object...)`. |
| The test-file change affected the benches | `ClosureCallHandleTest` is not
referenced from `ClosureBench`. |
| Packed closures hiding the path | Packing is opt-in (`@PackedClosures` /
`groovy.target.closure.pack`). `ClosureBench` is compiled without it; the
generated classes extend `Closure` and implement `GeneratedClosure` with
`doCall` only. |
---
## 6. What this report does *not* claim
- It does **not** claim that Groovy dynamic dispatch in general is 17%
faster. Groovy-indy `c(i)` was already ~2.2 ns/call and stays there.
- It does **not** claim a 17% reduction inside `doCall`. The ~5 ns is the
reflective-invoke overhead disappearing from the Java `call` wrapper around
`doCall`.
- It does **not** compare against the gh-pages 90-day dashboard. That
comparison is hardware-dominated; this one is a same-host A/B of two commits.
- It does **not** measure cold start, allocation (`-prof gc` was not
attached, to keep the timing run clean), or packed-closure dispatch
(`PackedClosure` overrides `call` and never uses this cache).
- Frequency-governor data is unavailable on this VM; calibration rulers are
the substitute.
---
## 7. Conclusion
On a same-host, 4-fork, 99.9%-CI JMH comparison of the isolated parent and
the MethodHandle commit:
1. **The advertised path is faster.** GDK `each` / `collect` / `findAll` /
`inject` improve by **1.13–1.21×** (geomean **1.168×**, **−4 to −6 ns per
callback**). All four 99.9% CIs are disjoint; all four fork sets move in the
same direction.
2. **The improvement is specific.** Groovy-indy sites that bind to `doCall`
(geomean **0.990×**) and `MethodClosure` (**0.994×**) do not move.
3. **The host did not move.** Calibration geomean **0.998×**.
4. **Secondary adapters behave as the architecture predicts.** Curry and
compose improve because they re-enter generated `call(...)`; they are not
counterexamples.
The performance claim of `1c3820b` is therefore **confirmed** for the
Java/GDK `Closure.call` hot path, at a magnitude that matches the cost of
`Method.invoke` being removed, and with no detectable regression on the paths
the change is designed to leave alone.
---
## Appendix A — Reproducing this run
```bash
git worktree add /tmp/groovy-base-9bb195d
9bb195dee52e82518bb5f4e5cda9ddbf08c16d39
git worktree add /tmp/groovy-mh-1c3820b
1c3820bff71419b5c40e73142bbf3e82210ea317
(cd /tmp/groovy-base-9bb195d && ./gradlew :performance:jmhJar --offline)
(cd /tmp/groovy-mh-1c3820b && ./gradlew :performance:jmhJar --offline)
java -jar
/tmp/groovy-base-9bb195d/subprojects/performance/build/libs/performance-6.0.0-SNAPSHOT-jmh.jar
\
org.apache.groovy.perf.ClosureBench
org.apache.groovy.perf.HostCalibrationBench \
-f 4 -wi 4 -i 5 -w 2s -r 2s -rf json -foe true \
-jvmArgsAppend '-Xms2g -Xmx2g -XX:+AlwaysPreTouch' \
-rff jmh-base.json -o jmh-base.txt
java -jar
/tmp/groovy-mh-1c3820b/subprojects/performance/build/libs/performance-6.0.0-SNAPSHOT-jmh.jar
\
org.apache.groovy.perf.ClosureBench
org.apache.groovy.perf.HostCalibrationBench \
-f 4 -wi 4 -i 5 -w 2s -r 2s -rf json -foe true \
-jvmArgsAppend '-Xms2g -Xmx2g -XX:+AlwaysPreTouch' \
-rff jmh-target.json -o jmh-target.txt
```
Raw JMH JSON from this run: `/tmp/grok-1000/jmh-base.json`,
`/tmp/grok-1000/jmh-target.json`.
## Appendix B — Artifact hashes
| Item | Value |
|---|---|
| Baseline SHA | `9bb195dee52e82518bb5f4e5cda9ddbf08c16d39` |
| Target SHA | `1c3820bff71419b5c40e73142bbf3e82210ea317` |
| Baseline JMH JAR SHA-256 |
`e321d268c7e099519e3f9e8b1b38d9bba5e11bf8151ee928f4a12156d4e40016` |
| Target JMH JAR SHA-256 |
`4cd0b63d1ca9a7f9cb383bae2b594adfc3cb079790260d3415aa2308d5f02b17` |
| Baseline window | 2026-08-15 23:49:53 – 2026-08-16 00:19:05 +09 |
| Target window | 2026-08-16 00:19:15 – 2026-08-16 00:48:23 +09 |
> Invoke cached Closure doCall targets via MethodHandle
> -----------------------------------------------------
>
> Key: GROOVY-12263
> URL: https://issues.apache.org/jira/browse/GROOVY-12263
> Project: Groovy
> Issue Type: Improvement
> Reporter: Daniel Sun
> Priority: Major
>
> The {{Closure.call(Object...)}} fast path (GROOVY-11911, GROOVY-12164,
> GROOVY-12165) already caches a per-arity {{doCall}} / {{call}} {{Method}} and
> invokes it with {{Method.invoke}}. That still puts a reflective invoke —
> access check, argument boxing, {{InvocationTargetException}} wrap — on every
> GDK {{each}} / {{collect}} / {{findAll}} / {{inject}} callback from Java.
> h2. Proposal
> At cache-build time, {{MethodHandles.unreflect}} the cached {{Method}} and
> adapt it to {{genericMethodType(arity+1)}}. {{call(Object...)}} then prefers
> {{invokeExact}} on that handle (specialized for arities 0–4).
> {{Method.invoke}} remains only when the method cannot be adapted, so the
> GROOVY-11911 {{call()}} / {{call(Object)}} carve-out still works if unreflect
> fails.
> Exception contracts stay as they were on the reflective path: a body-thrown
> throwable surfaces unwrapped. The handle path must not treat a body-thrown
> {{InvocationTargetException}} or {{IllegalAccessException}} as a reflection
> wrapper.
> Guards, {{CallOverride.NONE}} for {{MethodClosure}} / {{CurriedClosure}}, and
> the metaclass fallback for coercion (GROOVY-12164) are unchanged.
> h2. Why this path
> Java callers such as {{DefaultGroovyMethods}} resolve {{closure.call(item)}}
> to {{Closure.call(Object)}}, which wraps into {{call(Object...)}}. Groovy
> {{invokedynamic}} sites typically bind straight to {{doCall}} after warmup
> and never enter this method — they are out of scope.
> h2. Verification
> Same-host JMH, 4 forks, 99.9% CI, parent {{9bb195dee5}} vs {{1c3820bff7}},
> JDK 25. Host-calibration geomean 0.998x.
> || bench || speedup ||
> | {{eachWithClosure}} | 1.171x |
> | {{collectWithClosure}} | 1.163x |
> | {{findAllWithClosure}} | 1.128x |
> | {{injectWithClosure}} | 1.212x |
> | GDK geomean | 1.168x (~5 ns/callback) |
> | Groovy-indy {{doCall}} sites | 0.990x (flat) |
> | {{MethodClosure}} ({{list.&size}}) | 0.994x (flat) |
> All four GDK 99.9% CIs are disjoint. Full write-up:
> {{docs/closure-call-methodhandle-perf-report.md}}.
> h2. Related
> GROOVY-11911 introduced the reflective cache. GROOVY-12164 / GROOVY-12165
> extended it with typed and multi-arity guards. This change keeps that
> selection and only replaces the invoke.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
