daniellansun opened a new pull request, #2822: URL: https://github.com/apache/groovy/pull/2822
https://issues.apache.org/jira/browse/GROOVY-12284 # GROOVY-12284 Performance Verification Report **Specialize indy `sameClasses` guards for arity 1–4** | Item | Value | |---|---| | Issue | [GROOVY-12284](https://issues.apache.org/jira/browse/GROOVY-12284) | | Commit under test | `8e2b29b88e8a24b56440600de9f708fc73baf437` | | Baseline (parent) | `e801df39580ab480249e561e90cf661ce86917f5` | | Range | **one commit** (the treatment is isolated) | | Date | 2026-08-22 | | Verdict | **PASS** — specialised same-class guards are faster on the linked monomorphic hot path for arities 1–4; arity 5 and `@CompileStatic` / classic stay at parity; allocation of `Object[4]` disappears on HEAD | --- ## 1. Executive summary Commit `8e2b29b` changes how Groovy's invokedynamic runtime installs the **same-class guard** on a linked monomorphic call site when every argument is non-null. The parent always used ```text SAME_CLASSES.bindTo(expectedClasses).asCollector(Object[].class, n) ``` which, on every later invocation, collects the `n` arguments into a fresh `Object[]` and loops. HEAD specialises arities 1–4 (receiver plus 0–3 parameters) to scalar `sameClass` / `sameClasses` overloads bound with `bindTo`, and keeps the collector only for arity ≥ 5. On the same host, JDK, and JMH settings, two **order-balanced** A/B trials of a dedicated end-to-end bench (`SameClassesGuardBench`) show: | Row | What it tests | Order-balanced geomean (HEAD / parent) | |---|---|---| | `dynamic_arity1` | treatment (`recv.foo()`) | **1.56×** | | `dynamic_arity2` | treatment (`recv.foo(x)`) | **1.90×** | | `dynamic_arity3` | treatment (`recv.foo(x, y)`) | **2.00×** | | `dynamic_arity4` | treatment (`recv.foo(x, y, z)`) | **4.68×** | | `dynamic_arity5` | negative control (collector on both sides) | **1.08×** (noise) | | `cs_arity2` / `cs_arity5` | negative control (`invokevirtual`) | **~1.00×** on the clean trial | | classic (`-Pindy=false`), all rows | negative control (no indy guard) | **0.98–1.00×** | The GC profiler explains why arity 4 is the outlier: on this JDK, C2 **scalar-replaces** the collector array for `n ≤ 3` (both sides allocate ≈ 0 B/op) but **fails at `n = 4`**. Parent then pays a steady **32 B/op** (`Object[4]` with compressed oops); HEAD pays **0 B/op**. Arity 5 stays at **40 B/op** on both sides — the collector was not touched. An isolated MethodHandle microbench (`SameClassesGuardMhBench`, intra-HEAD) shows the combinator itself is **1.12–1.31×** faster without MOP selection. The larger end-to-end ratios are the same adapter sitting in a longer `guardWithTest` chain, where a fat collector blocks inlining of the rest of the site. > In one line: GROOVY-12284 removes `asCollector` from the common 1–4 arity > indy guard. Where the JVM already scalar-replaces the array (arity 1–3) > the site is still **1.6–2.0×** because the handle graph is simpler; where > it does not (arity 4) the site is **4.7×** and stops allocating 32 bytes > per call. **Performance verification: PASS.** --- ## 2. Existing benches, gap, and what was added ### 2.1 What was already in the tree | Bench | Why it is not a GROOVY-12284 isolator | |---|---| | `MethodInvocationBench` | Mixes instance/static/overload/interface; primitive args box; no arity-5 collector control; no allocation row | | `CallsiteBench.dispatch_1_monomorphic_groovy` | Arity-1 `hashCode()` only; mixed with poly/mega and Java/CS; cannot split collector vs specialised | | `DynamicDispatchBench` | `methodMissing` / interceptors — a different MOP path | | `DynamicDispatchColdBench` | Cold start / promotion, not the linked-guard hot path | | `ScopedInvalidationBench` | SwitchPoint domain (GROOVY-12191), not argument-class guards | None of those makes **arity** the independent variable, none has an **arity ≥ 5 collector control**, and none reports **bytes per invocation** for this guard. ### 2.2 What was added Two benches, documented in `subprojects/performance/README.adoc`: 1. **`SameClassesGuardBench`** (Groovy, package `org.apache.groovy.bench`) — one indy (or `@CompileStatic`) call per `@Benchmark` method so `gc.alloc.rate.norm` is bytes/invocation. Pre-allocated non-null `Arg` instances; no boxing. Rows: `dynamic_arity1`–`dynamic_arity5`, `cs_arity2`, `cs_arity5`. 2. **`SameClassesGuardMhBench`** (Java) — reconstructs the exact collector vs specialised MethodHandle graphs from the public `IndyGuardsFiltersAndSignatures` methods and invokes them with `invokeExact`. Compiles only on GROOVY-12284+; used intra-HEAD. The Groovy bench was **copied identically** into the parent worktree so the only production-code difference is `8e2b29b`. --- ## 3. Change mechanism and hypotheses ### 3.1 Parent (`e801df39`) For a linked site with all arguments non-null and at least one parameter type that is non-final (or a primitive wrapper — GROOVY-11782): ```text handle = guardWithTest( SAME_CLASSES.bindTo(classes).asCollector(Object[].class, n).asType((pt)boolean), fast, fallback) ``` Dynamic indy sites almost always have `Object` parameter types, so this is the **common** case, not a rare fallback. `asCollector` is specified to allocate a new array of length `n` on every invocation. C2 may scalar-replace that array; that is an empirical question (H4), not an assumption. If any argument is null at **link** time, `Selector` already installs per-slot `SAME_CLASS` / `IS_NULL` tests. That path is unchanged and is not measured here. ### 3.2 HEAD (`8e2b29b`) `Selector.sameClassesGuard(args, pt)`: | Arity (incl. receiver) | Guard | |---|---| | 0 | constant `true` | | 1 | existing `SAME_CLASS` | | 2–4 | new `SAME_CLASSES_2` / `_3` / `_4` | | ≥ 5 | existing `SAME_CLASSES` + `asCollector` (unchanged) | Semantics are unchanged: `false` if any argument is `null` or `getClass()` differs. Confirmed by `IndySameClassesGuardTest` (3/3). ### 3.3 Hypotheses | ID | Claim | Falsifier | |---|---|---| | H1 | Dynamic arity 1–4 throughput HEAD > parent, CIs separated | Overlap or ratio ≈ 1 | | H2 | Dynamic arity 5 ≈ 1× (collector on both sides) | Robust HEAD win/loss | | H3 | `@CompileStatic` arity 2 and 5 ≈ 1× | Robust CS delta | | H4 | `gc.alloc.rate.norm` drops on some specialised arity; arity 5 stays equal | Alloc delta on arity 5, or no alloc story at arity 4 | | H5 | Intra-HEAD specialised MH > collector MH | spec/collector ≤ 1 | | H6 | CPU calibration ruler ≈ 1× (no hardware drift) | cpuIntegerOps off by ≫ 5% | | H7 | Classic (`-Pindy=false`) ≈ 1× on every row | Classic HEAD win matching indy | | H8 | Correctness suite green | Any `IndySameClassesGuardTest` failure | --- ## 4. Methodology ### 4.1 Environment | Item | Value | |---|---| | OS | Linux 6.15.5 x86_64 (hostname `hera`) | | CPU | AMD EPYC 7763, 6 vCPUs visible (KVM) | | Memory | 23 GiB | | JDK | Amazon Corretto **25.0.2** (`25.0.2+10-LTS`) | | Groovy | 6.0.0-SNAPSHOT, indy default | | JMH | 1.37; `@Warmup(3×2s) @Measurement(5×2s) @Fork(2)` → 10 samples/row (rulers: 2×1s / 3×1s / 2 forks) | | Trees | HEAD: `/home/daniel/IdeaProjects/groovy` @ `8e2b29b`; parent: worktree `/tmp/groovy-12284-parent` @ `e801df39` | | Bench parity | `SameClassesGuardBench.groovy` byte-identical in both trees | | Run order | Sequential (no dual-JVM contention); no CPU pinning | | Trials | **T1** HEAD then parent; **T2** parent then HEAD (order-balanced) | ### 4.2 Suites | Suite | Trees | Purpose | |---|---|---| | `SameClassesGuardBench` indy, two trials | both | H1–H3 | | same bench + `-PjmhProfilers=gc` | both | H4 | | `SameClassesGuardMhBench` | HEAD only | H5 | | `org.apache.groovy.bench.CalibrationBench` | both | H6 | | `SameClassesGuardBench -Pindy=false` | both | H7 | | `IndySameClassesGuardTest` | HEAD | H8 | ### 4.3 Statistics - Scores are JMH means; `±` is JMH’s default **99.9% CI**. - Throughput ratio = HEAD / parent (>1 is faster). - Order-balanced result = **geomean of the two trial ratios**. - Treatment geomean = geomean of arity 1–4 ratios. Arity 4 is also reported separately because it is a different physical regime (allocation). - Conclusions rest on **CI separation and structural fingerprints** (32 vs 0 B/op, classic parity), not on ±1% micro-deltas. Raw JSON: `/tmp/groovy-12284-perf/{head,parent}/{t1-indy,t2-indy,gc,mh,classic,cal}/results.json`. --- ## 5. Results ### 5.1 End-to-end indy throughput (primary) Unit: ops/ms, higher is better. Inner work is a single dynamic (or CS) call. | Benchmark | T1 parent | T1 HEAD | T1 H/P | T2 parent | T2 HEAD | T2 H/P | Geomean | Class | |---|---:|---:|---:|---:|---:|---:|---:|---| | `dynamic_arity1` | 657 128 ± 22 407 | 1 026 659 ± 54 598 | 1.562× | 631 074 ± 40 254 | 981 191 ± 94 593 | 1.555× | **1.56×** | treatment | | `dynamic_arity2` | 488 745 ± 16 000 | 914 039 ± 57 519 | 1.870× | 479 100 ± 21 019 | 922 273 ± 62 420 | 1.925× | **1.90×** | treatment | | `dynamic_arity3` | 399 820 ± 29 545 | 790 867 ± 45 065 | 1.978× | 396 106 ± 24 308 | 796 682 ± 50 228 | 2.011× | **2.00×** | treatment | | `dynamic_arity4` | 133 989 ± 5 657 | 626 818 ± 20 809 | 4.678× | 133 960 ± 8 734 | 628 049 ± 33 533 | 4.688× | **4.68×** | treatment | | `dynamic_arity5` | 103 574 ± 5 245 | 117 248 ± 8 420 | 1.132× | 106 893 ± 4 623 | 111 050 ± 3 748 | 1.039× | **1.08×** | neg. control | | `cs_arity2` | 1 590 180 ± 72 701 | 1 586 803 ± 45 686 | 0.998× | 1 571 876 ± 63 590 | 1 287 505 ± 541 959† | 0.819× | 0.90׆ | neg. control | | `cs_arity5` | 1 586 542 ± 124 143 | 1 561 402 ± 47 614 | 0.984× | 1 604 839 ± 75 883 | 1 376 308 ± 410 742† | 0.858× | 0.92׆ | neg. control | † T2 HEAD `@CompileStatic` fork 2 is a dirty sample: raw scores drop to 666 k–993 k ops/ms against fork 1 at ~1.55 M (median 1.48 M). The 99.9% CI is 42% of the mean and is **not usable**. T1 CS is 0.998× / 0.984× with tight CIs; classic CS is 0.99× (§5.4). This is host noise at the `invokevirtual` ceiling, not a treatment effect. Treatment CIs **do not overlap** parent vs HEAD in either trial. Trial-to-trial ratios agree to two decimal places for every treatment row (arity 4: 4.678× and 4.688×). Equivalent ns/op (T1 mean, `1e6 / (ops/ms)`): | Row | Parent ns/op | HEAD ns/op | |---|---:|---:| | `cs_arity2` (ceiling) | 0.63 | 0.63 | | `dynamic_arity1` | 1.52 | 0.97 | | `dynamic_arity2` | 2.05 | 1.09 | | `dynamic_arity3` | 2.50 | 1.26 | | `dynamic_arity4` | 7.46 | 1.60 | | `dynamic_arity5` | 9.65 | 8.53 | HEAD arity 1–4 moves toward the CS ceiling; arity 5 stays on the slow collector plateau. Geomeans: - treatment arity 1–4: **2.29×** - treatment arity 1–3 only (EA already eats the array): **1.81×** - negative controls including noisy T2 CS: 0.97× (misleading; see †) - `dynamic_arity5` alone: **1.08×** (T2 CIs overlap) ### 5.2 Allocation (`gc.alloc.rate.norm`) One trial each tree, same bench, `-PjmhProfilers=gc`. Throughput on this run reproduced §5.1 (arity 4 = 4.71×). | Benchmark | Parent B/op | HEAD B/op | Object[] size (compressed oops) | |---|---:|---:|---| | `cs_arity2` | ≈ 0 | ≈ 0 | none | | `cs_arity5` | ≈ 0 | ≈ 0 | none | | `dynamic_arity1` | ≈ 0 | ≈ 0 | 24 B if allocated; **EA** | | `dynamic_arity2` | ≈ 0 | ≈ 0 | 24 B if allocated; **EA** | | `dynamic_arity3` | ≈ 0 | ≈ 0 | 32 B if allocated; **EA** | | `dynamic_arity4` | **32** | ≈ 0 | 16 + 4×4 = **32** | | `dynamic_arity5` | **40** | **40** | 16 + 5×4 = 36, aligned **40** | ≈ 0 means 10⁻⁶ B/op (profiler noise). This is a **structural fingerprint**, not a 1% delta: - C2 scalar-replaces `asCollector` arrays for `n ≤ 3` on this JDK, so arity 1–3 throughput wins (1.6–2.0×) are **combinator / inlining**, not GC. - At `n = 4` scalar replacement stops. Parent’s 32 B/op matches a live `Object[4]` exactly. HEAD does not allocate. That is why arity 4 is 4.7× rather than ~2×. - At `n = 5` both sides still collect; 40 B/op on both sides. H2 and H4 hold together. ### 5.3 Isolated MethodHandle combinator (intra-HEAD) `SameClassesGuardMhBench`: collector vs specialised, `invokeExact`, no MOP. | Arity | Collector ops/ms | Specialised ops/ms | Spec / collector | |---|---:|---:|---:| | 1 | 174 966 ± 7 386 | 195 310 ± 7 337 | **1.12×** | | 2 | 155 037 ± 8 686 | 183 437 ± 6 705 | **1.18×** | | 4 | 114 803 ± 7 351 | 150 054 ± 8 794 | **1.31×** | The leaf combinator is cheaper, and the gap grows with arity (fatter collector LambdaForm). It is **smaller** than the end-to-end gap because a lone `invokeExact` is a tiny compilation unit: C2 can digest a collector adapter in isolation more easily than the same adapter buried under `SwitchPoint.guardWithTest` + `asType` + the selected method. End-to-end amplifies the same mechanism; it does not contradict it. ### 5.4 Classic call sites (`-Pindy=false`) No indy same-class collector exists on this path (GROOVY-12185: classic cache lives in `groovy-callsite`). | Benchmark | Parent ops/ms | HEAD ops/ms | H/P | |---|---:|---:|---:| | `cs_arity2` | 549 963 ± 31 724 | 545 744 ± 31 083 | **0.99×** | | `cs_arity5` | 535 246 ± 29 755 | 522 777 ± 44 495 | **0.98×** | | `dynamic_arity1` | 108 286 ± 4 382 | 107 651 ± 2 788 | **0.99×** | | `dynamic_arity2` | 103 574 ± 2 311 | 101 707 ± 6 669 | **0.98×** | | `dynamic_arity3` | 100 825 ± 3 053 | 99 051 ± 6 455 | **0.98×** | | `dynamic_arity4` | 95 164 ± 2 119 | 95 148 ± 4 648 | **1.00×** | | `dynamic_arity5` | 91 998 ± 4 086 | 91 612 ± 3 842 | **1.00×** | Every row is 0.98–1.00×. The indy 1.6–4.7× pattern is **absent**. H7 holds: the measured indy win is not a silent compile, CPU, or bench-harness artifact. (Classic CS is slower than indy CS on this JDK because `-Pindy=false` changes bytecode for the whole module; that is a mode effect, not a HEAD-vs-parent effect.) ### 5.5 Calibration rulers (pure Java) AverageTime, µs/op, lower is better. Ratio below is parent/HEAD (speedup). | Ruler | Parent | HEAD | Speedup | |---|---:|---:|---:| | `cpuIntegerOps` | 417.1 ± 15.0 | 415.8 ± 9.1 | **1.00×** | | `memoryPointerChase` | 1544 ± 215 | 1470 ± 121 | **1.05×** | | `allocationChurn` | 95.7 ± 13.3 | 180 ± 412† | unusable | † HEAD `allocationChurn` fork 1 contains 237 / 457 / 84 µs; fork 2 is 102 / 102 / 98 µs, in line with parent. The 99.9% CI is wider than the mean. **`cpuIntegerOps` is the drift check: 1.00×.** Hardware did not move between the two trees. ### 5.6 Correctness ```text ./gradlew :test --tests org.codehaus.groovy.vmplugin.v8.IndySameClassesGuardTest --rerun-tasks ``` **3 passed / 0 failed** (overloads reject null and class change; arity 1–4 handles match collector semantics; a dynamic site relinks when an argument class changes). --- ## 6. Why it is faster ```text Parent hot path HEAD hot path (arity 1–4) recv.foo(a, b, c) │ │ ▼ ▼ SwitchPoint.guardWithTest SwitchPoint.guardWithTest │ │ ▼ ▼ asCollector(Object[].class, 4) SAME_CLASSES_4.bindTo(c0..c3) │ allocates Object[4] (n=4; EA fails) │ four getClass() == tests ▼ ▼ sameClasses(Class[], Object[]) return true → selected method │ ▼ selected method ``` Cost breakdown, matching the measurements: 1. **Simpler LambdaForm (all of 1–4).** `bindTo` of `Class` constants plus a scalar boolean method is a shorter combinator than `asCollector` + array loop. Isolated MH: 1.12–1.31×. End-to-end: 1.6–2.0× at arities where the array is already EA’d, because the collector adapter still pollutes inlining of the surrounding `guardWithTest` chain. 2. **Real allocation at arity 4.** Parent 32 B/op, HEAD 0. Write barriers and GC traffic sit on the hottest path of `recv.foo(x, y, z)`. Combined with (1) this is the 4.7× row. 3. **Arity 5 is deliberately unchanged.** Same collector, same 40 B/op, ~1.08× throughput (CIs overlap on T2). 4. **`@CompileStatic` and classic never install this guard.** Measured parity (T1 CS, all classic rows, CPU ruler). The classic MOP already specialised `MetaClassHelper.sameClasses` for 0–4 arguments. Indy had not. This commit closes that gap on the indy hot path. --- ## 7. Risks and boundaries | Topic | Assessment | |---|---| | H1 treatment win | **Robust.** Two order-balanced trials, ratios stable to two decimals, CIs separated | | H2 arity-5 control | **Holds.** 1.08× geomean; T2 CIs overlap; alloc 40 B/op both sides | | H3 CS control | **Holds on the clean trial and on classic.** T2 HEAD CS has a dirty fork (raw min 666 k vs fork-1 1.55 M); do not read the 0.82× mean as a regression | | H4 allocation | **Holds, and is JDK-specific.** EA ate `n ≤ 3` here (Corretto 25). A JDK that does not scalar-replace `n = 2` would show a B/op drop there too; the combinator win would remain | | H5 combinator | **Holds.** 1.12–1.31× intra-HEAD, growing with arity | | H6 drift | **Holds** on `cpuIntegerOps` (1.00×). Ignore HEAD `allocationChurn` (CI > mean) | | H7 classic | **Holds.** 0.98–1.00× on every row | | H8 correctness | **Holds.** 3/3 tests | | Shared 6-vCPU host | No pinning; order-balancing + classic + CPU ruler bound the noise | | Not claimed | A 2.29× language-wide geomean. Typical calls are arity 1–2 (**1.6–1.9×**). Arity 4 is **4.7×** and is a real `recv.foo(a,b,c)` shape, not a synthetic extreme | | Not covered | JDK 17/21 matrix, megamorphic argument-class oscillation (GROOVY-11152), null-at-link per-slot path | --- ## 8. Hypothesis scorecard | ID | Result | Evidence | |---|---|---| | H1 arity 1–4 faster | **Holds** | 1.56 / 1.90 / 2.00 / 4.68× geomean; CIs separated | | H2 arity 5 parity | **Holds** | 1.08×; T2 overlap; 40 B/op both | | H3 CS parity | **Holds** | T1 0.998 / 0.984×; classic CS 0.99×; T2 CS discarded as dirty fork | | H4 alloc drop on specialised, not on arity 5 | **Holds** | arity 4: 32 → 0 B/op; arity 5: 40 = 40; arity 1–3 already EA | | H5 specialised MH > collector | **Holds** | 1.12 / 1.18 / 1.31× | | H6 CPU ruler ~1× | **Holds** | `cpuIntegerOps` 1.00× | | H7 classic ~1× | **Holds** | 0.98–1.00× all seven rows | | H8 tests green | **Holds** | 3/3 | --- ## 9. Conclusions `8e2b29b` (GROOVY-12284) vs `e801df39` is a **single-commit, single-concern** indy hot-path change. Dedicated benches that the existing suite did not provide — arity as the factor, an arity-5 collector control, CS/classic controls, and bytes/op — show: 1. Linked monomorphic dynamic calls of arity 1–3 are **1.6–2.0×** on HEAD even when the parent’s collector array is already scalar-replaced. 2. Arity 4 is **4.7×** and stops allocating a live `Object[4]` (32 B/op). 3. Arity 5, `@CompileStatic`, and classic bytecode are **unchanged**. 4. The isolated combinator moves 1.12–1.31×; the rest of the end-to-end win is that combinator inlining into the full guard chain. **Performance verification: PASS.** Keep `SameClassesGuardBench` (and the GC profile of `dynamic_arity4` / `dynamic_arity5`) in regular indy JMH regression so a return to a universal `asCollector` guard cannot land silently. --- ## Appendix A — Reproduction ```bash # Correctness ./gradlew :test --tests org.codehaus.groovy.vmplugin.v8.IndySameClassesGuardTest # End-to-end (both trees; copy SameClassesGuardBench.groovy onto the parent) ./gradlew :perf:jmh -PbenchInclude=SameClassesGuardBench -PjmhResultFormat=JSON # Allocation ./gradlew :perf:jmh -PbenchInclude=SameClassesGuardBench -PjmhProfilers=gc -PjmhResultFormat=JSON # Isolated combinator (HEAD only) ./gradlew :perf:jmh -PbenchInclude=SameClassesGuardMhBench -PjmhResultFormat=JSON # Classic negative control ./gradlew :perf:jmh -PbenchInclude=SameClassesGuardBench -Pindy=false -PjmhResultFormat=JSON # CPU ruler ./gradlew :perf:jmh -PbenchInclude=org.apache.groovy.bench.CalibrationBench -PjmhResultFormat=JSON ``` Worktrees used: parent at `/tmp/groovy-12284-parent` (`e801df39`); HEAD as the GROOVY-12284 working tree (`8e2b29b`). Runner: `/tmp/groovy-12284-perf/run.sh`. ## Appendix B — Files Production (`8e2b29b`): - `src/main/java/org/codehaus/groovy/vmplugin/v8/IndyGuardsFiltersAndSignatures.java` - `src/main/java/org/codehaus/groovy/vmplugin/v8/Selector.java` - `src/test/groovy/org/codehaus/groovy/vmplugin/v8/IndySameClassesGuardTest.groovy` Measurement (this verification, not in `8e2b29b`): - `subprojects/performance/src/jmh/groovy/org/apache/groovy/bench/SameClassesGuardBench.groovy` - `subprojects/performance/src/jmh/groovy/org/apache/groovy/bench/SameClassesGuardMhBench.java` - `subprojects/performance/README.adoc` (section *Same-class guards*) ## Appendix C — Object array sizes used in §5.2 64-bit HotSpot, compressed oops + compressed class pointers (Corretto 25 default): array header 16 bytes (`mark` + `klass` + `length`), then `n` compressed refs (4 B), aligned to 8 B. | n | Payload | Aligned | Observed parent B/op | |---|---:|---:|---:| | 4 | 16 + 16 = 32 | 32 | **32** | | 5 | 16 + 20 = 36 | 40 | **40** | --- *Report generated from local JMH A/B measurements on 2026-08-22. Raw JSON: `/tmp/groovy-12284-perf/`.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
