[
https://issues.apache.org/jira/browse/GROOVY-12325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110672#comment-18110672
]
ASF GitHub Bot commented on GROOVY-12325:
-----------------------------------------
daniellansun commented on PR #2852:
URL: https://github.com/apache/groovy/pull/2852#issuecomment-5510968777
# GROOVY-12325 / PR 2852 — performance report
Working note. Not a JIRA comment.
## Setup
| Item | Value |
|---|---|
| Baseline | `03e303e9fdb2715c1be064849984c19a14cab4b8` (`GROOVY-12306:
compatibility note`) |
| Head | working tree on `GROOVY-12325` after review follow-ups |
| JDK | 25.0.4 (Amazon Corretto), 64-bit Server VM |
| JMH | 1.37 |
| Protocol | AverageTime, 1 fork, 3×1 s warmup, 5×1 s measurement, 1 thread |
| Command (head) | `./gradlew :performance:jmh
-PbenchInclude=CachedMethodInvoker` |
| Command (baseline) | worktree at `03e303e` + `CachedMethodBaselineBench`
(`-PbenchInclude=CachedMethodBaseline`) |
Operands are `@Param` fields so HotSpot cannot constant-fold `startsWith` to
`true`.
**Baseline for this change is reflective `CachedMethod.invoke`**, not a Java
direct call. Java is a lower bound only.
## Headline
Steady-state generated `CachedMethod.invoke` is **faster than `03e303e`**.
The kill-switch / pre-generation path is **statistically the same as
`03e303e`** after one hot-path fix (sticky skip when generation is disallowed).
Hidden-class definition is ~80 µs **once per inflated method**; it does not pay
off on a fresh `CachedMethod` that is discarded after a few hundred calls.
## Steady-state (the interned-MOP case)
`String.startsWith`, one `CachedMethod` reused across the whole fork.
| Path | Score (ns/op) | vs `03e303e` |
|---|---|---|
| Java `startsWith` (lower bound) | 7.34 ± 0.85 | 7.64 ± 1.35 on `03e303e` |
| **`03e303e` `CachedMethod.invoke`** | **18.04 ± 2.66** | — |
| HEAD, `disable=true` (reflective) | 17.85 ± 1.31 | ≈ baseline (overlapping
CI) |
| HEAD, `threshold=0` (generated, warmed) | **10.09 ± 2.52** | **~1.8×
faster** |
Megamorphic row (five distinct `CachedMethod`s, one `invoke` site, after
sticky opt):
| Path | Score (ns/op, 5 invokes) |
|---|---|
| `mega_reflective` | 77.0 ± 11.9 |
| `mega_generated` | 57.5 ± 4.6 |
## Generation cost and break-even
Fresh `CachedMethod` per invocation (`@Setup(Level.Invocation)`):
| Path | Score |
|---|---|
| `generate_reflective_fresh` | 0.088 ± 0.012 µs |
| `generate_then_invoke` (`threshold=0`) | 76.6 ± 5.4 µs |
Almost all of the 77 µs is `defineHiddenClass` + instantiate. Saving vs
reflective is ~8 ns/op once the trampoline exists (18 ns → 10 ns). Naïve
break-even:
```
77e3 ns / (18 − 10) ns/op ≈ 10⁴ invokes after generation
```
That is **once per interned `CachedMethod` per VM**, not per call site and
not per request. A method that stays on the MOP for the process lifetime
(typical `MetaClass` entry) recoups quickly. A throwaway `CachedMethod` used
250 times does not — see bursts.
Default threshold 100 is **below** `groovy.indy.optimize.threshold` (1000),
so cold indy can still sit on `doMethodInvoke` when the trampoline appears.
Methods that never reach 101 hits never pay `defineHiddenClass`.
## Bursts (fresh `CachedMethod`, hit counter starts at 0)
| Burst | Default threshold (100) | `disable=true` |
|---|---|---|
| 100 calls (never inflates) | 2.73 ± 0.47 µs | 1.71 ± 0.23 µs |
| 250 calls (inflates at hit 101) | 104 ± 16 µs | 4.06 ± 0.47 µs |
The 250-call default row **includes one hidden-class define**. It is the
cost blackdrag asked for, not the interned-MOP steady state. It is slower than
staying reflective for that short lifetime — expected, and why the threshold
exists.
## Iteration log
The user-required comparison is **head vs `03e303e`**, not generated vs
reflective only.
| Round | Change | `startsWith` generated | `startsWith` disable=true | vs
`03e303e` 18.0 ns |
|---|---|---|---|---|
| 0 | As reviewed (property re-read every invoke when disabled) | 9.98 ns |
**30.6 ns** | disable path regresses |
| 1 | Sticky `invokerAttempted` when `generationAllowed()` is false | 9.70
ns | **18.7 ns** | disable path matches baseline |
| 2 | Cache threshold on the `CachedMethod` (no `Long.getLong` per lukewarm
invoke) | 10.09 ns | **17.85 ns** | generated **1.8×**; disable **no
regression** |
No third round: the success criterion (generated faster than `03e303e`,
disable path not slower within error) is met.
## What we did *not* treat as a regression
- Burst-250 default vs burst-250 disable: measures **define cost on a
throwaway method**, not process-lifetime interned `CachedMethod`.
- `startsWith_generated` vs Java: Java is a lower bound; 10 ns vs 7 ns is
the remaining trampoline / boxing / `invokeinterface` tax.
## Repro
```bash
# head
./gradlew :performance:jmh -PbenchInclude=CachedMethodInvoker
# baseline (separate worktree)
git worktree add /tmp/groovy-baseline
03e303e9fdb2715c1be064849984c19a14cab4b8
# add CachedMethodBaselineBench.java (same shape as startsWith_reflective)
cd /tmp/groovy-baseline && ./gradlew :performance:jmh
-PbenchInclude=CachedMethodBaseline
```
> Speed up CachedMethod.invoke with a generated JIT-constant trampoline
> ---------------------------------------------------------------------
>
> Key: GROOVY-12325
> URL: https://issues.apache.org/jira/browse/GROOVY-12325
> Project: Groovy
> Issue Type: Improvement
> Reporter: Daniel Sun
> Priority: Major
>
> h2. Problem
> {{CachedMethod.invoke}} is the MOP/Java fallback used by {{MetaClassImpl}},
> classic uncompiled call sites, and the default indy cold tier
> ({{invokeColdReflective}} -> {{doMethodInvoke}}).
> That path still calls {{java.lang.reflect.Method.invoke}}. A {{MethodHandle}}
> held in an instance field is in the same performance band. After C2, only a
> JIT-constant callee (direct {{invokevirtual}} / {{invokestatic}} /
> {{invokeinterface}} in generated bytecode, or {{invokeExact}} of a {{static
> final}} / classData handle, or a linked {{invokedynamic}} CallSite) runs like
> a Java direct call.
> Hot monomorphic indy and {{@CompileStatic}} already have that shape.
> {{CachedMethod.invoke}} does not.
> h2. Approach
> After {{groovy.cachedmethod.invoker.threshold}} hits (default 100, below
> {{groovy.indy.optimize.threshold}} of 1000 so cold indy is still on
> {{doMethodInvoke}} when the trampoline appears), install a generated
> {{DirectInvoker}} behind {{CachedMethod.invoke}} only.
> Internal types live in {{org.apache.groovy.internal.runtime.invoke}}
> (japicmp-excluded). Definition reuses {{HiddenClassDefiner}} (GROOVY-12223)
> and {{ClassLoaderForClassArtifacts}}.
> Define order:
> # InvokerFactory nestmate + direct invoke when the member is publicly
> invocable from that class ({{String.startsWith}}).
> # Declaring-class nestmate + direct invoke when {{privateLookupIn}} is
> possible. Private class methods use {{invokevirtual}}; private interface
> methods use {{invokeinterface}} (hidden nestmates do not subclass the host,
> so {{invokespecial}} fails verification).
> # InvokerFactory nestmate + classData {{MethodHandle}} + {{invokeExact}} when
> types are still resolvable from the runtime loader.
> # {{ClassLoaderForClassArtifacts}} when the host loader can resolve
> {{DirectInvoker}} — never for bootstrap types.
> Failures sticky-return {{null}}; {{CachedMethod.invoke}} keeps
> {{Method.invoke}}. Generation is skipped for caller-sensitive and abstract
> methods, Android, native image, and when hidden classes are disabled.
> This is the MOP "Groovy as caller" path ({{makeAccessible}}). Indy continues
> to {{unreflect}} with the call-site {{Lookup}} and must not be fed the
> trampoline.
> h2. Configuration
> {noformat}
> -Dgroovy.cachedmethod.invoker.threshold=100
> -Dgroovy.cachedmethod.invoker.disable=true
> {noformat}
> The existing {{-Dgroovy.hidden.classes.disable=true}} also turns generation
> off.
> h2. Compatibility
> * No change to the {{MetaMethod.invoke}} / {{CachedMethod.invoke}} signatures.
> * Selection (categories, EMC, interceptable, per-instance MetaClass) is
> unchanged; the trampoline is bound to the Java {{Method}}, not to a
> {{MetaMethod}} wrapper.
> * Wrong-argument type on the generated path is {{ClassCastException}}
> (rethrown), matching DGM / {{CallSiteGenerator}}. The reflective path still
> wraps {{IllegalArgumentException}} in {{InvokerInvocationException}}.
> * Opt-out: {{-Dgroovy.cachedmethod.invoker.disable=true}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
