daniellansun commented on PR #2852: URL: https://github.com/apache/groovy/pull/2852#issuecomment-5510968777
# GROOVY-12325 / PR 2852 — performance report Working note. Not a JIRA comment. ## Setup | Item | Value | |---|---| | Baseline | `03e303e9fdb2715c1be064849984c19a14cab4b8` (`GROOVY-12306: compatibility note`) | | Head | working tree on `GROOVY-12325` after review follow-ups | | JDK | 25.0.4 (Amazon Corretto), 64-bit Server VM | | JMH | 1.37 | | Protocol | AverageTime, 1 fork, 3×1 s warmup, 5×1 s measurement, 1 thread | | Command (head) | `./gradlew :performance:jmh -PbenchInclude=CachedMethodInvoker` | | Command (baseline) | worktree at `03e303e` + `CachedMethodBaselineBench` (`-PbenchInclude=CachedMethodBaseline`) | Operands are `@Param` fields so HotSpot cannot constant-fold `startsWith` to `true`. **Baseline for this change is reflective `CachedMethod.invoke`**, not a Java direct call. Java is a lower bound only. ## Headline Steady-state generated `CachedMethod.invoke` is **faster than `03e303e`**. The kill-switch / pre-generation path is **statistically the same as `03e303e`** after one hot-path fix (sticky skip when generation is disallowed). Hidden-class definition is ~80 µs **once per inflated method**; it does not pay off on a fresh `CachedMethod` that is discarded after a few hundred calls. ## Steady-state (the interned-MOP case) `String.startsWith`, one `CachedMethod` reused across the whole fork. | Path | Score (ns/op) | vs `03e303e` | |---|---|---| | Java `startsWith` (lower bound) | 7.34 ± 0.85 | 7.64 ± 1.35 on `03e303e` | | **`03e303e` `CachedMethod.invoke`** | **18.04 ± 2.66** | — | | HEAD, `disable=true` (reflective) | 17.85 ± 1.31 | ≈ baseline (overlapping CI) | | HEAD, `threshold=0` (generated, warmed) | **10.09 ± 2.52** | **~1.8× faster** | Megamorphic row (five distinct `CachedMethod`s, one `invoke` site, after sticky opt): | Path | Score (ns/op, 5 invokes) | |---|---| | `mega_reflective` | 77.0 ± 11.9 | | `mega_generated` | 57.5 ± 4.6 | ## Generation cost and break-even Fresh `CachedMethod` per invocation (`@Setup(Level.Invocation)`): | Path | Score | |---|---| | `generate_reflective_fresh` | 0.088 ± 0.012 µs | | `generate_then_invoke` (`threshold=0`) | 76.6 ± 5.4 µs | Almost all of the 77 µs is `defineHiddenClass` + instantiate. Saving vs reflective is ~8 ns/op once the trampoline exists (18 ns → 10 ns). Naïve break-even: ``` 77e3 ns / (18 − 10) ns/op ≈ 10⁴ invokes after generation ``` That is **once per interned `CachedMethod` per VM**, not per call site and not per request. A method that stays on the MOP for the process lifetime (typical `MetaClass` entry) recoups quickly. A throwaway `CachedMethod` used 250 times does not — see bursts. Default threshold 100 is **below** `groovy.indy.optimize.threshold` (1000), so cold indy can still sit on `doMethodInvoke` when the trampoline appears. Methods that never reach 101 hits never pay `defineHiddenClass`. ## Bursts (fresh `CachedMethod`, hit counter starts at 0) | Burst | Default threshold (100) | `disable=true` | |---|---|---| | 100 calls (never inflates) | 2.73 ± 0.47 µs | 1.71 ± 0.23 µs | | 250 calls (inflates at hit 101) | 104 ± 16 µs | 4.06 ± 0.47 µs | The 250-call default row **includes one hidden-class define**. It is the cost blackdrag asked for, not the interned-MOP steady state. It is slower than staying reflective for that short lifetime — expected, and why the threshold exists. ## Iteration log The user-required comparison is **head vs `03e303e`**, not generated vs reflective only. | Round | Change | `startsWith` generated | `startsWith` disable=true | vs `03e303e` 18.0 ns | |---|---|---|---|---| | 0 | As reviewed (property re-read every invoke when disabled) | 9.98 ns | **30.6 ns** | disable path regresses | | 1 | Sticky `invokerAttempted` when `generationAllowed()` is false | 9.70 ns | **18.7 ns** | disable path matches baseline | | 2 | Cache threshold on the `CachedMethod` (no `Long.getLong` per lukewarm invoke) | 10.09 ns | **17.85 ns** | generated **1.8×**; disable **no regression** | No third round: the success criterion (generated faster than `03e303e`, disable path not slower within error) is met. ## What we did *not* treat as a regression - Burst-250 default vs burst-250 disable: measures **define cost on a throwaway method**, not process-lifetime interned `CachedMethod`. - `startsWith_generated` vs Java: Java is a lower bound; 10 ns vs 7 ns is the remaining trampoline / boxing / `invokeinterface` tax. ## Repro ```bash # head ./gradlew :performance:jmh -PbenchInclude=CachedMethodInvoker # baseline (separate worktree) git worktree add /tmp/groovy-baseline 03e303e9fdb2715c1be064849984c19a14cab4b8 # add CachedMethodBaselineBench.java (same shape as startsWith_reflective) cd /tmp/groovy-baseline && ./gradlew :performance:jmh -PbenchInclude=CachedMethodBaseline ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
