[
https://issues.apache.org/jira/browse/GROOVY-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18055591#comment-18055591
]
ASF GitHub Bot commented on GROOVY-10307:
-----------------------------------------
jamesfredley commented on PR #2374:
URL: https://github.com/apache/groovy/pull/2374#issuecomment-3827499847
**Test Date:** January 30, 2026
**Runs per Configuration:** 3
**Environment:** Windows 11, Java 17.0.18, 4GB heap, 20 CPUs
---
## Master Comparison Table
All values are **averages across 3 runs**. Times in milliseconds unless
noted. **Bold** indicates best performance.
### Loop Benchmarks (ComprehensiveBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Loop: each + toString | 33.09 ms | 33.23 ms | **32.70 ms** |
| Loop: collect | 52.62 ms | **52.22 ms** | 54.59 ms |
| Loop: findAll | **113.08 ms** | 116.15 ms | 114.58 ms |
---
### Method Invocation Benchmarks (ComprehensiveBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Method: simple instance | 8.77 ms | **8.55 ms** | 8.94 ms |
| Method: with params | **10.00 ms** | 10.08 ms | 10.52 ms |
| Method: static | 9.19 ms | **7.73 ms** | 8.25 ms |
| Method: polymorphic | **163.35 ms** | 1.70 s | 1.88 s |
---
### Closure Benchmarks (ComprehensiveBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Closure: creation + call | **21.45 ms** | 24.89 ms | 24.54 ms |
| Closure: reused | 19.44 ms | 19.86 ms | **18.59 ms** |
| Closure: nested | 39.25 ms | **37.51 ms** | 38.22 ms |
| Closure: curried | **154.37 ms** | 159.80 ms | 161.86 ms |
---
### Other Benchmarks (ComprehensiveBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Property: read/write | 20.80 ms | **18.70 ms** | 19.33 ms |
| Collection: each | 108.71 ms | **107.19 ms** | 110.03 ms |
| Collection: collect | 121.07 ms | **116.21 ms** | 121.74 ms |
| Collection: inject | **127.13 ms** | 132.85 ms | 139.18 ms |
| GString: simple | **101.22 ms** | 102.89 ms | 105.09 ms |
| GString: multi-value | **112.55 ms** | 116.85 ms | 117.61 ms |
---
### Method Invocation Benchmarks (MethodInvocationBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Simple instance method | 7.23 ms | **6.59 ms** | 8.22 ms |
| Method with parameters | **7.76 ms** | 8.08 ms | 8.31 ms |
| Method with object param | 11.41 ms | **10.74 ms** | 11.10 ms |
| Static method | **1.24 ms** | 3.47 ms | 3.40 ms |
| Static method with params | **1.17 ms** | 7.75 ms | 8.11 ms |
| Monomorphic call site | **46.89 ms** | 130.65 ms | 126.87 ms |
| Polymorphic call site | **382.26 ms** | 3.52 s | 3.85 s |
| Interface method | 6.34 ms | 3.53 ms | **3.42 ms** |
| Dynamic typed calls | 5.86 ms | **3.04 ms** | 3.18 ms |
| Property access | 27.52 ms | **21.49 ms** | 22.58 ms |
| Method reference | 34.40 ms | **31.01 ms** | 31.25 ms |
| GString method | **190.04 ms** | 193.28 ms | 194.07 ms |
---
### Closure Benchmarks (ClosureBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Simple closure creation | 32.52 ms | **30.21 ms** | 31.72 ms |
| Closure reuse | 22.79 ms | **19.86 ms** | 21.15 ms |
| Multi-param closure | 39.00 ms | **38.34 ms** | 39.22 ms |
| Closure with capture | 23.19 ms | **19.94 ms** | 20.34 ms |
| Closure modify capture | 14.30 ms | **12.55 ms** | 13.45 ms |
| Closure delegation | 40.33 ms | **27.24 ms** | 31.99 ms |
| Nested closures | 59.92 ms | **56.24 ms** | 59.72 ms |
| Curried closure | 291.59 ms | 296.21 ms | **289.39 ms** |
| Right curried closure | 188.59 ms | **187.98 ms** | 189.34 ms |
| Closure composition | 64.37 ms | **57.93 ms** | 59.37 ms |
| Closure as parameter | 42.64 ms | **40.58 ms** | 41.48 ms |
| Closure spread | **177.89 ms** | 2.10 s | 2.10 s |
| Closure.call() | 25.69 ms | **22.22 ms** | 25.49 ms |
| Closure trampoline | 55.88 ms | **50.72 ms** | 55.22 ms |
| each with closure | 28.55 ms | **26.44 ms** | 28.72 ms |
| collect with closure | 27.95 ms | **26.13 ms** | 28.00 ms |
| findAll with closure | **35.72 ms** | 37.14 ms | 42.34 ms |
| inject with closure | 33.69 ms | **33.41 ms** | 36.29 ms |
---
### Loop Benchmarks (LoopsBenchmark)
| Benchmark | NO-INDY | Optimized-INDY | Baseline-INDY |
|-----------|---------|----------------|---------------|
| Original: each + toString | **43.65 ms** | 45.55 ms | 45.90 ms |
| Simple: each only | **38.93 ms** | 42.03 ms | 42.67 ms |
| Closure call | 22.63 ms | **19.77 ms** | 21.99 ms |
| Method call | 8.14 ms | 6.22 ms | **5.39 ms** |
| Nested loops | **70.82 ms** | 72.94 ms | 77.47 ms |
| Loop with collect | **89.20 ms** | 90.26 ms | 90.67 ms |
| Loop with findAll | 209.40 ms | **204.75 ms** | 209.98 ms |
---
### Call Site Invalidation (Critical for GROOVY-10307)
| Metric | NO-INDY | Optimized-INDY | Baseline-INDY |
|--------|---------|----------------|---------------|
| Metaclass change ratio | **1.27x** | 69.4x | 102.6x |
| Baseline time (no changes) | 6.83 ms | 7.44 ms | **5.18 ms** |
| With changes time | **8.51 ms** | 439.54 ms | 531.72 ms |
---
## Summary Table: Key Metrics
| Metric | NO-INDY | Optimized-INDY | Baseline-INDY |
|--------|---------|----------------|---------------|
| Polymorphic call site | **382 ms** | 3.52 s | 3.85 s |
| Metaclass change ratio | **1.27x** | 69.4x | 102.6x |
| Static method | **1.24 ms** | 3.47 ms | 3.40 ms |
| Monomorphic call site | **46.89 ms** | 130.65 ms | 126.87 ms |
| Closure spread | **178 ms** | 2.10 s | 2.10 s |
| Simple instance method | 7.23 ms | **6.59 ms** | 8.22 ms |
| Closure delegation | 40.33 ms | **27.24 ms** | 31.99 ms |
| Property access | 27.52 ms | **21.49 ms** | 22.58 ms |
| Interface method | 6.34 ms | 3.53 ms | **3.42 ms** |
| Dynamic typed calls | 5.86 ms | **3.04 ms** | 3.18 ms |
---
> Groovy 4 runtime performance on average 2.4x slower than Groovy 3
> -----------------------------------------------------------------
>
> Key: GROOVY-10307
> URL: https://issues.apache.org/jira/browse/GROOVY-10307
> Project: Groovy
> Issue Type: Bug
> Components: bytecode, performance
> Affects Versions: 4.0.0-beta-1, 3.0.9
> Environment: OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9
> (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> WIN10 (tests) / REL 8 (web application)
> IntelliJ 2021.2
> Reporter: mgroovy
> Priority: Major
> Attachments: groovy_3_0_9_gc.png, groovy_3_0_9_loop2.png,
> groovy_3_0_9_loop4.png, groovy_3_0_9_mem.png, groovy_4_0_0_b1_loop2.png,
> groovy_4_0_0_b1_loop4.png, groovy_4_0_0_b1_loop4_gc.png,
> groovy_4_0_0_b1_loop4_mem.png,
> groovysql_performance_groovy4_2_xx_yy_zzzz.groovy, loops.groovy,
> profile3.txt, profile4-loops.txt, profile4.txt, profile4d.txt
>
>
> Groovy 4.0.0-beta-1 runtime performance in our framework is on average 2 to 3
> times slower compared to using Groovy 3.0.9 (regular i.e. non-INDY)
> * Our complete framework and application code is completely written in
> Groovy, spread over multiple IntelliJ modules
> ** mixed @CompileDynamic/@TypeChecked and @CompileStatic
> ** No Java classes left in project, i.e. no cross compilation occurs
> * We build using IntelliJ 2021.2 Groovy build process, then run / deploy the
> compiled class files
> ** We do _not_ use a Groovy based DSL, nor do we execute Groovy scripts
> during execution
> * Performance degradation when using Groovy 4.0.0-beta-1 instead of Groovy
> 3.0.9 (non-INDY):
> ** The performance of the largest of our web applications has dropped 3x
> (startup) / 2x (table refresh) respectively
> *** Stack: Tomcat/Vaadin/Ebean plus framework generated SQL
> ** Our test suite runs about 2.4 times as long as before (120 min when using
> G4, compared to about 50 min with G3)
> *** JUnit 5
> *** test suite also contains no scripts / dynamic code execution
> *** Individual test performance varies: A small number of tests runs faster,
> but the majority is slower, with some extreme cases taking nearly 10x as long
> to finish
> * Using Groovy 3.0.9 INDY displays nearly identical performance degradation,
> so it seems that the use of invoke dynamic is somehow at fault
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
