[
https://issues.apache.org/jira/browse/GROOVY-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18055522#comment-18055522
]
ASF GitHub Bot commented on GROOVY-10307:
-----------------------------------------
jonnybot0 commented on PR #2374:
URL: https://github.com/apache/groovy/pull/2374#issuecomment-3825526977
I'm also well outside my expertise here, but so long as we're doing this,
I'd be curious to see how this runs against the performance tests we have,
compared to baseline.
https://ci-builds.apache.org/job/Groovy/job/Groovy%204%20performance%20tests/
has a Jenkins build that runs these. That's no good for running against a
community repository, but perhaps we could push this up to a different branch
and run it. Little outside my usual wheelhouse, so I'd like the nod from
@paulk-asert, @blackdrag, or someone else on the PMC before I just run it
willy-nilly.
I tried running `./gradlew :performance:performanceTests` on my local
machine with Java 17 on this branch:
```
Groovy 4.0.7 Average 376.26ms ± 64.42ms
Groovy current Average 376.87ms ± 63.75ms (0.16% slower)
Groovy 3.0.14 Average 443.74ms ± 131.81ms (17.93% slower)
Groovy 2.5.20 Average 451.40999999999997ms ± 138.82ms (19.97% slower)
```
That seems comparable, so my assumption is this hasn't hurt the most basic
performance measures we do. Main reason I'd want CI confirmation is to make
sure it wasn't a quirk of my machine.
> Groovy 4 runtime performance on average 2.4x slower than Groovy 3
> -----------------------------------------------------------------
>
> Key: GROOVY-10307
> URL: https://issues.apache.org/jira/browse/GROOVY-10307
> Project: Groovy
> Issue Type: Bug
> Components: bytecode, performance
> Affects Versions: 4.0.0-beta-1, 3.0.9
> Environment: OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9
> (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> WIN10 (tests) / REL 8 (web application)
> IntelliJ 2021.2
> Reporter: mgroovy
> Priority: Major
> Attachments: groovy_3_0_9_gc.png, groovy_3_0_9_loop2.png,
> groovy_3_0_9_loop4.png, groovy_3_0_9_mem.png, groovy_4_0_0_b1_loop2.png,
> groovy_4_0_0_b1_loop4.png, groovy_4_0_0_b1_loop4_gc.png,
> groovy_4_0_0_b1_loop4_mem.png,
> groovysql_performance_groovy4_2_xx_yy_zzzz.groovy, loops.groovy,
> profile3.txt, profile4-loops.txt, profile4.txt, profile4d.txt
>
>
> Groovy 4.0.0-beta-1 runtime performance in our framework is on average 2 to 3
> times slower compared to using Groovy 3.0.9 (regular i.e. non-INDY)
> * Our complete framework and application code is completely written in
> Groovy, spread over multiple IntelliJ modules
> ** mixed @CompileDynamic/@TypeChecked and @CompileStatic
> ** No Java classes left in project, i.e. no cross compilation occurs
> * We build using IntelliJ 2021.2 Groovy build process, then run / deploy the
> compiled class files
> ** We do _not_ use a Groovy based DSL, nor do we execute Groovy scripts
> during execution
> * Performance degradation when using Groovy 4.0.0-beta-1 instead of Groovy
> 3.0.9 (non-INDY):
> ** The performance of the largest of our web applications has dropped 3x
> (startup) / 2x (table refresh) respectively
> *** Stack: Tomcat/Vaadin/Ebean plus framework generated SQL
> ** Our test suite runs about 2.4 times as long as before (120 min when using
> G4, compared to about 50 min with G3)
> *** JUnit 5
> *** test suite also contains no scripts / dynamic code execution
> *** Individual test performance varies: A small number of tests runs faster,
> but the majority is slower, with some extreme cases taking nearly 10x as long
> to finish
> * Using Groovy 3.0.9 INDY displays nearly identical performance degradation,
> so it seems that the use of invoke dynamic is somehow at fault
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
