Hi Groovy devs,
I have finally managed to extract a sample from our Groovy framework
that only uses our most basic modules and still exhibits the 2-3x
performance degradation between (non-indy) Groovy 3 and Groovy 4 I
already described in multiple posts a while ago.
The code was built using the latest Groovy releases (3.0.10 / 4.0.1),
you can find the 2 JARs & a test script at:
https://github.com/mgroovy/groovyperformance/tree/trunk/groovy4_performance_public_sample
To check the performance:
1. Open a GroovyConsole window for Groovy 3.0.10
2. Do Script\Add JAR(s) to ClassPath: groovyperformance-3.0.10.jar
3. Load & execute groovysql_performance_groovy4_2_xx_yy_zzzz.groovy
Analogue for Groovy 4.0.1 with groovyperformance-4.0.1.jar.
On 3 different PCs I consistently get results as shown below (each timed
test step executes the same code 100x, each time creating the same
simple SQL GString using our framework):
*Groovy 3.0.10*
0) dt = 7.99 ms
1) dt = 2.01 ms
2) dt = 1.53 ms
3) dt = 1.65 ms
4) dt = 1.36 ms
*Groovy 4.0.1*
0) dt = 16.51 ms
1) dt = 7.14 ms
2) dt = 5.83 ms
3) dt = 6.6 ms
4) dt = 6.24 ms
Throwing away the first loop, which includes framework setup time,
Groovy 3 outperforms Groovy 4 by a factor of about 3 (Note: On my
notebook the factor is closer to 2).
Execution times generally decrease when starting the script multiple
times, but the best dt I have observed on the PC the above measurements
were taken was 3.3ms, whereas Groovy 3.0.10 goes down below 1ms (In any
case this is irrelevant for practical applications, since here a short,
identical code segment will not be executed n x 500-times in a loop).
The performance degradation exhibited here is consistent with the
performance of a) executing our test suite and b) startup & refresh
cycle measurements I did in a web application based on our framwork
code, as described before.
I did not get anywhere trying to use VisualVM trying to pinpoint where
exactly performance was lost, apart from what I described in a previous
post.
Groovy 3 exhibits the same performance degradation when used in indy
mode, so it seems logical to assume that this mechanism is somehow to blame.
Based on this observation I tried to refactor parts of our framework
which create a lot of new objects / do a lot of method calls, but even
short-circuiting these did not lead to any relevant change in performance.
So this is hoping this finally supplies the information that leads to a
solution to this problem by someone that has knowledge about the inner
workings of Groovy & the JVM invokedynamic mechanism, and un-bars us
from ever switching to Groovy 4 G-)
Cheers,
mg
