Hi Paul & Jochen.
I have uploaded some repeated test execution timings & VisualVM hot spot
screenshots from a suitable looking test, comparing Groovy 3.0.9
(non-INDY) with Groovy 4.0.0-beta-1:
https://issues.apache.org/jira/browse/GROOVY-10307
Cheers,
mg
On 15/10/2021 10:21, Paul King wrote:
Thanks for the clarification. I didn't fully understand your scenario.
Both parsing and the Indy bytecode have been areas where we've had
feedback about performance. A bunch of projects have been impacted by
both. In your case it will be bytecode improvements that are needed.
We have made some good improvements for some Indy scenarios but have
more to look at.
Cheers, Paul.
On Fri, 15 Oct 2021, 10:08 am MG, <mg...@arscreat.com> wrote:
Hi Paul & Stephen,
unless I am missing something some miscommunication seems to have
happened: We compile our complete framework and application code
(all written in Groovy, mixed @CompileDynamic/@TypeChecked and
@CompileStatic) using IntelliJ Groovy build process, then run /
deploy the compiled class files.
We do not use a Groovy based DSL, nor do we execute Groovy scripts
during execution, so I do not see where (Parrot) parsing
performance would come into play... ?-)
The performance degradation occurs:
1. In e.g. one of our web applications (Tomcat/Vaadin/Ebean plus
framework generated SQL) when the class files deployed on the
server have been built using Groovy 4.0.0-beta-1 instead of
Groovy 3.0.9 (non-INDY)
2. When we execute our test suite (which also contains no scripts
/ dynamic code execution), the tests do not take longer to
build, they take longer to finish (overall the test suite
takes about 120 min when using G4, compared to about 50 min
with G3)
Ticket: https://issues.apache.org/jira/browse/GROOVY-10307
Cheers,
mg
On 14/10/2021 13:32, Paul King wrote:
Hi mg,
Antlr4 performance is something we want to work much more on but
it isn't an easy task and we have already picked off some of the
low-hanging fruit.
It is probably worth creating a Jira ticket for this. We tend to
progress much more efficiently on well-defined issues than broad
ones, but perhaps we need a broad one and can carve off some
specific points.
In the meantime, does the performance guide doco help at all:
https://github.com/apache/groovy/blob/master/src/spec/doc/performance-guide.adoc
In particular, does trying to use the SLL mode help for your
scenario. It generally only works for simple DSL code but offers
a great parsing performance boost when it does.
The caching parameters and parallel options are also potential
things to try but gains vary widely depending on your scenario.
Cheers, Paul.
On Thu, Oct 14, 2021 at 9:24 AM MG <mg...@arscreat.com> wrote:
Hi,
we have spiked using Groovy 4.0.0-beta-1 with our Groovy
framework, to see if the GString improvements would lead to a
speedup in our SQL generation code (typically this is
irrelevant, and performance is solely decided by SQL
optimizations, but we have recently come across a few cases
in practice where performance was not bound SQL execution but
by the SQL generating code itself).
We had to find workarounds for a few problems (tickets
coming), but overall it was much faster/hassle free to get
Groovy 4 up and running compared to our recent switch to from
Groovy 2.5 to 3, which was nice :-)
Alas performance-wise what we surprisingly found was, that
the performance of our main web application dropped by a
factor of 2 (table refresh) to 3 (startup).
Executing our test suite showed a similar picture. Since no
immediate source for this performance drop emerged, we
checked the performance of Groovy 3 Indy, more to rule out
that the performance reduction had something to do with
invokedynamic;
as can be seen in the table below, to our surprise
performance degradiation of Groovy 3.0.9 with full
invokedynamic (Groovy JARs & IntelliJ compiler
switch/checkbox active) was in fact in most cases close to
the one seen with Groovy 4.0.0-beta-1, pointing to
invokedynamic as the potential cause.
Right now it looks to us, as if Groovy with invokedynamic for
us is just "leaking performance everywhere", with no clear
source.
Speed changes range from about 0.6 (i.e. a speedup) to about
5.0, with a large bias towards a slowdown by a factor of 2.0
to 2.5. The overall time of the test suite increased by a
factor of 2.37 (G3 Indy) and 2.45 (G4) respectively.
Any ideas what could be the cause of this unexpected slowdown
or where we should put our focus in analyzing this, to create
a test case independent of our framwork ?
Cheers,
mg
*Groovy 3.0.9 [s]* *Groovy 3.0.9 INDY [s]* *Groovy
4.0.0-beta-1 [s]* *G3INDY/G3 Ratio* *G4/G3 Ratio*
3038 7200 7440 2.37 2.45
160.146 482.584 467.058 3.01 2.92
115.595 388.332 387.82 3.36 3.35
88.955 141.595 142.205 1.59 1.60
94.743 139.676 138.553 1.47 1.46
65.338 109.143 130.918 1.67 2.00
117.108 129.789 116.748 1.11 1.00
74.182 115.142 110.801 1.55 1.49
30.653 104.733 101.138 3.42 3.30
24.71 80.171 72.541 3.24 2.94
22.586 59.167 60.043 2.62 2.66
41.302 46.675 46.682 1.13 1.13
5.34 51.456 46.045 9.64 8.62
14.066 41.788 43.987 2.97 3.13
20.535 42.441 42.571 2.07 2.07
19.582 43.338 42.493 2.21 2.17
19.937 43.235 42.361 2.17 2.12
23.118 40.245 42.206 1.74 1.83
14.777 34.526 33.725 2.34 2.28
15.23 31.757 33.302 2.09 2.19
15.813 34.246 30.699 2.17 1.94
18.743 33.892 30.436 1.81 1.62
14.855 30.105 27.19 2.03 1.83
10.775 28.514 25.813 2.65 2.40
10.571 26.344 23.289 2.49 2.20
8.084 21.218 21.675 2.62 2.68
9.155 21.571 21.299 2.36 2.33
5.454 18.296 19.425 3.35 3.56
12.987 18.841 18.537 1.45 1.43
10.928 17.93 17.987 1.64 1.65
10.501 9.792 14.374 0.93 1.37
9.455 14.174 14.038 1.50 1.48
3.19 12.39 13.79 3.88 4.32
10.006 13.961 13.264 1.40 1.33
22.858 14.48 13.073 0.63 0.57
4.592 10.176 12.777 2.22 2.78
8.991 12.932 11.908 1.44 1.32
17.245 10.591 10.746 0.61 0.62
2.331 11.443 10.497 4.91 4.50
11.159 10.205 9.911 0.91 0.89
2.362 20.721 9.744 8.77 4.13
9.369 7.543 9.097 0.81 0.97
1.845 9.353 8.483 5.07 4.60
4.647 5.537 8.065 1.19 1.74
8.758 8.806 7.664 1.01 0.88
10.359 9.95 7.657 0.96 0.74
8.954 8.764 7.38 0.98 0.82
2.424 7.37 6.941 3.04 2.86
6.75 6.771 6.84 1.00 1.01
7.476 5.179 6.679 0.69 0.89
1.873 6.785 6.64 3.62 3.55
8.13 5.896 6.564 0.73 0.81
6.892 5.189 6.385 0.75 0.93
5.91 6.153 6.2 1.04 1.05
1.318 4.944 5.585 3.75 4.24
