Folks, as indicated, we performed performance tests of the OpenJDK upgrade PR https://github.com/apache/openwhisk/pull/4706 in our performance test environment (we are using the exact set of Gatling tests that are contributed to Openwhisk).
Though our first impressions looked good, we found severe degradations in the latency and warm invocation tests afterwards. Latency tests dropped to around 50% of the JDK8 results, warm invocation to a staggering 25%. These results were stable with using the original and the updated Gatling image and switching to parallelGC on OpenJDK11. Since GC does not seem to matter, the degradation might be caused by a changed implementation in the OpenJDK 11 (e.g. UUID generation). We did not perform any deeper research. Next we tried to change to use the OpenJ9 JDK11 instead of OpenJDK. Change in common/scala/Dockerfile: " -FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7 +FROM adoptopenjdk/openjdk11-openj9:x86_64-alpine-jdk-11.0.5_10_openj9-0.17.0 " With OpenJ9 in default configuration all performance metrics were looking good again. We did not see any degredation anymore. In the light of these test results, I herewith would like to start the discussion to base the JDK update PR to openJ9 JDK11. Kind Regards, Martin On 2019/12/23 01:05:20, Rodric Rabbah <r...@gmail.com> wrote: > Thanks Martin. What I’m most interested in is knowing what other than the > Gatling tests which we are running in Travis and can run in Jenkins if > anything you’re doing differently so that the project can replicate and > automate it where necessary. > > > -r> > > > On Dec 22, 2019, at 2:34 PM, Martin Henke <ma...@web.de> wrote:> > > > > > Rodric,> > > > > > I forgot to mention. When we find a degradation (and I honestly do not hope > > to see one) and we find the reason, we will of course share the information > > needed to setup tests in the Apache environment to catch those. > > > > > > Regards,> > > Martin> > > > > >> Am 22.12.2019 um 20:29 schrieb Martin Henke <ma...@web.de>:> > >> > > >> Rodric,> > >> > > >> It seems to be natural that the IBM environment differs in machine > >> configuration and network bandwidth and any other parameters from any > >> other OW installation.> > >> > > >> Nevertheless I see value for the project to test the JDK upgrade > >> additionally in our env to identify potential problems. As said, we saw > >> performance degredations before, which were not caught in the Apache > >> environments.> > >> > > >> Regards,> > >> Martin> > >> > > >> P.S I am away from keyboard for the next time and might be slow in > >> answering.> > >> > > >>>> Am 22.12.2019 um 19:33 schrieb Rodric Rabbah <ro...@gmail.com>:> > >>> > > >>> Thanks Martin. Can you share the environment details/how it’s different > >>> from the Jenkins job Vincent set up at one point for the project - so > >>> that any testing important to the project can be done in apache land. > > >>> > > >>> -r> > >>> > > >>>> On Dec 22, 2019, at 9:35 AM, Martin Henke <ma...@web.de> wrote:> > >>>> > > >>>> Rodric,> > >>>> > > >>>> there are no special tests beside the ones that were already contributed > >>>> by Christian from our team. > > >>>> > > >>>> The unknowns are given by our environment. > > >>>> > > >>>> Regards,> > >>>> Martin> > >>>> > > >>>> Von meinem iPhone gesendet> > >>>> > > >>>>> Am 21.12.2019 um 12:45 schrieb Rodric Rabbah <ro...@gmail.com>:> > >>>>> > > >>>>> Martin thanks for the input. Do you or others plan to contribute any > >>>>> of the> > >>>>> performance testing that you're doing/will do to the project - and > >>>>> what> > >>>>> would it take to facilitate this so that we are also increasing the > >>>>> project> > >>>>> velocity?> > >>>>> > > >>>>> -r> > >>>>> > > >>>>>>> On Sat, Dec 21, 2019 at 5:08 AM Martin Henke <ma...@web.de> wrote:> > >>>>>> > > >>>>>> Rodric,> > >>>>>> > > >>>>>> thank you for taking on this dangling task.> > >>>>>> > > >>>>>> Since we saw some severe performance impacts when we tried to move OW > >>>>>> to> > >>>>>> other Java 8 JDK versions> > >>>>>> before, we would like to run the final change through our performance> > >>>>>> tests.> > >>>>>> > > >>>>>> Therefore I would like to ask for a short deferral until midth of > >>>>>> January> > >>>>>> when people have returned> > >>>>>> from their Christmas holidays (you know that we Germans tend to have > >>>>>> long> > >>>>>> holidays :-)).> > >>>>>> > > >>>>>> Regards,> > >>>>>> Martin> > >>>>>> > > >>>>>> > > >>>>>>> On 2019/12/20 21:56:12, Rodric Rabbah <r....@gmail.com> wrote:> > >>>>>>> This PR https://github.com/apache/openwhisk/pull/4706 move openwhisk> > >>>>>> to>> > >>>>>>> using Java 11 for building and running the controller/invoker> > >>>>>> containers.>> > >>>>>>> It has been open since Oct 30 with no comments since Nov 14. It was> > >>>>>> covered>> > >>>>>>> at the Dec 4 community call>> > >>>>>>> > > >>>>>> https://cwiki.apache.org/confluence/display/OPENWHISK/2019-12-04+OW+Tech+Interchange+Meeting+Notes>> > >>>>>> > >>>>>> > > >>>>>>> and there was agreement to merge the PR.>> > >>>>>>> > > >>>>>>> Accordingly, I will merge it next week.>> > >>>>>>> > > >>>>>>> -r>> > >>>>>>> > > >>>> > > >> > > > > >
