Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
> On Sep 4, 2020, at 1:34 PM, David Blevins wrote: > >> On Sep 4, 2020, at 11:10 AM, Jonathan Gallimore >> wrote: >> >> This is the specific part of the code where the Geronimo OpenAPI service is >> filtered out: >> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/cdi/OptimizedLoaderService.java#L164-L165 >> . >> >> I hope that helps. > > It definitely did. I found a way to get it to pass. I don't know if there > are other consequences. > > It's effectively these three lines of code (first link) added in this commit > (second link) > > - > https://github.com/apache/tomee/blob/142f035e454b43192c51db7a66b9395982502d47/container/openejb-core/src/main/java/org/apache/openejb/cdi/OptimizedLoaderService.java#L159-L161 > - > https://github.com/apache/tomee/commit/1ebb0ee181622cd101df64a82f8af3bcb27211bf#diff-da64938534b0686bfc3d264149fac1f2R106-R108 > > There's no test or javadoc and the commit message says its for jcache. > > I'm going to try to get the build green again and then yank. > Ok, yanked those lines and added in the MicroProfile, Plus and Plume examples. Build looks happy! - https://ci.apache.org/builders/tomee-trunk-ubuntu-jvm8/builds/1426 These look good to go. From here I think it's just a matter of starting to use this more and more and see what if any issues arise. Might be nice to have an Arquillian version and potentially deprecate/remove the old incomplete embedded adapter. -David smime.p7s Description: S/MIME cryptographic signature
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
> On Sep 4, 2020, at 11:10 AM, Jonathan Gallimore > wrote: > > This is the specific part of the code where the Geronimo OpenAPI service is > filtered out: > https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/cdi/OptimizedLoaderService.java#L164-L165 > . > > I hope that helps. It definitely did. I found a way to get it to pass. I don't know if there are other consequences. It's effectively these three lines of code (first link) added in this commit (second link) - https://github.com/apache/tomee/blob/142f035e454b43192c51db7a66b9395982502d47/container/openejb-core/src/main/java/org/apache/openejb/cdi/OptimizedLoaderService.java#L159-L161 - https://github.com/apache/tomee/commit/1ebb0ee181622cd101df64a82f8af3bcb27211bf#diff-da64938534b0686bfc3d264149fac1f2R106-R108 There's no test or javadoc and the commit message says its for jcache. I'm going to try to get the build green again and then yank. -David > > Jon > > On Fri, 4 Sep 2020, 18:40 David Blevins, wrote: > >> If the commit did have some means to handle the duplicate classes, it's >> the exact insight I'm looking for even if it isn't working. >> >> Even knowing that there was something that was supposed to handle it is >> useful as that means I just need to find what was supposed to work and then >> figure out why it doesn't. >> >> If you didn't have to do anything at all and things just magically worked, >> then the starting point is less clear. >> >> >> -David >> >>> On Sep 4, 2020, at 10:12 AM, Jonathan Gallimore < >> jonathan.gallim...@gmail.com> wrote: >>> >>> It definitely sounds like fun :) >>> >>> On Fri, 4 Sep 2020, 18:10 David Blevins, wrote: >>> > On Sep 4, 2020, at 8:48 AM, Jonathan Gallimore < jonathan.gallim...@gmail.com> wrote: > > Off the top of my head - I don't know. I'll run your test and dig >> through > over the weekend. No worries, I'll continue digging. I thought there might be a "I had >> that issue and did the thing with the thing, you probably need to adjust it." If nothing jumps out, no need to dig unless that sounds fun :) -David > > On Fri, Sep 4, 2020 at 4:16 AM David Blevins > wrote: > >> Ok, so here's the error that keeps the TomEE MicroProfile, Plus and Plume >> examples from booting. >> >> It appears we get an AmbiguousResolutionException from OpenWebBeans >> because we have these two beans getting picked up and they conflict >> with >> each other: >> >> >> - TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API >> Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, >> org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, >> >> >> org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], >> >> >> Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] >> from >> >> >> jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar >> !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class >> >> >> - GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API >> Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, >> >> >> org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], >> >> >> Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] >> from >> >> >> jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar >> >> >> !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class >> >> >> It looks like TomEEOpenAPIExtension was added in this commit: >> >> - >> >> https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd >> >> Jon, looks like you added TomEEOpenAPIExtension. Is there any trick >> you >> needed to do to avoid the above conflict in our standalone servers? >> >> If anyone has any insight, it's very appreciated. This can be reproduced >> with this example: >> >> - >> >> https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile >> >> >> >> -David >> >> >> >> smime.p7s Description: S/MIME cryptographic signature
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
This is the specific part of the code where the Geronimo OpenAPI service is filtered out: https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/cdi/OptimizedLoaderService.java#L164-L165 . I hope that helps. I'm happy to look at your test over the weekend. Jon On Fri, 4 Sep 2020, 18:40 David Blevins, wrote: > If the commit did have some means to handle the duplicate classes, it's > the exact insight I'm looking for even if it isn't working. > > Even knowing that there was something that was supposed to handle it is > useful as that means I just need to find what was supposed to work and then > figure out why it doesn't. > > If you didn't have to do anything at all and things just magically worked, > then the starting point is less clear. > > > -David > > > On Sep 4, 2020, at 10:12 AM, Jonathan Gallimore < > jonathan.gallim...@gmail.com> wrote: > > > > It definitely sounds like fun :) > > > > On Fri, 4 Sep 2020, 18:10 David Blevins, wrote: > > > >> > >>> On Sep 4, 2020, at 8:48 AM, Jonathan Gallimore < > >> jonathan.gallim...@gmail.com> wrote: > >>> > >>> Off the top of my head - I don't know. I'll run your test and dig > through > >>> over the weekend. > >> > >> No worries, I'll continue digging. I thought there might be a "I had > that > >> issue and did the thing with the thing, you probably need to adjust it." > >> > >> If nothing jumps out, no need to dig unless that sounds fun :) > >> > >> > >> -David > >> > >> > >>> > >>> On Fri, Sep 4, 2020 at 4:16 AM David Blevins > >>> wrote: > >>> > Ok, so here's the error that keeps the TomEE MicroProfile, Plus and > >> Plume > examples from booting. > > It appears we get an AmbiguousResolutionException from OpenWebBeans > because we have these two beans getting picked up and they conflict > with > each other: > > > - TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API > Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, > org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, > > > org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], > > > Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] > from > > > >> > jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar > > >> !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class > > > - GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API > Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, > > > org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], > > > Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] > from > > > >> > jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar > > > >> > !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class > > > It looks like TomEEOpenAPIExtension was added in this commit: > > - > > >> > https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd > > Jon, looks like you added TomEEOpenAPIExtension. Is there any trick > you > needed to do to avoid the above conflict in our standalone servers? > > If anyone has any insight, it's very appreciated. This can be > >> reproduced > with this example: > > - > > >> > https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile > > > > -David > > > >> > >> > >
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
If the commit did have some means to handle the duplicate classes, it's the exact insight I'm looking for even if it isn't working. Even knowing that there was something that was supposed to handle it is useful as that means I just need to find what was supposed to work and then figure out why it doesn't. If you didn't have to do anything at all and things just magically worked, then the starting point is less clear. -David > On Sep 4, 2020, at 10:12 AM, Jonathan Gallimore > wrote: > > It definitely sounds like fun :) > > On Fri, 4 Sep 2020, 18:10 David Blevins, wrote: > >> >>> On Sep 4, 2020, at 8:48 AM, Jonathan Gallimore < >> jonathan.gallim...@gmail.com> wrote: >>> >>> Off the top of my head - I don't know. I'll run your test and dig through >>> over the weekend. >> >> No worries, I'll continue digging. I thought there might be a "I had that >> issue and did the thing with the thing, you probably need to adjust it." >> >> If nothing jumps out, no need to dig unless that sounds fun :) >> >> >> -David >> >> >>> >>> On Fri, Sep 4, 2020 at 4:16 AM David Blevins >>> wrote: >>> Ok, so here's the error that keeps the TomEE MicroProfile, Plus and >> Plume examples from booting. It appears we get an AmbiguousResolutionException from OpenWebBeans because we have these two beans getting picked up and they conflict with each other: - TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] from >> jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar >> !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class - GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] from >> jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar >> !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class It looks like TomEEOpenAPIExtension was added in this commit: - >> https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd Jon, looks like you added TomEEOpenAPIExtension. Is there any trick you needed to do to avoid the above conflict in our standalone servers? If anyone has any insight, it's very appreciated. This can be >> reproduced with this example: - >> https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile -David >> >>
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
It definitely sounds like fun :) On Fri, 4 Sep 2020, 18:10 David Blevins, wrote: > > > On Sep 4, 2020, at 8:48 AM, Jonathan Gallimore < > jonathan.gallim...@gmail.com> wrote: > > > > Off the top of my head - I don't know. I'll run your test and dig through > > over the weekend. > > No worries, I'll continue digging. I thought there might be a "I had that > issue and did the thing with the thing, you probably need to adjust it." > > If nothing jumps out, no need to dig unless that sounds fun :) > > > -David > > > > > > On Fri, Sep 4, 2020 at 4:16 AM David Blevins > > wrote: > > > >> Ok, so here's the error that keeps the TomEE MicroProfile, Plus and > Plume > >> examples from booting. > >> > >> It appears we get an AmbiguousResolutionException from OpenWebBeans > >> because we have these two beans getting picked up and they conflict with > >> each other: > >> > >> > >> - TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API > >> Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, > >>org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, > >> > >> org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], > >> > >> Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] > >> from > >> > >> > jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar > >> > !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class > >> > >> > >> - GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API > >> Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, > >> > >> org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], > >> > >> Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] > >> from > >> > >> > jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar > >> > >> > !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class > >> > >> > >> It looks like TomEEOpenAPIExtension was added in this commit: > >> > >> - > >> > https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd > >> > >> Jon, looks like you added TomEEOpenAPIExtension. Is there any trick you > >> needed to do to avoid the above conflict in our standalone servers? > >> > >> If anyone has any insight, it's very appreciated. This can be > reproduced > >> with this example: > >> > >> - > >> > https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile > >> > >> > >> > >> -David > >> > >> > >
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
> On Sep 4, 2020, at 8:48 AM, Jonathan Gallimore > wrote: > > Off the top of my head - I don't know. I'll run your test and dig through > over the weekend. No worries, I'll continue digging. I thought there might be a "I had that issue and did the thing with the thing, you probably need to adjust it." If nothing jumps out, no need to dig unless that sounds fun :) -David > > On Fri, Sep 4, 2020 at 4:16 AM David Blevins > wrote: > >> Ok, so here's the error that keeps the TomEE MicroProfile, Plus and Plume >> examples from booting. >> >> It appears we get an AmbiguousResolutionException from OpenWebBeans >> because we have these two beans getting picked up and they conflict with >> each other: >> >> >> - TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API >> Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, >>org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, >> >> org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], >> >> Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] >> from >> >> jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar >> !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class >> >> >> - GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API >> Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, >> >> org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], >> >> Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] >> from >> >> jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar >> >> !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class >> >> >> It looks like TomEEOpenAPIExtension was added in this commit: >> >> - >> https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd >> >> Jon, looks like you added TomEEOpenAPIExtension. Is there any trick you >> needed to do to avoid the above conflict in our standalone servers? >> >> If anyone has any insight, it's very appreciated. This can be reproduced >> with this example: >> >> - >> https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile >> >> >> >> -David >> >>
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
Off the top of my head - I don't know. I'll run your test and dig through over the weekend. Jon On Fri, Sep 4, 2020 at 4:16 AM David Blevins wrote: > Ok, so here's the error that keeps the TomEE MicroProfile, Plus and Plume > examples from booting. > > It appears we get an AmbiguousResolutionException from OpenWebBeans > because we have these two beans getting picked up and they conflict with > each other: > > >- TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API > Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, > org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, > > org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], > > Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] > from > > > jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar > !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class > > >- GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API > Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, > > org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], > > Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] > from > > > jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar > > !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class > > > It looks like TomEEOpenAPIExtension was added in this commit: > > - > https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd > > Jon, looks like you added TomEEOpenAPIExtension. Is there any trick you > needed to do to avoid the above conflict in our standalone servers? > > If anyone has any insight, it's very appreciated. This can be reproduced > with this example: > > - > https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile > > > > -David > >
Re: New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
Ok, so here's the error that keeps the TomEE MicroProfile, Plus and Plume examples from booting. It appears we get an AmbiguousResolutionException from OpenWebBeans because we have these two beans getting picked up and they conflict with each other: - TomEEOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, org.apache.tomee.microprofile.config.TomEEOpenAPIExtension, org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] from jar:file:/Users/dblevins/.m2/repository/org/apache/tomee/mp-common/8.0.5-SNAPSHOT/mp-common-8.0.5-SNAPSHOT.jar !/org/apache/tomee/microprofile/config/TomEEOpenAPIExtension.class - GeronimoOpenAPIExtension, WebBeansType:EXTENSION, Name:null, API Types:[java.lang.Object,javax.enterprise.inject.spi.Extension, org.apache.geronimo.microprofile.openapi.cdi.GeronimoOpenAPIExtension], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] from jar:file:/Users/dblevins/.m2/repository/org/apache/geronimo/geronimo-openapi-impl/1.0.12/geronimo-openapi-impl-1.0.12.jar !/org/apache/geronimo/microprofile/openapi/cdi/GeronimoOpenAPIExtension.class It looks like TomEEOpenAPIExtension was added in this commit: - https://github.com/apache/tomee/commit/10265708029ab2d0576f03aa1d5bb0a8a20bfffd Jon, looks like you added TomEEOpenAPIExtension. Is there any trick you needed to do to avoid the above conflict in our standalone servers? If anyone has any insight, it's very appreciated. This can be reproduced with this example: - https://github.com/apache/tomee/tree/master/examples/serverless-tomee-microprofile -David smime.p7s Description: S/MIME cryptographic signature
New TomEE embedded/serverless prototype (was Re: 9 Years of Startup times & Ideas for Improvement)
> On May 25, 2019, at 6:27 PM, David Blevins wrote: > > # Revisiting TomEE Embedded > > Already in the data, we can see 50% of our "new startup" time is extracting > the server from the tar.gz. We could speed things up by 50% for our build > and everyone's build, just by skipping that step. > > We have always had a TomEE Embedded distribution, but we've approached it > with a completely different mindset than regular TomEE and therefore it has > less functionality and simply could not complete with a TomEE zip. > > I believe we took a wrong turn at the start. > > In a plain Tomcat zip the server is started with an incredibly small > classpath, basically just what is the bin/ dir. Tomcat will then load > everything in the lib/ dir into a new classloader, grab a class from that new > classloader and tell it to finish the job. > > It would be possible for us to write an "embedded" version that does exactly > this, but using jars from the local Maven repo and would result in is being > able to start stop many Tomcat/TomEE instances in one JVM. We would have a > slightly different version of the Tomcat bootstrap jar, but everything after > that point would be 100% the same. The embedded version would start 2x > faster than a "remote" version but have the same functionality and cost us > very little in terms of maintenance. I've created a prototype of the above idea on a new embedded version of TomEE. The concepts at the heart of it are: - create a pom.xml for each server.zip that has a 1-to-1 mapping of every library in that zip - create a bootstrapper based on `org.apache.catalina.startup.Bootstrap` that can bootstrap a server using the classpath created by that pom - enjoy a server environment identical to the standalone zip, but running inside your JVM in an embedded or "serverless" fashion See the following two docs for an overview of what it looks like: - https://github.com/apache/tomee/tree/master/examples/serverless-tomee-webprofile#serverless-tomee-webprofile - https://github.com/apache/tomee/tree/master/examples/serverless-builder#serverless-builder-api Essentially there is a functional API that allows you to build an embedded server instance by passing in lambdas and method references that assist in the building. The approach still involves a very small number of config files in a temporary directory that represents the catalina.base. Where many embedded/serverless APIs go wrong is taking the perspective that any kind of configuration file is bad. When this perspective is taken, users must learn an elaborate API, usually a DSL, to build every aspect of the server instance by hand. The problem with this approach is that it takes an enormous effort to create these DSLs and get them to cover all possibilities in how a server can be configured. It sounds neat until you realize that you'd need a DSL to cover all the configuration files below: - catalina.policy - catalina.properties - context.xml - jaspic-providers.xml - logging.properties - server.xml - system.properties - tomcat-users.xml - tomee.xml - web.xml The effort to cover all that and properly document it is high. The documentation required is also enormously high. And while it's all very neat, the amount of effort a user must spend to learn it all is also quite high. And we go through all this effort as creators and users, just to get a version of the server we love that runs in our same VM as opposed to being a separate process. If the goal is simply to get a Tomcat/TomEE running in your JVM so you can right a test case or serverless app, there is a shorter path. When you boot Tomcat from the command line there is actually a very intentionally thin class called `Bootstrap` that does very little aside from constructing a ClassLoader out of the jars in lib/*.jar and bin/*.jar and then constructs an instance of `Catalina` and starts it. The real heavy lifting is done by Catalina from inside the newly constructed ClassLoader. The groundbreaking idea is that the simplest possible version of a Tomcat embedded is just a version of Bootstrap with some of those steps removed. When you take `Bootstrap` and pull out the classloader construction and the reflection needed to call `Catalina`, you really only have this code left: - https://github.com/apache/tomee/blob/master/tomee/tomee-bootstrap/src/main/java/org/apache/tomee/bootstrap/Server.java#L239-L247 From there you really only have two challenges, 1) ensure the right jars are in the classpath and 2) ensure there are config files on disk somewhere Tomcat can find. For challenge #1, the art of perfectly constructing a classpath that would be 100% identical to the corresponding Apache TomEE zip is actually a bit hard. The primary reason is Maven's transitive dependencies. To tackle this I wrote a tool that will extract a TomEE zip, analyze the libraries in it, map them back to Maven coordinates, then
Re: 9 Years of Startup times & Ideas for Improvement
Excellent explanation! I don't know when I'll have enough time to do this but it sounds exciting. If anyone has the inclination please do it! It's an important fix. On Mon, May 27, 2019 at 12:41 AM David Blevins wrote: > > On May 26, 2019, at 5:26 AM, Richard Monson-Haefel < > monsonhae...@gmail.com> wrote: > > > > This is really impressive, David! > > Thank you, Mentor :) > > > So, if I understand correctly, a new > > version of the bootstrap jar would load classes provided by Maven? Is > the > > time savings that Maven provides the uncompressed or is it something > else? > > Exactly right. > > Effectively, we take jars Maven has downloaded and put them into a Tomcat > zip and that becomes TomEE. When you write an Arquillian TomEE Remote test > case, we extract that zip each time the test runs. We boot the extracted > server, deploy your app, junit runs your tests in that test class, then we > delete the extracted server. Next test case the process repeats. > > On my mac it takes 4 seconds to extract TomEE Plus 8.0.0-M3, which is our > largest distro at 61MB. If you have 1000 tests, which is not a lot, you'll > spend 1 hour and 6 minutes just extracting TomEE zips. > > This is completely needless as the jars are already there downloaded by > Maven. We just need to use them. > > In the end the code that builds the Tomcat instance and would need to get > replaced with a Maven-aware version is not that big: > > mingus:/tmp 06:21:02 > $ unzip -l apache-tomee-webprofile-8.0.0-M3/bin/bootstrap.jar | grep > 'class$' > 8975 05-03-2019 23:26 > org/apache/catalina/security/SecurityClassLoad.class > 12196 05-03-2019 23:26 org/apache/catalina/startup/Bootstrap.class > 3494 05-03-2019 23:26 > org/apache/catalina/startup/CatalinaProperties.class > 1221 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$1.class > 1207 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$2.class > 1052 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$Repository.class > 1389 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$RepositoryType.class > 7429 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory.class > 858 05-03-2019 23:26 > > org/apache/catalina/startup/SafeForkJoinWorkerThreadFactory$SafeForkJoinWorkerThread.class > 934 05-03-2019 23:26 > org/apache/catalina/startup/SafeForkJoinWorkerThreadFactory.class > 5100 05-03-2019 23:26 org/apache/catalina/startup/Tool.class > 1414 05-03-2019 23:26 > org/apache/catalina/webresources/war/Handler.class > 1485 05-03-2019 23:26 > org/apache/catalina/webresources/war/WarURLConnection.class > 4387 05-03-2019 23:26 org/apache/tomcat/util/buf/UriUtil.class > > It's a 34k jar file with 14 classes in it. The Maven-aware version would > be slightly larger. > > The actual hard part will be eliminating the "build a TomEE zip" steps we > do in our build, such as TomEE webapps we discussed a few days back. > > > -David > > > > > > On Sat, May 25, 2019 at 8:34 PM David Blevins > > wrote: > > > >> All, > >> > >> Our startup time in the TomEE 8.0.0 milestones is much slower. We've > >> historically had great startup speed, something we've worked on and it's > >> been a point of pride for the project. A core part of the "Be small, Be > >> certified, be Tomcat" slogan. > >> > >> We definitely need to put some work into this again. I created a very > >> rough prototype that allows us to benchmark TomEE versions against each > >> other and see if we are trending up or down. I ran it on all our > >> webprofile and plus versions from TomEE 1.0.0 to 8.0.0-M3 (build 1136). > >> > >> - > >> > https://issues.apache.org/jira/secure/attachment/12969800/startup-times.png > >> - > >> > https://issues.apache.org/jira/secure/attachment/12969802/sort-by-total.txt > >> - > >> > https://issues.apache.org/jira/secure/attachment/12969801/sort-by-startup.txt > >> > >> I dumped all the data into a JIRA so none of us have to search email for > >> it in 5 years time. > >> > >> - https://issues.apache.org/jira/browse/TOMEE-2528 > >> > >> Historical data is great, but the effort was put in with the hope we > could > >> potentially make some improvements. A few opportunities come to mind. > >> > >> # Performance Testing in the build > >> > >> The trick with automating performance tests is you need a reliable way > to > >> pass/fail without human interaction or you haven't achieved much. You > >> can't put in a fixed number like "must be faster than 5 seconds" as > >> everyone's hardware is different. Scientific method to the rescue. We > >> could introduce a control group and measure against that. The test > could > >> measure the speed of the current code + the previous two releases and > >> assert the current code is not slower than the previous two releases by > >> some margin. > >> > >> The test is then asserting simply, "we can't
Re: 9 Years of Startup times & Ideas for Improvement
> On May 26, 2019, at 5:26 AM, Richard Monson-Haefel > wrote: > > This is really impressive, David! Thank you, Mentor :) > So, if I understand correctly, a new > version of the bootstrap jar would load classes provided by Maven? Is the > time savings that Maven provides the uncompressed or is it something else? Exactly right. Effectively, we take jars Maven has downloaded and put them into a Tomcat zip and that becomes TomEE. When you write an Arquillian TomEE Remote test case, we extract that zip each time the test runs. We boot the extracted server, deploy your app, junit runs your tests in that test class, then we delete the extracted server. Next test case the process repeats. On my mac it takes 4 seconds to extract TomEE Plus 8.0.0-M3, which is our largest distro at 61MB. If you have 1000 tests, which is not a lot, you'll spend 1 hour and 6 minutes just extracting TomEE zips. This is completely needless as the jars are already there downloaded by Maven. We just need to use them. In the end the code that builds the Tomcat instance and would need to get replaced with a Maven-aware version is not that big: mingus:/tmp 06:21:02 $ unzip -l apache-tomee-webprofile-8.0.0-M3/bin/bootstrap.jar | grep 'class$' 8975 05-03-2019 23:26 org/apache/catalina/security/SecurityClassLoad.class 12196 05-03-2019 23:26 org/apache/catalina/startup/Bootstrap.class 3494 05-03-2019 23:26 org/apache/catalina/startup/CatalinaProperties.class 1221 05-03-2019 23:26 org/apache/catalina/startup/ClassLoaderFactory$1.class 1207 05-03-2019 23:26 org/apache/catalina/startup/ClassLoaderFactory$2.class 1052 05-03-2019 23:26 org/apache/catalina/startup/ClassLoaderFactory$Repository.class 1389 05-03-2019 23:26 org/apache/catalina/startup/ClassLoaderFactory$RepositoryType.class 7429 05-03-2019 23:26 org/apache/catalina/startup/ClassLoaderFactory.class 858 05-03-2019 23:26 org/apache/catalina/startup/SafeForkJoinWorkerThreadFactory$SafeForkJoinWorkerThread.class 934 05-03-2019 23:26 org/apache/catalina/startup/SafeForkJoinWorkerThreadFactory.class 5100 05-03-2019 23:26 org/apache/catalina/startup/Tool.class 1414 05-03-2019 23:26 org/apache/catalina/webresources/war/Handler.class 1485 05-03-2019 23:26 org/apache/catalina/webresources/war/WarURLConnection.class 4387 05-03-2019 23:26 org/apache/tomcat/util/buf/UriUtil.class It's a 34k jar file with 14 classes in it. The Maven-aware version would be slightly larger. The actual hard part will be eliminating the "build a TomEE zip" steps we do in our build, such as TomEE webapps we discussed a few days back. -David > > On Sat, May 25, 2019 at 8:34 PM David Blevins > wrote: > >> All, >> >> Our startup time in the TomEE 8.0.0 milestones is much slower. We've >> historically had great startup speed, something we've worked on and it's >> been a point of pride for the project. A core part of the "Be small, Be >> certified, be Tomcat" slogan. >> >> We definitely need to put some work into this again. I created a very >> rough prototype that allows us to benchmark TomEE versions against each >> other and see if we are trending up or down. I ran it on all our >> webprofile and plus versions from TomEE 1.0.0 to 8.0.0-M3 (build 1136). >> >> - >> https://issues.apache.org/jira/secure/attachment/12969800/startup-times.png >> - >> https://issues.apache.org/jira/secure/attachment/12969802/sort-by-total.txt >> - >> https://issues.apache.org/jira/secure/attachment/12969801/sort-by-startup.txt >> >> I dumped all the data into a JIRA so none of us have to search email for >> it in 5 years time. >> >> - https://issues.apache.org/jira/browse/TOMEE-2528 >> >> Historical data is great, but the effort was put in with the hope we could >> potentially make some improvements. A few opportunities come to mind. >> >> # Performance Testing in the build >> >> The trick with automating performance tests is you need a reliable way to >> pass/fail without human interaction or you haven't achieved much. You >> can't put in a fixed number like "must be faster than 5 seconds" as >> everyone's hardware is different. Scientific method to the rescue. We >> could introduce a control group and measure against that. The test could >> measure the speed of the current code + the previous two releases and >> assert the current code is not slower than the previous two releases by >> some margin. >> >> The test is then asserting simply, "we can't be slower than we were >> before", which works on any hardware. >> >> This will likely cause intermittent build failures, but it will also help >> us squash performance issues early. They would be most likely to happen on >> library upgrades. This also means doing them the day before a release >> would become much more of a no-no, which is perhaps not a bad thing. >> >> # Experimenting with Performance Improvements >> >>
Re: 9 Years of Startup times & Ideas for Improvement
This is really impressive, David! So, if I understand correctly, a new version of the bootstrap jar would load classes provided by Maven? Is the time savings that Maven provides the uncompressed or is it something else? On Sat, May 25, 2019 at 8:34 PM David Blevins wrote: > All, > > Our startup time in the TomEE 8.0.0 milestones is much slower. We've > historically had great startup speed, something we've worked on and it's > been a point of pride for the project. A core part of the "Be small, Be > certified, be Tomcat" slogan. > > We definitely need to put some work into this again. I created a very > rough prototype that allows us to benchmark TomEE versions against each > other and see if we are trending up or down. I ran it on all our > webprofile and plus versions from TomEE 1.0.0 to 8.0.0-M3 (build 1136). > > - > https://issues.apache.org/jira/secure/attachment/12969800/startup-times.png > - > https://issues.apache.org/jira/secure/attachment/12969802/sort-by-total.txt > - > https://issues.apache.org/jira/secure/attachment/12969801/sort-by-startup.txt > > I dumped all the data into a JIRA so none of us have to search email for > it in 5 years time. > > - https://issues.apache.org/jira/browse/TOMEE-2528 > > Historical data is great, but the effort was put in with the hope we could > potentially make some improvements. A few opportunities come to mind. > > # Performance Testing in the build > > The trick with automating performance tests is you need a reliable way to > pass/fail without human interaction or you haven't achieved much. You > can't put in a fixed number like "must be faster than 5 seconds" as > everyone's hardware is different. Scientific method to the rescue. We > could introduce a control group and measure against that. The test could > measure the speed of the current code + the previous two releases and > assert the current code is not slower than the previous two releases by > some margin. > > The test is then asserting simply, "we can't be slower than we were > before", which works on any hardware. > > This will likely cause intermittent build failures, but it will also help > us squash performance issues early. They would be most likely to happen on > library upgrades. This also means doing them the day before a release > would become much more of a no-no, which is perhaps not a bad thing. > > # Experimenting with Performance Improvements > > Tools like JMeter, Grinder, etc. don't help at all as they test > throughput. Tools like YourKit, JProfiler etc are great for seeing where > time is spent in code and can be used for looking for optimizations. In > the end, however, they still take hours and it isn't always clear if you're > moving forward. > > With this technique of a+b perf testing, we can race the previous code and > our code with performance ideas and get bottom line feedback very quickly. > You can test multiple builds so you can try a few ideas out at once and see > which one moves the needle the most. > > If this tool is cleaned up enough and made very simple and command-line > friendly, I think it could turn us all into performance-minded contributors. > > # Revisiting TomEE Embedded > > Already in the data, we can see 50% of our "new startup" time is > extracting the server from the tar.gz. We could speed things up by 50% for > our build and everyone's build, just by skipping that step. > > We have always had a TomEE Embedded distribution, but we've approached it > with a completely different mindset than regular TomEE and therefore it has > less functionality and simply could not complete with a TomEE zip. > > I believe we took a wrong turn at the start. > > In a plain Tomcat zip the server is started with an incredibly small > classpath, basically just what is the bin/ dir. Tomcat will then load > everything in the lib/ dir into a new classloader, grab a class from that > new classloader and tell it to finish the job. > > It would be possible for us to write an "embedded" version that does > exactly this, but using jars from the local Maven repo and would result in > is being able to start stop many Tomcat/TomEE instances in one JVM. We > would have a slightly different version of the Tomcat bootstrap jar, but > everything after that point would be 100% the same. The embedded version > would start 2x faster than a "remote" version but have the same > functionality and cost us very little in terms of maintenance. > > > > -- > David Blevins > http://twitter.com/dblevins > http://www.tomitribe.com > > -- Richard Monson-Haefel https://twitter.com/rmonson https://www.linkedin.com/in/monsonhaefel/
