Hi Fabian, a few pointers and directions to check:
1. Ensure to test with securerandom.source=/dev/./urandom since you use a lot of crypto 2. You use hsqldb which is known to not scale very well with the concurrency, maybe switch to another database with a correctly configured connection pool 3. You use DataBaseRealm which does a lookup of the connection for each authentication which is synchronized and has no cache on the password hashes so it can be slow at runtime Personally i would start by using a fast realm (even if it always says "ok") to validate this hypothesis before investigating other cases. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/application-development/java-ee-8-high-performance> Le mar. 4 sept. 2018 à 12:15, <[email protected]> a écrit : > You were right, my student did something wrong with the first thread dump, > I > uploaded a correct one: > https://gist.github.com/TrustedGate/83781f05950f245a3bfd2388c5bfd7ab > > Also at the bottom there is a vmstat result, its definitely multithreaded > :) > > -----Original Message----- > From: Jean-Louis Monteiro <[email protected]> > Sent: Tuesday, September 04, 2018 11:59 AM > To: [email protected] > Subject: *EXT* [Newsletter] Re: TomEE Performance > > I would need to check on this one and probably investigate a bit more what > you > are doing. > > That being said, I can confirm only one thread is currently working. So > either > jmeter is only sending monothreaded requests or there is something else. > But you aren't doing multiple requests in parallel > > -- > Jean-Louis Monteiro > http://twitter.com/jlouismonteiro > http://www.tomitribe.com > > On Tue, Sep 4, 2018 at 11:53 AM, <[email protected]> > wrote: > > > We have > > > > @Resource > > WebServiceContext webserviceContext; > > > > in our SOAP api class, that should not work with @Singleton, or am I > > mistaken? > > > > > > -----Original Message----- > > From: Jean-Louis Monteiro <[email protected]> > > Sent: Tuesday, September 04, 2018 11:41 AM > > To: [email protected] > > Subject: *EXT* [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: > > TomEE Performance > > > > Ah ok. Well I was asking if you were injecting when you took the > > thread dump because from a server point of view I saw only one thread > > working. > > If you were using jmeter with multiple virtual users, I was expecting > > to see more than one thread working. > > > > I'll double check. > > > > @Singleton is by default using Lock WRITE which prevents multiple > > threads to access the singleton. > > If you don't need synchronization or if your code is thread safe (no > > instance variables, etc). You can safely use @Singleton with Lock READ > > which will be faster than @Stateless because there is no pool > > involved. And again, there is no tuning to do > > > > > > > > -- > > Jean-Louis Monteiro > > http://twitter.com/jlouismonteiro > > http://www.tomitribe.com > > > > On Tue, Sep 4, 2018 at 11:37 AM, <[email protected]> > > wrote: > > > > > Injecting? The jmeter was running during the threaddump. What do you > > > mean with injecting? > > > > > > Is Singleton (Lock.READ) not even more of a bottleneck when it comes > > > to multiple concurrent requests? IIRC we tried Singleton before, but > > > not sure what the reason was why we went with @Stateless... > > > > > > -----Original Message----- > > > From: Jean-Louis Monteiro <[email protected]> > > > Sent: Tuesday, September 04, 2018 11:30 AM > > > To: [email protected] > > > Subject: *EXT* [Newsletter] Re: [Newsletter] Re: TomEE Performance > > > > > > Yes exactly. > > > > > > Were you injecting anything when you took the jstack? > > > It seems that only one thread is working. > > > > > > We would need you to do it when you are injecting. > > > If you could also give us the CPU usage when you take the jstack > > > that'd be great. > > > You can run `vmstat 5` on another terminal so you can see what's > > > going > > on. > > > > > > You are using Stateless Session beans. Any reason you aren't using a > > > plain singleton? > > > With Singleton (Lock.READ) there is no pooling involved, so you > > > don't have anything to configure or tune. > > > > > > > > > > > > -- > > > Jean-Louis Monteiro > > > http://twitter.com/jlouismonteiro > > > http://www.tomitribe.com > > > > > > On Tue, Sep 4, 2018 at 10:53 AM, > > > <[email protected]> > > > wrote: > > > > > > > Like this: > > > > https://gist.github.com/TrustedGate/f670c079088404f42d69aabd409de7c4 > ? > > > > > > > > -----Original Message----- > > > > From: Jean-Louis Monteiro <[email protected]> > > > > Sent: Tuesday, September 04, 2018 8:38 AM > > > > To: [email protected] > > > > Subject: *EXT* [Newsletter] Re: TomEE Performance > > > > > > > > Hi, > > > > > > > > Around SOAP, there are a couple of possible optimizations. > > > > What would be helpful is to get into the docker container when you > > > > are over the linear zone and get a jstack of the tomee process. > > > > Post it here or in gist and put the link here. > > > > > > > > Jean-Louis > > > > > > > > -- > > > > Jean-Louis Monteiro > > > > http://twitter.com/jlouismonteiro > > > > http://www.tomitribe.com > > > > > > > > On Tue, Sep 4, 2018 at 8:27 AM, > > > > <[email protected]> > > > > wrote: > > > > > > > > > Hey, > > > > > > > > > > > > > > > > > > > > we have been running some performance tests with our application > > > > > (TomEE > > > > > 7.0.5 based) and are stuck: > > > > > > > > > > > > > > > > > > > > Until 4 core VMs (or docker containers) we see a linear increase > > > > > in performance, which is great and was anticipated. > > > > > > > > > > > > > > > > > > > > But after 4 cores, we barely get 10% (with 8 cores) more > > > > > performance of the system. > > > > > > > > > > > > > > > > > > > > We used jmeter based load tests (SOAP calls) with 10/20/30/40 > > > > > and > > > > > 100 threads, VM to VM via 1gbit, and 4 GB of RAM. > > > > > > > > > > > > > > > We played around with session bean pool sizes (min set to thread > > > > > count, max to 1000) and stateful bean pool settings and also > > > > > with jvm heap size and GC parameters, to no avail. > > > > > > > > > > > > > > > > > > > > Are there any more performance parameters we can toy around with > > > > > in TomEE or Tomcat that you can recommend? > > > > > > > > > > > > > > > > > > > > Thank you and best > > > > > > > > > > Fabian > > > > > > > > > > > > > > > > > > > > > > > > >
