On 10/01/2019 17:31, john.e.gr...@wellsfargo.com wrote: <snip/>
> The first troubleshooting step should be to take a heap dump and analyze it > with a tool like Eclipse MAT. If you read the entire thread you will see that I have already done that. The Java Heap (in fact all the JVM managed pools) were static whilst overall memory was growing. That points to a native memory leak that no Java profiler is going be able to find the root cause. I also tried various native memory leak detectors but the combination of the noise from the JVM and the performance impact meant the output was not useful. > I don't know anything about the OpenSSL implementation, but it probably has a > session cache. Is there an upper limit on the cache size? The JSSE cache > size is unlimited by default, I think, and objects only expire from the cache > after 1 day. Given that the APR/native connector is not affected but the NIO is, that points to something other than the SSL session cache (since the configuration is the same between the two). Again this was mentioned earlier in the thread. > When a client connects, if it already has a session id from a previous > handshake, it will ask the server to resume the session. If it happens to > connect to the same server, it will probably get to reuse the existing > (cached) session. If not, then a new session is created and cached. If you > have a lot of servers behind a load balancer with no stickiness, then the > session reuse will not be high. For example, if you have 2 servers, you're > only going to get 50% reuse. If you have 100 servers, only 1% reuse! )-: > The lower the session reuse rate, the more orphaned sessions there are > sitting on the server. > > Mark, in your test if you were connecting from the same client to the same > server over and over, you probably used the same session over and over, so > you wouldn't see much session cache growth, if any. Indeed. Which, given that the memory usage is growing, is further evidence that the session cache is not part of it. I'm currently looking at the Tomcat Native Library and written some simple tests that mimic TLS connection setup in a tight loop. These tests also exhibit native memory leaks. I've just tracked down one leak although it is a relatively small one. Next steps are to fix that leak and then find the next one. And then repeat. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
