A profiler has its place. VisualVM was vital in helping us solve a Java 8 memory leak last November (although it didn't involve Tomcat). We were moving a large, long running system from old Solaris machines with InformixDB to Intel hardware with SELinux and OracleDB. Part of the system's workflow involved a small Java utility that processed images from one step to another. That process had run on Oracle Java 7 and was now running on a RedHad build of OpenJDK 8. With the new environment we observed a memory leak where the Java process grew in memory used (resident set size increased continually) and the overall memory available on the machines just got lower and lower until the OS decided to kill one or more processes to free up some memory.
We poured over our code. Small though it was, I tried Java programs that were smaller yet--little more than connect and disconnect. We still saw a memory problem. Finally I profiled our Java utility with VisualVM, all on my Mac Mini, and quickly found a leak from java.util.zip. This was a surprise because we were not using java.util.zip anywhere, nor could I find any reference to java.util.zip when I looked at the code of the few open source libraries we were using. What we did find was two online references to a native memory leak in java.util.zip in Java 8 (one from OpenJDK and one from RedHat). Based on this, we took RedHat's recommendation to upgrade to OpenJDK 11. Memory stabilized immediately. We might still be restarting servers at 1am without VisualVM helping us identify this leak. (We are also moving our other Java apps and our Tomcat installations to Java 11.) On Fri, Feb 10, 2023 at 4:31 PM John Dale (DB2DOM) <jcdw...@gmail.com> wrote: > I've tried profilers in the past, but I've never had much luck since > you need a super computer to run them. Human intelligence rules .. > read the code carefully, review it, step it with a debugger, and look > for memory leak patterns. Mine have mostly been static and non static > collections and HashMaps that keep growing, or rampant string creation > wherein the GC couldn't keep up under load. > > Review the code .. become its master and empath. Memory leaks cause it > pain. > > On 2/9/23, James H. H. Lampert <jam...@touchtonecorp.com.invalid> wrote: > > I've obtained some heap and CPU numbers, taking data at 15 minute > > intervals, heap from WRKJVMJOB and CPU from WRKACTJOB. In two days of > > this, I didn't witness any crashes; I did witness a near-miss, in which > > heap-in-use hit 5011.938M (out of 5120). > > > > In discussion with our webapp developer (to whom we sent a catalina.out > > excerpt), he observed that they were running Tomcat on a six-year-old > > JVM (it identifies in a WRKJVMJOB as "1.8.0_151"; on the Manager page, > > it identifies as "8.0.5.5 - pap6480sr5fp5-20171114_01(SR5 FP5)") with a > > known issue (on Github, it's listed as 11493). He suggested that the > > customer ought to try updating to a more recent Java. > > > > I've also asked on the IBM Midrange Java List whether we can go any > > higher on the heap parameters (currently set at -Xms 4096 -Xmx 5120 for > > that particular installation). > > > > -- > > JHHL > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- "Hell hath no limits, nor is circumscrib'd In one self-place; but where we are is hell, And where hell is, there must we ever be" --Christopher Marlowe, *Doctor Faustus* (v. 111-13)
