My bad here are the correct dumps 10 secs apart https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMjIvMDIvMy8tLWFwaS0zNzY2ZmZmNy0wZDgyLTRhZTItYmE3Mi0zMWQyYTYwN2M1ZjgudHh0&
https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMjIvMDIvMy8tLWFwaS00NTE3MWUxNy1jYWRiLTRkY2UtODBlNS1lMDk0YTJjNTg1OGEudHh0& -----Original Message----- From: john.e.gr...@wellsfargo.com.INVALID <john.e.gr...@wellsfargo.com.INVALID> Sent: 03 February 2022 19:33 To: users@tomcat.apache.org Subject: RE: help with high cpu usage Alan, > -----Original Message----- > From: Alan F <shiva...@hotmail.com> > Sent: Thursday, February 03, 2022 12:19 PM > To: Tomcat Users List <users@tomcat.apache.org> > Subject: help with high cpu usage > > Had some issues today with one prod host. One is fine the other has > stuck around 80%Cpu. > Ive taken a thread dump from both hosts and would appreciate someone > give a once over what it may be before I kill and restart. They are > clustered nodes so running identical apps and loadbalanced by a hardware > balancer. > > Node1 is ok (relatively!) > https://urldefense.com/v3/__https://fastthread.io/my-thread- > report.jsp?p=c2hhcmVkLzIwMjIvMDIvMy8tLWFwaS1hODllYzBkZS01OGJjLTQ > 2ZDQtYWRhNS1kYjkxZjM2NjI1ZTAudHh0&__;!!F9svGWnIaVPGSwU!91fvMc > RzcMYr95RrClT-eCrcDNp3fKUDpupDSNtn-mDKbTl_dmUSa0LaAolXlhipl- > Fk2pQ$ > > > Node 2 still has high CPU usage > https://urldefense.com/v3/__https://fastthread.io/my-thread- > report.jsp?p=c2hhcmVkLzIwMjIvMDIvMy8tLWFwaS0yN2I0YWY4Mi05OWFhL > TQ3YjYtOGQ2My0wMDMwZjlkNDQzNjMudHh0&__;!!F9svGWnIaVPGSwU!9 > 1fvMcRzcMYr95RrClT-eCrcDNp3fKUDpupDSNtn- > mDKbTl_dmUSa0LaAolXlhipkmLFr3E$ Taking thread dumps was a good idea but I see very little activity on node 2. It looks like two threads are in Hibernate's EntityEntryContext.downgradeLocks() method, one is taking the thread dump itself, and another is reading a request from a client, I think. I would not expect this to add up to 80% CPU usage unless one of those threads is stuck in a loop. Comparing multiple thread dumps taken 5-10 seconds apart would help answer this question. How much GC is there? Could these Hibernate queries be pulling a huge amount of data from the DB, thus causing a lot of GC activity? Node 1 looks idle except for the thread taking the thread dump. Do you know for sure that it's the Tomcat process that is using the CPU?