Hi,

After an upgrade to Tomcat 8.5, we are experiencing an issue where
Tomcat starts generating a high CPU load (100%), probably after an
HTTP network scan. The bug seems to be related to Windows, NIO and
possibly SSL. I have a Yourkit dump and several thread dumps that show
the issue, and was wondering if anyone is interested in this, and if
we can gather any extra information to help debug this issue.

Setup: Windows 2k8r2, Tomcat 8.5.4, Java 8u102, NIO HTTP and NIO JSSE
HTTPS connector.

Out of nothing, Tomcat starts using 100% CPU. I made some thread
dumps, available here:

https://gist.github.com/MikeN123/f4a85f09231cfda7a9e632b64f27dcdc
https://gist.github.com/MikeN123/7dfe17ae95b8d516d86e0d7126cbaa02
https://gist.github.com/MikeN123/750da8580e04e0498f70b81dbd1a5c52
https://gist.github.com/MikeN123/2e83307b7c1216339d4fa73b30c02f1a
https://gist.github.com/MikeN123/8850ef2a60d39a4dc140b2d8fef18c3f

I also have some Yourkit stats available, but as these may contain
confidential information, I won't share them in public. Basically,
what we see is that the thread https-jsse-nio-443-ClientPoller-0 is
continuously runnable and using CPU on
sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(), and various other
https-jsse-nio-443-exec threads are waiting (parked) or running. These
threads together take up all the CPU. A Yourkit thread view showing
the issue starting around 11:02: https://dl.eveoh.nl/yc_fal.png

We _suspect_ the issue is triggered by an HTTP scan, which generates
the following requests in the access log, but we are still trying to
confirm this: https://gist.github.com/MikeN123/581d1f17aae100f06b8c65b86870a64a

Also, we are trying to confirm whether or not NIO2 shows the same behaviour.

The behaviour seems to be the same as in this tomcat-users thread:
https://mail-archives.apache.org/mod_mbox/tomcat-users/201604.mbox/%3CCAE-ydNF84pnoX2tP8BJ4vQisabgycP0y2vpnmjNhddz9+BKp=w...@mail.gmail.com%3E

A similar issue is mentioned for some other products, but I'm not sure
if there's a relation:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=357240
https://developer.jboss.org/thread/240618?start=0&tstart=0
https://github.com/netty/netty/issues/3857

Our next steps are:

- Switching the production site to NIO2, to see if that fixes the issue
- Checking if we can reproduce the issue by triggering the HTTP
vulnerability scan manually

Any ideas or requests for more information are more than welcome.

Regards,

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reply via email to