Mark,

On 11/20/20 11:08, Mark Thomas wrote:
On 20/11/2020 15:43, Eric Robinson wrote:
From: Mark Thomas <ma...@apache.org>


<snip/>

This information might turn out to be unnecessary. I have been re-looking at
the previous logs and I think I have enough evidence to ID a root cause. I'll
email off-list as I need to quote logs to prove my point.


I'll be watching my inbox like a hawk!

Sent.

For the curious, the analysis indicated either JVM code or a native
library was incorrectly closing the file descriptor associated with the
socket being used by an in progress HTTP request.

FWIW, Connector/J is a Type 4 JDBC Driver (aka "pure Java"), so no native components.

I'm not sure how it would be able to close the connection.

Also, v5.0.8 is like 13 years old. Eric, you guys *really* have to upgrade that. Somewhat surprisingly, there are CVEs against that library which allow unauthenticated remote attackers to take-over the MySQL client connections opened by that library.

A second look at the strace file from yesterday provided hard evidence
of a native library mis-using file descriptors and strong circumstantial
evidence linking that same library to a specific instance of the
observed failure.

Interesting. How can you tell it's a library instead of, for example, the JVM itself (which is of course itself a set of native libraries). I'm assiming that when you say "native library" you mean "some native component which is not a part of the core JVM".

TL;DR, an issue in an external library, not a Tomcat issue.

I've recently been trying to optimize IO throughput of certain web-to-database (and vice-versa) operations in my own web applications. The gist is that we are wrapping request InputStreams or JDBC InputStreams (depending upon whether we are reading from request->db or db->response) to allow streaming directly in or out of the database with a minimum of buffering.

If such a scheme were to be implemented particularly badly (e.g. allowing the database code direct-access to the request and response objects instead of just the stream(s)), it would certainly be possible for such an implementation to close the response's output stream before the servlet was expecting it.

But such an implementation would be caught red-handed if a simple wrapper around the response stream were to be installed and simply log all the calls to close(), which I think was one of your first debugging steps, here.

So does this really look like a (pretty serious) bug in a native library? Any idea which one?

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reply via email to