eurotrans-Verlag wrote:
Hello everybody,
I stumbled upon a strange problem with the ISAPI Redirector 1.2.31 on
Windows Server 2008 SP2 (32 bit) with IIS 7.0. The problem is, that when a
Servlet is generating lots of data (e.g. 200 MB) and a user downloads it
over the Isapi Redirector/IIS7, and cancels the download, the IIS Worker
process (w3wp.exe) will have 100% CPU usage for about 30 seconds. However if
the download is completed normally (not canceled), everything is fine and
the CPU usage will not go up that far.
The exact Components used are:
Windows Server 2008 SP2 (32 bit) with IIS 7.0,
Sun JDK 1.6.0_25,
Tomcat 7.0.14,
ISAPI Redirector 1.2.31.
I could reproduce the problem with a clean install of these components.
To reproduce:
1. Install Tomcat and the ISAPI Redirector on a Windows Server 2008 SP2 IIS
7.0 system, as described in the Tomcat Connectors Documentation
(The problem occurs with both settings of "enable_chunked_encoding", "true"
and "false").
2. Create a Servlet or JSP that produces some huge amount of random data,
for example:
@WebServlet("/DownloadServlet")
public class DownloadServlet extends HttpServlet {
protected void doGet(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {
response.setContentType("application/x-msdownload");
OutputStream out = response.getOutputStream();
Random r = new Random();
byte[] content = new byte[1024];
for (int i = 0; i < content.length; i++)
content[i] = (byte) (r.nextFloat() * 256);
for (int i = 0; i < 200000; i++)
out.write(content);
out.close();
}
}
3. Request the Servlet or JSP through the IIS port with a browser. After
some seconds, cancel the download.
4. One thread of the IIS Worker Process (w3wp.exe) will have 100% CPU usage
for some time (about 30 seconds), before it normalizes to 0%.
Note that is does not happen every time. Sometimes you will have to repeat
starting and canceling the download until the problem occurs.
5. In the ISAPI log, following lines are logged ("log_level"="info"):
[Sat May 14 14:48:55.654 2011] [3508:3560] [error]
isapi_write_client::jk_isapi_plugin.c (1210): WriteClient failed with 995
(0x000003e3)
[Sat May 14 14:48:55.654 2011] [3508:3560] [info]
ajp_process_callback::jk_ajp_common.c (1885): Writing to client aborted or
client network problems
[Sat May 14 14:48:55.654 2011] [3508:3560] [info]
ajp_service::jk_ajp_common.c (2543): (worker1) sending request to tomcat
failed (unrecoverable), because of client write error (attempt=1)
[Sat May 14 14:48:55.669 2011] [3508:3560] [info]
HttpExtensionProc::jk_isapi_plugin.c (2217): service() failed because client
aborted connection
These lines are logged immediately after the client cancels the connection
(not after the CPU usage of w3wp.exe goes down).
The worker process continues normally after the CPU usage went down.
However, if one would do this repeatedly, it could enable DoS attacks,
couldn't it?
What would cause these excessive CPU usages of the IIS worker process when
the user cancels the download?
A guess :
IIS and the isapi_redirector are one process, and Tomcat and the servlet are another
process. The isapi_redirector makes a connection to Tomcat, sends the request over it.
The servlet starts running and producing output, quite a lot and quite fast, over that
connection.
The isapi_redirector reads the servlet output, and writes it to the client
connection.
Now the client closes the connection, and isapi_redirector finds a closed socket the next
time it tries to write to the client. It writes this error to the logfile, and stops
trying to send output to the client (well, it cannot anymore, the socket is closed).
But in the meantime the servlet keeps producing output and sending it to
isapi_redirector.
So either isapi_redirector has a way to stop the servlet and discarding any servlet output
still pending, or else it is forced to read and discard whatever the servlet is still
sending, until the servlet decides to stop by itself.
Maybe the 100% CPU usage is happening while isapi_redirector is reading and discarding
whatever remains of the servlet output ? Since there is no client connection anymore to
slow things down, this is bound to be rather fast and cpu-intensive.
To figure out if this is what's happening, you could do some logging at the servlet end,
to see if it keeps sending data even when the client has canceled, or if it itself gets
some stop indication from the isapi_redirector (also a closed socket e.g.).
Or else Mladen or Rainer could tell us if I'm totally off-base here.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
