HTTP-NIO transport can permanently lock-up with larger messages and moderate
concurrency
----------------------------------------------------------------------------------------
Key: SYNAPSE-321
URL: https://issues.apache.org/jira/browse/SYNAPSE-321
Project: Synapse
Issue Type: Bug
Components: Transports
Affects Versions: 1.1.1
Environment: Windows (XP, Vista) and Linux (Red-Hat Enterprise 5 and
Ubuntu 7.10, 8.04) platforms, performance tuned and not.
Reporter: Jake Lambert
Priority: Critical
I can consistently reproduce an HTTP-NIO lockup by sending larger messages (>
300KB - both with and without attachments) *and* >50 threads of concurrency
using SoapUI as a client. I'm directing the messages through a simple
forwarding proxy service. This happens for me on both Windows and Linux
(Red-Hat and Ubuntu) platforms, both tuned and not.
After a lot of investigation, here's what I've found:
- each of the transport receiver I/O dispatcher threads can block writing to
the request pipe sink in ServerHandler's inputReady() method when there are no
ServerWorker processing threads left
- the block occurs because the pipe's sink can only buffer a limited number of
bytes until a ServerWorker thread is actively reading from the pipe's source
- a blocked I/O dispatcher thread stops all incoming reads from the client and
writes back to the client for its associated connections
- as more requests come in with no free ServerWorker threads, more of the
incoming I/O dispatcher threads are blocked until they are *all* permanently
blocked
- the ServerWorker threads are all blocked either waiting for a free
ClientWorker thread or blocked waiting for more input from the client (because
the incoming mediation can be complete before a request has been fully read
from the incoming socket - this is where I think the larger messages come into
play)
- the ClientWorker threads are all busy waiting to send to their responses back
to the client (as previously mentioned, the socket writes back to the client
have been for all I/O dispatcher threads)
- there's no way out of the situation, so Synapse is effectively disabled. As
you can see, increasing the number of I/O dispatchers and worker threads can
only delay and not fix the problem.
Since ClientHandler's inputReady() can also block in this way, it probably
should be fixed also.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
