Hi all
Checking this again, it seems that an OOM is being thrown after all. The
IoFilter throttling doesn't seem to kick in until too late, unfortunately.
Looking at the SocketIoProcessor class, if I apply the throttle before
the read call, this achieves the desired effect for me: (ie: limits
memory usage significantly):
private void process( Set<SelectionKey> selectedKeys )
{
...
if (!throttle(session)) {
read( session );
}
...
}
Unfortunately this solution is not clean as I am forced to modify the
Mina source code and not use the API.
Any better ideas anyone?
Chris Chalmers wrote:
Hi Peter
I'm now getting something strange:
- my read is throttling fine based on the number of outstanding writes
on a linked session
- the session.resumeRead() method is being called when the linked
writes have dropped
- however, the read is not resumed.
I have limited memory usage to 64Mb, and at the point of session
resumption, 63.8Mb has been read in.
When using 256Mb, the process completes; at the point of resume, it
seems like the allocated memory doubles - I am hazarding a guess that
this could be causing the problem when using 64Mb. However, I don't
get an out-of-memory exception.
Any ideas?
Thanks,
Chris
peter royal wrote:
On Jun 28, 2007, at 8:00 AM, Chris Chalmers wrote:
I have a Mina server that simply pumps data through (based on the
Mina Proxy example) -
however, when sending large files (500Mb+), the data is read in
quickly, but written slowly (the target uses a blocking Socket
read); this causes the server's memory usage to increase very quickly.
I have tried to use the ReadThrottleFilterBuilder in combination
with the WriteBufferLimitFilterBuilder (from
http://issues.apache.org/jira/browse/DIRMINA-302), but the best I
can get is 226Mb memory usage when setting the read/write
setMaximumConnectionBufferSize to 50000.
Is there any way that I can force Mina sessions to block incoming
messages instead? The source code explicitly sets the SocketChannels
to non-blocking and I can't see any way to change this using the API.
Alternatively is there a better method that I am overlooking?
IoSession.suspendRead / resumeRead ... you can use that to keep MINA
from reading the data, just turn it on/off in concert with your writing.
-pete
