As for reading - IoBuffer in each IoSession should also be limited.
It's interesting that AbstractIoSession has multiple statistical
variables, ex.: scheduledWriteBytes, readBytes, writtenBytes,
readBytesThroughput, writtenBytesThroughput. Maybe they can be useful
here...
Victor N
Victor wrote:
Emmanuel,
I think it should be implemented in some abstract way, giving us a way
to tune it in any way a specific application may desire (I mentioned our
sample in Jira - video streaming server capable of reducing frame rate
for slow clients). I do not know yet how this abstract
"IoThrottlingStrategy" should look like...
But in 90% of situations we could propose a predefined
"SimpleIoThrottlingStrategy" strategy as you propose - i.e. we may
configure some limits per IoSession and mina will check them.
First, we need to know all places in mina where OOM can occur for
fast-writing, slow-reading and frequently-connecting clients. I can
remember some of them:
- AbstractPollingIoProcessor: newSessions (and maybe) removingSessions,
flushingSessions, trafficControllingSessions
- AbstractPollingIoAcceptor: (not sure if this may impact)
registerQueue, cancelQueue, boundHandles
- AbstractIoSession: writeRequestQueue
- OrderedThreadPoolExecutor: waitingSessions and SessionTasksQueue
(DIRMINA-723)
Victor N
Emmanuel Lecharny wrote:
Hi,
today we discussed about DIRMINA-764, and about solutions to deal with
rogue clients (which are not necessarily malevolent).
The problem is that a client which send a lot of messages and does not
read the responses fast enough will impact the server in a very ad way
: at some point, you'll be hit by a OOM.
So the question arose about how to deal with such a situation. there
are many things we can control :
- number of clients per server
- number of message accepted for a client per unit of time
- number of message a client can have on the writing queue before we
stop accepting new requests
- size of message we accept for a client
- number of messages in the writing queue
- size of messages being processed globally
All those parameters (and I may have missed some) have an impact on
the server. The problem here is that we are at the limit between
configuration and protection. If we decide we accept up to 100 000
clients on a MINA server, then how do we set the other limits? What
size should we allowate to handle the load ?
Another problem is that if we limit the global number of messages
being processed, or the global size, then we will have to select which
client we will have to block.
Also limitating the writeQueue size might slow down the processing.
Right now, in order to avoid a situation where the server simply die,
I suggest to implement a very smple strategy on the server : we add a
parameter in the session config indicating the macimum number of
messages allowed in the writeQueue for a specific session, before this
session block new incoming messages. This is easy to implement, and
will protect us a bit from fast client but slow readers.
We can think more about those typical use cases in MINA 3.
thoughts ?
