Hi,
On 19/12/2018 16:28, Krishan Babbar wrote:
Dear All,
I am working on an IoT project. I have created a Java application using MINA
library. All the devices are connecting to this Java application using TCP
protocol.
So far, so good.
I have created simulator (to simulate devices) to do our load testing. In
simulator I have given limit of 3000 threads in thread pool so that one
instance of simulator can connect 3000 devices at once.
which means your simulator will try to connect 3000 times almost
simultaneously to your app...
I tried testing for 30000 devices by executing 5 instances of simulators in
parallel.
Well, that's 5 * 3000 = 15 000, not 30 000. Anyway, that is 15000/30000
simultaneous connections for your app, unless you do some ramp up
Each device was sending messages after interval of 30 seconds and I had 10
messages per device.
Again, that's 15000/30000 messages sent simultaneously to your app,
every 10 seconds.
But my Java Application did not receive all the messages in one go, it is
processing messages very slow.
It kind of makes sense... First of all, I do think you should try to
determinate what is the maximum number of connections and messages per
second your app can support. That mean ramping up the threads up to the
moment you start losing messages/connections. Your approach is brutal,
it will not allow you to understand what's going on.
Suppose 5 simulators have sent (3000 devices/threads per instance * 5 instances
* 5 mins) 150000 messages in 5 minutes but Java Application (with MINA) has
processed about 10000 messages only.
And even after sometime my Java Application was closed due to memory issue.
Please help and guide how MINA is working in case of lot of devices/sockets
connection?
It works pretty much as it does when having few connections/messages.
But I would distinguish between connections and message. A connection
will be handled by a thread processing the ACCEPT event. MINA allocates
one single acceptor and N+1 IoProcessor, where N is the number of core
you have on your server (unless you specified an explicit number).
Which means if all your clients are trying to connect at the very same
time, your acceptor will be quite overloaded... It's very likely that
some connection will simply be dropped.
Where is MINA storing messages which are yet to process?
It doesn't. MINA just process messages it was capable of reading from
the active sockets (ie the sockets for which it has been signaled there
is some data waiting to be read, or that the socket is ready to be
written). Each of those messages are processed sequentially, as we only
use a limited number of threads for that (9 on a 2 CPU quad cores
machine). If you have 30 000 active sockets at a given time, and
assuming processing one message takes 10ms, then your server will only
be capable of absorbing 9 * 100 messages per second (on a 2 CPU quad
cores machine), ie it will take 33s to process all of them. You can
immediately see that your scenario where the 30 000 threads are sending
a message every 10 s will simply not fly.
There are two solutions here :
- pour more CPU/cores on your machine (increasing the number of
IoProcessor threads above the default lilmit does not make sense, unless
your app is waiting for most of the 10ms - which ios likely anyway if
your app is waiting for, say, a remote database response)
- speed up the processing of one message. If it takes 10ms of processing
(ie, one CPU/core is at 100% during the whole time it takes to process
your message, the '10ms' I took as an example, for instance, then you
are pretty much screwed. Fall back to workaround 1, or scale up your
solution by spreading your app on more than 1 machine...
With 16 GB RAM Linux machine, and giving 4 GB and 6 GB as a heap parameters in
Java Application (java -Xms4096m -Xmx6144m -Dserver -jar Application.jar), how
many devices/sockets MINA library can handle?
No idea. It all depend on what you do when you process a message.
Is MINA using single thread to connect all devices and receive packets from
them?
No. One Acceptor, as many IoProcessor as your have CPU*cores.
I mean is it possible to receive all messages immediately in my Request Handler?
Yes. But MINA won't be able to process all of them simultaneously.
Below is given my code for MINA configuration and from request handler we are
starting a new thread for each message for processing.
What's the point in using a NIO framework if you are recreating a system
where each message is processed by a thread ? This is utter nonsense.
Realize that a thread use 100% of a core resource when it's executing,
so if you have 8 cores, you won't be able to execute more than 8 threads
*simultaneously* ! The only leverage you have is the time it takes to
process one message. It if takes 100ms of pure CPU usage, then your 8
cores machine will not be able to handle more than 80 messages per
second, no matter what you do.If it takes only 10ms, then you jump up to
800 messages per second. This is very basic math.
The only reason you may want to spawn new threads after the IoProcessor
would be if your message processing requires some asynchronous treatment
during which your thread is doing nothing but waiting for an external
resource (database, whatever).
There is no magic. You have to understand the very basic of how network
works, and on how a CPU processing works. MINA knows nothing about your
app, it just hides the dirty part of handling non blocking IO, nothing more.
Hope it helps.
