Emmanuel, thanks for the response. 1. The server processes incoming messages using the protocolCodecFilter. Currently it is the protocolCodecFilter responsibility to read the ENTIRE message and save it in the hard disk for future processing by the Handler. Reading few hundred MBs is an expensive process and might take a while, so dropping the first ExecutorFilter means that the NIOProcessor thread will handle this (long) task. If I'm not mistaken this is a bad choice because I have only few Processors so I'll easily cause my server to not excepting\processing other requests. Am I right? If I'm, maybe a better design for my server would be to move the (expensive) file reading to the handler... 2. In order to be efficient and consume as less memory as it can, the server sends the big files using FileWithHeaderWriteFilter (which extends AbstractStreamWriteFilter) I've noted that even if I have an ExecutorFilter in the chain Mina does not use its threads, instead it uses NIOProcessor threads. (Again,) this is an issue as I would expect my server to process simultaneously large number of files and not only few. How can I force Mina to use other threads than the NIOProcessor threads for that?
Thanks, Guy -----Original Message----- From: Emmanuel Lecharny [mailto:[email protected]] Sent: Sunday, February 13, 2011 1:32 PM To: [email protected] Subject: Re: where to place the ExecutorFilter? On 2/13/11 11:41 AM, Guy Itzhaki wrote: > (Sorry for my pre mature email an hour ago - my mistake) > > The ProtocolCodecFilter does an expensive process much more than the > handler, it is the ProtocolCodecFilter duty to read& persist incoming > messages, same goes for sending messages - the codec is responsible to > read the message from the disk and send it. ok, two things here : - I guess the decoding itself is not costly, so no need of using an executor for decoding. - You can add an executor to process the write operation only, but it should be placed *after* the codec filter, as the filters are executed in the reverse order when writing information. > > > I wonder where is best to place the ExecutorFilter? Is it before the > ProtocolCodecFilter or after? It seems to me that given what I explained > before I should use two executorFilters one to be placed before the > codec to digest incoming messages and the other after (for the sake of > the outgoing messages). You can't have 2 executors in the chain. That's by design. Probably bad, but you have to deal with it... > > > chain.addLast("sslFilter", sslFilter) > > chain.addLast("logger", new LoggingFilter()) > > chain.addLast("readPool", new ExecutorFilter(coreSize, > maxSize,keepAliveTime, TimeUnit.MILLISECONDS, > IoEventType.MESSAGE_RECEIVED)) > > chain.addLast("codec", protocolCodecFilter) > > chain.addLast("writePool", new ExecutorFilter(coreSize, > maxSize,keepAliveTime, TimeUnit.MILLISECONDS, IoEventType.WRITE)) > > > > Can you please approve that using the above chaining is best for my > needs? If you remove the first executor, that should be fine. > > > Emmanuel, I read once your reservation about placing the executor first, > you said: "...as the executor is the fist one in the filters chain, you > may not be able to deal with fragmented messages" > > Can you explain what did you mean fragmented messages? You have no guaranty whatsoever tht the data coming from a client will be received in one block. You can even expect to get the messageReceived() event be called for every single byte in the message (not likely to happen, but it's just to depict the worst possible situation). Generally speaking, if you have big messages (ie, more than a few Kb), you may get it split in more than one PDU. You have to deal with this, using a cumulativeProtocolDecoder, or writing your own logic to gather the bytes. > And in other post you said " Executor can leads to some issue if you are > under heavy load " can you elaborate on that? Taking into consideration > these comments of yours what is the best approach in case of heavy load? As soon as you spread the loads on many threads, at some point, you may creates thousands of threads if one of your clients send thousands of messages (let's say it's a DDOS). You probably have some ways to deal with such issues on a FW level, but it's important to keep in mind that such problems can arise. However, it's more or less a warning, as every application open to the outside world can have the exact same issues. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com Click here to report this message as SPAM: http://vsp.ateranetworks.com/ReportSpam.php?sid=20f993baa0bc4eff02b0f7eae7abc329_887ba19d853cfc2326e0e77f28a509c5 -- Powered by ATERA Networks --
