Hi André, Ah I see. There's a JIRA issue directly related with this:
https://issues.apache.org/jira/browse/DIRMINA-519 If you are interested in writing this filter and contribute to the MINA project, you are more than welcomed! :) Cheers, André Martin wrote: > Hi Trustin Lee, > I found the bottleneck: The problem is/was the text line length: The > average line length of my 18 MB test file was just 100 Bytes. > Unfortunately Mina doesn't seem to have an appropriate buffering > mechanisms on the sender side, so when I use the TextLineEncoder.java > (as suggested in the tutorials), it will create a bunch of (small) > IoBuffer objects for each individual line (or message object) which are > then put into the writer's queue. This is quite inefficient since > thousand of small IoBuffer objects will be sent over the wire instead of > creating one buffer with a more appropriate buffer size of e.g. 64kB.. > Instead of writing & using ProtocolCodecFilters as proposed in the > tutorial, I create a 64k large Buffer object now and let it fill up with > multiple message objects, and then I do a "flush" by calling the write > method and allocate a new 64k Buffer in order to put the next "incoming" > messages in. This performs way better. > Maybe we could/should provide an aggregate filter that does the > consolidation as described above if the buffer size is relatively small. > I think that would probably integrate in the current architecture? What > do you think? > > Cu on the 'net, > Bye - bye, > > <<<<< André <<<< >>>> èrbnA >>>>> > > > "이희승 (Trustin Lee) <[EMAIL PROTECTED]>" wrote: >> I failed to find any specific bottleneck in MINA implementation >> unfortunately, and it is also true that MINA-based client performs much >> worse than java.io based implementation. What's interesting is that >> MINA-based server performs pretty well comparing to the java.io based >> server implementation. It's still a mystery for me. Please let me know >> if you have any clue. >> >> Thanks, >> >> André Martin wrote: >> >>> Hi, >>> here you go - the server code first: >>> >>> import java.io.*; >>> import java.net.InetSocketAddress; >>> import java.sql.SQLException; >>> >>> import org.apache.commons.logging.*; >>> import org.apache.mina.common.*; >>> import org.apache.mina.filter.codec.ProtocolCodecFilter; >>> import org.apache.mina.transport.socket.nio.NioSocketAcceptor; >>> import org.apache.mina.filter.codec.textline.*; >>> >>> public class MinaPerformanceTesterServer >>> { >>> private static class InputStreamIOHandler extends IoHandlerAdapter >>> { >>> private BufferedWriter out; >>> public InputStreamIOHandler() >>> { >>> try {out = new BufferedWriter(new >>> FileWriter("dataOut.txt"));} >>> catch (IOException e) {e.printStackTrace();} >>> } >>> >>> public void messageReceived(IoSession session, Object message) >>> { >>> try {out.write(message.toString() + "\r\n");} >>> catch (IOException e) {e.printStackTrace();} >>> } >>> } >>> public static void main(String[] args) throws IOException, >>> SQLException, Exception >>> { >>> NioSocketAcceptor acceptor = new NioSocketAcceptor(); >>> >>> acceptor.getFilterChain().addLast("codec", new >>> ProtocolCodecFilter( >>> new TextLineCodecFactory())); >>> >>> acceptor.setHandler(new InputStreamIOHandler()); >>> try {acceptor.bind(new InetSocketAddress("localhost", 8081));} >>> catch (IOException e) {e.printStackTrace();} >>> } >>> } >>> >>> and here the client code: >>> >>> import java.io.*; >>> import java.net.InetSocketAddress; >>> import java.sql.SQLException; >>> import java.text.DecimalFormat; >>> >>> import org.apache.commons.logging.*; >>> import org.apache.mina.common.*; >>> import org.apache.mina.filter.codec.ProtocolCodecFilter; >>> import org.apache.mina.transport.socket.SocketConnector; >>> import org.apache.mina.transport.socket.nio.NioSocketConnector; >>> import org.apache.mina.filter.codec.textline.*; >>> >>> public class MinaPerformanceTesterClient >>> { >>> public static void main(String[] args) throws IOException, >>> SQLException, Exception >>> { >>> IoSession session = null; >>> SocketConnector connector = new NioSocketConnector(); >>> connector.getFilterChain().addLast("codec", >>> new ProtocolCodecFilter(new TextLineCodecFactory())); >>> connector.setHandler(new IoHandlerAdapter()); >>> ConnectFuture connectFuture = connector.connect( >>> new InetSocketAddress("localhost", 8081)); >>> connectFuture.awaitUninterruptibly(3000); >>> try {session = connectFuture.getSession();} >>> catch (RuntimeIoException e) {e.printStackTrace();} >>> try >>> { >>> BufferedReader in = >>> new BufferedReader(new FileReader("dataIn.txt")); >>> String line; >>> while((line = in.readLine())!=null) session.write(line); >>> in.close(); >>> } >>> catch (IOException e) {e.printStackTrace();} >>> } >>> } >>> >>> >>> "이희승 (Trustin Lee) <[EMAIL PROTECTED]>" wrote: >>> >>>> Hi André, >>>> >>>> Could you please provide the source code so we can figure out what's >>>> wrong? >>>> >>>> André Martin wrote: >>>> >>>> >>>>> Hi everyone, >>>>> I have a question concerning the performance & throughput of MINA. I >>>>> have the following setup: A client that reads a file line by line >>>>> using >>>>> the BufferedReader and a server that receives each line (using >>>>> TextLineCodec) and writes it on disk using a BufferedWriter. The test >>>>> file has a size of about 18 MBs. When I redirect the "input stream" >>>>> directly to the output stream, the entire job takes just one >>>>> second. If >>>>> there is the MINA client & server "in between", it takes 13 seconds >>>>> - a >>>>> throughput of less than 1 MB/s. The client & server are running on the >>>>> same machine. >>>>> Any ideas why the performance is so weak or how I can tweak it? Thanks >>>>> in advance for your insights. >>>>> >>>>> Cu on the 'net, >>>>> Bye - bye, >>>>> >>>>> <<<<< André <<<< >>>> èrbnA >>>>> >>>>> > > -- Trustin Lee - Principal Software Engineer, JBoss, Red Hat -- what we call human nature is actually human habit -- http://gleamynode.net/
signature.asc
Description: OpenPGP digital signature
