Re: What about creating a protocol repository ?
After the wonderful discussion yesterday, went back and had a quick look at the codec that I intended to design. Here is what I needed to replace existing blocking IO 1. POJO's - The High level representation of a protocol (already there) 2. Decoders - How I collect information from lower layer eg. CumulativeProtocolDecoder..etc and convert them to POJO 3. Encoder's - How to convert POJO to byte array For us, the tricky part is POJO's. There are two questions 1. Do we maintain protocol POJO's? 2. Do we use an 3rd party POJO library (in my case its jsmpp) Encoder/Decoder shall depend upon POJO's. So if we had to create a Codec repository, we have to decide which approach to use. It could be a mix of both. There are pro's and con's of each and not sure of the way forward. May be we can look at it once we actually create the repository. thanks ashish
Slow transfer rate with mina
Hi,
I'm currently experiencing some issue with MINA, I'm using it from the
scala language (which is definitely not the source of the problem).
When I send big amount of data through the network, the upload rate is
first high and then it slows down dramatically (I'm running the server
and the client on the same computer).
I believe it is due to bad usage of the library.
Did anyone experience a similar issue ? Does anyone can correct me if
I'm doing it wrong.
Here is how I encode message (this is scala, but it is still readable
for a java developer)
val bufferSize = 200 * 1024 * 1024
var buffer = IoBuffer.allocate( bufferSize , false )
buffer.putInt( bufferSize - 4 )
[...]
buffer.flip()
_session.write( buffer )
Here is how I decode messages
def doDecode(session : IoSession, in : IoBuffer, out :
ProtocolDecoderOutput) : Boolean = {
log.info( Decoding current buffer (size:{}ko) , in.remaining()/1024 )
if( in.prefixedDataAvailable(4) ) {
log.debug( Decoding started )
val remainingBytesToRead = in.getInt()
val buffer = new Array[Byte]( _cmdSize )
in.get( buffer )
val cmd = new String( buffer , _utf8Charset )
[...]
true
}
else {
false
}
}
Thanks,
Steve
[jira] Issue Comment Edited: (DIRMINA-32) Revise JavaDoc, PPT, and Tutorial
[ https://issues.apache.org/jira/browse/DIRMINA-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694034#action_12694034 ] Julien Vermillard edited comment on DIRMINA-32 at 3/31/09 2:31 AM: --- Thanks to last Emmanuel preso , we got a brand new presentation document. Tutorials javadoc are ok now. Let's close this issue ? WDYT ? was (Author: vrm): Thanks to last Emmanuel perso , we got a brand new presentation document. Tutorials javadoc are ok now. Let's close this issue ? WDYT ? Revise JavaDoc, PPT, and Tutorial - Key: DIRMINA-32 URL: https://issues.apache.org/jira/browse/DIRMINA-32 Project: MINA Issue Type: Task Components: Web Site / Documentation Affects Versions: 2.0.0-M3 Reporter: Trustin Lee Fix For: 2.0.0-RC1 0.9 will integrate I/O layer and Protocol layer. We need up-to-date documentation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Codec Factories
On Tue, Mar 31, 2009 at 10:24 AM, Emmanuel Lecharny elecha...@apache.org wrote: Maarten Bosteels wrote: The codecfactory in the tutorial is stateless : http://mina.apache.org/tutorial-on-protocolcodecfilter-for-mina-2x.html Well, if you create new instances of encoder/decoder per session, it doesn't matter too much that the factory is stateless or not. In fact, the factory being statefull or stateless is irrelevant. I was talking about the encoder/decoder of course. But it's the codecfactory that has to know whether the decoder is stateless or not. (I think encoder will almost always be stateless because all data is already available) It's the encoder/decoder state which is important. As soon as you have new instances of those guys for each new session, and as soon as you have a guarantee that you can't enter the encoder/decoder in two different threads for the same session, then you are safe. FWIK, if you encoder/decoder are stateless, you should never instanciate them for each session. If they are statefull, then again, you have to think about the consequences. I don't think it's a good idea to design a stateful codec by storing data into the encoder/decoder. I would rather store those data in a context object, stores within the session. This is what we do for the LDAP protocol. So at the end of the day, I don't think it make any sense to instanciate encoder/decoder at all ... That is/was my opinion as well, but in that thread other people expressed their preference to use stateful decoders... Thanks Marteen for the pointers ! You do it on purpose ! ;-) Maarten -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
[jira] Commented: (DIRMINA-32) Revise JavaDoc, PPT, and Tutorial
[ https://issues.apache.org/jira/browse/DIRMINA-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694037#action_12694037 ] Ashish Paliwal commented on DIRMINA-32: --- Lets close this issue. We have new PPT, most of the documentation section is updated. Javadoc's are decent, and can be refined as we go forward. Revise JavaDoc, PPT, and Tutorial - Key: DIRMINA-32 URL: https://issues.apache.org/jira/browse/DIRMINA-32 Project: MINA Issue Type: Task Components: Web Site / Documentation Affects Versions: 2.0.0-M3 Reporter: Trustin Lee Fix For: 2.0.0-RC1 0.9 will integrate I/O layer and Protocol layer. We need up-to-date documentation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Sai Pullabhotla is a new committer
Welcome Sai! Enjoy your stay 2009/3/31 Maarten Bosteels mbosteels@gmail.com: Welcome Sai ! Maarten On Mon, Mar 30, 2009 at 9:56 AM, Julien Vermillard jvermill...@archean.fr wrote: Le Sun, 29 Mar 2009 20:29:06 +0200, Niklas Gustavsson nik...@protocol7.com a écrit : Hi I'm very glad to announce that the MINA PMC has voted in Sai Pullabhotla as a new MINA committer based on his contributions on the FtpServer subproject. Welcome Sai, hope you'll have fun around here! :-) /niklas Warm Welcome Sai ! Julien
[jira] Closed: (DIRMINA-32) Revise JavaDoc, PPT, and Tutorial
[ https://issues.apache.org/jira/browse/DIRMINA-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Vermillard closed DIRMINA-32. Resolution: Fixed Revise JavaDoc, PPT, and Tutorial - Key: DIRMINA-32 URL: https://issues.apache.org/jira/browse/DIRMINA-32 Project: MINA Issue Type: Task Components: Web Site / Documentation Affects Versions: 2.0.0-M3 Reporter: Trustin Lee Fix For: 2.0.0-RC1 0.9 will integrate I/O layer and Protocol layer. We need up-to-date documentation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DIRMINA-453) Multiple IoServices for one java.nio.Selector
[ https://issues.apache.org/jira/browse/DIRMINA-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Vermillard updated DIRMINA-453: -- Fix Version/s: 3.0.0-M1 Multiple IoServices for one java.nio.Selector - Key: DIRMINA-453 URL: https://issues.apache.org/jira/browse/DIRMINA-453 Project: MINA Issue Type: New Feature Components: Core Affects Versions: 2.0.0-M1 Environment: All Reporter: Mark Webb Fix For: 3.0.0-M1 Please see the discussion: http://www.nabble.com/multiple-handlers-per-IoService-t4481513s16868.html#a12779214 Based on discussions with Trustin, this is functionality that will be in the 2.0 release. It seems to be inefficient to create a new selector/thread for each IoService. This means that an application that is listening on 2 ports, would need two java.nio.Selectors, two Selector worker threads and each have a pool of workers. Not only does this create much duplication, but more threads will be created than may be necessary. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DIRMINA-453) Multiple IoServices for one java.nio.Selector
[ https://issues.apache.org/jira/browse/DIRMINA-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694109#action_12694109 ] Emmanuel Lecharny commented on DIRMINA-453: --- I *think* we already have the capacity to use only one single selector for many sockets. No need to create new acceptors, just inject the ServerSocket into this NioAcceptor, and you are done. Multiple IoServices for one java.nio.Selector - Key: DIRMINA-453 URL: https://issues.apache.org/jira/browse/DIRMINA-453 Project: MINA Issue Type: New Feature Components: Core Affects Versions: 2.0.0-M1 Environment: All Reporter: Mark Webb Fix For: 3.0.0-M1 Please see the discussion: http://www.nabble.com/multiple-handlers-per-IoService-t4481513s16868.html#a12779214 Based on discussions with Trustin, this is functionality that will be in the 2.0 release. It seems to be inefficient to create a new selector/thread for each IoService. This means that an application that is listening on 2 ports, would need two java.nio.Selectors, two Selector worker threads and each have a pool of workers. Not only does this create much duplication, but more threads will be created than may be necessary. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DIRMINA-616) New release.xml file
[ https://issues.apache.org/jira/browse/DIRMINA-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694156#action_12694156 ] Julien Vermillard commented on DIRMINA-616: --- look like it's there : http://mvnrepository.com/artifact/org.apache.maven.plugins/maven-assembly-plugin going to give a try New release.xml file Key: DIRMINA-616 URL: https://issues.apache.org/jira/browse/DIRMINA-616 Project: MINA Issue Type: Improvement Reporter: Barend Garvelink Assignee: Julien Vermillard Attachments: release.xml In response to http://markmail.org/message/fhjl74dnlyaojrzr , here's an updated release.xml file. It generally works as it should, but it's affected by http://jira.codehaus.org/browse/MASSEMBLY-285, which causes duplicate entries in the lib/ directory inside the archive. When extracting the archive using 7zip for win32, I get prompted to overwrite/skip/cancel for each duplicate entry. Other archivers may file entirely (?). The only work-around I can come up with as a fix right now is to add a maven-antrun-plugin to the package phase of the root pom which post-processes the release file to fix this. I haven't done that, but if MASSEMBLY-285 isn't fixed before the next release, we may have to. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: svn commit: r760049 - in /mina/trunk: ./ core/ example/ filter-codec-netty/ filter-compression/ integration-beans/ integration-jmx/ integration-ognl/ integration-xbean/ legal/ legal/src/ legal/s
broke transport-serial bad, bad, bad :o) always use mvn with -Dwith-LGPL-dependencies for including transport-serial module. Julien 2009/3/30 akaras...@apache.org: Author: akarasulu Date: Mon Mar 30 17:10:38 2009 New Revision: 760049 URL: http://svn.apache.org/viewvc?rev=760049view=rev Log: [DIRMINAA-673]: Applying patch from Chris Custine to fix issues with OSGi Added: mina/trunk/legal/ mina/trunk/legal/pom.xml mina/trunk/legal/src/ mina/trunk/legal/src/main/ mina/trunk/legal/src/main/resources/ mina/trunk/legal/src/main/resources/notices.xml mina/trunk/parent/ mina/trunk/parent/pom.xml Modified: mina/trunk/core/pom.xml mina/trunk/example/pom.xml mina/trunk/filter-codec-netty/pom.xml mina/trunk/filter-compression/LICENSE.jzlib.txt mina/trunk/filter-compression/pom.xml mina/trunk/integration-beans/pom.xml mina/trunk/integration-jmx/pom.xml mina/trunk/integration-ognl/pom.xml mina/trunk/integration-xbean/pom.xml mina/trunk/pom.xml mina/trunk/release.xml mina/trunk/statemachine/pom.xml mina/trunk/transport-apr/pom.xml Modified: mina/trunk/core/pom.xml URL: http://svn.apache.org/viewvc/mina/trunk/core/pom.xml?rev=760049r1=760048r2=760049view=diff == --- mina/trunk/core/pom.xml (original) +++ mina/trunk/core/pom.xml Mon Mar 30 17:10:38 2009 @@ -23,7 +23,7 @@ modelVersion4.0.0/modelVersion parent groupIdorg.apache.mina/groupId - artifactIdbuild/artifactId + artifactIdmina-parent/artifactId version2.0.0-RC1-SNAPSHOT/version /parent artifactIdmina-core/artifactId Modified: mina/trunk/example/pom.xml URL: http://svn.apache.org/viewvc/mina/trunk/example/pom.xml?rev=760049r1=760048r2=760049view=diff == --- mina/trunk/example/pom.xml (original) +++ mina/trunk/example/pom.xml Mon Mar 30 17:10:38 2009 @@ -20,7 +20,7 @@ modelVersion4.0.0/modelVersion parent groupIdorg.apache.mina/groupId - artifactIdbuild/artifactId + artifactIdmina-parent/artifactId version2.0.0-RC1-SNAPSHOT/version /parent artifactIdmina-example/artifactId @@ -47,6 +47,7 @@ groupIdjmock/groupId artifactIdjmock/artifactId version1.2.0/version + scopetest/scope /dependency /dependencies /project Modified: mina/trunk/filter-codec-netty/pom.xml URL: http://svn.apache.org/viewvc/mina/trunk/filter-codec-netty/pom.xml?rev=760049r1=760048r2=760049view=diff == --- mina/trunk/filter-codec-netty/pom.xml (original) +++ mina/trunk/filter-codec-netty/pom.xml Mon Mar 30 17:10:38 2009 @@ -23,7 +23,7 @@ modelVersion4.0.0/modelVersion parent groupIdorg.apache.mina/groupId - artifactIdbuild/artifactId + artifactIdmina-parent/artifactId version2.0.0-RC1-SNAPSHOT/version /parent artifactIdmina-filter-codec-netty/artifactId Modified: mina/trunk/filter-compression/LICENSE.jzlib.txt URL: http://svn.apache.org/viewvc/mina/trunk/filter-compression/LICENSE.jzlib.txt?rev=760049r1=760048r2=760049view=diff == --- mina/trunk/filter-compression/LICENSE.jzlib.txt (original) +++ mina/trunk/filter-compression/LICENSE.jzlib.txt Mon Mar 30 17:10:38 2009 @@ -1,29 +0,0 @@ -JZlib 0.0.* were released under the GNU LGPL license. Later, we have switched -over to a BSD-style license. - --- -Copyright (c) 2000,2001,2002,2003 ymnk, JCraft,Inc. All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - - 1. Redistributions of source code must retain the above copyright notice, - this list of conditions and the following disclaimer. - - 2. Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in - the documentation and/or other materials provided with the distribution. - - 3. The names of the authors may not be used to endorse or promote products - derived from this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, -INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND -FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JCRAFT, -INC. OR ANY CONTRIBUTORS TO THIS SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT, -INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, -OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
FYI: epoll issue fixed in b54
Salut, The fix for epoll File exists and spinning Selector issue this community has experienced will be available with jdk7 build 54. I recommend you test it and report any failure as this fix will be backported to JDK 6 eventually (no idea which version...will report as soon as I know). A+ -- Jeanfrancois
Re: FYI: epoll issue fixed in b54
Jeanfrancois Arcand wrote: Salut, The fix for epoll File exists and spinning Selector issue this community has experienced will be available with jdk7 build 54. I recommend you test it and report any failure as this fix will be backported to JDK 6 eventually (no idea which version...will report as soon as I know). Great ! Nothing expected for Java 5 ? -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
Re: FYI: epoll issue fixed in b54
Le Tue, 31 Mar 2009 12:04:22 -0400, Jeanfrancois Arcand jeanfrancois.arc...@sun.com a écrit : Salut, The fix for epoll File exists and spinning Selector issue this community has experienced will be available with jdk7 build 54. I recommend you test it and report any failure as this fix will be backported to JDK 6 eventually (no idea which version...will report as soon as I know). A+ -- Jeanfrancois Merci for the hint, going to add that to the jira issue. Julien signature.asc Description: PGP signature
[jira] Commented: (DIRMINA-678) NioProcessor 100% CPU usage on Linux (epoll selector bug)
[ https://issues.apache.org/jira/browse/DIRMINA-678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694171#action_12694171 ] Julien Vermillard commented on DIRMINA-678: --- From Jeanfrancois Arcand : The fix for epoll File exists and spinning Selector issue this community has experienced will be available with jdk7 build 54. I recommend you test it and report any failure as this fix will be backported to JDK 6 eventually (no idea which version...will report as soon as I know). NioProcessor 100% CPU usage on Linux (epoll selector bug) - Key: DIRMINA-678 URL: https://issues.apache.org/jira/browse/DIRMINA-678 Project: MINA Issue Type: Bug Components: Core Affects Versions: 2.0.0-M4 Environment: CentOS 5.x, 32/64-bit, 32/64-bit Sun JDK 1.6.0_12, also _11/_10/_09 and Sun JDK 1.7.0 b50, Kernel 2.6.18-92.1.22.el5 and also older versions, Reporter: Serge Baranov Fix For: 2.0.0 Attachments: snap973.png, snap974.png It's the same bug as described at http://jira.codehaus.org/browse/JETTY-937 , but affecting MINA in the very similar way. NioProcessor threads start to eat 100% resources per CPU. After 10-30 minutes of running depending on the load (sometimes after several hours) one of the NioProcessor starts to consume all the available CPU resources probably spinning in the epoll select loop. Later, more threads can be affected by the same issue, thus 100% loading all the available CPU cores. Sample trace: NioProcessor-10 [RUNNABLE] CPU time: 5:15 sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) sun.nio.ch.EPollArrayWrapper.poll(long) sun.nio.ch.EPollSelectorImpl.doSelect(long) sun.nio.ch.SelectorImpl.lockAndDoSelect(long) sun.nio.ch.SelectorImpl.select(long) org.apache.mina.transport.socket.nio.NioProcessor.select(long) org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run() org.apache.mina.util.NamePreservingRunnable.run() java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) java.util.concurrent.ThreadPoolExecutor$Worker.run() java.lang.Thread.run() It seems to affect any NIO based Java server applications running in the specified environment. Some projects provide workarounds for similar JDK bugs, probably MINA can also think about a workaround. As far as I know, there are at least 3 users who experience this issue with Jetty and all of them are running CentOS (some distribution default setting is a trigger?). As for MINA, I'm not aware of similar reports yet. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DIRMINA-678) NioProcessor 100% CPU usage on Linux (epoll selector bug)
[ https://issues.apache.org/jira/browse/DIRMINA-678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694177#action_12694177 ] Victor N commented on DIRMINA-678: -- I see a similar behavior on our production servers, with the same picture of CPU usage after several hours. But at the same time I can see that our server begins an intensive garbage allocation, so I am not sure yet whether this is the Selector problem or something wrong in our code. I will come back later to clarify the result. This problem of CPU usage occurs in our latest server version where we use NioSocketConnector and MinaSocketAcceptor. In previous versions, we used only NioSocketAcceptor and it worked well for weeks. NioProcessor 100% CPU usage on Linux (epoll selector bug) - Key: DIRMINA-678 URL: https://issues.apache.org/jira/browse/DIRMINA-678 Project: MINA Issue Type: Bug Components: Core Affects Versions: 2.0.0-M4 Environment: CentOS 5.x, 32/64-bit, 32/64-bit Sun JDK 1.6.0_12, also _11/_10/_09 and Sun JDK 1.7.0 b50, Kernel 2.6.18-92.1.22.el5 and also older versions, Reporter: Serge Baranov Fix For: 2.0.0 Attachments: snap973.png, snap974.png It's the same bug as described at http://jira.codehaus.org/browse/JETTY-937 , but affecting MINA in the very similar way. NioProcessor threads start to eat 100% resources per CPU. After 10-30 minutes of running depending on the load (sometimes after several hours) one of the NioProcessor starts to consume all the available CPU resources probably spinning in the epoll select loop. Later, more threads can be affected by the same issue, thus 100% loading all the available CPU cores. Sample trace: NioProcessor-10 [RUNNABLE] CPU time: 5:15 sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) sun.nio.ch.EPollArrayWrapper.poll(long) sun.nio.ch.EPollSelectorImpl.doSelect(long) sun.nio.ch.SelectorImpl.lockAndDoSelect(long) sun.nio.ch.SelectorImpl.select(long) org.apache.mina.transport.socket.nio.NioProcessor.select(long) org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run() org.apache.mina.util.NamePreservingRunnable.run() java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) java.util.concurrent.ThreadPoolExecutor$Worker.run() java.lang.Thread.run() It seems to affect any NIO based Java server applications running in the specified environment. Some projects provide workarounds for similar JDK bugs, probably MINA can also think about a workaround. As far as I know, there are at least 3 users who experience this issue with Jetty and all of them are running CentOS (some distribution default setting is a trigger?). As for MINA, I'm not aware of similar reports yet. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: FYI: epoll issue fixed in b54
Salut, Emmanuel Lecharny wrote: Jeanfrancois Arcand wrote: Salut, The fix for epoll File exists and spinning Selector issue this community has experienced will be available with jdk7 build 54. I recommend you test it and report any failure as this fix will be backported to JDK 6 eventually (no idea which version...will report as soon as I know). Great ! Nothing expected for Java 5 ? No, unfortunately. The fix will be available in 6u16 and no more lower version. A+ -- Jeanfrancois
Re: FYI: epoll issue fixed in b54
Jeanfrancois Arcand wrote: Salut, Emmanuel Lecharny wrote: Jeanfrancois Arcand wrote: Salut, The fix for epoll File exists and spinning Selector issue this community has experienced will be available with jdk7 build 54. I recommend you test it and report any failure as this fix will be backported to JDK 6 eventually (no idea which version...will report as soon as I know). Great ! Nothing expected for Java 5 ? No, unfortunately. The fix will be available in 6u16 and no more lower version. That should be enough. Java 6 is way better that Java 5 anyway :) Thanks for the info ! -- -- cordialement, regards, Emmanuel Lécharny www.iktek.com directory.apache.org
