[jira] Closed: (DIRMINA-289) Change MINA to create heap buffers by default.
[ http://issues.apache.org/jira/browse/DIRMINA-289?page=all ] Trustin Lee closed DIRMINA-289. --- Resolution: Fixed Assignee: Trustin Lee > Change MINA to create heap buffers by default. > -- > > Key: DIRMINA-289 > URL: http://issues.apache.org/jira/browse/DIRMINA-289 > Project: Directory MINA > Issue Type: Task > Components: Core >Reporter: Trustin Lee > Assigned To: Trustin Lee >Priority: Trivial > Fix For: 1.1 > > > Many benchmark reports have shown that heap buffer performs better in many > modern JVMs. Using heap buffers also helps us from keep answering to > increase the direct buffer memory size in user's JVM settings. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Moving setUseDirectBuffers (now setPreferDirectBuffers) to ByteBufferAllocator implementations.
Hi, I think we need to move useDirectBuffers property to the respective ByteBufferAllocator implementations to give users full control over how buffers are allocated. What do you think? By default, allocating only heap buffers could be a viable option. Trustin -- what we call human nature is actually human habit -- http://gleamynode.net/ -- PGP key fingerprints: * E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E * B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
[jira] Created: (DIRMINA-290) Make IoService (IoConnector and IoAcceptor) manage only one service/connection.
Make IoService (IoConnector and IoAcceptor) manage only one service/connection. --- Key: DIRMINA-290 URL: http://issues.apache.org/jira/browse/DIRMINA-290 Project: Directory MINA Issue Type: Task Components: Core Reporter: Trustin Lee Fix For: 1.1 Allowing IoAcceptor to bind more than one ports per one instance makes the API complex. Restricting an acceptor to bind to only one port will dramatically simplify the API. The properties of IoServiceConfig will be merged into IoService. Applying this practice to IoConnector will also be very helpful to implement reconnection feature. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (DIRMINA-289) Change MINA to create heap buffers by default.
Change MINA to create heap buffers by default. -- Key: DIRMINA-289 URL: http://issues.apache.org/jira/browse/DIRMINA-289 Project: Directory MINA Issue Type: Task Components: Core Reporter: Trustin Lee Priority: Trivial Fix For: 1.1 Many benchmark reports have shown that heap buffer performs better in many modern JVMs. Using heap buffers also helps us from keep answering to increase the direct buffer memory size in user's JVM settings. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: ThreadPool in Mina with concurrent from Java5 Howto ?
Hi Frederic, On 9/26/06, Frédéric Brégier <[EMAIL PROTECTED]> wrote: Sorry to come back on this subject, but I am a bit confused with the Mina logic for ThreadPool. First, what I suppose to understand : - First IoAcceptor have one ThreadPool to handle incoming IoSessions. - Second, the IoConfig use a ThreadPool to handle process of filters. - Eventually, in the filter chain, we can insert a new ThreadPool for specific filters that are supposed to consume a lot of cpu (for example compression or ssl ?). For me, the doc says there are two level of ThreadPool management, but in the API I found three levels. Do I missed something ? The documentation is outdated seriously. First, there's no such level in MINA. In MINA 0.8, we had two layers, but they are merged into one layer, and there's only one filter chain per session. The thread model is decided by the number of ExecutorFilter you inserted into the chain. By default, one ExecutorFilter is inserted, but you can change this behavior by setting a different ThreadModel. Please refer to ThreadModel JavaDoc and its subclasses. 1) Put a fixed threadPool to IoAcceptor : let me known if someone can say if it is better here to use a CachedThreadPool or a FixedThreadPool ? It highly depends I woul say how Mina is using it. IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1); You usually don't need to specify the executor as a constructor parameter. It's for some containers or microkernels which want to manage threading. 2) Put a cached threadPool in IoAcceptorConfig : I don't know here if it is the same than (1) or not ? When I look into the code of Mina, it seems not, but I am really not sure. (1) and (2) are very different as I answered above. IoAccaptorConfig config = new SocketAcceptorConfig(); config.setThreadModel(executor2); Also I don't understand why Peter (and in the old doc) I should use : IoConfig.setThreadModel( ThreadModel.MANUAL ); You can use ThreadModel.MANUAL if you want, but then you have to insert ExecutorFilter by yourself. The default thread model is a PooledThreadModel. ThreadModel extends IoFilterChainBuilder, and inserts an appropriate thread pool filter into the filter chain when a session is created. So, if you set the right ThreadModel, you don't need to insert another thread pool filter by yourself. HTH, Trustin -- what we call human nature is actually human habit -- http://gleamynode.net/ -- PGP key fingerprints: * E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E * B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: [mina] memory issues
Hi Colin,MINA has its own mailing list. Please send an empty message to [EMAIL PROTECTED] to subscribe to the list, and send your question to mina-dev@directory.apache.org from the next time.On 10/18/06, Colin Koeck < [EMAIL PROTECTED]> wrote:I recently upgraded from netty2 to mina, and noticed mina is much more stable which I am very happy about. However, it seems to be a memory hog which concerns me a little. I'm trying to push about 75,000 messages, each about 400 characters in length, through a client/server architecture using mina..and the software throws a OutOfMemoryError about half-way through. The only way I got it to successfully finish was to execute the software with -Xmx1000M!You will have to increase your direct buffer memory. Please refer to our FAQ page.: http://directory.apache.org/subprojects/mina/faq.html#connection-resetI think it might be a good idea to use heap buffers by default. What do you think, everyone? I'm going to profile the software later this week, but I'm wondering if the results I got with mina are expected, and if there is any way to cut corners and use less memory with mina. I'd appreciate it if anyone has any input on the subject. Thank you. Please let us know whenever you have any idea to improve MINA. All ideas will be appreciated.HTH,Trustin-- what we call human nature is actually human habit --http://gleamynode.net/--PGP key fingerprints:* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
[mina] memory issues
I recently upgraded from netty2 to mina, and noticed mina is much more stable which I am very happy about. However, it seems to be a memory hog which concerns me a little. I'm trying to push about 75,000 messages, each about 400 characters in length, through a client/server architecture using mina..and the software throws a OutOfMemoryError about half-way through. The only way I got it to successfully finish was to execute the software with -Xmx1000M! I'm going to profile the software later this week, but I'm wondering if the results I got with mina are expected, and if there is any way to cut corners and use less memory with mina. I'd appreciate it if anyone has any input on the subject. Thank you. Colin Koeck
ApacheCon presentation materials
Hi all, I gave two talks at the last ApacheCon in Austin where we really enjoyed our time. Presentations and other materials are available under the following svn repos: https://svn.apache.org/repos/asf/directory/sandbox/ersiner/apachecon-us06-fr14 https://svn.apache.org/repos/asf/directory/sandbox/ersiner/apachecon-us06-fr20 Feel free to ask any questions about these stuff. Cheers, -- Ersin
Re: Solaris x86 installer - headless
Graham Leggett wrote: Alex Karasulu wrote: Sorry I've been horible with email but as we already discussed at ApacheCon there is no solaris pkg for ApacheDS. Perhaps we should consider making a tarball and a pkg for solaris platforms. I did the Solaris package scripts for httpd, apr and subversion, I can see what I can do for this - what tool did you use for packaging, I remember you said it was a maven plugin? Yeah it's a custom Maven plugin that I wrote a while back. The thing has no documentation :(. The code is located over here if you want to take a look? http://svn.apache.org/repos/asf/directory/trunks/daemon/plugin I have a RPM spec template here which might help shed some light as to the variable replacements going on in such files. http://svn.apache.org/repos/asf/directory/trunks/daemon/plugin/src/main/resources/org/apache/directory/daemon/installers/rpm/ Perhaps you might build a static solaris package for ApacheDS and I can incorporate that as a target in the plugin. Thanks, Alex P.S. Hope your trip home was a safe and relaxing one.
Re: Thread Pool filter - performance
Hello, Robert Greig wrote: In my tests the throughput in our application is more than doubled when using our thread pool as compared with the default MINA one. Impressive. If other people are interested in trying our filter I'd be very interested in seeing the performance they get. I've adopted Asyncweb (a Mina base HTTP server) do use your thread pool and tried to run some benchmarks against it, although it works, when I fire the load tester I get lots of errors. Read errors: ERROR [SocketAcceptorIoProcessor-0.0] 17 Oct 2006 17:02:24 - Exception event constructed: java.io.IOException: Connection reset by peer java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) at sun.nio.ch.IOUtil.read(IOUtil.java:206) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) at org.apache.mina.transport.socket.nio.SocketIoProcessor.read(SocketIoProcessor.java:272) at org.apache.mina.transport.socket.nio.SocketIoProcessor.process(SocketIoProcessor.java:246) at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$4(SocketIoProcessor.java:235) at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:564) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:43) at java.lang.Thread.run(Thread.java:619) INFO [SocketAcceptorIoProcessor-0.0] 17 Oct 2006 17:02:24 - Destroy called on PoolingFilter AsynchronousWriteFilter INFO [SocketAcceptorIoProcessor-0.0] 17 Oct 2006 17:02:24 - Destroy called on PoolingFilter AsynchronousReadFilter Write errors: ERROR [SocketAcceptorIoProcessor-0.0] 17 Oct 2006 18:12:13 - Exception event constructed: java.io.IOException: Broken pipe java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcher.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104) at sun.nio.ch.IOUtil.write(IOUtil.java:75) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334) at org.apache.mina.transport.socket.nio.SocketIoProcessor.doFlush(SocketIoProcessor.java:487) at org.apache.mina.transport.socket.nio.SocketIoProcessor.doFlush(SocketIoProcessor.java:419) at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$5(SocketIoProcessor.java:380) at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:572) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:43) at java.lang.Thread.run(Thread.java:619) These errors don't show up with the default thread pool. Probably due to this errors I didn't get any throughput increase ... however as a quick experiment I commented out the: session.getFilterChain().fireExceptionCaught( session, e ); lines in the SocketIoProcessor and I did get a slight throughput increase with lower CPU utilization. -- Luis Neves
Re: WARN: Attribute does not have normalizer
1. IMHO, a level of INFO would be better for the 'normalizer' entries - would you agree? My users are equally sensitive to both ERROR and WARN log entries.Yep, this can be changed. Can you fill a JIRA for that, so we won't loose the request ? 2. I think the BTreePartition WARNs could be avoided if the code that creates the system partition explicitly sets the cache size on the index attributes.Sure. INFO seems much more adequat for defaulting a value. I know this as I create a custom partition and it was generating WARN messages! I see from the Spring configuration file that it sets the index attributes. Is there a way to ensure the system partition sets the index attributes when I'm not using Spring?Hmmm. AFAIR, index are always defaulted for some attributes in system partition like ObjectClass, but not for any other attribute (I may be wrong, this has to be doublec checked). If you don't use Spring, but embed ADS? then you will have to set those index programaticaly. I'm not an expert in this part of the server, but I'm sure someone else can confirm that. Emmanuel Thanks Simon T 17 October 2006 13:53To: "Apache Directory Developers List"cc: From: "Emmanuel Lecharny" <[EMAIL PROTECTED]>Subject: Re: WARN: Attribute does not have normalizerHi !So far, you haven't done anything wrong. The warnings are just here to warn about the fact that, as we don't have declared any normalizer for sme attributes, ADS will just use a kind of Identity normalizer (ie, a normalizer that don't normalize :). It's always possible to declare some normalizer into the schema files, but I'm not sure this is necessary to normalize a photo (!?), for instance. FYI, normalizer are used to transform an attribute value in a way that allow comparizons between values. For instance, if an attribute is case insensitive, then the normalization will transform the value to a lower cased string, and every further comparizons will be done using lower cased comparizons. I think that you just get more logs just because 1.0 is just a little bit more verbose.Emmanuel On 10/17/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I just installed the latest V1.0 release on Windows. I'm running DS embedded in jBoss. I am experiencing a number of log4j warnings that I did not see in RC3. The first batch or WARNs seem to relate to the Bootstrap schema loader and the second batch relate to the creation of the system partition. What have I done wrong? BootstrapAttributeTypeRegistry) Attribute teletexTerminalIdentifier does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute photo does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute facsimileTelephoneNumber does not have normalizer : using NoopNormalizer BTreePartition) Using default cache size of 100 for index on attribute objectClassBTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.4BTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.1 TIA Simon T-- Cordialement,Emmanuel Lécharny -- Cordialement,Emmanuel Lécharny
[RESULTS] [VOTE] Moving Naming to Sandbox
Hi all, Here's the results of the vote to move naming to the sandbox: +1 votes Ole Ersoy Phil Steitz Trustin Lee Enrique Rodriguez Emmanuel Lecharny Alex Karasulu John E. Conlon Pierre-Arnaud Marcelot Stefan Zoerner Ersin Er No +/-0 or -1 votes were cast. There are sufficient binding votes. I will move naming into the sandbox later today. Thanks, Alex
Re: [VOTE] Move Naming to sandbox
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Emmanuel Lecharny wrote: [ X ] +1 Remove naming from site and move it to sandbox +1 Alex -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQGD3PPcPVto+tI7JJARAqZ5AKDaidw3Aw930aISGcQVdEEKwjZdOACggEYy calZLsl02b4DD0KIx70tM1w= =HdEA -END PGP SIGNATURE-
[jira] Created: (DIRMINA-288) Add SessionLog.info( session, "CREATED" ); to LoggingFilter.sessionCreated method
Add SessionLog.info( session, "CREATED" ); to LoggingFilter.sessionCreated method - Key: DIRMINA-288 URL: http://issues.apache.org/jira/browse/DIRMINA-288 Project: Directory MINA Issue Type: Bug Components: Core Affects Versions: 1.0 Reporter: Michael Newcomb Priority: Trivial Please add the following to the LoggingFilter.sessionCreated method: public void sessionCreated( NextFilter nextFilter, IoSession session ) { SessionLog.info( session, "CREATED" ); nextFilter.sessionCreated( session ); } Thanks, Michael -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (DIRMINA-284) Filter.sessionCreated/Filter.sessionOpened called AFTER Filter.filterWrite...
[ http://issues.apache.org/jira/browse/DIRMINA-284?page=all ] Michael Newcomb closed DIRMINA-284. --- Fix looks good. > Filter.sessionCreated/Filter.sessionOpened called AFTER Filter.filterWrite... > - > > Key: DIRMINA-284 > URL: http://issues.apache.org/jira/browse/DIRMINA-284 > Project: Directory MINA > Issue Type: Bug > Components: Core, Filter >Affects Versions: 1.0 >Reporter: Michael Newcomb > Assigned To: Trustin Lee > Fix For: 1.0.1 > > > Using the sumup example: > Move the following code out of ClientSessionHandler.sessionOpened(): > for( int i = 0; i < values.length; i++ ) > { > AddMessage m = new AddMessage(); > m.setSequence( i ); > m.setValue( values[ i ] ); > session.write( m ); > } > And put it into the Client right after: > ConnectFuture future = connector.connect( > new InetSocketAddress( HOSTNAME, PORT ), > new ClientSessionHandler( values ), cfg ); > future.join(); > session = future.getSession(); > And you will see the following in the Client console: > Oct 13, 2006 11:19:05 AM org.apache.mina.util.SessionLog info > INFO: [localhost/127.0.0.1:8080] CREATED > Oct 13, 2006 11:19:05 AM org.apache.mina.util.SessionLog info > INFO: [localhost/127.0.0.1:8080] WRITE: 0:ADD(5) > Oct 13, 2006 11:19:05 AM org.apache.mina.util.SessionLog info > INFO: [localhost/127.0.0.1:8080] OPENED > Oct 13, 2006 11:19:05 AM org.apache.mina.util.SessionLog info > INFO: [localhost/127.0.0.1:8080] WRITE: 1:ADD(45) > Oct 13, 2006 11:19:05 AM org.apache.mina.util.SessionLog info > INFO: [localhost/127.0.0.1:8080] WRITE: 2:ADD(334) > In this run, only OPENED happened after the WRITE, but in my real application > CREATED and OPENED happen *after* a WRITE on the Session. > IMHO, ConnectFuture.join(), should ensure that Session.sessionCreated and > Session.sessionOpened have been called. > Michael -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: WARN: Attribute does not have normalizer
Emmanuel Thanks for the prompt reply. 1. IMHO, a level of INFO would be better for the 'normalizer' entries - would you agree? My users are equally sensitive to both ERROR and WARN log entries. 2. I think the BTreePartition WARNs could be avoided if the code that creates the system partition explicitly sets the cache size on the index attributes. I know this as I create a custom partition and it was generating WARN messages! I see from the Spring configuration file that it sets the index attributes. Is there a way to ensure the system partition sets the index attributes when I'm not using Spring? Thanks Simon T 17 October 2006 13:53To: "Apache Directory Developers List" cc: From: "Emmanuel Lecharny" <[EMAIL PROTECTED]>Subject: Re: WARN: Attribute does not have normalizerHi !So far, you haven't done anything wrong. The warnings are just here to warn about the fact that, as we don't have declared any normalizer for sme attributes, ADS will just use a kind of Identity normalizer (ie, a normalizer that don't normalize :). It's always possible to declare some normalizer into the schema files, but I'm not sure this is necessary to normalize a photo (!?), for instance. FYI, normalizer are used to transform an attribute value in a way that allow comparizons between values. For instance, if an attribute is case insensitive, then the normalization will transform the value to a lower cased string, and every further comparizons will be done using lower cased comparizons. I think that you just get more logs just because 1.0 is just a little bit more verbose.Emmanuel On 10/17/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I just installed the latest V1.0 release on Windows. I'm running DS embedded in jBoss. I am experiencing a number of log4j warnings that I did not see in RC3. The first batch or WARNs seem to relate to the Bootstrap schema loader and the second batch relate to the creation of the system partition. What have I done wrong? BootstrapAttributeTypeRegistry) Attribute teletexTerminalIdentifier does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute photo does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute facsimileTelephoneNumber does not have normalizer : using NoopNormalizer BTreePartition) Using default cache size of 100 for index on attribute objectClassBTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.4BTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.1 TIA Simon T-- Cordialement,Emmanuel Lécharny
Re: WARN: Attribute does not have normalizer
Hi !So far, you haven't done anything wrong. The warnings are just here to warn about the fact that, as we don't have declared any normalizer for sme attributes, ADS will just use a kind of Identity normalizer (ie, a normalizer that don't normalize :). It's always possible to declare some normalizer into the schema files, but I'm not sure this is necessary to normalize a photo (!?), for instance. FYI, normalizer are used to transform an attribute value in a way that allow comparizons between values. For instance, if an attribute is case insensitive, then the normalization will transform the value to a lower cased string, and every further comparizons will be done using lower cased comparizons. I think that you just get more logs just because 1.0 is just a little bit more verbose.EmmanuelOn 10/17/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I just installed the latest V1.0 release on Windows. I'm running DS embedded in jBoss. I am experiencing a number of log4j warnings that I did not see in RC3. The first batch or WARNs seem to relate to the Bootstrap schema loader and the second batch relate to the creation of the system partition. What have I done wrong? BootstrapAttributeTypeRegistry) Attribute teletexTerminalIdentifier does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute photo does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute facsimileTelephoneNumber does not have normalizer : using NoopNormalizer BTreePartition) Using default cache size of 100 for index on attribute objectClassBTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.4BTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.1 TIA Simon T -- Cordialement,Emmanuel Lécharny
WARN: Attribute does not have normalizer
I just installed the latest V1.0 release on Windows. I'm running DS embedded in jBoss. I am experiencing a number of log4j warnings that I did not see in RC3. The first batch or WARNs seem to relate to the Bootstrap schema loader and the second batch relate to the creation of the system partition. What have I done wrong? BootstrapAttributeTypeRegistry) Attribute teletexTerminalIdentifier does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute photo does not have normalizer : using NoopNormalizerBootstrapAttributeTypeRegistry) Attribute facsimileTelephoneNumber does not have normalizer : using NoopNormalizer BTreePartition) Using default cache size of 100 for index on attribute objectClassBTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.4BTreePartition) Using default cache size of 100 for index on attribute 1.2.6.1.4.1.18060.1.1.1.3.1 TIA Simon T