[VOTE] MINA 2.0.12 release, take 2

[Emmanuel Lécharny]

Hi,

Here is another vote for a new bug fix release of Mina : 2.0.12.

This release fixed a serious issue in the IoProcessor loop, that may lead in 
some cases to a 100% CPU consumption. 
It also fixes the IoProcessor loop exit conditions. Another two bugs found by 
Radovan Semancik has been fixed.

Here is the list of fixed issues :

Bugs :
--

[DIRMINA-1001 ] mina2.0.9 
session.close cpu100%
[DIRMINA-1006 ] mina2.0.9 
NioProcessor thread make cpu 100%
[DIRMINA-1024 ] There is 
no way to start a SslHandshake when the autoStart flag is set to false
[DIRMINA-1026 ] Session 
may be removed twice from the removedSession queue



A temporary tag has been created (it can be removed if the vote is not
approved):

Updated Tags:  refs/tags/2.0.12 [created] 5498c66fb

Project: http://git-wip-us.apache.org/repos/asf/mina/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina/commit/d35959b8
Tree: http://git-wip-us.apache.org/repos/asf/mina/tree/d35959b8
Diff: http://git-wip-us.apache.org/repos/asf/mina/diff/d35959b8

- Nexus 
repository:https://repository.apache.org/content/repositories/orgapachemina-1019

- Binaries:http://people.apache.org/~elecharny

Let us vote :
[ ] +1 | Release MINA 2.0.12
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release MINA 2.0.12

Thanks !

Result, was : [VOTE] MINA 2.0.12 release, take 2

[Emmanuel Lécharny]

Hi guys,

I'm closing the vote, with 4 +1 binding votes and one non binding vote.

I'll push the packages and make an announcement asap.

Note that we will certainly have to cut a new release, because a new
serious issue has been found in the SSLHandler class, although I don't
think we should wait for the current version to be released.

Many thanks !


Le 03/02/16 10:21, Emmanuel Lécharny a écrit :
> Hi,
>
> Here is another vote for a new bug fix release of Mina : 2.0.12.
>
> This release fixed a serious issue in the IoProcessor loop, that may lead in 
> some cases to a 100% CPU consumption. 
> It also fixes the IoProcessor loop exit conditions. Another two bugs found by 
> Radovan Semancik has been fixed.
>
> Here is the list of fixed issues :
>
> Bugs :
> --
>
> [DIRMINA-1001 <https://issues.apache.org/jira/browse/DIRMINA-1001";>] 
> mina2.0.9 session.close cpu100%
> [DIRMINA-1006 <https://issues.apache.org/jira/browse/DIRMINA-1006";>] 
> mina2.0.9 NioProcessor thread make cpu 100%
> [DIRMINA-1024 <https://issues.apache.org/jira/browse/DIRMINA-1024";>] There is 
> no way to start a SslHandshake when the autoStart flag is set to false
> [DIRMINA-1026 <https://issues.apache.org/jira/browse/DIRMINA-1026";>] Session 
> may be removed twice from the removedSession queue
>
>
>
> A temporary tag has been created (it can be removed if the vote is not
> approved):
>
> Updated Tags:  refs/tags/2.0.12 [created] 5498c66fb
>
> Project: http://git-wip-us.apache.org/repos/asf/mina/repo
> Commit: http://git-wip-us.apache.org/repos/asf/mina/commit/d35959b8
> Tree: http://git-wip-us.apache.org/repos/asf/mina/tree/d35959b8
> Diff: http://git-wip-us.apache.org/repos/asf/mina/diff/d35959b8
>
> - Nexus 
> repository:https://repository.apache.org/content/repositories/orgapachemina-1019
>
> - Binaries:http://people.apache.org/~elecharny
>
> Let us vote :
> [ ] +1 | Release MINA 2.0.12
> [ ] ± | Abstain
> [ ] -1 | Do **NOT**  release MINA 2.0.12
>
> Thanks !
>

MINA jenkins Build on windows

[Emmanuel Lécharny]

Hi guys,

it's a couple of days we get some jenkins error every hour for windows.

I have askd Infra about it, and in the mean time, changed the
periodicity to get only one build per day.

If anyone has a clue about what should be done to get this fixed, this
would be very appreciated.

[VOTE] MINA 2.0.13 release

[Emmanuel Lécharny]

Hi,

MINA 2.0.13 is a release that fixes a serious bug in the SslHandler that was 
causing a deadlock in some specific case.

Here is the fixed issues :

Bugs :
--

[DIRMINA-1019 ] SslHandler 
flushScheduledEvents race condition
[DIRMINA-1027 ] SSLHandler 
writes corrupt messages under heavy load



A temporary tag has been created (it can be removed if the vote is not
approved):

Updated Tags:  refs/tags/2.0.13 [created] 882dbe52a

Project: http://git-wip-us.apache.org/repos/asf/mina/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina/commit/2cfadcf3
Tree: http://git-wip-us.apache.org/repos/asf/mina/tree/2cfadcf3
Diff: http://git-wip-us.apache.org/repos/asf/mina/diff/2cfadcf3

- Nexus 
repository:https://repository.apache.org/content/repositories/orgapachemina-1020

- Binaries:http://people.apache.org/~elecharny

Let us vote :
[ ] +1 | Release MINA 2.0.13
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release MINA 2.0.13

Thanks !

Re: svn commit: r1730433 - /mina/site/trunk/content/mina-project/developer-guide.mdtext

[Emmanuel Lécharny]

Le 15/02/16 09:45, Jeff MAURY a écrit :
> Should we really promote skipping tests which is a bad practice ?

This is not what I wrote. This is a very specific case, where we *have*
to run a clean install before running a perform, otherwise you have
failures. Tests have already been ran 2 times in the release process
(dry test and prepare). Here, I'm just skipping test to save the 3
minutes it takes.

What I would rather suggest is that we fix the problem that forces us to
run the clean install at this step...

Result, was: [VOTE] MINA 2.0.13 release

[Emmanuel Lécharny]

Hi guys,

I'm closing this vote with 4 binding +1s and one non binding +1 :

bindings :
Jeff
Jean-François
Ashish
and me

Non binding :
Jamie

I'll publish the packages and update the web site asap.

Many thanks !

Re: We still experience DIRMINA-1006 despite using 2.0.13

[Emmanuel Lécharny]

Le 26/02/16 12:07, Buddy Butterfly a écrit :
> Hi,
>
> we still experience isse DIRMINA-1006. We have an RFID device that
> rejects connections when connection limit is reached. Our mina client
> thread then goes into 100% cpu usage. Any idea if the patch maybe
> did not cover the all situations?
>
> Cheers,
> Buddy
>
>
>
>
Cr*p...

Can you reopen the issue, and attach a stacktrace (kill -3) when you
observe a 100% CPU ?

Re: We still experience DIRMINA-1006 despite using 2.0.13

[Emmanuel Lécharny]

Le 27/02/16 18:08, Buddy Butterfly a écrit :
> Obviously netty fixed this with a workaround.

Thanks for the interesting analysis and pointers.

It seems that in some case, we are still looping on the select().

Can you do one more thing to confirm that it's teh real problem ? Set
the log to WARN, you should get messages like :

Create a new selector. Selected is 0, delta = XXX

This would be the signal that we detected a stalled selector (ie a
selector that return immediately on select(), even if no event has
arrived, leading to a fast loop eating all the CPU). In thsi case, we
recreate a new selector. In your case, and accordingly to the bug
report, there might be a case where creating a new selector does not
solve the issue.

Many thanks !

Re: We still experience DIRMINA-1006 despite using 2.0.13

[Emmanuel Lécharny]

Le 28/02/16 18:00, Buddy Butterfly a écrit :
> Hi,
>
> if I understand it correct, you are telling me that there will be no
> sultion for it? 

Did I say that ???

> Or did I get it wrong.
Probably... I'm just trying to see what could be the root cause.

> This problem strikes
> us heavily! Project responsible is over the top :-(.
>
> We saw 4 lines of this type:
>
>   Line 41771: 2016-02-28 01:52:01,147 [   NioProcessor-2358] WARN
> ing.AbstractPollingIoProcessor$Processor  run - Create a
> new selector. Selected is 0, delta = 0
>
>   Line 71754: 2016-02-28 04:27:14,918 [   NioProcessor-3238] WARN
> ing.AbstractPollingIoProcessor$Processor  run - Create a
> new selector. Selected is 0, delta = 0
>
>   Line 74744: 2016-02-28 06:52:45,293 [   NioProcessor-4054] WARN
> ing.AbstractPollingIoProcessor$Processor  run - Create a
> new selector. Selected is 0, delta = 0
>
>   Line 87830: 2016-02-28 17:24:35,806 [   NioProcessor-7346] WARN
> ing.AbstractPollingIoProcessor$Processor  run - Create a
> new selector. Selected is 0, delta = 0

So clearly something else is wrong. If we were looping because the
select() exits immediately with no socket being active, we would have
way more of such messages. Here, it happens only from time to time.

Now, reading teh various links you have provided, I wonder if the
problem is not in the JDK code itself...

I'll investigate more tonite.

Would you be ready to test a nightly build if I provide one ?

Thanks !

Re: CumulativeProtocolDecoder and UDP

[Emmanuel Lécharny]

Le 01/03/16 15:05, tunca a écrit :
> Hi Trustin,
> We're using Mina Core 2.0.9.

Have you checked with 2.0.13 ?

> The Udp Cumulative Protocol doesn't seem to accumulate.
> The Decoder is working with TCP however, with UDP the 
> The size of the IoBuffer doesn't increase.

Using such a protocol decoder does not make any sense in UDP : either
you receive the full message, or you receive nothing. Moreover, there is
no guarantee that a message N will arrive *before* a message N+1 in UDP.

What exactly are you trying to do ?

Re: CumulativeProtocolDecoder and UDP

[Emmanuel Lécharny]

Le 01/03/16 16:58, tunca a écrit :
> Our customer has strange requirement about merging multiple udp/TCP packages
> to create a single  message. 
> There is a well defined protocol that defines message boundaries. 
> The decoder is working good with TCP packages.  It can create a single
> message from multiple TCP packages. 
> However when a message is fragmented into multiple packages the do doDecode
> method always gives the same ioBuffer. 

Define fragmented in a UDP context : how do you manage the packet ordering ?
> I'll try 2.0.13 next day. 

It won't solve your issue. You most certainly handling the incoming
packets incorrectly somewhere in your IoHandler. Typically, you are not
in the same session when 2 UDP packets are coming, thus the IoBuffer,
which is stored into the session, is different for each incoming packet.

Re: CumulativeProtocolDecoder and UDP

[Emmanuel Lécharny]

Le 02/03/16 07:38, tunca a écrit :
> Hi Emmanuel.
> http://pastebin.com/embed_iframe/EpL4WE4b
> This is the link to my doDecode method.
>
> A complete message starts with x01 and ends with x03.
> For test I send 2 udp messages.
> First Udp message starts with x01 
> Second Udp messages ends with x03.
>
> When I send first message doDecode message is called with the content of the
> first Udp message.
> the doDecode method returns false as the message is not complete.
>
> Then the second Udp message is sent.
> However doDecode method is still called with only First Udp message's
> content from in IoBuffer parameter.

Ok, I think I see what's going on. The
CumulativeProtocolDecoder.decode() method starts with :

public void decode(IoSession session, IoBuffer in,
ProtocolDecoderOutput out) throws Exception {
if (!session.getTransportMetadata().hasFragmentation()) {
while (in.hasRemaining()) {
if (!doDecode(session, in, out)) {
break;
}
}

return;
}

and for a UDP session, we initialize the transport metadata this way :

static final TransportMetadata METADATA = new
DefaultTransportMetadata("nio", "datagram", true, false,
InetSocketAddress.class, DatagramSessionConfig.class,
IoBuffer.class);


where the forth parameter is 'boolean fragmentation' :

public DefaultTransportMetadata(String providerName, String name,
boolean connectionless, boolean fragmentation,
Class addressType, Class sessionConfigType,
Class... envelopeTypes) {

It's set to 'false', so the decode method simply don't accumulate the
data in one single buffer.

There is no way to change the flag once the metadata instance has been
created.

What you can do is to define your own version of the
CumulativeProtocolDecoder, copying the original code and simply discard
the test on the transportMetadata.

   

Re: Student looking to contribute

[Emmanuel Lécharny]

Le 03/03/16 06:00, Alberto Torres a écrit :
> Hi,
>
> My name is Alberto Torres and I am a 4th year computer science student. I am 
> looking to contribute to an open source project as part of my school 
> semester. I currently work in and have 1 year experience in IT support for a 
> Charter School Organization. I don’t have much programming experience but I 
> am hoping to be able to contribute in other ways a few hours a week for the 
> next month. I look forward to getting involved! 

Hi Alberto !

you can obviously contribute to MINA even if you are not a experienced
developper ! One good startng point would be the documentation which is
lacking seriously. Typically, we have a lot of missing Javadoc in many
of MINA classes and that would be great to get that corrected.

Do not hesitate to ask for directions.

Thanks !

Re: CumulativeProtocolDecoder and UDP

[Emmanuel Lécharny]

Le 03/03/16 08:29, tunca a écrit :
> Hi Emmanuel,
>
> Thanks a lot, you find out the root cause of the problem I believe.
>
> I created my own version of CumulativeProtocolDecoder
> (http://pastebin.com/m0qmbEVS) and as you suggested removed the has
> fragmentation check.
> It still doesn't accumulate, however the second package arrives without
> accumulating for the same session.

Remove completely the initial check, includinhg the call to doDecode(),
otherwise it will be called twice, consuming the buffer.

Re: CumulativeProtocolDecoder and UDP

[Emmanuel Lécharny]

Le 03/03/16 09:09, tunca a écrit :
> Hi Emmanuel,
>
> I downloaded apache mina 2.0.13 and updated  NioDatagramSession METADATA
> creation code.
> Set the 4th parameter to true, then for UDP CumulativeProtocolDecoder
> started to work as expected.

Indeed.

Although doing so you create a dependency on a patched version of MINA.
That means if someone in the future switch to a newer version of MINA,
your code will simply break.

Here is what I suggest : fill a JIRA asking for exposing the MetaData
parameter, allowing teh developper (you, here) to set the flag to
something else than the defautl value.

Re: Student looking to contribute

[Emmanuel Lécharny]

Le 03/03/16 15:16, Alberto Torres a écrit :
> Great! I would love to help out with documentation. Can you provide me with 
> more information on where to start ? 

Sure : we have all the documentation stored on
http://svn.apache.org/viewvc/mina/site/trunk/

You can checkout the content doing :

$ svn co http://svn.apache.org/viewvc/mina/site/trunk/ mina-site

Now, all the content is in content/mina-project/userguide, and we are
using markdown format for the documentation
(https://daringfireball.net/projects/markdown/).

Regarding the Javadoc, the simpler is to checkout the code doing :

$ git clone http://git-wip-us.apache.org/repos/asf/mina.git mina then $ git 
checkout -b 2.0

This is the current developpement branch for MINA 2.0 You can load the
project in Eclipse or Intellij and browse the code, you'll quickly find
some classes with no Javadoc at all...

Re: Mina build JDK

[Emmanuel Lécharny]

Le 08/03/16 23:08, Mondain a écrit :
> I noticed that the last couple builds were done with JDK8; is this the
> trend from now on for Mina?
>
> Paul
>
No. We will continue support Java 7.

Re: Mina build JDK

[Emmanuel Lécharny]

Le 09/03/16 17:32, Mondain a écrit :
> Will the jdk7 build artifacts have a different name? 

No, the idea would be to build with JDK7 as a target. I don't know why I
haven't done that, it's clearly a mistake.

Re: Mina build JDK

[Emmanuel Lécharny]

Le 09/03/16 19:53, Mondain a écrit :
> The "catch" here is that you can't build with JDK8 for JDK7; if you do
> this, users on JDK7 will see the concurrent keyset error.

Damn... The problem is that Java 7 is not available anymore for download
since April 2015...

Re: Mina build JDK

[Emmanuel Lécharny]

Le 09/03/16 20:20, Emmanuel Lécharny a écrit :
> Le 09/03/16 19:53, Mondain a écrit :
>> The "catch" here is that you can't build with JDK8 for JDK7; if you do
>> this, users on JDK7 will see the concurrent keyset error.
> Damn... The problem is that Java 7 is not available anymore for download
> since April 2015...
>
Ah, hopefully, I have a Java 7 installed :

localhost:mina-2.0 elecharny$ mvn -version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
2015-11-10T17:41:47+01:00)
Maven home: /opt/apache-maven-3.3.9
Java version: 1.7.0_55, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.7.0_55.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: US-ASCII
OS name: "mac os x", version: "10.9.5", arch: "x86_64", family: "mac"

Re: Vysper update

[Emmanuel Lécharny]

Le 19/03/16 09:45, Praveen Prabhu a écrit :
> Hi,
> I have recently downloaded the source from the vysper svn trunk and have made 
> it compatible with jdk 1.7.
> Also, a number of changes made:
> 1. Upgraded the deprecated DocumentSpecCompliantAnnotationFactory to 
> DocumentSpecCompliantAnnotationProcessor to support latest annotations 
> processor framework2. Migrated to Mina 2.0.133. Upgraded slfj to 1.7.18 
> (supports varargs for logging)4. Upgraded ehcache to 2.10.1 (can support 
> locking features for better concurancy)5. Upgraded websocket end point to use 
> sslcontextfactory instead of depricated api.6. Code cleanup to remove warning 
> messages
> Compilation and packaging is successfull, though I have not run any other 
> tests other than the unit tests during compilation.
> I am planning to work on clustering multiple vysper instances such that users 
> can logon to multiple hosts within a cluster and outbound and inbound message 
> processing can be done on different servers based on where the user is logged 
> in. This should enable horizontal scalability to vysper.
> I would very much like to contribute this to the vypser community. I can send 
> the source for review so that you can review and commit if you deem fit.
That would be great !!

The best would be to create one (or more) JIRA and attach the source to
it, or to create a pull request on github
(https://github.com/apache/vysper) so that we can review teh changes.

FTR, Vysper has been kind of dorman for that few years, but if you feel
like taking over the project, we could most certainly vote you in as a
committer, whoch would be frnkly simpler.

Thanks  a lot !

Re: [VOTE] Release Apache Mina SSHD 1.2.0

[Emmanuel Lécharny]

Le 31/03/16 09:15, Guillaume Nodet a écrit :
> We're still missing a few binding votes 

Damn.. Totally missed the last votes :/


I have tested the package, and teh first time I got an error :

Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.098
sec <<< FAILURE! - in
org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest
testCopyDataExtension[size=8,192, readOffset=4,096, readLength=2,048,
writeOffset=4,096](org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest)
 
Time elapsed: 5.014 sec  <<< ERROR!
org.apache.sshd.common.SshException: Failed to get operation result
within specified timeout: 5000
at
org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest.testCopyDataExtension(CopyDataExtensionImplTest.java:174)
at
org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest.testCopyDataExtension(CopyDataExtensionImplTest.java:135)
at
org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest.testCopyDataExtension(CopyDataExtensionImplTest.java:121)



Re-running it twice, both were successful.

Otherwise, I think there is a missing Copyright for jcraft, which is
part of the dependencies :

"JZlib 0.0.* were released under the GNU LGPL license.  Later, we have
switched
over to a BSD-style license.

--
Copyright (c) 2000,2001,2002,2003 ymnk, JCraft,Inc. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice,
 this list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright
 notice, this list of conditions and the following disclaimer in
 the documentation and/or other materials provided with the
distribution.
"


I know this should have been detected before, and that I (and many
others) haven't check that for years... Am I wrong ?

Re: [VOTE] Release Apache Mina SSHD 1.2.0

[Emmanuel Lécharny]

Le 31/03/16 23:15, Guillaume Nodet a écrit :
> 2016-03-31 15:13 GMT+02:00 Emmanuel Lécharny :
>
>> Le 31/03/16 09:15, Guillaume Nodet a écrit :
>>> We're still missing a few binding votes 
>> Damn.. Totally missed the last votes :/
>>
>>
>> I have tested the package, and teh first time I got an error :
>>
>> Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.098
>> sec <<< FAILURE! - in
>>
>> org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest
>> testCopyDataExtension[size=8,192, readOffset=4,096, readLength=2,048,
>>
>> writeOffset=4,096](org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest)
>> Time elapsed: 5.014 sec  <<< ERROR!
>> org.apache.sshd.common.SshException: Failed to get operation result
>> within specified timeout: 5000
>> at
>>
>> org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest.testCopyDataExtension(CopyDataExtensionImplTest.java:174)
>> at
>>
>> org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest.testCopyDataExtension(CopyDataExtensionImplTest.java:135)
>> at
>>
>> org.apache.sshd.client.subsystem.sftp.extensions.helpers.CopyDataExtensionImplTest.testCopyDataExtension(CopyDataExtensionImplTest.java:121)
>>
>>
>>
>> Re-running it twice, both were successful.
>>
> Not sure if this is an environmental problem or not.  Lyor, any idea ?
>
>
>> Otherwise, I think there is a missing Copyright for jcraft, which is
>> part of the dependencies :
>>
>> "JZlib 0.0.* were released under the GNU LGPL license.  Later, we have
>> switched
>> over to a BSD-style license.
>>
>>
>> --
>> Copyright (c) 2000,2001,2002,2003 ymnk, JCraft,Inc. All rights reserved.
>>
>> Redistribution and use in source and binary forms, with or without
>> modification, are permitted provided that the following conditions are met:
>>
>>   1. Redistributions of source code must retain the above copyright notice,
>>  this list of conditions and the following disclaimer.
>>
>>   2. Redistributions in binary form must reproduce the above copyright
>>  notice, this list of conditions and the following disclaimer in
>>  the documentation and/or other materials provided with the
>> distribution.
>> "
>>
>>
>> I know this should have been detected before, and that I (and many
>> others) haven't check that for years... Am I wrong ?
>>
>>
> Those dependencies are not distributed in any distribution, so I don't
> really see why their copyright / notice should be included.

okie.

My +1

Re: Recent releases are not included on the web site

[Emmanuel Lécharny]

Le 11/05/16 16:46, Eugene Petrenko a écrit :
> Hi,
>
> According to Maven we have 1.1.1 and 1.2.0 released. But on the web site
> https://mina.apache.org/sshd-project/downloads.html
> it shows the latest release is 1.1.0.

I just re-published the site. It should now reference the 2 missing
versions.

Re: can't visit the webpage of http://mirrors.hust.edu.cn/apache/mina/ftpserver/1.0.6/ftpserver-1.0.6.zip

[Emmanuel Lécharny]

Le 13/05/16 05:27, huangqihua (A) a écrit :
> Hi,there.I want to download the apache ftpserver from the url: 
> http://mirrors.hust.edu.cn/apache/mina/ftpserver/1.0.6/ftpserver-1.0.6.zip
>
> but it only returns 404,the othere sites that are suggested returns the same 
> error code.I think it's a bad experience of web browsing.Please fix the 
> problems,thanks.
>
Many thanks !

I have fixed the broken links. Please give it a try !

Re: svn commit: r1756636 - /mina/site/trunk/content/vysper-project/download_0.7.mdtext

[Emmanuel Lécharny]

Le 17/08/16 à 17:04, sebb a écrit :
> On 17 August 2016 at 15:00,   wrote:
>> Author: elecharny
>> Date: Wed Aug 17 14:00:48 2016
>> New Revision: 1756636
>>
>> URL: http://svn.apache.org/viewvc?rev=1756636&view=rev
>> Log:
>> Fixed a broken link"
>>
>> Modified:
>> mina/site/trunk/content/vysper-project/download_0.7.mdtext
>>
>> Modified: mina/site/trunk/content/vysper-project/download_0.7.mdtext
>> URL: 
>> http://svn.apache.org/viewvc/mina/site/trunk/content/vysper-project/download_0.7.mdtext?rev=1756636&r1=1756635&r2=1756636&view=diff
>> ==
>> --- mina/site/trunk/content/vysper-project/download_0.7.mdtext (original)
>> +++ mina/site/trunk/content/vysper-project/download_0.7.mdtext Wed Aug 17 
>> 14:00:48 2016
>> @@ -29,8 +29,8 @@ For a detailed view of new features and
>>
>>  | Description | Download Link | PGP Signature file of download |
>>  |---|---|---|
>> -| Windows Distribution | 
>> [vysper-0.7-bin.zip](http://www.apache.org/dyn/closer.lua/mina/vysper/0.7/vysper-0.7-bin.zip)
>>  | 
>> [vysper-0.7-bin.zip.asc](http://www.apache.org/dist/mina/vysper/0.7/vysper-0.7-bin.zip.asc)
>>  |
>> -| Unix/Linux/Cygwin Distribution | 
>> [vysper-0.7-bin.tar.gz](http://www.apache.org/dist/mina/vysper/0.7/vysper-0.7-bin.tar.gz)
>>  | 
>> [vysper-0.7-bin.tar.gz.asc](http://www.apache.org/dist/mina/vysper/0.7/vysper-0.7-bin.tar.gz.asc)
>>  |
>> +| Windows Distribution | 
>> [vysper-0.7-bin.zip](http://www.apache.org/dyn/closer.lua/mina/vysper/0.7/dist/vysper-0.7-bin.zip)
>>  | 
>> [vysper-0.7-bin.zip.asc](http://www.apache.org/dist/mina/vysper/0.7/dist/vysper-0.7-bin.zip.asc)
>>  |
>> +| Unix/Linux/Cygwin Distribution | 
>> [vysper-0.7-bin.tar.gz](http://www.apache.org/dist/mina/vysper/0.7/dist/vysper-0.7-bin.tar.gz)
>>  | 
>> [vysper-0.7-bin.tar.gz.asc](http://www.apache.org/dist/mina/vysper/0.7/dist/vysper-0.7-bin.tar.gz.asc)
>>  |
> -1, this must use the mirror system, not the ASF mirror source.

Thanks, fixed !

On-going work in MINA 2.0 branch

[Emmanuel Lécharny]

Hi guys,


I'm currrently reviewing the pending JIRA's on MINA 2.0 branch (and
there are many !).

I was able to fix the long lasting issue with session remaining in
pending state when being closed while some messages were nt written to
the remote peer. That should fix an issue in SSHd.

Now, I'm trying to fix DIRMINA-1041 (writeFuture is not always updated
when a session is being closed on a remote peer, leaving the connection
pending). This is a different flavor of the previous problem.


At the same time, I discovered that under stress, the NioProcessor
thread was creating a *lot* of new Selector (and on a loop of 1000
sessions being created, 9% of them ended with the creation of a new
selector). This is really problematic, and might very well lead to a
performance degradation. For the record, we create a new selector when
we have some good reason to think that the current seletor is not any
more working well : typically, when the select(TIMEOUT) call returns
immediately with 0 selected keys, and no wakeup/interrupt has occured
(this is a well lnow NIO bug, which has not been fixed for more than a
decade, may thinks to the SUN then Oracle JDK developpers...). When we
detect this bad behaviour, we assume the selector is broken, we create a
new one and register all the keys on this selector. If we have plenty of
keys, this is an expensive operation.

In order to limit the number of selector being created, I give the
current selector some 'room' : we try more than once to do a select(),
and we bail out after ten unsuccessful select() - ie all of the select()
calls return a wrong 0 result -. Doing that, no new selector is being
created... I have no idea why calling twice the select() method works,
but in any case, it should nbot hurt us.


Next will be the integration fo DIRMINA-1039 patch.


If all goes well, I expect to cut a release soon.


Thanks !

[VOTE] MINA 2.0.14 release

[Emmanuel Lécharny]

Hi,

Here is a new release, MINA 2.0.14. It fixes many issues, some of them being a 
real burden for SSHD DIRMINA-1021). Some patches were also applied (thanks to 
Maria Petridan).

Here are the fixed issues :

Bugs :
--

[DIRMINA-760 ]   Client 
fails to detect disconnection
[DIRMINA-976 ]   
ScheduledWriteBytes Increases after Exception on Writing
[DIRMINA-1021 ] MINA-CORE 
does not remove sessions if exceptions occur while closing
[DIRMINA-1025 ] A call to 
session.closed(true) may still flush messages.
[DIRMINA-1028 ] The 
supported ciphers configuration might not be used
[DIRMINA-1029 ] The sent 
buffer is reset to its original position when using the SSL Filter after a 
session.write()
[DIRMINA-1037 ] Throw 
exception in NioProcessor.write if the session is closing
[DIRMINA-1039 ] Response 
messages queue up on the server side waiting to be written to socket, while the 
server continues to read more request messages, causing out of heap memory
[DIRMINA-1042 ] Epoll 
spinning causes memory leak


Task :
--

DIRMINA-986 ] Update the web 
site to reflect the switch to git for the release process



A temporary tag has been created (it can be removed if the vote is not 
approved):

Updated Tags:  refs/tags/2.0.14 [created] 34bed70ca

Project: http://git-wip-us.apache.org/repos/asf/mina/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina/commit/9ed90e0d
Tree: http://git-wip-us.apache.org/repos/asf/mina/tree/9ed90e0d
Diff: http://git-wip-us.apache.org/repos/asf/mina/diff/9ed90e0d

- Nexus repository: 
https://repository.apache.org/content/repositories/orgapachemina-1023

- Binaries and sources: https://dist.apache.org/repos/dist/dev/mina/mina/2.0.14/

Let us vote :
[ ] +1 | Release MINA 2.0.14
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release MINA 2.0.14

Thanks !

Re: [VOTE] MINA 2.0.14 release

[Emmanuel Lécharny]

Le 29/08/16 à 18:24, Mondain a écrit :
> Not that I count, but +1 from me :)

All votes count !

Result, was : [VOTE] MINA 2.0.14 release

[Emmanuel Lécharny]

Hi guys,

I'm closing the vote for the MINA 2.0.14 release. We have had 6 +1 (4
binding votes):

Ashish,
Guillaume,
Jeff,
Maria (non binding),
Mondain (non binding),
and me.

Many thanks to the voters !

I'll update the web site, push the packages and announce the release in
the coming 24h.

Re: [VOTE] Release SSHD 1.3.0

[Emmanuel Lécharny]

Le 19/09/16 à 09:27, Guillaume Nodet a écrit :
> I've prepared a candidate for SSHD 1.3.0 release at
>   https://repository.apache.org/content/repositories/orgapachemina-1024

Both tag and packages built, signature tested, N&L files checked : my +1

[VOTE] MINA 2.0.15 release

[Emmanuel Lécharny]

Hi,

Here is a new release, MINA 2.0.15. It fixes one critical SSL issue, and a NPE. 
Javadoc has also been a bit cleaned up.

Here are the fixed issues :

Bugs :
--

[DIRMINA-1044 ] Non-Secure 
(no TLS/SSL) based client could successfully send message to secure Mina 
endpoint after second attempt
[DIRMINA-1à43 ] 
NullPointerException after upgrade to mina 2.0.14



A temporary tag has been created (it can be removed if the vote is not 
approved):

Updated Tags:  refs/tags/2.0.15 [created] 0399f97db

Project: http://git-wip-us.apache.org/repos/asf/mina/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina/commit/8bae1d4f
Tree: http://git-wip-us.apache.org/repos/asf/mina/tree/8bae1d4f
Diff: http://git-wip-us.apache.org/repos/asf/mina/diff/8bae1d4f

- Nexus repository: 
https://repository.apache.org/content/repositories/orgapachemina-1025

- Binaries and sources: https://dist.apache.org/repos/dist/dev/mina/mina/2.0.15/

Let us vote :
[ ] +1 | Release MINA 2.0.15
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release MINA 2.0.15

Thanks !

MINA userguide site updated

[Emmanuel Lécharny]

Hi guys,


I have slightly improved the Session pages in the MINA userGuide
(http://mina.staging.apache.org/mina-project/userguide/ch4-session/ch4-session.html)

Typically, I have added two sub-chapters, and added a lot of information.


Feel free to review those pages, and comment them, they are not yet on
the production site.


Many thanks !

Re: observing hangs after update to MINA 2.0.14

[Emmanuel Lécharny]

Le 21/09/16 à 13:17, Christoph John a écrit :
> Hi,
>
> do you need any more information from my side? Should I create a JIRA
> issue? The problem can be reproduced with QuickFIX/J code but I could
> check if can condense a MINA-only test case if required.

A test case would be very valuable, and, yes, a JIRA will definitvely help.

My take on this issue is that if both the client and the server close
the session more or less at the same time, one side will hang, due to
some race condition.

Re: observing hangs after update to MINA 2.0.14

[Emmanuel Lécharny]

Le 26/09/16 à 09:01, Christoph John a écrit :
> Until now I could not reproduce the problem reliably. However, I think
> it might not be a "hang" but have to do with the changed behaviour
> regarding flushing that has been fixed in MINA 2.0.14 (DIRMINA-1025/1021)
> E.g. the tests wait for a message to arrive that the other side has
> sent but it doesn't seem to be received.
>
> I guess I'll have to review the closeNow/closeOnFlush calls in
> QuickFIX/J then? Maybe it is only related to the test suite, though.

I think you sent a test case that reproduce the issue (AFAIR). I ran
this test and got a hang, if I try to close both ends, that is what I'd
like to investigate.

Sadly, I have been pretty busy lately, and I had to cut a new release
2.0.15 urgently, due to a big bug in SSL, so I didn't had chance to go
any further. I hope to have time to do that this week.

Result, was: [VOTE] MINA 2.0.15 release

[Emmanuel Lécharny]

Hi,

I'm closing this vote which got 4 binding +1 :
- Ashish
- Jeff (Genender)
- Jeff (Maury)
and me.

I will push the packages and update the web site.


Many thans !

Re: Apache Mina sharing Session in UDP

[Emmanuel Lécharny]

Le 28/09/16 à 15:13, rafrl a écrit :
> Dears,
>
> I have a problem using apache mina UDP server. I receive two connections are
> two sources with different ip but leaving the same port. The mina it is not
> creat a new sessions for each connection and is differentiating sessions and
> mixing the defined attributes. Someone passed a similar problem ?
>
> Thank's

Please don't cross post. This mailing list is for discussion related to
the MINA project development, not for MINA users problems.

Re: ConcurrentLinkedQueue vs MpscChunkedArrayQueue

[Emmanuel Lécharny]

Le 17/10/16 à 11:12, Guido Medina a écrit :
> Hi Emmanuel,
>
> At the mina-core class AbstractNioSession the variable:
>
> /** the queue of pending writes for the session, to be dequeued by the
> {@link SelectorLoop} */
> private final Queue writeQueue = new DefaultWriteQueue();
>
> such queue is being consumed by a selector loop? 
It's consumed by the IoProcessor selector loop (so you have many of them).

The thread that has been selected to process the read will go on up to
the point it has written the bytes into teh write queue, then it go back
to teh point it was called from, and the  it processes the writes :

select :
1) read -> head -> filter -> filter -> ... -> handler ->
session.write(something) -> filter -> filter -> ... -> head -> put in
write queue and return (unstacking all teh calls)
2) write -> get the write queue, and write the data in it until the
queue is empty or the socket is full (and in thsi case, set the OP_WRITE
status)   



> which makes me think it is
> a single thread loop hence making it MpSc and ideal for low GC optimization.
> But maybe such optimization is so unnoticeable that is not worth.
Actually, we have as many thread as we have IoProcessor instances. The
thing is that I'm not even sure that we need a synchronize queue, as the
IoProcessor is execution all the processing on ts own thread. The
cincurrentQueue is there just because one can use an executor filter,
that will spread the processing into many other threads, and then we
*may* have more than one thread accessing this queue.

Regarding GC, we have removed some useless object allocations in 2.0.15,
so the GC should be slightly less under pressure.

If you want to alleviate the GC load, I think there are other areas
where some improvement can be made. Typically, there is a
BufferAllocator that pool the CachedBufferAllocator that allows you to
reuse buffers. Now, this is a tricky solution, as it has to be
synchronized, so expect some bottleneck here. Ideally, we should have
another implementation that use the ThreadLocalStorage, but that would
be memory expensive.


>
> That's the only place I think it is worth replacing by a low GC footprint
> queue, it will avoid the creation of GC-able linked nodes
> (ConcurrentLinkedQueue)
>
> In fact, further in that logic you try to avoid writing to the queue if is
> empty by passing the message directly to the next handler which is a
> micro-optimization,
> isEmpty() will 99.99% of the cases render to be false for systems with high
> load.
Well, it depends. The queue will be emptied if the socket is able to
swallow the data. If your system is under high load, I suspect that all
the socket will be full already, so you'll end up with some bigger
problem ! (typically, the queue will grow, and at some pont, you'll get
a OOM... I have alredy experienced that, so yes, it may happen).

My point is that you shoud always expect that your OS and your network
are capable of allocating enouh buffer for teh sockects, and have enough
bandwith to send them fast enough so that the socket buffer can always
accept any write done on it. In this case, the write queue will always
be empty, except if you are writing a huge message (typically above the
socket buffer size, whoch default to 1Kb - from the top of my head- but
that you can set up to 64Kb more or less). Even on a loaded system.

Re: ConcurrentLinkedQueue vs MpscChunkedArrayQueue

[Emmanuel Lécharny]

Le 17/10/16 à 11:35, Guido Medina a écrit :
> And DefaultIoSessionDataStructureFactory has the following public static
> inner class:
>
> private static class DefaultWriteRequestQueue implements
> WriteRequestQueue {
> /** A queue to store incoming write requests */
> private final Queue q = new
> ConcurrentLinkedQueue();
>
> I'm assuming that's also a queue consumed by a loop thread, anything in
> fact that is consumable by a loop thread and has a chance of receiving many
> messages is optimizable by a low GC MpSc version of JC tools APIs.
> You can look at other queues they have available but TBH I prefer the array
> based queues as they have a practically zero GC impact:
> https://github.com/JCTools/JCTools/tree/master/jctools-core/src/main/java/org/jctools/queues

Maybe.

I would suggest that you do your experiment, and measure the imrpovement
you get with the JCtools queue. If it's any better, we would be pleased
to swap what we have with something better :-)

It's pretty hard to tell what would be the hypothetical gain *in
theory*, this has to be tested in silicio.

Now, MINA is an OSS project, so we warmly welcome any contribution !
Keep in mind this is also *your* project, if you want to participate !

Re: JDK7 for MINA 2.0.14/15?

[Emmanuel Lécharny]

Le 18/10/16 à 08:56, Christoph John a écrit :
> Hi,
>
> just a quick question. I have noticed that MINA 2.0.15 is compiled
> with JDK7. Was that intentional? 


This is enforced in the maven configuration  :


  org.apache.maven.plugins
  maven-compiler-plugin
  ${version.compiler.plugin}
  
1.7
1.7
true
true
ISO-8859-1
  


Java 7 is actually EOLed, so we may change that to Java 8 soon, as it's
the currently supported Java version.

For those using a older version of Java, obviously, they will have to
build MINA with a change maven configuration. For those using Java 8, I
don't think it makes any difference (I have Java 8 installe don my machine).

We haven't yet tested MINA with Java 9, as it's not yet released
(http://www.java9countdown.xyz/).

Re: JDK7 for MINA 2.0.14/15?

[Emmanuel Lécharny]

Le 18/10/16 à 10:35, Guido Medina a écrit :
> But still I wouldn't change the major version from Java 7 to Java 8 in
> 2.0.x, some legacy projects don't have the luxury to move to Java 8,
> in my case all my projects are running in Java 8 so in my case it won't
> make any difference.

I don't think we intent to move to Java 8 only any time soon for MINA 2.

In any case, we don't use any Java 8 construct in MINA, and the reason
we have kept Java 7 enforcment in Maven is for users taht are still
using it to not be forced to move to Java 8.

Hopefully, we didn't had any complaint about the switch from Java 6 to
Java 7 :-)

Re: JDK7 for MINA 2.0.14/15?

[Emmanuel Lécharny]

Le 18/10/16 à 08:56, Christoph John a écrit :
> Hi,
>
> just a quick question. I have noticed that MINA 2.0.15 is compiled
> with JDK7. Was that intentional? It is not a big problem, but I would
> not have expected that in a patch release and also have found no JIRA
> issue for this, so I could not check it earlier.
> But I'll now go the same route and increase the JDK level in a patch
> version for QuickFIX/J now. ;)

Just curious : you are still supporting Java 6 ?

[VOTE] MINA 2.0.16 release

[Emmanuel Lécharny]

Hi,

Here is a new release, MINA 2.0.16. It fixes a NPE, an some race condition that 
would make application block forever.

Here are the fixed issues :

Bugs :
--

[DIRMINA-1041 ] 
WriteFuture.await() hangs when the connection is closed remotely before await 
is invoked
[DIRMINA-1047 ] 
NullPointerException in AbstractIoSession.destroy()
[DIRMINA-1049 ] Error in 
mina-statemachine manifest prevents using it in Apache Karaf



A temporary tag has been created (it can be removed if the vote is not 
approved):

Updated Tags:  refs/tags/2.0.16 [created] b3f8caaa3

Project: http://git-wip-us.apache.org/repos/asf/mina/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina/commit/9be38df3
Tree: http://git-wip-us.apache.org/repos/asf/mina/tree/9be38df3
Diff: http://git-wip-us.apache.org/repos/asf/mina/diff/9be38df3

- Nexus repository: 
https://repository.apache.org/content/repositories/orgapachemina-1026

- Binaries and sources: https://dist.apache.org/repos/dist/dev/mina/mina/2.0.16/

Let us vote :
[ ] +1 | Release MINA 2.0.16
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release MINA 2.0.16

Thanks !


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

[VOTE] Apache FTPServer 1.1.0 release

[Emmanuel Lécharny]

Hi,

this is a new release of Apache MINA FTPServer, 1.1.0. It's a bug fix release, 
that makes FTPServer
using MINA 2.0.16. 

Java 7 is required to run this version.


Here are the fixed issues :

Bugs :
--

[FTPSERVER-475 ] FTPServer 
should use latest MINA-Core (2.0.15) and prevent abstract method error caused 
by missing FtpHandlerAdapter.inputClosed()
[FTPSERVER-461 ] FtpServer 
implement inputClosed() or extend IoHandlerAdapter for MINA 2.0.8 or higher



A temporary tag has been created (it can be removed if the vote is not 
approved):

Updated Branches:
  refs/heads/master 50e47f44f -> 8f39414e2

Project: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/commit/8f39414e
Tree: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/tree/8f39414e
Diff: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/diff/8f39414e

- Nexus repository: 
https://repository.apache.org/content/repositories/orgapachemina-1027

- Binaries and sources: 
https://dist.apache.org/repos/dist/dev/mina/ftpserver/1.1.0/

Let us vote :
[ ] +1 | Release FTPServer 1.1.0
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release FTPServer 1.1.0

Thanks !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: CMS diff: Why MINA

[Emmanuel Lécharny]

Whoah!!! Many thanks for all those fixes !


I have applied them right away :
http://svn.apache.org/viewvc?rev=1766550&view=rev

The result should be visible on
http://mina.staging.apache.org/mina-project/userguide/ch1-getting-started/why-mina.html

I will push the site later this week, as we have a coming release that
wil require some updates.


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [jira] [Commented] (FTPSERVER-475) FTPServer should use latest MINA-Core (2.0.15) and prevent abstract method error caused by missing FtpHandlerAdapter.inputClosed()

[Emmanuel Lécharny]

Le 25/10/16 à 16:43, Guido Medina a écrit :
> I have seen MINA steadily releasing versions, not as fast as I wish but
> steady enough to classify as reliable.
> But, there is a but, I don't like submit patch process, I have contributed
> to several OS projects, lots of code using GitHub.
>
> I just can't seem to get along with patch like contributions, I wonder if
> you have plans of moving it a more fast pace style like GitHub.
> I would like very to contribute
>
> *-with my time constraints just like anyone else-*

You mean, something like https://github.com/apache/mina ? ;-)

 

Re: [jira] [Commented] (FTPSERVER-475) FTPServer should use latest MINA-Core (2.0.15) and prevent abstract method error caused by missing FtpHandlerAdapter.inputClosed()

[Emmanuel Lécharny]

Le 25/10/16 à 22:14, Guido Medina a écrit :
> Forget I mentioned it :D, brain fart, things like Google didn't exist in my
> head when I was thinking about this ;-)

Actually, it's not mentionned on the web site, so there is no reason for
you to expect this repository to exist on github !

May be we should update the web site to provide this information...

Re: [VOTE] Apache FTPServer 1.1.0 release

[Emmanuel Lécharny]

Le 26/10/16 à 10:57, Dave Roberts a écrit :

> FWIW, here's my vote:-
>
> -1 | Do **NOT**  release FTPServer 1.1.0
>
> There are pending pull requests that have been around for over 3
> years which I'd really like to see make the cut...
>
> For FTPSERVER-344:
> https://github.com/saaconsltd/mina-ftpserver/pull/1
>
> For FTPSERVER-446:
> https://github.com/saaconsltd/mina-ftpserver/pull/2
>
> For FTPSERVER-438:
> https://github.com/saaconsltd/mina-ftpserver/pull/3
>
> I know Niklas hasn't really been around lately so no-one is actively
> working on this project, but if a release is going to be made, it
> might as well take some of the fixes on offer. Obviously I'm a
> little biased in these cases. :)
>
> Kind regards
> Dave

Hi Dave,

IMHO, the urgency is to have a release cut. As it was 2 years it was
pending, it was lagging behing all the MINA fixes. Getting this release
out was already quite painful, with many things having to be fixed in
teh code and teh build system to have it working.

I would consider that this version (1.1.0) is a stabilized base on which
we will be able to move forward.

There is nothing bad in having a 1.1.1 out in one week, containing the
pending PR. Differing the release will just make things a bit more
painful...

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

MINA 2.0.16 and FTPServer release : votes needed !

[Emmanuel Lécharny]

Hi guys,


I have pushed 2 release vote last monday, and I really need more binding
votes for them. We currently only have 2...


Many thanks !


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Mina FTP Server : session.close(true) problem

[Emmanuel Lécharny]

Hi,


Le 28/10/16 à 11:52, Robin Vermes a écrit :
> Hello,
>
>
>
> I have got a problem developing a FTP server with the Apache Mina Ftp
> Server API.
>
>
>
> I need my server to be able to shut down immediately a transferring
> session. I am trying to do this using the FtpIoSession.close(true) method.
> What I do not understand is that it is working when I am connecting a
> client via FileZilla and uploading a file, but if I use my own FTP client
> (based on this tutorial : http://www.codejava.net/java-
> se/networking/ftp/java-ftp-file-upload-tutorial-and-example ), the session
> only closes after the transfer has finished.
>
>
>
> After having a closer look via the eclipse debugger, it appears the workers
> in org.apache.mina.filter.executor.OrderedThreadPoolExecutor do run the
> “session closed” task only when the transfer has finished (for the second
> case).

This is a bug that has been fixed in MINA 2.0.16, which is going to be
used in FtpServer 1.1.0, both being currently voted.

I expect those two packages to be available this week-end.

Re: [VOTE] MINA 2.0.16 release

[Emmanuel Lécharny]

For the record, building MINA 2.0.16 with Java 8 may fail. Oracle have
demoted the MD5withRSA cipher, and sadly the certificate we use in some
tests have been created with this cipher.

It's possible to workaround the issue, by changing the
JRE_HOME/lib/security/java.security file, and more specifically the
jdk.certpath.disabledAlgorithms and jdk.tls.disabledAlgorithms
parameters : you will have to remove MD5 and MD5withRSA from those lists.

We will have that fixed in 2.017, by regenerating the certificates.

FtpServer had the same issue, but I have enforced teh use of this cipher
programatically.

Result, was: [VOTE] MINA 2.0.16 release

[Emmanuel Lécharny]

Ok I close the vote with 3 binding +1 and one non binding +1 :

Ashish, jeff and me (bindings) and John (non bnding).


I'll push the release today.


Thanks to the voters !

Result, was: [VOTE] Apache FTPServer 1.1.0 release

[Emmanuel Lécharny]

Hi !

I'm closing this vote, which got 3 binding +1 (Jeff, Ashish and mine)
and one non-bidning +1 (Dave).


I will push the packages, update the site and announce the release in
teh next coming hours.


Many thanks !

MIA for a while...

[Emmanuel Lécharny]

Hi !


just for you to know, I may be MIA for a while, or at least way less
responsive : My wife just gave birth to our wonderful daughter today,
and I'm afraid it's going to be our priority number one in the next few
weeks :-)


Thanks !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [RESULT] [VOTE] Release Apache Mina SSHD 1.4.0

[Emmanuel Lécharny]

Sorry for hving missed teh vote.. Was busy ;-)



Le 03/03/2017 à 07:54, Guillaume Nodet a écrit :
> Closing this vote with 3 binding +1s.
> I'll publish the artifacts asap.
>
> 2017-02-28 18:13 GMT+01:00 Lyor Goldstein :
>
>> +1
>>
>> Proper disclosure: I am the one who made most of the changes and initiated
>> the vote (through Guillaume).
>>
>
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

[VOTE] Apache FTPServer 1.1.1 release

[Emmanuel Lécharny]

Hi,

this is a new release of Apache MINA FTPServer, 1.1.1. It's a bug fix
release, that fixes a few urgent bugs (gathered in FTPSERVER-424).

There will be a need for a 1.2 sooner or later, which should go through the 
existing JIRA tickets. 


In the mean time, let's get this bug fix release out.



A temporary tag has been created (it can be removed if the vote is not
approved):

Updated Branches:
   refs/heads/master 4ac606b55 -> 49788dfc4

Project: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/repo
Commit: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/commit/49788dfc
Tree: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/tree/49788dfc
Diff: http://git-wip-us.apache.org/repos/asf/mina-ftpserver/diff/49788dfc


- Nexus repository: 
https://repository.apache.org/content/repositories/orgapachemina-1029

- Binaries and sources: 
https://dist.apache.org/repos/dist/dev/mina/ftpserver/1.1.1/

Let us vote :
[ ] +1 | Release FTPServer 1.1.1
[ ] ± | Abstain
[ ] -1 | Do **NOT**  release FTPServer 1.1.1

Thanks !

--
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [VOTE] Apache FTPServer 1.1.1 release

[Emmanuel Lécharny]

Thanks to the voters !


I close the vote, with 3 +1 :

Niklas, Ashish and me.


I'll push the releases, update the site and announce it later today.


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Hello! I have found bug in Apache Mina Core!

[Emmanuel Lécharny]

Hi Nikita !


Thanks for the report !

May I suggest a couple of things :

- fill a JIRA (https://issues.apache.org/jira/browse/SSHD), this is for
everybody the best way to have this issue tracked, and the fix will
appear in the release note

- providing a diff is a better way for us

- be accurate, this is a SSHD issue : we have other subprojects
(MINA/FtpServer/Vysper/Asyncweb)

- avoid cross-posting (users/dev mailing list), we follow bth of tjem
anyway :-)


Many thanks !


Le 26/04/2017 à 09:32, Nikita Anishhenko a écrit :
> Hello!
> ScpTimestamp. parseTime must be like this:
>
> public static ScpTimestamp parseTime(String line) throws 
> NumberFormatException {
> String[] numbers = GenericUtils.split(line.substring(1), ' ');
> //return new 
> ScpTimestamp(TimeUnit.SECONDS.toMillis(Long.parseLong(numbers[0])), // It is 
> an error!
> //TimeUnit.SECONDS.toMillis(Long.parseLong(numbers[2]))); 
>   // It is an error!
> return new 
> ScpTimestamp(TimeUnit.SECONDS.toMillis(Long.parseLong(numbers[1])),// 
> Must be like this!
> TimeUnit.SECONDS.toMillis(Long.parseLong(numbers[3])));   
>   // Must be like this!
> }
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [ANNOUNCE] Apache FtpServer 1.1.1 released

[Emmanuel Lécharny]

Correction :


The provided link in teh announce is wrong It's

http://mina.apache.org/ftpserver-project/downloads.html


Thanks !



-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [VOTE] Release Apache Mina SSHD 1.5.0

[Emmanuel Lécharny]

I have a question regarding N&L.


I see that sshd is using the net.i2p.crypto package, and I'm not sure
which license applies : https://geti2p.net/en/get-involved/develop/licenses


In any case, there is no mention of it in N&L files (in assembly) It may
not be necessary, but I'd like a double check.


Same thing for jgit, which is used, and require their LICENSE file to be
added (http://git.eclipse.org/c/gerrit/jgit/jgit.git/tree/LICENSE)


FTR, here are the relevant dependencies (and their transitive
dependencies, which must also be taken into account ) :


$ mvn depenceny:tree

...

[INFO]

[INFO] Building Apache Mina SSHD :: Core 1.5.0
[INFO]

[INFO]
[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ sshd-core ---
[INFO] org.apache.sshd:sshd-core:jar:1.5.0
[INFO] +- org.slf4j:slf4j-api:jar:1.7.24:compileOK
[INFO] +- org.apache.mina:mina-core:jar:2.0.16:compile  OK
[INFO] +- tomcat:tomcat-apr:jar:5.5.23:compile  OK
[INFO] +- org.bouncycastle:bcpg-jdk15on:jar:1.56:compileOK
[INFO] |  \- org.bouncycastle:bcprov-jdk15on:jar:1.56:compile   OK
[INFO] +- org.bouncycastle:bcpkix-jdk15on:jar:1.56:compile  OK
[INFO] +- net.i2p.crypto:eddsa:jar:0.2.0:compileTo be
checked

...

[INFO]

[INFO] Building Apache Mina SSHD :: Git 1.5.0
[INFO]

[INFO]
[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ sshd-git ---
[INFO] org.apache.sshd:sshd-git:jar:1.5.0   
OK
[INFO] +- org.apache.sshd:sshd-core:jar:1.5.0:compile   
OK
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.24:compile   
  OK
[INFO] +-
org.eclipse.jgit:org.eclipse.jgit:jar:4.6.0.201612231935-r:compile   
 To be checked
[INFO] |  +- com.googlecode.javaewah:JavaEWAH:jar:1.1.6:compile   
  To be checked
[INFO] |  \- org.apache.httpcomponents:httpclient:jar:4.4.1:compile   
  OK
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.1:compile   
 OK
[INFO] | \- commons-codec:commons-codec:jar:1.9:compile   
  OK
[INFO] +-
org.eclipse.jgit:org.eclipse.jgit.pgm:jar:4.6.0.201612231935-r:compile   
 To be checked
[INFO] |  +- args4j:args4j:jar:2.0.15:compile   
To be checked
[INFO] |  +- org.apache.commons:commons-compress:jar:1.6:compile   
 OK
[INFO] |  |  \- org.tukaani:xz:jar:1.4:compile   
   To be checked
[INFO] |  +-
org.eclipse.jgit:org.eclipse.jgit.archive:jar:4.6.0.201612231935-r:compile   
  To be checked
[INFO] |  |  \- org.osgi:org.osgi.core:jar:4.3.1:compile   
 To be checked
[INFO] |  +-
org.eclipse.jgit:org.eclipse.jgit.ui:jar:4.6.0.201612231935-r:compile   
   To be checked
[INFO] |  +-
org.eclipse.jgit:org.eclipse.jgit.http.apache:jar:4.6.0.201612231935-r:compile 
To be checked
[INFO] |  +- log4j:log4j:jar:1.2.17:compile   
  OK
[INFO] |  +-
org.eclipse.jetty:jetty-servlet:jar:9.2.13.v20150730:compile   
To be checked
[INFO] |  |  \-
org.eclipse.jetty:jetty-security:jar:9.2.13.v20150730:compile   
To be checked
[INFO] |  | \-
org.eclipse.jetty:jetty-server:jar:9.2.13.v20150730:compile   
   To be checked
[INFO] |  |+-
javax.servlet:javax.servlet-api:jar:3.1.0:compile 
To be checked
[INFO] |  |+-
org.eclipse.jetty:jetty-http:jar:9.2.13.v20150730:compile 
To be checked
[INFO] |  ||  \-
org.eclipse.jetty:jetty-util:jar:9.2.13.v20150730:compile   To
be checked
[INFO] |  |\-
org.eclipse.jetty:jetty-io:jar:9.2.13.v20150730:compile   
To be checked
[INFO] |  +-
org.eclipse.jgit:org.eclipse.jgit.lfs:jar:4.6.0.201612231935-r:compile   
  To be checked
[INFO] |  \-
org.eclipse.jgit:org.eclipse.jgit.lfs.server:jar:4.6.0.201612231935-r:compile  
To be checked



All in all, its just about updating the assembly N&L file in
assembly/src/main/distribution.


Atm, I will cast a -1.


Side note : sorry if I haven't expressed such a concern for any previous
distribution, I'm just trying to catch up with those complex
requirements, and I have spent a huge amount of time last week reading
the ASF doco about N&L.



-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [VOTE] Release Apache Mina SSHD 1.5.0

[Emmanuel Lécharny]

Le 30/05/2017 à 15:47, Guillaume Nodet a écrit :
> Our source code depends on those 2 libraries, however they are optional,
> and we don't ship them in our assembly.

Optional in what sense ? Seems they are used in compile scope, so my
guess is that they are part of the package. Can you check that ?

Typically, how do you make them part of the package, or how do you
exclude them ?

> So I don't think their LICENSE/NOTICE files should end up in our binary
> distribution.  I'm not sure they need to be in the source distribution
> either.
If they are not included in the source package and in teh binary
package, then they dn't need to be added in the N&L files.
>>
>>
>> --
>> Emmanuel Lecharny
>>
>> Symas.com
>> directory.apache.org
>>
>>
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [VOTE] Apache Mina SSHD 1.6.0

[Emmanuel Lécharny]

+1




Le 29/06/2017 à 11:36, Guillaume Nodet a écrit :
> I've staged a release for SSHD 1.6.0 at:
>   https://repository.apache.org/content/repositories/orgapachemina-1031
>
> I've moved issues targeted to 1.5.0 to 1.6.0, so we have the overall
> following changes:
>
> Release Notes - MINA SSHD - Version 1.6.0
> ** Bug
> * [SSHD-85] - Port Forward closes connection before all bytes are sent
> * [SSHD-728] - sshd-core sftp not working with FileZilla sftp client
> * [SSHD-731] - Vulnerability in SimpleAccessControlSftpEventListener
>  implementation
> * [SSHD-732] - ClientIdentitiesWatcher should load keys one at a time
> * [SSHD-734] - When ClientSessionImpl construction fails,
> AbstractSessionIoHandler#exceptionCaught may throw NPE
> * [SSHD-749] - SFTP client sends wrong open flags for protocol version 4
> * [SSHD-752] - HostConfigEntry can't parse tabbed SSH config files
> * [SSHD-753] - SSHD cannot read its keyfile through a symlink
> ** Improvement
> * [SSHD-739] - A call to resolvePropertyValue can be very inefficient
> * [SSHD-740] - Add default methods on PropertyResolver to actually do
> the property resolution
> * [SSHD-747] - Add support for non-standard port specification in
> known_hosts file
> * [SSHD-748] - Activate Maven PMD plugin in the project
> ** New Feature
> * [SSHD-656] - Support The PROXY protocol
> * [SSHD-710] - Cannot connect standard OpenSSH client/server using
> ed25519 keys
> * [SSHD-741] - Provide seamless replacement for Spring integation SFTP
> adapter
> * [SSHD-750] - Accept SSH clients advertising vesion 1.99
> ** Task
> * [SSHD-727] - Upgrade used EdDSA artifact version to 1.1
> ** Wish
> * [SSHD-733] - SSHD server displays file symlinks instead of dir
> symlinks
>
>
> Please review and vote !
>
> Cheers,
> Guillaume Nodet
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Maven releases are missing?

[Emmanuel Lécharny]

1.5.0 ha snever been 'officially' released. It will never exist, as we
are currently voting for a 1.6.0.




Le 05/07/2017 à 00:39, Eugene Petrenko a écrit :
> Hi,
>
> I noticed there is 1.5.0 release visible on GitHub, but there are no jar
> files released for the version on maven. Is it an intended?
>
> Releases on GitHub
> https://github.com/apache/mina-sshd/releases
>
> Maven search
> https://mvnrepository.com/artifact/org.apache.sshd/sshd-core
>
> Best regards,
> Eugene Petrenko
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Le 26/07/2017 à 13:59, Christoph John a écrit :
> Hi,
>
> I am a developer and maintainer of the QuickFIX/J project
> (https://github.com/quickfix-j/quickfixj) and I have a question
> regarding NioSocketConnectors.
>
> We are facing a problem when there is a process that constantly (every
> 30 seconds) tries to connect to a counterparty and the connection is
> established but dropped shortly after. Then sometimes the
> NioProcessors/NioSocketConnectors are not cleaned up properly. In the
> stack trace we see them hanging in a call to dispose:
>
> "NioProcessor-1140" #239 prio=5 os_prio=0 tid=0x01fe1800
> nid=0x2523 runnable [0x7f9c67e8f000]
>java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xf6699e60> (a sun.nio.ch.Util$3)
> - locked <0xf6699e50> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0xf6699c18> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.mina.transport.socket.nio.NioProcessor.select(NioProcessor.java:98)
> at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1075)
> at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
>
> "NioSocketConnector-68" #238 prio=5 os_prio=0 tid=0x7f9c70caf000
> nid=0x2522 in Object.wait() [0x7f9c6af9f000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:209)
> - locked <0xf66ac718> (a
> org.apache.mina.core.future.DefaultIoFuture)
> at
> org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:141)
> at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.dispose(AbstractPollingIoProcessor.java:188)
> at
> org.apache.mina.core.service.SimpleIoProcessorPool.dispose(SimpleIoProcessorPool.java:329)
> - locked <0xf66ac750> (a java.lang.Object)
> at
> org.apache.mina.core.polling.AbstractPollingIoConnector$Connector.run(AbstractPollingIoConnector.java:582)
> at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
>
> It does not happen very often: about 5% of the connection attempts
> leave a NioSocketConnector hanging.
> It only seems to happen though when the connection is disconnected by
> "javax.net.ssl.SSLHandshakeException: SSL handshake failed". Although
> there are cases when there is no leak even on an SSLHandshakeException.
> If the connection was reset "normally" by "java.io.IOException:
> Connection reset by peer" then the leak does not seem to occur. It
> also does not occur when the connection is refused right away.
>
> Since this seems to be related to SSL connections: is there something
> that we need to take care of when using the SSL filter?
>
> The code for the IoSessionInitiator can be found here:
> https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/main/java/quickfix/mina/initiator/IoSessionInitiator.java
> I have added some comments in this gist (starting with "chrjohn"):
> https://gist.github.com/chrjohn/2671f06d80e8d917d9061b573477ec5b
>
> I cannot rule out that we might be doing something wrong here, so any
> pointer is appreciated. :)

I see in your code that you are waiting 2s for the connection to be
established, and if this timeout is reached, you try again, up to teh
point you bail out. In tjis case, teh connection is not cleared up, AFAICT.

Is that correct ?


OTOH, it does not necessarily makes a lot of sense to poll the connector
: as MINA is fully asynchronous, you'll be informed when the connection
is established, and if not, you can use the idle event to know that your
connection is idling (an idle event is generated every second, so
waiting for, say, 30 idle events will let you manage a 30s timeout, for
instance). If your connection idle for too long, simply dispose it.

>
> Thanks in advance for your help and best regards,
> Chris.
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Le 26/07/2017 à 16:40, Christoph John a écrit :
> Hi,
>
> thanks for your reply. I will check the points you mentioned and come
> back.
> Most of the code in that class is quite some years old and started off
> as a C++ project (quickfix) and was also using MINA 1.x before. So it
> might be that nowadays there are some suboptimal things done there. :)
In any case, if you have question related to the code, feel free to ask !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

(Please just reply to the list)

comments in line


Le 27/07/2017 à 10:25, Christoph John a écrit :
> One question that comes to my mind after looking at our code: there is
> a Boolean attribute get/set on the IoSession in various places
> (SessionConnector.QFJ_RESET_IO_CONNECTOR). We get/set this from
> different locations and threads. But we neither synchronize on the
> IoSession nor the get/set of the attribute. So IMHO it could happen
> that the attribute is set to a different value than actually expected,
> triggering unexpected behaviour.
> I only searched briefly but could not find anything in the MINA code
> that makes getting/setting the attribute thread-safe.

Attributes within a session are not protected : it's up to you to make
sure they are not modified concurently. Now, for a boolean, using an
AtomicBoolean would certainly help.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Le 27/07/2017 à 17:22, Christoph John a écrit :
> But if I access IoSession.getAttribute() and setAttribute() from
> different threads: can't it happen that one thread does not "see" if I
> put an attribute into the IoSession? 
Not if you add this attribute when you create the session : it will be
present no mater what for every threads. Of course, you should not
remove it from the session.

And btw, the session attributes are stored in a concurrentHashmap, so
accessing it is threadsafe.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Le 28/07/2017 à 23:02, Christoph John a écrit :
> Any idea regarding my filter question?
> I tried to close the filters but somehow there were none registered.
> But most probably I missed something. Will try further.

So let's clarify something : filters are added in the chain when the
connection is created (ie, for every new connection, the first step is
the initialization step, where MINA creates a list of filter this
session will use). You can add also a filter on the fly (like you have a
non  secure connection, and you decide to secure it while 'talking' to
the remote peer, you just add the filter in your session filter chain.
This is what we do when dealing with StartTLS, for instance. In any
case, it's not that frequent, and usually, nce the conneciton has been
created, the filters used by this connection does not change.

That being said, SSL filter is kind of a beast : it has to first ensure
the handshake is completed before being able to transmit data. If you
had a exception during the handshake, you most cettainly need to drop
teh connection anyway, because there is little you can do to restore it.
There is a exceptionCaught() event you can catch in your application
that can be used to dispose the session.
>
> Thanks,
> Chris.
>
>
> On 27/07/17 17:13, Christoph John wrote:
>>
>> I have now a test which tries to connect several FIX sessions with
>> enabled SSL filter. The connection is established and then dropped
>> because of an SSLHandshakeException. This is done for about a minute
>> and leads to the mentioned problem with the connectors hanging in
>> dispose.
>> When I change the test to not use SSL I can see no "old" connectors
>> hanging around. Is there something that I must specifically do when
>> using the SSL filter? E.g. when there is an exception caught? First
>> destroy all filters in the chain and then close the session?
>>
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Le 29/07/2017 à 23:15, Christoph John a écrit :
> Hi,
>
> thanks, I'll test what you suggested.
> I've noticed that we have an exceptionCaught handler in one of our
> classes which extend the MINA IoHandlerAdapter. How is that different
> from the exceptionCaught handler in the filter?
No difference. The filter hadles events the same way the IoHandler does,
except that those events are handled by the filters *before* it reaches
the handler.

Actually, once a filter handles an event, it may passes it to the next
filter, and ultimately it bubbles up to the hoHandler. Here is the
Logging filter implementation :

@Override
public void exceptionCaught(NextFilter nextFilter, IoSession
session, Throwable cause) throws Exception {
log(exceptionCaughtLevel, "EXCEPTION :", cause);
nextFilter.exceptionCaught(session, cause);
}


As you can see it calls the next filter exceptionCaught event. The last
filter in a chain is the TailFilter (you don't have to add it in your
chain, it's always there) and it calls your handler :


@Override
public void exceptionCaught(NextFilter nextFilter, IoSession
session, Throwable cause) throws Exception {
AbstractIoSession s = (AbstractIoSession) session;

try {
s.getHandler().exceptionCaught(s, cause);
} finally {
if (s.getConfig().isUseReadOperation()) {
s.offerFailedReadFuture(cause);
}
}
}




-- 

Emmanuel Lecharny

Symas.com
directory.apache.org

Re: MINA3.0 recommended

[Emmanuel Lécharny]

Le 18/08/2017 à 07:53, 胡阳 a écrit :
> Hi guys:

Hi ! (Sorry, I can't use your first name, my keyboard is a bit limited ;-)

> I read the source code of MINA3.0M2. The style of the code is very good, the 
> structure is clear, the design is concise and efficient, especially the use 
> of Selector is unexpected. However, the enqueueWriteRequest method and the 
> processWrite method in the AbstractNioSession are somewhat flawed.

Probably :-) This is just an effort to improve from MINA 2, and we are
very open to any proposal !
> I see the source code in the enqueueWriteRequest method was originally 
> "synchronized (writeQueue)", but was commented out, personal speculation may 
> be the author feel that this treatment will affect performance.
You are looking at an old version of the code base. The current (not
released) version is using a ConcurrentLinkedQueue, which does not
require a synchronized anymore.

> My approach is to use CAS to ensure memory visibility and atomic, see I see 
> the startSync, finishSync method, feeling that this may be more secure after 
> some of the performance will not lose too much.
> A little personal humble opinion.
And we don't disagree :-)

If you are interested in this code, feel free to dig the trunk, which
contains the latest changes. Note that it's not a very active project
atm, but we would really appreciate any contribution !

You can also use github to push PR, would you feel like contributing :
https://github.com/apache/mina

Thanks !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Mailing Lists

[Emmanuel Lécharny]

Le 30/08/2017 à 16:30, Guido Medina a écrit :
> Google forums are nice for that, I don't like the old way of handling
> forums TBH i.e.: e-mails,
> Google forums also have nice code formatting, friendly search capability,
> etc.

Certainly not. This is the 'best' way to fragment the community :
creating mailing list/discussion group on whatever social media one
would like. Plus we have no visibility about when private companies like
Google will decide to brutally discontinue the server - as they already
did many times in the past -, losing all the history at the same time.

I'd rather split the existing list into a SSHd list, a FTPServer list, a
MINA list, etc.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Mailing Lists

[Emmanuel Lécharny]

Le 30/08/2017 à 17:52, Guido Medina a écrit :
> I forgot to mention that the Gitter interface uses your Github user as
> login and allows you to reference PR and ping Github users.
> It is similar to Slack, but IMHO it is better designed for code and of
> course much better integration with Github.
All those ideas have been floated many times at The ASF.

The real problem is that we are not on the same TZ, and mail is perfect
for that : it's asynchronous.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: MINA3.0 recommended

[Emmanuel Lécharny]

Le 18/08/2017 à 07:53, 胡阳 a écrit :
> Hi guys:
> I read the source code of MINA3.0M2. The style of the code is very good, the 
> structure is clear, the design is concise and efficient, especially the use 
> of Selector is unexpected. However, the enqueueWriteRequest method and the 
> processWrite method in the AbstractNioSession are somewhat flawed.
> I see the source code in the enqueueWriteRequest method was originally 
> "synchronized (writeQueue)", but was commented out, personal speculation may 
> be the author feel that this treatment will affect performance.
> My approach is to use CAS to ensure memory visibility and atomic, see I see 
> the startSync, finishSync method, feeling that this may be more secure after 
> some of the performance will not lose too much.
> A little personal humble opinion.

Sorry for coming back so late... Busy days :/

I do agree that we should use CAS whenever we can, and Thread Local
Storage, instead of any other synchronisation solution.

Feel free to propose patches that we could inject into teh code !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: MINA3.0 recommended

[Emmanuel Lécharny]

Le 19/09/2017 à 00:24, Jonathan Valliere a écrit :
> Synchronization, unlike Locks, does not create any memory garbage and are
> just as fast as CAS under low locking activity.  Just because CAS/ Locks
> are faster when 8 threads are accessing one object, doesn't mean that
> standard Mutex Synchronization doesn't work just find when almost all of
> the time a single thread acquires the lock.  In reactor IO frameworks,
> there is almost never more than one thread accessing the resource.  Full
> Locks are usually overkill unless there is something specific you want to
> attempt.  NIO internally uses the synchronized keyword for all state and
> read / write locking anyway.

FTR, the number of threads MINA uses by default is pretty limited : nb
Cores +1. The main place where yo ight have contention is when responses
are enqueued, because a session might be active on more than one thread
(like, for LDAP, you might have a Search request *and* an Abandon
Request executed in parallel).

IMHO, such a situation should be quite rare, as you said, and it would
worth it analyse the pros and cons of each solution, my point being that
we simply discarded using alternative synchronisation mechanisms years
ago (but that was back when Java 5 was out, when we were stuck on Java
4). We haven't revisited the item since them, or barely.

What I mean is that the code base is pretty old and we should, at some
point, check if any other solution wouldn't be better all things being
equal. And if synchronized sections are proven o be more efficient in
the general case, there is no reason to ditch it, of course !


Thanks for your valuable input !
>
> On Mon, Sep 18, 2017 at 5:35 PM, Emmanuel Lécharny 
> wrote:
>
>>
>> Le 18/08/2017 à 07:53, 胡阳 a écrit :
>>> Hi guys:
>>> I read the source code of MINA3.0M2. The style of the code is very good,
>> the structure is clear, the design is concise and efficient, especially the
>> use of Selector is unexpected. However, the enqueueWriteRequest method and
>> the processWrite method in the AbstractNioSession are somewhat flawed.
>>> I see the source code in the enqueueWriteRequest method was originally
>> "synchronized (writeQueue)", but was commented out, personal speculation
>> may be the author feel that this treatment will affect performance.
>>> My approach is to use CAS to ensure memory visibility and atomic, see I
>> see the startSync, finishSync method, feeling that this may be more secure
>> after some of the performance will not lose too much.
>>> A little personal humble opinion.
>> Sorry for coming back so late... Busy days :/
>>
>> I do agree that we should use CAS whenever we can, and Thread Local
>> Storage, instead of any other synchronisation solution.
>>
>> Feel free to propose patches that we could inject into teh code !
>>
>> --
>> Emmanuel Lecharny
>>
>> Symas.com
>> directory.apache.org
>>
>>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [RESULT] [VOTE] Apache Mina SSHD 1.6.0

[Emmanuel Lécharny]

Le 19/09/2017 à 08:26, Jeff MAURY a écrit :
> Just noticed while working on the board report that the web site still
> advertise 1.2.0 as the latest release

Fixed. Thanks Jeff !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [RESULT] [VOTE] Apache Mina SSHD 1.6.0

[Emmanuel Lécharny]

Le 19/09/2017 à 08:26, Jeff MAURY a écrit :
> Just noticed while working on the board report that the web site still
> advertise 1.2.0 as the latest release


That makes me wonder were the previous versions (1.3.0, 1.4.0 and 1.5.0)
are stored ? I haven't found them on any repo...

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: MINA3.0 recommended

[Emmanuel Lécharny]

Le 19/09/2017 à 15:28, Jonathan Valliere a écrit :
> I forget, does MINA uses the same Processor for both read and write
> operations?  If so, there will never be contention unless you use a Thread
> Executor Pool in the Filters.

It does. MINA use always the same IoProcessor for one session, to avoid
having to manage the session attributes access across may threads. As
you said, one can add a Executor filter, but then, it's up to the
application to deal with concuttent accesses.

>
> If you look at the tests here:
> https://dzone.com/articles/synchronized-vs-lock  You can see that until you
> really hit > 2 threads, sync actually is faster.

I find this article a bit dubious. The numbe this guy get for 2 threads
is clearly problematic. You will find other blogs that shows Locks are
better as soon as you have more than a thread :

http://lycog.com/concurency/performance-reentrantlock-synchronized/
or
https://objectpartners.com/2011/11/29/java-synchronization-method-performance-comparison/

I won't say I fully trust those blog posts : there are too many moving
pieces (Java version, benchmark, OS used, etc)

But overall, I would say it's not a clear cut.


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: MINA3.0 recommended

[Emmanuel Lécharny]

I would add Brian Goetz take on synchronized vs Lock :


https://www.ibm.com/developerworks/library/j-jtp10264/



Le 19/09/2017 à 15:28, Jonathan Valliere a écrit :
> I forget, does MINA uses the same Processor for both read and write
> operations?  If so, there will never be contention unless you use a Thread
> Executor Pool in the Filters.
>
> If you look at the tests here:
> https://dzone.com/articles/synchronized-vs-lock  You can see that until you
> really hit > 2 threads, sync actually is faster.
>
> Personally, I use a custom ReentantLock but my use-case is much more
> complicated.
>
> On Tue, Sep 19, 2017 at 1:31 AM, Emmanuel Lécharny 
> wrote:
>
>>
>> Le 19/09/2017 à 00:24, Jonathan Valliere a écrit :
>>> Synchronization, unlike Locks, does not create any memory garbage and are
>>> just as fast as CAS under low locking activity.  Just because CAS/ Locks
>>> are faster when 8 threads are accessing one object, doesn't mean that
>>> standard Mutex Synchronization doesn't work just find when almost all of
>>> the time a single thread acquires the lock.  In reactor IO frameworks,
>>> there is almost never more than one thread accessing the resource.  Full
>>> Locks are usually overkill unless there is something specific you want to
>>> attempt.  NIO internally uses the synchronized keyword for all state and
>>> read / write locking anyway.
>> FTR, the number of threads MINA uses by default is pretty limited : nb
>> Cores +1. The main place where yo ight have contention is when responses
>> are enqueued, because a session might be active on more than one thread
>> (like, for LDAP, you might have a Search request *and* an Abandon
>> Request executed in parallel).
>>
>> IMHO, such a situation should be quite rare, as you said, and it would
>> worth it analyse the pros and cons of each solution, my point being that
>> we simply discarded using alternative synchronisation mechanisms years
>> ago (but that was back when Java 5 was out, when we were stuck on Java
>> 4). We haven't revisited the item since them, or barely.
>>
>> What I mean is that the code base is pretty old and we should, at some
>> point, check if any other solution wouldn't be better all things being
>> equal. And if synchronized sections are proven o be more efficient in
>> the general case, there is no reason to ditch it, of course !
>>
>>
>> Thanks for your valuable input !
>>> On Mon, Sep 18, 2017 at 5:35 PM, Emmanuel Lécharny 
>>> wrote:
>>>
>>>> Le 18/08/2017 à 07:53, 胡阳 a écrit :
>>>>> Hi guys:
>>>>> I read the source code of MINA3.0M2. The style of the code is very
>> good,
>>>> the structure is clear, the design is concise and efficient, especially
>> the
>>>> use of Selector is unexpected. However, the enqueueWriteRequest method
>> and
>>>> the processWrite method in the AbstractNioSession are somewhat flawed.
>>>>> I see the source code in the enqueueWriteRequest method was originally
>>>> "synchronized (writeQueue)", but was commented out, personal speculation
>>>> may be the author feel that this treatment will affect performance.
>>>>> My approach is to use CAS to ensure memory visibility and atomic, see I
>>>> see the startSync, finishSync method, feeling that this may be more
>> secure
>>>> after some of the performance will not lose too much.
>>>>> A little personal humble opinion.
>>>> Sorry for coming back so late... Busy days :/
>>>>
>>>> I do agree that we should use CAS whenever we can, and Thread Local
>>>> Storage, instead of any other synchronisation solution.
>>>>
>>>> Feel free to propose patches that we could inject into teh code !
>>>>
>>>> --
>>>> Emmanuel Lecharny
>>>>
>>>> Symas.com
>>>> directory.apache.org
>>>>
>>>>
>> --
>> Emmanuel Lecharny
>>
>> Symas.com
>> directory.apache.org
>>
>>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Regarding MINA Project

[Emmanuel Lécharny]

Le 20/09/2017 à 22:32, harmeet singh a écrit :
> Hi,I was just exploring sshd project, could you provide me documentation of 
> this project.

I would say all you can get is on the web site
(http://mina.apache.org/sshd-project/index.html and
http://mina.apache.org/sshd-project/documentation.html)

If you want to see some samples on how to use it, check the samples :
https://git-wip-us.apache.org/repos/asf?p=mina-sshd.git;a=tree;f=sshd-core/src/test/java/org/apache/sshd;h=f6e0e89724bc188797d39c53c61e61dd342a6ad1;hb=b737b07d251479dd186ce877cad5b0d13b9bb054
> I wanted to check dual authentication feature, looks like it support only 
> single authentication.

I can't tell, but Lyor or Guillaume could answer.

> Also could you please let me know if it can be used for production and is it 
> being patched regularly for security vulnerabilities, PCI compliance etc.
We just released 1.6.0 two weeks ago. Patches are being applied
regularly. Of course, this is OpenSource code, so if you find something
wrong, feel free to participate. I'm pretty sure it's used in
production, but again, we are just providing source code and packages
(for convenience), people using it are not necesseraly coming back to us
about what they use it for.

FTR, Apache SSHD packages are being downloaded around 70 000 times per
month, and growing (source : https://repository.apache.org/#central-stat)

Regarding PCI compliance, I would say it's on you : due diligence is
required on your side, as we have no idea what you are going to do with
the project. Add to that it would require a huge amount of time if we
were to check all teh aspects of PCI, time we don't have being volunteers.

Again, I'll stress out the fact this is Open Source, we are not paid to
write this code, we offer it to the community, up to the people using it
to either participate or use it at will, but we aren't responsible for
anything else.

Hope it helps !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [jira] [Commented] (DIRMINA-1057) AbstractIoSession getScheduledWriteMessages always -negative?

[Emmanuel Lécharny]

Hi Jonathan,


it's probably better to put your comment in the JIRA ticket, for a
better follow up...


That being said,


Le 06/10/2017 à 20:08, Jonathan Valliere a écrit :
> I haven’t looked at the Mina code in a while.  Looking at
> DefaultIoFilterChain.java under HeadFilter#filterWrite
>
> WriteRequestQueue writeRequestQueue = s.getWriteRequestQueue();
>
> if (!s.isWriteSuspended()) {
> if (writeRequestQueue.isEmpty(session)) {
> // We can write directly the message
> s.getProcessor().write(s, writeRequest);
> } else {
> s.getWriteRequestQueue().offer(s, writeRequest);
> s.getProcessor().flush(s);
> }
> } else {
> s.getWriteRequestQueue().offer(s, writeRequest);
> }
>
> Checking and working with the WriteRequestQueue is unnecessary because the
> AbstractPollingIoProcessor adds all write requests to the Queue anyway.

Those are two different queues. The HeadFilter stacks the message in a
session queue, while the AbstractPollingIoProcessor stacks the message
in the IoProcessor queue. You may have thousands of sessions but nly a
few IoProcessor which handle many sessions.
>
> Similarly, it is confusing that the HeadFilter is controlling the
> increaseScheduledWriteBytes.  This should be controlled by the
> AbstractPollingIoProcessor to ensure safety of the increase/decrease
> operations.

Again, this is a per session counter vs a per IoProcessor counter. The
application is really interested on what happens on ifself.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [jira] [Commented] (DIRMINA-1057) AbstractIoSession getScheduledWriteMessages always -negative?

[Emmanuel Lécharny]

Le 07/10/2017 à 16:43, Jonathan Valliere a écrit :
> Emmanuel,
>
> I’m not sure my discussions is warranted inside of the ticket unless I come
> to some sort of conclusion.

Well, we can discuss in the ticket, to keep the context around. But it's
up to you...
>
> I don’t think that is true that the message is stacked in the IoProcessor
> Queue.

I don't think I'm too :-) Actually, I re-checked teh code a bit more
throughously and you are right there is only one queue which is teh
session's queue. The IoProcessor simply loop on all the sessions that ar
eintersted in write (OP_WRITE is set) and flush the message in the socket.
>
> @Override
> public void write(S session, WriteRequest writeRequest) {
> WriteRequestQueue writeRequestQueue =
> session.getWriteRequestQueue();
>
> writeRequestQueue.offer(session, writeRequest);
>
> if (!session.isWriteSuspended()) {
> this.flush(session);
> }
> }
>
> WriteRequestQueue writeRequestQueue = s.getWriteRequestQueue();
>
> if (!s.isWriteSuspended()) {
> if (writeRequestQueue.isEmpty(session)) {
> // We can write directly the message
> s.getProcessor().write(s, writeRequest);
> } else {
> s.getWriteRequestQueue().offer(s, writeRequest);
> s.getProcessor().flush(s);
> }
> } else {
> s.getWriteRequestQueue().offer(s, writeRequest);
> }
>
> @Override
> public final void flush(S session) {
> // add the session to the queue if it's not already
> // in the queue, then wake up the select()
> if (session.setScheduledForFlush(true)) {
> flushingSessions.add(session);
> wakeup();
> }
> }
>
> Both use Session.getWriteRequestQueue.  Seems like the IoProcessor has a
> Queue which contains IoSession objects and not the Writable Data.

Indeed. Was too long away from the code...
> Basically, I’m looking for a concise place to unify and make the counters
> thread-safe.
Some of the counters are AtomicInt/AtomicLong ad should be thread safe.
Some are protected by a lock :

    public final void increaseScheduledWriteBytes(int increment) {
    throughputCalculationLock.lock();

    try {
    scheduledWriteBytes += increment;
    } finally {
    throughputCalculationLock.unlock();
    }
    }

but some others are simply not thread safe : writtenMessages for
instance, which is a int, incremented outside any lock.

At this point, I think we should try to be generic and only use
Atomic(Int|Long) for all the stats variables, and get rid of the locks.

wdyt ?

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Hi Christophe, Jonathan,

when dispose() is called, is first set the 'disposing' flag, which will
be checked in the IoProcessor select loop. The disposalFuture is
signaled after the select loop, in a try... catch... finally.


That supposes the loop ends at some point, and the only reason for the
infinite loop to exit is that we have no session associated with the
processor. Session are removed from the IoProcessor by this piece of code :


    if (isDisposing()) {            // Clearly, the
'disposing' flag is set
    boolean hasKeys = false;

    for (Iterator i = allSessions(); i.hasNext();) {
    IoSession session = i.next();

    if (session.isActive()) {            // A
session is active if its selectionKey is accepting OP_READ ror OP_WRITE
    scheduleRemove((S) session);   
    hasKeys = true;
    }
    }

    if (hasKeys) {
    wakeup();
    }
    }

The only reason I see for a session to not be removed from the
IoProcessor is that it's not active anymore. That may be because its
channel has been closed, but the session has not been removed from the
IoProcessor. This is soemthing to investigate (typically, what happens
when teh HS fails...)


Le 10/10/2017 à 10:49, Christoph John a écrit :
> Hi,
>
> thanks for your reply.
> In fact it is hanging forever, i.e. until the process stops. I have
> attached the original message I've sent to the mailing list. It only
> does occur sometimes for SSL connections with a failing handshake.
> Unfortunately I have no reproducable example for MINA itself. I could
> probably put something together for QuickFIX/J (the open source
> project I am working on).
>
> My OS is Ubuntu 14.04.5, JDK1.8_144 and the problem appears not so
> often on my machine but almost every time on the TravisCI build server
> (https://travis-ci.org/quickfix-j/quickfixj/builds/283210509). As a
> result, some of the SSL related tests are failing. TravisCI has almost
> similar setup with JDK1.8_144 and Debian Linux.
>
> What would be a good starting point to create a test? I see that there
> is an SslTest in the mina-core module. So I probably have to change
> that test to repeatedly connect and get a handshake exception
> everytime and then take a number of stack traces.
>
> Thanks,
> Chris.
>
>
>
>
> On 09/10/17 14:51, Jonathan Valliere wrote:
>> What OS / Java Version / etc;  Do you have a reproducible example?
>>
>> On Mon, Oct 9, 2017 at 8:34 AM, Jonathan Valliere
>> mailto:jon.valli...@emoten.com>> wrote:
>>
>>     Let me know if its hanging more than 1s
>>
>>     On Mon, Oct 9, 2017 at 5:08 AM, Christoph John
>> >     > wrote:
>>
>>     Hi,
>>
>>     I have another question regarding this one. There is
>>     https://issues.apache.org/jira/browse/DIRMINA-1060
>>      which
>> also sounds a little like the
>>     problem I'm having. When the connectors are hanging in the
>> call to dispose() then there
>>     always is an accompanying NioProcessor which is hanging in
>> select().
>>
>>     Example:
>>     "NioProcessor-60" #100328 prio=5 os_prio=0
>> tid=0x7f2a10003000 nid=0x2e71 runnable
>>     [0x7f2a388b1000]
>>        java.lang.Thread.State: RUNNABLE
>>         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>>         at
>> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
>>         at
>> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
>>         at
>> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
>>         - locked <0xe239c118> (a sun.nio.ch.Util$3)
>>         - locked <0xe239c108> (a
>> java.util.Collections$UnmodifiableSet)
>>         - locked <0xe239bed0> (a
>> sun.nio.ch.EPollSelectorImpl)
>>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
>>         at
>> org.apache.mina.transport.socket.nio.NioProcessor.select(NioProcessor.java:98)
>>         at
>>    
>> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1075)
>>         at
>> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>         at java.lang.Thread.run(Thread.java:748)
>>
>>
>>     "NioSocketConnector-38" #100326 prio=5 os_prio=0
>> tid=0x7f2a3001d800 nid=0x2e6f in
>>     Object.wait() [0x7f2a1f2d3000]
>>      

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Hi,


can you test with this patch ?


diff --git
a/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
b/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
index 50ebd4e..575b2f4 100644
---
a/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
+++
b/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
@@ -695,8 +695,9 @@
 for (Iterator i = allSessions(); i.hasNext();) {
 IoSession session = i.next();
 
+    scheduleRemove((S) session);
+
 if (session.isActive()) {
-    scheduleRemove((S) session);
 hasKeys = true;
 }
 }


Le 11/10/2017 à 12:04, Christoph John a écrit :
> Hi,
>
> thanks for the patch. I am using 2.0.16.
> Oddly enough when I run all MINA tests then the ConnectorTest is
> hanging on my machine in the testTCPWithSSL method. But I don't know
> if this is related. (stack trace attached)
> However, I will try out your patch and let you know.
>
> Thanks again,
> Chris.
>
>
> On 10/10/17 20:47, Jonathan Valliere wrote:
>> Which version of Mina are you using or are you building from Git?
>>
>> Please pull tag/2.0.16 from GIT and apply the attached patch. Let me
>> know if that fixes your problem.  Sorry about the excess changes in
>> the patch; the java code formatter made a lot of changes. If this
>> works then we can create a JIRA bug.
>>
>> On Tue, Oct 10, 2017 at 4:49 AM, Christoph John
>> mailto:christoph.j...@macd.com>> wrote:
>>
>>     Hi,
>>
>>     thanks for your reply.
>>     In fact it is hanging forever, i.e. until the process stops. I
>> have attached the original
>>     message I've sent to the mailing list. It only does occur
>> sometimes for SSL connections with a
>>     failing handshake.
>>     Unfortunately I have no reproducable example for MINA itself. I
>> could probably put something
>>     together for QuickFIX/J (the open source project I am working on).
>>
>>     My OS is Ubuntu 14.04.5, JDK1.8_144 and the problem appears not
>> so often on my machine but
>>     almost every time on the TravisCI build server
>>     (https://travis-ci.org/quickfix-j/quickfixj/builds/283210509
>>     ).
>> As a result, some of the SSL
>>     related tests are failing. TravisCI has almost similar setup with
>> JDK1.8_144 and Debian Linux.
>>
>>     What would be a good starting point to create a test? I see that
>> there is an SslTest in the
>>     mina-core module. So I probably have to change that test to
>> repeatedly connect and get a
>>     handshake exception everytime and then take a number of stack
>> traces.
>>
>>     Thanks,
>>     Chris.
>>
>>
>>
>>
>>
>>     On 09/10/17 14:51, Jonathan Valliere wrote:
>>>     What OS / Java Version / etc;  Do you have a reproducible example?
>>>
>>>     On Mon, Oct 9, 2017 at 8:34 AM, Jonathan Valliere
>>> >>     > wrote:
>>>
>>>     Let me know if its hanging more than 1s
>>>
>>>     On Mon, Oct 9, 2017 at 5:08 AM, Christoph John
>>> >>     > wrote:
>>>
>>>     Hi,
>>>
>>>     I have another question regarding this one. There is
>>>     https://issues.apache.org/jira/browse/DIRMINA-1060
>>>     
>>> which also sounds a little like
>>>     the problem I'm having. When the connectors are hanging
>>> in the call to dispose() then
>>>     there always is an accompanying NioProcessor which is
>>> hanging in select().
>>>
>>>     Example:
>>>     "NioProcessor-60" #100328 prio=5 os_prio=0
>>> tid=0x7f2a10003000 nid=0x2e71 runnable
>>>     [0x7f2a388b1000]
>>>        java.lang.Thread.State: RUNNABLE
>>>         at sun.nio.ch.EPollArrayWrapper.epollWait(Native
>>> Method)
>>>         at
>>> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
>>>         at
>>> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
>>>         at
>>> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
>>>         - locked <0xe239c118> (a sun.nio.ch.Util$3)
>>>         - locked <0xe239c108> (a
>>> java.util.Collections$UnmodifiableSet)
>>>         - locked <0xe239bed0> (a
>>> sun.nio.ch.EPollSelectorImpl)
>>>         at
>>> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
>>>         at
>>> org.apache.mina.transport.socket.nio.NioProcessor.select(NioProcessor.java:98)
>>>         at
>>>    
>>> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPolli

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Le 11/10/2017 à 14:51, Christoph John a écrit :
> Thanks, will try. Did this patch apply to 2.0.16? The line numbers do
> not match. I have added it manually in line 1162 and removed the
> statement from line 1164.

It's for 2.0.17-SNAPSHOT, so you may have differences with 2.0.16, but
minor.

The idea here is to remove a session when it's i teh list of managed
version, even if it's not valid (that might be a corner case when teh
SSLFilter does not properly handle an exception and close the channel.
Call it a workaround atm)

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: leaking NioProcessors/NioSocketConnectors hanging in call to dispose

[Emmanuel Lécharny]

Hi Christoph,


have you tried with this patch ? :


diff --git
a/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
b/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
index 50ebd4e..575b2f4 100644
---
a/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
+++
b/mina-core/src/main/java/org/apache/mina/core/polling/AbstractPollingIoProcessor.java
@@ -695,8 +695,9 @@
 for (Iterator i = allSessions(); i.hasNext();) {
 IoSession session = i.next();
 
+    scheduleRemove((S) session);
+
 if (session.isActive()) {
-    scheduleRemove((S) session);
 hasKeys = true;
 }
 }


Le 16/10/2017 à 14:41, Christoph John a écrit :
> Hi Emmanuel, Jonathan,
>
> thanks for your patches. Unfortunately neither of them does correct
> the problem I am facing.
> I'd suggest that I'll try to reproduce this with a MINA-based test
> first. Otherwise it is probably just guess-work.
>
> Thanks for your help and best regards,
> Chris.
>
>
> On 11/10/17 16:14, Jonathan Valliere wrote:
>> Basically my proposed patch moves the check dispose conditional into a
>> finally loop forcing it to run even if exceptions were caught.
>>
>> If the SSLFilter did something weird then it would be blocked on that
>> filter. What is happening is that the Processor is not dying as
>> requested.
>> That's why I think my patch ought to resolve the problem.
>>
>> On Wed, Oct 11, 2017 at 10:05 AM Emmanuel Lécharny 
>> wrote:
>>
>>>
>>> Le 11/10/2017 à 14:51, Christoph John a écrit :
>>>> Thanks, will try. Did this patch apply to 2.0.16? The line numbers do
>>>> not match. I have added it manually in line 1162 and removed the
>>>> statement from line 1164.
>>> It's for 2.0.17-SNAPSHOT, so you may have differences with 2.0.16, but
>>> minor.
>>>
>>> The idea here is to remove a session when it's i teh list of managed
>>> version, even if it's not valid (that might be a corner case when teh
>>> SSLFilter does not properly handle an exception and close the channel.
>>> Call it a workaround atm)
>>>
>>> -- 
>>> Emmanuel Lecharny
>>>
>>> Symas.com
>>> directory.apache.org
>>>
>>>
>

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

New release soon

[Emmanuel Lécharny]

Hi guys,


I'm going to cut MINA-2.0.17 in the coming days.


Just FYI.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [jira] [Commented] (DIRMINA-1060) Handle the spinning selectors in Socket/Datagram Acceptor and Connector

[Emmanuel Lécharny]

Le 27/10/2017 à 15:58, Jonathan Valliere a écrit :
> For connector, I would allow the error to flow all the way up the stack.

Currently, the excption is caught, a Thread.sleep(1000) is called, and
the ExceptionMonitor.exceptionCaught() method is called.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: SslFilter.exceptionCaught()

[Emmanuel Lécharny]

Le 15/12/2017 à 13:09, Christoph John a écrit :
> Hi,
> 
> I have a question regarding the method above. I have noticed that it
> swallows an Exception in some cases (WriteToClosedSessionException).
> Is there any harm in overriding the exceptionCaught() method in a
> subclass to get notified of the WriteToClosedSessionException? Or could
> this cause problems in the general SSL connection/disconnection process?

>From the top of my head, this message is sent when an exception has
occured, and it's propagated to the IoHandler would it want to do
somemething in this case.

Somehow, it does not make sense for it to be swallowed, for teh exact
reason the IoHandler might do something with it, so if it is, then it's
probably a bug.

Do you have a scenario that reproduces this beahavior ? (or at least the
places where the exception is swallowed) ?

Thanks !

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: SslFilter.exceptionCaught()

[Emmanuel Lécharny]

Le 15/12/2017 à 15:29, Christoph John a écrit :
> Hi,
> 
> actually, the only scenario where this caused "problems" was in a unit
> test. There we set up some sessions and expect them to fail with an
> Exception e.g. when the certificate is invalid. In the tests we install
> an SSL Exception Handler as last part of the chain to get notified of
> the SSLExceptions.
> However, sometimes the SSLException is not thrown and the tests
> failbecause of that.

Ok. The SsHandler is probably the worst piece of MINA code, and I wold
assume it needs some love here. If you have a reproductible test, that
could help (even if it only fails from time to time).

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: New release soon

[Emmanuel Lécharny]

Sorry, was distracted by many other tasks.

It' still on my agenda.


Le 11/01/2018 à 17:22, elijah baley a écrit :
> Any progress ?...
> 
> 
> 
> From: Emmanuel Lécharny 
> Sent: Thursday, October 26, 2017 3:56 PM
> To: dev
> Subject: New release soon
> 
> Hi guys,
> 
> 
> I'm going to cut MINA-2.0.17 in the coming days.
> 
> 
> Just FYI.
> 
> --
> Emmanuel Lecharny
> 
> Symas.com
> directory.apache.org
> 
> 

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: [VOTE] Release Apache Mina SSHD 1.7.0

[Emmanuel Lécharny]

Hi !

Sorry for the long delay... Being caught in end of year/new year things.

Built from package and from tag, N&L checked, my +1

Note that some dependencies could be bumped up.


Thanks !

Le 22/01/2018 à 07:50, Lyor Goldstein a écrit :
> Hi guys,
> 
> 
> I was wondering where we are on this since I don't see anyone else voting...
> 
> I vote +1
> 
> ...
> 
> Lyor G. (a.k.a. lgoldstein...)
> 
> 
> From: Guillaume Nodet 
> Sent: Thursday, January 11, 2018 8:59 PM
> To: dev@mina.apache.org
> Subject: [VOTE] Release Apache Mina SSHD 1.7.0
> 
> I've staged a candidate release for SSHD 1.7.0.
> 
> Staging repository:
>   https://repository.apache.org/content/repositories/orgapachemina-1033
> 
> Release Notes - MINA SSHD - Version 1.7.0
> 
> ** Bug
> * [SSHD-737] - "Invalid encoding: redundant leading 0s" when
> establishing session
> * [SSHD-738] - "BufferException: Underflow" warning is frequently
> reported
> * [SSHD-743] - Nio2Session sporadically leaks exceptions from nio2
> threads
> * [SSHD-754] - OOM in sending data for channel
> * [SSHD-755] - Nio2Connector leaks socket if address is unresolvable or
> unreachable
> * [SSHD-760] - Unable to read PKCS8 key files
> * [SSHD-761] - The agent forwarding channel is not closed at the end of
> the session
> * [SSHD-765] - SFTP client fails to retrieve a binary file using an
> InputStream
> * [SSHD-771] - SFTP server closes the connection when hmac-sha2-512 is
> used
> * [SSHD-774] - SshConfigFileReader cannot parse tab delimiter in config
> file
> * [SSHD-776] - SSHD local port forwarding close session unexpectedly
> * [SSHD-779] - Deadlock in ChannelOutputStream
> * [SSHD-785] - ChannelPipedInputStream attempts to read from client
> even for 0-byte reads.
> * [SSHD-786] - Clients can't authenticate after unexpected exception in
> Nio2Acceptor
> * [SSHD-793] - Continuously connecting to sshd server lead to memory
> leaks
> 
> ** Improvement
> * [SSHD-762] - Keyboard Interactive Authentication only supports
> one-time interaction
> * [SSHD-763] - Add support for reading ECDSA PUTTY key files
> * [SSHD-764] - Override default jUnit tests runner so as not to create
> a new instance for each executed test method
> * [SSHD-766] - Separate forwarding filter functionality according to
> sshd-config options
> * [SSHD-773] - Add support for ed25519 Putty key file
> * [SSHD-775] - SftpSubSystem::sendStatus leaks Exception information
> * [SSHD-777] - Make the code handling channels more error tolerant
> 
> ** New Feature
> * [SSHD-700] - SSHD does not suppot agent forwarding for XShell and
> XAgent
> * [SSHD-746] - IPv6 addresses are not supported as client sessions
> connection target
> * [SSHD-768] - Avoid unbounded message queueing when sending large
> amounts of data to slow clients
> * [SSHD-787] - Secure trasnfer file's directoy existence is alway
> checked (on real FS) no metter of any other settings
> * [SSHD-790] - SFTP client does not support filename encoding other
> than UTF-8.
> 
> ** Question
> * [SSHD-781] - The heartbeat task of client not support keepalive
> function well
> 
> 
> Please review and vote !
> 

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Fatal Error Handling

[Emmanuel Lécharny]

Le 11/02/2018 à 15:52, Jonathan Valliere a écrit :
> Hi Team,
> 
> What is the standard way in Mina of handing super fatal errors?  i.e.
> unrecoverable invalid internal state

Exception are trapped and probagated to the application through the
ExcetionCaught message, the session being closed in some specific case.
As we catch Exception, that covers RintimeException.

Errors aren't catched.

Do you have a bit more of context ?

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Fatal Error Handling

[Emmanuel Lécharny]

Le 11/02/2018 à 19:14, Jonathan Valliere a écrit :
> There is a condition in the IoProcessor where the expected Session Count
> turns negative. Basically the IoProcessor is now in a Fatal state. What
> should happen?

Uh oH...


The expected Session count should *never* become negative. If it's the
case, that needs to be fixed.

Do you have a clue about what can be the cause ?

Otherwise, everthing we would do to manage this situation would probably
be just a bad workaround...

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Pease welcome Jonathan Vallliere, who has been voted as a new MINA committer !

[Emmanuel Lécharny]

Hi !

I'm pleased to announce that Jonathan Valliere has been voted in this
week)end as a new committer of the MINA project !

Jonathan, here are a few pointers about what it means to be a cmmiter at
The Apache Foundation :

How it works : https://www.apache.org/foundation/how-it-works.html

Guide to new committers : https://www.apache.org/dev/contributors.html

You can manage your account, subscribe to e-mails,and a few more things
using https://whimsy.apache.org/

In any case, would you have any question feel free to post them on
dev@mina.apache.org.

Thanks and welcome !


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Linter / Code Formatting

[Emmanuel Lécharny]

Le 19/02/2018 à 16:47, Jonathan Valliere a écrit :
> Any interest in adding support for a linter to automatically check for
> formatting / quality issues and formatting the code to enforce consistent
> style across the entire codebase?

We can use Checkstyle for that purpose.

And, yes, that would be a good idea.

For the record, we are following teh standard java convention for the
formatting.


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Linter / Code Formatting

[Emmanuel Lécharny]

Le 19/02/2018 à 16:47, Jonathan Valliere a écrit :
> Any interest in adding support for a linter to automatically check for
> formatting / quality issues and formatting the code to enforce consistent
> style across the entire codebase?
> 

I would add twi things :

- months ago, I ran the code against Java 8 with the javadoc linter. I
spent days fixing all the missing Javadoc and incorrect ones. The
javadoc linter is on by default now on. It's possible to disbale it,
there is a commented property in the pom.xml file :

...


...

- Checkstyle is pretty good at verifying the code format. We don't
currently use it in MINA, but would we want to set it up, I'll suggest
we mimic what we did in teh Apache Directory project, where it's on by
default. It's based on a checkstyle configuration, and a plugin :

https://gitbox.apache.org/repos/asf?p=directory-buildtools.git;a=tree;f=checkstyle-configuration;h=67f80df6c18be403fcecd13464b6854bd31eb89e;hb=HEAD

https://gitbox.apache.org/repos/asf?p=directory-ldap-api.git;a=blob;f=pom.xml;h=3bfaec9c08bba75ff71b8810bbf092d101280d2e;hb=HEAD

with :


  org.apache.maven.plugins
  maven-checkstyle-plugin
  
true
false
  
  

  validate
  validate
  
check
  

  



That works pretty well.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: How to handle NetworkRecycledException in AbstractPollingIoAcceptor?

[Emmanuel Lécharny]

Le 20/02/2018 à 13:12, dgriff a écrit :
> Hi, on z/OS it is possible to stop and restart the underlying TCP/IP stack.
> When this is done, the JVM throws the following exception:
> 
> 
> 
> (This is with 2.0.1 but problem still appears to be in latest version)

Please provide the exeption and stack trace, it has not been added to
your mail.

Also be sure to test with 2.0.16.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Needs Immutable Iterator

[Emmanuel Lécharny]

Le 25/02/2018 à 00:59, Jonathan Valliere a écrit :
> Hi Everyone,
> 
> Working on a couple of bugs and needed an Immutable Iterator.  Is there an
> Immutable Iterator in any of the existing dependencies or the codebase of
> Mina that I should use instead of adding one.

We don't have any dependencies in mina-core. The idea was to make it as
easy as possible to embed it.

As suggested by Christopher, Collections.unmodifiableList( list
).iterator() is an option.


In which context do you need such a thing ?

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: Release Targets in JIRA

[Emmanuel Lécharny]

Le 26/02/2018 à 17:06, Jonathan Valliere a écrit :
> Just a thought…
> 
> At work, we always have a release target similar to 2.0-NEXT which always
> represents the next release.  When a new release is completed, the target
> is renamed to the correct version number and a new 2.0-NEXT release is
> created.  This style makes it easier to configure and schedule issues for
> the “next” release without having to figure out the exact name of the next
> release.

ATM, we are simply using the next version (because we are mainly in
maintenance mode, it makes sense), but I'm not against chaning that.


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Errors when buuilding MINA 2.0 branch

[Emmanuel Lécharny]

Hi,

I'm trying to build MINA 2.0 with the latest changes made by Jonathan,
and I get some random error in a test :

[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
11.378 s <<< FAILURE! - in org.apache.mina.example.echoserver.ConnectorTest
[ERROR] testTCPWithSSL(org.apache.mina.example.echoserver.ConnectorTest)
 Time elapsed: 11.079 s  <<< FAILURE!
java.lang.AssertionError: expected:<160> but was:<32>
at
org.apache.mina.example.echoserver.ConnectorTest.waitForResponse(ConnectorTest.java:222)
at
org.apache.mina.example.echoserver.ConnectorTest.testConnector0(ConnectorTest.java:192)
at
org.apache.mina.example.echoserver.ConnectorTest.testConnector(ConnectorTest.java:141)
at
org.apache.mina.example.echoserver.ConnectorTest.testConnector(ConnectorTest.java:103)
at
org.apache.mina.example.echoserver.ConnectorTest.testTCPWithSSL(ConnectorTest.java:90)



or


[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
11.528 s <<< FAILURE! - in org.apache.mina.example.echoserver.ConnectorTest
[ERROR] testTCPWithSSL(org.apache.mina.example.echoserver.ConnectorTest)
 Time elapsed: 11.226 s  <<< FAILURE!
java.lang.AssertionError: expected:<160> but was:<0>
at
org.apache.mina.example.echoserver.ConnectorTest.waitForResponse(ConnectorTest.java:222)
at
org.apache.mina.example.echoserver.ConnectorTest.testConnector0(ConnectorTest.java:192)
at
org.apache.mina.example.echoserver.ConnectorTest.testConnector(ConnectorTest.java:141)
at
org.apache.mina.example.echoserver.ConnectorTest.testConnector(ConnectorTest.java:106)
at
org.apache.mina.example.echoserver.ConnectorTest.testTCPWithSSL(ConnectorTest.java:90)



It's pretty random.


I'm going to investigate what's going on, but that was one of the reason
I wasn't able to cut the release last december...

If anyone has an idea...


-- 
Emmanuel Lecharny

Symas.com
directory.apache.org