subject:"\[jira\] \[Commented\] \(SSHD\-1070\) OutOfMemoryError when use port forwarding"

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-22 Thread Guillaume Nodet (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199891#comment-17199891
 ] 

Guillaume Nodet commented on SSHD-1070:
---

I've fixed the memory issue, but I've also disabled async mode by default.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-15 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17196129#comment-17196129
 ] 

Feng Jiajie commented on SSHD-1070:
---

[~gnodet] Thanks! I've raised a new issue. 
https://issues.apache.org/jira/browse/SSHD-1079

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-15 Thread Guillaume Nodet (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17196107#comment-17196107
 ] 

Guillaume Nodet commented on SSHD-1070:
---

[~fengjiajie] please raise a JIRA for async mode on the local forwarder.
At first glance, a simple patch like the following should work:
{code}
diff --git 
a/sshd-core/src/main/java/org/apache/sshd/common/forward/TcpipClientChannel.java
 
b/sshd-core/src/main/java/org/apache/sshd/common/forward/TcpipClientChannel.java
index ae780769..e22e1a57 100644
--- 
a/sshd-core/src/main/java/org/apache/sshd/common/forward/TcpipClientChannel.java
+++ 
b/sshd-core/src/main/java/org/apache/sshd/common/forward/TcpipClientChannel.java
@@ -33,8 +33,11 @@ import org.apache.sshd.common.Closeable;
 import org.apache.sshd.common.NamedResource;
 import org.apache.sshd.common.SshConstants;
 import org.apache.sshd.common.SshException;
+import org.apache.sshd.common.channel.ChannelAsyncInputStream;
+import org.apache.sshd.common.channel.ChannelAsyncOutputStream;
 import org.apache.sshd.common.channel.ChannelOutputStream;
 import org.apache.sshd.common.channel.Window;
+import org.apache.sshd.common.future.CloseFuture;
 import org.apache.sshd.common.io.IoSession;
 import org.apache.sshd.common.session.Session;
 import org.apache.sshd.common.util.ValidateUtils;
@@ -86,6 +89,7 @@ public class TcpipClientChannel extends AbstractClientChannel 
implements Forward
 this.localEntry = new SshdSocketAddress((InetSocketAddress) 
serverSession.getLocalAddress());
 this.remote = remote;
 this.messagesQueue = new ClientChannelPendingMessagesQueue(this);
+this.streaming = Streaming.Async;
 }

 public OpenFuture getOpenFuture() {
@@ -161,12 +165,24 @@ public class TcpipClientChannel extends 
AbstractClientChannel implements Forward
 @Override
 protected synchronized void doOpen() throws IOException {
 if (streaming == Streaming.Async) {
-throw new IllegalArgumentException("Asynchronous streaming isn't 
supported yet on this channel");
+asyncIn = new ChannelAsyncOutputStream(this, 
SshConstants.SSH_MSG_CHANNEL_DATA) {
+@SuppressWarnings("synthetic-access")
+@Override
+protected CloseFuture doCloseGracefully() {
+try {
+sendEof();
+} catch (IOException e) {
+getSession().exceptionCaught(e);
+}
+return super.doCloseGracefully();
+}
+};
+asyncOut = new ChannelAsyncInputStream(this);
+} else {
+out = new ChannelOutputStream(
+this, getRemoteWindow(), log, 
SshConstants.SSH_MSG_CHANNEL_DATA, true);
+invertedIn = out;
 }
-
-out = new ChannelOutputStream(
-this, getRemoteWindow(), log, 
SshConstants.SSH_MSG_CHANNEL_DATA, true);
-invertedIn = out;
 }

 @Override
{code}

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-14 Thread Guillaume Nodet (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195275#comment-17195275
 ] 

Guillaume Nodet commented on SSHD-1070:
---

I agree that a bounded queue is complicated, as what is important is the total 
size of messages, not their number, so the idea 
[https://github.com/apache/mina-sshd/pull/163] is a good one imho.

I've tried to come up with something more esy to integrate: 
[https://github.com/apache/mina-sshd/pull/166,] but it would need a 
reproducible test to actually verify the behavior.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-10 Thread Thomas Wolf (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17193695#comment-17193695
 ] 

Thomas Wolf commented on SSHD-1070:
---

[~lgoldstein], I don't think this can be left to be configured by the user. 
This is server-side. If someone implements a server, s/he cannot know how fast 
the two ends will be, and what kind of throttling might needed. Which is why I 
wrote on the PR I wouldn't know how to configure something like that. Perhaps 
s/he could figure it out at run-time and implement some super-clever adaptive 
scheme, but that probably means rewrite the whole forwarding mechanism.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-10 Thread Lyor Goldstein (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17193681#comment-17193681
 ] 

Lyor Goldstein commented on SSHD-1070:
--

No - you did not misunderstand - I was just wondering if we could introduce the 
use of {{ChannelStreamPacketWriterResolver}} here as well and let the user 
decide whether to implement a throttling mechanism. Anyway, I am not opposed to 
using a limited queue - but it should have configurable size and max. wait 
time. Note that "configuration" can be done per-session via a 
{{SessionListener}} - i.e., the user can decide how to configure these 
parameters per session (or can do it globally).

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$complet

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-09 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17193360#comment-17193360
 ] 

Feng Jiajie commented on SSHD-1070:
---

Hi [~lgoldstein],  I took a look at ChannelStreamPacketWriterResolver.

NIO constantly writes data to the queue(BufferedIoOutputStream.writes),
BufferedIoOutputStream.writes writes to ChannelAsyncOutputStream one by one 
(asynchronous mode).
Finally, the OOM is because the queue(BufferedIoOutputStream.writes) is too big.

So we need to set up blocking before queue (BufferedIoOutputStream.writes)?  
@Override resolveChannelStreamPacketWriter in TcpipServerChannel can setup a 
block in ChannelAsyncOutputStream downstream, but may not block read into 
BufferedIoOutputStream.writes. Or maybe I misunderstood.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the ses

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-09 Thread Lyor Goldstein (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192989#comment-17192989
 ] 

Lyor Goldstein commented on SSHD-1070:
--

{quote}
https://github.com/fengjiajie/mina-sshd/commit/7b3809a0a5030990dc9d3877ee95a272d643ce62
{quote}
Code seems rather complex + it is not really production ready (too many logs 
without information, {{System.exit}}, etc,). I rather like [PR 
#163|https://github.com/apache/mina-sshd/pull/163/] approach (though with some 
changes)

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.co

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-09 Thread Lyor Goldstein (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17193001#comment-17193001
 ] 

Lyor Goldstein commented on SSHD-1070:
--

Have you looked into using a {{ChannelStreamPacketWriterResolver}} ? We 
introduced it to allow users to apply their own traffic shaping policy. There 
is even an (experimental) {{ThrottlingPacketWriter}} example. Perhaps this 
approach can be extended to this case as well.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invo

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-08 Thread Thomas Wolf (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192639#comment-17192639
 ] 

Thomas Wolf commented on SSHD-1070:
---

{quote}
I think your implementation PR #163 has the minimal overall impact and is more 
suitable for a quick fix to the problem.
{quote}

I'm not sure. I actually closed/abandoned that PR after having tried your 
change. I did observe that with my change, I consistently did run into into 
SSHD-1069 with the iperf3 test you suggested here (with -P 8; my machine has 4 
cores and thus 5 NIO threads), but with yours, it just worked fine even with -P 
8, and the throughputs for -P 1 and -P 2 were identical for both approaches.

Also, if I understand your change right, it is specific for the forwarding 
case, whereas mine has the risk of affecting other uses of the 
{{BufferedIoOutputstream}}. But forwarding should be the only scenario where a 
server would encounter such speed differences between the two ends. (Or are 
there other cases?) So probably your approach is less risky.
 

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Ses

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-08 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192609#comment-17192609
 ] 

Feng Jiajie commented on SSHD-1070:
---

Hi [~wolft],
{quote}It also seems to solve SSHD-1069?
{quote}
No, but once the SSHD-1070 is resolved, the SSHD-1069 can be resolved in the 
same way as the SSHD-1003.

I've considered it again. I think your implementation [PR 
#163|https://github.com/apache/mina-sshd/pull/163/commits/37efbff75fed6eae374c7012277c433f002cd6d7]
 has the minimal overall impact and is more suitable for a quick fix to the 
problem.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Nati

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-08 Thread Thomas Wolf (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192396#comment-17192396
 ] 

Thomas Wolf commented on SSHD-1070:
---

That's rather nice. Indeed it works well. (y)

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)17:52:26.045
>  [sshd-SshServer[5bcea91b

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-08 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17192197#comment-17192197
 ] 

Feng Jiajie commented on SSHD-1070:
---

Hi [~wolft] [~lgoldstein]

I wrote a demo code,

[https://github.com/fengjiajie/mina-sshd/commit/7b3809a0a5030990dc9d3877ee95a272d643ce62]

Just tried the above idea and it works.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-07 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191631#comment-17191631
 ] 

Feng Jiajie commented on SSHD-1070:
---

Hi [~wolft] Thanks for the reply.

The whole story goes like that:
 # I need to use ssh -R (remote port forwarding), and then I met the problem 
described by SSHD-1069.
 # I found SSHD-1003 and SSHD-557, they solved the ssh -L (local port 
forwarding) problem (before SSHD-1003, ssh -L had the same problem, see 
SSHD-557).
 # SSHD-1003 modified TcpipServerChannel, So I modified the TcpipClientChannel 
in the same way to solve the -R problem. 
https://github.com/fengjiajie/mina-sshd/commit/a448cae1ca6b68a08b34e8d375fb56114561b144
 . The OOM occurred while testing the new code.
 # I think -L might have the same OOM problem, then I created SSHD-1070.

The whole thing started with resolving the SSHD-1069 deadlock, so when I see 
BlockingQueue, I have the above question.
{quote}I think it should not, since a {{put()}} that succeeds then executes in 
the same thread {{startWriting()}}, and {{startWriting()}} calls itself once a 
buffered element has been written to process the rest of the queue. So 
eventually there should be space again, and a pending {{put()}} should then 
succeed.
{quote}
I don't really have a good understanding of the SSHD processing process and 
present an idea that may not be correct：
Consider a scenario, if there is only one thread, and BlockingQueue max size is 
small, eg 1. startWriting doesn't actually output data, but instead calls nio's 
future write. If the next scheduling still reads the data instead of output 
data, the queue would be full. So when the maximum length of the queue is 
larger, the probability of deadlock is lower, but it's still possible?

I think maybe we could handle this situation closer to the network layer. For 
example, if it is found that it is not suitable to read data from Channel (e.g. 
the queue in the program is full), then do not read the data first. For 
example, if the *Nio2CompletionHandler handleReadCycleCompletion* finds that 
the queue is full, then *doReadCycle* is no longer called, and if the thread 
executing the output operation finds that doReadCycle was not called, then 
doReadCycle is called by that thread, blocking the data outside of the JVM.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
>

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-06 Thread Thomas Wolf (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191465#comment-17191465
 ] 

Thomas Wolf commented on SSHD-1070:
---

{quote}
1. Could this cause a deadlock?
{quote}
I think it should not, since a {{put()}} that succeeds then executes in the 
same thread {{startWriting()}}, and {{startWriting()}} calls itself once a 
buffered element has been written to process the rest of the queue. So 
eventually there should be space again, and a pending {{put()}} should then 
succeed.

What you describe, though, is exactly what happens in the SSHD-1069 case (all 
threads only reading, no thread is writing), but since that problem already 
existed before, it isn't caused by this change. I wonder if it happens because 
the {{ChannelAsyncOutputStream}} delays the write until there'd be enough 
window space; but if all threads are busy reading I don't see where a thread 
would request a window adjust and eventually retrigger the pending write.
{quote}
2. The performance of asynchronous frameworks will be degraded...
{quote}
Perhaps. But the whole process here doesn't seem to be very asynchronous 
anyway, with the same thread initiating the write. I didn't check in detail 
what the  {{ChannelAsyncOutputStream}} does ; I don't see it using different 
threads for writing... I would expect different connections to use different 
{{BufferedIoOutputStreams}} in any case. With a fixed thread pool for reading 
and writing it seems to me that it is always possible that a fast connection is 
slowed down by slower connections, or that a fast connection starves slower 
ones.

Adding yet another queue like you suggest will just re-introduce the problem of 
unbounded queues again, and then we can get OOMEs again.

But feel free to provide a better fix.

 

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchro

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-06 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191424#comment-17191424
 ] 

Feng Jiajie commented on SSHD-1070:
---

[~technobcn] Thanks for helping reproduce the issue.

[~wolft] Thanks for helping to fix the problem.
I think there maybe some problems with the commit: [PR 
#163|https://github.com/apache/mina-sshd/pull/163/commits/37efbff75fed6eae374c7012277c433f002cd6d7]
BlockingQueue is a blocking queue, whereas SSHD is based on an asynchronous 
framework, maybe this modification will cause blocking of asynchronous 
processes?
a. Could this cause a deadlock? For example: if all the working threads are 
blocking on BlockingQueue.put, the process would fail to provide service, 
because there would be no thread (all already blocking on BlockingQueue.put) to 
retrieve elements from BlockingQueues.
b. The performance of asynchronous frameworks will be degraded, and fast 
connections may be slower in scenarios where both fast and slow connections 
exist.
I don't know much about asynchronous frameworks, but I guess it's possible here 
to 
[offer|https://docs.oracle.com/javase/8/docs/api/java/util/Queue.html#offer-E-] 
elements to the queue (unlike the blocking put operation), and if the queue is 
full then immediately return the failure and put that "offer operation" into an 
operation queue and wait until the next time there's a free working thread to 
try to perform the operation?

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Ni

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-06 Thread Thomas Wolf (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191375#comment-17191375
 ] 

Thomas Wolf commented on SSHD-1070:
---

[PR #163|https://github.com/apache/mina-sshd/pull/163] implements this 
throttling. Seems to work fine in my tests using Feng's iperf3 test (with -P 1 
or -P 2; with -P 8 I run into SSHD-1069).

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:1

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-06 Thread Thomas Wolf (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191369#comment-17191369
 ] 

Thomas Wolf commented on SSHD-1070:
---

>From code inspection it looks like the queue in 
>{{org.apache.sshd.common.channel.BufferedIoOutputStream}} should be a 
>{{java.util.concurrent.LinkedBlockingQueue}} with a maximum capacity of a 
>certain number of elements (say 10; maybe configurable?), and the 
>{{BufferedIoOutputStream}} should use {{LinkedBlockingQueue.put()}} to enqueue 
>the write futures. Unless I'm mistaken that would throttle reading from the 
>fast producer to the speed of writing to the slower consumer?

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessControlle

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-06 Thread Guillermo Grandes (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17191233#comment-17191233
 ] 

Guillermo Grandes commented on SSHD-1070:
-

If it helps... I have reproduced the issue (OutOfMemoryError), running a simple 
http-server on machine "A", running sshd-code (client mode connected to 
openssh-server on same machine "A") using dynamic-socks-forward (-Xmx32m) on 
same machine "A"; and running "http-client" (firefox) on other machine "B", 
downloading a 2GB iso file, over slow link (wifi), after a while, crash. 
Therefore, if producer is faster than consumer, OOM.

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-01 Thread Feng Jiajie (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188957#comment-17188957
 ] 

Feng Jiajie commented on SSHD-1070:
---

Hi [~lgoldstein].
{quote}How large are we talking ?
{quote}
greater than JVM heap memory.

Here's another example for reproducing the problem.

Step1. start a nginx server, and put a large(4GB+) file in the html directory, 
then the file can be download using wget:
{code:java}
[root@debugboxcreate2204 nginx]# ll html/
total 4372592
-rw-r--r-- 1 work work537 May 22 19:31 50x.html
-rw-r--r-- 1 work work 4477522014 Sep  2 10:37 b
-rw-r--r-- 1 work work612 May 22 19:31 index.html
[root@debugboxcreate2204 nginx]#
[root@debugboxcreate2204 nginx]# wget 127.0.0.1:8080/b
{code}
Step2. start SSHD and ssh client:
{code:java}
java -Xmx2G TestSshd2

ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
0.0.0.0:15678:127.0.0.1:8080 test5@127.0.0.1
{code}
Step3. wget file through forwarding port:
{code:java}
wget 127.0.0.1:15678/b
{code}
Then OOM will occur in a few seconds:
{code:java}
11:34:33.631 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-2] WARN 
org.apache.sshd.common.channel.IoWriteFutureImpl - 
notifyListener(IoWriteFutureImpl[id=TcpipServerChannel[id=0, 
recipient=1]-ServerSessionImpl[test5@/127.0.0.1:54820][SSH_MSG_CHANNEL_DATA]][value=true])
 failed (OutOfMemoryError) to invoke 
org.apache.sshd.common.channel.BufferedIoOutputStream$1@4eb77084: Java heap 
space11:34:33.631 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-2] WARN 
org.apache.sshd.common.channel.IoWriteFutureImpl - 
notifyListener(IoWriteFutureImpl[id=TcpipServerChannel[id=0, 
recipient=1]-ServerSessionImpl[test5@/127.0.0.1:54820][SSH_MSG_CHANNEL_DATA]][value=true])
 failed (OutOfMemoryError) to invoke 
org.apache.sshd.common.channel.BufferedIoOutputStream$1@4eb77084: Java heap 
space11:34:33.772 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-2] WARN 
org.apache.sshd.common.io.nio2.Nio2DefaultIoWriteFuture - 
notifyListener(Nio2DefaultIoWriteFuture[id=/127.0.0.1:54820][value=true]) 
failed (OutOfMemoryError) to invoke 
org.apache.sshd.common.channel.ChannelAsyncOutputStream$$Lambda$86/673218654@3d3a7dd4:
 GC overhead limit exceeded11:34:34.207 
[sshd-SshServer[5bcea91b](port=12133)-nio2-thread-3] WARN 
org.apache.sshd.common.io.nio2.Nio2Session - 
exceptionCaught(Nio2Session[local=/127.0.0.1:26744, remote=/127.0.0.1:8106]) 
Exception handler threw OutOfMemoryError, closing the session: GC overhead 
limit exceededException in thread 
"sshd-SshServer[5bcea91b](port=12133)-nio2-thread-2" 
org.apache.sshd.common.RuntimeSshException11:34:35.525 
[sshd-SshServer[5bcea91b](port=12133)-nio2-thread-3] WARN 
org.apache.sshd.common.io.nio2.Nio2Session - 
exceptionCaught(Nio2Session[local=/127.0.0.1:26744, remote=/127.0.0.1:8106]) 
Exception handler threw OutOfMemoryError, closing the session: GC overhead 
limit exceeded at 
org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:411) 
at org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495)
{code}
I think the reason is:

nginx -> SSHD -> ssh -> wget

"nginx -> SSHD" speed about 1GB/s (Asynchronous)

"SSHD -> ssh" speed about 60MB/s (Data encryption and decryption)

after few second, SSHD would buffer GBs data in memory.

 

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingF

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

2020-09-01 Thread Lyor Goldstein (Jira)



[ 
https://issues.apache.org/jira/browse/SSHD-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188611#comment-17188611
 ] 

Lyor Goldstein commented on SSHD-1070:
--

{quote}
when a large amount of data is returned from upstream application
{quote}
How large are we talking ?

> OutOfMemoryError when use port forwarding
> -
>
> Key: SSHD-1070
> URL: https://issues.apache.org/jira/browse/SSHD-1070
> Project: MINA SSHD
>  Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Feng Jiajie
>Priority: Major
>
> Hi [~gnodet], I found this commit implemented asynchronous port forwarding:
> [https://github.com/apache/mina-sshd/commit/45f84aab59b2e11d72942cffe9d810e37ab64959#diff-33823b8546f71d77bb1d653358ecde70]
> However, when a large amount of data is returned from upstream application, 
> it is possible to cause an OOM in SSHD.
> Step1. SSHD server:
> {code:java}
> import org.apache.sshd.common.FactoryManager;
> import org.apache.sshd.common.PropertyResolverUtils;
> import org.apache.sshd.common.util.security.SecurityUtils;
> import org.apache.sshd.server.SshServer;
> import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
> import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;
> import java.io.IOException;
> import java.nio.file.Paths;
> public class TestSshd2 {
>   public static void main(String[] args) throws IOException, 
> InterruptedException {
> SecurityUtils.setAPrioriDisabledProvider("BC", true);
> SshServer sshd = SshServer.setUpDefaultServer();
> sshd.setPort(12133);
> sshd.setKeyPairProvider(new 
> SimpleGeneratorHostKeyProvider(Paths.get("/tmp/aa.key")));
> sshd.setPasswordAuthenticator((username, password, session) -> true);
> sshd.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
> sshd.start();
> Thread.sleep(1);
>   }
> }
> {code}
> Step2. start ssh client and iperf3 server: 
> {code:java}
> ssh -o 'ExitOnForwardFailure yes' -p 12133 -f -x -N -T -L 
> 0.0.0.0:15678:127.0.0.1:12345 test5@127.0.0.1
> iperf3 -s -p 12345
> {code}
> Step3. run iperf3 client: 
> {code:java}
> iperf3 -c 127.0.0.1  -i 1 -t 120 -p 15678 -P 8 --reverse
> {code}
> *-R, --reverse run in reverse mode (server sends, client receives)*
> SSHD will receive a lot of data but will not be able to forward it in time.
> log when OOM:
> {code:java}
> 17:52:25.195 [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded17:52:25.195 
> [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-1] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33524, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:382)
>  at 
> sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session.doWriteCycle(Nio2Session.java:420) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.startWriting(Nio2Session.java:404) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.finishWrite(Nio2Session.java:495) 
> at 
> org.apache.sshd.common.io.nio2.Nio2Session.handleCompletedWriteCycle(Nio2Session.java:465)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:429)
>  at 
> org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:426)17:52:25.639
>  [sshd-SshServer[5bcea91b](port=12133)-nio2-thread-6] WARN 
> org.apache.sshd.common.io.nio2.Nio2Session - 
> exceptionCaught(Nio2Session[local=/127.0.0.1:33530, remote=/127.0.0.1:12345]) 
> Exception handler threw OutOfMemoryError, closing the session: GC overhead 
> limit exceeded at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
>  at java.security.AccessController.doPrivileged(Native Method) at 
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
>  at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) at 
> sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) at 
> sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixAsynchronousSocketChannelImpl.java:736)17:52:26.045
>  [sshd-Ss

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

[jira] [Commented] (SSHD-1070) OutOfMemoryError when use port forwarding

21 matches

Site Navigation

Mail list logo

Footer information