[jira] [Commented] (DIRMINA-946) ByteBufferDumper fails to dump ByteBuffers that are not backed by an array or that are readonly
[ https://issues.apache.org/jira/browse/DIRMINA-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687813#comment-13687813 ] Raphaël P. Barazzutti commented on DIRMINA-946: --- Hi Emmanuel, Thanks for your comment, then I'll propose another patch this later today. ByteBufferDumper fails to dump ByteBuffers that are not backed by an array or that are readonly --- Key: DIRMINA-946 URL: https://issues.apache.org/jira/browse/DIRMINA-946 Project: MINA Issue Type: Bug Components: Core Affects Versions: 3.0.0-trunk Reporter: Raphaël P. Barazzutti Assignee: Julien Vermillard ByteBufferDumper does a call to array() of the ByteBuffer, it throws an exception if that last one isn't read/write and backed by an array. fix available here: https://github.com/rbarazzutti/mina/tree/bufferdumper-fix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (DIRMINA-946) ByteBufferDumper fails to dump ByteBuffers that are not backed by an array or that are readonly
[ https://issues.apache.org/jira/browse/DIRMINA-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raphaël P. Barazzutti closed DIRMINA-946. - Fixed with commit fad44fe. We can close this issue now. ByteBufferDumper fails to dump ByteBuffers that are not backed by an array or that are readonly --- Key: DIRMINA-946 URL: https://issues.apache.org/jira/browse/DIRMINA-946 Project: MINA Issue Type: Bug Components: Core Affects Versions: 3.0.0-trunk Reporter: Raphaël P. Barazzutti Assignee: Julien Vermillard ByteBufferDumper does a call to array() of the ByteBuffer, it throws an exception if that last one isn't read/write and backed by an array. fix available here: https://github.com/rbarazzutti/mina/tree/bufferdumper-fix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (DIRMINA-947) select git trunk as default branch on github
Raphaël P. Barazzutti created DIRMINA-947: - Summary: select git trunk as default branch on github Key: DIRMINA-947 URL: https://issues.apache.org/jira/browse/DIRMINA-947 Project: MINA Issue Type: Task Reporter: Raphaël P. Barazzutti Priority: Minor Currently the Apache MINA GitHub page presents the branch 1.0. It seems that github looks first for a branch called master, then it selects the first branch in alpha/numerical order. An admin can select trunk as the main branch from the following URL: https://github.com/apache/mina/settings Currently a visitor coming to the GitHub page sees a message announcing that the last commit was done almost one year ago if he doesn't find a way to look at the right branch. Cheers, Raphaël -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (DIRMINA-947) select git trunk as default branch on github
[ https://issues.apache.org/jira/browse/DIRMINA-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raphaël P. Barazzutti closed DIRMINA-947. - Resolution: Fixed Thanks Julien. I created an issue on INFRA. select git trunk as default branch on github Key: DIRMINA-947 URL: https://issues.apache.org/jira/browse/DIRMINA-947 Project: MINA Issue Type: Task Reporter: Raphaël P. Barazzutti Priority: Minor Labels: git Currently the Apache MINA GitHub page presents the branch 1.0. It seems that github looks first for a branch called master, then it selects the first branch in alpha/numerical order. An admin can select trunk as the main branch from the following URL: https://github.com/apache/mina/settings Currently a visitor coming to the GitHub page sees a message announcing that the last commit was done almost one year ago if he doesn't find a way to look at the right branch. Cheers, Raphaël -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (SSHD-259) Provide Base64 in sshd.util
Jean-Baptiste Onofré created SSHD-259: - Summary: Provide Base64 in sshd.util Key: SSHD-259 URL: https://issues.apache.org/jira/browse/SSHD-259 Project: MINA SSHD Issue Type: Improvement Reporter: Jean-Baptiste Onofré Fix For: 0.10.0 Since sshd 0.9.0, using JDK 1.7, the mina-core dependency is no more required. However, in order to parse .ssh/known_hosts file, the client has to use org.apache.mina.util.Base64. So, even if sshd has not the mina-core dependency, the sshd users has to depend to mina-core. It would be great if sshd itself provide org.apache.sshd.util.Base64. Like this, the users don't need mina-core at all (just sshd). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (SSHD-259) Provide Base64 in sshd.util
[ https://issues.apache.org/jira/browse/SSHD-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré updated SSHD-259: -- Attachment: SSHD-259.patch Provide Base64 in sshd.util --- Key: SSHD-259 URL: https://issues.apache.org/jira/browse/SSHD-259 Project: MINA SSHD Issue Type: Improvement Reporter: Jean-Baptiste Onofré Fix For: 0.10.0 Attachments: SSHD-259.patch Since sshd 0.9.0, using JDK 1.7, the mina-core dependency is no more required. However, in order to parse .ssh/known_hosts file, the client has to use org.apache.mina.util.Base64. So, even if sshd has not the mina-core dependency, the sshd users has to depend to mina-core. It would be great if sshd itself provide org.apache.sshd.util.Base64. Like this, the users don't need mina-core at all (just sshd). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (SSHD-332) Nio2 security
Gaël Lalire created SSHD-332: Summary: Nio2 security Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaël Lalire updated SSHD-332: - Attachment: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaël Lalire updated SSHD-332: - Attachment: securesshd.zip Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar, securesshd.zip Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047647#comment-14047647 ] Gaël Lalire commented on SSHD-332: -- I found a way to run sshd in secure env by using mina even in java 7 8 : sshd.setIoServiceFactoryFactory(new MinaServiceFactoryFactory()); However it can be interesting to know if Nio2 is misused or misimplemented. I attached a jar and its sources so you can reproduce the issue with below commands : java -jar securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar ssh -p 127.0.0.1 An exception should occurs on java side if Nio2 is available. Exception occurs at least with Oracle JDK8 on Mac OS X and OpenJDK7 on fedora. Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar, securesshd.zip Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055224#comment-14055224 ] Gaël Lalire commented on SSHD-332: -- My version is newer java version 1.8.0_05 Java(TM) SE Runtime Environment (build 1.8.0_05-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode) I found the commit which avoid all permissions for NIO2 handler (6 month ago) in openjdk http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/c4baa68f4e3a I think it is a JVM bug to use it for NIO2 handler. Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar, securesshd.zip Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaël Lalire reopened SSHD-332: -- Sorry I get an answer from Oracle and it is expected. So you may close as wont fix if you doesn't want to fix it but not at resolved. Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Assignee: Guillaume Nodet Fix For: 0.12.0 Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar, securesshd.zip Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056203#comment-14056203 ] Gaël Lalire commented on SSHD-332: -- I check your code and an AsynchronousChannelGroup is used. The associated ExecutorService is a fixed thread pool and should have normal permissions. Maybe JVM issue. Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Assignee: Guillaume Nodet Fix For: 0.12.0 Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar, securesshd.zip Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SSHD-332) Nio2 security
[ https://issues.apache.org/jira/browse/SSHD-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056214#comment-14056214 ] Gaël Lalire commented on SSHD-332: -- You added AccessController.doPrivileged that should be ok, thanks. Nio2 security --- Key: SSHD-332 URL: https://issues.apache.org/jira/browse/SSHD-332 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0 Environment: Oracle Java 8 Reporter: Gaël Lalire Assignee: Guillaume Nodet Fix For: 0.12.0 Attachments: securesshd-0.0.1-SNAPSHOT-jar-with-dependencies.jar, securesshd.zip Original Estimate: 96h Remaining Estimate: 96h I don't know if it is a JVM bug or normal behavior but a ProtectionDomain with no permission is associated with completionHandler thread by sun.misc.InnocuousThread class. As a result if a security manager is set all code in completionHandler has no permission (event if policy grants all permission). If the behavior of JVM is correct then you should add AccessController.doPrivileged() when entering completionHandler. You can also check if a SecurityManager is set and run without Nio2 as a quick fix. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (SSHD-338) Authentication with same krb5.keytab file works with OpenSSH and not with Apache MINA
Christian Dröscher created SSHD-338: --- Summary: Authentication with same krb5.keytab file works with OpenSSH and not with Apache MINA Key: SSHD-338 URL: https://issues.apache.org/jira/browse/SSHD-338 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0, 0.9.0 Environment: RedHat Enterprise Linux, RHEL 6.4 (x86_64), 2.6.32-358.6.2.el6.x86_64 build 23093132 Reporter: Christian Dröscher Priority: Minor In the krb5.keytab file I have four different key versions(KVNO) with the same host/** (aes256-cts-hmac-sha1-96) - principial. OpenSSH can handle this multiple entries and there are no authentication problems. But the authentication with the same config, fails with Apache MINA. I deleted the last two KVNO of the principial so that we have just two entries left. With just two different KVNOs for the same principial, MINA is working as expected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (SSHD-337) Authentication with same krb5.keytab file works with OpenSSH and not with Apache MINA
Christian Dröscher created SSHD-337: --- Summary: Authentication with same krb5.keytab file works with OpenSSH and not with Apache MINA Key: SSHD-337 URL: https://issues.apache.org/jira/browse/SSHD-337 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.11.0, 0.9.0 Environment: RedHat Enterprise Linux, RHEL 6.4 (x86_64), 2.6.32-358.6.2.el6.x86_64 build 23093132 Reporter: Christian Dröscher Priority: Minor In the krb5.keytab file I have four different key versions(KVNO) with the same host/** (aes256-cts-hmac-sha1-96) - principial. OpenSSH can handle this multiple entries and there are no authentication problems. But the authentication with the same config, fails with Apache MINA. I deleted the last two KVNO of the principial so that we have just two entries left. With just two different KVNOs for the same principial, MINA is working as expected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Closed] (SSHD-337) Authentication with same krb5.keytab file works with OpenSSH and not with Apache MINA
[ https://issues.apache.org/jira/browse/SSHD-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Dröscher closed SSHD-337. --- Resolution: Duplicate Authentication with same krb5.keytab file works with OpenSSH and not with Apache MINA - Key: SSHD-337 URL: https://issues.apache.org/jira/browse/SSHD-337 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.9.0, 0.11.0 Environment: RedHat Enterprise Linux, RHEL 6.4 (x86_64), 2.6.32-358.6.2.el6.x86_64 build 23093132 Reporter: Christian Dröscher Priority: Minor Labels: newbie In the krb5.keytab file I have four different key versions(KVNO) with the same host/** (aes256-cts-hmac-sha1-96) - principial. OpenSSH can handle this multiple entries and there are no authentication problems. But the authentication with the same config, fails with Apache MINA. I deleted the last two KVNO of the principial so that we have just two entries left. With just two different KVNOs for the same principial, MINA is working as expected. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135422#comment-14135422
]
Saša Živkov commented on SSHD-348:
--
Can you try to shutdown the clients if you reproduce the problem ?
There are hundreds of clients who are reading the stream-events and I am not
aware
of any possibility to find out which specific client(s) to shutdown. Shutting
down all
clients would be almost impossible to organize as they are owned and maintained
by
our development teams.
Is there any other way we could approach this issue?
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135557#comment-14135557
]
Hugo Arès commented on SSHD-348:
Can you try to shutdown the clients if you reproduce the problem ?
I have tried to reproduced that problem by slowing down client so the server
would wait for space, then I disconnected client (clean disconnect, killed
client) but all the times the server behaved as expected and saw that the
client disconnected and stopped waiting. I also tried with different ssh client
but I focused on JSCH 0.1.46 because on our production server all the stuck
thread were connected to a jsch client version 0.1.46 and older (JSCH is
library used by CI server to connect to Gerrit using persistant ssh connection).
I investigated the stuck thread on our production server and I found out that
it is waiting for space on a disconnected client which made me thought that the
issue was the server missing the disconnection but like I said, I did not find
a way to reproduce the problem.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135736#comment-14135736
]
Hugo Arès commented on SSHD-348:
I did that already but the problem is that the only place I can reproduce it in
my production server and I can only change the application during the
maintenance window (next one is on September 27-28). I will install my patched
version of Gerrit that revert sshd to 0.9.0.201311081, if the problem is not
fixed by then and let you know about my findings.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util
[jira] [Comment Edited] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135736#comment-14135736
]
Hugo Arès edited comment on SSHD-348 at 9/16/14 5:07 PM:
-
May be it worth trying to downgrade SSHD to
org.apache.sshd:sshd-core:0.9.0.201311081
I did that already but the problem is that the only place I can reproduce it in
my production server and I can only change the application during the
maintenance window (next one is on September 27-28). I will install my patched
version of Gerrit that revert sshd to 0.9.0.201311081, if the problem is not
fixed by then and let you know about my findings.
was (Author: hugares):
I did that already but the problem is that the only place I can reproduce it in
my production server and I can only change the application during the
maintenance window (next one is on September 27-28). I will install my patched
version of Gerrit that revert sshd to 0.9.0.201311081, if the problem is not
fixed by then and let you know about my findings.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137500#comment-14137500
]
Hugo Arès commented on SSHD-348:
More info: the thread is stuck waiting for space but the state of the session
is CLOSE and the state of the channel is CLOSE_RECV, is it normal that the
channel wait for space when its state is closed?
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run()V(ThreadPoolExecutor.java
[jira] [Comment Edited] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137534#comment-14137534
]
Hugo Arès edited comment on SSHD-348 at 9/17/14 5:36 PM:
-
Let me know if you need more info, I still have a stuck thread on my production
server so I can attach and give you the info you need.
was (Author: hugares):
Let me know if you need more info, I still have a stuck thread on my production
server so attach and give you the info you need.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143305#comment-14143305
]
Hugo Arès commented on SSHD-348:
here is the info:
remoteWindow.size: 0
remoteWindow.waiting: true
remoteWindow.closed: false
commandExitFuture.result: true
gracefulState.value: 2
gracefulFuture.result: null
state.value: 1
closeFuture.result: null
service.state.value: 3
service.closeFuture.result: true
session.state.value:3
session.closeFuture.result:true
command.done: did not find that variable, are you sure of the name?
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147548#comment-14147548
]
Saša Živkov commented on SSHD-348:
--
In our production Gerrit we have been now using the sshd 0.10.1 plus the ssh
handshake failure fix (2aed686bdb2)
cherry-picked.
After 2 days of Gerrit uptime there is no single SSH-Stream-Worker thread
blocked in the waitForSpace method.
This means that the issue was introduced somewhere between the 0.10.1 and 0.11
as we know that both 0.11 and 0.12
have this issue.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148919#comment-14148919
]
Saša Živkov commented on SSHD-348:
--
My previous report was too optimistic. Checked our production Gerrit and 7 out
of 10 threads where blocked in the waitForSpace method.
Means: 0.10.1 also contains this bug!
FYI, I built 0.10.1 with 2aed686bdb2 cherry-picked on top and with the
following changes in the pom.xml files:
$ git diff
diff --git a/assembly/pom.xml b/assembly/pom.xml
index 85af850..4800b97 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -24,12 +24,12 @@
parent
groupIdorg.apache.sshd/groupId
artifactIdsshd/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
/parent
groupIdorg.apache.sshd/groupId
artifactIdapache-sshd/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
nameApache Mina SSHD :: Assembly/name
packagingpom/packaging
diff --git a/pom.xml b/pom.xml
index 1d3fbbf..901bc59 100644
--- a/pom.xml
+++ b/pom.xml
@@ -29,7 +29,7 @@
groupIdorg.apache.sshd/groupId
artifactIdsshd/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
nameApache Mina SSHD/name
packagingpom/packaging
inceptionYear2008/inceptionYear
diff --git a/sshd-core/pom.xml b/sshd-core/pom.xml
index c190b20..0d3ed07 100644
--- a/sshd-core/pom.xml
+++ b/sshd-core/pom.xml
@@ -24,12 +24,12 @@
parent
groupIdorg.apache.sshd/groupId
artifactIdsshd/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
/parent
groupIdorg.apache.sshd/groupId
artifactIdsshd-core/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
nameApache Mina SSHD :: Core/name
packagingjar/packaging
inceptionYear2008/inceptionYear
diff --git a/sshd-git/pom.xml b/sshd-git/pom.xml
index 3dcdc4a..1123da8 100644
--- a/sshd-git/pom.xml
+++ b/sshd-git/pom.xml
@@ -24,12 +24,12 @@
parent
groupIdorg.apache.sshd/groupId
artifactIdsshd/artifactId
-version0.9.0-SNAPSHOT/version
+version0.10.1-2aed686bdb2/version
/parent
groupIdorg.apache.sshd/groupId
artifactIdsshd-git/artifactId
-version0.9.0-SNAPSHOT/version
+version0.10.1-2aed686bdb2/version
nameApache Mina SSHD :: Git/name
packagingjar/packaging
inceptionYear2008/inceptionYear
diff --git a/sshd-pam/pom.xml b/sshd-pam/pom.xml
index 78d38c1..71683f4 100644
--- a/sshd-pam/pom.xml
+++ b/sshd-pam/pom.xml
@@ -24,12 +24,12 @@
parent
groupIdorg.apache.sshd/groupId
artifactIdsshd/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
/parent
groupIdorg.apache.sshd/groupId
artifactIdsshd-pam/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
nameApache Mina SSHD :: PAM/name
!--
packagingbundle/packaging
diff --git a/sshd-sftp/pom.xml b/sshd-sftp/pom.xml
index 50590aa..c71b79f 100644
--- a/sshd-sftp/pom.xml
+++ b/sshd-sftp/pom.xml
@@ -24,12 +24,12 @@
parent
groupIdorg.apache.sshd/groupId
artifactIdsshd/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
/parent
groupIdorg.apache.sshd/groupId
artifactIdsshd-sftp/artifactId
-version0.10.1/version
+version0.10.1-2aed686bdb2/version
nameApache Mina SSHD :: SFTP/name
packagingjar/packaging
inceptionYear2008/inceptionYear
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream
[jira] [Comment Edited] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[ https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148919#comment-14148919 ] Saša Živkov edited comment on SSHD-348 at 9/26/14 8:44 AM: --- My previous report was too optimistic. Checked our production Gerrit and 7 out of 10 threads where blocked in the waitForSpace method. Means: 0.10.1 also contains this bug! FYI, I built 0.10.1 with 2aed686bdb2 cherry-picked on top and with the following changes in the pom.xml files (see the attached diff). was (Author: zivkov): My previous report was too optimistic. Checked our production Gerrit and 7 out of 10 threads where blocked in the waitForSpace method. Means: 0.10.1 also contains this bug! FYI, I built 0.10.1 with 2aed686bdb2 cherry-picked on top and with the following changes in the pom.xml files: $ git diff diff --git a/assembly/pom.xml b/assembly/pom.xml index 85af850..4800b97 100644 --- a/assembly/pom.xml +++ b/assembly/pom.xml @@ -24,12 +24,12 @@ parent groupIdorg.apache.sshd/groupId artifactIdsshd/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version /parent groupIdorg.apache.sshd/groupId artifactIdapache-sshd/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version nameApache Mina SSHD :: Assembly/name packagingpom/packaging diff --git a/pom.xml b/pom.xml index 1d3fbbf..901bc59 100644 --- a/pom.xml +++ b/pom.xml @@ -29,7 +29,7 @@ groupIdorg.apache.sshd/groupId artifactIdsshd/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version nameApache Mina SSHD/name packagingpom/packaging inceptionYear2008/inceptionYear diff --git a/sshd-core/pom.xml b/sshd-core/pom.xml index c190b20..0d3ed07 100644 --- a/sshd-core/pom.xml +++ b/sshd-core/pom.xml @@ -24,12 +24,12 @@ parent groupIdorg.apache.sshd/groupId artifactIdsshd/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version /parent groupIdorg.apache.sshd/groupId artifactIdsshd-core/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version nameApache Mina SSHD :: Core/name packagingjar/packaging inceptionYear2008/inceptionYear diff --git a/sshd-git/pom.xml b/sshd-git/pom.xml index 3dcdc4a..1123da8 100644 --- a/sshd-git/pom.xml +++ b/sshd-git/pom.xml @@ -24,12 +24,12 @@ parent groupIdorg.apache.sshd/groupId artifactIdsshd/artifactId -version0.9.0-SNAPSHOT/version +version0.10.1-2aed686bdb2/version /parent groupIdorg.apache.sshd/groupId artifactIdsshd-git/artifactId -version0.9.0-SNAPSHOT/version +version0.10.1-2aed686bdb2/version nameApache Mina SSHD :: Git/name packagingjar/packaging inceptionYear2008/inceptionYear diff --git a/sshd-pam/pom.xml b/sshd-pam/pom.xml index 78d38c1..71683f4 100644 --- a/sshd-pam/pom.xml +++ b/sshd-pam/pom.xml @@ -24,12 +24,12 @@ parent groupIdorg.apache.sshd/groupId artifactIdsshd/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version /parent groupIdorg.apache.sshd/groupId artifactIdsshd-pam/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version nameApache Mina SSHD :: PAM/name !-- packagingbundle/packaging diff --git a/sshd-sftp/pom.xml b/sshd-sftp/pom.xml index 50590aa..c71b79f 100644 --- a/sshd-sftp/pom.xml +++ b/sshd-sftp/pom.xml @@ -24,12 +24,12 @@ parent groupIdorg.apache.sshd/groupId artifactIdsshd/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version /parent groupIdorg.apache.sshd/groupId artifactIdsshd-sftp/artifactId -version0.10.1/version +version0.10.1-2aed686bdb2/version nameApache Mina SSHD :: SFTP/name packagingjar/packaging inceptionYear2008/inceptionYear Some SSH threads get blocked in Object.wait() method forever Key: SSHD-348 URL: https://issues.apache.org/jira/browse/SSHD-348 Project: MINA SSHD Issue Type: Bug Affects Versions: 0.12.0 Environment: Gerrit Code Review 2.9.1 Reporter: David Ostrovsky Attachments: diff This seems to be a regression compared to previous versions (0.6-0 and later). In Gerrit we have SSH commamds that returns immediately and so called stream-events command which keeps connection open until clients disconnect. Basically for days or weeks. This is used for example to inform CI system (jenkins) about events in gerrit, like new patch set upload etc. In Gerrit we have configurable SSH-Stream-Worker thread pool which is dedicated to the mentioned stream-events SSH command. The observed behaviour
[jira] [Updated] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saša Živkov updated SSHD-348:
-
Attachment: diff
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
Attachments: diff
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run()V(ThreadPoolExecutor.java:615)
at java.lang.Thread.run()V(Thread.java:812)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149136#comment-14149136
]
Hugo Arès commented on SSHD-348:
There are two commit that look suspicious:
8baa36e0e50bbd683dd7f4275d194c94504da4f0 [SSHD-228] Introduce a Closeable
interface and related utilities
18bcdf797dfba5e462413a6ccce9dd6d3da2025b [SSHD-228] Add Closeable to SshClient,
SshService, IoService and forward helpers
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
Attachments: diff
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V
[jira] [Comment Edited] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149136#comment-14149136
]
Hugo Arès edited comment on SSHD-348 at 9/26/14 1:30 PM:
-
There are two commits that look suspicious:
8baa36e0e50bbd683dd7f4275d194c94504da4f0 [SSHD-228] Introduce a Closeable
interface and related utilities
18bcdf797dfba5e462413a6ccce9dd6d3da2025b [SSHD-228] Add Closeable to SshClient,
SshService, IoService and forward helpers
was (Author: hugares):
There are two commit that look suspicious:
8baa36e0e50bbd683dd7f4275d194c94504da4f0 [SSHD-228] Introduce a Closeable
interface and related utilities
18bcdf797dfba5e462413a6ccce9dd6d3da2025b [SSHD-228] Add Closeable to SshClient,
SshService, IoService and forward helpers
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.12.0
Environment: Gerrit Code Review 2.9.1
Reporter: David Ostrovsky
Attachments: diff
This seems to be a regression compared to previous versions (0.6-0 and later).
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V
[jira] [Created] (SSHD-358) Create Builder and Factories for SShServer and SShClient so they can be extended properly
Paweł Skierczyński created SSHD-358: --- Summary: Create Builder and Factories for SShServer and SShClient so they can be extended properly Key: SSHD-358 URL: https://issues.apache.org/jira/browse/SSHD-358 Project: MINA SSHD Issue Type: Improvement Reporter: Paweł Skierczyński Priority: Minor I want to extend them and use default configuration. Right now it's impossible without copying all configuration code to my class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-358) Create Builder and Factories for SShServer and SShClient so they can be extended properly
[ https://issues.apache.org/jira/browse/SSHD-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paweł Skierczyński updated SSHD-358: Attachment: SSHD-358.patch I'm adding patch. It shouldn't change any behavior as it's mostly refactoring stuff. Also it adds classes but I believe that it shouldn't break previous api. It's compiling and mvn test succeded. Create Builder and Factories for SShServer and SShClient so they can be extended properly - Key: SSHD-358 URL: https://issues.apache.org/jira/browse/SSHD-358 Project: MINA SSHD Issue Type: Improvement Reporter: Paweł Skierczyński Priority: Minor Attachments: SSHD-358.patch I want to extend them and use default configuration. Right now it's impossible without copying all configuration code to my class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DIRMINA-990) Control flow over exceptional path in AbstractIoBuffer
Jörg Michelberger created DIRMINA-990:
-
Summary: Control flow over exceptional path in AbstractIoBuffer
Key: DIRMINA-990
URL: https://issues.apache.org/jira/browse/DIRMINA-990
Project: MINA
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.8
Reporter: Jörg Michelberger
There is control flow over exceptional path in
ObjectOutputStream.writeClassDescriptor() method used in
AbstractIoBuffer.putObject, so that serialization of primitive types is done
via exception path. As an other point, serialization of array types is done via
asking the classloader with a ClassForName call. Both could be a performance
issue. I append the original sourcce and a possible fix for this issue, which
hopefully hits all cases the inventor of the original code wants to hit. Is it
possible to get this in the upcomming 2.0.9 release of MINA?
Original code:
{code}
protected void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
try {
Class? clz = Class.forName(desc.getName());
if (!Serializable.class.isAssignableFrom(clz)) { //
NON-Serializable class
write(0);
super.writeClassDescriptor(desc);
} else { // Serializable class
write(1);
writeUTF(desc.getName());
}
} catch (ClassNotFoundException ex) { // Primitive types
write(0);
super.writeClassDescriptor(desc);
}
}
{code}
Possible fix
{code}
protected void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
if (desc.forClass().isArray() ||
desc.forClass().isPrimitive() ||
!Serializable.class.isAssignableFrom(desc.forClass())) {
write(0);
super.writeClassDescriptor(desc);
} else {
// Serializable class
write(1);
writeUTF(desc.getName());
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DIRMINA-991) Possible faster deserialization in AbstractIoBuffer object deserialization.
Jörg Michelberger created DIRMINA-991:
-
Summary: Possible faster deserialization in AbstractIoBuffer
object deserialization.
Key: DIRMINA-991
URL: https://issues.apache.org/jira/browse/DIRMINA-991
Project: MINA
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.8
Reporter: Jörg Michelberger
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is a
possibility to avoid duplicate call to Class.forName(). First call is done in
readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call of
desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming 2.0.9
release of MINA?
Original
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
}
{source}
Possible fix
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
if (null == desc.forClass()) { //this works for
serializable desc classes.
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
} else {
return desc.forClass();
}
}
{source}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DIRMINA-990) Control flow over exceptional path in AbstractIoBuffer
[
https://issues.apache.org/jira/browse/DIRMINA-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jörg Michelberger updated DIRMINA-990:
--
Description:
There is control flow over exceptional path in
ObjectOutputStream.writeClassDescriptor() method used in
AbstractIoBuffer.putObject, so that serialization of primitive types is done
via exception path. As an other point, serialization of array types is done via
asking the classloader with a ClassForName call. Both could be a performance
issue. I append the original source and a possible fix for this issue, which
hopefully hits all cases the inventor of the original code wants to hit. Is it
possible to get this in the upcomming 2.0.9 release of MINA?
Original code:
{code}
protected void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
try {
Class? clz = Class.forName(desc.getName());
if (!Serializable.class.isAssignableFrom(clz)) { //
NON-Serializable class
write(0);
super.writeClassDescriptor(desc);
} else { // Serializable class
write(1);
writeUTF(desc.getName());
}
} catch (ClassNotFoundException ex) { // Primitive types
write(0);
super.writeClassDescriptor(desc);
}
}
{code}
Possible fix
{code}
protected void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
if (desc.forClass().isArray() ||
desc.forClass().isPrimitive() ||
!Serializable.class.isAssignableFrom(desc.forClass())) {
write(0);
super.writeClassDescriptor(desc);
} else {
// Serializable class
write(1);
writeUTF(desc.getName());
}
}
{code}
was:
There is control flow over exceptional path in
ObjectOutputStream.writeClassDescriptor() method used in
AbstractIoBuffer.putObject, so that serialization of primitive types is done
via exception path. As an other point, serialization of array types is done via
asking the classloader with a ClassForName call. Both could be a performance
issue. I append the original sourcce and a possible fix for this issue, which
hopefully hits all cases the inventor of the original code wants to hit. Is it
possible to get this in the upcomming 2.0.9 release of MINA?
Original code:
{code}
protected void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
try {
Class? clz = Class.forName(desc.getName());
if (!Serializable.class.isAssignableFrom(clz)) { //
NON-Serializable class
write(0);
super.writeClassDescriptor(desc);
} else { // Serializable class
write(1);
writeUTF(desc.getName());
}
} catch (ClassNotFoundException ex) { // Primitive types
write(0);
super.writeClassDescriptor(desc);
}
}
{code}
Possible fix
{code}
protected void writeClassDescriptor(ObjectStreamClass desc)
throws IOException {
if (desc.forClass().isArray() ||
desc.forClass().isPrimitive() ||
!Serializable.class.isAssignableFrom(desc.forClass())) {
write(0);
super.writeClassDescriptor(desc);
} else {
// Serializable class
write(1);
writeUTF(desc.getName());
}
}
{code}
Control flow over exceptional path in AbstractIoBuffer
--
Key: DIRMINA-990
URL: https://issues.apache.org/jira/browse/DIRMINA-990
Project: MINA
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.8
Reporter: Jörg Michelberger
Labels: patch, performance
There is control flow over exceptional path in
ObjectOutputStream.writeClassDescriptor() method used in
AbstractIoBuffer.putObject, so that serialization of primitive types is done
via exception path. As an other point, serialization of array types is done
via asking the classloader with a ClassForName call. Both could be a
performance issue. I append
[jira] [Updated] (DIRMINA-991) Possible faster deserialization in AbstractIoBuffer object deserialization.
[
https://issues.apache.org/jira/browse/DIRMINA-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jörg Michelberger updated DIRMINA-991:
--
Description:
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is a
possibility to avoid duplicate call to Class.forName(). First call is done in
readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call of
desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming 2.0.9
release of MINA?
Original
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
{source}
Possible fix
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
if (null == desc.forClass()) { //this works for
serializable desc classes.
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
} else {
return desc.forClass();
}
}
{source}
was:
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is a
possibility to avoid duplicate call to Class.forName(). First call is done in
readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call of
desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming 2.0.9
release of MINA?
Original
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
}
{source}
Possible fix
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
if (null == desc.forClass()) { //this works for
serializable desc classes.
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
} else {
return desc.forClass();
}
}
{source}
Possible faster deserialization in AbstractIoBuffer object deserialization.
---
Key: DIRMINA-991
URL: https://issues.apache.org/jira/browse/DIRMINA-991
Project: MINA
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.8
Reporter: Jörg Michelberger
Labels: patch, performance
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is
a possibility to avoid duplicate call to Class.forName(). First call is done
in readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call
of desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming
2.0.9 release of MINA?
Original
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
{source}
Possible fix
{source}
protected
[jira] [Updated] (DIRMINA-991) Possible faster deserialization in AbstractIoBuffer object deserialization.
[
https://issues.apache.org/jira/browse/DIRMINA-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jörg Michelberger updated DIRMINA-991:
--
Description:
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is a
possibility to avoid duplicate call to Class.forName(). First call is done in
readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call of
desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming 2.0.9
release of MINA?
Original
{code}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
}
{code}
Possible fix
{code}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
if (null == desc.forClass()) { //this works for
serializable desc classes.
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
} else {
return desc.forClass();
}
}
{code}
was:
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is a
possibility to avoid duplicate call to Class.forName(). First call is done in
readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call of
desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming 2.0.9
release of MINA?
Original
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
{source}
Possible fix
{source}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
if (null == desc.forClass()) { //this works for
serializable desc classes.
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
} else {
return desc.forClass();
}
}
{source}
Possible faster deserialization in AbstractIoBuffer object deserialization.
---
Key: DIRMINA-991
URL: https://issues.apache.org/jira/browse/DIRMINA-991
Project: MINA
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.8
Reporter: Jörg Michelberger
Labels: patch, performance
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is
a possibility to avoid duplicate call to Class.forName(). First call is done
in readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call
of desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming
2.0.9 release of MINA?
Original
{code}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
}
{code}
Possible fix
{code}
protected Class? resolveClass
[jira] [Commented] (DIRMINA-991) Possible faster deserialization in AbstractIoBuffer object deserialization.
[
https://issues.apache.org/jira/browse/DIRMINA-991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176855#comment-14176855
]
Jörg Michelberger commented on DIRMINA-991:
---
Great, no duplicate desc.forClass() call. Looking forward to 2.0.10. :-)
Possible faster deserialization in AbstractIoBuffer object deserialization.
---
Key: DIRMINA-991
URL: https://issues.apache.org/jira/browse/DIRMINA-991
Project: MINA
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.8
Reporter: Jörg Michelberger
Labels: patch, performance
In ObjectInputStream.resolveClass() of AbstractIoBuffer.getObject() there is
a possibility to avoid duplicate call to Class.forName(). First call is done
in readClassDescriptor() and second in resolveClass() in case we deal with
Serializables, class descriptors are cached by the java platform, and a call
of desc.forClass() in resolveClass() returns a previous resolved class, which
allows skipping the ClassLoader call. I append the original source and a
possible fix for this issue. Is it possible to get this in the upcomming
2.0.9 release of MINA?
Original source :
{code}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
{code}
Possible fix :
{code}
protected Class? resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
if (null == desc.forClass()) { //this works for
serializable desc classes.
String name = desc.getName();
try {
return Class.forName(name, false, classLoader);
} catch (ClassNotFoundException ex) {
return super.resolveClass(desc);
}
} else {
return desc.forClass();
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SSHD-358) Create Builders for SShServer and SShClient so they can be extended properly
[ https://issues.apache.org/jira/browse/SSHD-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177001#comment-14177001 ] Paweł Skierczyński commented on SSHD-358: - Thank you for merging my patch, however after your changes I need 2 more tweaks... First I need builders to be public - otherwise I can't call their methods from the outside. Second I need serverKeyVerifier method in client builder. I'm attaching a patch for convenience. Create Builders for SShServer and SShClient so they can be extended properly Key: SSHD-358 URL: https://issues.apache.org/jira/browse/SSHD-358 Project: MINA SSHD Issue Type: New Feature Reporter: Paweł Skierczyński Assignee: Guillaume Nodet Priority: Minor Fix For: 0.13.0 Attachments: SSHD-358.patch I want to extend them and use default configuration. Right now it's impossible without copying all configuration code to my class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-358) Create Builders for SShServer and SShClient so they can be extended properly
[ https://issues.apache.org/jira/browse/SSHD-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paweł Skierczyński updated SSHD-358: Attachment: SSHD-358-2.patch Create Builders for SShServer and SShClient so they can be extended properly Key: SSHD-358 URL: https://issues.apache.org/jira/browse/SSHD-358 Project: MINA SSHD Issue Type: New Feature Reporter: Paweł Skierczyński Assignee: Guillaume Nodet Priority: Minor Fix For: 0.13.0 Attachments: SSHD-358-2.patch, SSHD-358.patch I want to extend them and use default configuration. Right now it's impossible without copying all configuration code to my class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (SSHD-358) Create Builders for SShServer and SShClient so they can be extended properly
[ https://issues.apache.org/jira/browse/SSHD-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paweł Skierczyński reopened SSHD-358: - Create Builders for SShServer and SShClient so they can be extended properly Key: SSHD-358 URL: https://issues.apache.org/jira/browse/SSHD-358 Project: MINA SSHD Issue Type: New Feature Reporter: Paweł Skierczyński Assignee: Guillaume Nodet Priority: Minor Fix For: 0.13.0 Attachments: SSHD-358-2.patch, SSHD-358.patch I want to extend them and use default configuration. Right now it's impossible without copying all configuration code to my class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-358) Create Builders for SShServer and SShClient so they can be extended properly
[ https://issues.apache.org/jira/browse/SSHD-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178208#comment-14178208 ] Paweł Skierczyński commented on SSHD-358: - wow, that was quick :) Create Builders for SShServer and SShClient so they can be extended properly Key: SSHD-358 URL: https://issues.apache.org/jira/browse/SSHD-358 Project: MINA SSHD Issue Type: New Feature Reporter: Paweł Skierczyński Assignee: Guillaume Nodet Priority: Minor Fix For: 0.13.0 Attachments: SSHD-358-2.patch, SSHD-358.patch I want to extend them and use default configuration. Right now it's impossible without copying all configuration code to my class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SSHD-368) When ssh server is slow in respond or some packages lost client in proxy has no timeout and may hang forever
Paweł Skierczyński created SSHD-368: --- Summary: When ssh server is slow in respond or some packages lost client in proxy has no timeout and may hang forever Key: SSHD-368 URL: https://issues.apache.org/jira/browse/SSHD-368 Project: MINA SSHD Issue Type: Improvement Reporter: Paweł Skierczyński I've found that client can globally timeout with global timeout. What I need is to client timeout if there is no communication for some time. If response lasts 1 hour but some bytes are coming now and then it's perfectly alright. My use case here is git checkout that can take a while with big git repository. I've found similar feature is already implemented for server but not for client. It may be that there is some socket configuration that will just do that, but I couldn't find it... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-368) When ssh server is slow in respond or some packages lost client in proxy has no timeout and may hang forever
[ https://issues.apache.org/jira/browse/SSHD-368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paweł Skierczyński updated SSHD-368: Attachment: SSHD-368.patch It moves feature from server to common and adjust client so it can use it. It also adds some status so it can be read later to figure out what happened. Tests are green though I didn't write any new one. When ssh server is slow in respond or some packages lost client in proxy has no timeout and may hang forever Key: SSHD-368 URL: https://issues.apache.org/jira/browse/SSHD-368 Project: MINA SSHD Issue Type: Improvement Reporter: Paweł Skierczyński Attachments: SSHD-368.patch I've found that client can globally timeout with global timeout. What I need is to client timeout if there is no communication for some time. If response lasts 1 hour but some bytes are coming now and then it's perfectly alright. My use case here is git checkout that can take a while with big git repository. I've found similar feature is already implemented for server but not for client. It may be that there is some socket configuration that will just do that, but I couldn't find it... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242763#comment-14242763
]
Hugo Arès commented on SSHD-348:
stream-events is a command: ssh -p29418 localhost gerrit stream-events
Every time I saw a thread stuck, the client was jsch.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242768#comment-14242768
]
Hugo Arès commented on SSHD-348:
More info on the stream-events command and why it's used:
stream-events command is used by anyone who wants to see in events hapenning on
the gerrit server in real time. The connection will not disconnect until the
client disconnect and every time an event is hapenning, client will receive the
event detail, one event per line.
The most common use case is for build tools to get notified when something
happens on Gerrit and trigger builds accordingly. The most common integration
(a least for our company) is Gerrit-Trigger, which is a Jenkins plugin.
Gerrit-Trigger is using jsch so this is why I only saw stuck threads with jsch
since almost all the stream-events command towards our Gerrit server are
initiated by Gerrit-Trigger.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243596#comment-14243596
]
Hugo Arès commented on SSHD-348:
Yes, this is correct. On my side I was not able to get the log of the jenkins
server since there are more than 50 jenkins servers connecting to Gerrit and I
cannot know wich one had a stuck connection. Admin of the jenkins is not likely
to notify a stuck connection because Gerrit-Trigger plugin reconnect
automatically.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314301#comment-14314301
]
Hugo Arès commented on SSHD-348:
I can give you the client side application but you will not be able to
reproduce the problem because it requires the server side (Gerrit) to have some
level of activity. I wrote a Gerrit plugin that simulates the load we have in
our production environment and to write that plugin, I logged one day of stream
events from our production server into a file a server and that plugin is
replaying those stream-events but I cannot give you that plugin because the
data it uses is confidential.
I will try to remove the confidential data from the gerrit plugin so I can give
it to you but in the meantime, I can run any test you want (test possible fix,
reproduce problem with more logging,...).
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff,
threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323073#comment-14323073
]
Hugo Arès commented on SSHD-348:
Here is the info, including command.done. I will try to isolate the server logs
that correspond to that connection (it took more that 24h to reproduce the
problem so I have around 25GB of logs):
remoteWindow.size 0
remoteWindow.waiting true
remoteWindow.closed false
commandExitFuture.result true
gracefulState.value CloseReceived
gracefulFuture.result null
state.value Graceful
closeFuture.result null
service.state.value Closed
service.closeFuture.result true
session.state.value Closed
session.closeFuture.result true
command.done false
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff,
threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327412#comment-14327412
]
Hugo Arès commented on SSHD-348:
So far so good, no stuck thread. I will let the test run for the whole week-end
to be sure, I will let you know Monday if the problem is still there.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch,
debugLogs.txt, diff, threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue
[jira] [Updated] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hugo Arès updated SSHD-348:
---
Attachment: debugLogs.txt
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch,
debugLogs.txt, diff, threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V(ThreadPoolExecutor.java:1145
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323288#comment-14323288
]
Hugo Arès commented on SSHD-348:
I uploaded a file called debugLogs.txt which contains logs about 3 stuck
threads. I isolated logs related to the channel and recipient that is stuck in
waitForSpace and I also filtered out the log Send SSH_MSG_CHANNEL_DATA on
channel 1 since it is flooding and I am not convinced it is helping to debug
this issue.
In 3 examples, we see the 2 following lines but never the one saying that the
channel is closed:
DEBUG org.apache.sshd.server.channel.ChannelSession : Closing
ChannelSession[id=x, recipient=x] gracefully
DEBUG org.apache.sshd.server.channel.ChannelSession : Wait 5000 ms for shell to
exit cleanly
In 2 examples (1 and 2), there is a stack trace but I do not know if it is
related to the problem so I included it.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch,
debugLogs.txt, diff, threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326140#comment-14326140
]
Hugo Arès commented on SSHD-348:
Sure, I will start the test today and keep you posted. Thanks for looking into
this issue.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch,
debugLogs.txt, diff, threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run()V(ScheduledThreadPoolExecutor.java:292)
at
com.google.gerrit.server.git.WorkQueue$Task.run()V(WorkQueue.java:364
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270925#comment-14270925
]
Saša Živkov commented on SSHD-348:
--
Unfortunately, on our productive Gerrit, where we use the 0.9.0 version or more
precisely the 0.9.x branch of mina-sshd,
the same issue occurred today. All threads blocked in the waitForSpace method.
The 0.9.0 version was long believed to be free of this issue. It doesn't seem
to be true... maybe the issue is just less
likely to happen on 0.9.0 than on a newer version.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266436#comment-14266436
]
Hugo Arès commented on SSHD-348:
I managed to reproduce the issue so I can give you the info you need. I wrote a
simple application that connects to gerrit using the same libraries that
Jenkins Gerrit-Trigger uses (gerrit-events which use jsch). This application
simulates clients that stay connected, some other that disconnects/connect
every 10 seconds and others that disconnects/connect when they did not receive
any events for 1 minutes.
I took a full thread dump and there are no other thread stuck other than the
ones stuck on Window.waitForSpace.
Here is the info you asked(I have 5 threads stuck on Window.waitForSpace and
all the values are the same):
remoteWindow.size 0
remoteWindow.waiting true
remoteWindow.closed false
commandExitFuture.result true
gracefulState.value CloseReceived
gracefulFuture.result null
state.value Graceful
closeFuture.result null
service.state.value Closed
service.closeFuture.result true
session.state.value Closed
session.closeFuture.result true
command.done (Are you sure about the name, I did not find it)
I looked on the client side and there is no errors. Is it possible that the
client disconnect and the server miss the disconnection event and then continue
to write to the client until the buffer is full?
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch, diff
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter
[jira] [Commented] (SSHD-348) Some SSH threads get blocked in Object.wait() method forever
[
https://issues.apache.org/jira/browse/SSHD-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334884#comment-14334884
]
Hugo Arès commented on SSHD-348:
Good news, I ran the test from Friday night to Monday morning, no stuck
threads. Just to be sure, I ran a test again last night without the fix and it
took less that 12 hours to get 3 stuck threads so I am pretty confident that
the problem is fixed.
Some SSH threads get blocked in Object.wait() method forever
Key: SSHD-348
URL: https://issues.apache.org/jira/browse/SSHD-348
Project: MINA SSHD
Issue Type: Bug
Affects Versions: 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0
Environment: Gerrit Code Review 2.9.1 (SSHD 0.12.0)
Gerrit Code Review 2.9.2 (SSHD 0.13.0)
Gerrit Code Review 2.9.3 (Downgraded to SSHD 0.9)
Reporter: David Ostrovsky
Assignee: Guillaume Nodet
Priority: Blocker
Fix For: 0.14.0
Attachments: 0001-Prepare-release-sshd-0.13.0-72f868e26.patch,
debugLogs.txt, diff, threaddump.txt
This seems to be a regression started from 0.10.1.
In Gerrit we have SSH commamds that returns immediately and so called
stream-events command which keeps connection open until clients disconnect.
Basically for days or weeks. This is used for example to inform CI system
(jenkins) about events in gerrit, like new patch set upload etc.
In Gerrit we have configurable SSH-Stream-Worker thread pool which is
dedicated to the mentioned stream-events SSH command. The observed behaviour
on latest SSHD release is that after some time all threads are stuck. They
aren't stuck at the same time, but one after another untill at some time all
threads are stuck and Gerrit must be restarted. Usually after one week. The
stack dump of all such stuck thread are the same, see below. If we had a
patch we could apply it to our production Gerrit instance and try if this
helps.
{code}
SSH-Stream-Worker-10 cpu=95400.00 [reset 95400.00] ms elapsed=146444.30
[reset 146444.30] s allocated=552670 B (5.15 GB) [reset 552670 B
(5.15 GB)] defined_classes=0
io= file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
[reset file i/o: 15622752/0 B, net i/o: 0/0 B, files opened:0, socks opened:0
]
prio=10 tid=0x7f54514df800 nid=0x1c71 / 7281 pthread-id=13281374976
in Object.wait() [_thread_blocked (_at_safepoint),
stack(0x7f541f5f6000,0x7f541f6f7000)] [0x7f541f6f5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(J)V(Native Method)
- waiting on 0x7f553aa530d0 (a
org.apache.sshd.common.channel.Window)
at java.lang.Object.wait()V(Object.java:503)
at
org.apache.sshd.common.channel.Window.waitForSpace()I(Window.java:148)
- locked 0x7f553aa530d0 (a org.apache.sshd.common.channel.Window)
at
org.apache.sshd.common.channel.ChannelOutputStream.flush()V(ChannelOutputStream.java:116)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at
org.apache.sshd.common.channel.ChannelOutputStream.write([BII)V(ChannelOutputStream.java:84)
- locked 0x7f553aa55060 (a
org.apache.sshd.common.channel.ChannelOutputStream)
at sun.nio.cs.StreamEncoder.writeBytes()V(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer()V(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush()V(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush()V(StreamEncoder.java:141)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush()V(OutputStreamWriter.java:229)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:254)
- locked 0x7f553aa7e258 (a java.io.OutputStreamWriter)
at java.io.PrintWriter.flush()V(PrintWriter.java:320)
- locked 0x7f553aa7e210 (a java.io.BufferedWriter)
at java.io.PrintWriter.checkError()Z(PrintWriter.java:357)
at
com.google.gerrit.sshd.commands.StreamEvents.writeEvents()V(StreamEvents.java:186)
at
com.google.gerrit.sshd.commands.StreamEvents.access$100(Lcom/google/gerrit/sshd/commands/StreamEvents;)V(StreamEvents.java:43)
at
com.google.gerrit.sshd.commands.StreamEvents$3.run()V(StreamEvents.java:82)
at
java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;(Executors.java:471)
at java.util.concurrent.FutureTask.run()V(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Ljava/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask;)V(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor
[jira] [Commented] (SSHD-553) Make sure all code can be compiled using JDK 7
[ https://issues.apache.org/jira/browse/SSHD-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702797#comment-14702797 ] Stefan Magnus Landrø commented on SSHD-553: --- How about setting up a build with travis? Make sure all code can be compiled using JDK 7 -- Key: SSHD-553 URL: https://issues.apache.org/jira/browse/SSHD-553 Project: MINA SSHD Issue Type: Bug Affects Versions: 1.0.0 Reporter: Goldstein Lyor Assignee: Goldstein Lyor Priority: Blocker Fix For: 1.0.0 Seems that some JDK 8 methods / constructors leaked into the code. Need to detect and fix them -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FTPSERVER-470) Support NLST according to RFC 1123
Andreas Schörk created FTPSERVER-470:
Summary: Support NLST according to RFC 1123
Key: FTPSERVER-470
URL: https://issues.apache.org/jira/browse/FTPSERVER-470
Project: FtpServer
Issue Type: Improvement
Reporter: Andreas Schörk
Hello,
Having problems using client code working with unix ftpservers on the apache
ftpserver I found FTPSERVER-287 and won't fix.
I know, rfc-1123 is not officially supported but the sentence:
{code}
The data returned by an NLST command MUST contain only a
simple list of legal pathnames, such that the server can use
them directly as the arguments of subsequent data transfer
commands for the individual files.
{code}
i.m.o.means, that the returned names must be usable as arguments for e.g. RETR
or DELE.
With the current implementation this works only if the listed files are in the
current working directory.
So I propose to reopen FTPSERVER-287 to support rfc-1123
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SSHD-611) Servers rejecting keyboard-interactive authentication not handled
Oliver Stöneberg created SSHD-611:
-
Summary: Servers rejecting keyboard-interactive authentication not
handled
Key: SSHD-611
URL: https://issues.apache.org/jira/browse/SSHD-611
Project: MINA SSHD
Issue Type: Bug
Reporter: Oliver Stöneberg
I am trying to communicate with a server that advertises keyboard-interactive
authentication but it fails with "Too many authentication failures". When the
client sends the request it gets a failure and requests it again and again
until it hits the maximum retries value. It also never reaches the
UserInteraction object that was assigned to the client. It seems when the
request fails it should move on to the next authentication method.
Here's the output of sshd-core:
DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (ClientUserAuthService.java:234)
tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting
method=keyboard-interactive
DEBUG [sshd-SshClient[48c40605]-nio2-thread-1]
(UserAuthKeyboardInteractive.java:110)
process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send
SSH_MSG_USERAUTH_REQUEST for keyboard-interactive
TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862)
encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 00
04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00
14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 00
00 00 00
DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing
100 bytes
DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) Finished
writing
DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84
bytes
TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003)
decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00
00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f
61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00
TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415)
doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process
SSH_MSG_USERAUTH_FAILURE
DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (ClientUserAuthService.java:181)
processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received
SSH_MSG_USERAUTH_FAILURE - partial=false,
methods=publickey,password,keyboard-interactive
Here's the putty output:
Outgoing packet #0x4, type 5 / 0x05 (SSH2_MSG_SERVICE_REQUEST)
00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth
Incoming packet #0x4, type 6 / 0x06 (SSH2_MSG_SERVICE_ACCEPT)
00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth
Outgoing packet #0x5, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST)
00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh-
0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 04 6e 6f connectionno
0020 6e 65ne
Incoming packet #0x5, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE)
00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa
0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard-
0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive.
Outgoing packet #0x6, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST)
00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh-
0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 14 6b 65 connectionke
0020 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 yboard-interacti
0030 76 65 00 00 00 00 00 00 00 00ve
Event Log: Attempting keyboard-interactive authentication
Incoming packet #0x6, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE)
00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa
0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard-
0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive.
Event Log: Server refused keyboard-interactive authentication
Outgoing packet #0x7, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST)
00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh-
0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 08 70 61 connectionpa
0020 73 73 77 6f 72 64 00 XX XX XX XX XX XX XX XX XX ssword.X
0030 XX XX XX XXX
Outgoing packet #0x8, type 2 / 0x02 (SSH2_MSG_IGNORE)
00 00 00 a0 dd aa 67 0a 8d 42 d0 2a 5c 82 1e 5e ..g..B.*\..^
0010 ef 3b 9f 2a c2 5d 71 8a 28 ff 5d ca 1f 28 94 20 .;.*.]q.(.]..(.
0020 ec f4 2d dd 34 dc cf 99 94 da c1 40 7d a4 d9 09 ..-.4..@}...
0030 0e 7c 15 f6 01 56 6b e8 a4 3c 45 a6 c9 bd 00 e3 .|...Vk..
[jira] [Created] (SSHD-610) Disconnect reason should be propagated as error message
Oliver Stöneberg created SSHD-610: - Summary: Disconnect reason should be propagated as error message Key: SSHD-610 URL: https://issues.apache.org/jira/browse/SSHD-610 Project: MINA SSHD Issue Type: Bug Reporter: Oliver Stöneberg Priority: Minor While connecting to a server I get the following error org.apache.sshd.common.SshException: Session is closed - at org.apache.sshd.client.session.ClientUserAuthService.preClose() in ClientUserAuthService.java:244. at org.apache.sshd.common.util.closeable.AbstractCloseable.close() in AbstractCloseable.java:68. at org.apache.sshd.common.util.closeable.ParallelCloseable.doClose() in ParallelCloseable.java:65. at org.apache.sshd.common.util.closeable.SimpleCloseable.close() in SimpleCloseable.java:52. at org.apache.sshd.common.util.closeable.SequentialCloseable$1.operationComplete() in SequentialCloseable.java:56. at org.apache.sshd.common.util.closeable.SequentialCloseable$1.operationComplete() in SequentialCloseable.java:46. at org.apache.sshd.common.util.closeable.SequentialCloseable.doClose() in SequentialCloseable.java:69. at org.apache.sshd.common.util.closeable.SimpleCloseable.close() in SimpleCloseable.java:52. at org.apache.sshd.common.util.closeable.AbstractInnerCloseable.doCloseImmediately() in AbstractInnerCloseable.java:47. at org.apache.sshd.common.util.closeable.AbstractCloseable.close() in AbstractCloseable.java:69. at org.apache.sshd.common.session.AbstractSession.handleDisconnect() in AbstractSession.java:498. at org.apache.sshd.common.session.AbstractSession.doHandleMessage() in AbstractSession.java:420. at org.apache.sshd.common.session.AbstractSession.handleMessage() in AbstractSession.java:394. at org.apache.sshd.client.session.ClientSessionImpl.handleMessage() in ClientSessionImpl.java:248. at org.apache.sshd.common.session.AbstractSession.decode() in AbstractSession.java:1010. at org.apache.sshd.common.session.AbstractSession.messageReceived() in AbstractSession.java:374. at org.apache.sshd.common.session.AbstractSessionIoHandler.messageReceived() in AbstractSessionIoHandler.java:59. at org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted() in Nio2Session.java:225. at org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted() in Nio2Session.java:217. at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$1.run() in Nio2CompletionHandler.java:37. at java.security.AccessController.doPrivileged() in AccessController.java:-2. at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed() in Nio2CompletionHandler.java:34. at sun.nio.ch.Invoker.invokeUnchecked() in Invoker.java:126. at sun.nio.ch.Invoker$2.run() in Invoker.java:218. at sun.nio.ch.AsynchronousChannelGroupImpl$1.run() in AsynchronousChannelGroupImpl.java:112. at java.util.concurrent.ThreadPoolExecutor.runWorker() in ThreadPoolExecutor.java:1142. at java.util.concurrent.ThreadPoolExecutor$Worker.run() in ThreadPoolExecutor.java:617. at java.lang.Thread.run() in Thread.java:745. The actual error can be found in the log: DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (ClientUserAuthService.java:234) tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting method=keyboard-interactive DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (UserAuthKeyboardInteractive.java:110) process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send SSH_MSG_USERAUTH_REQUEST for keyboard-interactive TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862) encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 00 00 00 00 DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing 100 bytes DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) Finished writing DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84 bytes TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003) decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415) doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process SSH_MSG_USERAUTH_FAILURE DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (ClientUserAuthService.java:181) processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received SSH_MSG_USERAUTH_FAILURE - partial=false, methods=publickey,password,keyboard-interactive repeats a few time TRACE [sshd-SshClient[48c40605]-nio2-thread-6] (AbstractSession.java:415) doHandleMessage(ClientSessionImpl[root
[jira] [Commented] (SSHD-611) Servers rejecting keyboard-interactive authentication not handled
[ https://issues.apache.org/jira/browse/SSHD-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046920#comment-15046920 ] Oliver Stöneberg commented on SSHD-611: --- I set up the SshClient using SshClient.setupDefaultClient() and at the very end I mentioned that I am using "28faad4 of master". In case you meant the server version - it happened with two different servers so far: - busybox with "SSH-2.0-OpenSSH_7.1" - OpenBSD 5.1 with "SSH-2.0-OpenSSH_6.0" > Servers rejecting keyboard-interactive authentication not handled > - > > Key: SSHD-611 > URL: https://issues.apache.org/jira/browse/SSHD-611 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > > I am trying to communicate with a server that advertises keyboard-interactive > authentication but it fails with "Too many authentication failures". When the > client sends the request it gets a failure and requests it again and again > until it hits the maximum retries value. It also never reaches the > UserInteraction object that was assigned to the client. It seems when the > request fails it should move on to the next authentication method. > Here's the output of sshd-core: > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (ClientUserAuthService.java:234) > tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting > method=keyboard-interactive > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (UserAuthKeyboardInteractive.java:110) > process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send > SSH_MSG_USERAUTH_REQUEST for keyboard-interactive > TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862) > encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 > 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 > 00 14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 > 00 00 00 00 > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing > 100 bytes > DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) > Finished writing > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84 > bytes > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003) > decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00 > 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f > 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415) > doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process > SSH_MSG_USERAUTH_FAILURE > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] > (ClientUserAuthService.java:181) > processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received > SSH_MSG_USERAUTH_FAILURE - partial=false, > methods=publickey,password,keyboard-interactive > Here's the putty output: > Outgoing packet #0x4, type 5 / 0x05 (SSH2_MSG_SERVICE_REQUEST) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Incoming packet #0x4, type 6 / 0x06 (SSH2_MSG_SERVICE_ACCEPT) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Outgoing packet #0x5, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 04 6e 6f connectionno > 0020 6e 65ne > Incoming packet #0x5, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. > Outgoing packet #0x6, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 14 6b 65 connectionke > 0020 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 yboard-interacti > 0030 76 65 00 00 00 00 00 00 00 00ve > Event Log: Attempting keyboard-interactive authentication > Incoming packet #0x6, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. &
[jira] [Comment Edited] (SSHD-611) Servers rejecting keyboard-interactive authentication not handled
[ https://issues.apache.org/jira/browse/SSHD-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048462#comment-15048462 ] Oliver Stöneberg edited comment on SSHD-611 at 12/9/15 10:48 AM: - I attached the log but it doesn't seem the logging you added made any different. The problem is, that the SSH_MSG_USERAUTH_REQUEST prodcues a SSH_MSG_USERAUTH_FAILURE response leading it to always entering the "buffer == null" block in org.apache.sshd.client.auth.UserAuthKeyboardInteractive#process(). It never moves on since it requires a SSH_MSG_USERAUTH_REQUEST response for that. was (Author: firewave): DEBUG output when trying to connect to a server that rejects keyboard-interactive authentication > Servers rejecting keyboard-interactive authentication not handled > - > > Key: SSHD-611 > URL: https://issues.apache.org/jira/browse/SSHD-611 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: sshd-core_keyboard-interactive_rejection.txt > > > I am trying to communicate with a server that advertises keyboard-interactive > authentication but it fails with "Too many authentication failures". When the > client sends the request it gets a failure and requests it again and again > until it hits the maximum retries value. It also never reaches the > UserInteraction object that was assigned to the client. It seems when the > request fails it should move on to the next authentication method. > Here's the output of sshd-core: > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (ClientUserAuthService.java:234) > tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting > method=keyboard-interactive > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (UserAuthKeyboardInteractive.java:110) > process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send > SSH_MSG_USERAUTH_REQUEST for keyboard-interactive > TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862) > encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 > 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 > 00 14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 > 00 00 00 00 > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing > 100 bytes > DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) > Finished writing > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84 > bytes > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003) > decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00 > 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f > 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415) > doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process > SSH_MSG_USERAUTH_FAILURE > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] > (ClientUserAuthService.java:181) > processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received > SSH_MSG_USERAUTH_FAILURE - partial=false, > methods=publickey,password,keyboard-interactive > Here's the putty output: > Outgoing packet #0x4, type 5 / 0x05 (SSH2_MSG_SERVICE_REQUEST) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Incoming packet #0x4, type 6 / 0x06 (SSH2_MSG_SERVICE_ACCEPT) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Outgoing packet #0x5, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 04 6e 6f connectionno > 0020 6e 65ne > Incoming packet #0x5, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. > Outgoing packet #0x6, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 14 6b 65 connectionke > 0020 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 yboard-interacti > 0030 76 65 00 00 00 00 00 00 00 00ve > Event Log: Attempting keyboard-interactive authent
[jira] [Updated] (SSHD-611) Servers rejecting keyboard-interactive authentication not handled
[ https://issues.apache.org/jira/browse/SSHD-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-611: -- Attachment: putty_core_keyboard-interactive_rejection.log I added the putty log for one of the servers. It seems if I encountered a failure it just moves on to the next authentication method. I checked RFC 4252 section 5.1 and it says The client MAY send several authentication requests without waiting for responses from previous requests. The server MUST process each request completely and acknowledge any failed requests with a SSH_MSG_USERAUTH_FAILURE message before processing the next request. So it seems fine to just move on if you get SSH_MSG_USERAUTH_FAILURE. Checking RFC 4256 section 3.1 I don't see any mention that has has to send a subsequent info request after failure. It's either one or the other judging from The server MUST reply with an SSH_MSG_USERAUTH_SUCCESS, SSH_MSG_USERAUTH_FAILURE, or SSH_MSG_USERAUTH_INFO_REQUEST message. An info would only happen if it actually wants to do keyboard-interactive in that case. > Servers rejecting keyboard-interactive authentication not handled > - > > Key: SSHD-611 > URL: https://issues.apache.org/jira/browse/SSHD-611 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: putty_core_keyboard-interactive_rejection.log, > sshd-core_keyboard-interactive_rejection.txt > > > I am trying to communicate with a server that advertises keyboard-interactive > authentication but it fails with "Too many authentication failures". When the > client sends the request it gets a failure and requests it again and again > until it hits the maximum retries value. It also never reaches the > UserInteraction object that was assigned to the client. It seems when the > request fails it should move on to the next authentication method. > Here's the output of sshd-core: > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (ClientUserAuthService.java:234) > tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting > method=keyboard-interactive > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (UserAuthKeyboardInteractive.java:110) > process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send > SSH_MSG_USERAUTH_REQUEST for keyboard-interactive > TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862) > encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 > 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 > 00 14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 > 00 00 00 00 > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing > 100 bytes > DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) > Finished writing > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84 > bytes > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003) > decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00 > 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f > 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415) > doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process > SSH_MSG_USERAUTH_FAILURE > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] > (ClientUserAuthService.java:181) > processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received > SSH_MSG_USERAUTH_FAILURE - partial=false, > methods=publickey,password,keyboard-interactive > Here's the putty output: > Outgoing packet #0x4, type 5 / 0x05 (SSH2_MSG_SERVICE_REQUEST) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Incoming packet #0x4, type 6 / 0x06 (SSH2_MSG_SERVICE_ACCEPT) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Outgoing packet #0x5, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 04 6e 6f connectionno > 0020 6e 65ne > Incoming packet #0x5, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. > Outgoing packet #0x6, type 50 / 0x32 (S
[jira] [Commented] (SSHD-611) Client incorrectly handles rejected keyboard-interactive authentication by server
[ https://issues.apache.org/jira/browse/SSHD-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057611#comment-15057611 ] Oliver Stöneberg commented on SSHD-611: --- Great! Thanks! > Client incorrectly handles rejected keyboard-interactive authentication by > server > - > > Key: SSHD-611 > URL: https://issues.apache.org/jira/browse/SSHD-611 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Fix For: 1.1.0 > > Attachments: putty_core_keyboard-interactive_rejection.log, > sshd-core_keyboard-interactive_rejection.txt > > > I am trying to communicate with a server that advertises keyboard-interactive > authentication but it fails with "Too many authentication failures". When the > client sends the request it gets a failure and requests it again and again > until it hits the maximum retries value. It also never reaches the > UserInteraction object that was assigned to the client. It seems when the > request fails it should move on to the next authentication method. > Here's the output of sshd-core: > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (ClientUserAuthService.java:234) > tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting > method=keyboard-interactive > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (UserAuthKeyboardInteractive.java:110) > process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send > SSH_MSG_USERAUTH_REQUEST for keyboard-interactive > TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862) > encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 > 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 > 00 14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 > 00 00 00 00 > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing > 100 bytes > DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) > Finished writing > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84 > bytes > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003) > decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00 > 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f > 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415) > doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process > SSH_MSG_USERAUTH_FAILURE > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] > (ClientUserAuthService.java:181) > processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received > SSH_MSG_USERAUTH_FAILURE - partial=false, > methods=publickey,password,keyboard-interactive > Here's the putty output: > Outgoing packet #0x4, type 5 / 0x05 (SSH2_MSG_SERVICE_REQUEST) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Incoming packet #0x4, type 6 / 0x06 (SSH2_MSG_SERVICE_ACCEPT) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Outgoing packet #0x5, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 04 6e 6f connectionno > 0020 6e 65ne > Incoming packet #0x5, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. > Outgoing packet #0x6, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 14 6b 65 connectionke > 0020 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 yboard-interacti > 0030 76 65 00 00 00 00 00 00 00 00ve > Event Log: Attempting keyboard-interactive authentication > Incoming packet #0x6, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. > Event Log: Server refused keyboard-interactive authentication
[jira] [Created] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
Oliver Stöneberg created SSHD-614: - Summary: SCP upload fails with dropbear server (aka server with paket size limitation) Key: SSHD-614 URL: https://issues.apache.org/jira/browse/SSHD-614 Project: MINA SSHD Issue Type: Bug Reporter: Oliver Stöneberg Attachments: dropbear_not_working.log, openssh_working.log When trying to upload a file via SCP to a server running dropbear I get the following error: java.io.EOFException: readAck - EOF before ACK - at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. at org.apache.sshd.client.scp.DefaultScpClient.upload() in DefaultScpClient.java:104. at org.apache.sshd.client.scp.AbstractScpClient.upload() in AbstractScpClient.java:201. The main difference between dropbear and OpenSSH I have encountered is that dropbear has a paket size limitation in place by default. That's the reason in the first place why I even need to upload files via SCP since using trying to execute bigger scripts via an exec channel it will fail. I attached logs from the system that fails and another system. The main difference seems to be this: working (OpenSSH): DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractConnectionService.java:236) channelOpenConfirmation(ChannelExec[id=0, recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) setRecipient(ChannelExec[id=0, recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) recipient=0 DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: client remote window) size=0, max.=0, packet=32768 DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) doOpen(ChannelExec[id=0, recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh not working (dropbear): DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractConnectionService.java:236) channelOpenConfirmation(ChannelExec[id=0, recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) setRecipient(ChannelExec[id=0, recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: client remote window) size=24576, max.=24576, packet=32759 DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) doOpen(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh As you can see that's a min/max set for the not working dropbear. There's also two typos in this log message (the additional whitespace at "client local" and the period at "max": DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) size=2097152, max.=2097152, packet=32768 I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-614: -- Attachment: openssh_working.log dropbear_not_working.log > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-611) Client incorrectly handles rejected keyboard-interactive authentication by server
[ https://issues.apache.org/jira/browse/SSHD-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055947#comment-15055947 ] Oliver Stöneberg commented on SSHD-611: --- The latest master is working for me. Thanks. I am not sure though if raising the priority of the "password" over the "keyboards-interactive" authentication method is a good change. > Client incorrectly handles rejected keyboard-interactive authentication by > server > - > > Key: SSHD-611 > URL: https://issues.apache.org/jira/browse/SSHD-611 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Fix For: 1.1.0 > > Attachments: putty_core_keyboard-interactive_rejection.log, > sshd-core_keyboard-interactive_rejection.txt > > > I am trying to communicate with a server that advertises keyboard-interactive > authentication but it fails with "Too many authentication failures". When the > client sends the request it gets a failure and requests it again and again > until it hits the maximum retries value. It also never reaches the > UserInteraction object that was assigned to the client. It seems when the > request fails it should move on to the next authentication method. > Here's the output of sshd-core: > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (ClientUserAuthService.java:234) > tryNext(ClientSessionImpl[root@/10.48.43.215:22]) attempting > method=keyboard-interactive > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] > (UserAuthKeyboardInteractive.java:110) > process(root@ClientSessionImpl[root@/10.48.43.215:22])[ssh-connection] Send > SSH_MSG_USERAUTH_REQUEST for keyboard-interactive > TRACE [sshd-SshClient[48c40605]-nio2-thread-1] (AbstractSession.java:862) > encode(ClientSessionImpl[root@/10.48.43.215:22]) Sending packet #5: 32 00 00 > 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 > 00 14 6b 65 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 00 00 00 > 00 00 00 00 > DEBUG [sshd-SshClient[48c40605]-nio2-thread-1] (Nio2Session.java:114) Writing > 100 bytes > DEBUG [sshd-SshClient[48c40605]-nio2-thread-4] (Nio2Session.java:274) > Finished writing > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] (Nio2Session.java:223) Read 84 > bytes > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:1003) > decode(ClientSessionImpl[root@/10.48.43.215:22]) Received packet #6: 33 00 00 > 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 73 73 77 6f 72 64 2c 6b 65 79 62 6f > 61 72 64 2d 69 6e 74 65 72 61 63 74 69 76 65 00 > TRACE [sshd-SshClient[48c40605]-nio2-thread-5] (AbstractSession.java:415) > doHandleMessage(ClientSessionImpl[root@/10.48.43.215:22]) process > SSH_MSG_USERAUTH_FAILURE > DEBUG [sshd-SshClient[48c40605]-nio2-thread-5] > (ClientUserAuthService.java:181) > processUserAuth(ClientSessionImpl[root@/10.48.43.215:22]) Received > SSH_MSG_USERAUTH_FAILURE - partial=false, > methods=publickey,password,keyboard-interactive > Here's the putty output: > Outgoing packet #0x4, type 5 / 0x05 (SSH2_MSG_SERVICE_REQUEST) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Incoming packet #0x4, type 6 / 0x06 (SSH2_MSG_SERVICE_ACCEPT) > 00 00 00 0c 73 73 68 2d 75 73 65 72 61 75 74 68 ssh-userauth > Outgoing packet #0x5, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 04 6e 6f connectionno > 0020 6e 65ne > Incoming packet #0x5, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 72 64 2c 6b 65 79 62 6f 61 72 64 2d ssword,keyboard- > 0020 69 6e 74 65 72 61 63 74 69 76 65 00 interactive. > Outgoing packet #0x6, type 50 / 0x32 (SSH2_MSG_USERAUTH_REQUEST) > 00 00 00 04 72 6f 6f 74 00 00 00 0e 73 73 68 2d rootssh- > 0010 63 6f 6e 6e 65 63 74 69 6f 6e 00 00 00 14 6b 65 connectionke > 0020 79 62 6f 61 72 64 2d 69 6e 74 65 72 61 63 74 69 yboard-interacti > 0030 76 65 00 00 00 00 00 00 00 00ve > Event Log: Attempting keyboard-interactive authentication > Incoming packet #0x6, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE) > 00 00 00 27 70 75 62 6c 69 63 6b 65 79 2c 70 61 ...'publickey,pa > 0010 73 73 77 6f 7
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057925#comment-15057925 ] Oliver Stöneberg commented on SSHD-614: --- No. This library is supposed to replace another SSH library which is used via JNI at the moment and there's only SCP implemented at the moment so using SFTP is not an option. > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-614: -- Comment: was deleted (was: I meant that I can't use the ScpCommandMain application since it doesn't seem to have password authentication implemented. I don't see it anywhere in the source. It just seems to support keys. I will look into it again later today. Also the main SCP code of ScpCommandMain looks very much the same as our code (we just don't use ScpLocation).) > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082803#comment-15082803 ] Oliver Stöneberg commented on SSHD-614: --- I meant that I can't use the ScpCommandMain application since it doesn't seem to have password authentication implemented. I don't see it anywhere in the source. It just seems to support keys. I will look into it again later today. Also the main SCP code of ScpCommandMain looks very much the same as our code (we just don't use ScpLocation). > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082805#comment-15082805 ] Oliver Stöneberg commented on SSHD-614: --- I meant that I can't use the ScpCommandMain application since it doesn't seem to have password authentication implemented. I don't see it anywhere in the source. It just seems to support keys. I will look into it again later today. Also the main SCP code of ScpCommandMain looks very much the same as our code (we just don't use ScpLocation). > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Attachment: sftp.log > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp.log, sftp_hang.txt > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085465#comment-15085465 ] Oliver Stöneberg commented on SSHD-621: --- I also tried psftp by PuTTY and sftp of an ubuntu installation and it also fails with those: sftp (added full output as attachment): debug1: channel 0: new [client-session] debug1: Entering interactive session. debug1: Sending environment. debug1: Sending env LANG = en_US.UTF-8 debug1: Sending subsystem: sftp debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 sh: /usr/libexec/sftp-server: not found debug1: channel 0: free: client-session, nchannels 1 debug1: fd 0 clearing O_NONBLOCK Transferred: sent 2480, received 2048 bytes, in 0.0 seconds Bytes per second: sent 66418.5, received 54848.9 debug1: Exit status 127 Connection closed psftp: sh: /usr/libexec/sftp-server: not found Fatal: Received unexpected end-of-file from SFTP server So the server doesn't support SFTP but they both give a different error message. Still it shouldn't hang. > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp.log, sftp_hang.txt > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Summary: ClientSession.createSftpClient() doesn't return when server doesn't support SFTP (was: ClientSession.createSftpClient() doesn't return when server doesn) > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Attachment: sftp_hang.txt > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: sftp_hang.txt > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn
Oliver Stöneberg created SSHD-621: - Summary: ClientSession.createSftpClient() doesn't return when server doesn Key: SSHD-621 URL: https://issues.apache.org/jira/browse/SSHD-621 Project: MINA SSHD Issue Type: Bug Reporter: Oliver Stöneberg [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] onClose(ChannelSubsystem[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before version negotiated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083073#comment-15083073 ] Oliver Stöneberg commented on SSHD-614: --- Thanks a lot. It seems something is up with that server though. I tried to connect with WinSCP (I though I did that) and it does connect but has problems getting the file listing afterwards. I also tried to implement SFTP as a fallback but that just brought up another issue for me - files that as SSHD-621. > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083212#comment-15083212 ] Oliver Stöneberg commented on SSHD-621: --- It seems it doesn't support SFTP. I tried WinSCP to connect and it just goes back to the dialog that lets you select. > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp_hang.txt > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Description: I am trying to connect to a server via SFTP and the ClientSession.createSftpClient() call hangs. I see the following warning which doesn't seem to be propagated: [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] onClose(ChannelSubsystem[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before version negotiated I tried to connect to server with another SFTP client and that client can't connect either. was: I am trying to connect to a server via SFTP and the [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] onClose(ChannelSubsystem[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before version negotiated > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085472#comment-15085472 ] Oliver Stöneberg commented on SSHD-614: --- Okay, that file listing issue is expected. So it works with WinSCP. I also tried pscp by PuTTY and scp on ubuntu and both of them work. I am not sure if the log is actually helpful, but I am attaching it anyways. > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Attachment: (was: sftp.log) > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp_hang.txt, sftp_vvv.log > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Attachment: sftp_vvv.log > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp_hang.txt, sftp_vvv.log > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-614: -- Attachment: scp_vvv.log > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log, > scp_vvv.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Attachment: sftp_vvv.log > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp_hang.txt, sftp_vvv.log > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-621) ClientSession.createSftpClient() doesn't return when server doesn't support SFTP
[ https://issues.apache.org/jira/browse/SSHD-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-621: -- Attachment: (was: sftp_vvv.log) > ClientSession.createSftpClient() doesn't return when server doesn't support > SFTP > > > Key: SSHD-621 > URL: https://issues.apache.org/jira/browse/SSHD-621 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Assignee: Goldstein Lyor > Attachments: sftp_hang.txt, sftp_vvv.log > > > I am trying to connect to a server via SFTP and the > ClientSession.createSftpClient() call hangs. I see the following warning > which doesn't seem to be propagated: > [org.apache.sshd.client.subsystem.sftp.DefaultSftpClient] > onClose(ChannelSubsystem[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44][sftp]) closed before > version negotiated > I tried to connect to server with another SFTP client and that client can't > connect either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085783#comment-15085783 ] Oliver Stöneberg commented on SSHD-614: --- I think I figured out what the issue is. The problem is that no commands are available on the target system (I didn't set it up) so when it tries to run "scp -p -t -- /tmp/X_script.sh" it will fail since "scp" is not available on the system. Using pscp you are actually getting the same error as I do with the library: sh: scp: not found Fatal: Received unexpected end-of-file from server The only thing missing is the error by sh that it didn't find the command. So removing the scp command from the target system would do the trick to reproduce this "issue". > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log, > scp_vvv.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-614) SCP upload fails with dropbear server (aka server with paket size limitation)
[ https://issues.apache.org/jira/browse/SSHD-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067749#comment-15067749 ] Oliver Stöneberg commented on SSHD-614: --- I can't get it to work. I aways get org.apache.sshd.common.SshException: No more authentication methods available Looking at the code it seems like password authentication is not yet implemented. Unfortunately I have no key file for the server I am testing against and I cannot create one. > SCP upload fails with dropbear server (aka server with paket size limitation) > - > > Key: SSHD-614 > URL: https://issues.apache.org/jira/browse/SSHD-614 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg >Priority: Minor > Attachments: dropbear_not_working.log, openssh_working.log > > > When trying to upload a file via SCP to a server running dropbear I get the > following error: > java.io.EOFException: readAck - EOF before ACK - > at org.apache.sshd.common.scp.ScpHelper.readAck() in ScpHelper.java:703. > at org.apache.sshd.common.scp.ScpHelper.sendStream() in ScpHelper.java:539. > at org.apache.sshd.client.scp.DefaultScpClient.upload() in > DefaultScpClient.java:104. > at org.apache.sshd.client.scp.AbstractScpClient.upload() in > AbstractScpClient.java:201. > The main difference between dropbear and OpenSSH I have encountered is that > dropbear has a paket size limitation in place by default. That's the reason > in the first place why I even need to upload files via SCP since using trying > to execute bigger scripts via an exec channel it will fail. I attached logs > from the system that fails and another system. The main difference seems to > be this: > working (OpenSSH): > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > Received SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > recipient=0 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (Window.java:122) > init(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]: > client remote window) size=0, max.=0, packet=32768 > DEBUG [sshd-SshClient[51a8313b]-nio2-thread-9] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[ssht...@x..xxx/XX.XX.XX.XXX:22]) > send SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > not working (dropbear): > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] > (AbstractConnectionService.java:236) > channelOpenConfirmation(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) Received > SSH_MSG_CHANNEL_OPEN_CONFIRMATION recipient= > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (AbstractChannel.java:121) > setRecipient(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) recipient=0 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (Window.java:122) > init(ChannelExec[id=0, recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]: > client remote window) size=24576, max.=24576, packet=32759 > DEBUG [sshd-SshClient[11389053]-nio2-thread-7] (ChannelExec.java:47) > doOpen(ChannelExec[id=0, > recipient=0]-ClientSessionImpl[test@/XX.XX.XX.XXX:44]) send > SSH_MSG_CHANNEL_REQUEST exec command=scp -p -t -- /tmp/X_script.sh > As you can see that's a min/max set for the not working dropbear. > There's also two typos in this log message (the additional whitespace at > "client local" and the period at "max": > DEBUG [forceDeviceActionAsync] (Window.java:122) init(ChannelExec[id=0, > recipient=-1]-ClientSessionImpl[test@/10.48.43.214:44]: client local window) > size=2097152, max.=2097152, packet=32768 > I am using the latest version of master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SSHD-600) Actual authentication error is just a warning in the log
Oliver Stöneberg created SSHD-600: - Summary: Actual authentication error is just a warning in the log Key: SSHD-600 URL: https://issues.apache.org/jira/browse/SSHD-600 Project: MINA SSHD Issue Type: Bug Affects Versions: 1.0.0 Reporter: Oliver Stöneberg Priority: Minor I was getting the following exception on various systems when calling auth().verify() on a ClientSession object. org.apache.sshd.common.SshException: Session is closed - at org.apache.sshd.client.session.ClientUserAuthService.preClose() in ClientUserAuthService.java:225. at org.apache.sshd.common.util.CloseableUtils$AbstractCloseable.close() in CloseableUtils.java:346. at org.apache.sshd.common.util.CloseableUtils$ParallelCloseable.doClose() in CloseableUtils.java:235. at org.apache.sshd.common.util.CloseableUtils$SimpleCloseable.close() in CloseableUtils.java:202. at org.apache.sshd.common.util.CloseableUtils$SequentialCloseable$1.operationComplete() in CloseableUtils.java:260. at org.apache.sshd.common.util.CloseableUtils$SequentialCloseable$1.operationComplete() in CloseableUtils.java:254. at org.apache.sshd.common.util.CloseableUtils$SequentialCloseable.doClose() in CloseableUtils.java:270. at org.apache.sshd.common.util.CloseableUtils$SimpleCloseable.close() in CloseableUtils.java:202. at org.apache.sshd.common.util.CloseableUtils$AbstractInnerCloseable.doCloseImmediately() in CloseableUtils.java:441. at org.apache.sshd.common.session.AbstractSession.doCloseImmediately() in AbstractSession.java:538. at org.apache.sshd.common.util.CloseableUtils$AbstractCloseable.close() in CloseableUtils.java:347. at org.apache.sshd.common.session.AbstractSession.exceptionCaught() in AbstractSession.java:525. at org.apache.sshd.common.session.AbstractSessionIoHandler.exceptionCaught() in AbstractSessionIoHandler.java:49. at org.apache.sshd.common.io.nio2.Nio2Session.exceptionCaught() in Nio2Session.java:137. at org.apache.sshd.common.io.nio2.Nio2Session.access$500() in Nio2Session.java:46. at org.apache.sshd.common.io.nio2.Nio2Session$2.onFailed() in Nio2Session.java:240. at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$2.run() in Nio2CompletionHandler.java:45. at java.security.AccessController.doPrivileged() in AccessController.java:-2. at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.failed() in Nio2CompletionHandler.java:42. at org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted() in Nio2Session.java:233. at org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted() in Nio2Session.java:212. at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$1.run() in Nio2CompletionHandler.java:34. at java.security.AccessController.doPrivileged() in AccessController.java:-2. at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed() in Nio2CompletionHandler.java:31. at sun.nio.ch.Invoker.invokeUnchecked() in Invoker.java:126. at sun.nio.ch.Invoker$2.run() in Invoker.java:218. at sun.nio.ch.AsynchronousChannelGroupImpl$1.run() in AsynchronousChannelGroupImpl.java:112. at java.util.concurrent.ThreadPoolExecutor.runWorker() in ThreadPoolExecutor.java:1142. at java.util.concurrent.ThreadPoolExecutor$Worker.run() in ThreadPoolExecutor.java:617. After enabling all log levels it turned out the actual error is a different one and is only logged as a warning: java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 2048 (inclusive) at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DHKeyPairGenerator.java:120) at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:674) at java.security.KeyPairGenerator.initialize(KeyPairGenerator.java:411) at org.apache.sshd.common.kex.DHG.getE(DHG.java:66) at org.apache.sshd.client.kex.DHGEXClient.next(DHGEXClient.java:110) at org.apache.sshd.common.session.AbstractSession.doHandleMessage(AbstractSession.java:395) at org.apache.sshd.common.session.AbstractSession.handleMessage(AbstractSession.java:349) at org.apache.sshd.client.session.ClientSessionImpl.handleMessage(ClientSessionImpl.java:487) at org.apache.sshd.common.session.AbstractSession.decode(AbstractSession.java:848) at org.apache.sshd.common.session.AbstractSession.messageReceived(AbstractSession.java:331) at org.apache.sshd.common.session.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:57) at org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:220) at org.apache.sshd.common.io.nio2.Nio2Session$2.onCompleted(Nio2Session.java:212) at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$1.run(Nio2CompletionHandler.java:34) at java.security.AccessController.doPrivileged(Native Method
[jira] [Commented] (SSHD-635) potential hang in AuthFuture.verify(Long.MAX_VALUE) on error
[ https://issues.apache.org/jira/browse/SSHD-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132255#comment-15132255 ] Oliver Stöneberg commented on SSHD-635: --- I have some tests I run against a server and if I execute those multi-threaded then the issue occurs very often. That's what I meant with reproducible. Unfortunately this didn't fix it. Some other verify() calls now get another error message but this still isn't working. I will try to debug it and get back to you soon - hopefully with more information. > potential hang in AuthFuture.verify(Long.MAX_VALUE) on error > > > Key: SSHD-635 > URL: https://issues.apache.org/jira/browse/SSHD-635 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: ssh.log > > > It appears that after the SshClient.connect().verify() call was successful > the ClientSession.auth().verify() call might hang if the connection was > closed in-between those calls. If AuthFuture.verify() is called with a > timeout it correctly times out, but I would expect it to detect that the > connection is closed/closing and throw a proper error even without the > timeout. > The errors I see in the log are these (more completel log attached): > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:137) Caught > IOException[An established connection was aborted by the software in your > host machine] - calling handler > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:142) > Exception handler threw IllegalStateException, closing the session: No > session available > It's hard to debug for me since it only happens in a multi-threaded scenario > and I haven't been able to generate separate logs per session (is this even > possible?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SSHD-626) ssh scp command is broken on mina backend
[
https://issues.apache.org/jira/browse/SSHD-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144675#comment-15144675
]
Hugo Arès edited comment on SSHD-626 at 2/12/16 3:10 PM:
-
I bisected and found the commit causing this issue:
commit 0d86de5384857e29b5a6f9e679fcdbf2e5b32f51
Author: Lyor Goldstein <...>
Date: Thu Jul 16 16:42:28 2015 +0300
[SSHD-541] Re-use incoming request buffer for outgoing response (where
applicable)
was (Author: hugares):
I bisected and found the commit causing this issue:
commit 0d86de5384857e29b5a6f9e679fcdbf2e5b32f51
Author: Lyor Goldstein <lgoldst...@vmware.com>
Date: Thu Jul 16 16:42:28 2015 +0300
[SSHD-541] Re-use incoming request buffer for outgoing response (where
applicable)
> ssh scp command is broken on mina backend
> -
>
> Key: SSHD-626
> URL: https://issues.apache.org/jira/browse/SSHD-626
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 1.0.0
> Environment: Gerrit Code Review
>Reporter: David Ostrovsky
>
> After upgrade of sshd-core to 1.0.0 and mina to 2.10
> ssh scp command doesn't work with mina backend.
> * Upgrade change: https://gerrit-review.googlesource.com/73033
> * Debug server trace: http://paste.openstack.org/show/484740
> * Debug client trace: http://paste.openstack.org/show/484741
> Workaround: use nio2 backend in gerrit.config:
> {code}
> [sshd]
> backend = nio2
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SSHD-626) ssh scp command is broken on mina backend
[
https://issues.apache.org/jira/browse/SSHD-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15144675#comment-15144675
]
Hugo Arès commented on SSHD-626:
I bisected and found the commit causing this issue:
commit 0d86de5384857e29b5a6f9e679fcdbf2e5b32f51
Author: Lyor Goldstein <lgoldst...@vmware.com>
Date: Thu Jul 16 16:42:28 2015 +0300
[SSHD-541] Re-use incoming request buffer for outgoing response (where
applicable)
> ssh scp command is broken on mina backend
> -
>
> Key: SSHD-626
> URL: https://issues.apache.org/jira/browse/SSHD-626
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 1.0.0
> Environment: Gerrit Code Review
>Reporter: David Ostrovsky
>
> After upgrade of sshd-core to 1.0.0 and mina to 2.10
> ssh scp command doesn't work with mina backend.
> * Upgrade change: https://gerrit-review.googlesource.com/73033
> * Debug server trace: http://paste.openstack.org/show/484740
> * Debug client trace: http://paste.openstack.org/show/484741
> Workaround: use nio2 backend in gerrit.config:
> {code}
> [sshd]
> backend = nio2
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SSHD-635) potential hang in AuthFuture.verify(Long.MAX_VALUE) on error
[ https://issues.apache.org/jira/browse/SSHD-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130432#comment-15130432 ] Oliver Stöneberg commented on SSHD-635: --- Thanks a lot. It's quite easy for me to reproduce this and now knowing which part of the code I have to look at for the error handling I should be able to get more insight in case your change didn't fix it. > potential hang in AuthFuture.verify(Long.MAX_VALUE) on error > > > Key: SSHD-635 > URL: https://issues.apache.org/jira/browse/SSHD-635 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: ssh.log > > > It appears that after the SshClient.connect().verify() call was successful > the ClientSession.auth().verify() call might hang if the connection was > closed in-between those calls. If AuthFuture.verify() is called with a > timeout it correctly times out, but I would expect it to detect that the > connection is closed/closing and throw a proper error even without the > timeout. > The errors I see in the log are these (more completel log attached): > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:137) Caught > IOException[An established connection was aborted by the software in your > host machine] - calling handler > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:142) > Exception handler threw IllegalStateException, closing the session: No > session available > It's hard to debug for me since it only happens in a multi-threaded scenario > and I haven't been able to generate separate logs per session (is this even > possible?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-635) potential hang in AuthFuture.verify(Long.MAX_VALUE) on error
[ https://issues.apache.org/jira/browse/SSHD-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128021#comment-15128021 ] Oliver Stöneberg commented on SSHD-635: --- Unfortunately it's a different issue and I am using 1.1 (or at least a prior revision very close to it). I am sure since I reported SSHD-600. > potential hang in AuthFuture.verify(Long.MAX_VALUE) on error > > > Key: SSHD-635 > URL: https://issues.apache.org/jira/browse/SSHD-635 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: ssh.log > > > It appears that after the SshClient.connect().verify() call was successful > the ClientSession.auth().verify() call might hang if the connection was > closed in-between those calls. If AuthFuture.verify() is called with a > timeout it correctly times out, but I would expect it to detect that the > connection is closed/closing and throw a proper error even without the > timeout. > The errors I see in the log are these (more completel log attached): > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:137) Caught > IOException[An established connection was aborted by the software in your > host machine] - calling handler > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:142) > Exception handler threw IllegalStateException, closing the session: No > session available > It's hard to debug for me since it only happens in a multi-threaded scenario > and I haven't been able to generate separate logs per session (is this even > possible?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-635) potential hang in AuthFuture.verify(Long.MAX_VALUE) on error
[ https://issues.apache.org/jira/browse/SSHD-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Stöneberg updated SSHD-635: -- Attachment: ssh.log > potential hang in AuthFuture.verify(Long.MAX_VALUE) on error > > > Key: SSHD-635 > URL: https://issues.apache.org/jira/browse/SSHD-635 > Project: MINA SSHD > Issue Type: Bug >Reporter: Oliver Stöneberg > Attachments: ssh.log > > > It appears that after the SshClient.connect().verify() call was successful > the ClientSession.auth().verify() call might hang if the connection was > closed in-between those calls. If AuthFuture.verify() is called with a > timeout it correctly times out, but I would expect it to detect that the > connection is closed/closing and throw a proper error even without the > timeout. > The errors I see in the log are these (more completel log attached): > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:137) Caught > IOException[An established connection was aborted by the software in your > host machine] - calling handler > DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:142) > Exception handler threw IllegalStateException, closing the session: No > session available > It's hard to debug for me since it only happens in a multi-threaded scenario > and I haven't been able to generate separate logs per session (is this even > possible?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SSHD-635) potential hang in AuthFuture.verify(Long.MAX_VALUE) on error
Oliver Stöneberg created SSHD-635: - Summary: potential hang in AuthFuture.verify(Long.MAX_VALUE) on error Key: SSHD-635 URL: https://issues.apache.org/jira/browse/SSHD-635 Project: MINA SSHD Issue Type: Bug Reporter: Oliver Stöneberg It appears that after the SshClient.connect().verify() call was successful the ClientSession.auth().verify() call might hang if the connection was closed in-between those calls. If AuthFuture.verify() is called with a timeout it correctly times out, but I would expect it to detect that the connection is closed/closing and throw a proper error even without the timeout. The errors I see in the log are these (more completel log attached): DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:137) Caught IOException[An established connection was aborted by the software in your host machine] - calling handler DEBUG [sshd-SshClient[329a1f8d]-nio2-thread-5] (Nio2Session.java:142) Exception handler threw IllegalStateException, closing the session: No session available It's hard to debug for me since it only happens in a multi-threaded scenario and I haven't been able to generate separate logs per session (is this even possible?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SSHD-626) ssh scp command is broken on mina backend
[
https://issues.apache.org/jira/browse/SSHD-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147672#comment-15147672
]
Hugo Arès commented on SSHD-626:
I investigated and narrow it down to the change done in
AbstractChannel.handleRequest method. If revert the change done to reuse the
incoming buffer in that method, it fixes the problem.
> ssh scp command is broken on mina backend
> -
>
> Key: SSHD-626
> URL: https://issues.apache.org/jira/browse/SSHD-626
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 1.0.0
> Environment: Gerrit Code Review
>Reporter: David Ostrovsky
>
> After upgrade of sshd-core to 1.0.0 and mina to 2.10
> ssh scp command doesn't work with mina backend.
> * Upgrade change: https://gerrit-review.googlesource.com/73033
> * Debug server trace: http://paste.openstack.org/show/484740
> * Debug client trace: http://paste.openstack.org/show/484741
> Workaround: use nio2 backend in gerrit.config:
> {code}
> [sshd]
> backend = nio2
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SSHD-626) ssh scp command is broken on mina backend
[
https://issues.apache.org/jira/browse/SSHD-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148767#comment-15148767
]
Hugo Arès commented on SSHD-626:
I tracked it down hoping that the information I gave would ring a bell to
someone from this project. I can upload a revert patch but it feels like going
backward. The intention of the author was clearly to reuse incoming buffer for
the outgoing response. Because of my lack of knowledge in this project, I
cannot tell if it is safe to reuse incoming buffer in AbstractChannel which
would mean that there is a bug in Mina since the same code works with NIO2.
I will try spend more time on this issue soon but in the meantime, I will
upload a revert patch.
> ssh scp command is broken on mina backend
> -
>
> Key: SSHD-626
> URL: https://issues.apache.org/jira/browse/SSHD-626
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 1.0.0
> Environment: Gerrit Code Review
>Reporter: David Ostrovsky
>
> After upgrade of sshd-core to 1.0.0 and mina to 2.10
> ssh scp command doesn't work with mina backend.
> * Upgrade change: https://gerrit-review.googlesource.com/73033
> * Debug server trace: http://paste.openstack.org/show/484740
> * Debug client trace: http://paste.openstack.org/show/484741
> Workaround: use nio2 backend in gerrit.config:
> {code}
> [sshd]
> backend = nio2
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SSHD-662) Neither 1.1.1 nor 1.2.0 are indeed released and available on central
[ https://issues.apache.org/jira/browse/SSHD-662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stéphane Landelle updated SSHD-662: --- Summary: Neither 1.1.1 nor 1.2.0 are indeed released and available on central (was: Neither 1.1.1 nor 1.2.0 are indeed released and available on cntral) > Neither 1.1.1 nor 1.2.0 are indeed released and available on central > > > Key: SSHD-662 > URL: https://issues.apache.org/jira/browse/SSHD-662 > Project: MINA SSHD > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: Stéphane Landelle > > Hi, > Commit log contains commits for 1.1.1 and 1.2.0 releases, and pom version is > master is currently 1.2.1-SNAPSHOT. > However, binaries are actually stucked in the Apache Nexus staging > repository: > https://repository.apache.org/content/groups/staging/org/apache/sshd/sshd/. > Would it be possible to complete the release cycle for for versions and > publish them on maven central, please? > Regards -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SSHD-662) Neither 1.1.1 nor 1.2.0 are indeed released and available on central
[ https://issues.apache.org/jira/browse/SSHD-662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stéphane Landelle updated SSHD-662: --- Description: Hi, Commit log contains commits for 1.1.1 and 1.2.0 releases, and pom version in master is currently 1.2.1-SNAPSHOT. However, binaries are actually stucked in the Apache Nexus staging repository: https://repository.apache.org/content/groups/staging/org/apache/sshd/sshd/. Would it be possible to complete the release cycle for for versions and publish them on maven central, please? Regards was: Hi, Commit log contains commits for 1.1.1 and 1.2.0 releases, and pom version is master is currently 1.2.1-SNAPSHOT. However, binaries are actually stucked in the Apache Nexus staging repository: https://repository.apache.org/content/groups/staging/org/apache/sshd/sshd/. Would it be possible to complete the release cycle for for versions and publish them on maven central, please? Regards > Neither 1.1.1 nor 1.2.0 are indeed released and available on central > > > Key: SSHD-662 > URL: https://issues.apache.org/jira/browse/SSHD-662 > Project: MINA SSHD > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: Stéphane Landelle > > Hi, > Commit log contains commits for 1.1.1 and 1.2.0 releases, and pom version in > master is currently 1.2.1-SNAPSHOT. > However, binaries are actually stucked in the Apache Nexus staging > repository: > https://repository.apache.org/content/groups/staging/org/apache/sshd/sshd/. > Would it be possible to complete the release cycle for for versions and > publish them on maven central, please? > Regards -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SSHD-662) Neither 1.1.1 nor 1.2.0 are indeed released and available on cntral
Stéphane Landelle created SSHD-662: -- Summary: Neither 1.1.1 nor 1.2.0 are indeed released and available on cntral Key: SSHD-662 URL: https://issues.apache.org/jira/browse/SSHD-662 Project: MINA SSHD Issue Type: Bug Affects Versions: 1.1.1 Reporter: Stéphane Landelle Hi, Commit log contains commits for 1.1.1 and 1.2.0 releases, and pom version is master is currently 1.2.1-SNAPSHOT. However, binaries are actually stucked in the Apache Nexus staging repository: https://repository.apache.org/content/groups/staging/org/apache/sshd/sshd/. Would it be possible to complete the release cycle for for versions and publish them on maven central, please? Regards -- This message was sent by Atlassian JIRA (v6.3.4#6332)
