[jira] [Closed] (SSHD-940) Query on Thread pool and the ssh threads hanging in blocked state
[ https://issues.apache.org/jira/browse/SSHD-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goldstein Lyor closed SSHD-940. --- Resolution: Won't Fix > Query on Thread pool and the ssh threads hanging in blocked state > - > > Key: SSHD-940 > URL: https://issues.apache.org/jira/browse/SSHD-940 > Project: MINA SSHD > Issue Type: Question >Reporter: Anudeep >Priority: Blocker > > Hi Team, > We face a problem with sshd-core-0.13.0 (whenever a thread wait for some > space on the window and the another thread initiate a close on the same > connection threads go into blocked state and if the thread count of such > threads(BLOCKED state) crosses three no new connections are opened). > We have a simple question ,we know this is a bug in 0.13.0 and it was > mentioned as fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348) > and now we need a clear picture if this issue is completely resolved in the > later versions. > If yes please let us know the version. > We will migrate to later version based on your confirmation ,we cannot simply > go and migrate to a later one and if the problem still exists it will not be > a compromising one. > Below are my previous requests raised for the same.please have a look on them > and let us know if any further information is required from us. > https://issues.apache.org/jira/browse/SSHD-938 > https://issues.apache.org/jira/browse/SSHD-939 > Thanks in Advance. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org For additional commands, e-mail: dev-h...@mina.apache.org
[jira] [Commented] (SSHD-940) Query on Thread pool and the ssh threads hanging in blocked state
[
https://issues.apache.org/jira/browse/SSHD-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929285#comment-16929285
]
Goldstein Lyor commented on SSHD-940:
-
{quote}
We have a simple question ,we know this is a bug in 0.13.0 and it was mentioned
as fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348) and now we
need a clear picture if this issue is completely resolved in the later versions.
If yes please let us know the version.
We will migrate to later version based on your confirmation ,we cannot simply
go and migrate to a later one and if the problem still exists it will not be a
compromising one.
{quote}
I am sorry, but I cannot provide any such guarantees - especially on such an
old version. I understand your reluctance to upgrade, but I see no other
option. You keep opening bugs and asking for assistance on a version that is
long gone, with many changes since then. I am sorry I cannot be of more help -
but like I said, we do not have the manpower to provide support for more than 1
version...
> Query on Thread pool and the ssh threads hanging in blocked state
> -
>
> Key: SSHD-940
> URL: https://issues.apache.org/jira/browse/SSHD-940
> Project: MINA SSHD
> Issue Type: Question
>Reporter: Anudeep
>Priority: Blocker
>
> Hi Team,
> We face a problem with sshd-core-0.13.0 (whenever a thread wait for some
> space on the window and the another thread initiate a close on the same
> connection threads go into blocked state and if the thread count of such
> threads(BLOCKED state) crosses three no new connections are opened).
> We have a simple question ,we know this is a bug in 0.13.0 and it was
> mentioned as fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348)
> and now we need a clear picture if this issue is completely resolved in the
> later versions.
> If yes please let us know the version.
> We will migrate to later version based on your confirmation ,we cannot simply
> go and migrate to a later one and if the problem still exists it will not be
> a compromising one.
> Below are my previous requests raised for the same.please have a look on them
> and let us know if any further information is required from us.
> https://issues.apache.org/jira/browse/SSHD-938
> https://issues.apache.org/jira/browse/SSHD-939
> Thanks in Advance.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org
For additional commands, e-mail: dev-h...@mina.apache.org
[jira] [Commented] (SSHD-939) SSH threads get blocked in Object.wait() method forever resulting in CLOSE_WAIT connections
[
https://issues.apache.org/jira/browse/SSHD-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929282#comment-16929282
]
Goldstein Lyor commented on SSHD-939:
-
I don't see how
> SSH threads get blocked in Object.wait() method forever resulting in
> CLOSE_WAIT connections
> ---
>
> Key: SSHD-939
> URL: https://issues.apache.org/jira/browse/SSHD-939
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Anudeep
>Assignee: Goldstein Lyor
>Priority: Major
> Attachments: svc-1-bnsiserv_06_09_2019-server.7z,
> svc-1-bnsiserv_thread.log
>
>
> Ssh threads get blocked in object.wait() for ever and if the count of such
> threads crosses 3 no more ssh requests are served.
> We had an analysis of the logs and from the thread model of apache we see the
> maximum number of I/O threads is number of cores +1 in our server the number
> of cores is 2.
> SO after three such blocked threads we are not able to open any new ssh
> connection.
> Can you please confirm if the above logic is correct .
>
> We too found a ticket raised in 0.14.0
> (https://issues.apache.org/jira/browse/SSHD-348)which says threads in blocked
> state are fixed with 0.14.0 so we need a confirmation from you if 0.14.0 will
> fix the threads to be released and do not hung the process even if there are
> more than 3 of such a kind.
>
> Logs from thread dumps
> {code:java}
> From the thread dump it looks like we have a deadlock, e.g.
> In one thread we're trying to send out data on an SSH connection, but we're
> waiting for "space"
> "pool-7-thread-2213" #2677 prio=5 os_prio=0 tid=0x7f85554cc800 nid=0x61e3
> in Object.wait() [0x7f84ea29f000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.sshd.common.channel.Window.waitForSpace(Window.java:148)
> - locked <0x000764942ad8> (a
> org.apache.sshd.common.channel.Window)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.flush(ChannelOutputStream.java:116)
> - eliminated <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.write(ChannelOutputStream.java:84)
> - locked <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> - locked <0x000764943500> (a java.io.OutputStreamWriter)
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> at java.io.BufferedWriter.write(BufferedWriter.java:188)
> - locked <0x000764943500> (a java.io.OutputStreamWriter)
> at java.io.PrintWriter.write(PrintWriter.java:426)
> - locked <0x0007649434d8> (a java.io.BufferedWriter)
> at java.io.PrintWriter.write(PrintWriter.java:443)
> In another thread, we trying to close the same connection
> "sshd-SshServer[1ae27e4e]-nio2-thread-3" #356 prio=5 os_prio=0
> tid=0x015f6000 nid=0x30b8 waiting for monitor entry
> [0x7f851ab9c000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.close(ChannelOutputStream.java:146)
> - waiting to lock <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommandContext.shutdown(SshCommandContext.java:146)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommand$ContextStarter.shutdown(SshCommand.java:104)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommand.destroy(SshCommand.java:78)
> at
> org.apache.sshd.server.channel.ChannelSession.doCloseImmediately(ChannelSession.java:242)
> at
> org.apache.sshd.common.util.CloseableUtils$AbstractCloseable.close(CloseableUtils.java:276)
> at
> org.apache.sshd.common.util.CloseableUtils$ParallelCloseable.doClose(CloseableUtils.java:182)
> at
> org.apache.sshd.common.util.CloseableUtils$SimpleCloseable.close(CloseableUtils.java:151)
> at
> org.apache.sshd.common.io.nio2.Nio2Session$1.onFailed(Nio2Session.java:199)
> at
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler$2.run(Nio2CompletionHandler.java:41)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler.failed(Nio2CompletionHandler.java:39)
> at
>
[jira] [Updated] (SSHD-940) Query on Thread pool and the ssh threads hanging in blocked state
[ https://issues.apache.org/jira/browse/SSHD-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anudeep updated SSHD-940: - Description: Hi Team, We face a problem with sshd-core-0.13.0 (whenever a thread wait for some space on the window and the another thread initiate a close on the same connection threads go into blocked state and if the thread count of such threads(BLOCKED state) crosses three no new connections are opened). We have a simple question ,we know this is a bug in 0.13.0 and it was mentioned as fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348) and now we need a clear picture if this issue is completely resolved in the later versions. If yes please let us know the version. We will migrate to later version based on your confirmation ,we cannot simply go and migrate to a later one and if the problem still exists it will not be a compromising one. Below are my previous requests raised for the same.please have a look on them and let us know if any further information is required from us. https://issues.apache.org/jira/browse/SSHD-938 https://issues.apache.org/jira/browse/SSHD-939 Thanks in Advance. was: Hi Team, We face a problem with sshd-core-0.13.0 (whenever a thread wait for some space on the window and the another thread initiate a close on the same connection threads go into blocked state and if the thread count of such threads(BLOCKED state) crosses three no new connections are opened). We have a simple question ,we know this is a bug in 0.13.0 and was fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348) and now we need a clear picture if this issue is completely resolved in the later versions. If yes please let us know the version. We will migrate to later version based on your confirmation ,we cannot simply go and migrate to a later one and if the problem still exists it will not be a compromising one. Below are my previous requests raised for the same.please have a look on them and let us know if any further information is required from us. https://issues.apache.org/jira/browse/SSHD-938 https://issues.apache.org/jira/browse/SSHD-939 Thanks in Advance. > Query on Thread pool and the ssh threads hanging in blocked state > - > > Key: SSHD-940 > URL: https://issues.apache.org/jira/browse/SSHD-940 > Project: MINA SSHD > Issue Type: Question >Reporter: Anudeep >Priority: Blocker > > Hi Team, > We face a problem with sshd-core-0.13.0 (whenever a thread wait for some > space on the window and the another thread initiate a close on the same > connection threads go into blocked state and if the thread count of such > threads(BLOCKED state) crosses three no new connections are opened). > We have a simple question ,we know this is a bug in 0.13.0 and it was > mentioned as fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348) > and now we need a clear picture if this issue is completely resolved in the > later versions. > If yes please let us know the version. > We will migrate to later version based on your confirmation ,we cannot simply > go and migrate to a later one and if the problem still exists it will not be > a compromising one. > Below are my previous requests raised for the same.please have a look on them > and let us know if any further information is required from us. > https://issues.apache.org/jira/browse/SSHD-938 > https://issues.apache.org/jira/browse/SSHD-939 > Thanks in Advance. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org For additional commands, e-mail: dev-h...@mina.apache.org
[jira] [Created] (SSHD-940) Query on Thread pool and the ssh threads hanging in blocked state
Anudeep created SSHD-940: Summary: Query on Thread pool and the ssh threads hanging in blocked state Key: SSHD-940 URL: https://issues.apache.org/jira/browse/SSHD-940 Project: MINA SSHD Issue Type: Question Reporter: Anudeep Hi Team, We face a problem with sshd-core-0.13.0 (whenever a thread wait for some space on the window and the another thread initiate a close on the same connection threads go into blocked state and if the thread count of such threads(BLOCKED state) crosses three no new connections are opened). We have a simple question ,we know this is a bug in 0.13.0 and was fixed in 0.14.0 (https://issues.apache.org/jira/browse/SSHD-348) and now we need a clear picture if this issue is completely resolved in the later versions. If yes please let us know the version. We will migrate to later version based on your confirmation ,we cannot simply go and migrate to a later one and if the problem still exists it will not be a compromising one. Below are my previous requests raised for the same.please have a look on them and let us know if any further information is required from us. https://issues.apache.org/jira/browse/SSHD-938 https://issues.apache.org/jira/browse/SSHD-939 Thanks in Advance. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org For additional commands, e-mail: dev-h...@mina.apache.org
[jira] [Commented] (SSHD-939) SSH threads get blocked in Object.wait() method forever resulting in CLOSE_WAIT connections
[
https://issues.apache.org/jira/browse/SSHD-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929037#comment-16929037
]
Anudeep commented on SSHD-939:
--
Hi [~lgoldstein],
This was my previous question is it posible to figure out the cipher from the
packet which we receive.
Regards,
Anudeep
> SSH threads get blocked in Object.wait() method forever resulting in
> CLOSE_WAIT connections
> ---
>
> Key: SSHD-939
> URL: https://issues.apache.org/jira/browse/SSHD-939
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Anudeep
>Assignee: Goldstein Lyor
>Priority: Major
> Attachments: svc-1-bnsiserv_06_09_2019-server.7z,
> svc-1-bnsiserv_thread.log
>
>
> Ssh threads get blocked in object.wait() for ever and if the count of such
> threads crosses 3 no more ssh requests are served.
> We had an analysis of the logs and from the thread model of apache we see the
> maximum number of I/O threads is number of cores +1 in our server the number
> of cores is 2.
> SO after three such blocked threads we are not able to open any new ssh
> connection.
> Can you please confirm if the above logic is correct .
>
> We too found a ticket raised in 0.14.0
> (https://issues.apache.org/jira/browse/SSHD-348)which says threads in blocked
> state are fixed with 0.14.0 so we need a confirmation from you if 0.14.0 will
> fix the threads to be released and do not hung the process even if there are
> more than 3 of such a kind.
>
> Logs from thread dumps
> {code:java}
> From the thread dump it looks like we have a deadlock, e.g.
> In one thread we're trying to send out data on an SSH connection, but we're
> waiting for "space"
> "pool-7-thread-2213" #2677 prio=5 os_prio=0 tid=0x7f85554cc800 nid=0x61e3
> in Object.wait() [0x7f84ea29f000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.sshd.common.channel.Window.waitForSpace(Window.java:148)
> - locked <0x000764942ad8> (a
> org.apache.sshd.common.channel.Window)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.flush(ChannelOutputStream.java:116)
> - eliminated <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.write(ChannelOutputStream.java:84)
> - locked <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> - locked <0x000764943500> (a java.io.OutputStreamWriter)
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> at java.io.BufferedWriter.write(BufferedWriter.java:188)
> - locked <0x000764943500> (a java.io.OutputStreamWriter)
> at java.io.PrintWriter.write(PrintWriter.java:426)
> - locked <0x0007649434d8> (a java.io.BufferedWriter)
> at java.io.PrintWriter.write(PrintWriter.java:443)
> In another thread, we trying to close the same connection
> "sshd-SshServer[1ae27e4e]-nio2-thread-3" #356 prio=5 os_prio=0
> tid=0x015f6000 nid=0x30b8 waiting for monitor entry
> [0x7f851ab9c000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.close(ChannelOutputStream.java:146)
> - waiting to lock <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommandContext.shutdown(SshCommandContext.java:146)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommand$ContextStarter.shutdown(SshCommand.java:104)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommand.destroy(SshCommand.java:78)
> at
> org.apache.sshd.server.channel.ChannelSession.doCloseImmediately(ChannelSession.java:242)
> at
> org.apache.sshd.common.util.CloseableUtils$AbstractCloseable.close(CloseableUtils.java:276)
> at
> org.apache.sshd.common.util.CloseableUtils$ParallelCloseable.doClose(CloseableUtils.java:182)
> at
> org.apache.sshd.common.util.CloseableUtils$SimpleCloseable.close(CloseableUtils.java:151)
> at
> org.apache.sshd.common.io.nio2.Nio2Session$1.onFailed(Nio2Session.java:199)
> at
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler$2.run(Nio2CompletionHandler.java:41)
> at java.security.AccessController.doPrivileged(Native Method)
> at
>
[jira] [Commented] (SSHD-939) SSH threads get blocked in Object.wait() method forever resulting in CLOSE_WAIT connections
[
https://issues.apache.org/jira/browse/SSHD-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929033#comment-16929033
]
Anudeep commented on SSHD-939:
--
Hi [~lgoldstein],
I am sorry but i need to ask for a clear picture. Cipher here you mean the
client one or the server one? What do you mean by decode bug ? The exceptions
invalid packet length is seen when we receive a packet from client.
Regards,
Anudeep
> SSH threads get blocked in Object.wait() method forever resulting in
> CLOSE_WAIT connections
> ---
>
> Key: SSHD-939
> URL: https://issues.apache.org/jira/browse/SSHD-939
> Project: MINA SSHD
> Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Anudeep
>Assignee: Goldstein Lyor
>Priority: Major
> Attachments: svc-1-bnsiserv_06_09_2019-server.7z,
> svc-1-bnsiserv_thread.log
>
>
> Ssh threads get blocked in object.wait() for ever and if the count of such
> threads crosses 3 no more ssh requests are served.
> We had an analysis of the logs and from the thread model of apache we see the
> maximum number of I/O threads is number of cores +1 in our server the number
> of cores is 2.
> SO after three such blocked threads we are not able to open any new ssh
> connection.
> Can you please confirm if the above logic is correct .
>
> We too found a ticket raised in 0.14.0
> (https://issues.apache.org/jira/browse/SSHD-348)which says threads in blocked
> state are fixed with 0.14.0 so we need a confirmation from you if 0.14.0 will
> fix the threads to be released and do not hung the process even if there are
> more than 3 of such a kind.
>
> Logs from thread dumps
> {code:java}
> From the thread dump it looks like we have a deadlock, e.g.
> In one thread we're trying to send out data on an SSH connection, but we're
> waiting for "space"
> "pool-7-thread-2213" #2677 prio=5 os_prio=0 tid=0x7f85554cc800 nid=0x61e3
> in Object.wait() [0x7f84ea29f000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.sshd.common.channel.Window.waitForSpace(Window.java:148)
> - locked <0x000764942ad8> (a
> org.apache.sshd.common.channel.Window)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.flush(ChannelOutputStream.java:116)
> - eliminated <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.write(ChannelOutputStream.java:84)
> - locked <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> - locked <0x000764943500> (a java.io.OutputStreamWriter)
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> at java.io.BufferedWriter.write(BufferedWriter.java:188)
> - locked <0x000764943500> (a java.io.OutputStreamWriter)
> at java.io.PrintWriter.write(PrintWriter.java:426)
> - locked <0x0007649434d8> (a java.io.BufferedWriter)
> at java.io.PrintWriter.write(PrintWriter.java:443)
> In another thread, we trying to close the same connection
> "sshd-SshServer[1ae27e4e]-nio2-thread-3" #356 prio=5 os_prio=0
> tid=0x015f6000 nid=0x30b8 waiting for monitor entry
> [0x7f851ab9c000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.sshd.common.channel.ChannelOutputStream.close(ChannelOutputStream.java:146)
> - waiting to lock <0x000764942c20> (a
> org.apache.sshd.common.channel.ChannelOutputStream)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommandContext.shutdown(SshCommandContext.java:146)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommand$ContextStarter.shutdown(SshCommand.java:104)
> at
> com.ericsson.oss.nbi.connection.ssh.SshCommand.destroy(SshCommand.java:78)
> at
> org.apache.sshd.server.channel.ChannelSession.doCloseImmediately(ChannelSession.java:242)
> at
> org.apache.sshd.common.util.CloseableUtils$AbstractCloseable.close(CloseableUtils.java:276)
> at
> org.apache.sshd.common.util.CloseableUtils$ParallelCloseable.doClose(CloseableUtils.java:182)
> at
> org.apache.sshd.common.util.CloseableUtils$SimpleCloseable.close(CloseableUtils.java:151)
> at
> org.apache.sshd.common.io.nio2.Nio2Session$1.onFailed(Nio2Session.java:199)
> at
> org.apache.sshd.common.io.nio2.Nio2CompletionHandler$2.run(Nio2CompletionHandler.java:41)
>
