[jira] [Updated] (IGNITE-3606) Node sometimes fails to detect broken connection
[
https://issues.apache.org/jira/browse/IGNITE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ksenia Rybakova updated IGNITE-3606:
Attachment: logs.zip
> Node sometimes fails to detect broken connection
>
>
> Key: IGNITE-3606
> URL: https://issues.apache.org/jira/browse/IGNITE-3606
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Semen Boikov
>Priority: Critical
> Fix For: 2.2
>
> Attachments: logs.zip
>
>
> Here is test reproducing issue
> https://github.com/rossdanderson/IgniteDeadlock.
> When I run this test observe this sequence:
> - server starts
> - client starts
> - server sends 2000 messages to client, on client node communication
> backpressure pauses reads
> - server gets write timeout and closes socket
> - for some reason client does not detect that existing connection was broken
> and thinks that connection is still established (most probably because reads
> are paused and node does not try to access connection)
> - when server tries to re-connec, client sees that connection already
> established and rejects connection, so server constantly tries to reconnect
> and does not exit from reconnect loop:
> {noformat}
> "main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
> [0x021ed000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (IGNITE-3606) Node sometimes fails to detect broken connection
[
https://issues.apache.org/jira/browse/IGNITE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-3606:
Fix Version/s: (was: 2.1)
2.2
> Node sometimes fails to detect broken connection
>
>
> Key: IGNITE-3606
> URL: https://issues.apache.org/jira/browse/IGNITE-3606
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Semen Boikov
>Priority: Critical
> Fix For: 2.2
>
>
> Here is test reproducing issue
> https://github.com/rossdanderson/IgniteDeadlock.
> When I run this test observe this sequence:
> - server starts
> - client starts
> - server sends 2000 messages to client, on client node communication
> backpressure pauses reads
> - server gets write timeout and closes socket
> - for some reason client does not detect that existing connection was broken
> and thinks that connection is still established (most probably because reads
> are paused and node does not try to access connection)
> - when server tries to re-connec, client sees that connection already
> established and rejects connection, so server constantly tries to reconnect
> and does not exit from reconnect loop:
> {noformat}
> "main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
> [0x021ed000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (IGNITE-3606) Node sometimes fails to detect broken connection
[
https://issues.apache.org/jira/browse/IGNITE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semen Boikov updated IGNITE-3606:
-
Fix Version/s: (was: 2.0)
2.1
> Node sometimes fails to detect broken connection
>
>
> Key: IGNITE-3606
> URL: https://issues.apache.org/jira/browse/IGNITE-3606
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Semen Boikov
>Priority: Critical
> Fix For: 2.1
>
>
> Here is test reproducing issue
> https://github.com/rossdanderson/IgniteDeadlock.
> When I run this test observe this sequence:
> - server starts
> - client starts
> - server sends 2000 messages to client, on client node communication
> backpressure pauses reads
> - server gets write timeout and closes socket
> - for some reason client does not detect that existing connection was broken
> and thinks that connection is still established (most probably because reads
> are paused and node does not try to access connection)
> - when server tries to re-connec, client sees that connection already
> established and rejects connection, so server constantly tries to reconnect
> and does not exit from reconnect loop:
> {noformat}
> "main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
> [0x021ed000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (IGNITE-3606) Node sometimes fails to detect broken connection
[
https://issues.apache.org/jira/browse/IGNITE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semen Boikov updated IGNITE-3606:
-
Fix Version/s: (was: 1.8)
2.0
> Node sometimes fails to detect broken connection
>
>
> Key: IGNITE-3606
> URL: https://issues.apache.org/jira/browse/IGNITE-3606
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Semen Boikov
>Priority: Critical
> Fix For: 2.0
>
>
> Here is test reproducing issue
> https://github.com/rossdanderson/IgniteDeadlock.
> When I run this test observe this sequence:
> - server starts
> - client starts
> - server sends 2000 messages to client, on client node communication
> backpressure pauses reads
> - server gets write timeout and closes socket
> - for some reason client does not detect that existing connection was broken
> and thinks that connection is still established (most probably because reads
> are paused and node does not try to access connection)
> - when server tries to re-connec, client sees that connection already
> established and rejects connection, so server constantly tries to reconnect
> and does not exit from reconnect loop:
> {noformat}
> "main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
> [0x021ed000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (IGNITE-3606) Node sometimes fails to detect broken connection
[
https://issues.apache.org/jira/browse/IGNITE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yakov Zhdanov updated IGNITE-3606:
--
Description:
Here is test reproducing issue https://github.com/rossdanderson/IgniteDeadlock.
When I run this test observe this sequence:
- server starts
- client starts
- server sends 2000 messages to client, on client node communication
backpressure pauses reads
- server gets write timeout and closes socket
- for some reason client does not detect that existing connection was broken
and thinks that connection is still established (most probably because reads
are paused and node does not try to access connection)
- when server tries to re-connec, client sees that connection already
established and rejects connection, so server constantly tries to reconnect and
does not exit from reconnect loop:
{noformat}
"main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
[0x021ed000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
at
org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
{noformat}
was:
Here is test reproducing issue https://github.com/rossdanderson/IgniteDeadlock.
When I run this test observe this sequence:
- server starts
- client starts
- server sends 2000 messages to client, on client node communication
backpressure pauses reads
- server gets write timeout and closes socket
- for some reason client does not detect that existing connection was broken
and thinks that connection is still established (most probably because reads
are paused and node does not try to access connection)
- when server tries to re-connect then client sees that connection already
established and rejects connection, so server constantly tries to reconnect and
does not exist from reconnect loop:
{noformat}
"main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
[0x021ed000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
at
org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
{noformat}
> Node sometimes fails to detect broken connection
>
>
> Key: IGNITE-3606
> URL: https://issues.apache.org/jira/browse/IGNITE-3606
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Semen Boikov
>Priority: Critical
> Fix For: 1.8
>
>
> Here is test reproducing issue
> https://github.com/rossdanderson/IgniteDeadlock.
> When I run this test observe this sequence:
> - server starts
> - client starts
> - server sends 2000 messages to client, on client node communication
> backpressure pauses reads
> - server gets write timeout and closes socket
> - for some reason client does not detect that existing connection was broken
> and thinks that connection is still established (most probably because reads
> are paused and node does not try to access connection)
> - when server tries to re-connec, client sees that connection already
> established and rejects connection, so server constantly tries to reconnect
> and does not exit from reconnect loop:
> {noformat}
> "main" prio=6 tid=0x01f4a000 nid=0x3588 waiting on condition
> [0x021ed000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessa
