RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-13 Thread dkarachentsev
Hi,

This thread dump is absolutely fine, you confused socket state and java
thread state. These two things are absolutely unrelated.
There should not be so many socket connections (TIME_WAIT means that socket
already closed and waiting for last packages) for three nodes. Could you
please share your configuration and netstat output?

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-12 Thread HEWA WIDANA GAMAGE, SUBASH
And I took a thread dump by then.

I could see about 10 of following WAITING. Is this looks fine ?

"sys-stripe-6-#29637%null%" #46762 prio=5 os_prio=0 tid=0x7fa24402b320 
nid=0x3900 waiting on condition [0x7fa22d454000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:526)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:422)
at java.lang.Thread.run(Thread.java:748)


-Original Message-
From: HEWA WIDANA GAMAGE, SUBASH [mailto:subash.hewawidanagam...@fmr.com] 
Sent: Tuesday, June 12, 2018 2:06 PM
To: user@ignite.apache.org
Subject: RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED 
event

I have only 3 nodes, in three separate physical boxes. Question is, can there 
be any possibility for creating such number of sockets on 47500, between two 
nodes ? I cannot reproduce the issue. Hence trying to find any theoretical 
possibility.



-Original Message-
From: dkarachentsev [mailto:dkarachent...@gridgain.com] 
Sent: Tuesday, June 12, 2018 11:37 AM
To: user@ignite.apache.org
Subject: RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED 
event

There is no difference on how do you start/stop your node. 
Node on start will examine all connections specified in address list: it
takes one address and port and tries to connect to it. If not successfull,
get another address and port. For instance if you have address
1.2.3.4:47500..47509, node will check 10 addresses.

Does this impact you somehow?

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-12 Thread HEWA WIDANA GAMAGE, SUBASH
I have only 3 nodes, in three separate physical boxes. Question is, can there 
be any possibility for creating such number of sockets on 47500, between two 
nodes ? I cannot reproduce the issue. Hence trying to find any theoretical 
possibility.



-Original Message-
From: dkarachentsev [mailto:dkarachent...@gridgain.com] 
Sent: Tuesday, June 12, 2018 11:37 AM
To: user@ignite.apache.org
Subject: RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED 
event

There is no difference on how do you start/stop your node. 
Node on start will examine all connections specified in address list: it
takes one address and port and tries to connect to it. If not successfull,
get another address and port. For instance if you have address
1.2.3.4:47500..47509, node will check 10 addresses.

Does this impact you somehow?

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


RE: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-12 Thread HEWA WIDANA GAMAGE, SUBASH
We are using TcpDiscoveryVmIpFinder with PARTITIONED cache with one backup. 

But I specifically noticed this behavior ONLY after received a NODE_FAILED 
event on the node, and then we restart Ignite Process in the same JVM. So is 
there any known/possible implications of restarting Ignite without killing the 
JVM ?



-Original Message-
From: dkarachentsev [mailto:dkarachent...@gridgain.com] 
Sent: Tuesday, June 12, 2018 4:58 AM
To: user@ignite.apache.org
Subject: Re: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED 
event

Hi,

TcpDiscoveryMulticastIpFinder produces such a big number of connections. I'd
recommend to switch to TcpDiscoveryVmIpFinder with static set of addresses.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


Re: Ignite opens/close 5000 sockets in every 5mins after NODE_FAILED event

2018-06-12 Thread dkarachentsev
Hi,

TcpDiscoveryMulticastIpFinder produces such a big number of connections. I'd
recommend to switch to TcpDiscoveryVmIpFinder with static set of addresses.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/