Re: IgniteCompute.broadcast() stuck

2016-09-01 Thread bintisepaha
Also, it was stuck like this for hours. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCompute-broadcast-stuck-tp7255p7472.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCompute.broadcast() stuck

2016-09-01 Thread bintisepaha
Val, this dump was from the client node which I sent on the original email. the zipped up dumps were from all the server nodes that participate in the distributed cache. anyways, changing it to run() fixed the issue. But we never understand the root cause of the hanging, its always that the

Re: IgniteCompute.broadcast() stuck

2016-08-26 Thread bintisepaha
It was hanging because all of our clients were stuck at the below Thread dump Name: main State: WAITING on org.apache.ignite.internal.ComputeTaskInternalFuture@3fd1be52 Total blocked: 5 Total waited: 5,975 Stack trace: sun.misc.Unsafe.park(Native Method)

Re: IgniteCompute.broadcast() stuck

2016-08-25 Thread vkulichenko
Binti, These are just listener threads for JMX and they are not locking anything. And as I said, I don't see any evidence of blocked compute Actually, I just tried to search for 'Compute', 'broadcast' and other related words in dumps and found nothing which means that nobody is executing any

Re: IgniteCompute.broadcast() stuck

2016-08-25 Thread bintisepaha
Also, it looks like a lot of other nodes did not have these TCP threads. Could that be the reason for this issue? server nodes not accepting connections? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCompute-broadcast-stuck-tp7255p7314.html Sent from the

Re: IgniteCompute.broadcast() stuck

2016-08-25 Thread bintisepaha
Vlad, Look at this one below in file 2511995.txt "RMI TCP Connection(8)-10.10.11.100" #202 daemon prio=5 os_prio=0 tid=0x7ff050009000 nid=0x2a7a23 in Object.wait() [0x7fef5aef5000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native

Re: IgniteCompute.broadcast() stuck

2016-08-25 Thread Vladislav Pyatkov
Hello, I do not see locks on RMI threads. I think, you gave wrong dump files. Please, could you check files or provide the fragment from the dump where a thread locked? On Wed, Aug 24, 2016 at 11:53 PM, bintisepaha wrote: > Did you see the dumps for RMI threads? We are

Re: IgniteCompute.broadcast() stuck

2016-08-24 Thread bintisepaha
Did you see the dumps for RMI threads? We are seeing some RMI TCP Communication threads locked. Computation is stuck because clients are hanging in broadcast(), could this be related to the rmi tcp threads being locked. -- View this message in context:

Re: IgniteCompute.broadcast() stuck

2016-08-24 Thread vkulichenko
Binti, Broadcast method is broadcasting :) I.e., it sends a closure to all nodes. For you case you should use run(). Dumps look empty, like nothing is actually executed. Why do you think that the computation is stuck? -Val -- View this message in context:

Re: IgniteCompute.broadcast() stuck

2016-08-24 Thread bintisepaha
Val, if I selecting only one random node like below ClusterGroup serverGroup = ignite.cluster().forServers().forRandom(); IgniteCompute serverCompute = ignite.compute(serverGroup); The broadcast should only send the task to the pre-selected random node from the

IgniteCompute.broadcast() stuck

2016-08-23 Thread bintisepaha
Hi We are on ignite 1.5.0-final and are recently facing this issue from the client side broadcaasting a job to a random remote server node. how can we avoid this and what is causing this? We can 20 parallel clients for load testing, and 15 completed with no issues, 5 got stuck here. code that