Re: [akka-user] How to tune failure-detector on high load cluster

2014-07-21 Thread Akka Team
Hi Xingrun, You should try to upgrade to Akka 2.3.4 since it has a fix that might help in this case. Can you try and see if that fixes your issue? -Endre On Fri, Jul 18, 2014 at 2:25 PM, Xingrun CHEN wrote: > Our server keeps long live connections with client mobile app, and send > heartbeat

Re: [akka-user] How to tune failure-detector on high load cluster

2014-07-18 Thread Xingrun CHEN
Our server keeps long live connections with client mobile app, and send heartbeat packet to client every 30s. The Akka version is 2.3.3. On Thursday, July 17, 2014 5:00:54 PM UTC+8, Patrik Nordwall wrote: > > Can you please describe the purpose of 100k heartbeating actors. That is > not somethi

Re: [akka-user] How to tune failure-detector on high load cluster

2014-07-17 Thread Patrik Nordwall
Can you please describe the purpose of 100k heartbeating actors. That is not something that is done by akka itself. Which Akka version are you using? /Patrik > 15 jul 2014 kl. 11:00 skrev Xingrun CHEN : > > > We have about 100k actors who sent heartbeat to client side evey several > seconds.

Re: [akka-user] How to tune failure-detector on high load cluster

2014-07-15 Thread Xingrun CHEN
We have about 100k actors who sent heartbeat to client side evey several seconds. (use system.scheduler.scheduleOnce) And we found it may cause the cluster heartbeat delay. So we change the config `akka.cluster. scheduler.tick-duration` to 9 ms, and then the akka cluster created a second schedu

Re: [akka-user] How to tune failure-detector on high load cluster

2014-07-15 Thread √iktor Ҡlang
Can you send us a stack trace of the scheduler thread? On Tue, Jul 15, 2014 at 7:45 AM, Xingrun CHEN wrote: > We found that "ClusterSystem-scheduler-1" thread is very busy and consume > 50% cpu (but not more than 50%). > And we meet 'node unreachable' issue every day. > Here's our configs: > >

[akka-user] How to tune failure-detector on high load cluster

2014-07-14 Thread Xingrun CHEN
We found that "ClusterSystem-scheduler-1" thread is very busy and consume 50% cpu (but not more than 50%). And we meet 'node unreachable' issue every day. Here's our configs: akka.cluster { failure-detector { acceptable-heartbeat-pause = 6 s # default 3 s threshold = 12.0# def