Re: [akka-user] Re: how to tune akka remoting performance
Hi Ivan, You can try to set the tcp-nodelay option to be false, so you get some batching from the TCP driver. You might want to try to tweak the netty pool-size, too: - akka.remote.netty.tcp.server-socket-worker-pool - akka.remote.netty.tcp.client-socket-worker-pool (http://doc.akka.io/docs/akka/2.3.9/general/configuration.html#akka-remote) On Thu, Apr 2, 2015 at 6:30 PM, Ivan Balashov wrote: > Patrik, > > 2015-04-01 20:12 GMT+03:00 Patrik Nordwall : > >> Are you sending all messages in one go? >> > > Unfortunately, yes, which is rather unrealistic and creates needless GC > activity, I fully admit my oversight on this. > > >> You can compare with this benchmark: >> https://github.com/akka/akka/tree/release-2.3/akka-samples/akka-sample-remote-scala/src/main/scala/sample/remote/benchmark >> > > Finally I managed to run this example in Intellij IDEA. The key to success > was to take master branch rather than 2.x, which is broken Idea-wise ( > http://goo.gl/uUJdGD) > > Here are some results as run on 4 core laptop under OS X: > > == It took 38830 ms to deliver 100 messages, throughtput 25753 msg/s, >> max round-trip 62 ms, burst size 100, payload size 100 >> == It took 35742 ms to deliver 100 messages, throughtput 27978 msg/s, >> max round-trip 1000 ms, burst size 1000, payload size 100 >> == It took 34356 ms to deliver 100 messages, throughtput 29106 msg/s, >> max round-trip 804 ms, burst size 1, payload size 100 >> == It took 21636 ms to deliver 100 messages, throughtput 46219 msg/s, >> max round-trip 2033 ms, burst size 5, payload size 100 > > > The results are somewhat comparable to what was observed in my tests with > high CPU usage caused particularly by context switching, and partly by Akka > Remoting. > > Running Receiver took 1m 20s CPU time > > "Method","Time (ms)","Own Time (ms)" > > *"sun.misc.Unsafe.unpark(Object) Unsafe.java (native)","9716","9716"* >> "sun.nio.ch.KQueueArrayWrapper.kevent0(int, long, int, long) >> KQueueArrayWrapper.java (native)","3841","3838" >> "java.lang.ClassLoader.defineClass1(String, byte[], int, int, >> ProtectionDomain, String) ClassLoader.java (native)","2673","2050" >> "java.net.URI$Parser.scan(int, int, long, long) URI.java","1420","1420" >> "com.yourkit.probes.Table.createRow() Table.java","1123","1123" >> "java.lang.Integer.parseInt(String) Integer.java","1091","1085" >> "java.net.URLClassLoader$1.run() URLClassLoader.java","4844","906" >> "scala.collection.AbstractIterable.() Iterable.scala","508","469" >> "java.lang.Thread.sleep(long) Thread.java (native)","416","416" >> "akka.remote.WireFormats$ActorRefData.(CodedInputStream, >> ExtensionRegistryLite) WireFormats.java","419","387" >> "java.lang.AbstractStringBuilder.expandCapacity(int) >> AbstractStringBuilder.java","381","371" > > > If we divide 1M messages / 80 Sec CPU Time = 12K rps / Core max, which > corresponds to what is observed in test. > > In case my calculations are correct, doesn't this looks a bit unsettling > for Akka to be a CPU-bound system, even when the only thing we do is pass > around chunks of bytes? ;-) > To be honest, akka-remote is not the fastest kid on the block (it is ripe for a redesign), but Patrik and I have seen much better results than you have here. The unpark method that you highlight up there is not really relevant though. -Endre > > > -- > >> Read the docs: http://akka.io/docs/ > >> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Akka Team Typesafe - Reactive apps on the JVM Blog: letitcrash.com Twitter: @akkateam -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: how to tune akka remoting performance
Patrik, 2015-04-01 20:12 GMT+03:00 Patrik Nordwall : > Are you sending all messages in one go? > Unfortunately, yes, which is rather unrealistic and creates needless GC activity, I fully admit my oversight on this. > You can compare with this benchmark: > https://github.com/akka/akka/tree/release-2.3/akka-samples/akka-sample-remote-scala/src/main/scala/sample/remote/benchmark > Finally I managed to run this example in Intellij IDEA. The key to success was to take master branch rather than 2.x, which is broken Idea-wise ( http://goo.gl/uUJdGD) Here are some results as run on 4 core laptop under OS X: == It took 38830 ms to deliver 100 messages, throughtput 25753 msg/s, > max round-trip 62 ms, burst size 100, payload size 100 > == It took 35742 ms to deliver 100 messages, throughtput 27978 msg/s, > max round-trip 1000 ms, burst size 1000, payload size 100 > == It took 34356 ms to deliver 100 messages, throughtput 29106 msg/s, > max round-trip 804 ms, burst size 1, payload size 100 > == It took 21636 ms to deliver 100 messages, throughtput 46219 msg/s, > max round-trip 2033 ms, burst size 5, payload size 100 The results are somewhat comparable to what was observed in my tests with high CPU usage caused particularly by context switching, and partly by Akka Remoting. Running Receiver took 1m 20s CPU time "Method","Time (ms)","Own Time (ms)" *"sun.misc.Unsafe.unpark(Object) Unsafe.java (native)","9716","9716"* > "sun.nio.ch.KQueueArrayWrapper.kevent0(int, long, int, long) > KQueueArrayWrapper.java (native)","3841","3838" > "java.lang.ClassLoader.defineClass1(String, byte[], int, int, > ProtectionDomain, String) ClassLoader.java (native)","2673","2050" > "java.net.URI$Parser.scan(int, int, long, long) URI.java","1420","1420" > "com.yourkit.probes.Table.createRow() Table.java","1123","1123" > "java.lang.Integer.parseInt(String) Integer.java","1091","1085" > "java.net.URLClassLoader$1.run() URLClassLoader.java","4844","906" > "scala.collection.AbstractIterable.() Iterable.scala","508","469" > "java.lang.Thread.sleep(long) Thread.java (native)","416","416" > "akka.remote.WireFormats$ActorRefData.(CodedInputStream, > ExtensionRegistryLite) WireFormats.java","419","387" > "java.lang.AbstractStringBuilder.expandCapacity(int) > AbstractStringBuilder.java","381","371" If we divide 1M messages / 80 Sec CPU Time = 12K rps / Core max, which corresponds to what is observed in test. In case my calculations are correct, doesn't this looks a bit unsettling for Akka to be a CPU-bound system, even when the only thing we do is pass around chunks of bytes? ;-) -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: how to tune akka remoting performance
Are you sending all messages in one go?
You can compare with this benchmark:
https://github.com/akka/akka/tree/release-2.3/akka-samples/akka-sample-remote-scala/src/main/scala/sample/remote/benchmark
/Patrik
> 1 apr 2015 kl. 15:54 skrev Viktor Klang :
>
> Looks like you're using Java Serialization, a good first start would be to
> switch serializers for your messages.
>
>> On Wed, Apr 1, 2015 at 3:49 PM, Ivan Balashov wrote:
>> Viktor,
>>
>> Here is the latest configuration that allowed to me reach 30msg/sec between
>> 2 x 4cpu hosts in GCE network.
>>
>> http://goo.gl/sPnjQS
>>
>> On both sides context switches jumped to 30-50K/sec
>>
>> Client:
>> 0 0 0 1528652 94396 79422800 0 4 468 772 0 0 100 >> 0
>> 2 0 0 1533988 94408 79422400 0 5 9051 16555 11 4 85
>> 0
>> 1 0 0 1326560 94412 79422400 0 6 28710 52792 34 13
>> 54 0
>> 2 0 0 1326188 94412 79422800 0 3 30209 59439 26 8
>> 66 0
>> 1 0 0 1325444 94420 79422800 0 4 26194 51556 24 6
>> 70 0
>> 1 0 0 1328172 94424 79422800 0 1 468 725 0 0 100
>> 0
>>
>> Server:
>> 3 0 0 827812 30372 206631600 0 3 9762 13188 48 7 45
>> 0
>> 1 0 0 564080 30388 206631600 0 3 19969 23901 29 13
>> 59 0
>> 4 0 0 416900 30388 206631600 0 0 24864 31988 39 13
>> 48 0
>> 1 0 0 241876 30388 206631600 0 3 24113 34808 42 11
>> 47 0
>> 4 0 0 145280 30404 201443200 0 2 21127 40357 44 6
>> 50 0
>> 2 0 0 106608 30404 196002400 0 0 19162 36545 47 6
>> 47 0
>> 4 0 0 138560 30328 186709200 0 6 14574 27803 57 4
>> 39 0
>> 2 0 0 293616 30336 186709200 0 4 6390 10023 42 12 46
>> 0
>>
>> Couple more things that concerned me:
>>
>> 1) [WARN] [04/01/2015 12:54:26.125] [systemB-network-dispatcher-5]
>> [akka.tcp://systemB@10.110.112.155:2001/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FsystemA%4010.110.47.70%3A2001-0/endpointWriter]
>> [
>> 473494] buffered messages in EndpointWriter for
>> [akka.tcp://systemA@10.110.47.70:2001]. You should probably implement flow
>> control to avoid flooding the remote connection.
>>
>> Looks like either side is sending more messages than receiving is capable to
>> accept. This however, might be cause by 1M network buffer, which, OTOH is
>> needed for higher throughput.
>> jFTR, GCE network is quite fast (my last measurement gives me 72Mb/sec,
>> while during the test we barely hit 10Mb/sec).
>>
>> 2) Significant heap pressure, apparently caused by 1) and, as a consequence,
>> some GC activity. Profiler gives me estimate of ~700bytes per every message
>> in the queue, quickly growing heap. The slower messages get processed, the
>> more of them are accumulated, the more cpu is needed for GC, chicken-egg.
>>
>> Any configuration advice to achieve better throughput in this scenario of
>> many small messages?
>>
>>
>>> On Wednesday, April 1, 2015 at 10:59:32 AM UTC+3, √ wrote:
>>> Could you share your entire config?
>>>
On Tue, Mar 31, 2015 at 5:31 PM, Ivan Balashov wrote:
> On Tuesday, March 31, 2015 at 11:23:25 AM UTC+3, √ wrote:
> Sounds like you're using too many FJ threads.
I wish it was that simple. For both remoting and actor pool the same
dispatcher is used (4 core box):
> type = Dispatcher
> executor = "fork-join-executor"
> throughput = 1000 // Does this apply to FJ, or only to
> ThreadPoolEx?
> fork-join-executor {
> parallelism-min = 1
> parallelism-max = 4
> }
I get lower CS values if I set parallelism-max=1, maintaining about the
same total throughput only with less cpu burn.
However, it looks like CS should not depend much on whether we have 1 or 4
or 10 FJ threads, most switching must be happing on deeper level, e.g.
controlled by `throughput`.
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> √
>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ
Re: [akka-user] Re: how to tune akka remoting performance
Looks like you're using Java Serialization, a good first start would be to
switch serializers for your messages.
On Wed, Apr 1, 2015 at 3:49 PM, Ivan Balashov wrote:
> Viktor,
>
> Here is the latest configuration that allowed to me reach 30msg/sec
> between 2 x 4cpu hosts in GCE network.
>
> http://goo.gl/sPnjQS
>
> On both sides context switches jumped to 30-50K/sec
>
> Client:
> 0 0 0 1528652 94396 79422800 0 4 468 772 0 0
> 100 0
> 2 0 0 1533988 94408 79422400 0 5 9051 16555 11 4
> 85 0
> 1 0 0 1326560 94412 79422400 0 6 28710 52792 34 13
> 54 0
> 2 0 0 1326188 94412 79422800 0 3 30209 59439 26 8
> 66 0
> 1 0 0 1325444 94420 79422800 0 4 26194 51556 24 6
> 70 0
> 1 0 0 1328172 94424 79422800 0 1 468 725 0 0
> 100 0
>
> Server:
> 3 0 0 827812 30372 206631600 0 3 9762 13188 48 7
> 45 0
> 1 0 0 564080 30388 206631600 0 3 19969 23901 29 13
> 59 0
> 4 0 0 416900 30388 206631600 0 0 24864 31988 39 13
> 48 0
> 1 0 0 241876 30388 206631600 0 3 24113 34808 42 11
> 47 0
> 4 0 0 145280 30404 201443200 0 2 21127 40357 44 6
> 50 0
> 2 0 0 106608 30404 196002400 0 0 19162 36545 47 6
> 47 0
> 4 0 0 138560 30328 186709200 0 6 14574 27803 57 4
> 39 0
> 2 0 0 293616 30336 186709200 0 4 6390 10023 42 12
> 46 0
>
> Couple more things that concerned me:
>
> 1) [WARN] [04/01/2015 12:54:26.125] [systemB-network-dispatcher-5]
> [akka.tcp://
> systemB@10.110.112.155:2001/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FsystemA%4010.110.47.70%3A2001-0/endpointWriter]
> [
> *473494*] buffered messages in EndpointWriter for [akka.tcp://
> systemA@10.110.47.70:2001]. *You should probably implement flow control
> to avoid flooding the remote connection.*
>
> Looks like either side is sending more messages than receiving is capable
> to accept. This however, might be cause by 1M network buffer, which, OTOH
> is needed for higher throughput.
> jFTR, GCE network is quite fast (my last measurement gives me 72Mb/sec,
> while during the test we barely hit 10Mb/sec).
>
> 2) Significant heap pressure, apparently caused by 1) and, as a
> consequence, some GC activity. Profiler gives me estimate of ~700bytes per
> every message in the queue, quickly growing heap. The slower messages get
> processed, the more of them are accumulated, the more cpu is needed for GC,
> chicken-egg.
>
> Any configuration advice to achieve better throughput in this scenario of
> many small messages?
>
>
> On Wednesday, April 1, 2015 at 10:59:32 AM UTC+3, √ wrote:
>>
>> Could you share your entire config?
>>
>> On Tue, Mar 31, 2015 at 5:31 PM, Ivan Balashov wrote:
>>
>>>
>>>
>>> On Tuesday, March 31, 2015 at 11:23:25 AM UTC+3, √ wrote:
Sounds like you're using too many FJ threads.
>>>
>>> I wish it was that simple. For both remoting and actor pool the same
>>> dispatcher is used (4 core box):
>>>
>>> type = Dispatcher
executor = "fork-join-executor"
throughput = 1000 // Does this apply to FJ, or only to
ThreadPoolEx?
fork-join-executor {
parallelism-min = 1
parallelism-max = 4
}
>>>
>>>
>>> I get lower CS values if I set parallelism-max=1, maintaining about the
>>> same total throughput only with less cpu burn.
>>> However, it looks like CS should not depend much on whether we have 1 or
>>> 4 or 10 FJ threads, most switching must be happing on deeper level, e.g.
>>> controlled by `throughput`.
>>>
>>>
>>>
>>>
--
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: http://doc.akka.io/docs/akka/
>>> current/additional/faq.html
>>> >> Search the archives: https://groups.google.com/
>>> group/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Cheers,
>> √
>>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@
Re: [akka-user] Re: how to tune akka remoting performance
Viktor,
Here is the latest configuration that allowed to me reach 30msg/sec between
2 x 4cpu hosts in GCE network.
http://goo.gl/sPnjQS
On both sides context switches jumped to 30-50K/sec
Client:
0 0 0 1528652 94396 79422800 0 4 468 772 0 0 100
0
2 0 0 1533988 94408 79422400 0 5 9051 16555 11 4
85 0
1 0 0 1326560 94412 79422400 0 6 28710 52792 34 13
54 0
2 0 0 1326188 94412 79422800 0 3 30209 59439 26 8
66 0
1 0 0 1325444 94420 79422800 0 4 26194 51556 24 6
70 0
1 0 0 1328172 94424 79422800 0 1 468 725 0 0
100 0
Server:
3 0 0 827812 30372 206631600 0 3 9762 13188 48 7
45 0
1 0 0 564080 30388 206631600 0 3 19969 23901 29 13
59 0
4 0 0 416900 30388 206631600 0 0 24864 31988 39 13
48 0
1 0 0 241876 30388 206631600 0 3 24113 34808 42 11
47 0
4 0 0 145280 30404 201443200 0 2 21127 40357 44 6
50 0
2 0 0 106608 30404 196002400 0 0 19162 36545 47 6
47 0
4 0 0 138560 30328 186709200 0 6 14574 27803 57 4
39 0
2 0 0 293616 30336 186709200 0 4 6390 10023 42 12
46 0
Couple more things that concerned me:
1) [WARN] [04/01/2015 12:54:26.125] [systemB-network-dispatcher-5]
[akka.tcp://systemB@10.110.112.155:2001/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FsystemA%4010.110.47.70%3A2001-0/endpointWriter]
[
*473494*] buffered messages in EndpointWriter for
[akka.tcp://systemA@10.110.47.70:2001]. *You should probably implement flow
control to avoid flooding the remote connection.*
Looks like either side is sending more messages than receiving is capable
to accept. This however, might be cause by 1M network buffer, which, OTOH
is needed for higher throughput.
jFTR, GCE network is quite fast (my last measurement gives me 72Mb/sec,
while during the test we barely hit 10Mb/sec).
2) Significant heap pressure, apparently caused by 1) and, as a
consequence, some GC activity. Profiler gives me estimate of ~700bytes per
every message in the queue, quickly growing heap. The slower messages get
processed, the more of them are accumulated, the more cpu is needed for GC,
chicken-egg.
Any configuration advice to achieve better throughput in this scenario of
many small messages?
On Wednesday, April 1, 2015 at 10:59:32 AM UTC+3, √ wrote:
>
> Could you share your entire config?
>
> On Tue, Mar 31, 2015 at 5:31 PM, Ivan Balashov > wrote:
>
>>
>>
>> On Tuesday, March 31, 2015 at 11:23:25 AM UTC+3, √ wrote:
>>>
>>> Sounds like you're using too many FJ threads.
>>>
>>
>> I wish it was that simple. For both remoting and actor pool the same
>> dispatcher is used (4 core box):
>>
>> type = Dispatcher
>>> executor = "fork-join-executor"
>>> throughput = 1000 // Does this apply to FJ, or only to
>>> ThreadPoolEx?
>>> fork-join-executor {
>>> parallelism-min = 1
>>> parallelism-max = 4
>>> }
>>
>>
>> I get lower CS values if I set parallelism-max=1, maintaining about the
>> same total throughput only with less cpu burn.
>> However, it looks like CS should not depend much on whether we have 1 or
>> 4 or 10 FJ threads, most switching must be happing on deeper level, e.g.
>> controlled by `throughput`.
>>
>>
>>
>>
>>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com .
>> To post to this group, send email to akka...@googlegroups.com
>> .
>> Visit this group at http://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Cheers,
> √
>
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: how to tune akka remoting performance
Could you share your entire config?
On Tue, Mar 31, 2015 at 5:31 PM, Ivan Balashov wrote:
>
>
> On Tuesday, March 31, 2015 at 11:23:25 AM UTC+3, √ wrote:
>>
>> Sounds like you're using too many FJ threads.
>>
>
> I wish it was that simple. For both remoting and actor pool the same
> dispatcher is used (4 core box):
>
> type = Dispatcher
>> executor = "fork-join-executor"
>> throughput = 1000 // Does this apply to FJ, or only to
>> ThreadPoolEx?
>> fork-join-executor {
>> parallelism-min = 1
>> parallelism-max = 4
>> }
>
>
> I get lower CS values if I set parallelism-max=1, maintaining about the
> same total throughput only with less cpu burn.
> However, it looks like CS should not depend much on whether we have 1 or 4
> or 10 FJ threads, most switching must be happing on deeper level, e.g.
> controlled by `throughput`.
>
>
>
>
>> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
--
Cheers,
√
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: how to tune akka remoting performance
On Tuesday, March 31, 2015 at 11:23:25 AM UTC+3, √ wrote:
>
> Sounds like you're using too many FJ threads.
>
I wish it was that simple. For both remoting and actor pool the same
dispatcher is used (4 core box):
type = Dispatcher
> executor = "fork-join-executor"
> throughput = 1000 // Does this apply to FJ, or only to ThreadPoolEx?
> fork-join-executor {
> parallelism-min = 1
> parallelism-max = 4
> }
I get lower CS values if I set parallelism-max=1, maintaining about the
same total throughput only with less cpu burn.
However, it looks like CS should not depend much on whether we have 1 or 4
or 10 FJ threads, most switching must be happing on deeper level, e.g.
controlled by `throughput`.
>
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: how to tune akka remoting performance
Sounds like you're using too many FJ threads. On Mon, Mar 30, 2015 at 6:36 PM, Ivan Balashov wrote: > > While running the test I see that interrupts and context switches are > rather high (for a 4 cpu core box) > > r b swpd free buff cache si sobibo in cs us sy id >> wa > > 5 0 0 2083556 11296 39838000 0 0 15491 39242 58 21 >>> 21 0 >> >> 1 0 0 2110728 11304 39838000 0 7 17791 42876 50 >>> 21 29 0 >> >> 0 0 0 2114272 11304 39838000 0 0 22580 47400 34 >>> 15 51 0 >> >> 3 0 0 2132924 11304 39838000 0 4 26861 53711 26 >>> 10 65 0 >> >> 0 0 0 2119648 11304 39838000 0 2 27225 54133 26 >>> 10 64 0 >> >> > > Trying to adjust "akka.remote.default-remote-dispatcher.throughput" did > not have any noticeable effect. > > Could anyone suggest how to reduce context switches with akka remoting in > case of many small messages? > > Thanks, > > > On Monday, March 30, 2015 at 1:38:40 PM UTC+3, Ivan Balashov wrote: >> >> Hi, >> >> Running simple Ping-Pong test (http://goo.gl/6gyWsd) between two JVMs on >> same box (4 Cores, tested both on my laptop and GCE) gives me total >> throughput 20K msg/sec, which are a CPU bound processes on both sides. >> >> Any ideas how you could manage 60K with almost idle CPU? >> >> Thanks, >> >> On Saturday, August 9, 2014 at 5:08:18 PM UTC+3, Sean Zhong wrote: >>> >>> For Akka 2.3.4, 2 machines, GbE network, one machine contains a sender >>> actor, the other contains a reciver actor. Sender actor will continuously >>> send message (100 bytes each, pre-created in memory) to receiver actor with >>> simple flow control based on sliding window and acking(one ack per 100 >>> messages). >>> >>> I can only get throughput about 60K messages/second , which is much >>> slower than my expectation. The CPU usage is almost idle(3%). Network usage >>> is about 20% of the bandwidth. I use kryo serialization, and >>> singleConsumerUnboundedMailBox, 4 local dispatcher, 4 remote >>> dispatcher, default setting for netty. >>> >>> Is there performance baseline number for akka remoting in your smoke >>> test? >>> >>> >>> >>> >>> -- > >> Read the docs: http://akka.io/docs/ > >> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
