Re: [akka-user] Re: Akka http vs Spray performance

[Olga Gorun]

I added scalatra (scalatra 2.4.1 on embedded jetty) to tests set.  Two 
testing tools from three (ab and weighttp) give me approximately the same 
results for all three frameworks in ~70MB/s throughput when I test long 
static result. So  it is similar to network bandwidth saturation. On the 
other hand in case of scalatra wrk is able to get to 86MB/s and iperf from 
client to server with default parameters can give me ~88MB/s.

Can it be networking problem? How do I check it?

Code can be found 
at https://gist.github.com/olga-gorun/825e823027aadb2962b3b7229e4d5d48
Tests results 
- 
https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing

On Thursday, October 6, 2016 at 10:42:04 PM UTC+3, Olga Gorun wrote:
>
> For more information, I give here versions I test with.
>
> ubuntu 14.04, scala 2.11.8, oracle jdk 1.8
>
> akka-http: 2.4.11
>
> spray: 1.3.1 with akka 2.3.6
>
> On Thursday, October 6, 2016 at 10:26:48 PM UTC+3, Olga Gorun wrote:
>>
>> Hi Christian. Thank you for response. I checked that equal Xms/Xmx both 
>> set to more than enough memory (4GB while used heap is not more than 1.5GB 
>> during load test with or without  explicitly defined memory) don't 
>> influence results. I'll try other JVM settings you propose. But generally 
>> speaking I don't understand why I should see something different in GC 
>> behavior. String used as response defined as val in object. It's not 
>> created in some class instantiated per request. So GC should not clear it. 
>> When GC should make more work if response length is increased?
>>
>> On Thursday, October 6, 2016 at 10:03:47 PM UTC+3, Christian Schmitt 
>> wrote:
>>>
>>> garbadge collection I guess. also I didn't see if you've set Xms/Xmx you 
>>> should set them equal. And also try the following: -XX:+UseNUMA 
>>> -XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+PerfDisableSharedMem 
>>> -XX:+ParallelRefProcEnabled paramters, not all of them will be helpful, but 
>>> you should try some of them.
>>>
>>> Am Donnerstag, 6. Oktober 2016 18:08:57 UTC+2 schrieb Olga Gorun:

 I continue playing with given example projects and see strange results. 
 If I increase string length of static response from 7 to 2490 characters 
 throughput is reduced to ~24K rps for both spray and akka-http. 
 Does anybody else see such results? How can they be explained? 



 On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:
>
> Hi Olga,
>
> establishing a connection is a very expensive operation, which is why 
> all modern clients/browsers reuse them for multiple requests. The ab tool 
> needs the -k switch to enable this behavior.
>
> Regards, Roland 
>
> Sent from my iPhone
>
> On 5 Oct 2016, at 01:32, Olga Gorun  wrote:
>
>
> I tried to repeat this benchmark and would be glad to see comments to 
> results I got.  
> Tested porjects: gists from original post with the following versions: 
>
> akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
> 2.11.8, jvm 8.
>
> The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 
> 16GB memory, ulimit - 50). As I can see from jvm and system 
> monitoring, 
> it is far from being limited by CPU, memory, disk IO or networking. OS: 
> Ubuntu 14.04
>
> See results at: 
> https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing
>
> Starting from the fact that my results are different (spray gives 
> better results than akka-http). In addition I see interesting effects 
> when 
> I use different benchmark tools. I also started from wrk, but in addition 
> to throughput I wanted to see failures if exist, and latency percentiles, 
> so I added  tests with weighttp and ab. 
>
> Throughput reported by ab is much different than the rest and shows 
> drastic difference between akka-http and spray. Reported latency also 
> don't 
> have much in common. I can think that ab (as a single-threaded tool) can 
> be 
> bottleneck itself. But how to explain such a difference in throughput for 
> akka-htttp and spray? And how to explain difference in reported latency.
>
> Regards,
> Olga Gorun
>
>
>
>
>
> -- 
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: 
> https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to the Google 
> Groups "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to akka-user+...@googlegroups.com.
> To post to this group, send email to akka...@googlegroups.com.
> Visit this group at https://groups.

Re: [akka-user] Re: Akka http vs Spray performance

[Olga Gorun]

For more information, I give here versions I test with.

ubuntu 14.04, scala 2.11.8, oracle jdk 1.8

akka-http: 2.4.11

spray: 1.3.1 with akka 2.3.6

On Thursday, October 6, 2016 at 10:26:48 PM UTC+3, Olga Gorun wrote:
>
> Hi Christian. Thank you for response. I checked that equal Xms/Xmx both 
> set to more than enough memory (4GB while used heap is not more than 1.5GB 
> during load test with or without  explicitly defined memory) don't 
> influence results. I'll try other JVM settings you propose. But generally 
> speaking I don't understand why I should see something different in GC 
> behavior. String used as response defined as val in object. It's not 
> created in some class instantiated per request. So GC should not clear it. 
> When GC should make more work if response length is increased?
>
> On Thursday, October 6, 2016 at 10:03:47 PM UTC+3, Christian Schmitt wrote:
>>
>> garbadge collection I guess. also I didn't see if you've set Xms/Xmx you 
>> should set them equal. And also try the following: -XX:+UseNUMA 
>> -XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+PerfDisableSharedMem 
>> -XX:+ParallelRefProcEnabled paramters, not all of them will be helpful, but 
>> you should try some of them.
>>
>> Am Donnerstag, 6. Oktober 2016 18:08:57 UTC+2 schrieb Olga Gorun:
>>>
>>> I continue playing with given example projects and see strange results. 
>>> If I increase string length of static response from 7 to 2490 characters 
>>> throughput is reduced to ~24K rps for both spray and akka-http. 
>>> Does anybody else see such results? How can they be explained? 
>>>
>>>
>>>
>>> On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:

 Hi Olga,

 establishing a connection is a very expensive operation, which is why 
 all modern clients/browsers reuse them for multiple requests. The ab tool 
 needs the -k switch to enable this behavior.

 Regards, Roland 

 Sent from my iPhone

 On 5 Oct 2016, at 01:32, Olga Gorun  wrote:


 I tried to repeat this benchmark and would be glad to see comments to 
 results I got.  
 Tested porjects: gists from original post with the following versions: 

 akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
 2.11.8, jvm 8.

 The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 
 16GB memory, ulimit - 50). As I can see from jvm and system 
 monitoring, 
 it is far from being limited by CPU, memory, disk IO or networking. OS: 
 Ubuntu 14.04

 See results at: 
 https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing

 Starting from the fact that my results are different (spray gives 
 better results than akka-http). In addition I see interesting effects when 
 I use different benchmark tools. I also started from wrk, but in addition 
 to throughput I wanted to see failures if exist, and latency percentiles, 
 so I added  tests with weighttp and ab. 

 Throughput reported by ab is much different than the rest and shows 
 drastic difference between akka-http and spray. Reported latency also 
 don't 
 have much in common. I can think that ab (as a single-threaded tool) can 
 be 
 bottleneck itself. But how to explain such a difference in throughput for 
 akka-htttp and spray? And how to explain difference in reported latency.

 Regards,
 Olga Gorun





 -- 
 >> Read the docs: http://akka.io/docs/
 >> Check the FAQ: 
 http://doc.akka.io/docs/akka/current/additional/faq.html
 >> Search the archives: 
 https://groups.google.com/group/akka-user
 --- 
 You received this message because you are subscribed to the Google 
 Groups "Akka User List" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to akka-user+...@googlegroups.com.
 To post to this group, send email to akka...@googlegroups.com.
 Visit this group at https://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.



-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Akka http vs Spray performance

[Olga Gorun]

Hi Christian. Thank you for response. I checked that equal Xms/Xmx both set 
to more than enough memory (4GB while used heap is not more than 1.5GB 
during load test with or without  explicitly defined memory) don't 
influence results. I'll try other JVM settings you propose. But generally 
speaking I don't understand why I should see something different in GC 
behavior. String used as response defined as val in object. It's not 
created in some class instantiated per request. So GC should not clear it. 
When GC should make more work if response length is increased?

On Thursday, October 6, 2016 at 10:03:47 PM UTC+3, Christian Schmitt wrote:
>
> garbadge collection I guess. also I didn't see if you've set Xms/Xmx you 
> should set them equal. And also try the following: -XX:+UseNUMA 
> -XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+PerfDisableSharedMem 
> -XX:+ParallelRefProcEnabled paramters, not all of them will be helpful, but 
> you should try some of them.
>
> Am Donnerstag, 6. Oktober 2016 18:08:57 UTC+2 schrieb Olga Gorun:
>>
>> I continue playing with given example projects and see strange results. 
>> If I increase string length of static response from 7 to 2490 characters 
>> throughput is reduced to ~24K rps for both spray and akka-http. 
>> Does anybody else see such results? How can they be explained? 
>>
>>
>>
>> On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:
>>>
>>> Hi Olga,
>>>
>>> establishing a connection is a very expensive operation, which is why 
>>> all modern clients/browsers reuse them for multiple requests. The ab tool 
>>> needs the -k switch to enable this behavior.
>>>
>>> Regards, Roland 
>>>
>>> Sent from my iPhone
>>>
>>> On 5 Oct 2016, at 01:32, Olga Gorun  wrote:
>>>
>>>
>>> I tried to repeat this benchmark and would be glad to see comments to 
>>> results I got.  
>>> Tested porjects: gists from original post with the following versions: 
>>>
>>> akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
>>> 2.11.8, jvm 8.
>>>
>>> The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB 
>>> memory, ulimit - 50). As I can see from jvm and system monitoring, it 
>>> is far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 
>>> 14.04
>>>
>>> See results at: 
>>> https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing
>>>
>>> Starting from the fact that my results are different (spray gives better 
>>> results than akka-http). In addition I see interesting effects when I use 
>>> different benchmark tools. I also started from wrk, but in addition to 
>>> throughput I wanted to see failures if exist, and latency percentiles, so I 
>>> added  tests with weighttp and ab. 
>>>
>>> Throughput reported by ab is much different than the rest and shows 
>>> drastic difference between akka-http and spray. Reported latency also don't 
>>> have much in common. I can think that ab (as a single-threaded tool) can be 
>>> bottleneck itself. But how to explain such a difference in throughput for 
>>> akka-htttp and spray? And how to explain difference in reported latency.
>>>
>>> Regards,
>>> Olga Gorun
>>>
>>>
>>>
>>>
>>>
>>> -- 
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: 
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >> Search the archives: 
>>> https://groups.google.com/group/akka-user
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Akka http vs Spray performance

[Christian Schmitt]

garbadge collection I guess. also I didn't see if you've set Xms/Xmx you 
should set them equal. And also try the following: -XX:+UseNUMA 
-XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+PerfDisableSharedMem 
-XX:+ParallelRefProcEnabled paramters, not all of them will be helpful, but 
you should try some of them.

Am Donnerstag, 6. Oktober 2016 18:08:57 UTC+2 schrieb Olga Gorun:
>
> I continue playing with given example projects and see strange results. If 
> I increase string length of static response from 7 to 2490 characters 
> throughput is reduced to ~24K rps for both spray and akka-http. 
> Does anybody else see such results? How can they be explained? 
>
>
>
> On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:
>>
>> Hi Olga,
>>
>> establishing a connection is a very expensive operation, which is why all 
>> modern clients/browsers reuse them for multiple requests. The ab tool needs 
>> the -k switch to enable this behavior.
>>
>> Regards, Roland 
>>
>> Sent from my iPhone
>>
>> On 5 Oct 2016, at 01:32, Olga Gorun  wrote:
>>
>>
>> I tried to repeat this benchmark and would be glad to see comments to 
>> results I got.  
>> Tested porjects: gists from original post with the following versions: 
>>
>> akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
>> 2.11.8, jvm 8.
>>
>> The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB 
>> memory, ulimit - 50). As I can see from jvm and system monitoring, it 
>> is far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 
>> 14.04
>>
>> See results at: 
>> https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing
>>
>> Starting from the fact that my results are different (spray gives better 
>> results than akka-http). In addition I see interesting effects when I use 
>> different benchmark tools. I also started from wrk, but in addition to 
>> throughput I wanted to see failures if exist, and latency percentiles, so I 
>> added  tests with weighttp and ab. 
>>
>> Throughput reported by ab is much different than the rest and shows 
>> drastic difference between akka-http and spray. Reported latency also don't 
>> have much in common. I can think that ab (as a single-threaded tool) can be 
>> bottleneck itself. But how to explain such a difference in throughput for 
>> akka-htttp and spray? And how to explain difference in reported latency.
>>
>> Regards,
>> Olga Gorun
>>
>>
>>
>>
>>
>> -- 
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Akka http vs Spray performance

[Olga Gorun]

I continue playing with given example projects and see strange results. If 
I increase string length of static response from 7 to 2490 characters 
throughput is reduced to ~24K rps for both spray and akka-http. 
Does anybody else see such results? How can they be explained? 



On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:
>
> Hi Olga,
>
> establishing a connection is a very expensive operation, which is why all 
> modern clients/browsers reuse them for multiple requests. The ab tool needs 
> the -k switch to enable this behavior.
>
> Regards, Roland 
>
> Sent from my iPhone
>
> On 5 Oct 2016, at 01:32, Olga Gorun > 
> wrote:
>
>
> I tried to repeat this benchmark and would be glad to see comments to 
> results I got.  
> Tested porjects: gists from original post with the following versions: 
>
> akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
> 2.11.8, jvm 8.
>
> The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB 
> memory, ulimit - 50). As I can see from jvm and system monitoring, it 
> is far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 
> 14.04
>
> See results at: 
> https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing
>
> Starting from the fact that my results are different (spray gives better 
> results than akka-http). In addition I see interesting effects when I use 
> different benchmark tools. I also started from wrk, but in addition to 
> throughput I wanted to see failures if exist, and latency percentiles, so I 
> added  tests with weighttp and ab. 
>
> Throughput reported by ab is much different than the rest and shows 
> drastic difference between akka-http and spray. Reported latency also don't 
> have much in common. I can think that ab (as a single-threaded tool) can be 
> bottleneck itself. But how to explain such a difference in throughput for 
> akka-htttp and spray? And how to explain difference in reported latency.
>
> Regards,
> Olga Gorun
>
>
>
>
>
> -- 
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+...@googlegroups.com .
> To post to this group, send email to akka...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Akka http vs Spray performance

[Olga Gorun]

Hi rkurn,

Thank you for reply. You are right. keep-alive makes the difference. I 
updated in google spreadshit.

On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:
>
> Hi Olga,
>
> establishing a connection is a very expensive operation, which is why all 
> modern clients/browsers reuse them for multiple requests. The ab tool needs 
> the -k switch to enable this behavior.
>
> Regards, Roland 
>
> Sent from my iPhone
>
> On 5 Oct 2016, at 01:32, Olga Gorun > 
> wrote:
>
>
> I tried to repeat this benchmark and would be glad to see comments to 
> results I got.  
> Tested porjects: gists from original post with the following versions: 
>
> akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
> 2.11.8, jvm 8.
>
> The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB 
> memory, ulimit - 50). As I can see from jvm and system monitoring, it 
> is far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 
> 14.04
>
> See results at: 
> https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing
>
> Starting from the fact that my results are different (spray gives better 
> results than akka-http). In addition I see interesting effects when I use 
> different benchmark tools. I also started from wrk, but in addition to 
> throughput I wanted to see failures if exist, and latency percentiles, so I 
> added  tests with weighttp and ab. 
>
> Throughput reported by ab is much different than the rest and shows 
> drastic difference between akka-http and spray. Reported latency also don't 
> have much in common. I can think that ab (as a single-threaded tool) can be 
> bottleneck itself. But how to explain such a difference in throughput for 
> akka-htttp and spray? And how to explain difference in reported latency.
>
> Regards,
> Olga Gorun
>
>
>
>
>
> -- 
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+...@googlegroups.com .
> To post to this group, send email to akka...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: Akka http vs Spray performance

[Roland Kuhn]

Hi Olga,

establishing a connection is a very expensive operation, which is why all 
modern clients/browsers reuse them for multiple requests. The ab tool needs the 
-k switch to enable this behavior.

Regards, Roland 

Sent from my iPhone

> On 5 Oct 2016, at 01:32, Olga Gorun  wrote:
> 
> 
> I tried to repeat this benchmark and would be glad to see comments to results 
> I got.  
> Tested porjects: gists from original post with the following versions: 
> 
> akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 2.11.8, 
> jvm 8.
> 
> The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB 
> memory, ulimit - 50). As I can see from jvm and system monitoring, it is 
> far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 14.04
> 
> See results at: 
> https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing
> 
> Starting from the fact that my results are different (spray gives better 
> results than akka-http). In addition I see interesting effects when I use 
> different benchmark tools. I also started from wrk, but in addition to 
> throughput I wanted to see failures if exist, and latency percentiles, so I 
> added  tests with weighttp and ab. 
> 
> Throughput reported by ab is much different than the rest and shows drastic 
> difference between akka-http and spray. Reported latency also don't have much 
> in common. I can think that ab (as a single-threaded tool) can be bottleneck 
> itself. But how to explain such a difference in throughput for akka-htttp and 
> spray? And how to explain difference in reported latency.
> 
> Regards,
> Olga Gorun
> 
> 
> 
>
> 
> -- 
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> >> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Akka http vs Spray performance

[Olga Gorun]

I tried to repeat this benchmark and would be glad to see comments to 
results I got.  
Tested porjects: gists from original post with the following versions: 

akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 
2.11.8, jvm 8.

The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB 
memory, ulimit - 50). As I can see from jvm and system monitoring, it 
is far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 
14.04

See results 
at: 
https://docs.google.com/spreadsheets/d/1yuFD7WDOzhWB5_Ob7XgAfFAdWt48SrthbvZSZ45GE-k/edit?usp=sharing

Starting from the fact that my results are different (spray gives better 
results than akka-http). In addition I see interesting effects when I use 
different benchmark tools. I also started from wrk, but in addition to 
throughput I wanted to see failures if exist, and latency percentiles, so I 
added  tests with weighttp and ab. 

Throughput reported by ab is much different than the rest and shows drastic 
difference between akka-http and spray. Reported latency also don't have 
much in common. I can think that ab (as a single-threaded tool) can be 
bottleneck itself. But how to explain such a difference in throughput for 
akka-htttp and spray? And how to explain difference in reported latency.

Regards,
Olga Gorun



   

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Akka http vs Spray performance

[Владимир Морозов]

I perform some test for 2.4.9 release

> wrk -t30 -c64 
http://127.0.0.1:9000/dictionaries/hello/suggestions?ngr=hond
Running 10s test @ 
http://127.0.0.1:9000/dictionaries/hello/suggestions?ngr=hond
  30 threads and 64 connections
  Thread Stats   Avg  Stdev Max   +/- Stdev
Latency10.00ms   37.94ms 640.12ms   94.49%
Req/Sec 1.66k   738.92 4.73k63.21%
  487404 requests in 10.07s, 69.26MB read
Requests/sec:  48405.49
Transfer/sec:  6.88MB

> wrk -t30 -c64 
http://127.0.0.1:9000/dictionaries/hello/suggestions?ngr=hond
Running 10s test @ 
http://127.0.0.1:9000/dictionaries/hello/suggestions?ngr=hond
  30 threads and 64 connections
  Thread Stats   Avg  Stdev Max   +/- Stdev
Latency 6.58ms   18.10ms 212.50ms   92.59%
Req/Sec 1.94k   672.59 7.34k69.63%
  577713 requests in 10.08s, 82.09MB read
Requests/sec:  57308.72
Transfer/sec:  8.14MB

> wrk -t30 -c64 
http://127.0.0.1:9000/dictionaries/hello/suggestions?ngr=hond
Running 10s test @ 
http://127.0.0.1:9000/dictionaries/hello/suggestions?ngr=hond
  30 threads and 64 connections
  Thread Stats   Avg  Stdev Max   +/- Stdev
Latency 7.87ms   28.93ms 456.46ms   95.19%
Req/Sec 1.89k   639.70 4.98k74.75%
  561358 requests in 10.09s, 79.77MB read
Requests/sec:  55630.70
Transfer/sec:  7.90MB

пятница, 2 сентября 2016 г., 11:19:03 UTC+3 пользователь Konrad 'ktoso' 
Malawski написал:
>
> Figured I should link to the 2.4.9 annoucement from this old thread (as it 
> shows up high in google).
> In 2.4.9 we did quite some improvements, annoucement here:
> http://akka.io/news/2016/08/19/akka-2.4.9-released.html  
> 
>
> Highlights of the performance improvements include:
>
>- Overall Akka HTTP throughput and transfer rate has been improved by 
>30-40%
>- Performance is on-par or better than Spray.
>   - Matching it both in raw throughput as well as latency 
>   distribution across a spectrum of request rates.
>   - We measured an overall improvement of ~14% over Spray
>- Short lived connections throughput, which remains to be the 
>worst-case scenario for Akka HTTP thought remains rare in the real world 
>(due to connection pooling), has been doubled.
>- Given our EC2 infrastructure (m4.2xlarge instances) the server 
>currently reaches a maximum of ~160.000 “ping / pong” requests per second 
>(measured using 100 concurrent connections).
>
> Happy hakking!
>
> -- Konrad
>
> W dniu środa, 27 maja 2015 12:44:49 UTC+1 użytkownik zergood napisał:
>>
>> I've done little benchmarks to compare spray and akka-http performance. I 
>> use default jvm and akka settings. So you can see that there is 
>> an significant performance difference between it.
>>
>> code:
>> spray https://gist.github.com/zergood/18bae0adc2e774c31233. 
>> akka-http https://gist.github.com/zergood/53977efd500985a34ea1.
>>
>> versions:
>> spray 1.3.3 
>> akka-http 1.0-RC3
>> scala 2.11.6
>> java 1.8
>>
>> wrk output for spray:
>> Running 1m test @ 
>> http://127.0.0.1:8080/dictionaries/hello/suggestions?ngr=hond
>>   30 threads and 64 connections
>>   Thread Stats   Avg  Stdev Max   +/- Stdev
>> Latency 2.14ms9.82ms  78.22ms   98.22%
>> Req/Sec 2.55k   609.68 4.22k78.12%
>>   4322357 requests in 1.00m, 614.20MB read
>> Requests/sec:  72044.97
>> Transfer/sec: 10.24MB
>>
>> wrk output for akka-http:
>> Running 1m test @ 
>> http://127.0.0.1:3535/dictionaries/hello/suggestions?ngr=hond
>>   30 threads and 64 connections
>>   Thread Stats   Avg  Stdev Max   +/- Stdev
>> Latency 5.39ms6.82ms 108.07ms   92.80%
>> Req/Sec   454.43126.73   679.00 77.77%
>>   811836 requests in 1.00m, 115.36MB read
>> Requests/sec:  13531.62
>> Transfer/sec:  1.92MB
>>
>> Is there any akka-http config options to increase performance to the same 
>> level as spray?
>>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Akka http vs Spray performance

[Konrad 'ktoso' Malawski]

Figured I should link to the 2.4.9 annoucement from this old thread (as it 
shows up high in google).
In 2.4.9 we did quite some improvements, annoucement here:
http://akka.io/news/2016/08/19/akka-2.4.9-released.html  


Highlights of the performance improvements include:

   - Overall Akka HTTP throughput and transfer rate has been improved by 
   30-40%
   - Performance is on-par or better than Spray.
  - Matching it both in raw throughput as well as latency distribution 
  across a spectrum of request rates.
  - We measured an overall improvement of ~14% over Spray
   - Short lived connections throughput, which remains to be the worst-case 
   scenario for Akka HTTP thought remains rare in the real world (due to 
   connection pooling), has been doubled.
   - Given our EC2 infrastructure (m4.2xlarge instances) the server 
   currently reaches a maximum of ~160.000 “ping / pong” requests per second 
   (measured using 100 concurrent connections).

Happy hakking!

-- Konrad

W dniu środa, 27 maja 2015 12:44:49 UTC+1 użytkownik zergood napisał:
>
> I've done little benchmarks to compare spray and akka-http performance. I 
> use default jvm and akka settings. So you can see that there is 
> an significant performance difference between it.
>
> code:
> spray https://gist.github.com/zergood/18bae0adc2e774c31233. 
> akka-http https://gist.github.com/zergood/53977efd500985a34ea1.
>
> versions:
> spray 1.3.3 
> akka-http 1.0-RC3
> scala 2.11.6
> java 1.8
>
> wrk output for spray:
> Running 1m test @ 
> http://127.0.0.1:8080/dictionaries/hello/suggestions?ngr=hond
>   30 threads and 64 connections
>   Thread Stats   Avg  Stdev Max   +/- Stdev
> Latency 2.14ms9.82ms  78.22ms   98.22%
> Req/Sec 2.55k   609.68 4.22k78.12%
>   4322357 requests in 1.00m, 614.20MB read
> Requests/sec:  72044.97
> Transfer/sec: 10.24MB
>
> wrk output for akka-http:
> Running 1m test @ 
> http://127.0.0.1:3535/dictionaries/hello/suggestions?ngr=hond
>   30 threads and 64 connections
>   Thread Stats   Avg  Stdev Max   +/- Stdev
> Latency 5.39ms6.82ms 108.07ms   92.80%
> Req/Sec   454.43126.73   679.00 77.77%
>   811836 requests in 1.00m, 115.36MB read
> Requests/sec:  13531.62
> Transfer/sec:  1.92MB
>
> Is there any akka-http config options to increase performance to the same 
> level as spray?
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: Akka http vs Spray performance

[Jakub Kahovec]

Hi,

first thank you so much for making such a great piece of software. As the 
2.4.0 is out, have you made any progress on improving performance of akka 
http or at least is there any roadmap for this ?

Thanks

Jakub


On Wednesday, May 27, 2015 at 1:44:49 PM UTC+2, zergood wrote:
>
> I've done little benchmarks to compare spray and akka-http performance. I 
> use default jvm and akka settings. So you can see that there is 
> an significant performance difference between it.
>
> code:
> spray https://gist.github.com/zergood/18bae0adc2e774c31233. 
> akka-http https://gist.github.com/zergood/53977efd500985a34ea1.
>
> versions:
> spray 1.3.3 
> akka-http 1.0-RC3
> scala 2.11.6
> java 1.8
>
> wrk output for spray:
> Running 1m test @ 
> http://127.0.0.1:8080/dictionaries/hello/suggestions?ngr=hond
>   30 threads and 64 connections
>   Thread Stats   Avg  Stdev Max   +/- Stdev
> Latency 2.14ms9.82ms  78.22ms   98.22%
> Req/Sec 2.55k   609.68 4.22k78.12%
>   4322357 requests in 1.00m, 614.20MB read
> Requests/sec:  72044.97
> Transfer/sec: 10.24MB
>
> wrk output for akka-http:
> Running 1m test @ 
> http://127.0.0.1:3535/dictionaries/hello/suggestions?ngr=hond
>   30 threads and 64 connections
>   Thread Stats   Avg  Stdev Max   +/- Stdev
> Latency 5.39ms6.82ms 108.07ms   92.80%
> Req/Sec   454.43126.73   679.00 77.77%
>   811836 requests in 1.00m, 115.36MB read
> Requests/sec:  13531.62
> Transfer/sec:  1.92MB
>
> Is there any akka-http config options to increase performance to the same 
> level as spray?
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.