subject:"Re\: \[akka\-user\] Akka HTTP performance in 2.4.9"

Re: [akka-user] Akka HTTP performance in 2.4.9

2016-09-06 Thread Adam

Thanks!

Adding -k indeed fixes this right away:

ab -k -c 400 -n 10 -m GET http://127.0.0.1:3000/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1 requests
Completed 2 requests
Completed 3 requests
Completed 4 requests
Completed 5 requests
Completed 6 requests
Completed 7 requests
Completed 8 requests
Completed 9 requests
Completed 10 requests
Finished 10 requests


Server Software:akka-http/2.4.9
Server Hostname:127.0.0.1
Server Port:3000

Document Path:  /
Document Length:2 bytes

Concurrency Level:  400
Time taken for tests:   3.401 seconds
Complete requests:  10
Failed requests:0
Keep-Alive requests:10
Total transferred:  1670 bytes
HTML transferred:   20 bytes
Requests per second:29403.12 [#/sec] (mean)
Time per request:   13.604 [ms] (mean)
Time per request:   0.034 [ms] (mean, across all concurrent requests)
Transfer rate:  4795.23 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:00   2.8  0 509
Processing: 0   11  62.6  61566
Waiting:0   11  62.6  61566
Total:  0   11  62.8  61566

Percentage of the requests served within a certain time (ms)
  50%  6
  66%  8
  75%  8
  80%  9
  90% 13
  95% 16
  98% 26
  99% 32
 100%   1566 (longest request)


P.S.  - I wasn't familiar with wrk, but, of course, I realize running on 
the local machine is not really valid.
I only used this method to pick up on really basic issues (like this 
example of not using persistent connections illustrates perfectly)



On Tuesday, September 6, 2016 at 5:39:02 PM UTC+3, Konrad Malawski wrote:
>
> Without looking at the code, two things to fix right away in your 
> methodology:
>
> 1) You're running `ab` without persistent connections (also known as 
> "useless mode").
> Please run: `ab -k` to use keep-alive connections, which is what all http 
> clients and browsers do.
>
> 2) Please do not benchmark using the same host for running the app and 
> load generator...
> You're stealing resources from the http server as you're load generator is 
> trying to hit it as hard as it can, and the other way around.
> The benchmark will be completely skewed by that.
>
> ab is also not very good in general, use wrk instead - a more modern load 
> generator.
>
> -- 
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
> On 6 September 2016 at 16:34:22, Adam (adam...@gmail.com ) 
> wrote:
>
> Hi,
>
> I've previously seen bad performance form Akka HTTP for a simple ping/pong 
> scenario (which I realize according to the release notes is still a 
> relatively bad scenario for Akka HTTP), but the results I'm getting are bad 
> enough that I'm wondering if my test is even valid.
>
> Here's my code:
>
> package com.example.rest
>
> import akka.NotUsed
> import akka.actor.ActorSystem
> import akka.http.scaladsl.Http
> import akka.http.scaladsl.model._
> import akka.stream.scaladsl.{Flow, Keep, RunnableGraph, Sink, Source}
> import akka.stream.{ActorMaterializer, ActorMaterializerSettings, Fusing}
>
> import scala.concurrent.Future
> import scala.io.StdIn
>
> object WebServer extends App {
>   implicit val system = ActorSystem()
>   implicit val materializer =  
> ActorMaterializer(ActorMaterializerSettings(system).withAutoFusing(false))
>   implicit val executionContext = system.dispatcher
>
>   val response = HttpResponse(entity = 
> HttpEntity(ContentTypes.`text/plain(UTF-8)`,"Ok"))
>   val requestHandler: HttpRequest => HttpResponse = {
> case _ => response
>   }
>
>   val flow: Flow[HttpRequest, HttpResponse, NotUsed] = Flow[HttpRequest] map 
> requestHandler
>   val prefused = Fusing.aggressive(flow)
>   val httpHandler: Flow[HttpRequest, HttpResponse, NotUsed] = 
> Flow.fromGraph(prefused)
>
>   if (args(0) == "http"){
> runHttpServer()
>   } else {
> runPingPong(args(1).toInt)
>   }
>
>   def runHttpServer() = {
> val bindingFuture = Http().bindAndHandle(handler = httpHandler, interface 
> = "127.0.0.1  ", port = 3000)
>
> println("Type RETURN to exit")
> StdIn.readLine()
>
> bindingFuture
>   .flatMap(_.unbind()) // trigger unbinding from the port
>   .onComplete(_ => system.terminate())
>   }
>
>   def runPingPong(times: Int) = {
> val ping = HttpRequest().withUri(Uri("/"))
>
> val graph: RunnableGraph[Future[Int]] = Source(1 to times)
>   .map(_ => ping)
>   .via(httpHandler)
>   .map(_ => 1)
>   .toMat(Sink.fold[Int, Int](0)((a, b) => a + 1))(Keep.right)
>
> val startTime = System.nanoTime()
> val count: Future[Int] = graph.run()
>
>

Re: [akka-user] Akka HTTP performance in 2.4.9

2016-09-06 Thread Viktor Klang

And it can be considered recommended to use wrk2 over wrk (coordinated
omission correction)

On Tue, Sep 6, 2016 at 4:43 PM, Viktor Klang  wrote:

> Also, I'd recommend avoiding App for benches, I don't know how it works
> nowadays but I recall seeing issues with inlining (OSR) back in the day.
>
> On Tue, Sep 6, 2016 at 4:38 PM, Konrad Malawski <
> konrad.malaw...@lightbend.com> wrote:
>
>> Without looking at the code, two things to fix right away in your
>> methodology:
>>
>> 1) You're running `ab` without persistent connections (also known as
>> "useless mode").
>> Please run: `ab -k` to use keep-alive connections, which is what all http
>> clients and browsers do.
>>
>> 2) Please do not benchmark using the same host for running the app and
>> load generator...
>> You're stealing resources from the http server as you're load generator
>> is trying to hit it as hard as it can, and the other way around.
>> The benchmark will be completely skewed by that.
>>
>> ab is also not very good in general, use wrk instead - a more modern load
>> generator.
>>
>> --
>> Konrad `ktoso` Malawski
>> Akka  @ Lightbend 
>>
>> On 6 September 2016 at 16:34:22, Adam (adamho...@gmail.com) wrote:
>>
>> Hi,
>>
>> I've previously seen bad performance form Akka HTTP for a simple
>> ping/pong scenario (which I realize according to the release notes is still
>> a relatively bad scenario for Akka HTTP), but the results I'm getting are
>> bad enough that I'm wondering if my test is even valid.
>>
>> Here's my code:
>>
>> package com.example.rest
>>
>> import akka.NotUsed
>> import akka.actor.ActorSystem
>> import akka.http.scaladsl.Http
>> import akka.http.scaladsl.model._
>> import akka.stream.scaladsl.{Flow, Keep, RunnableGraph, Sink, Source}
>> import akka.stream.{ActorMaterializer, ActorMaterializerSettings, Fusing}
>>
>> import scala.concurrent.Future
>> import scala.io.StdIn
>>
>> object WebServer extends App {
>>   implicit val system = ActorSystem()
>>   implicit val materializer =  
>> ActorMaterializer(ActorMaterializerSettings(system).withAutoFusing(false))
>>   implicit val executionContext = system.dispatcher
>>
>>   val response = HttpResponse(entity = 
>> HttpEntity(ContentTypes.`text/plain(UTF-8)`,"Ok"))
>>   val requestHandler: HttpRequest => HttpResponse = {
>> case _ => response
>>   }
>>
>>   val flow: Flow[HttpRequest, HttpResponse, NotUsed] = Flow[HttpRequest] map 
>> requestHandler
>>   val prefused = Fusing.aggressive(flow)
>>   val httpHandler: Flow[HttpRequest, HttpResponse, NotUsed] = 
>> Flow.fromGraph(prefused)
>>
>>   if (args(0) == "http"){
>> runHttpServer()
>>   } else {
>> runPingPong(args(1).toInt)
>>   }
>>
>>   def runHttpServer() = {
>> val bindingFuture = Http().bindAndHandle(handler = httpHandler, 
>> interface = "127.0.0.1  ", port = 3000)
>>
>> println("Type RETURN to exit")
>> StdIn.readLine()
>>
>> bindingFuture
>>   .flatMap(_.unbind()) // trigger unbinding from the port
>>   .onComplete(_ => system.terminate())
>>   }
>>
>>   def runPingPong(times: Int) = {
>> val ping = HttpRequest().withUri(Uri("/"))
>>
>> val graph: RunnableGraph[Future[Int]] = Source(1 to times)
>>   .map(_ => ping)
>>   .via(httpHandler)
>>   .map(_ => 1)
>>   .toMat(Sink.fold[Int, Int](0)((a, b) => a + 1))(Keep.right)
>>
>> val startTime = System.nanoTime()
>> val count: Future[Int] = graph.run()
>>
>> count.onComplete(res => {
>>   if (res.isFailure) {
>> Console.err.println(s"failed with: ${res.failed}")
>>   } else {
>> println(s"ran for ${res.get} times")
>> val et = System.nanoTime() - startTime
>> val etMillis = et / 100.0
>> println(s"et: ${etMillis}ms")
>>   }
>>   system.terminate()
>> })
>>   }
>> }
>>
>> The in memory scenario (runPingPong) yields the following results on my 
>> local windows machine:
>>
>> ran for 10 times
>> et: 322.150515ms
>>
>>
>> Running ab to test actual local HTTP calls get me this:
>>
>> ab -c 400 -n 10 -m GET http://127.0.0.1:3000/
>>
>> This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>>
>> Benchmarking 127.0.0.1 (be patient)
>> Completed 1 requests
>> Completed 2 requests
>> Completed 3 requests
>> Completed 4 requests
>> Completed 5 requests
>> Completed 6 requests
>> Completed 7 requests
>> Completed 8 requests
>> Completed 9 requests
>> Completed 10 requests
>> Finished 10 requests
>>
>>
>> Server Software:akka-http/2.4.9
>> Server Hostname:127.0.0.1
>> Server Port:3000
>>
>> Document Path:  /
>> Document Length:2 bytes
>>
>> Concurrency Level:  400
>> Time taken for tests:   139.083 seconds
>> Complete requests:  10
>>

Re: [akka-user] Akka HTTP performance in 2.4.9

2016-09-06 Thread Viktor Klang

Also, I'd recommend avoiding App for benches, I don't know how it works
nowadays but I recall seeing issues with inlining (OSR) back in the day.

On Tue, Sep 6, 2016 at 4:38 PM, Konrad Malawski <
konrad.malaw...@lightbend.com> wrote:

> Without looking at the code, two things to fix right away in your
> methodology:
>
> 1) You're running `ab` without persistent connections (also known as
> "useless mode").
> Please run: `ab -k` to use keep-alive connections, which is what all http
> clients and browsers do.
>
> 2) Please do not benchmark using the same host for running the app and
> load generator...
> You're stealing resources from the http server as you're load generator is
> trying to hit it as hard as it can, and the other way around.
> The benchmark will be completely skewed by that.
>
> ab is also not very good in general, use wrk instead - a more modern load
> generator.
>
> --
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
> On 6 September 2016 at 16:34:22, Adam (adamho...@gmail.com) wrote:
>
> Hi,
>
> I've previously seen bad performance form Akka HTTP for a simple ping/pong
> scenario (which I realize according to the release notes is still a
> relatively bad scenario for Akka HTTP), but the results I'm getting are bad
> enough that I'm wondering if my test is even valid.
>
> Here's my code:
>
> package com.example.rest
>
> import akka.NotUsed
> import akka.actor.ActorSystem
> import akka.http.scaladsl.Http
> import akka.http.scaladsl.model._
> import akka.stream.scaladsl.{Flow, Keep, RunnableGraph, Sink, Source}
> import akka.stream.{ActorMaterializer, ActorMaterializerSettings, Fusing}
>
> import scala.concurrent.Future
> import scala.io.StdIn
>
> object WebServer extends App {
>   implicit val system = ActorSystem()
>   implicit val materializer =  
> ActorMaterializer(ActorMaterializerSettings(system).withAutoFusing(false))
>   implicit val executionContext = system.dispatcher
>
>   val response = HttpResponse(entity = 
> HttpEntity(ContentTypes.`text/plain(UTF-8)`,"Ok"))
>   val requestHandler: HttpRequest => HttpResponse = {
> case _ => response
>   }
>
>   val flow: Flow[HttpRequest, HttpResponse, NotUsed] = Flow[HttpRequest] map 
> requestHandler
>   val prefused = Fusing.aggressive(flow)
>   val httpHandler: Flow[HttpRequest, HttpResponse, NotUsed] = 
> Flow.fromGraph(prefused)
>
>   if (args(0) == "http"){
> runHttpServer()
>   } else {
> runPingPong(args(1).toInt)
>   }
>
>   def runHttpServer() = {
> val bindingFuture = Http().bindAndHandle(handler = httpHandler, interface 
> = "127.0.0.1  ", port = 3000)
>
> println("Type RETURN to exit")
> StdIn.readLine()
>
> bindingFuture
>   .flatMap(_.unbind()) // trigger unbinding from the port
>   .onComplete(_ => system.terminate())
>   }
>
>   def runPingPong(times: Int) = {
> val ping = HttpRequest().withUri(Uri("/"))
>
> val graph: RunnableGraph[Future[Int]] = Source(1 to times)
>   .map(_ => ping)
>   .via(httpHandler)
>   .map(_ => 1)
>   .toMat(Sink.fold[Int, Int](0)((a, b) => a + 1))(Keep.right)
>
> val startTime = System.nanoTime()
> val count: Future[Int] = graph.run()
>
> count.onComplete(res => {
>   if (res.isFailure) {
> Console.err.println(s"failed with: ${res.failed}")
>   } else {
> println(s"ran for ${res.get} times")
> val et = System.nanoTime() - startTime
> val etMillis = et / 100.0
> println(s"et: ${etMillis}ms")
>   }
>   system.terminate()
> })
>   }
> }
>
> The in memory scenario (runPingPong) yields the following results on my local 
> windows machine:
>
> ran for 10 times
> et: 322.150515ms
>
>
> Running ab to test actual local HTTP calls get me this:
>
> ab -c 400 -n 10 -m GET http://127.0.0.1:3000/
>
> This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Licensed to The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking 127.0.0.1 (be patient)
> Completed 1 requests
> Completed 2 requests
> Completed 3 requests
> Completed 4 requests
> Completed 5 requests
> Completed 6 requests
> Completed 7 requests
> Completed 8 requests
> Completed 9 requests
> Completed 10 requests
> Finished 10 requests
>
>
> Server Software:akka-http/2.4.9
> Server Hostname:127.0.0.1
> Server Port:3000
>
> Document Path:  /
> Document Length:2 bytes
>
> Concurrency Level:  400
> Time taken for tests:   139.083 seconds
> Complete requests:  10
> Failed requests:0
> Total transferred:  1620 bytes
> HTML transferred:   20 bytes
> Requests per second:719.00 [#/sec] (mean)
> Time per request:   556.332 [ms] (mean)
> Time per request:   1.391 [ms] (mean, across all concurrent requests)
> Transfer rate:  113.75 [Kbytes/sec]

Re: [akka-user] Akka HTTP performance in 2.4.9

2016-09-06 Thread Konrad Malawski

Without looking at the code, two things to fix right away in your
methodology:

1) You're running `ab` without persistent connections (also known as
"useless mode").
Please run: `ab -k` to use keep-alive connections, which is what all http
clients and browsers do.

2) Please do not benchmark using the same host for running the app and load
generator...
You're stealing resources from the http server as you're load generator is
trying to hit it as hard as it can, and the other way around.
The benchmark will be completely skewed by that.

ab is also not very good in general, use wrk instead - a more modern load
generator.

-- 
Konrad `ktoso` Malawski
Akka  @ Lightbend 

On 6 September 2016 at 16:34:22, Adam (adamho...@gmail.com) wrote:

Hi,

I've previously seen bad performance form Akka HTTP for a simple ping/pong
scenario (which I realize according to the release notes is still a
relatively bad scenario for Akka HTTP), but the results I'm getting are bad
enough that I'm wondering if my test is even valid.

Here's my code:

package com.example.rest

import akka.NotUsed
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.scaladsl.{Flow, Keep, RunnableGraph, Sink, Source}
import akka.stream.{ActorMaterializer, ActorMaterializerSettings, Fusing}

import scala.concurrent.Future
import scala.io.StdIn

object WebServer extends App {
  implicit val system = ActorSystem()
  implicit val materializer =
ActorMaterializer(ActorMaterializerSettings(system).withAutoFusing(false))
  implicit val executionContext = system.dispatcher

  val response = HttpResponse(entity =
HttpEntity(ContentTypes.`text/plain(UTF-8)`,"Ok"))
  val requestHandler: HttpRequest => HttpResponse = {
case _ => response
  }

  val flow: Flow[HttpRequest, HttpResponse, NotUsed] =
Flow[HttpRequest] map requestHandler
  val prefused = Fusing.aggressive(flow)
  val httpHandler: Flow[HttpRequest, HttpResponse, NotUsed] =
Flow.fromGraph(prefused)

  if (args(0) == "http"){
runHttpServer()
  } else {
runPingPong(args(1).toInt)
  }

  def runHttpServer() = {
val bindingFuture = Http().bindAndHandle(handler = httpHandler,
interface = "127.0.0.1  ", port = 3000)

println("Type RETURN to exit")
StdIn.readLine()

bindingFuture
  .flatMap(_.unbind()) // trigger unbinding from the port
  .onComplete(_ => system.terminate())
  }

  def runPingPong(times: Int) = {
val ping = HttpRequest().withUri(Uri("/"))

val graph: RunnableGraph[Future[Int]] = Source(1 to times)
  .map(_ => ping)
  .via(httpHandler)
  .map(_ => 1)
  .toMat(Sink.fold[Int, Int](0)((a, b) => a + 1))(Keep.right)

val startTime = System.nanoTime()
val count: Future[Int] = graph.run()

count.onComplete(res => {
  if (res.isFailure) {
Console.err.println(s"failed with: ${res.failed}")
  } else {
println(s"ran for ${res.get} times")
val et = System.nanoTime() - startTime
val etMillis = et / 100.0
println(s"et: ${etMillis}ms")
  }
  system.terminate()
})
  }
}

The in memory scenario (runPingPong) yields the following results on
my local windows machine:

ran for 10 times
et: 322.150515ms


Running ab to test actual local HTTP calls get me this:

ab -c 400 -n 10 -m GET http://127.0.0.1:3000/

This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1 requests
Completed 2 requests
Completed 3 requests
Completed 4 requests
Completed 5 requests
Completed 6 requests
Completed 7 requests
Completed 8 requests
Completed 9 requests
Completed 10 requests
Finished 10 requests


Server Software:akka-http/2.4.9
Server Hostname:127.0.0.1
Server Port:3000

Document Path:  /
Document Length:2 bytes

Concurrency Level:  400
Time taken for tests:   139.083 seconds
Complete requests:  10
Failed requests:0
Total transferred:  1620 bytes
HTML transferred:   20 bytes
Requests per second:719.00 [#/sec] (mean)
Time per request:   556.332 [ms] (mean)
Time per request:   1.391 [ms] (mean, across all concurrent requests)
Transfer rate:  113.75 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:01  24.7  0 593
Processing:13  552 394.65622144
Waiting:2  388 347.15261583
Total: 13  554 395.25622144

Percentage of the requests served within a certain time (ms)
  50%562
  66%586
  75%639
  80%   1047
  90%   1065
  95%   1096
  98%   1560
  99%   1574
 100%   2144 (longest request)


I was able to use the same approach with other frameworks (e.g. Play

Re: [akka-user] Akka HTTP performance in 2.4.9

Re: [akka-user] Akka HTTP performance in 2.4.9

Re: [akka-user] Akka HTTP performance in 2.4.9

Re: [akka-user] Akka HTTP performance in 2.4.9

4 matches

Site Navigation

Mail list logo

Footer information