On Friday, September 1, 2017 at 8:19:19 PM UTC+3, dbohdan wrote:
> My exceptions were [...]
This, of course, should say "expectations".
On Friday, September 1, 2017 at 9:38:25 PM UTC+3, Neil Van Dyke wrote:
> Thank you very much for doing this work, D. Bohdan.
You're welcome! I had fun doing it.
> This performance of Racket SCGI+nginx relative to the others you tested
is surprising to me, since I made the Racket `scgi` package for
particular non-performance requirements, and performance was really
secondary.
Thanks for making the 'scgi package. I rather like the SCGI protocol. It's a
pity that it isn't as widely supported as FastCGI, considering that it's much
simpler to implement (second only to plain old CGI), but still has a
performance profile similar to FastCGI's.
> Not to look a gift horse in the mouth, [...]
No worries. The horse is given very much with that in mind. :-) To address your
specific concerns:
> errors can cause good performance numbers. Sometimes I used
> JMeter instead of `ab` to rule out that cause of bad numbers in
> performance testing (as well as to implement some testing).
I think the SCGI benchmark works correctly because of the data sizes that
ApacheBench reports. For example, here is the request data from one run:
> Complete requests: 178572
> Failed requests: 0
> Total transferred: 755002416 bytes
> HTML transferred: 733038060 bytes
733038060 / 178572 = 4105, which is exactly the size of the text message the
application serves. The same is true of other data I've examined so far (5
runs). To help detect errors, the benchmark is also programmed to abort if the
first request to an application doesn't serve exactly the right text (see
`run-when-ready.sh`) or if ApacheBench sees enough of nginx's status 502 pages,
which are served when the SCGI server doesn't respond correctly or at all.
I'll look into using JMeter in addition to ApacheBench.
> the OS pushing into swap
Good point. I thought I'd already disabled the containers' access to swap, but
apparently it didn't work because of a thing with cgroups. The "benchmarked"
container still must have used swap, because it began to run out of memory for
some applications when I disabled the swap on the VM itself. I've increased
"benchmarked's" memory quota to 768 MiB and added a recommendation to disable
the swap system-wide in README.md.
> sporadic network latency (though looks like you might've controlled for
that one)
The application and the load generator communicate through a virtual network
between two Docker containers on the same host, so this should not be an issue.
> some other OS/hardware/network burp outside of your Racket
process(es).
Such burps are possible, and even likely, because I run the VM on a machine I
use for other tasks. I try to ensure no taxing tasks run alongside the
benchmark and mitigate the inevitable CPU spikes by simply benchmarking every
application for longer (three minutes by default).
On Friday, September 1, 2017 at 9:51:13 PM UTC+3, Neil Van Dyke wrote:
> `#:scgi-max-allow-wait
Thanks for the suggestion. This turned out to be the key to SCGI performance.
Increasing #:scgi-max-allow-wait from 1 to 4 (default), 16, 64, 256 gives a
moderate increase in throughput (from ~2350 req/s to ~2900 req/s), but
*decreases* the maximum latency in a very major way (from ~50000 ms to ~250
ms). See scgi-max-allow.md in the attachments for some detailed data samples.
The effect levels out at 256. There isn't an obvious difference between 256,
1024, 4096, and 16384. I've pushed the update to run the tests at
#:scgi-max-allow-wait 256.
Besides scgi-max-allow.md, I've also attached the results for 1) a five-minute
benchmark with one concurrent connection, 768 MiB RAM, no swap,
#:scgi-max-allow-wait 4; 2) a rerun of the first benchmark with the updated
settings (three minutes, 100 connections, 768 MiB RAM, no swap,
#:scgi-max-allow-wait 256).
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
> grep Requests results/*
results/caddy.txt:Requests per second: 4117.47 [#/sec] (mean)
results/compojure.txt:Requests per second: 5127.98 [#/sec] (mean)
results/flask.txt:Requests per second: 1077.55 [#/sec] (mean)
results/guile.txt:Requests per second: 2053.89 [#/sec] (mean)
results/plug.txt:Requests per second: 5042.51 [#/sec] (mean)
results/scgi.txt:Requests per second: 2760.30 [#/sec] (mean)
results/sinatra.txt:Requests per second: 312.66 [#/sec] (mean)
results/stateful.txt:Requests per second: 529.93 [#/sec] (mean)
results/stateless.txt:Requests per second: 620.32 [#/sec] (mean)
> grep -A 29 'Concurrency Level' results/*
results/caddy.txt:Concurrency Level: 100
results/caddy.txt-Time taken for tests: 180.001 seconds
results/caddy.txt-Complete requests: 741147
results/caddy.txt-Failed requests: 0
results/caddy.txt-Total transferred: 3206206248 bytes
results/caddy.txt-HTML transferred: 3042412540 bytes
results/caddy.txt-Requests per second: 4117.47 [#/sec] (mean)
results/caddy.txt-Time per request: 24.287 [ms] (mean)
results/caddy.txt-Time per request: 0.243 [ms] (mean, across all
concurrent requests)
results/caddy.txt-Transfer rate: 17394.73 [Kbytes/sec] received
results/caddy.txt-
results/caddy.txt-Connection Times (ms)
results/caddy.txt- min mean[+/-sd] median max
results/caddy.txt-Connect: 0 1 0.9 0 21
results/caddy.txt-Processing: 0 24 11.7 22 207
results/caddy.txt-Waiting: 0 23 11.2 21 207
results/caddy.txt-Total: 0 24 11.6 23 207
results/caddy.txt-WARNING: The median and mean for the initial connection time
are not within a normal deviation
results/caddy.txt- These results are probably not that reliable.
results/caddy.txt-
results/caddy.txt-Percentage of the requests served within a certain time (ms)
results/caddy.txt- 50% 23
results/caddy.txt- 66% 28
results/caddy.txt- 75% 31
results/caddy.txt- 80% 33
results/caddy.txt- 90% 39
results/caddy.txt- 95% 45
results/caddy.txt- 98% 52
results/caddy.txt- 99% 56
results/caddy.txt- 100% 207 (longest request)
--
results/compojure.txt:Concurrency Level: 100
results/compojure.txt-Time taken for tests: 180.000 seconds
results/compojure.txt-Complete requests: 923037
results/compojure.txt-Failed requests: 0
results/compojure.txt-Total transferred: 3913676880 bytes
results/compojure.txt-HTML transferred: 3789066885 bytes
results/compojure.txt-Requests per second: 5127.98 [#/sec] (mean)
results/compojure.txt-Time per request: 19.501 [ms] (mean)
results/compojure.txt-Time per request: 0.195 [ms] (mean, across all
concurrent requests)
results/compojure.txt-Transfer rate: 21233.05 [Kbytes/sec] received
results/compojure.txt-
results/compojure.txt-Connection Times (ms)
results/compojure.txt- min mean[+/-sd] median max
results/compojure.txt-Connect: 0 9 90.7 0 3048
results/compojure.txt-Processing: 0 11 8.0 9 424
results/compojure.txt-Waiting: 0 10 7.9 8 424
results/compojure.txt-Total: 1 19 91.8 10 3074
results/compojure.txt-
results/compojure.txt-Percentage of the requests served within a certain time
(ms)
results/compojure.txt- 50% 10
results/compojure.txt- 66% 12
results/compojure.txt- 75% 14
results/compojure.txt- 80% 15
results/compojure.txt- 90% 19
results/compojure.txt- 95% 24
results/compojure.txt- 98% 35
results/compojure.txt- 99% 61
results/compojure.txt- 100% 3074 (longest request)
--
results/flask.txt:Concurrency Level: 100
results/flask.txt-Time taken for tests: 180.000 seconds
results/flask.txt-Complete requests: 193959
results/flask.txt-Failed requests: 0
results/flask.txt-Total transferred: 826653258 bytes
results/flask.txt-HTML transferred: 796201695 bytes
results/flask.txt-Requests per second: 1077.55 [#/sec] (mean)
results/flask.txt-Time per request: 92.803 [ms] (mean)
results/flask.txt-Time per request: 0.928 [ms] (mean, across all
concurrent requests)
results/flask.txt-Transfer rate: 4484.87 [Kbytes/sec] received
results/flask.txt-
results/flask.txt-Connection Times (ms)
results/flask.txt- min mean[+/-sd] median max
results/flask.txt-Connect: 0 0 0.2 0 10
results/flask.txt-Processing: 3 93 4.5 92 146
results/flask.txt-Waiting: 3 92 4.5 92 146
results/flask.txt-Total: 13 93 4.4 92 146
results/flask.txt-
results/flask.txt-Percentage of the requests served within a certain time (ms)
results/flask.txt- 50% 92
results/flask.txt- 66% 93
results/flask.txt- 75% 94
results/flask.txt- 80% 95
results/flask.txt- 90% 97
results/flask.txt- 95% 99
results/flask.txt- 98% 103
results/flask.txt- 99% 112
results/flask.txt- 100% 146 (longest request)
--
results/guile.txt:Concurrency Level: 100
results/guile.txt-Time taken for tests: 180.006 seconds
results/guile.txt-Complete requests: 369712
results/guile.txt-Failed requests: 0
results/guile.txt-Total transferred: 1547614432 bytes
results/guile.txt-HTML transferred: 1517667760 bytes
results/guile.txt-Requests per second: 2053.89 [#/sec] (mean)
results/guile.txt-Time per request: 48.688 [ms] (mean)
results/guile.txt-Time per request: 0.487 [ms] (mean, across all
concurrent requests)
results/guile.txt-Transfer rate: 8396.08 [Kbytes/sec] received
results/guile.txt-
results/guile.txt-Connection Times (ms)
results/guile.txt- min mean[+/-sd] median max
results/guile.txt-Connect: 0 0 0.2 0 10
results/guile.txt-Processing: 18 49 5.3 50 96
results/guile.txt-Waiting: 17 49 5.3 50 96
results/guile.txt-Total: 23 49 5.3 50 96
results/guile.txt-
results/guile.txt-Percentage of the requests served within a certain time (ms)
results/guile.txt- 50% 50
results/guile.txt- 66% 51
results/guile.txt- 75% 52
results/guile.txt- 80% 53
results/guile.txt- 90% 54
results/guile.txt- 95% 56
results/guile.txt- 98% 60
results/guile.txt- 99% 63
results/guile.txt- 100% 96 (longest request)
--
results/plug.txt:Concurrency Level: 100
results/plug.txt-Time taken for tests: 180.000 seconds
results/plug.txt-Complete requests: 907654
results/plug.txt-Failed requests: 0
results/plug.txt-Total transferred: 3895655260 bytes
results/plug.txt-HTML transferred: 3725923775 bytes
results/plug.txt-Requests per second: 5042.51 [#/sec] (mean)
results/plug.txt-Time per request: 19.831 [ms] (mean)
results/plug.txt-Time per request: 0.198 [ms] (mean, across all
concurrent requests)
results/plug.txt-Transfer rate: 21135.23 [Kbytes/sec] received
results/plug.txt-
results/plug.txt-Connection Times (ms)
results/plug.txt- min mean[+/-sd] median max
results/plug.txt-Connect: 0 1 1.4 1 32
results/plug.txt-Processing: 0 19 5.9 18 80
results/plug.txt-Waiting: 0 18 6.0 17 79
results/plug.txt-Total: 1 20 5.7 19 80
results/plug.txt-
results/plug.txt-Percentage of the requests served within a certain time (ms)
results/plug.txt- 50% 19
results/plug.txt- 66% 21
results/plug.txt- 75% 22
results/plug.txt- 80% 23
results/plug.txt- 90% 26
results/plug.txt- 95% 30
results/plug.txt- 98% 35
results/plug.txt- 99% 40
results/plug.txt- 100% 80 (longest request)
--
results/scgi.txt:Concurrency Level: 100
results/scgi.txt-Time taken for tests: 180.000 seconds
results/scgi.txt-Complete requests: 496854
results/scgi.txt-Failed requests: 0
results/scgi.txt-Total transferred: 2100698712 bytes
results/scgi.txt-HTML transferred: 2039585670 bytes
results/scgi.txt-Requests per second: 2760.30 [#/sec] (mean)
results/scgi.txt-Time per request: 36.228 [ms] (mean)
results/scgi.txt-Time per request: 0.362 [ms] (mean, across all
concurrent requests)
results/scgi.txt-Transfer rate: 11397.02 [Kbytes/sec] received
results/scgi.txt-
results/scgi.txt-Connection Times (ms)
results/scgi.txt- min mean[+/-sd] median max
results/scgi.txt-Connect: 0 1 1.4 0 19
results/scgi.txt-Processing: 0 35 17.6 35 263
results/scgi.txt-Waiting: 0 35 17.6 34 263
results/scgi.txt-Total: 0 36 17.3 35 264
results/scgi.txt-
results/scgi.txt-Percentage of the requests served within a certain time (ms)
results/scgi.txt- 50% 35
results/scgi.txt- 66% 39
results/scgi.txt- 75% 42
results/scgi.txt- 80% 46
results/scgi.txt- 90% 55
results/scgi.txt- 95% 63
results/scgi.txt- 98% 76
results/scgi.txt- 99% 90
results/scgi.txt- 100% 264 (longest request)
--
results/sinatra.txt:Concurrency Level: 100
results/sinatra.txt-Time taken for tests: 180.002 seconds
results/sinatra.txt-Complete requests: 56279
results/sinatra.txt-Failed requests: 0
results/sinatra.txt-Total transferred: 243298440 bytes
results/sinatra.txt-HTML transferred: 231029400 bytes
results/sinatra.txt-Requests per second: 312.66 [#/sec] (mean)
results/sinatra.txt-Time per request: 319.839 [ms] (mean)
results/sinatra.txt-Time per request: 3.198 [ms] (mean, across all
concurrent requests)
results/sinatra.txt-Transfer rate: 1319.96 [Kbytes/sec] received
results/sinatra.txt-
results/sinatra.txt-Connection Times (ms)
results/sinatra.txt- min mean[+/-sd] median max
results/sinatra.txt-Connect: 0 0 0.2 0 10
results/sinatra.txt-Processing: 10 319 215.2 279 2305
results/sinatra.txt-Waiting: 3 314 215.3 269 2305
results/sinatra.txt-Total: 10 319 215.2 279 2306
results/sinatra.txt-
results/sinatra.txt-Percentage of the requests served within a certain time (ms)
results/sinatra.txt- 50% 279
results/sinatra.txt- 66% 334
results/sinatra.txt- 75% 402
results/sinatra.txt- 80% 454
results/sinatra.txt- 90% 611
results/sinatra.txt- 95% 752
results/sinatra.txt- 98% 931
results/sinatra.txt- 99% 1054
results/sinatra.txt- 100% 2306 (longest request)
--
results/stateful.txt:Concurrency Level: 100
results/stateful.txt-Time taken for tests: 180.001 seconds
results/stateful.txt-Complete requests: 95388
results/stateful.txt-Failed requests: 0
results/stateful.txt-Total transferred: 407128788 bytes
results/stateful.txt-HTML transferred: 391580055 bytes
results/stateful.txt-Requests per second: 529.93 [#/sec] (mean)
results/stateful.txt-Time per request: 188.704 [ms] (mean)
results/stateful.txt-Time per request: 1.887 [ms] (mean, across all
concurrent requests)
results/stateful.txt-Transfer rate: 2208.81 [Kbytes/sec] received
results/stateful.txt-
results/stateful.txt-Connection Times (ms)
results/stateful.txt- min mean[+/-sd] median max
results/stateful.txt-Connect: 0 0 0.3 0 11
results/stateful.txt-Processing: 1 188 199.0 165 1271
results/stateful.txt-Waiting: 1 184 197.9 161 1269
results/stateful.txt-Total: 1 189 199.0 165 1271
results/stateful.txt-
results/stateful.txt-Percentage of the requests served within a certain time
(ms)
results/stateful.txt- 50% 165
results/stateful.txt- 66% 192
results/stateful.txt- 75% 273
results/stateful.txt- 80% 282
results/stateful.txt- 90% 316
results/stateful.txt- 95% 663
results/stateful.txt- 98% 912
results/stateful.txt- 99% 1014
results/stateful.txt- 100% 1271 (longest request)
--
results/stateless.txt:Concurrency Level: 100
results/stateless.txt-Time taken for tests: 180.011 seconds
results/stateless.txt-Complete requests: 111665
results/stateless.txt-Failed requests: 0
results/stateless.txt-Total transferred: 476658776 bytes
results/stateless.txt-HTML transferred: 458454610 bytes
results/stateless.txt-Requests per second: 620.32 [#/sec] (mean)
results/stateless.txt-Time per request: 161.206 [ms] (mean)
results/stateless.txt-Time per request: 1.612 [ms] (mean, across all
concurrent requests)
results/stateless.txt-Transfer rate: 2585.88 [Kbytes/sec] received
results/stateless.txt-
results/stateless.txt-Connection Times (ms)
results/stateless.txt- min mean[+/-sd] median max
results/stateless.txt-Connect: 0 1 0.5 1 21
results/stateless.txt-Processing: 14 160 132.7 138 1397
results/stateless.txt-Waiting: 6 133 122.3 110 1320
results/stateless.txt-Total: 15 160 132.8 138 1398
results/stateless.txt-
results/stateless.txt-Percentage of the requests served within a certain time
(ms)
results/stateless.txt- 50% 138
results/stateless.txt- 66% 160
results/stateless.txt- 75% 174
results/stateless.txt- 80% 183
results/stateless.txt- 90% 215
results/stateless.txt- 95% 276
results/stateless.txt- 98% 710
results/stateless.txt- 99% 872
results/stateless.txt- 100% 1398 (longest request)
> grep Requests results/*
results/caddy.txt:Requests per second: 1942.18 [#/sec] (mean)
results/compojure.txt:Requests per second: 2173.21 [#/sec] (mean)
results/flask.txt:Requests per second: 1010.19 [#/sec] (mean)
results/guile.txt:Requests per second: 1693.35 [#/sec] (mean)
results/plug.txt:Requests per second: 2382.33 [#/sec] (mean)
results/scgi.txt:Requests per second: 1735.37 [#/sec] (mean)
results/sinatra.txt:Requests per second: 349.71 [#/sec] (mean)
results/stateful.txt:Requests per second: 742.28 [#/sec] (mean)
results/stateless.txt:Requests per second: 749.27 [#/sec] (mean)
> grep -A 21 'Requests per second' results/*
results/caddy.txt:Requests per second: 1942.18 [#/sec] (mean)
results/caddy.txt-Time per request: 0.515 [ms] (mean)
results/caddy.txt-Time per request: 0.515 [ms] (mean, across all
concurrent requests)
results/caddy.txt-Transfer rate: 8204.94 [Kbytes/sec] received
results/caddy.txt-
results/caddy.txt-Connection Times (ms)
results/caddy.txt- min mean[+/-sd] median max
results/caddy.txt-Connect: 0 0 0.0 0 10
results/caddy.txt-Processing: 0 0 0.2 0 10
results/caddy.txt-Waiting: 0 0 0.1 0 10
results/caddy.txt-Total: 0 0 0.2 0 10
results/caddy.txt-
results/caddy.txt-Percentage of the requests served within a certain time (ms)
results/caddy.txt- 50% 0
results/caddy.txt- 66% 0
results/caddy.txt- 75% 1
results/caddy.txt- 80% 1
results/caddy.txt- 90% 1
results/caddy.txt- 95% 1
results/caddy.txt- 98% 1
results/caddy.txt- 99% 1
results/caddy.txt- 100% 10 (longest request)
--
results/compojure.txt:Requests per second: 2173.21 [#/sec] (mean)
results/compojure.txt-Time per request: 0.460 [ms] (mean)
results/compojure.txt-Time per request: 0.460 [ms] (mean, across all
concurrent requests)
results/compojure.txt-Transfer rate: 8998.45 [Kbytes/sec] received
results/compojure.txt-
results/compojure.txt-Connection Times (ms)
results/compojure.txt- min mean[+/-sd] median max
results/compojure.txt-Connect: 0 0 0.0 0 4
results/compojure.txt-Processing: 0 0 0.2 0 19
results/compojure.txt-Waiting: 0 0 0.2 0 19
results/compojure.txt-Total: 0 0 0.2 0 19
results/compojure.txt-
results/compojure.txt-Percentage of the requests served within a certain time
(ms)
results/compojure.txt- 50% 0
results/compojure.txt- 66% 0
results/compojure.txt- 75% 0
results/compojure.txt- 80% 0
results/compojure.txt- 90% 0
results/compojure.txt- 95% 1
results/compojure.txt- 98% 1
results/compojure.txt- 99% 1
results/compojure.txt- 100% 19 (longest request)
--
results/flask.txt:Requests per second: 1010.19 [#/sec] (mean)
results/flask.txt-Time per request: 0.990 [ms] (mean)
results/flask.txt-Time per request: 0.990 [ms] (mean, across all
concurrent requests)
results/flask.txt-Transfer rate: 4204.51 [Kbytes/sec] received
results/flask.txt-
results/flask.txt-Connection Times (ms)
results/flask.txt- min mean[+/-sd] median max
results/flask.txt-Connect: 0 0 0.0 0 4
results/flask.txt-Processing: 1 1 0.2 1 22
results/flask.txt-Waiting: 0 1 0.2 1 21
results/flask.txt-Total: 1 1 0.2 1 22
results/flask.txt-
results/flask.txt-Percentage of the requests served within a certain time (ms)
results/flask.txt- 50% 1
results/flask.txt- 66% 1
results/flask.txt- 75% 1
results/flask.txt- 80% 1
results/flask.txt- 90% 1
results/flask.txt- 95% 1
results/flask.txt- 98% 1
results/flask.txt- 99% 1
results/flask.txt- 100% 22 (longest request)
--
results/guile.txt:Requests per second: 1693.35 [#/sec] (mean)
results/guile.txt-Time per request: 0.591 [ms] (mean)
results/guile.txt-Time per request: 0.591 [ms] (mean, across all
concurrent requests)
results/guile.txt-Transfer rate: 6922.23 [Kbytes/sec] received
results/guile.txt-
results/guile.txt-Connection Times (ms)
results/guile.txt- min mean[+/-sd] median max
results/guile.txt-Connect: 0 0 0.0 0 6
results/guile.txt-Processing: 0 0 0.6 0 19
results/guile.txt-Waiting: 0 0 0.6 0 19
results/guile.txt-Total: 0 1 0.6 0 19
results/guile.txt-WARNING: The median and mean for the total time are not
within a normal deviation
results/guile.txt- These results are probably not that reliable.
results/guile.txt-
results/guile.txt-Percentage of the requests served within a certain time (ms)
results/guile.txt- 50% 0
results/guile.txt- 66% 0
results/guile.txt- 75% 1
results/guile.txt- 80% 1
results/guile.txt- 90% 1
results/guile.txt- 95% 1
results/guile.txt- 98% 1
--
results/plug.txt:Requests per second: 2382.33 [#/sec] (mean)
results/plug.txt-Time per request: 0.420 [ms] (mean)
results/plug.txt-Time per request: 0.420 [ms] (mean, across all
concurrent requests)
results/plug.txt-Transfer rate: 9985.31 [Kbytes/sec] received
results/plug.txt-
results/plug.txt-Connection Times (ms)
results/plug.txt- min mean[+/-sd] median max
results/plug.txt-Connect: 0 0 0.0 0 4
results/plug.txt-Processing: 0 0 0.1 0 11
results/plug.txt-Waiting: 0 0 0.1 0 10
results/plug.txt-Total: 0 0 0.1 0 11
results/plug.txt-
results/plug.txt-Percentage of the requests served within a certain time (ms)
results/plug.txt- 50% 0
results/plug.txt- 66% 0
results/plug.txt- 75% 0
results/plug.txt- 80% 0
results/plug.txt- 90% 0
results/plug.txt- 95% 0
results/plug.txt- 98% 1
results/plug.txt- 99% 1
results/plug.txt- 100% 11 (longest request)
--
results/scgi.txt:Requests per second: 1735.37 [#/sec] (mean)
results/scgi.txt-Time per request: 0.576 [ms] (mean)
results/scgi.txt-Time per request: 0.576 [ms] (mean, across all
concurrent requests)
results/scgi.txt-Transfer rate: 7165.17 [Kbytes/sec] received
results/scgi.txt-
results/scgi.txt-Connection Times (ms)
results/scgi.txt- min mean[+/-sd] median max
results/scgi.txt-Connect: 0 0 0.0 0 7
results/scgi.txt-Processing: 0 0 0.5 0 115
results/scgi.txt-Waiting: 0 0 0.5 0 115
results/scgi.txt-Total: 0 1 0.5 1 115
results/scgi.txt-
results/scgi.txt-Percentage of the requests served within a certain time (ms)
results/scgi.txt- 50% 1
results/scgi.txt- 66% 1
results/scgi.txt- 75% 1
results/scgi.txt- 80% 1
results/scgi.txt- 90% 1
results/scgi.txt- 95% 1
results/scgi.txt- 98% 1
results/scgi.txt- 99% 1
results/scgi.txt- 100% 115 (longest request)
--
results/sinatra.txt:Requests per second: 349.71 [#/sec] (mean)
results/sinatra.txt-Time per request: 2.860 [ms] (mean)
results/sinatra.txt-Time per request: 2.860 [ms] (mean, across all
concurrent requests)
results/sinatra.txt-Transfer rate: 1476.37 [Kbytes/sec] received
results/sinatra.txt-
results/sinatra.txt-Connection Times (ms)
results/sinatra.txt- min mean[+/-sd] median max
results/sinatra.txt-Connect: 0 0 0.0 0 5
results/sinatra.txt-Processing: 2 3 0.4 3 35
results/sinatra.txt-Waiting: 2 3 0.4 2 35
results/sinatra.txt-Total: 2 3 0.4 3 35
results/sinatra.txt-ERROR: The median and mean for the waiting time are more
than twice the standard
results/sinatra.txt- deviation apart. These results are NOT reliable.
results/sinatra.txt-
results/sinatra.txt-Percentage of the requests served within a certain time (ms)
results/sinatra.txt- 50% 3
results/sinatra.txt- 66% 3
results/sinatra.txt- 75% 3
results/sinatra.txt- 80% 3
results/sinatra.txt- 90% 3
results/sinatra.txt- 95% 3
results/sinatra.txt- 98% 4
--
results/stateful.txt:Requests per second: 742.28 [#/sec] (mean)
results/stateful.txt-Time per request: 1.347 [ms] (mean)
results/stateful.txt-Time per request: 1.347 [ms] (mean, across all
concurrent requests)
results/stateful.txt-Transfer rate: 3093.81 [Kbytes/sec] received
results/stateful.txt-
results/stateful.txt-Connection Times (ms)
results/stateful.txt- min mean[+/-sd] median max
results/stateful.txt-Connect: 0 0 0.0 0 4
results/stateful.txt-Processing: 1 1 3.5 1 421
results/stateful.txt-Waiting: 0 1 2.8 1 387
results/stateful.txt-Total: 1 1 3.5 1 421
results/stateful.txt-
results/stateful.txt-Percentage of the requests served within a certain time
(ms)
results/stateful.txt- 50% 1
results/stateful.txt- 66% 1
results/stateful.txt- 75% 1
results/stateful.txt- 80% 1
results/stateful.txt- 90% 1
results/stateful.txt- 95% 1
results/stateful.txt- 98% 2
results/stateful.txt- 99% 2
results/stateful.txt- 100% 421 (longest request)
--
results/stateless.txt:Requests per second: 749.27 [#/sec] (mean)
results/stateless.txt-Time per request: 1.335 [ms] (mean)
results/stateless.txt-Time per request: 1.335 [ms] (mean, across all
concurrent requests)
results/stateless.txt-Transfer rate: 3122.93 [Kbytes/sec] received
results/stateless.txt-
results/stateless.txt-Connection Times (ms)
results/stateless.txt- min mean[+/-sd] median max
results/stateless.txt-Connect: 0 0 0.0 0 7
results/stateless.txt-Processing: 1 1 2.8 1 387
results/stateless.txt-Waiting: 0 1 2.6 1 216
results/stateless.txt-Total: 1 1 2.8 1 387
results/stateless.txt-
results/stateless.txt-Percentage of the requests served within a certain time
(ms)
results/stateless.txt- 50% 1
results/stateless.txt- 66% 1
results/stateless.txt- 75% 1
results/stateless.txt- 80% 1
results/stateless.txt- 90% 1
results/stateless.txt- 95% 1
results/stateless.txt- 98% 2
results/stateless.txt- 99% 2
results/stateless.txt- 100% 387 (longest request)
60-second runs.
# scgi-max-allow-wait 1
## run 1
Requests per second: 2301.85 [#/sec] (mean)
Time per request: 43.443 [ms] (mean)
Time per request: 0.434 [ms] (mean, across all concurrent requests)
Transfer rate: 9504.14 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 13
Processing: 0 31 420.4 1 54393
Waiting: 0 31 420.4 1 54393
Total: 0 31 420.4 1 54394
Percentage of the requests served within a certain time (ms)
50% 1
66% 2
75% 2
80% 2
90% 3
95% 5
98% 1002
99% 1024
100% 54394 (longest request)
## run 2
Requests per second: 2373.20 [#/sec] (mean)
Time per request: 42.137 [ms] (mean)
Time per request: 0.421 [ms] (mean, across all concurrent requests)
Transfer rate: 9798.70 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 13
Processing: 0 28 305.7 1 54836
Waiting: 0 28 305.7 1 54835
Total: 0 28 305.7 1 54837
Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 2
80% 2
90% 3
95% 4
98% 19
99% 1024
100% 54837 (longest request)
# scgi-max-allow-wait 4
## run 1
Requests per second: 2691.02 [#/sec] (mean)
Time per request: 37.161 [ms] (mean)
Time per request: 0.372 [ms] (mean, across all concurrent requests)
Transfer rate: 11110.95 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 9
Processing: 0 36 279.0 3 27390
Waiting: 0 36 279.0 3 27390
Total: 0 37 279.0 3 27391
Percentage of the requests served within a certain time (ms)
50% 3
66% 4
75% 4
80% 5
90% 7
95% 10
98% 1017
99% 1030
100% 27391 (longest request)
## run 2
Requests per second: 2592.17 [#/sec] (mean)
Time per request: 38.578 [ms] (mean)
Time per request: 0.386 [ms] (mean, across all concurrent requests)
Transfer rate: 10702.88 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 12
Processing: 0 37 322.3 3 27311
Waiting: 0 37 322.2 3 27310
Total: 0 37 322.3 3 27311
Percentage of the requests served within a certain time (ms)
50% 3
66% 4
75% 5
80% 5
90% 7
95% 10
98% 1016
99% 1029
100% 27311 (longest request)
# scgi-max-allow-wait 16
## run 1
Requests per second: 2838.87 [#/sec] (mean)
Time per request: 35.225 [ms] (mean)
Time per request: 0.352 [ms] (mean, across all concurrent requests)
Transfer rate: 11721.42 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 11
Processing: 0 35 170.1 7 4327
Waiting: 0 35 170.1 7 4327
Total: 0 35 170.1 7 4328
Percentage of the requests served within a certain time (ms)
50% 7
66% 9
75% 11
80% 12
90% 16
95% 20
98% 1017
99% 1031
100% 4328 (longest request)
## run 2
Requests per second: 2785.81 [#/sec] (mean)
Time per request: 35.896 [ms] (mean)
Time per request: 0.359 [ms] (mean, across all concurrent requests)
Transfer rate: 11502.34 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 8
Processing: 0 35 170.6 7 3054
Waiting: 0 35 170.6 7 3054
Total: 0 36 170.7 8 3056
Percentage of the requests served within a certain time (ms)
50% 8
66% 10
75% 11
80% 13
90% 16
95% 21
98% 1018
99% 1032
100% 3056 (longest request)
# scgi-max-allow-wait 64
## run 1
Requests per second: 2920.96 [#/sec] (mean)
Time per request: 34.235 [ms] (mean)
Time per request: 0.342 [ms] (mean, across all concurrent requests)
Transfer rate: 12060.39 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.1 0 19
Processing: 0 33 92.5 24 1257
Waiting: 0 33 92.5 24 1257
Total: 0 34 92.5 24 1257
Percentage of the requests served within a certain time (ms)
50% 24
66% 28
75% 30
80% 32
90% 40
95% 48
98% 63
99% 137
100% 1257 (longest request)
## run 2
Requests per second: 2889.72 [#/sec] (mean)
Time per request: 34.605 [ms] (mean)
Time per request: 0.346 [ms] (mean, across all concurrent requests)
Transfer rate: 11931.38 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.1 0 16
Processing: 0 34 96.2 24 3036
Waiting: 0 33 96.2 24 3036
Total: 0 35 96.2 24 3036
Percentage of the requests served within a certain time (ms)
50% 24
66% 28
75% 31
80% 33
90% 40
95% 48
98% 64
99% 142
100% 3036 (longest request)
Requests per second: 2880.75 [#/sec] (mean)
Time per request: 34.713 [ms] (mean)
Time per request: 0.347 [ms] (mean, across all concurrent requests)
Transfer rate: 11894.35 [Kbytes/sec] received
# scgi-max-allow-wait 256
## run 1
Requests per second: 2880.75 [#/sec] (mean)
Time per request: 34.713 [ms] (mean)
Time per request: 0.347 [ms] (mean, across all concurrent requests)
Transfer rate: 11894.35 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.4 0 18
Processing: 0 34 17.0 33 205
Waiting: 0 33 17.0 33 205
Total: 0 35 16.8 34 209
Percentage of the requests served within a certain time (ms)
50% 34
66% 37
75% 41
80% 45
90% 52
95% 60
98% 72
99% 86
100% 209 (longest request)
## run 2
Requests per second: 2926.96 [#/sec] (mean)
Time per request: 34.165 [ms] (mean)
Time per request: 0.342 [ms] (mean, across all concurrent requests)
Transfer rate: 12085.21 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.3 0 19
Processing: 0 33 16.2 33 265
Waiting: 0 33 16.3 33 265
Total: 0 34 16.1 33 266
Percentage of the requests served within a certain time (ms)
50% 33
66% 36
75% 39
80% 42
90% 51
95% 59
98% 72
99% 84
100% 266 (longest request)
# scgi-max-allow-wait 1024
## run 1
Requests per second: 2834.44 [#/sec] (mean)
Time per request: 35.280 [ms] (mean)
Time per request: 0.353 [ms] (mean, across all concurrent requests)
Transfer rate: 11703.40 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.3 0 18
Processing: 0 34 16.4 34 230
Waiting: 0 34 16.4 33 230
Total: 0 35 16.2 34 233
Percentage of the requests served within a certain time (ms)
50% 34
66% 37
75% 41
80% 45
90% 53
95% 60
98% 72
99% 85
100% 233 (longest request)
## run 2
Requests per second: 2976.19 [#/sec] (mean)
Time per request: 33.600 [ms] (mean)
Time per request: 0.336 [ms] (mean, across all concurrent requests)
Transfer rate: 12288.40 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.4 0 19
Processing: 0 33 16.3 33 217
Waiting: 0 32 16.4 32 216
Total: 0 34 16.1 33 217
Percentage of the requests served within a certain time (ms)
50% 33
66% 36
75% 39
80% 43
90% 51
95% 58
98% 70
99% 83
100% 217 (longest request)
