Summary: First, I'll speak in the context of a single instance of Rocket. I'll talk about pound in a bit.
ApacheBench, which I used to test Rocket, unfairly accentuates the benefits of Rocket. httperf allows for a much fairer test. The httperf configuration that Kuba used tested a non-standard situation (while applicable to a project he's working on) that accentuates a known weakness of Rocket over Cherrypy. Even though the single-instance test was inconclusive the multi-instance test implied that Rocket would be slower in the single instance. Because my tests and Kuba's tests focused on polar opposite situations, the numbers were different. Nicholas Piel tested version 1.0.1 which did not include epoll support so his initial conclusions, while correct for the time, are no longer accurate. The difference in situations revolves around how many HTTP requests are pipelined over a single connection. ApacheBench puts them all in a few connections, httperf allows for configuring this. Kuba's benchmark settings put one request per connection. A real-world setting is something around 10, which Nicholas Piel uses. Kuba released another round of tests that follow Nicholas Piel's HTTP/1.1 tests (10 requests per connection). The results showed Rocket as performing slightly faster. Now, let's talk about pound. I've not used pound for any tests before so this was all new information to me. The first test showed 4 instances of Rocket behind pound to be slower than 4 instances of Cherrypy behind pound on a Quad-core machine. There are several possible explanations for this. All of the explanations require more development on Rocket to work around. The difference in performance would not be a show-stopper for me, but others may disagree. I've asked Kuba to retest 4xRocket vs. 4xCherrypy with the second test configuration. Vasile Ermicioi, put in a vote for Rocket to be included in web2py because I'm in the web2py community and there is still plenty of room for Rocket to be optimized (which I noted). Now you're up-to-date. -tim -----Original Message----- From: "mdipierro" <mdipie...@cs.depaul.edu> Sent: Friday, March 19, 2010 9:01pm To: "web2py-users" <web2py@googlegroups.com> Subject: [web2py] Re: benchmarking: rocket vs pound with four rockets had a long day, can somebody provide an execute summary of all the tests? On Mar 19, 3:33 pm, Timothy Farrell <tfarr...@swgen.com> wrote: > Thank you Kuba. Would you mind re-running the 4x pound test like this also? > > On 3/19/2010 3:09 PM, Kuba Kucharski wrote: > > > One instance of each, with 10 calls in a connection as it is closer to > > reallife scenario: > > (numbers speak for themselves) > > > CHERRYPY: > > > r...@kubatron:/home/kuba/httperf-0.9.0/src# ./httperf --hog --server > > 192.168.0.1 --port=8000 ==uri=/vae/default/benchmark2 > > --num-conns=10000 --num-calls=10 > > httperf --hog --client=0/1 --server=192.168.0.1 --port=8000 --uri=/ > > --send-buffer=4096 --recv-buffer=16384 --num-conns=10000 > > --num-calls=10 > > > Maximum connect burst length: 1 > > > Total: connections 10000 requests 100000 replies 100000 test-duration > > 67.659 s > > > Connection rate: 147.8 conn/s (6.8 ms/conn,<=1 concurrent connections) > > Connection time [ms]: min 6.2 avg 6.8 max 10.5 median 6.5 stddev 0.2 > > Connection time [ms]: connect 0.1 > > Connection length [replies/conn]: 10.000 > > > Request rate: 1478.0 req/s (0.7 ms/req) > > Request size [B]: 64.0 > > > Reply rate [replies/s]: min 1474.7 avg 1478.0 max 1480.3 stddev 2.0 (13 > > samples) > > Reply time [ms]: response 0.6 transfer 0.0 > > Reply size [B]: header 205.0 content 66.0 footer 2.0 (total 273.0) > > Reply status: 1xx=0 2xx=0 3xx=100000 4xx=0 5xx=0 > > > CPU time [s]: user 25.67 system 41.99 (user 37.9% system 62.1% total 100.0%) > > Net I/O: 483.5 KB/s (4.0*10^6 bps) > > > Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0 > > Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0 > > > ROCKET: > > > r...@kubatron:/home/kuba/httperf-0.9.0/src# ./httperf --hog --server > > 192.168.0.1 --port=8000 ==uri=/vae/default/benchmark2 > > --num-conns=10000 --num-calls=10 > > httperf --hog --client=0/1 --server=192.168.0.1 --port=8000 --uri=/ > > --send-buffer=4096 --recv-buffer=16384 --num-conns=10000 > > --num-calls=10 > > Maximum connect burst length: 1 > > > Total: connections 10000 requests 100000 replies 100000 test-duration > > 64.760 s > > > Connection rate: 154.4 conn/s (6.5 ms/conn,<=1 concurrent connections) > > Connection time [ms]: min 5.9 avg 6.5 max 72.7 median 6.5 stddev 1.0 > > Connection time [ms]: connect 0.1 > > Connection length [replies/conn]: 10.000 > > > Request rate: 1544.2 req/s (0.6 ms/req) > > Request size [B]: 64.0 > > > Reply rate [replies/s]: min 1526.9 avg 1544.2 max 1555.9 stddev 8.6 (12 > > samples) > > Reply time [ms]: response 0.6 transfer 0.0 > > Reply size [B]: header 216.0 content 66.0 footer 0.0 (total 282.0) > > Reply status: 1xx=0 2xx=0 3xx=100000 4xx=0 5xx=0 > > > CPU time [s]: user 24.18 system 40.58 (user 37.3% system 62.7% total 100.0%) > > Net I/O: 521.8 KB/s (4.3*10^6 bps) > > > Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0 > > Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0 -- You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web...@googlegroups.com. To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/web2py?hl=en. -- You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web...@googlegroups.com. To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/web2py?hl=en.
