This is a different test than the one I presented. The test I presented
was run on Windows with one instance and tested with ApacheBench. I've
looked at httperf a little and it seems to be a more realistic test than
ApacheBench.
Due to the nature of how Rocket handles listening sockets, it is a
little slower at accepting connections compared to Cherrypy. Nicholas
Piel's test handles 10 requests per connection whereas Apachebench would
handle 1000. So there will be a difference by virtue of the difference
in fresh connections. This could explain why Rocket is slower with 4
instances, but that being the case it should also be slower with one
instance (even though they hit some arbitrary external wall) which is
inconclusive at this point.
I'd be curious which version Nicholas Piel tested. I just fixed a
performance issue yesterday for linux. If he tested prior to that
version (1.0.2) then yes, it would appear much slower.
Are these numbers consistent with Tim numbers? Could this be dues to a
different memory usage?
Note that my tests were run on Windows. I'm not sure what Cherrypy's
bottleneck on Windows is, but Rocket is not subject to it on that
platform. Also, Rocket uses less memory (by almost 2MB) than Cherrypy
on Windows 7. I haven't looked at memory usage in Linux but due to
Rocket's less-custom code-base we should see a similarly smaller memory
usage amount.
In the 4-instance test, this is not a use-case I'd considered yet. As
previously mentioned, Rocket is slower at accepting connections. If
pound was closing the connection (HTTP 1.0 behavior and some HTTP 1.1
proxies) after every request, this could explain why Rocket comes up slower.
Some other things to consider:
- Kuba, how many processor cores are on your test machine? Having more
processes than processors will hurt Rocket more than Cherrypy.
- It seems that you are testing this against web2py (notice how all the
responses are 3xx), perhaps you should just test the servers themselves
for now. If that's not the case, may we see the invocation code?
In the bigger picture, there are some other matters to consider:
- Who will likely run web2py with the build-in webserver? New users
testing things out or devs running relatively small jobs.
- What platforms will those run on? Windows in the former. The latter
is anyone's guess. (Let's not start a rabbit-trail about which
operating system is better, just consider which one most students run.)
So here are some things to consider in this situation:
- Rocket measures (not horribly) slower than Cherrypy on Linux with 4
instances running. How common of a situation is this?
- Rocket is not affected by a major concurrency issue with
single-instance Cherrypy on Windows.
I think going forward we should figure out which one is truly faster as
a single-instance on Linux. I wouldn't be surprised if Rocket is
slightly slower than Cherrypy but it should not be vastly slower. The
goal of Rocket was not to be faster than Cherrypy but to be more
concurrent. So far that's true for Windows and inconclusive on Linux.
I don't have access to a Mac, but I would be surprised if Macs performed
differently than Linux.
Anyone know how to identify that wall that both servers are hitting on
Linux?
-tim
On 3/19/2010 5:36 AM, Kuba Kucharski wrote:
Are these numbers consistent with Tim numbers? Could this be dues to a
different memory usage?
1. Tim?
2. I have a lot of free memory while testing
I wrote email to an author of the blog entry about wsgi webserver
benchmarks - Nicholas Piël
http://nichol.as/benchmark-of-python-web-servers
In short he says:
make sure you do not use ab
yes, in my tests I use httperf
make sure you are running from other machine with limits also tweaked
this is done like that by me
use recompiled httperf
done already
this also comes from him:
"I did a quick benchmark after being pointed to Rocket and I could not
see the same performance advantage for Rocket over CherryPy, more the
opposite."
--
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to web...@googlegroups.com.
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en.
