Tarek, My response inline to your:
> You are not getting my point. What happens to weezhy or XXX framework > when you are running it in a given stack, under heavy load ? let me correct you, it is wheezy.web (not `weezhy`). Tell me your definition of web framework heavy load. If you have one, we can try benchmark it. > There are many interactions that may impact the behavior of the stack - > most of them are in the web server itself, but they can be things in the > framework too, depending on the architectural choice. The reason I choose uWSGI is due to minimal possible impact that application server may cause. Since this component `equally influence` productivity of each framework it can be to some degree ignored. > And you will not know it with an hello world app. To put it more > bluntly, your benchmark is going to join the big pile of hello worlds > benchmarks that are completely meaningless. Can not agree. This is just simple thing. Simple things should run fast, no doubt. If you can provide a better idea as to which framework calls to put into benchmark, I will be happy extend the benchmark case. > If you want to prove that weezhy is faster than another py framework, > because, I dunno, the number of function calls are smaller ? then you > need to isolate this and > do a different kind of bench. > > Have a look at http://plope.com/pyroptimization , it's a good example The numbers provided in that article are incorrect. They didn't match results from the file they provide (result.txt in each framework dir) at the time of writing. I have used that idea to re-run things (isolated benchmark; report with total time, total number of calls and number of distinct functions used). See here: https://bitbucket.org/akorn/helloworld/src/tip/benchmark.py I will update original post a bit later (to let you comment on this). > Same thing for the raw speed of your templating engine - isolation is > required. Improved bigtable benchmark report by adding total number of calls and number distinct functions used: https://bitbucket.org/akorn/wheezy.template/src/tip/demos/bigtable/bigtable.py Original post not updated yet. > One good read: > http://blog.ianbicking.org/2010/03/16/web-server-benchmarking-we-need/ Sounds not so bad. It points to some specific workloads. Any attempt to prioritize and/or practically implement them? Thanks. Andriy ---------------------------------------- > Date: Wed, 26 Sep 2012 11:41:26 +0200 > From: ta...@ziade.org > To: andriy.kornats...@live.com > CC: python-list@python.org > Subject: Re: Fastest web framework > > On 9/26/12 11:26 AM, Andriy Kornatskyy wrote: > > Tarek, > > > > Thank you for the response back. Yes, your idea is pretty clear to me. The > > point is that higher workload you put in your application business logic, > > repository, backend, whatever... less you will see in final results > > comparison. This is obvious and we, as technical people, very well > > understand this (somebody even laugh). > > I am happy somebody got a good laugh, I had a myself a good coffee :) > > > The reality is that not all web applications do heavy CPU computations > > and/or experience IO delays (due to response from database running a query > > over table that has no index, let say), some use caches, some split jobs to > > be run in background, some parallel them... I have to state that simple > > things must perform really fast to give more room for one that are not so. > > That in turn makes your infrastructure more effective. Some prefer to add a > > box, some see that a likely to be a problem further it goes. The good thing > > - you have a choice, you are not locked, and as result you are responsible > > for the effectiveness of the system you build today and definitely next one. > > > > Take care. > > > > Andriy > > You are not getting my point. What happens to weezhy or XXX framework > when you are running it in a given stack, under heavy load ? > > There are many interactions that may impact the behavior of the stack - > most of them are in the web server itself, but they can be things in the > framework too, depending on the architectural choice. > > And you will not know it with an hello world app. To put it more > bluntly, your benchmark is going to join the big pile of hello worlds > benchmarks that are completely meaningless. > > If you want to prove that weezhy is faster than another py framework, > because, I dunno, the number of function calls are smaller ? then you > need to isolate this and > do a different kind of bench. > > Have a look at http://plope.com/pyroptimization , it's a good example > > Same thing for the raw speed of your templating engine - isolation is > required. > > One good read: > http://blog.ianbicking.org/2010/03/16/web-server-benchmarking-we-need/ > > > Cheers > Tarek > > > > > > > ---------------------------------------- > >> Date: Wed, 26 Sep 2012 11:08:19 +0200 > >> From: ta...@ziade.org > >> To: andriy.kornats...@live.com > >> CC: python-list@python.org > >> Subject: Re: Fastest web framework > >> > >> On 9/25/12 3:21 PM, Andriy Kornatskyy wrote: > >>> Tarek, > >>> > >>> With all respect, running benchmark on something that has sleeps, etc is > >>> pretty far from real world use case. So I went a little bit different way. > >> That's not a good summary of what the function does. It does not just > >> sleep. It does some I/O and CPU bound tasks. The sleep is here to > >> simulate a blocking I/O call, besides the DB calls. > >> > >> The whole function tries to simulate a real application, unlike printing > >> 'Hello World' - to put the stack under realistic conditions. > >> > >> The multiplication is cached by the processor, but will still push some > >> CPU work on every call. > >> > >>> Here is a live demo (a semi real world web application) that comes with > >>> wheezy.web framework as a template: > >>> > >>> http://wheezy.pythonanywhere.com/ > >>> > >>> I have implemented it in a way that it uses one web framework > >>> (wheezy.web) and various template engines (jinja2, mako, tenjin, > >>> wheezy.template and wheezy.template with preprocessor)... Please see the > >>> following post under `Real World Example` section: > >>> > >>> http://mindref.blogspot.com/2012/07/python-fastest-template.html > >>> > >>> Source code here: > >>> > >>> https://bitbucket.org/akorn/wheezy.web/src/tip/demos/template > >>> > >>> The real world example shows the difference between template engines > >>> implementing the same things. The same applies to web frameworks (more or > >>> less depending on your choice). > >>> > >>> Thanks. > >> Great, thanks for the update ! -- that's cool to bench the template > >> engines, but this is still not what I had in mind. > >> > >> What I had in mind was to try each one of the framework with an > >> application that does things, and see how the whole stack reacts on high > >> load. > >> > >> But I guess we have different goals - wheezy seems really fast, congrats. > >> > >> > >> Cheers > >> Tarek > >> > >>> Andriy > >>> > >>> > >>> ---------------------------------------- > >>>> Date: Mon, 24 Sep 2012 13:50:31 +0200 > >>>> From: ta...@ziade.org > >>>> To: python-list@python.org > >>>> Subject: Re: Fastest web framework > >>>> > >>>> On 9/23/12 11:19 AM, Andriy Kornatskyy wrote: > >>>>> I have run recently a benchmark of a trivial 'hello world' application > >>>>> for various python web frameworks (bottle, django, flask, pyramid, > >>>>> web.py, wheezy.web) hosted in uWSGI/cpython2.7 and gunicorn/pypy1.9... > >>>>> you might find it interesting: > >>>>> > >>>>> http://mindref.blogspot.com/2012/09/python-fastest-web-framework.html > >>>>> > >>>>> Comments or suggestions are welcome. > >>>>> > >>>>> Thanks. > >>>>> > >>>>> Andriy Kornatskyy > >>>>> > >>>> I would try this with a web app that does more than 'Hello World' > >>>> > >>>> You may argue that you're just trying the server stack, but that's not > >>>> realistic because you don't really measure how the server behaves with a > >>>> real app. > >>>> > >>>> Have a look at > >>>> https://github.com/mozilla-services/chaussette/blob/master/chaussette/util.py#L188 > >>>> > >>>> (setup_bench and teardow_bench have to be run on startup and tear down > >>>> of the server) > >>>> > >>>> I would be curious to see how things goes then > >>>> > >>>> Cheers > >>>> Tarek > >>>> -- > >>>> http://mail.python.org/mailman/listinfo/python-list > > > -- http://mail.python.org/mailman/listinfo/python-list
