The difference between the two is that in former case the output returned by the DB is normalized (each record is a Row object) while in the executesql case the output is not normalized (each record is a tuple containing data in the database specific representation). It does take time to loops over a list of tuples and convert it into into a list (Rows) or dictionaries (Row) of normalized objects.
You can in fact do: rows = db(....).select(cacheable=True) # some speedup rows = db(....).select(processor=lambda rows,*a,**b: rows) # as fast as executesql but no parsing On Saturday, 13 July 2013 16:56:56 UTC-5, Michele Comitini wrote: > > I don't care about ORM performance which is worse both in term of bare > performance and in development speed, but using large databases and doing > profiling I find that most of the time that is "spent on DB IO" is ispent > inside python code unless executesql is used. There is still a lot of room > for improvement even if a lot has been done already. > > Our target should be that in the future: > > rows = db(query).select(...) > > vs. > > sql=db(query)._select(...) > rows = db.exequtesql(sql) > > have a not a difference of one order of magnitude anymore. > > > 2013/7/13 Massimo Di Pierro <massimo....@gmail.com <javascript:>> > >> It really depends on what one compares. >> >> I do not think we ever published benchmarks claiming web2py is faster >> than other frameworks. If one want speed raw WSGI (perhaps with gunicorn or >> gevent) without templates and without databases beats every framework. The >> more a framework does, the slower it gets. I do claim web2py is better than >> other frameworks because it is easier to use and because it does more for >> you than other frameworks do. >> >> According to my tests: >> - the DAL is faster in generating SQL queries than ORM based on >> metaclasses but this is pointless since all the time is in DB IO, not in >> generating queries. That time is the same. Django in some cases >> (postgresql) has the ability of returning an iterators while web2py always >> returns the data. The Django approach is better from a memory management >> way and faster is only plan to loop over a subset of the selected records. >> If you plan to loop over all of the records selected the web2py approach is >> better. >> - The web2py template language is faster than the Django templates >> because uses the python interpreter and not its own. Yet it depends on the >> details of the view page. >> - Apart for templates and DAL, the execution of models and controllers is >> slower in web2py. In Django models are executed only once (at startup) >> while in web2py at every request. We do this for a reason. It is a small >> performance penalty to pay for a greater flexibility (multi-app support, >> hot plug and play, no library conflicts, etc.). This only hits in hello >> world kind of tests and in the case of very very long models (that is why >> some of use prefer to put code in modules and not in models) and then >> import the modules in models. >> >> Massimo >> >> On Friday, 12 July 2013 18:44:11 UTC-5, Arnon Marcus wrote: >> >>> I seem to remember vividly seeing a benchmark about 4 years ago that >>> showed that web2py was indeed the fastest. >>> >>> Either that benchmark was misleading, or things have changed. >>> >>> In either case, it would still remain important to explain to the public >>> the reasons for the trade-offs that were chosen. If "speed-of-development" >>> or "security" had to come with a performance-penalty, and still it was a >>> conscious-choice to for-go with performance, than this trade-off choice >>> should be out there in the open and explicit. It's a form of >>> expectation-management and quality public-relation. >>> >>> If people out-there understand that "hello-world" benchmarks are >>> meaningless for real-world use-cases, as the bottleneck always end-up being >>> the database whenever one is used (which is practically 100% of real-world >>> use-cases), they would be more apt to trying-out web2py, and have less >>> irrelevant/erroneous-biases against it. >>> >>> And if the DAL is really faster than an ORM, than the bottleneck should >>> theoretically be less severe in web2py, since less queries would be >>> executed to the database. Meaning, that any performance-loss of >>> non-database-related executions, should be grossly overshadows and >>> nullified by virtue of having fewer database accesses. Meaning, that for >>> real-world production use - web2py ultimately should be faster for >>> database-intensive applications. >>> >>> >>> On Fri, Jul 12, 2013 at 5:11 AM, Ovidio Marinho <ovid...@gmail.com>wrote: >>> >>>> Web2py can not be the fastest, but it is the most simple and functional >>>> for everyone. >>>> >>>> >>>> >>>> >>>> Ovidio Marinho Falcao Neto >>>> ITJP.NET.BR >>>> ovid...@gmail.com >>>> 83 8826 9088 - Oi >>>> 83 9336 3782 - Claro >>>> Brasil >>>> >>>> >>>> >>>> 2013/7/12 Massimo Di Pierro <massimo....@gmail.com> >>>> >>>>> Because I think it is pointless for various reasons: >>>>> >>>>> 1) I am biased and people outside the community would not trust it >>>>> 2) Code changes so it would become obsolete quick >>>>> 3) One can produce benchmarks to produce almost any result one wants >>>>> 4) People who are concerned about 2x factors in speed are not web2py >>>>> audience. The point of web2py is security and speed of development. For >>>>> example we can make web2py more than 2x faster moving sessions management >>>>> to the app level (like Flask) and not creating the request.env object. >>>>> Yet >>>>> we do not do it. What matters is not speed but scalability and web2py >>>>> scales no better or worse than any other major Python framework. >>>>> 5) Every time I published any comparison between web2py and other >>>>> frameworks in the past, somebody got upset and attacked us. Not worth it. >>>>> >>>>> On Thursday, 11 July 2013 20:44:07 UTC-5, viniciusban wrote: >>>>> >>>>>> Massimo, how about you writing an article about this subject and >>>>>> share with us? >>>>>> >>>>>> So, this could be spread. >>>>>> >>>>>> On Thu, Jul 11, 2013 at 2:12 PM, Massimo Di Pierro >>>>>> <massimo....@gmail.com> wrote: >>>>>> > I agree. I will do. >>>>>> > >>>>>> > On Thursday, 11 July 2013 11:51:39 UTC-5, Arnon Marcus wrote: >>>>>> >> >>>>>> >> I see. >>>>>> >> In that case, I think it would be advisable to note that in >>>>>> presentations, >>>>>> >> as peope might get the wrong impression... >>>>>> >> >>>>>> >> On Thursday, July 11, 2013, Massimo Di Pierro < >>>>>> massimo....@gmail.com> >>>>>> >> wrote: >>>>>> >> > It is true but not an issue. Django is faster only in hello >>>>>> world >>>>>> >> > examples because does not perform as many header >>>>>> validation/conversions as >>>>>> >> > web2py does and because you cannot turn off sessions in web2py. >>>>>> As soon as >>>>>> >> > one uses templates, web2py is faster. If you use databases the >>>>>> speed is >>>>>> >> > about the same because that becomes the bottle neck. >>>>>> >> > Massimo >>>>>> >> > >>>>>> >> > On Monday, 8 July 2013 17:08:53 UTC-5, Arnon Marcus wrote: >>>>>> >> >> >>>>>> >> >> BTW, is it really true that web2py is twice as slower than >>>>>> django >>>>>> >> >> nowadays? >>>>>> >> >> How can that be? >>>>>> >> >> Didn't it used to be twice as fast? >>>>>> >> >> When I first evaluated it 3 years ago, it was by-far the >>>>>> fastest - what >>>>>> >> >> changed? >>>>>> >> >> You said that one of the core principles of accepting changes >>>>>> to >>>>>> >> >> web2py, is that they should always make it run faster - never >>>>>> slower. Has >>>>>> >> >> that principle been broken? >>>>>> >> >> And what about the whole ORM-vs-DAL fiasco? Didn't you guys >>>>>> always say >>>>>> >> >> that a DAL is always faster than an ORM? Given that it makes >>>>>> sense, how come >>>>>> >> >> Django is faster? Shouldn't it's ORM make it slower? >>>>>> >> >> And as for executing-vs-importing - didn't you say before that >>>>>> it >>>>>> >> >> should be a non-issue in terms of performance? What changed? >>>>>> >> > >>>>>> >> > -- >>>>>> >> > >>>>>> >> > --- >>>>>> >> > You received this message because you are subscribed to a topic >>>>>> in the >>>>>> >> > Google Groups "web2py-users" group. >>>>>> >> > To unsubscribe from this topic, visit >>>>>> >> > https://groups.google.com/d/**to**pic/web2py/kWwIuQjeatQ/** >>>>>> unsubsc**ribe<https://groups.google.com/d/topic/web2py/kWwIuQjeatQ/unsubscribe>. >>>>>> >>>>>> >>>>>> >> > To unsubscribe from this group and all its topics, send an email >>>>>> to >>>>>> >> > web2py+un...@googlegroups.com. >>>>>> >> > For more options, visit https://groups.google.com/**grou** >>>>>> ps/opt_out <https://groups.google.com/groups/opt_out>. >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> > >>>>>> > -- >>>>>> > >>>>>> > --- >>>>>> > You received this message because you are subscribed to the Google >>>>>> Groups >>>>>> > "web2py-users" group. >>>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>>> send an >>>>>> > email to web2py+un...@**googlegroups.com. >>>>>> > For more options, visit https://groups.google.com/**grou** >>>>>> ps/opt_out <https://groups.google.com/groups/opt_out>. >>>>>> > >>>>>> > >>>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "web2py-users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to web2py+un...@**googlegroups.com. >>>>> >>>>> For more options, visit >>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>>> . >>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "web2py-users" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/** >>>> topic/web2py/kWwIuQjeatQ/**unsubscribe<https://groups.google.com/d/topic/web2py/kWwIuQjeatQ/unsubscribe> >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> web2py+un...@**googlegroups.com. >>>> For more options, visit >>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>> . >>>> >>>> >>>> >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "web2py-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to web2py+un...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
