On Fri, Dec 9, 2011 at 7:43 PM, Joe & Anne Tennies <tenn...@gmail.com> wrote: > The thing is, we aren't trying to "scientifically correct" statistics. What > we're aiming to say is, "This is not so wildly different as to be of any > concern." We aren't looking for minor difference, but orders of magnitude > difference. > > If you are that worried about a <2% difference in speed, you probably > shouldn't be using Python... or at least Django. You should be finding the > parts that you can optimize for your specific application and optimizing > those. Python 3 *IS* the future. There isn't much way around that at this > point. I believe the general idea is to make sure the solution at this point > does not slow everything down to the point where it would be impossible for > someone to switch to Python 3. Don't get me wrong, I don't want to see a 2% > increase in timing all the time, but Python 3 support is a bullet that will > HAVE to get bitten.
Py3k is the future, changing DB adapter from a C library with a python wrapper to a pure python adaptor should merit some performance testing of the new adaptor, which is what I really wanted to test. However, it is bad science (we should all consider ourselves scientists) to do one run and say that performance hasn't changed. Regression testing is not just about changes in test results, changes should also not make the framework unnecessarily slow, and the only way to determine how much effect a changeset has is to benchmark it, and the only way to benchmark it is scientifically, using statistics and to a high degree of confidence. We should only care about large changes in performance, but how do you determine if something is a large change, without statistically valid benchmarks? > > Also, I am expecting I could make bigger gains by just using that pure > Python MySQL driver and running under PyPy. If the major part of the time > wasn't spent inside the actual database (which should be fairly obviously > the case as switching from SQLite in memory to MySQL is a >5x increase in > time). > Statistics or shut up :) Only joking :) How the pure python mysql driver performs compared to one built around libmysqlclient is particularly interesting. Have you actually tried this, or are you speculating? Ian, thanks for running those tests. Running the numbers through ministat tells us that the patches are slightly slower (3.2% ± 2.5%, at a 99% confidence). Cheers Tom -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
