Hi, Just thanks for this big contribution. And maybe this project could give new ideas to optimize Python, who knows ?
At least, you've win a beer for the FOSDEM event this week-end ;-) Have a nice week. -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ 2016-01-25 19:16 GMT+01:00 Victor Stinner <victor.stin...@gmail.com>: > Hi, > > Summary: FAT Python is not faster, but it will be ;-) > > -- > > When I started the FAT Python as a fork of CPython 3.6, I put > everything in the same repository. Last weeks, I focused on splitting > my giant patch (10k lines) into small reviewable patches. I wrote 3 > PEP (509 dict version, 510 function specialziation, 511 code > tranformers) and I enhanced the API to make it usable for more use > cases than just FAT Python. I also created fatoptimizer (the AST > optimizer) and fat (runtime dependency of the optimizer) projects on > GitHub to separate clearly what should be outside Python core. For all > links, see: > > http://faster-cpython.readthedocs.org/fat_python.html > > For the fatoptimizer project, my constraint is to be able to run the > full Python test suite unmodified. In practice, I have to disable some > optimizations by putting a "__fatoptimizer__= {...}" configuration to > some test files. For example, I have to disable constant folding on > test_bool because it tests that False+2 gives 2 at runtime, whereas > the optimizer replaces directly False+2 with 2 during the compilation. > Well, test_bool.py is not the best example because all tests pass with > the constant folding optimization (if I comment my > "__fatoptimizer__={...}" change). > > This constraint ensures that the optimizer "works" and doesn't break > (too much ;-)) the Python semantics, but it's more difficult to > implement powerful optimizations. > > I also found and fixed various kinds of bugs. In my code obviously, > but also in the Python core, in various places. Some bugs only concern > AST transformers which is a new feature, but I had to fix them. For > example, Python didn't support negative line number delta in > co_lntotab of code objects, and so line number were all wrong on > optimized code. I merged my enhancement in the default branch of > CPython (issue #26107). > > In short, I focused on having something working (respecting the Python > semantics), rather than spending time on writing optimizations. > > -- > > When I asked explicitly "Is someone opposed to this PEP 509 [dict > verion] ?", Barry Warsaw answered that a performance analysis is > required. Extract of his mail: > > "I still think this is maintenance and potential performance > overhead we don't want to commit to long term unless it enables > significant optimization. Since you probably can't prove that without > some experimentation, this API should be provisional." > > Last week, I ran some benchmarks and I have to admin that I was > disappointed. Not only fatoptimizer doesn't make Python faster, but it > makes it much slower on some tests! > > http://fatoptimizer.readthedocs.org/en/latest/benchmarks.html > > Quickly, I identified a major performance issue when nested functions > are specialized, especially in Lib/json/encoder.py (tested by > bm_json_v2.py benchmark). I fixed my optimizer to not specialize > nested functions anymore. This simple change fixed the main > performance issue. Reminder: in performance critical code, don't use > nested functions! I will maybe propose patches for Lib/json/encoder.py > to stop using nested functions. > > I only ran benchmarks with the optimizer enabled. I now have to > measure the overhead of my patches (PEP 509, 510 and 511) adding the > API fat AST optimizers. The overhead must be negligible. For me, it's > a requirement of the whole project. Changes must not make Python > slower when the optimizer is not used. > > fatoptimizer is faster on microbenchmarks, but I had to write manually > some optimizations: > > http://fatoptimizer.readthedocs.org/en/latest/microbenchmarks.html > > IMHO fatoptimizer is not faster on macro benchmarks because it is not > smart enough (yet) to generate the most interesting optimizations, > like function inlining and specialization for argument types. You can > estimate the speedup if you specialize manually your functions. > > -- > > Barry also wrote: "Did you address my suggestion on python-ideas to > make the new C API optionally compiled in?" > > Well, it is an option, but I would prefer to have the API for AST > optimizer directly built in Python. > > The first beta version of Python 3.6 is scheduled in September 2016 > (deadline for new features in Python 3.6), so I still have a few > months to implement more powerful optimizations and prove that it can > be faster ;-) > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/gmludo%40gmail.com >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com