Hi Victor,
great to hear. I think everybody here appreciates your efforts.
Do you think there will be any change of merging this back into CPython?
Best,
Sven
On 04.11.2015 09:50, Victor Stinner wrote:
Hi,
I'm writing a new "FAT Python" project to try to implement
optimizations in CPython (inlining, constant folding, move invariants
out of loops, etc.) using a "static" optimizer (not a JIT). For the
background, see the thread on python-ideas:
https://mail.python.org/pipermail/python-ideas/2015-October/036908.html
See also the documentation:
https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst
https://hg.python.org/sandbox/fatpython/file/tip/ASTOPTIMIZER.rst
I implemented the most basic optimization to test my code: replace
calls to builtin functions (with constant arguments) with the result.
For example, len("abc") is replaced with 3. I reached the second
milestone: it's now possible to run the full Python test suite with
these optimizations enabled. It confirms that the optimizations don't
break the Python semantic.
Example:
---
>>> def func():
... return len("abc")
...
>>> import dis
>>> dis.dis(func)
2 0 LOAD_GLOBAL 0 (len)
3 LOAD_CONST 1 ('abc')
6 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
9 RETURN_VALUE
>>> len(func.get_specialized())
1
>>> specialized=func.get_specialized()[0]
>>> dis.dis(specialized['code'])
2 0 LOAD_CONST 1 (3)
3 RETURN_VALUE
>>> len(specialized['guards'])
2
>>> func()
3
>>> len=lambda obj: "mock"
>>> func()
'mock'
>>> func.get_specialized()
[]
---
The function func() has specialized bytecode which returns directly 3
instead of calling len("abc"). The specialized bytecode has two guards
dictionary keys: builtins.__dict__['len'] and globals()['len']. If one
of these keys is modified, the specialized bytecode is simply removed
(when the function is called) and the original bytecode is executed.
You cannot expect any speedup at this milestone, it's just to validate
the implementation. You can only get speedup if you implement
*manually* optimizations. See for example posixpath.isabs() which
inlines manually the call to the _get_sep() function. More
optimizations will be implemented in the third milestone. I don't know
yet if I will be able to implement constant folding, function inlining
and/or moving invariants out of loops.
Download, compile and test FAT Python with:
hg clone http://hg.python.org/sandbox/fatpython
./configure && make && ./python -m test test_astoptimizer test_fat
Currently, only 24 functions are specialized in the standard library.
Calling a builtin function with constant arguments in not common (it
was expected, it's only the first step for my optimizer). But 161
functions are specialized in tests.
To be honest, I had to modify some tests to make them pass in FAT
mode. But most changes are related to the .pyc filename, or to the
exact size in bytes of dictionary objects.
FAT Python is still experimental. Currently, the main bug is that the
AST optimizer can optimize a call to a function which is not the
expected builtin function. I already started to implement code to
understand namespaces (detect global and local variables), but it's
not enough yet to detect when a builtin is overriden. See TODO.rst for
known bugs and limitations.
Victor
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/srkunze%40mail.de
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
