[issue26297] Move constant folding to AST level

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Issue1346238 has a patch.

--
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> A constant folding optimization pass for the AST

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26297] Move constant folding to AST level

[STINNER Victor]

STINNER Victor added the comment:

> we need to move some optimizations from bytecode level to AST level

You may be aware that there is a long list of similar (old) issues proposing 
something similar:
https://www.python.org/dev/peps/pep-0511/#usage-1-ast-optimizer

My PEP 511 tries to propose a generic API to support all proposed optimizations.

--

I didn't try yet to do something like importlib to compile an AST optimizer 
written in Python into a frozen module and then use it in Python. It would 
allow to implement an AST optimizer in pure Python to ease its maintenance 
*and* have it builtin into CPython, rather than having to play with AST in C 
(less fun).

My plan for the PEP 511 is more to start to experiment multiple AST optimizers 
and bytecode optimizers outside CPython, wait until the code becomes more 
stable, and later discuss if one optimizer deserves to become part of Python 
stdlib.

"Writing an optimizer or a preprocessor is out of the scope of this PEP."

--

But maybe you would prefer to have a single and simple AST optimizer builtin 
Python without new additional API?

I don't want to block your effort on experimentation :-) I just see a nice 
opportunity to sell you the PEP 511 ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26297] Move constant folding to AST level

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

For using more efficient bytecode (either specialized 8-bit opcodes or 16-bit 
opcodes) we need to move some optimizations from bytecode level to AST level, 
since LOAD_CONST variants could have variable size. Now with the Constant node 
this should be easy.

--
components: Interpreter Core
messages: 259680
nosy: abarnert, benjamin.peterson, brett.cannon, georg.brandl, haypo, ncoghlan, 
serhiy.storchaka, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Move constant folding to AST level
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26297] Move constant folding to AST level

[STINNER Victor]

STINNER Victor added the comment:

Yury Selivanov added the comment:
> In fact I'm -1 on using any kind of flag to trigger optimizations on or off.  
> Those flags will only drive confusion.  If the optimizations are any good, 
> people will simply start using the flag all the time.

Yeah, Guido calls this a missed optimization opportunity.

My only concern is:

> ast.Constant for tuples and frozensets has a limitation: it doesn't store the 
> location of items (line number, column offset).

Is it a problem in practice? I didn't have time to investigate this.

Since the PEP 511 is still under discussion, I prefer to focus on the
PEP itself than this specific implementation detail ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26297] Move constant folding to AST level

[STINNER Victor]

STINNER Victor added the comment:

*If* the PEP 511 is accepted, it will be super easy to start with an
implementation in pure Python.

But I think that we should benchmark the overhead of the Python API of
PEP 511, because we need to convert all AST internal objects to Python
objects, run the AST optimizer, and then again convert Python objects
to internal AST objects.

For a long running process, the time to compile a .py file doesn't
matter. For a short script, it matters. At least, we need to compile
the script itself.

By the way, would it be insane to *not* optimize the script when
running "python script.py"?

> Now with the Constant node this should be easy.

ast.Constant is *not* emited by the compiler to not break backward
compatibility. I *know* that there is no stable API on AST, but I
noticed some issues when working on my AST project. For example, pip
doesn't work because an internal library uses AST and the code doesn't
handle ast.Constant (it's probably super easy to fix it).

I'm open to change the compiler to emit ast.Constant directly, but
maybe only in "optimized mode" (ex: python -O?). ast.Constant for
tuples and frozensets has a limitation: it doesn't store the location
of items (line number, column offset).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26297] Move constant folding to AST level

[Yury Selivanov]

Yury Selivanov added the comment:

> ast.Constant is *not* emited by the compiler to not break backward
compatibility. I *know* that there is no stable API on AST, but I
noticed some issues when working on my AST project. For example, pip
doesn't work because an internal library uses AST and the code doesn't
handle ast.Constant (it's probably super easy to fix it).

Yeah, it's probably very easy to fix.  And if pip stops working on 3.6-alpha it 
will be fixed.  We don't give any guarantees on AST backwards compatibility.  
In fact, we even tell that the AST might change in every release in the docs 
"The abstract syntax itself might change with each Python release".


> I'm open to change the compiler to emit ast.Constant directly, but
maybe only in "optimized mode" (ex: python -O?). 

I'm -1 on using -O flag since some people would prefer to have optimizations 
*and* have their assert statements.

In fact I'm -1 on using any kind of flag to trigger optimizations on or off.  
Those flags will only drive confusion.  If the optimizations are any good, 
people will simply start using the flag all the time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com