[issue1346238] A constant folding optimization pass for the AST

[Alexander Belopolsky]

Alexander Belopolsky <[EMAIL PROTECTED]> added the comment:

Raymond wrote in his recent response on issue2499 (a patch that adds
unary '+' and 'not' folding to peephole optimizer):

"""
 More importantly, we decided that the peepholer is the wrong place to 
do much of this work.  Most of the peepholer is going to be migrated 
up the chain, after the AST is generated, but before the opcodes are 
generated.  That is a faster, more reliable, and more general 
approach.
""" (See msg64618.)

This looks like the relevant patch.  I would like to take a look at the
patch, but since it is more than 2 years old, maybe someone has an
updated version.  Please advise.

--
nosy: +belopolsky
type:  -> performance

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Thomas Lee]

Thomas Lee <[EMAIL PROTECTED]> added the comment:

I'm working on the AST optimization code for 2.7 (?) in the
tlee-ast-optimize branch. I've since adopted some of the ideas from this
patch, but I'll take another look when I get a chance. The folding
operation is already mostly in-place.

--
nosy: +thomas.lee

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Jeremy Hylton]

Changes by Jeremy Hylton <[EMAIL PROTECTED]>:


--
nosy: +jhylton

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Dave Malcolm]

Changes by Dave Malcolm :


--
nosy: +dmalcolm

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Georg Brandl]

Changes by Georg Brandl :


--
nosy:  -georg.brandl, georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Dave Malcolm]

Dave Malcolm  added the comment:

FWIW, I'm working on fixing up the this patch to work against py3k; I'm 
assuming there's still interest in the AST visitor + specific optimization 
passes approach.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +benjamin.peterson
versions: +Python 3.2 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Raymond Hettinger]

Raymond Hettinger  added the comment:

David, it would be great if an optional AST optimization pass could do 
something that we don't already have (perhaps, loop invariant code motion when 
python is called with -OO or somesuch).  The AST tree makes it possible for the 
first time to provide some non-trivial optimizations, so aim high.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Dave Malcolm]

Dave Malcolm  added the comment:

I've been working on this against the py3k branch.  I'm attaching what I've got 
so far.

I foolishly didn't check the tlee-ast-optimize branch, instead using file6850 
as a base.

Rune Holm/titanstar, I assume you've signed a PSF contributor agreement?

Changes since titanstar's original patch:
  - rework to apply against py3k
  - reformatted tabs to 4-space indentation; tried to reformat to PEP-7 as much 
as possible
  - added stmt types: With_kind, Nonlocal_kind
  - added expr types: IfExp_kind, Bytes_kind, SetComp_kind, DictComp_kind, 
Ellipsis_Kind
  - removed Print_kind, Exec_kind, Repr_kind
  - reworked Raise_kind
  - added "col_offset" and "arena" arguments; pass in the PyArena from the 
compiler as the context of the visitor
  - removal of all "free_expr" and "asdl_seq_free" calls on the assumption that 
PyArena now handles all of this (am I correct in thinking this?)
  - String -> Bytes in create_ast_from_constant_object
  - added test_optimize selftest suite, though this is based on bytecode 
disassembly, rather than direct inspection of the AST
  - I've fixed it up so it compiles and passes regrtest, but I suspect I've 
missed optimization possibilities

I did a little performance testing using the py3k version of the benchmark 
suite; currently it's a slight regression for some tests, a slight improvement 
for others; nothing impressive yet.

Thomas Lee's AST optimization branch (branched from r62457) has lots of 
interesting work:
  e.g. 
http://svn.python.org/view/python/branches/tlee-ast-optimize/Python/optimize.c?view=log

This appears to not be quite the same starting point; he added a 
PyCF_NO_OPTIMIZE flag to Include/pythonrun.h (and other places), which seems 
like a good way to see the effect of the optimization pass.  He also removed 
the peepholer; maybe it's worth doing that, but it seems worth at least keeping 
the test suite around to ensure a lack of regressions.

I can look at cherrypicking Thomas' work/porting it to py3k.

Re: "aiming high": I'd love to add new optimizations, but it's not clear to me 
what's permissable.  In particular, is it permissable for an optimization pass 
to assume that there are no external modifications to the locals within a frame?

It's possible to write code like this:
frame = inspect.currentframe()
inspect.getouterframes(frame)[-depth][0].f_locals[name] = value
to manipulate locals; whether or not this actually affects running code in the 
current implementation of CPython seems hit-or-miss to me right now, I think 
depending on exactly when fastlocals get written back to the f_locals 
dictionary (I could have miswritten the precise code).

By strategically manipulating locals in other frames, we can break pretty-much 
any typical compiler optimization: locals can appear or change from under us, 
or change attribute values, or gain side-effects to their __getattr__ (e.g. 
writing to disk).

If it is permissable for an optimization pass to assume that there are no 
external modifications to the locals within a frame, then issue 4264 might be 
one to investigate: this is a patch on top of Tom Lee's work (to do local 
type-inference to replace list.append with LIST_APPEND).   Ideas for other 
optimizations would be most welcome.

--
Added file: 
http://bugs.python.org/file19513/py3k-ast-optimization-2010-11-05-001.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Terry J. Reedy]

Terry J. Reedy  added the comment:

#11549 Rewrite peephole to work on AST
includes constant folding. I have not compared.

--
nosy: +terry.reedy
versions: +Python 3.3 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Dave Malcolm]

Dave Malcolm  added the comment:

Another optimization idea: detect local dictionaries that are only ever used in 
non-mutating ways, and convert them to constants, rather than rebuilding the 
dict from scratch each time.

See e.g. htmlparser.py:adjustSVGAttributes etc within the bm_html5lib benchmark 
(though this doesn't seem to be ported to py3k yet)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Alex]

Alex  added the comment:

ISTM that you don't need to worry about mutating locals, the locals() function 
is explicitly documented as not necessarily affecting local variables (not sure 
if frame objects have the same documentation).

If you want a free optimization opportunity: BINARY_SUBSCR with a list for the 
first argument who's contents are all constant, this can be optimized to turn 
it into a tuple which can be load const'd (as we do in the case of an in 
check), note that this cannot be used in the case of::

l = [1, 2, 3]
l[v]

As v.__index__ could theoretically mutate l.

--
nosy: +alex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1346238] A constant folding optimization pass for the AST

[Jeremy Hylton]

Changes by Jeremy Hylton :


--
priority: high -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com