[issue11549] Rewrite peephole to work on AST

2011-06-06 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: Eugene raised the question of AST changes on python-dev [1] and the verdict was that so long as ast.__version__ is updated, AST clients will be able to cope with changes. Benjamin Peterson made some subsequent changes to the AST (bringing the

[issue11549] Rewrite peephole to work on AST

2011-03-29 Thread anatoly techtonik
anatoly techtonik techto...@gmail.com added the comment: Is there any tool to see how it works step-by-step. The whole stuff is extremely interesting, but I can't fit all the details of AST processing in my head. -- nosy: +techtonik ___ Python

[issue11549] Rewrite peephole to work on AST

2011-03-29 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: Eugene: I suggest raising the question on python-dev. The answer potentially affects the PEP 380 patch as well (which adds a new attribute to the Yield node). Anatoly: If you just want to get a feel for the kind of AST various pieces of code

[issue11549] Rewrite peephole to work on AST

2011-03-28 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: If we have to preserve backward compatibility of Python AST API, we can do this relatively easily (at the expense of some code complexity): * Add 'version' argument to compile() and ast.parse() with default value of 1 (old AST). Value 2 will

[issue11549] Rewrite peephole to work on AST

2011-03-28 Thread Raymond Hettinger
Raymond Hettinger raymond.hettin...@gmail.com added the comment: Eugene, I think you're doing great work here and would like to see you succeed. In the near term, I don't have time to participate, but don't let that stop you. -- ___ Python tracker

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: Finally got around to reviewing this (just a visual scan at this stage) - thanks for the effort. These are mostly big picture type comments, so I'm keeping them here rather than burying them amongst all the details in the code review tool.

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: I think the biggest thing to take out of my review is that I strongly encourage deferring the changes for 5(b) and 5(c). I like the basic idea of using a template-based approach to try to get rid of a lot of the boilerplate code currently

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: As a more general policy question... where do we stand in regards to backwards compatibility of the AST? The ast module docs don't have any caveats to say that it may change between versions, but it obviously *can* change due to new language

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Georg Brandl
Georg Brandl ge...@python.org added the comment: I would provide this via another compile flag a la PyCF_ONLY_AST. If you give only this flag, you get the original AST. If you give (e.g.) PyCF_OPTIMIZED_AST, you get the resulting AST after the optimization stage (or the same, if optimization

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Thanks. string concatenation will now work, and errors like 'hello' - 'world' should give a more informative TypeError Yes, 'x'*5 works too. Bikeshed: We use Attribute rather than Attr for that node type, perhaps the full Literal name would

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: and with __future__ it should work on 2.5 as well. Actually, seems that at least str.format is not in 2.5 as well. Still the question is should I make it run on 2.5 or 2.4 or is 2.6 OK (then __future__ can be removed). --

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Daniel Urban
Daniel Urban urban.dani...@gmail.com added the comment: not x == 2 can be theoretically optimized to x != 2, ... I don't think it can: class X: ... def __eq__(self, other): ... return True ... def __ne__(self, other): ... return True ... x = X() not x ==

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: I don't think it can: That already doesn't work in dict and set (eq not consistent with hash), I don't think it's a big problem if that stops working in some other cases. Anyway, I said theoretically -- maybe after some conservative type

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Also, to avoid any confusion -- currently my patch only runs AST optimizations before code generation, so compile() with ast.PyCF_ONLY_AST returns non-optimized AST. -- ___ Python tracker

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Terry J. Reedy
Terry J. Reedy tjre...@udel.edu added the comment: While I would not be happy to use class X above, the 3.2 manual explicitly says There are no implied relationships among the comparison operators. The truth of x==y does not imply that x!=y is false. . --

[issue11549] Rewrite peephole to work on AST

2011-03-27 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: OK, I missed the fact that the new optimisation pass isn't run under PyCF_ONLY_AST. However, as Eugene picked up, my concern is actually with the collapsing of Str/Num/Bytes/Ellipsis into the new Lit node, and the changes to the way

[issue11549] Rewrite peephole to work on AST

2011-03-23 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Is anyone looking or planing to look at the patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-23 Thread Terry J. Reedy
Terry J. Reedy tjre...@udel.edu added the comment: I suspect someone will sometime. There is bit of a backlog of older issues. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-19 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: AFAICT my patch has everything that #1346238 has, except BoolOps, which can be easily added (there's a TODO). I don't want to add any new code, though, until the current patch will get reviewed -- adding code will only make reviewing harder.

[issue11549] Rewrite peephole to work on AST

2011-03-19 Thread Georg Brandl
Changes by Georg Brandl ge...@python.org: -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list

[issue11549] Rewrite peephole to work on AST

2011-03-19 Thread Skip Montanaro
Changes by Skip Montanaro s...@pobox.com: -- nosy: -skip.montanaro ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list

[issue11549] Rewrite peephole to work on AST

2011-03-18 Thread Mark Dickinson
Changes by Mark Dickinson dicki...@gmail.com: -- nosy: +mark.dickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___

[issue11549] Rewrite peephole to work on AST

2011-03-18 Thread Nadeem Vawda
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nvawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list

[issue11549] Rewrite peephole to work on AST

2011-03-18 Thread Terry J. Reedy
Terry J. Reedy tjre...@udel.edu added the comment: A couple of somewhat related issues: #10399 AST Optimization: inlining of function calls #1346238 A constant folding optimization pass for the AST Obviously, ast optimizers should work together and not duplicate. Nice to see increased attention.

[issue11549] Rewrite peephole to work on AST

2011-03-16 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Any comments on the code so far or suggestions on how we should move forward? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-16 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: I've been focusing on softer targets during the sprints - I'll dig into this once I'm back home and using my desktop machine again. -- ___ Python tracker rep...@bugs.python.org

[issue11549] Rewrite peephole to work on AST

2011-03-16 Thread Alex Gaynor
Changes by Alex Gaynor alex.gay...@gmail.com: -- nosy: +alex ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list mailing

[issue11549] Rewrite peephole to work on AST

2011-03-16 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: I've updated patches on Rietveld with some small changes. This includes better code generation for boolops used outside of conditions and cleaned up optimize_jumps(). This is probably the last change before I get some feedback. Also, I forgot

[issue11549] Rewrite peephole to work on AST

2011-03-16 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Just for fun I've run pystones. W/o my changes it averages to about 70k, with my changes to about 72.5k. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Daniel Urban
Changes by Daniel Urban urban.dani...@gmail.com: -- nosy: +durban ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Nick Coghlan
Changes by Nick Coghlan ncogh...@gmail.com: -- nosy: +dmalcolm, ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Nick Coghlan
Changes by Nick Coghlan ncogh...@gmail.com: -- nosy: +brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Skip Montanaro
Skip Montanaro s...@pobox.com added the comment: I'm confused. Why aren't there review links? -- nosy: +skip.montanaro ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Because I don't know how to make them. Any pointers? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Because I don't know how to make them. Any pointers? Martin is hacking on the tool these days... So it's no surprise it doesn't work perfectly yet ;) If you have a Google account you can upload these patches to http://codereview.appspot.com/,

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: Thanks. Review link: http://codereview.appspot.com/4281051 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: The review links didn't come up automatically because 336137a359ae isn't a hg.python.org/cpython revision ID. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Eugene Toder
Eugene Toder elto...@gmail.com added the comment: I see. Should I attach diffs vs. some revision from hg.python.org? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-15 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: No need, since you manually created a review on appspot. The local Reitveld links are just a convenience that can avoid the need to manually create a review instance. -- ___ Python tracker

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
New submission from Eugene Toder elto...@gmail.com: As pointed out by Raymond, constant folding should be done on AST rather than on generated bytecode. Here's a patch to do that. It's rather long, so overview first. The patch replaces existing peephole pass with a folding pass on AST and a

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
Changes by Eugene Toder elto...@gmail.com: -- keywords: +patch Added file: http://bugs.python.org/file21198/0_ast.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
Changes by Eugene Toder elto...@gmail.com: Added file: http://bugs.python.org/file21199/0_fold.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
Changes by Eugene Toder elto...@gmail.com: Added file: http://bugs.python.org/file21200/0_compile.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
Changes by Eugene Toder elto...@gmail.com: Added file: http://bugs.python.org/file21201/0_generated.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
Changes by Eugene Toder elto...@gmail.com: Added file: http://bugs.python.org/file21202/0_tests.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Eugene Toder
Changes by Eugene Toder elto...@gmail.com: -- nosy: +pitrou, rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Andreas Stührk
Changes by Andreas Stührk andy-pyt...@hammerhartes.de: -- nosy: +Trundle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Santoso Wijaya
Changes by Santoso Wijaya santoso.wij...@gmail.com: -- nosy: +santa4nt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___

[issue11549] Rewrite peephole to work on AST

2011-03-14 Thread Antoine Pitrou
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11549 ___ ___ Python-bugs-list