[issue21074] Too aggressive constant folding

Serhiy Storchaka added the comment: Fixed in issue30416. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed superseder: Build-out an AST optimizer, moving some functionality out of the peephole optimizer -> constant folding opens compiler to quadratic

[issue21074] Too aggressive constant folding

Change by Serhiy Storchaka : -- pull_requests: +4789 ___ Python tracker ___ ___

[issue21074] Too aggressive constant folding

Serhiy Storchaka added the comment: New changeset b580f4f2bf49fd3b11f2a046911993560c02492a by Serhiy Storchaka in branch '3.6': [3.6] bpo-30416: Protect the optimizer during constant folding. (#4865)

[issue21074] Too aggressive constant folding

Serhiy Storchaka added the comment: New changeset 2e3f5701858d1fc04caedefdd9a8ea43810270d2 by Serhiy Storchaka in branch 'master': bpo-30416: Protect the optimizer during constant folding. (#4860)

[issue21074] Too aggressive constant folding

Change by Serhiy Storchaka : -- pull_requests: +4756 ___ Python tracker ___ ___

[issue21074] Too aggressive constant folding

Change by Serhiy Storchaka : -- keywords: +patch pull_requests: +4750 stage: needs patch -> patch review ___ Python tracker

[issue21074] Too aggressive constant folding

Antoine Pitrou added the comment: > "I apologize if you were offended", which I will take as being sincere, > happens to also be one of the most common examples of an insincere apology. > Bowing out when there is a reference to the CoC gives undue power to others, > and hinders the ability to

[issue21074] Too aggressive constant folding

Changes by Antoine Pitrou : -- nosy: -pitrou ___ Python tracker ___ ___ Python-bugs-list

[issue21074] Too aggressive constant folding

Andrew Dalke added the comment: Again, I do not propose any changes to the existing optimizer. I do not need anything changed for my code to work. My goal is to counter-balance comments which suggest that perfectly normal code is somehow folly and arcane. These caused me some bewilderment and

[issue21074] Too aggressive constant folding

Serhiy Storchaka added the comment: See the patch in issue30416. I'm not sure these checks should be added to the peephole optimizer, but this isn't very difficult. -- nosy: +serhiy.storchaka ___ Python tracker

[issue21074] Too aggressive constant folding

Changes by Raymond Hettinger : -- nosy: -rhettinger ___ Python tracker ___ ___

[issue21074] Too aggressive constant folding

Raymond Hettinger added the comment: > I do not think quoting the Zen of Python helps anything. Instead of practicality-beats-purity, the better aphorism is perfect-is-the-enemy-of-good. > The Code of Conduct which governs comments here requests > that we be considerate and respective The

[issue21074] Too aggressive constant folding

Andrew Dalke added the comment: I do not think quoting the Zen of Python helps anything. As I wrote, "it gives different answers depending on where one draws the line." This includes "practicality beats purity". >From my viewpoint, the peephole optimizer isn't going to change because the

[issue21074] Too aggressive constant folding

Raymond Hettinger added the comment: > (I reopen the issue since it got a long new comment.) Andrew specifically said, "I don't propose re-opening it". FWIW, the zen of python also says, "practicality beats purity". For most users, most of the time, the peephole optimizer is a net win and it

[issue21074] Too aggressive constant folding

STINNER Victor added the comment: (I reopen the issue since it got a long new comment.) -- resolution: duplicate -> status: closed -> open ___ Python tracker

[issue21074] Too aggressive constant folding

Andrew Dalke added the comment: I know this issue was closed many years ago, and I don't propose re-opening it. I write this comment because some of the earlier comments here make it sound like only a foolish or perverse programmer might be affected by this 'too aggressive constant folding'.

[issue21074] Too aggressive constant folding

Nick Coghlan added the comment: I'm marking this as a duplicate of issue 11549 (which is about building out an AST optimiser), mostly because I don't think this kind of parameter tweaking makes sense with the current peephole optimiser. With a more sophisticated Python implemented optimiser

[issue21074] Too aggressive constant folding

STINNER Victor added the comment: The change that I proposed (check parameters before compute the result is simple), whereas implementing a new optimizer working on the AST requires much more work. -- ___ Python tracker rep...@bugs.python.org

[issue21074] Too aggressive constant folding

Raymond Hettinger added the comment: check parameters before compute the result is simple Handling the general case (all sequences and all operators) isn't really simple. It also involves doing more computations and making more assumptions than we want to do in this code (it is not a great

[issue21074] Too aggressive constant folding

Steve Holden added the comment: Is a fix really required? Are we really supposed to protect programmers from their own folly by second-guessing when constant folding might be required and when it might not? How is hte interpreter supposed to know the function isn't called? The simple

[issue21074] Too aggressive constant folding

STINNER Victor added the comment: Is a fix really required? Yes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074 ___ ___ Python-bugs-list

[issue21074] Too aggressive constant folding

INADA Naoki added the comment: For example. I want to write test like this: @unittest.skip(This test requires huge memory) def test_largebuf(): client = self.create_client() data = b'x' * (2**32 - 1) client.send(data) -- ___ Python

[issue21074] Too aggressive constant folding

STINNER Victor added the comment: For example. I want to write test like this: @unittest.skip(This test requires huge memory) There is @test.support.bigmemtest(size=_2G, memuse=1) decorator which is more convinient than the skip decorators. See Lib/test/test_bigmem.py. You can use

[issue21074] Too aggressive constant folding

Raymond Hettinger added the comment: Special cases aren't special enough to break the rules. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074 ___

[issue21074] Too aggressive constant folding

Raymond Hettinger added the comment: FWIW, I concur with Steve Holden and recommend closing this. It would be ashamed to hack-up constant-folding to handle an arcane case of an uncalled function that does space intensive constant operation. -- assignee: - rhettinger nosy: +rhettinger

[issue21074] Too aggressive constant folding

Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: rhettinger - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074 ___ ___

[issue21074] Too aggressive constant folding

Changes by Alex Gaynor alex.gay...@gmail.com: -- nosy: +alex ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074 ___ ___ Python-bugs-list mailing

[issue21074] Too aggressive constant folding

STINNER Victor added the comment: Hi, I have a project called astoptimizer which does the same job than the CPython peephole optimizer, but it is implemented in Python. To avoid this issue (create an huge object and then discard it because it is too large), I have a check_binop_cst function

[issue21074] Too aggressive constant folding

Antoine Pitrou added the comment: Not only is a lot of memory allocated but it also eats quite a bit of CPU time. -- nosy: +pitrou stage: - needs patch versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074

[issue21074] Too aggressive constant folding

New submission from INADA Naoki: When I run following script: def uncalled(): x = b'x' * (2**32) print('Hello') Python 3.4 consumes huge memory in spite of uncalled() function isn't called. $ /usr/bin/time -l /usr/local/bin/python2 constant_folding.py Hello 0.02 real 0.01

[issue21074] Too aggressive constant folding

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074 ___ ___ Python-bugs-list mailing

[issue21074] Too aggressive constant folding

Benjamin Peterson added the comment: Note the memory is allocated but is immediately thrown away. -- nosy: +benjamin.peterson priority: normal - low ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21074

[issue21074] Too aggressive constant folding

eryksun added the comment: The redesigned peephole optimizer in PY3 improves constant folding. Limiting this would be a step back. Plus you can get the same behavior in PY2 if you first expand the power. For example: # using 2**30, for a 32-bit process def uncalled(): x =