New submission from Andrew Dalke: The peephole optimizer is an overall benefit to Python but it has some side-effects that occasionally cause problems. These are well-known in the issue tracker, but there is no other documentation which will help a Python programmer figure out which constructs may be a problem.
1) it will compute large integer constants and save them in the .pyc file. The following results in a 19M .pyc file. def compute_largest_known_prime(): return 2**74207281 - 1 As an example of someone affected by the compile-time evaluation, see https://stackoverflow.com/questions/34113609/why-does-python-preemptively-hang-when-trying-to-calculate-a-very-large-number/ . Note the many people who struggled to find definitive documentation. 2) it will create and discard large strings. Consider this variant of the code in test_zlib.py, where I have replaced the imported module variable "_1G" with its value: @bigmemtest(size=_4G + 4, memuse=1, dry_run=False) def test_big_buffer(self, size): data = b"nyan" * (2**30 + 1) # the original uses "_1G" self.assertEqual(zlib.crc32(data), 1044521549) self.assertEqual(zlib.adler32(data), 2256789997) The byte compiler will create the ~4GB string then discard it, even though the function will not be called on machines with insufficient RAM. As an example of how I was affected by this, see #21074 . 3) The optimizer affects control flow such that the coverage.py gives false positives about unreached code. As examples of how people are affected, see #2506 and https://bitbucket.org/ned/coveragepy/issues/198/continue-marked-as-not-covered . The last item on the coverage.py tracker asks "Is this limitation documented anywhere?" I do not believe that the current peephole optimizer should be changed to support these use cases, only that there should be documentation about how the optimizer may negatively affect real-world code. The domain expert on this topic is Raymond Hettinger. He does not feel safe in issues where I am involved. As I believe my continued presence on this issue will inhibit the documentation changes which I think are needed, I will remove my name from this issue and not be further involved. ---------- assignee: docs@python components: Documentation messages: 294248 nosy: dalke, docs@python priority: normal severity: normal status: open title: document peephole optimizer effects versions: Python 3.7 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30440> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
