[issue27129] Wordcode, part 2

Pablo Galindo Salgado added the comment: Closing as this is fixed. Feel free to reopen if there is something missing -- nosy: +pablogsal resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue27129] Wordcode, part 2

Mark Shannon added the comment: New changeset c368ce74d2c9bcbf1ec320466819c2d4768252f7 by Dennis Sweeney in branch 'master': bpo-27129: Update magic numbers and bootstrapping for GH-25069 (GH-25172) https://github.com/python/cpython/commit/c368ce74d2c9bcbf1ec320466819c2d4768252f7

[issue27129] Wordcode, part 2

Change by Ned Deily : -- priority: normal -> release blocker versions: +Python 3.10 -Python 3.6 ___ Python tracker ___ ___

[issue27129] Wordcode, part 2

Change by Dennis Sweeney : -- pull_requests: +23913 pull_request: https://github.com/python/cpython/pull/25172 ___ Python tracker ___

[issue27129] Wordcode, part 2

Dennis Sweeney added the comment: I notice this in _bootstrap_external.py: the magic number did not get changed, only the comment: # Python 3.10a2 3433 (RERAISE restores f_lasti if oparg != 0) # Python 3.10a6 3434 (PEP 634: Structural Pattern Matching) # Python 3.10a7 3435 Use

[issue27129] Wordcode, part 2

David Bolen added the comment: I don't think reverting the commit at this point would necessarily be helpful. While it might fix some systems, it could newly break anyone who happened to do their first build since the commit was in place. I didn't want to bug anyone over the weekend, but

[issue27129] Wordcode, part 2

STINNER Victor added the comment: > New changeset fcb55c0037baab6f98f91ee38ce84b6f874f034a by Mark Shannon in > branch 'master': > bpo-27129: Use instruction offsets, not byte offsets, in bytecode and > internally. (GH-25069) This change broke buildbots, please revert it to repair

[issue27129] Wordcode, part 2

David Bolen added the comment: I've opened issue #43709 for fixing the buildbot clean script under Windows. It needs to clean the Tools and Parser trees, not just Lib (and there are a few other folders involved besides clinic) -- ___ Python

[issue27129] Wordcode, part 2

David Bolen added the comment: I'm out of time for a bit, but it appears that the root issue is old pyc files in Tools/clinic/__pycache__ that aren't removed during a clean process, and appear to be the source of all of the errors. Manually pruning that folder fixes things. I believe the

[issue27129] Wordcode, part 2

David Bolen added the comment: Ah, Victor, that helps. I was having trouble reproducing the problem on a different system. I was suspecting a small difference in compiler version, but I hadn't considered it being because I started fresh. >From what I can see, a particular build tree can

[issue27129] Wordcode, part 2

STINNER Victor added the comment: I get two crashes on Windows with Python built in debug mode: * I got a crash. I wasn't sure if it was an issue of incremental build, so I rebuilt Python. * On a fresh build, Python crashed on the CALL_FUNCTION_KW opcode, when loading names, names was equal

[issue27129] Wordcode, part 2

David Bolen added the comment: Unfortunately, not at the moment - what's in the buildbot log is what's available. The RTL assertion aborts the process. The tests involved (such as test_clinic) do seem reproducible in a few separate tries, though again, all they do is terminate. As the

[issue27129] Wordcode, part 2

Mark Shannon added the comment: That assertion is correct, and hasn't changed. Do you have a traceback? The buildbot just shows the assertion message with no context. -- ___ Python tracker

[issue27129] Wordcode, part 2

David Bolen added the comment: Note that this commit appears to be causing exceptions for the Win10 buildbot, failing the PyCode_Addr2Line assertion in codeobject.c line 1252. The assertion seems to pop up at differing points during each test run, but the builder has yet to complete a full

[issue27129] Wordcode, part 2

Mark Shannon added the comment: New changeset fcb55c0037baab6f98f91ee38ce84b6f874f034a by Mark Shannon in branch 'master': bpo-27129: Use instruction offsets, not byte offsets, in bytecode and internally. (GH-25069)

[issue27129] Wordcode, part 2

Change by Mark Shannon : -- pull_requests: +23819 pull_request: https://github.com/python/cpython/pull/25069 ___ Python tracker ___

[issue27129] Wordcode, part 2

Mark Shannon added the comment: frame.f_lasti and traceback.tb_lasti are best left as byte offsets. There is no guarantee that we won't go back to variable length instructions. For example, a "LONG_JUMP" instruction which is 4 bytes long and takes a 3 byte offset might well be a worthwhile

[issue27129] Wordcode, part 2

Change by Brett Cannon : -- nosy: -brett.cannon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue27129] Wordcode, part 2

Kirill Balunov added the comment: Hello, what is the future of this patch? Such a feeling that the transition to wordcode is still in some half-way state. -- nosy: +godaygo ___ Python tracker

[issue27129] Wordcode, part 2

STINNER Victor added the comment: Serhiy Storchaka added the comment: > No, only the simpler and safer part was committed. I divided the original > patch on four parts, but since the code was significantly evolved, they no > longer applied clearly. Oh ok. I saw that a change was pushed, I

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: No, only the simpler and safer part was committed. I divided the original patch on four parts, but since the code was significantly evolved, they no longer applied clearly. -- ___ Python tracker

[issue27129] Wordcode, part 2

STINNER Victor added the comment: This issue can now be closed, no? -- ___ Python tracker ___ ___

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: Thanks Berker! -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue27129] Wordcode, part 2

Roundup Robot added the comment: New changeset b49a938eaa31 by Serhiy Storchaka in branch 'default': Fixed refactoring bug in dd046963bd42 (issue27129). https://hg.python.org/cpython/rev/b49a938eaa31 -- ___ Python tracker

[issue27129] Wordcode, part 2

Berker Peksag added the comment: >From >http://buildbot.python.org/all/builders/s390x%20Debian%203.x/builds/1811/steps/compile/logs/stdio > (I also saw the same compile error on another Linux boxes) _freeze_importlib: Python/peephole.c:524: PyCode_Optimize: Assertion `((codestr[i]) >> 8) ==

[issue27129] Wordcode, part 2

Roundup Robot added the comment: New changeset dd046963bd42 by Serhiy Storchaka in branch 'default': Issue #27129: Replaced wordcode related magic constants with macros. https://hg.python.org/cpython/rev/dd046963bd42 -- nosy: +python-dev ___ Python

[issue27129] Wordcode, part 2

Demur Rumed added the comment: The patches LGTM & seem to be implementation of follow up ideas outlined in the first portion. It'd be good to verify that benchmarks are relatively unaffected -- ___ Python tracker

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: update patch replaces yet few magic constants. -- Added file: http://bugs.python.org/file4/wordcode-refactor2.patch ___ Python tracker

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: This is not just about cleaning (to my eyes current code is not very readable, and I read it many times, perhaps more times than any other core developer in last months). There are other benefits. Changing jump offsets allows to get rid of EXTENDED_ARGs for

[issue27129] Wordcode, part 2

Raymond Hettinger added the comment: I really don't think any of this should be done at all. The current code is clean and fast (and to my eyes at least is very readable). -- ___ Python tracker

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: Here is a patch that just refactors the code. It renames OPCODE and OPCODE to _Py_OPCODE and _Py_OPCODE and moves them to code.h for reusing in other files. Introduces _Py_CODEUNIT as an alias to unsigned short (if it is 16-bit, otherwise a compile error is

[issue27129] Wordcode, part 2

Changes by Serhiy Storchaka : -- nosy: +Mark.Shannon ___ Python tracker ___ ___

[issue27129] Wordcode, part 2

Changes by Eric Fahlgren : -- nosy: +eric.fahlgren ___ Python tracker ___ ___

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: word-jump-offsets.patch doesn't simplify the code, but rather complicates it, because this is minimal patch that doesn't include the refactoring. The main benefit of this patch is that it extends the range addressed by short jump instruction. The other

[issue27129] Wordcode, part 2

Changes by Demur Rumed : Removed file: http://bugs.python.org/file43048/forbegin.patch ___ Python tracker ___

[issue27129] Wordcode, part 2

Changes by Emanuel Barry : -- Removed message: http://bugs.python.org/msg266607 ___ Python tracker ___

[issue27129] Wordcode, part 2

Changes by Emanuel Barry : -- Removed message: http://bugs.python.org/msg266608 ___ Python tracker ___

[issue27129] Wordcode, part 2

Demur Rumed added the comment: Oops wrong issue, sorry. Please delete? -- ___ Python tracker ___ ___

[issue27129] Wordcode, part 2

Demur Rumed added the comment: Attached is the larger potential change of replacing GET_ITER/FOR_ITER with FOR_BEGIN/FOR_ITER. I took awhile to getting this put together due to missing that CONTINUE_LOOP was jumping to the top of the loop, skipping past FOR_ITER -- Added file:

[issue27129] Wordcode, part 2

Raymond Hettinger added the comment: I don't see how this is a simplification. The additional /2 and *2 on the affected lines makes the code a little harder to reason about and it loses some of the cleanness achieved by the last patch. To me, it also increases conceptual complexity

[issue27129] Wordcode, part 2

Changes by Serhiy Storchaka : Added file: http://bugs.python.org/file43040/word-jump-offsets.patch ___ Python tracker ___

[issue27129] Wordcode, part 2

Changes by Serhiy Storchaka : Removed file: http://bugs.python.org/file43039/word-jump-offsets.patch ___ Python tracker ___

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: Here is a patch that implements only the first change -- makes jump offsets be in 16-bit units, not bytes. This is minimal change, it doesn't include refactoring. -- Added file: http://bugs.python.org/file43039/word-jump-offsets.patch

[issue27129] Wordcode, part 2

STINNER Victor added the comment: > if the co_code object is remaining a bytes object then it seems intuitive to > keep f_lasti as a bytes offset Right. > In reality most of the results on github all seem to be copying a few > distinct uses. So maybe backwards compatibiltiy isn't so

[issue27129] Wordcode, part 2

Demur Rumed added the comment: https://github.com/search?q=f_lasti=Code Popular use of f_lasti is checking it for -1, checking the instruction at the byte offset of f_lasti, checking the argument with code[f_lasti+1] (Some bad code checking f_lasti+3 which'll break with 3.6) abarnert

[issue27129] Wordcode, part 2

Serhiy Storchaka added the comment: The patch contains the change of Lib/asyncio/coroutines.py. This is the only change in Python code besides the dis module. I can keep f_lasti to be twice the number of instructions, but this will complicate the patch. The simplest way perhaps is to convert

[issue27129] Wordcode, part 2

Brett Cannon added the comment: So is avoiding changing f_lasti just to minimize breakage of tools? Aren't they going to have to update to support the wordcode changes anyway? -- ___ Python tracker

[issue27129] Wordcode, part 2

Changes by Brett Cannon : -- nosy: +brett.cannon ___ Python tracker ___ ___

[issue27129] Wordcode, part 2

STINNER Victor added the comment: > Changes f_lasti, tb_lasti etc to count code units instead of bytes. I asked Demur to not break f_lasti. I don't understand if this change breaks applications using f_lasti or not. For example, asyncio/coroutines.py uses: if

[issue27129] Wordcode, part 2

New submission from Serhiy Storchaka: This is the second stage of converting to wordcode (issue26647). Proposed patch makes bytecodecode consisting of code units (16-bit words) instead of bytes. It includes following changes: * Changes meaning of jump offsets. They counts not bytes, but code