[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:


New changeset 93602e3af70d3b9f98ae2da654b16b3382b68d50 by Nick Coghlan in 
branch '3.5':
[3.5] bpo-29537: Tolerate legacy invalid bytecode (#169)
https://github.com/python/cpython/commit/93602e3af70d3b9f98ae2da654b16b3382b68d50


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:


New changeset 0410bee6e6c87f37e91f6cae3627d8e3704075f1 by Nick Coghlan in 
branch '3.5':
bpo-29537: Also cover 3.5.2 in NEWS entry
https://github.com/python/cpython/commit/0410bee6e6c87f37e91f6cae3627d8e3704075f1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +459

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

Merged (with test cases) in 
https://github.com/python/cpython/commit/93602e3af70d3b9f98ae2da654b16b3382b68d50

The test cases even cover ensuring the backwards compatibility also applies to 
frozen bytecode :)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

I just realised I need to add some test cases to 
test_importlib/source/test_file_loader.py before merging it, but since Larry 
hasn't objected to the proposed approach, I'm going to go ahead and implement 
this fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Iryna]

Iryna added the comment:

If I may ask, what was the decision on this matter?

We are planning to rebase Python 3.5 for Fedora and this currently blocks us, 
if we do not work this around with a patch.
Let me know if there is anything I can help with to speed up the process.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

I updated the PR to cover both Serhiy's change to make the legacy bytecode safe 
to interpret, and a tweaked version of Petr's change to allow the legacy 
bytecode to be imported.

The main tweaks to the latter were:

- more in depth comments explaining the changes
- switching the C level changes to rely on a couple of extern variables 
exported from import.c rather than scattering the backwards compatibility 
numbers throughout the code

I'm also cc'ing Larry into the discussion as 3.5 release manager. Larry, most 
of the context for this change is actually in http://bugs.python.org/issue29514 
where we reported the problems with the magic number change that prompted Petr 
to create a downstream patch for Fedora to restore compatibility with the 
legacy bytecode magic number.

While the answer for 3.6+ is "Let's try to avoid ever doing this again", it 
turns out this is tricky enough to handle that it would be nice to have a 
shared solution upstream in 3.5.4+ that redistributors can collectively adopt, 
rather than everyone needing to come up with their own workaround for the 
problem.

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

For easier cross-referencing, note that http://bugs.python.org/issue29514 is 
the issue proposing a test case that ensures future maintainers are aware of 
the practical problems created by bumping the bytecode magic number in a 
maintenance release.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Changes by Nick Coghlan :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Changes by Nick Coghlan :


--
assignee:  -> ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

In putting together a PR for this, I think it *only* makes sense if we also 
push Petr's change upstream to accept the legacy bytecode files in 3.5.4+.

The reason is that the patch actually causes a test case to fail due to "f()" 
change to "function" in an error message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Changes by Nick Coghlan :


--
pull_requests: +135

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Mark Dickinson]

Changes by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Iryna]

Changes by Iryna :


--
nosy: +ishcherb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Nick Coghlan]

Nick Coghlan added the comment:

Thanks Serhiy, I think that will work well downstream in combination with 
Petr's patch to accept both magic numbers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Proposed patch is an alternative fix of issue27286 for Python 3.5. Rather than 
fixing the compiler and bumping the magic number it just avoids using incorrect 
value in eval loop. The patch can be useful for disro maintainers that don't 
want to recompile all .pyc files when update Python to 3.5.3. I think it would 
be the best way to fix issue27286. Now it can be combined with either reverting 
issue27286 changes or applying the workaround 
https://github.com/encukou/cpython/commit/magic-number-workaround .

The patch makes the high byte of oparg be used only when it is unambiguous 
(equal to 1). In that case the error message contains a function name. 
Otherwise the error message doesn't contain a function name. This is small 
usability regression, but the BUILD_MAP_UNPACK_WITH_CALL opcode is very rarely 
used and shouldn't raise an exception in normal case.

--
components: Interpreter Core
messages: 287637
nosy: barry, encukou, ncoghlan, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29537] Alternative fix for BUILD_MAP_UNPACK_WITH_CALL opcode in 3.5

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +patch
Added file: 
http://bugs.python.org/file46634/BUILD_MAP_UNPACK_WITH_CALL-no-function_location.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com