On 30.08.16 20:42, Nick Coghlan wrote:
On 28 August 2016 at 08:25, Terry Reedy <tjre...@udel.edu> wrote:
Slicing can be made to malfunction and even crash with an 'evil' __index__
method. https://bugs.python.org/issue27867
The crux of the problem is this: PySlice_GetIndicesEx
receives a slice object and a sequence length. Calling __index__ on the
start, stop, and step components can mutate the sequence and invalidate the
length. Adjusting the int values of start and stop according to an invalid
length (in particular, one that is too long) will result in invalid results
or a crash.
Possible actions -- very briefly. For more see end of
https://bugs.python.org/issue27867?@ok_message=msg 273801
0. Do nothing.
1. Detect length change and raise.
2. Retrieve length after any possible changes and proceed as normal.
Possible implementation strategies for 1. and 2.
A. Change all functions that call PySlice_GetIndicesEx.
B. Add PySlice_GetIndicesEx2 (or ExEx?), which would receive *collection
instead of length, so the length could be retrieved after the __index__
calls. Change calls. Deprecate PySlice_GetIndicesEx.
Given Serhiy's clarification that this is primarily a thread safety
problem, I'm more supportive of the "PySlice_GetIndicesForObject"
approach (since that can call all the __index__ methods first, leaving
the final __len__ call as the only problematic case).
This doesn't work with multidimensional slicing (like
_testbuffer.ndarray or NumPy arrays).
However, given the observation that __len__ can also release the GIL,
I'm not clear on how 2A is supposed to work - a poorly timed thread
switch means there's always going to be a risk of len(obj) returning
outdated information if a container implemented in Python is being
mutated concurrently from different threads, so what can be done
differently in the calling functions that couldn't be done in a new
API that accepted the container reference?
Current code doesn't use __len__. It uses something like
PyUnicode_GET_LENGTH().
The solution was found easier than I afraid. See my patch to issue27867.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
