New submission from Benjamin Keen <benjamin.k...@hpe.com>:
Currently the array.array object can export a memoryview, but there is no way
to construct one from a memoryview without making a copy of the underlying
data. So in that sense array.array only supports one half of the buffer
protocol and this is to allow for the other half.
This proposal is to allow the array object to be constructed from an existing
exported buffer without copying and reallocating the memory, permitting
operations that can modify the underlying buffer's contents but not the
allocation.
This is useful when working with many small pieces of one very large underlying
buffer that you do not want to copy, when desiring to work with different parts
of it with different types, and as part of a way to work with shared memory in
multiple processes.
I will shortly have a PR for this, including updates for the documentation and
unit tests.
- Modules/arraymodule.c already must check if the array object has exported a
buffer for methods that might resize. If the array was constructed from an
imported buffer, the same restrictions apply. So the object just needs to know
whether it is constructed from a Py_Buffer or not and check in the same places
it checks for the export count being nonzero. So the code doesn't need to be
perturbed that much.
- Only MemoryView objects with contiguous layout, size, and alignment
compatible with the data type of the array element are allowed.
- I'm proposing this is only for when it's an actual memoryview object, not
just if the object can export buffers. This preserves more of the existing
behavior.
- Currently you /can/ initialize an array with a type-compatible memoryview -
but it makes a copy, iterating the elements and the types have to match, not
just in size. We could maintain exact backward compatibility by adding an extra
argument to array.array() or another letter to the format specifier; my current
patch doesn't do this though.
-----------------------------------------------------------
Example of current behavior:
>>> import array
>>> x = array.array('b', [1,2,3,4])
>>> y = memoryview(x)
>>> z = array.array('b', y)
>>> z
array('b', [1, 2, 3, 4])
>>> z[0] = 42
>>> x
array('b', [1, 2, 3, 4])
>>> z
array('b', [42, 2, 3, 4])
# x and z are backed by different memory
>>> x.append(17)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
BufferError: cannot resize an array that is exporting buffers
# this is because y is still a live object
>>> z.append(17)
# it is really a copy, x and y are irrelevant to z
>>> z
array('b', [42, 2, 3, 4, 17])
----------------------------------------
Example of new behavior:
>>> import array
>>> x = array.array('b', [1,2,3,4])
>>> x
array('b', [1, 2, 3, 4])
>>> y = memoryview(x)
>>> z = array.array('b', y)
>>> z
array('b', [1, 2, 3, 4])
>>> z[0] = 42
>>> x
array('b', [42, 2, 3, 4])
>>> x.append(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
BufferError: cannot resize an array that is exporting buffers
>>> z.append(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
BufferError: cannot resize an array constructed from an imported buffer
----------
components: Library (Lib)
messages: 367688
nosy: bjkeen
priority: normal
severity: normal
status: open
title: allow array.array construction from memoryview w/o copy
type: enhancement
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue40440>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
