Nick Coghlan <ncogh...@gmail.com> added the comment: On Mon, Feb 14, 2011 at 8:33 AM, Antoine Pitrou <rep...@bugs.python.org> wrote: >> I'm still not comfortable with a convention that relies on *clients* >> of the PEP 3118 API not mucking with the internals of the Py_buffer >> struct. > > Which clients? Those who export the buffer, or those who consume it?
Consumers. (I'll try to stick to provider/consumer terminology, as that's clearer in this context >> I'm *much* happier with the rule based on malloc/free semantics where >> the *pointer* passed to PyObject_GetBuffer must match a single later >> call to PyObject_ReleaseBuffer. > > Agreed that Py_buffer should have been a PyObject from the start, but > the PEP chose differently. malloc/free modelled semantics have *nothing* to do with Py_buffer being a full PyObject in its own right. All I mean is that whatever pointer you call ReleaseBuffer with should be the one you passed to GetBuffer, and the only thing tp_releasebuffer implementations should rely on is the address of that pointer rather than the struct contents. However, from what Pauli has said, we may want to go with the alternative approach of saying the struct address is irrelevant, and only the content matter, using the "internal" field to disambiguate different exported buffers. I believe either will work, and either places additional constraints on buffer API consumers that aren't currently clearly documented. > We now have backwards compatibility constraints. What do we do with > PyMemoryView_FromBuffer? Also, there's probably some code out there that > likes to copy Py_buffers around. Such code is likely to be broken regardless of how we clarify the semantics, in the same way that our own dup_buffer is currently broken under either set of semantics (i.e. calling ReleaseBuffer with a different address in one case, clobbering the "internal" field in other cases). We will probably need to expose an official Py_buffer copying function that gets all the subtle details right so that extension authors can more easily avoid making the same mistakes. >> As far as the question of re-exporting the underlying view or not >> goes, I agree having "memoryview(a)" potentially refer to different >> underlying memory from "a" itself (because the source object has >> changed since the first view was exported) is a recipe for confusion. > > If an object changes its buffer while it's exported somewhere, it will > always result in confusion for the user, regardless of how the > memoryview object is implemented. All normal uses of the buffer API > assume that the buffer's memory doesn't change while it's being accessed > by its consumer (what would it mean to SHA1-hash or zlib-compress a > changing piece of memory?). > So I don't know why the memoryview object *in particular* should care > about this. I'm not talking about an exported view changing its details (that's explicitly disallowed by the PEP), I'm talking about the fact that sequential calls to PyObject_GetBuffer are permitted to return different answers. That's the point Pauli's PictureSet example illustrated - even though the toy example uses a somewhat clumsy API, it's perfectly legitimate according to the documentation, and it shows that the current memoryview implementation may behave strangely when you copy or slice a view of a mutable object, even though the view itself is guaranteed to remain valid. Consider the following: Traceback (most recent call last): File "<stdin>", line 1, in <module> BufferError: Existing exports of data: object cannot be re-sized Now suppose that instead of disallowing the resize, bytearray (or a similar object) permitted it by allocating a new memory buffer, while keeping a reference to the old buffer around until the memoryview releases it (an approach that is perfectly legitimate according to the PEP). In that case, our current "use the source object" approach to memoryview copying and slicing will backfire badly, since copies and slices will be working off the *new* (empty) state of the object, while the original memoryview will still be looking at the old populated state. I think Pauli's right, we need to make memoryview re-exporting significantly smarter in order to cope correctly with mutable objects. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue10181> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com