Nick Coghlan ncoghlan at gmail.com writes:
I don't see anything wrong with the PEP 3118 protocol.
Apart from the fact that:
- it uses something (Py_buffer) which is not a PyObject and has totally
different allocation/lifetime semantics (which makes it non-trivial to adapt to
for anyone used to
Antoine Pitrou wrote:
If the memoryview wasn't holding onto a Py_buffer, one couldn't rely on its
length or anything else because the underlying object could be mutated at any
moment
Hmm, it seems there are two different approaches that could
be taken here to the design of a memoryview
Greg Ewing wrote:
In any case, I think it should be possible to implement
either version without the memoryview having to own
more than one Py_buffer and one set of shape/strides
at a time. Slicing the memoryview creates another
memoryview with its own Py_buffer and shape/strides.
The
Antoine Pitrou wrote:
Hello,
The Py_buffer struct has two pointers named `shape` and `strides`. Each points
to an array of Py_ssize_t values whose length is equal to the number of
dimensions of the buffer object. Unfortunately, the buffer protocol spec doesn't
explain how allocation of these
Alexander Belopolsky wrote:
On Mon, Dec 8, 2008 at 6:25 PM, Antoine Pitrou [EMAIL PROTECTED] wrote:
..
Alexander's suggestion of going and looking at what the numpy folks have
done in this area is probably a good idea too.
Well, I'm open to others doing this, but I won't do it myself. My
Antoine Pitrou wrote:
Alexander Belopolsky alexander.belopolsky at gmail.com writes:
I did not follow numpy development for the last year or more, so I
won't qualify as the numpy folks, but my understanding is that numpy
does exactly what Nick recommended: the viewed object owns shape and
Antoine Pitrou wrote:
Nick Coghlan ncoghlan at gmail.com writes:
For the slicing problem in particular, memoryview is currently trying to
get away with only one Py_buffer object when it needs TWO.
Why should it need two? Why couldn't the embedded Py_buffer fullfill all the
needs of the
Greg Ewing wrote:
Antoine Pitrou wrote:
Why should it need two? Why couldn't the embedded Py_buffer fullfill
all the
needs of the memoryview object?
Two things here:
1) The memoryview should *not* be holding onto a Py_buffer
in between calls to its getitem and setitem methods. It
Nick Coghlan wrote:
Antoine Pitrou wrote:
In all honesty, I admit I am annoyed by all the problems with the buffer API /
memoryview object, many of which are caused by its utterly bizarre design (and
the fact that the design team went missing in action after imposing such a
bizarre and complex
Antoine Pitrou wrote:
Nick Coghlan ncoghlan at gmail.com writes:
I don't see anything wrong with the PEP 3118 protocol.
Apart from the fact that:
- it uses something (Py_buffer) which is not a PyObject and has totally
different allocation/lifetime semantics (which makes it non-trivial to
Greg Ewing wrote:
Nick Coghlan wrote:
Maintaining a PyDict instance to map from view pointers to shapes
and strides info doesn't strike me as a complex scheme though.
I don't see why a given buffer provider should ever need
more than one set of shape/strides arrays at a time. It
can allocate
Nick Coghlan wrote:
The multi-dimensional cases get pretty tricky though, since they will
almost always end up dealing with non-contiguous data. The PEP 3118
protocol is up to handling the task, but the implementation of the index
mapping to handle these multi-dimensional cases is highly
Antoine Pitrou wrote:
- it uses something (Py_buffer) which is not a PyObject and has totally
different allocation/lifetime semantics
This was a deliberate decision -- in fact I argued for it myself.
The buffer interface is meant to be a minimal-overhead way for
C code to get at the
Antoine Pitrou wrote:
Nick Coghlan ncoghlan at gmail.com writes:
No, you misunderstand what I meant. Py_buffer doesn't need to be changed
at all. The *issuing type* would define a new structure with the
additional fields, such as:
With to the current buffer API, this is not possible. It's
Antoine Pitrou wrote:
Alexander Belopolsky alexander.belopolsky at gmail.com writes:
I did not follow numpy development for the last year or more, so I
won't qualify as the numpy folks, but my understanding is that numpy
does exactly what Nick recommended: the viewed object owns shape and
Antoine Pitrou wrote:
Alexander Belopolsky alexander.belopolsky at gmail.com writes:
I did not follow numpy development for the last year or more, so I
won't qualify as the numpy folks, but my understanding is that numpy
does exactly what Nick recommended: the viewed object owns shape and
Le mardi 09 décembre 2008 à 22:33 +1000, Nick Coghlan a écrit :
I have zero problem whatsoever if slice assignment TO a memoryview
object is permitted only if the shape stays the same (i.e. I think that
issue should be closed as not a bug).
I'm not even talking about slice /assignment/ here,
Antoine Pitrou solipsis at pitrou.net writes:
The first is that memoryview is treating the len field in the
Py_buffer struct as the number of objects in the view in a few places
instead of as the total number of bytes being exposed (it is actually
the latter, as defined in PEP 3118).
Nick Coghlan wrote:
Maintaining a PyDict instance to map from view pointers to shapes
and strides info doesn't strike me as a complex scheme though.
I don't see why a given buffer provider should ever need
more than one set of shape/strides arrays at a time. It
can allocate them on creation,
Antoine Pitrou wrote:
That doesn't work if e.g. you take a slice of a memoryview object, since the
shape changes in the process.
See http://bugs.python.org/issue4580
I haven't looked in detail at how memoryview is currently
implemented, but it seems to me that the way it should work
is that
Nick Coghlan wrote:
[from the PEP] If the exporter wants to be able to change an
object's shape, strides, and/or suboffsets before releasebuffer is
called then it should allocate those arrays when getbuffer is called
(pointing to them in the buffer-info structure provided) and free them
when
Nick Coghlan ncoghlan at gmail.com writes:
For the slicing problem in particular, memoryview is currently trying to
get away with only one Py_buffer object when it needs TWO.
Why should it need two? Why couldn't the embedded Py_buffer fullfill all the
needs of the memoryview object? If the
Antoine Pitrou wrote:
Why should it need two? Why couldn't the embedded Py_buffer fullfill all the
needs of the memoryview object?
Two things here:
1) The memoryview should *not* be holding onto a Py_buffer
in between calls to its getitem and setitem methods. It
should request
Greg Ewing greg.ewing at canterbury.ac.nz writes:
1) The memoryview should *not* be holding onto a Py_buffer
in between calls to its getitem and setitem methods. It
should request one from the underlying object when needed
and release it again as soon as possible.
If
Hello,
The Py_buffer struct has two pointers named `shape` and `strides`. Each points
to an array of Py_ssize_t values whose length is equal to the number of
dimensions of the buffer object. Unfortunately, the buffer protocol spec doesn't
explain how allocation of these arrays should be handled.
Antoine Pitrou wrote:
For the one-dimensional case, I had in mind a simple scheme where the
Py_buffer
struct has an additional two-member Py_ssize_t array. Then `shape` and
`strides`
can point to the first and second member of this array, respectively. This
wouldn't solve the
I don't have much to add to Nick's reply other than to point you to
numpy, http://projects.scipy.org/scipy/numpy, as a reference
implementation. You may also get better responses on the numpy list,
[EMAIL PROTECTED].
On Mon, Dec 8, 2008 at 3:46 PM, Nick Coghlan [EMAIL PROTECTED] wrote:
Antoine
Nick Coghlan ncoghlan at gmail.com writes:
Actually, I think your suggested scheme for the one-dimensional case
shows the way forward: ownership of the shape and strides memory belongs
to the object issuing the Py_buffer struct, and that object needs to
deal with it when the buffer is
Antoine Pitrou wrote:
Nick Coghlan ncoghlan at gmail.com writes:
Actually, I think your suggested scheme for the one-dimensional case
shows the way forward: ownership of the shape and strides memory belongs
to the object issuing the Py_buffer struct, and that object needs to
deal with it when
Nick Coghlan ncoghlan at gmail.com writes:
No, you misunderstand what I meant. Py_buffer doesn't need to be changed
at all. The *issuing type* would define a new structure with the
additional fields, such as:
With to the current buffer API, this is not possible. It's the caller who
allocates
On Mon, Dec 8, 2008 at 6:25 PM, Antoine Pitrou [EMAIL PROTECTED] wrote:
..
Alexander's suggestion of going and looking at what the numpy folks have
done in this area is probably a good idea too.
Well, I'm open to others doing this, but I won't do it myself. My interest is
in
fixing the most
Antoine Pitrou wrote:
(of course complex schemes can be devised where the callee maintains its own
separate storage for shape and strides, but I don't think we want to go there)
But that's exactly where you're supposed to be going.
If the object providing the buffer has variable-sized
shape
32 matches
Mail list logo
