Re: [Python-Dev] Extended Buffer Interface/Protocol

[Carl Banks]

(cc'ing back to Python-dev; the original reply was intended for it by I 
had an email malfunction.)

Travis Oliphant wrote:
 >Carl Banks wrote:
>> 3. Allow getbuffer to return an array of "derefence offsets", one for 
>> each dimension.  For a given dimension i, if derefoff[i] is 
>> nonnegative, it's assumed that the current position (base pointer + 
>> indexing so far) is a pointer to a subarray, and derefoff[i] is the 
>> offest in that array where the current position goes for the next 
>> dimension.  If derefoff[i] is negative, there is no dereferencing.  
>> Here is an example of how it'd work:
> 
> 
> This sounds interesting, but I'm not sure I totally see it.  I probably 
> need a picture to figure out what you are proposing. 

I'll get on it sometime.  For now I hope an example will do.


> The derefoff 
> sounds like some-kind of offset.   Is that enough?  Why not just make 
> derefoff[i] == 0 instead of negative?

I may have misunderstood something.  I had thought the values exported 
by getbuffer could change as the view narrowed, but I'm not sure if it's 
the case now.  I'll assume it isn't for now, because it simplifies 
things and demonstrates the concept better.

Let's start from the beginning.  First, change the prototype to this:

 typedef PyObject *(*getbufferproc)(PyObject *obj, void **buf,
Py_ssize_t *len, int *writeable,
char **format, int *ndims,
Py_ssize_t **shape,
Py_ssize_t **strides,
int **isptr)

"isptr" is a flag indicating whether, for a certain dimension, the 
positision we've strided to so far is a pointer that should be followed 
before proceeding with the rest of the strides.

Now here's what a general "get_item_pointer" function would look like, 
given a set of indices:

void* get_item_pointer(int ndim, void* buf, Py_ssize_t* strides,
Py_ssize_t* derefoff, Py_ssize_t *indices) {
 char* pointer = (char*)buf;
 int i;
 for (i = 0; i < ndim; i++) {
 pointer += strides[i]*indices[i];
 if (isptr[i]) {
 pointer = *(char**)pointer;
 }
 }
 return (void*)pointer;
}


> I don't fully understand the PIL example you gave.

Yeah.  How about more details.  Here is a hypothetical image data object 
structure:

struct rgba {
 unsigned char r, g, b, a;
};

struct ImageObject {
 PyObject_HEAD;
 ...
 struct rgba** lines;
 Py_ssize_t height;
 Py_ssize_t width;
 Py_ssize_t shape_array[2];
 Py_ssize_t stride_array[2];
 Py_ssize_t view_count;
};

"lines" points to malloced 1-D array of (struct rgba*).  Each pointer in 
THAT block points to a seperately malloced array of (struct rgba).  Got 
that?

In order to access, say, the red value of the pixel at x=30, y=50, you'd 
use "lines[50][30].r".

So what does ImageObject's getbuffer do?  Leaving error checking out:

PyObject* getbuffer(PyObject *self, void **buf, Py_ssize_t *len,
 int *writeable, char **format, int *ndims,
 Py_ssize_t **shape, Py_ssize_t **strides,
 int **isptr) {

 static int _isptr[2] = { 1, 0 };

 *buf = self->lines;
 *len = self->height*self->width;
 *writable = 1;
 *ndims = 2;
 self->shape_array[0] = height;
 self->shape_array[1] = width;
 *shape = &self->shape_array;
 self->stride_array[0] = sizeof(struct rgba*);  /* yep */
 self->stride_array[1] = sizeof(struct rgba);
 *strides = &self->stride_array;
 *isptr = _isptr;

 self->view_count ++;
 /* create and return view object here, but for what? */
}


There are three essential differences from a regular, contiguous array.

1. buf is set to point at the array of pointers, not directly to the data.

2. The isptr thing.  isptr[0] is true to indicate that the first 
dimension is an array of pointers, not the actual data.

3. stride[0] is sizeof(struct rgba*), not self->width*sizeof(struct 
rgba) like it would be for a contiguous array.  This is because your 
first stride is through an array of pointers, not the data itself.


So let's examine what "get_item_pointer" above will do given these 
values.  Once again, we're looking for the pixel at x=30, y=50.

First, we set pointer to buf, that is, self->lines.

Then we take the first stride: we add index[0]+strides[0], that is, 
50*4=200, to poitner.  pointer now equals &self->lines[50].

Now, we check isptr[0].  We see that it is true.  Thus, the position 
we've strided to is, in fact, a pointer to a subarray where the actual 
data is.  So we follow it: pointer = *pointer.  pointer now equals 
self->lines[50] which equals &self->lines[50][0].

Next dimension.  We take the second stride: we add index[1]+strides[1], 
that is, 30*4=120, to pointer.  pointer now equals &self->lines[50][30].

Now, we check is

Re: [Python-Dev] Fwd: RFC - GoogleSOC proposal -cleanupurllib

[Senthil]

On 3/24/07, Josiah Carlson <[EMAIL PROTECTED]> wrote:
>
> If I were to summarize your proposal, it would be "merge, clean up, and
> add features to urllib, urllib2, and urlparse".  Sounds reasonable for
> Python 3, but remember that you will need to update any standard library
> module that currently uses urllib, urllib2, and urlparse to make sure
> that they pass the test suite, probably add tests to the test suite for
> your new module, and include documentation (I can't remember if any of
> that was in your proposal).

Thanks Josiah,
I have included the tasks of updating other standard library modules
that use urllib, urllib2, and urlparse, including tests and
documentation in my proposal.
That was a very helpful feedback.

I have submitted my proposal in the web app as well.
http://puggy.symonds.net/~senthil/summer_code_urllib.txt

Thanks,
Senthil


> "Senthil Kumaran" <[EMAIL PROTECTED]> wrote:
> > Resending this; as i think my sf.net email alias got blocked by the 
> > python-dev.
> >
> >
> > -- Forwarded message --
> > From: Senthil <[EMAIL PROTECTED]>
> > Date: Mar 24, 2007 6:05 AM
> > Subject: RFC - GoogleSOC proposal -cleanupurllib
> > To: python-dev@python.org
> >
> >
> > Hi All,
> > I have written a proposal to cleanup urllib as part of Google SoC. I
> > am attaching the file 'soc1' with this email. Requesting you to go
> > through the proposal and provide any feedback which I can incorporate
> > in my submission.
>
>


-- 
O.R.Senthil Kumaran

http://phoe6.livejournal.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extended Buffer Interface/Protocol

[Travis Oliphant]

Carl Banks wrote:
> We're done.  Return pointer.

Thank you for this detailed example.  I will have to parse it in more 
depth but I think I can see what you are suggesting.

> 
> First, I'm not sure why getbuffer needs to return a view object. 

The view object in your case would just be the ImageObject.  The reason 
I was thinking the function should return "something" is to provide more 
flexibility in what a view object actually is.

I've also been going back and forth between explicitly passing all this 
information around or placing it in an actual view-object.  In some 
sense, a view object is a NumPy array in my mind.  But, with the 
addition of isptr we are actually expanding the memory abstraction of 
the view object beyond an explicit NumPy array.

In the most common case, I envisioned the view object would just be the 
object itself in which case it doesn't actually have to be returned. 
While returning the view object would allow unspecified flexibilty in 
the future, it really adds nothing to the current vision.

We could add a view object separately as an abstract API on top of the 
buffer interface.

> 
> 
> Second question: what happens if a view wants to re-export the buffer? 
> Do the views of the buffer ever change?  Example, say you create a 
> transposed view of a Numpy array.  Now you want a slice of the 
> transposed array.  What does the transposed view's getbuffer export?

Basically, you could not alter the internal representation of the object 
while views which depended on those values were around.

In NumPy, a transposed array actually creates a new NumPy object that 
refers to the same data but has its own shape and strides arrays.

With the new buffer protocol, the NumPy array would not be able to alter 
it's shape/strides/or reallocate its data areas while views were being 
held by other objects.

With the shape and strides information, the format information, and the 
data buffer itself, there are actually several pieces of memory that may 
need to be protected because they may be shared with other objects. 
This makes me wonder if releasebuffer should contain an argument which 
states whether or not to release the memory, the shape and strides 
information, the format information, or all of it.

Having such a thing as a view object would actually be nice because it 
could hold on to a particular view of data with a given set of shape and 
strides (whose memory it owns itself) and then the exporting object 
would be free to change it's shape/strides information as long as the 
data did not change.

> 
> The reason I ask is: if things like "buf" and "strides" and "shape" 
> could change when a buffer is re-exported, then it can complicate things 
> for PIL-like buffers.  (How would you account for offsets in a dimension 
> that's in a subarray?)

I'm not sure what you mean, offsets are handled by changing the starting 
location of the pointer to the buffer.

-Travis
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: RFC - GoogleSOC proposal -cleanupurllib

[Senthil]

On 3/25/07, Mike Brown <[EMAIL PROTECTED]> wrote:
> From your proposal:
>
> > 2) In all modules, Follow the new RFC 2396 in favour of RFC 1738 and RFC 
> > 1808.
>
> The "new" RFC 2396 was superseded by STD 66 (RFC 3986) two years ago. Your
> failure to notice this development doesn't bode well :) j/k, although it does
> undermine confidence somewhat.

I apologize, Mike. I over-looked it and perhaps did not do sufficient
research on the RFCs.
I read through the RFC3986 and RFC3987, and have changed the proposal
on changes urllib to be compliant with these current RFCs.

> In any case, I have a few suggestions:

Thank you for your suggestions. I went through your discussions on
urllib utf issues and I think a careful study and discussions with
people is much neccessary before any implementation thought.
4Suite.org references to RFC3986 comtible methods are helpful.

With the comments incorporated, I have submited the proposal:
http://puggy.symonds.net/~senthil/summer_code_urllib.txt

Thanks,

O.R.Senthil Kumaran

http://phoe6.livejournal.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SoC proposal: multimedia library

[Pete Shinners]

Lino Mastrodomenico  gmail.com> writes:
> I would like to participate as a student in google Summer of Code and
> I'm interested in feedback on a multimedia library for Python.
> 
> The library I propose should have the following features:
> * the capability to extract and decompress video and audio from a
> few common multimedia file format;
> * and, vice versa, it can create a compressed video file from a
> series of still images and from uncompressed audio;
> * it should have an interface as simple and pythonic as possible;

My main question is what is the image and sound container passed back to Python?
This single thing along would be worth a SoC if it could be implemented across
all libraries.

Will your image objects be transferrable to PIL, Pygame, PyOpengl, Numpy,
Pythonmagick, Pycairo, wxPython, etc? It would be best if this could avoid the
"fromstring/tostring" mojo that always requires and extra copy of the data for
each transfer.

If the core numpy arrays could ever get added to the Python standard library,
this problem would be mostly solved. I believe there was a SoC for Python last
year for this exact array problem.

Your proposal sounds dangerously close to implementing some file demuxing by
yourself. Do not dare touch any of the file bits in your own library. This
proposal should only be for getting data from existing decompression libraries
into some general Python container. 


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extended Buffer Interface/Protocol

[Carl Banks]

Travis Oliphant wrote:
> Carl Banks wrote:
>> We're done.  Return pointer.
> 
> Thank you for this detailed example.  I will have to parse it in more 
> depth but I think I can see what you are suggesting.
> 
>> First, I'm not sure why getbuffer needs to return a view object. 
> 
> The view object in your case would just be the ImageObject.  

ITSM that we are using the word "view" very differently.  Consider this 
example:

A = zeros((100,100))
B = A.transpose()

In this scenario, A would be the exporter object, I think we both would 
call it that.  When I use the word "view", I'm referring to B.  However, 
you seem to be referring to the object returned by A.getbuffer, right? 
What term have you been using to refer to B?  Obviously, it would help 
the discussion if we could get our terminology straight.

(Frankly, I don't agree with your usage; it doesn't agree with other 
uses of the word "view".  For example, consider the proposed Python 3000 
dictionary views:

D = dict()
V = D.items()

Here, V is the view, and it's analogous to B in the above example.)

I'd suggest the object returned by A.getbuffer should be called the 
"buffer provider" or something like that.

For the sake of discussion, I'm going to avoid the word "view" 
altogether.  I'll call A the exporter, as before.  B I'll refer to as 
the requestor.  The object returned by A.getbuffer is the provider.


 > The reason
 > I was thinking the function should return "something" is to provide more
 > flexibility in what a view object actually is.
 >
> I've also been going back and forth between explicitly passing all this 
> information around or placing it in an actual view-object.  In some 
> sense, a view object is a NumPy array in my mind.  But, with the 
> addition of isptr we are actually expanding the memory abstraction of 
> the view object beyond an explicit NumPy array.
>
> In the most common case, I envisioned the view object would just be the 
> object itself in which case it doesn't actually have to be returned. 
> While returning the view object would allow unspecified flexibilty in 
> the future, it really adds nothing to the current vision.
 >
> We could add a view object separately as an abstract API on top of the 
> buffer interface.

Having thought quite a bit about it, and having written several abortive 
replies, I now understand it and see the importance of it.  getbuffer 
returns the object that you are to call releasebuffer on.  It may or may 
not be the same object as exporter.  Makes sense, is easy to explain.

It's easy to see possible use cases for returning a different object.  A 
  hypothetical future incarnation of NumPy might shift the 
responsibility of managing buffers from NumPy array object to a hidden 
raw buffer object.  In this scenario, the NumPy object is the exporter, 
but the raw buffer object the provider.

Considering this use case, it's clear that getbuffer should return the 
shape and stride data independently of the provider.  The raw buffer 
object wouldn't have that information; all it does is store a pointer 
and keep a reference count.  Shape and stride is defined by the exporter.


>> Second question: what happens if a view wants to re-export the buffer? 
>> Do the views of the buffer ever change?  Example, say you create a 
>> transposed view of a Numpy array.  Now you want a slice of the 
>> transposed array.  What does the transposed view's getbuffer export?
> 
> Basically, you could not alter the internal representation of the object 
> while views which depended on those values were around.
>
> In NumPy, a transposed array actually creates a new NumPy object that 
> refers to the same data but has its own shape and strides arrays.
> 
> With the new buffer protocol, the NumPy array would not be able to alter 
> it's shape/strides/or reallocate its data areas while views were being 
> held by other objects.

But requestors could alter their own copies of the data, no?  Back to 
the transpose example: B itself obviously can't use the same "strides" 
array as A uses.  It would have to create its own strides, right?

So, what if someone takes a slice out of B?  When calling B.getbuffer,
does it get B's strides, or A's?

I think it should get B's.  After all, if you're taking a slice of B, 
don't you care about the slicing relative to B's axes?  I can't really 
think of a use case for exporting A's stride data when you take a slice 
of B, and it doesn't seem to simplify memory management, because B has 
to make it's own copies anyways.


> With the shape and strides information, the format information, and the 
> data buffer itself, there are actually several pieces of memory that may 
> need to be protected because they may be shared with other objects. 
> This makes me wonder if releasebuffer should contain an argument which 
> states whether or not to release the memory, the shape and strides 
> information, the format information, or all of it.

Here's what I think: the lock should only app

Re: [Python-Dev] SoC proposal: multimedia library

[Greg Ewing]

Pete Shinners wrote:

> Will your image objects be transferrable to PIL, Pygame, PyOpengl, Numpy,
> Pythonmagick, Pycairo, wxPython, etc? It would be best if this could avoid the
> "fromstring/tostring" mojo that always requires and extra copy of the data for
> each transfer.

This would be an excellent test case for the enhanced
buffer protocol that's currently being worked on.

I wouldn't worry about this too much initially, though.
Just get some kind of library-independent image/sound
object working, and a buffer interface can be added
later once the details are worked out.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extended Buffer Interface/Protocol

[Greg Ewing]

Carl Banks wrote:

> It's easy to see possible use cases for returning a different object.  A 
>   hypothetical future incarnation of NumPy might shift the 
> responsibility of managing buffers from NumPy array object to a hidden 
> raw buffer object.  In this scenario, the NumPy object is the exporter, 
> but the raw buffer object the provider.

But since the NumPy object has to know about the provider,
it can simply pass the release call on to it if appropriate.
I don't see how this case necessitates making the release call
on a different object.

I'm -1 on involving any other objects or returning object
references from the buffer interface, unless someone can
come up with a use case which actually *requires* this
(as opposed to it just being something which might be
"nice to have"). The buffer interface should be Blazingly
Fast(tm), and messing with PyObject*s is not the way to
get that.

> > This makes me wonder if releasebuffer should contain an argument which 
> > states whether or not to release the memory, the shape and strides 
> > information, the format information, or all of it.
>
> Here's what I think: the lock should only apply to the buffer itself, 
> and not to shape and stride data at all.  If the requestor wants to keep
> its own copies of the data, it would have to malloc its own storage for 
> it.  I expect that this would be very rare.

Seems to me the lock should apply to *everything* returned
by getbuffer. If the requestor is going to iterate over the
data, and there are multiple dimensions, surely it's going to
want to refer to the shape and stride info at regular intervals
while it's doing that. Requiring it to make its own copy
would be a burden.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] An updated extended buffer PEP

[Travis Oliphant]

Hi Carl and Greg,

Here is my updated PEP which incorporates several parts of the 
discussions we have been having. 

-Travis


 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] An updated extended buffer PEP

[Travis Oliphant]

Travis Oliphant wrote:
> Hi Carl and Greg,
> 
> Here is my updated PEP which incorporates several parts of the 
> discussions we have been having. 
> 

And here is the actual link:

http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/doc/pep_buffer.txt




> -Travis
> 
> 
>  
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
> 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SoC proposal: multimedia library

[Andrew Bennetts]

Lino Mastrodomenico wrote:
> Hello everyone,
> 
> I would like to participate as a student in google Summer of Code and
> I'm interested in feedback on a multimedia library for Python.
> 
> The library I propose should have the following features:
> * the capability to extract and decompress video and audio from a
> few common multimedia file format;
> * and, vice versa, it can create a compressed video file from a
> series of still images and from uncompressed audio;
> * it should have an interface as simple and pythonic as possible;
> * it must be cross-platform, easy to extend and not be limited to
> a single file container format or to some specific audio or video
> compression format;
> * at least a subset of the supported formats should be available
> without any external dependency.

Except I guess for the last point, don't the Python bindings for GStreamer
already provide this?

-Andrew.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)

[Lino Mastrodomenico]

2007/3/26, Pete Shinners <[EMAIL PROTECTED]>:
> My main question is what is the image and sound container passed back to 
> Python?
> This single thing along would be worth a SoC if it could be implemented across
> all libraries.
>
> Will your image objects be transferrable to PIL, Pygame, PyOpengl, Numpy,
> Pythonmagick, Pycairo, wxPython, etc? It would be best if this could avoid the
> "fromstring/tostring" mojo that always requires and extra copy of the data for
> each transfer.

I agree. I withdrew my original "multimedia library" idea and
submitted a proposal for the creation of two standard Image and Sound
classes.

If the PSF accepts my proposal and if Google grants a student slot for
me, I will try to implement an interface for the two classes that can
work well for everyone and be easily usable from both the Python and
the C side. I guess this will require a PEP, if this is the case I
will volunteer to write it.

And, most important, I will create patches for the appropriate
standard library modules and for a lot of external libraries, to try
to make the use of the two new classes really interoperable.

Finally, as Greg suggested, adding the new buffer interface to the two
classes can be a really good idea. I'm following with interest that
discussion.

Keeping fingers crossed...

-- 
Lino Mastrodomenico
E-mail: [EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extended Buffer Interface/Protocol

[Carl Banks]

Travis Oliphant wrote:
> Carl Banks wrote:
> 
>> Tr
>> ITSM that we are using the word "view" very differently.  Consider 
>> this example:
>>
>> A = zeros((100,100))
>> B = A.transpose()
> 
> 
> You are thinking of NumPy's particular use case.  I'm thinking of a 
> generic use case.  So, yes I'm using the word view in two different 
> contexts.
> 
> In this scenario, NumPy does not even use the buffer interface.  It 
> knows how to transpose it's own objects and does so by creating a new 
> NumPy object (with it's own shape and strides space) with a data buffer 
> pointed to by "A".

I realized that as soon as I tried a simple Python demonstration of it. 
  So it's a poor example.  But I hope it's obvious how it would 
generalize to a different type.


>>> Having such a thing as a view object would actually be nice because 
>>> it could hold on to a particular view of data with a given set of 
>>> shape and strides (whose memory it owns itself) and then the 
>>> exporting object would be free to change it's shape/strides 
>>> information as long as the data did not change.
>>
>>
>> What I don't undestand is why it's important for the provider to 
>> retain this data.  The requestor only needs the information when it's 
>> created; it will calculate its own versions of the data, and will not 
>> need the originals again, so no need to the provider to keep them around.
> 
> That is certainly a policy we could enforce (and pretty much what I've 
> been thinking).  We just need to make it explicit that the shape and 
> strides provided is only guaranteed up until a GIL release (i.e. 
> arbitrary Python code could change these memory areas both their content 
> and location) and so if you need them later, make your own copies.
> 
> If this were the policy, then NumPy could simply pass pointers to its 
> stored shape and strides arrays when the buffer interface is called but 
> then not worry about re-allocating these arrays before the "buffer" lock 
> is released.   Another object could hold on to the memory area of the 
> NumPy array but would have to store shape and strides information if it 
> wanted to keep it.
> NumPy could also just pass a pointer to the char * representation of the 
> format (which in NumPy would be stored in the dtype object) and would 
> not have to worry about the dtype being re-assigned later.

Bingo!  This is my preference.


 The reason I ask is: if things like "buf" and "strides" and "shape" 
 could change when a buffer is re-exported, then it can complicate 
 things for PIL-like buffers.  (How would you account for offsets in 
 a dimension that's in a subarray?)
>>>
>>>
>>> I'm not sure what you mean, offsets are handled by changing the 
>>> starting location of the pointer to the buffer.
>>
>>
>>
>> But to anwser your question: you can't just change the starting 
>> location because there's hundreds of buffers.  You'd either have to 
>> change the starting location of each one of them, which is highly 
>> undesirable, or to factor in an offset somehow.  (This was, in fact, 
>> the point of the "derefoff" term in my original suggestion.)
> 
> 
> I get better what your derefoff was doing now.  I was missing the 
> de-referencing that was going on.   Couldn't you still just store a 
> pointer to the start of the array.  In other words, isn't your **isptr  
> suggestion sufficient?   It seems to be.

No.  The problem arises when slicing.  In a single buffer, you would 
adjust the base pointer to point at the element [0,0] of the slice.  But 
you can't do that with multiple buffers.  Instead, you have to add an 
offset after deferencing the pointer to the subarray.

Hence my derefoff proposal.  It dereferenced the pointer, then added an 
offset to get you to the 0 position in that dimension.


>> Anyways, despite the miscommunications so far, I now have a very good 
>> idea of what's going on.  We definitely need to get terms straight.  I 
>> agree that getbuffer should return an object.  I think we need to 
>> think harder about the case when requestors re-export the buffer.  
>> (Maybe it's time to whip up some experimental objects?)
> 
> I'm still not clear what you are concerned about.   If an object 
> consumes the buffer interface and then wants to be able to later export 
> it to another, then from our discussion about the shape/strides and 
> format information, it would have to maintain it's own copies of these 
> things, because it could not rely on the original provider (or exporter) 
> to keep them around once the GIL is released.

Right.  So, if someone calls getbuffer, it would send its own copies of 
the buffer information, and not the original exporter's.  The values 
returned by getbuffer can vary for a given buffer, depending on the 
exporter.  Which means the data returned by getbuffer could reflect 
slicing.  Which means the isptr array is not sufficient for the 
PIL-style multiple buffers.


> This is the reason we would have to be very clear abo

Re: [Python-Dev] SoC proposal: multimedia library

[Navtej Singh]

I would
personally suggest limiting support and going for ffmpeg (via ctypes
would be quickest, though a SWIG wrapping could make it more convenient).



Python wrapper for FFmpeg using Pyrex is available on google code (
http://code.google.com/p/pyffmpeg/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] An updated extended buffer PEP

[Greg Ewing]

Travis Oliphant wrote:

> > Here is my updated PEP which incorporates several parts of the 
> > discussions we have been having.

It looks pretty good.

However, I'm still having trouble seeing what use it is returning
a different object from getbuffer. There seems to be no rationale
set out for this in the PEP. Can you give me a concrete example of
a case where it would be necessary?

Also it appears that you're returning a borrowed reference, so
if the provider object is not the same as the main object, this
would seem to require the main object to keep references to all
the provider objects that it has handed out, until releasebuffer
has been called on them. This seems very odd to me.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] An updated extended buffer PEP

[Greg Ewing]

Here's another idea, to allow multiple views of the same
buffer with different shape/stride info to coexist, but
without extra provider objects or refcount weirdness.
Also it avoids using calls with a brazillion arguments.

   struct bufferinfo {
 void **buf;
 Py_ssize_t *len;
 int *writeable;
 char **format;
 int *ndims;
 Py_ssize_t **shape;
 Py_ssize_t **strides;
 int **isptr;
   };

   int (*getbuffer)(PyObject *obj, struct bufferinfo *info);

   int (*releasebuffer)(PyObject *obj, struct bufferinfo *info);

If the object has constant shape/stride info, it just fills
in the info struct with pointers to its own memory, and does
nothing when releasebuffer is called (other than unlocking
its buffer).

If its shape/stride info can change, it mallocs memory for
them and copies them into the info struct. When releasebuffer
is called, it frees this memory.

It is the responsibility of the consumer to ensure that the
base object remains alive until releasebuffer has been called
on the info struct (to avoid leaking any memory that has
been malloced for shapes/strides).

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com