Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-03-06 Thread Josiah Carlson
"Guido van Rossum" <[EMAIL PROTECTED]> wrote: > I haven't been following that as closely as perhaps I should have. I'd > be glad to drop this and go back to a string > representation/implementation that's essentially the 2.x unicode type, > with a compile-time configuration choice between 16 or 32

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-03-06 Thread Guido van Rossum
On 2/28/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > Travis Oliphant <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > > >Travis Oliphant <[EMAIL PROTECTED]> wrote: > > >>I think you are right. In the discussions for unifying string/unicode I > > >>really like the proposals that are lean

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-03-01 Thread Jim Jewett
On 2/27/07, Travis E. Oliphant <[EMAIL PROTECTED]> wrote: > Rationale (This was a good section; thank you.) > 5. There is no shape information provided for the memory. But, >several array-like Python types could make use of a standard >way to describe the shape-interpretati

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Travis Oliphant
> Travis E. Oliphant wrote: > > >>Yes, this was the reason for my dtype object. But, I think that folks >>felt it was too much, especially since the struct-style syntax is >>already there in Python. > > > Making it a full-blown Python object would be too much > for this application. But it c

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Greg Ewing
Daniel Stutzbach wrote: > On 2/27/07, Travis E. Oliphant <[EMAIL PROTECTED]> wrote: >>But I've heard that the reference counts on >>Python objects can be larger than 1 in some cases (even though there >>isn't really anything "viewing" the memory). > > Is that true? The interpreter machinery some

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Greg Ewing
Travis E. Oliphant wrote: > Yes, this was the reason for my dtype object. But, I think that folks > felt it was too much, especially since the struct-style syntax is > already there in Python. Making it a full-blown Python object would be too much for this application. But it could be somethin

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Josiah Carlson
Travis Oliphant <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > >Travis Oliphant <[EMAIL PROTECTED]> wrote: > >>I think you are right. In the discussions for unifying string/unicode I > >>really like the proposals that are leaning toward having a unicode > >>object be an immutable string o

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Travis Oliphant
Josiah Carlson wrote: >Travis Oliphant <[EMAIL PROTECTED]> wrote: > > >>I think you are right. In the discussions for unifying string/unicode I >>really like the proposals that are leaning toward having a unicode >>object be an immutable string of either ucs-1, ucs-2, or ucs-4 depending >>on

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Josiah Carlson
Travis Oliphant <[EMAIL PROTECTED]> wrote: > I think you are right. In the discussions for unifying string/unicode I > really like the proposals that are leaning toward having a unicode > object be an immutable string of either ucs-1, ucs-2, or ucs-4 depending > on what is in the string. Exce

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Travis Oliphant
Thomas Heller wrote: > >>Additions to the struct string-syntax >> >> The struct string-syntax is missing some characters to fully >> implement data-format descriptions already available elsewhere (in >> ctypes and NumPy for example). Here are the proposed additions: >> >> Character

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Daniel Stutzbach
On 2/27/07, Travis E. Oliphant <[EMAIL PROTECTED]> wrote: > Maybe I'm not understanding you correctly. Perhaps what you are saying > is that we should have all memory allocation go through a light-weight > memory-object. Then, you would get this object + an offset when you > wanted a pointer into

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-28 Thread Thomas Heller
Travis E. Oliphant schrieb: > Attached is my current draft of the enhanced buffer protocol for Python > 3000. It is basically what has been discussed except for some issues > with non single-segment memory areas (such as a sub-array). > > Comments are welcome. > > -Travis Oliphant > Addition

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Travis E. Oliphant wrote: > > Attached is my current draft of the enhanced buffer protocol for Python > 3000. It is basically what has been discussed except for some issues > with non single-segment memory areas (such as a sub-array). > > Comments are welcome. > The latest version of the PEP

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Greg Ewing wrote: > > The buffer API is for the use of C code, and it should > be designed with the convenience of C code in mind. > Using Python data structures unnecessarily seems like > the wrong way to go about that. > > The following alternative would seem to provide most of > the things tha

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Travis E. Oliphant wrote: > > The memory management is the only issue. In fact, the PEP still has the issue of who manages the memory for the format-description string when it is returned. The easiest thing to do is to return a Python String and let reference counting handle the memory managem

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Greg Ewing wrote: > Travis E. Oliphant wrote: > >> typedef char *(*formatbufferproc)(PyObject *view, int *itemsize) >> >> Get the format-string of the memory using the struct-module >> string syntax > > I'm not sure whether a struct-format string would be > the most convenient for

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Greg Ewing
Josiah Carlson wrote: > About the only issue I can see with implementing the mechanism as you > describe is that everything that wants to offer the buffer interface > would need to store its data in a PyArray structure. And that wouldn't be acceptable, because the point of the buffer interface is

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Greg Ewing
Travis E. Oliphant wrote: > typedef char *(*formatbufferproc)(PyObject *view, int *itemsize) > > Get the format-string of the memory using the struct-module > string syntax I'm not sure whether a struct-format string would be the most convenient form for use by C-level code, as i

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Nick Coghlan
Greg Ewing wrote: > Nick Coghlan wrote: >> I'd also like to know what should happen if >> the object's Python refcount drops to zero, but the view count is still >> greater than 0. > > That shouldn't happen, because the code using the view > ought to be responsible for holding a reference to the

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Greg Ewing
Nick Coghlan wrote: > I'd also like to know what should happen if > the object's Python refcount drops to zero, but the view count is still > greater than 0. That shouldn't happen, because the code using the view ought to be responsible for holding a reference to the containing object as long as

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Daniel Stutzbach wrote: > I know I'm joining this discussion late in the game, so I apologize if > my look through the list archives was not sufficiently exhaustive and > this has been proposed and shot down before... No, I don't think you are late. But this discussion has been going on off and

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Daniel Stutzbach
On 2/27/07, Josiah Carlson <[EMAIL PROTECTED]> wrote: > About the only issue I can see with implementing the mechanism as you > describe is that everything that wants to offer the buffer interface > would need to store its data in a PyArray structure. Bytes, unicode, > array.array, mmap, etc. Mos

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Josiah Carlson
"Daniel Stutzbach" <[EMAIL PROTECTED]> wrote: [snip] > The container cannot reallocate the memory unless the reference > counter on the PyArray is exactly 1. Alternatively, some objects could opt to create a new PyArray of sufficient size, copy data as necessary, and leave all previous views to p

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Nick Coghlan
Daniel Stutzbach wrote: > When an array-like container allocates memory, it allocates a PyArray > to store the actual data in. When a caller request a view, the > container increments the PyArray's reference counter and returns a > pointer to the PyArray. The caller is responsible for decrementin

Re: [Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Daniel Stutzbach
I know I'm joining this discussion late in the game, so I apologize if my look through the list archives was not sufficiently exhaustive and this has been proposed and shot down before... What if the locking mechanism were put into the array's memory instead of the container's memory? If the arra

[Python-3000] PEP Draft: Enhancing the buffer protcol

2007-02-27 Thread Travis E. Oliphant
Attached is my current draft of the enhanced buffer protocol for Python 3000. It is basically what has been discussed except for some issues with non single-segment memory areas (such as a sub-array). Comments are welcome. -Travis Oliphant PEP: Title: Revising the buffer protocol Version: