[Python-Dev] PEP 3118: Extended buffer protocol (new version)
Travis Oliphant wrote: Carl Banks wrote: Ok, I've thought quite a bit about this, and I have an idea that I think will be ok with you, and I'll be able to drop my main objection. It's not a big change, either. The key is to explicitly say whether the flag allows or requires. But I made a few other changes as well. I'm good with using an identifier to differentiate between an allowed flag and a require flag. I'm not a big fan of VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it means but not so much that it takes forever to type and uses up horizontal real-estate. That's fine with me. I'm not very particular about spellings, as long as they're not misleading. Now, here is a key point: for these functions to work (indeed, for PyObject_GetBuffer to work at all), you need enough information in bufinfo to figure it out. The bufferinfo struct should be self-contained; you should not need to know what flags were passed to PyObject_GetBuffer in order to know exactly what data you're looking at. Naturally. Therefore, format must always be supplied by getbuffer. You cannot tell if an array is contiguous without the format string. (But see below.) No, I don't think this is quite true. You don't need to know what kind of data you are looking at if you don't get strides. If you use the SIMPLE interface, then both consumer and exporter know the object is looking at bytes which always has an itemsize of 1. But doesn't this violate the above maxim? Suppose these are the contents of bufinfo: ndim = 1 len = 20 shape = (10,) strides = (2,) format = NULL How does it know whether it's looking at contiguous array of 10 two-byte objects, or a discontiguous array of 10 one-byte objects, without having at least an item size? Since item size is now in the mix, it's moot, of course. The idea that Py_BUF_SIMPLE implies bytes is news to me. What if you want a contiguous, one-dimensional array of an arbitrary type? I was thinking this would be acceptable with Py_BUF_SIMPLE. It seems you want to require Py_BUF_FORMAT for that, which would suggest to me that Py_BUF_ALLOW_ND amd Py_BUF_ALLOW_NONCONTIGUOUS, etc., would imply Py_BUF_FORMAT? IOW, pretty much anything that's not SIMPLE implies FORMAT? If that's the case, then most of the issues I brought up about item size don't apply. Also, if that's the case, you're right that Py_BUF_FORMAT makes more sense than Py_BUF_DONT_NEED_FORAMT. But it now it seems even more unnecessary than it did before. Wouldn't any consumer that just wants to look at a chunk of bytes always use Py_BUF_FORMAT, especially if there's danger of a presumptuous exporter raising an exception? I'll update the PEP with my adaptation of your suggestions in a little while. Ok. Thanks for taking the lead, and for putting up with my verbiose nitpicking. :) Carl Banks ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)
Carl Banks wrote: Travis Oliphant wrote: Carl Banks wrote: Ok, I've thought quite a bit about this, and I have an idea that I think will be ok with you, and I'll be able to drop my main objection. It's not a big change, either. The key is to explicitly say whether the flag allows or requires. But I made a few other changes as well. I'm good with using an identifier to differentiate between an allowed flag and a require flag. I'm not a big fan of VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it means but not so much that it takes forever to type and uses up horizontal real-estate. That's fine with me. I'm not very particular about spellings, as long as they're not misleading. Now, here is a key point: for these functions to work (indeed, for PyObject_GetBuffer to work at all), you need enough information in bufinfo to figure it out. The bufferinfo struct should be self-contained; you should not need to know what flags were passed to PyObject_GetBuffer in order to know exactly what data you're looking at. Naturally. Therefore, format must always be supplied by getbuffer. You cannot tell if an array is contiguous without the format string. (But see below.) No, I don't think this is quite true. You don't need to know what kind of data you are looking at if you don't get strides. If you use the SIMPLE interface, then both consumer and exporter know the object is looking at bytes which always has an itemsize of 1. But doesn't this violate the above maxim? Suppose these are the contents of bufinfo: ndim = 1 len = 20 shape = (10,) strides = (2,) format = NULL In my thinking, format/itemsize is necessary if you have strides (as you do here) but not needed if you don't have strides information (i.e. you are assuming a C_CONTIGUOUS memory-chunk). The intent of the simple interface is to basically allow consumers to mimic the old buffer protocol, very easily. How does it know whether it's looking at contiguous array of 10 two-byte objects, or a discontiguous array of 10 one-byte objects, without having at least an item size? Since item size is now in the mix, it's moot, of course. My only real concern is to have some way to tell the exporter that it doesn't need to figure out the format if the consumer doesn't care about it. Given the open-ended nature of the format string, it is possible that a costly format-string construction step could be under-taken even when the consumer doesn't care about it. I can see you are considering the buffer structure as a self-introspecting structure where I was considering it only in terms of how the consumer would be using its members (which implied it knew what it was asking for and wouldn't touch anything else). How about we assume FORMAT will always be filled in but we add a Py_BUF_REQUIRE_PRIMITIVE flag that will only return primitive format strings (i.e. basic c-types)? An exporter receiving this flag will have to return complicated data-types as 'bytes'. I would add this to the Py_BUF_SIMPLE default. The idea that Py_BUF_SIMPLE implies bytes is news to me. What if you want a contiguous, one-dimensional array of an arbitrary type? I was thinking this would be acceptable with Py_BUF_SIMPLE. Unsigned bytes are just the lowest common denominator. They represent the old way of sharing memory. Doesn't an arbitrary type mean bytes? Or did you mean what if you wanted a contiguous, one-dimensional array of a *specific* type? It seems you want to require Py_BUF_FORMAT for that, which would suggest to me that But it now it seems even more unnecessary than it did before. Wouldn't any consumer that just wants to look at a chunk of bytes always use Py_BUF_FORMAT, especially if there's danger of a presumptuous exporter raising an exception? I'll put in the REQUIRE_PRIMITIVE_FORMAT idea in the next update to the PEP. I can just check in my changes to SVN, so it should show up by Friday. Thanks again, -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] RELEASED Python 2.5.1, FINAL
On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.5.1 (FINAL) This is the first bugfix release of Python 2.5. Python 2.5 is now in bugfix-only mode; no new features are being added. According to the release notes, over 150 bugs and patches have been addressed since Python 2.5, including a fair number in the new AST compiler (an internal implementation detail of the Python interpreter). This is a production release of Python, and should be a painless upgrade from 2.5. Since the release candidate, we have backed out a couple of small changes that caused 2.5.1 to behave differently to 2.5. See the release notes for more. For more information on Python 2.5.1, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.5.1/ Highlights of this new release include: Bug fixes. According to the release notes, at least 150 have been fixed. Highlights of the previous major Python release (2.5) are available from the Python 2.5 page, at http://www.python.org/2.5/highlights.html Enjoy this release, Anthony Anthony Baxter [EMAIL PROTECTED] Python Release Manager (on behalf of the entire python-dev team) pgpIqpS4BdDy1.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] python3k change to slicing
There is one thing I'd like to see changed in a future python. I always found it surprising, that x = [1,2,3,4,5] x[1:10] [2, 3, 4, 5] is not an error. This is perhaps the only case (but a fundamental one!) where an error is silently ignored. I really can't think of a good justification for it. If I really meant x[1:] I would have said so. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python3k change to slicing
[+python-3000; followups please remove python-dev] -1 While this may be theoretically preferable, I believe that in practice changing this would be a major pain for very little gain. I don't recall ever finding a bug related to this feature, and I believe it's occasionally useful. Here's something that would be much more cumbersome with your proposed change: suppose I have a string of unknown length and I want to get the first three characters, or less if it's not that long. Today I can write s[:3]. With your proposal I would have to write s[:min(3, len(s))]. --Guido On 4/19/07, Neal Becker [EMAIL PROTECTED] wrote: There is one thing I'd like to see changed in a future python. I always found it surprising, that x = [1,2,3,4,5] x[1:10] [2, 3, 4, 5] is not an error. This is perhaps the only case (but a fundamental one!) where an error is silently ignored. I really can't think of a good justification for it. If I really meant x[1:] I would have said so. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python3k change to slicing
Guido van Rossum [EMAIL PROTECTED] wrote: -1 Me too. While this may be theoretically preferable, I believe that in practice changing this would be a major pain for very little gain. I don't recall ever finding a bug related to this feature, and I believe it's occasionally useful. I find it quite frequently useful. Here's something that would be much more cumbersome with your proposed change: suppose I have a string of unknown length and I want to get the first three characters, or less if it's not that long. Today I can write s[:3]. With your proposal I would have to write s[:min(3, len(s))]. ... and that's exactly the sort of situation I find it useful ;-) I certainly think it's easier to check that your result is the length you wanted on the occasions you need to, than it would be to make the replacement you show above everywhere else. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)
Travis Oliphant [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] | I'm good with using an identifier to differentiate between an allowed | flag and a require flag. I'm not a big fan of | VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it | means but not so much that it takes forever to type and uses up | horizontal real-estate. To save fingers and real-estate, adopt the following convention: by default, adjectives like writable and contiguous are 'required' unless tagged with 'OK', as in WRITABLE_OK. Explain that in the flag doc just before the flags themselves. And yes, ND for N_DIMENSIONAL or MULTIDIMENSIONAL is also a great win that can also be explained in the same intro. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)
Travis Oliphant wrote: you would like to make the original memory read-only until you are done with the algorithm and have copied the data back into memory. Okay, I see what you mean now. Maybe this should be called Py_BUF_LOCK_CONTENTS or something to make the semantics clearer. Otherwise it could mean either that *you* don't intend to write to it, or that you require nobody ever to write to it. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
