Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Stephen J. Turnbull
Glenn Linderman writes: significantly seems to be the only word at question; it seems that there are a fair number of validation checks that could be performed; the numeric part of UTF-8 decoding is just a sequence of shifts, masks, and ORs, so can be coded pretty tightly in C or

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Ulrich Eckhardt
On Friday 05 December 2008, James Y Knight wrote: On Dec 5, 2008, at 5:27 AM, Ulrich Eckhardt wrote: Using the byte variant is equally fubar, because e.g. on MS Windows it is not supported, except through a very lossy roundtrip through the locale's codepage, limiting your functionality.

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Glenn Linderman
On approximately 12/8/2008 12:57 AM, came the following characters from the keyboard of Stephen J. Turnbull: Internal decoding is (or should be) an oxymoron. Why would your software be passing around text in any format other than internal? So decoding will happen (a) on I/O, which is itself

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Christian Heimes
Martin v. Löwis wrote: I wasn't (primarily) talking about fixing this particular issue. Time needs to be made available also for the upcoming 2.4.6 and 2.5.3 releases (which should, IMO, get priority over a 3.0 bugfix release at this point) I've no opinion on the priority of the releases.

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Dec 7, 2008, at 7:56 PM, Christian Heimes wrote: Barry Warsaw wrote: I'm personally okay with performance fixes in point releases, as long it doesn't change API or add additional features. Does your okay include or exclude new internal APIs

[Python-Dev] Deciding on dbm API in setup.py

2008-12-08 Thread skip
Several packages provide a dbm-compatible API. Currently, the code in Python's setup.py hardcodes the order of consideration: ndbm, then gdbm, then Berkeley DB. While the APIs are compatible, the file formats are all different as far as I know. If you have ndbm but want to use Berkeley DB

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Nick Coghlan
Terry Reedy wrote: This to be is an argument for keeping the default the current behavior, but not for rejecting flexibility. The computing world seems to be messier than we would like and worse that I realized until this week. As you say below, people need to better anticipate the future,

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Ulrich Eckhardt
On Sunday 07 December 2008, Guido van Rossum wrote: My problem with raising exceptions *by default* when an undecodable name exists is that it may render an app completely useless in a situation where the developer is no longer around. This happened all the time with the 2.x Unicode API, where

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread M.-A. Lemburg
On 2008-12-06 01:48, Nick Coghlan wrote: You can't display a non-decodable filename to the user, hence the user will have no idea what they're working on. Non-filesystem related apps have no business trying to deal with insane filenames. This is not entirely true: OSes, shells, and

[Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Antoine Pitrou
Hello, The Py_buffer struct has two pointers named `shape` and `strides`. Each points to an array of Py_ssize_t values whose length is equal to the number of dimensions of the buffer object. Unfortunately, the buffer protocol spec doesn't explain how allocation of these arrays should be handled.

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread rdmurray
On Sun, 7 Dec 2008 at 13:33, Guido van Rossum wrote: My problem with raising exceptions *by default* when an undecodable name exists is that it may render an app completely useless in a situation where the developer is no longer around. This happened all I think Nick Coghlan's suggestion of

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Bill Janssen
Nick Coghlan [EMAIL PROTECTED] wrote: - I think the binary and Unicode APIs should be available (and fully functional) on all platforms (including Windows) so that app developers don't create portability problems for themselves when they make the decision as to which API to use +1 I'm

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Terry Reedy
Nick Coghlan wrote: Terry Reedy wrote: This to be is an argument for keeping the default the current behavior, but not for rejecting flexibility. The computing world seems to be messier than we would like and worse that I realized until this week. As you say below, people need to better

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Guido van Rossum
On Sun, Dec 7, 2008 at 3:53 PM, Terry Reedy [EMAIL PROTECTED] wrote: Guido van Rossum wrote: On Sun, Dec 7, 2008 at 1:20 PM, Terry Reedy [EMAIL PROTECTED] wrote: Toshio Kuratomi wrote: - If this is true, a definition of os.listdir(type 'str') that would better meet programmer expectation

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Brett Cannon
On Mon, Dec 8, 2008 at 05:11, Barry Warsaw [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Dec 7, 2008, at 7:56 PM, Christian Heimes wrote: Barry Warsaw wrote: I'm personally okay with performance fixes in point releases, as long it doesn't change API or add

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Guido van Rossum
On Mon, Dec 8, 2008 at 10:34 AM, [EMAIL PROTECTED] wrote: On Mon, 8 Dec 2008 at 13:16, Terry Reedy wrote: And the decoding problems don't pass silently either - they just get emitted as a warning by default instead of causing the application to crash. Do they get automatically logged?

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Scott Dial
Guido van Rossum wrote: On Mon, Dec 8, 2008 at 10:34 AM, [EMAIL PROTECTED] wrote: On Mon, 8 Dec 2008 at 13:16, Terry Reedy wrote: And the decoding problems don't pass silently either - they just get emitted as a warning by default instead of causing the application to crash. Do they get

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Bugbee, Larry
I'm perhaps biased here; most of my Python programs don't have user interfaces, because they don't talk to people, they talk to other programs. The binary APIs for the OS are essential. I use and deeply appreciate all the string handling features in Python, particularly its firm grip on

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread rdmurray
On Mon, 8 Dec 2008 at 11:25, Guido van Rossum wrote: On Mon, Dec 8, 2008 at 10:34 AM, [EMAIL PROTECTED] wrote: I'm in favor of an option to control what happens. I just really really don't want the _default_ to be ignore. Defaulting to a warning is fine with me, as would be defaulting to a

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Guido van Rossum
On Mon, Dec 8, 2008 at 12:07 PM, [EMAIL PROTECTED] wrote: On Mon, 8 Dec 2008 at 11:25, Guido van Rossum wrote: On Mon, Dec 8, 2008 at 10:34 AM, [EMAIL PROTECTED] wrote: I'm in favor of an option to control what happens. I just really really don't want the _default_ to be ignore.

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Nick Coghlan
Brett Cannon wrote: On Mon, Dec 8, 2008 at 05:11, Barry Warsaw [EMAIL PROTECTED] wrote: On Dec 7, 2008, at 7:56 PM, Christian Heimes wrote: Barry Warsaw wrote: I'm personally okay with performance fixes in point releases, as long it doesn't change API or add additional features. Does your

[Python-Dev] [PATCH] Make 2to3 --write preserve file mode (eg. execution bit)

2008-12-08 Thread Adeodato Simó
Hello, after using 2to3 --write over some scripts, I found it very cumbersome having to run `chmod +x` on each of them afterwards. The attached patch is a possible way to fix this issue. It'd be great if somebody could apply it, or write a more appropriate fix. Many thanks in advance! P.S.:

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread M.-A. Lemburg
On 2008-12-08 19:26, Guido van Rossum wrote: On Sun, Dec 7, 2008 at 3:53 PM, Terry Reedy [EMAIL PROTECTED] wrote: Here is a possible use case: I want filenames as 3.0 strings and I anticipate no problems at present but, as you say above, something might happen years in the future. I am using

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Antoine Pitrou
Nick Coghlan ncoghlan at gmail.com writes: Where would adding a (undocumented) get_filename() method to ZipImporter objects for the benefit of the -m switch fit then? Why not call it _get_filename() in 3.0 and get_filename() in 3.1? ___

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Antoine Pitrou
M.-A. Lemburg mal at egenix.com writes: Such application specific error handlers could then also apply whatever fancy round-trip safe encoding of non-decodable bytes to Unicode escapes, private code points, etc. as seen fit by the application. I'd argue that such fancy round-trip safe error

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Nick Coghlan
Antoine Pitrou wrote: For the one-dimensional case, I had in mind a simple scheme where the Py_buffer struct has an additional two-member Py_ssize_t array. Then `shape` and `strides` can point to the first and second member of this array, respectively. This wouldn't solve the

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Nick Coghlan
Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: Where would adding a (undocumented) get_filename() method to ZipImporter objects for the benefit of the -m switch fit then? Why not call it _get_filename() in 3.0 and get_filename() in 3.1? Actually, since it should only be a

Re: [Python-Dev] [PATCH] Make 2to3 --write preserve file mode (eg. execution bit)

2008-12-08 Thread Mark Dickinson
On Mon, Dec 8, 2008 at 6:51 PM, Adeodato Simó [EMAIL PROTECTED] wrote: The attached patch is a possible way to fix this issue. It'd be great if somebody could apply it, or write a more appropriate fix. Please could you submit your patch to the bug tracker, at http://bugs.python.org That way

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Dec 8, 2008, at 3:39 PM, Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: Where would adding a (undocumented) get_filename() method to ZipImporter objects for the benefit of the -m switch fit then? Why not call it

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread M.-A. Lemburg
On 2008-12-08 21:45, Antoine Pitrou wrote: M.-A. Lemburg mal at egenix.com writes: Such application specific error handlers could then also apply whatever fancy round-trip safe encoding of non-decodable bytes to Unicode escapes, private code points, etc. as seen fit by the application. I'd

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Nick Coghlan
Terry Reedy wrote: Nick Coghlan wrote: Terry Reedy wrote: This to be is an argument for keeping the default the current behavior, but not for rejecting flexibility. The computing world seems to be messier than we would like and worse that I realized until this week. As you say below, people

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Alexander Belopolsky
I don't have much to add to Nick's reply other than to point you to numpy, http://projects.scipy.org/scipy/numpy, as a reference implementation. You may also get better responses on the numpy list, [EMAIL PROTECTED]. On Mon, Dec 8, 2008 at 3:46 PM, Nick Coghlan [EMAIL PROTECTED] wrote: Antoine

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Adam Olsen
On Mon, Dec 8, 2008 at 1:45 PM, Antoine Pitrou [EMAIL PROTECTED] wrote: M.-A. Lemburg mal at egenix.com writes: Such application specific error handlers could then also apply whatever fancy round-trip safe encoding of non-decodable bytes to Unicode escapes, private code points, etc. as seen

Re: [Python-Dev] 3.0.1 possibilities

2008-12-08 Thread Nick Coghlan
Barry Warsaw wrote: On Dec 8, 2008, at 3:39 PM, Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: Where would adding a (undocumented) get_filename() method to ZipImporter objects for the benefit of the -m switch fit then? Why not call it _get_filename() in 3.0 and

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Antoine Pitrou
Nick Coghlan ncoghlan at gmail.com writes: Actually, I think your suggested scheme for the one-dimensional case shows the way forward: ownership of the shape and strides memory belongs to the object issuing the Py_buffer struct, and that object needs to deal with it when the buffer is

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Antoine Pitrou
Adam Olsen rhamph at gmail.com writes: Except they're clearly NOT part of the unicode spec. This is always the same discussion going in circles. I know they're not part of the unicode spec, but practicality beats purity and if the said error handler comes with an appropriate warning in the

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Adam Olsen
On Mon, Dec 8, 2008 at 2:01 PM, M.-A. Lemburg [EMAIL PROTECTED] wrote: On 2008-12-08 21:45, Antoine Pitrou wrote: M.-A. Lemburg mal at egenix.com writes: Such application specific error handlers could then also apply whatever fancy round-trip safe encoding of non-decodable bytes to Unicode

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Toshio Kuratomi
Guido van Rossum wrote: On Mon, Dec 8, 2008 at 12:07 PM, [EMAIL PROTECTED] wrote: On Mon, 8 Dec 2008 at 11:25, Guido van Rossum wrote: But I'm happy with just issuing a warning by default. That would mean it doesn't fail silently, but neither does it crash. Seems like the best compromise

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Nick Coghlan
Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: Actually, I think your suggested scheme for the one-dimensional case shows the way forward: ownership of the shape and strides memory belongs to the object issuing the Py_buffer struct, and that object needs to deal with it when

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread M.-A. Lemburg
On 2008-12-08 22:32, Adam Olsen wrote: On Mon, Dec 8, 2008 at 2:01 PM, M.-A. Lemburg [EMAIL PROTECTED] wrote: On 2008-12-08 21:45, Antoine Pitrou wrote: M.-A. Lemburg mal at egenix.com writes: Such application specific error handlers could then also apply whatever fancy round-trip safe

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Adam Olsen
On Mon, Dec 8, 2008 at 1:12 PM, Guido van Rossum [EMAIL PROTECTED] wrote: On Mon, Dec 8, 2008 at 12:07 PM, [EMAIL PROTECTED] wrote: But I'm happy with just issuing a warning by default. That would mean it doesn't fail silently, but neither does it crash. Seems like the best compromise with

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread M.-A. Lemburg
On 2008-12-08 22:39, Victor Stinner wrote: ('strict', 'ignore', 'replace', 'xmlcharrefreplace') replace (or xmlcharrefreplace) is just useless because you will not be unable to open or rename the file... You just know that there is a strange file in the directory. Right, but that's

Re: [Python-Dev] [PATCH] Make 2to3 --write preserve file mode (eg. execution bit)

2008-12-08 Thread Adeodato Simó
* Mark Dickinson [Mon, 08 Dec 2008 20:56:25 +]: On Mon, Dec 8, 2008 at 6:51 PM, Adeodato Simó [EMAIL PROTECTED] wrote: The attached patch is a possible way to fix this issue. It'd be great if somebody could apply it, or write a more appropriate fix. Please could you submit your patch

Re: [Python-Dev] as keyword woes

2008-12-08 Thread Guido van Rossum
On Sun, Dec 7, 2008 at 1:06 PM, Paul Boddie [EMAIL PROTECTED] wrote: On Sat Dec 6 21:29:09 CET 2008, Guido van Rossum wrote: On Sat, Dec 6, 2008 at 11:38 AM, Warren DeLano warren at delsci.com wrote: As someone somewhat knowledgable of how parsers work, I do not understand why a

[Python-Dev] Self in method body

2008-12-08 Thread Filip Gruszczyński
There is a large discussion on python-list about Guido's article about new self syntax, therefore I would like to use that to raise similar question: self in the body. Some time ago I was coding in Magik language (http://en.wikipedia.org/wiki/Magik_(programming_language), which is dynamically

Re: [Python-Dev] Nonlocal shortcut

2008-12-08 Thread Guido van Rossum
On Sun, Dec 7, 2008 at 2:45 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED] wrote: Hello, Fabio Zadrozny wrote: Hi, I'm currently implementing a parser to handle Python 3.0, and one of the points I found conflicting with the grammar specification is the PEP 3104. It says that a shortcut

Re: [Python-Dev] as keyword woes

2008-12-08 Thread Paul Boddie
On Monday 08 December 2008 22:54:41 Guido van Rossum wrote: From my experience with SQL, it's nearly as bad as Python in that every single one of the 200+ reserved words in a typical implementation cannot be used as a name in any context without using double quotes. SQL is a big language; I

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Adam Olsen
On Mon, Dec 8, 2008 at 2:44 PM, M.-A. Lemburg [EMAIL PROTECTED] wrote: On 2008-12-08 22:32, Adam Olsen wrote: On Mon, Dec 8, 2008 at 2:01 PM, M.-A. Lemburg [EMAIL PROTECTED] wrote: On 2008-12-08 21:45, Antoine Pitrou wrote: M.-A. Lemburg mal at egenix.com writes: Such application specific

Re: [Python-Dev] Nonlocal shortcut

2008-12-08 Thread Calvin Spealman
Did the original PEP discussion cover debates about the shortcut working for all assignment operators (like += and x[i] =) and the difference between it being one-shot (doesnt affect x for the rest of the function) or simply the unrolling into nonlocal x; x= y as it is? On Mon, Dec 8, 2008 at

Re: [Python-Dev] Self in method body

2008-12-08 Thread Steven D'Aprano
On Tue, 9 Dec 2008 08:55:21 am Filip Gruszczyński wrote: There is a large discussion on python-list about Guido's article about new self syntax, therefore I would like to use that to raise similar question: self in the body. Some time ago I was coding in Magik language

Re: [Python-Dev] Self in method body

2008-12-08 Thread Filip Gruszczyński
I've just grabbed a random, heavily OO module from my own code library. It has 60 instances of self, or 240 characters, out of 18,839 characters in total (including newlines). Removing self will decrease the number of my keystrokes and the amount of pure typing time (excluding thinking time,

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Antoine Pitrou
Nick Coghlan ncoghlan at gmail.com writes: No, you misunderstand what I meant. Py_buffer doesn't need to be changed at all. The *issuing type* would define a new structure with the additional fields, such as: With to the current buffer API, this is not possible. It's the caller who allocates

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Terry Reedy
M.-A. Lemburg wrote: On Sun, Dec 7, 2008 at 3:53 PM, Terry Reedy [EMAIL PROTECTED] wrote: try: files = os.listdir(somedir, errors = strict) except OSError as e: log(verbose error message that includes somedir and e) files = os.listdir(somedir) If that error parameter is the same as in

Re: [Python-Dev] Self in method body

2008-12-08 Thread Terry Reedy
Filip Gruszczyński wrote: There is a large discussion on python-list about Guido's article about That discussion should stay there. new self syntax, therefore I would like to use that to raise similar question: self in the body. That has also be heavily discussed, many times, there and

Re: [Python-Dev] Holding a Python Language Summit at PyCon

2008-12-08 Thread A.M. Kuchling
On Sat, Dec 06, 2008 at 02:42:38PM -0800, Brett Cannon wrote: No, I am saying I had told AMK I was interested in championing the session. He chose you, and that's that. One less thing for me to worry about. =) Brett, I actually think you'd be a good champion for the 11AM transition-planning

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Alexander Belopolsky
On Mon, Dec 8, 2008 at 6:25 PM, Antoine Pitrou [EMAIL PROTECTED] wrote: .. Alexander's suggestion of going and looking at what the numpy folks have done in this area is probably a good idea too. Well, I'm open to others doing this, but I won't do it myself. My interest is in fixing the most

Re: [Python-Dev] Holding a Python Language Summit at PyCon

2008-12-08 Thread Brett Cannon
On Mon, Dec 8, 2008 at 18:53, A.M. Kuchling [EMAIL PROTECTED] wrote: On Sat, Dec 06, 2008 at 02:42:38PM -0800, Brett Cannon wrote: No, I am saying I had told AMK I was interested in championing the session. He chose you, and that's that. One less thing for me to worry about. =) Brett, I

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-08 Thread Greg Ewing
Antoine Pitrou wrote: (of course complex schemes can be devised where the callee maintains its own separate storage for shape and strides, but I don't think we want to go there) But that's exactly where you're supposed to be going. If the object providing the buffer has variable-sized shape

Re: [Python-Dev] Python-3.0, unicode, and os.environ

2008-12-08 Thread Glenn Linderman
On approximately 12/8/2008 9:30 AM, came the following characters from the keyboard of [EMAIL PROTECTED]: If warnings were emitted, then files would not be silently ignored, yet the program could still be used. Yep, this is sounding useful. PS: I'd like to see a similar warning issued