Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Nick Coghlan
David Hopwood wrote: > Martin v. Löwis wrote: >> Programs that rely on sys.getfilesystemencoding() being able to >> represent arbitrary file names on Windows might have a bug; >> programs that rely on sys.getfilesystemencoding() being able >> to encode all elements of sys.path do not (at least not

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: > Elements of sys.path can be Unicode strings in Python 2.5, and should be > pathnames supported by the underlying OS. Where is it documented that there > is any further restriction on them? And why should there be any further > restriction on them? It's not documented in th

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Martin v. Löwis wrote: > David Hopwood schrieb: > >>On Windows, file system pathnames can contain arbitrary Unicode characters >>(well, almost). Despite the existence of "ANSI" filesystem APIs, and >>regardless of what 'sys.getfilesystemencoding()' returns, the underlying >>file system encoding fo

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: > On Windows, file system pathnames can contain arbitrary Unicode characters > (well, almost). Despite the existence of "ANSI" filesystem APIs, and > regardless of what 'sys.getfilesystemencoding()' returns, the underlying > file system encoding for NTFS and FAT filesystems i

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Nick Coghlan wrote: > David Hopwood wrote: >> Martin v. Löwis wrote: >>> Nick Coghlan schrieb: >>> So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. >>> >>> I doubt it does without side effects. For example, an application th

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
Nick Coghlan schrieb: > I think MvL is looking at it from the point of view of consumers of the list > of strings in sys.path, such as PEP 302 importer and loader objects, and > tools > like module_finder. Currently, the list of values in sys.path is limited to: That, and all kinds of inspectio

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Nick Coghlan
David Hopwood wrote: > Martin v. Löwis wrote: >> Nick Coghlan schrieb: >> >>> So this is taking something that *already works properly on POSIX >>> systems* and making it work on Windows as well. >> I doubt it does without side effects. For example, an application that >> would go through sys.path,

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: >> I doubt it does without side effects. For example, an application that >> would go through sys.path, and encode everything with >> sys.getfilesystemencoding() currently works, but will break if the patch >> is applied and non-mbcs strings are put on sys.path. > > Huh? It

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Martin v. Löwis wrote: > Nick Coghlan schrieb: > >>So this is taking something that *already works properly on POSIX >>systems* and making it work on Windows as well. > > I doubt it does without side effects. For example, an application that > would go through sys.path, and encode everything with

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Steve Holden
Martin v. Löwis wrote: > Nick Coghlan schrieb: > >>So this is taking something that *already works properly on POSIX >>systems* and making it work on Windows as well. > > > I doubt it does without side effects. For example, an application that > would go through sys.path, and encode everything w

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
Nick Coghlan schrieb: > So this is taking something that *already works properly on POSIX > systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through sys.path, and encode everything with sys.getfilesystemencoding() curre

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Nick Coghlan
Martin v. Löwis wrote: > Steve Holden schrieb: >> Or simply that this inability isn't currently >> described in a bug report on Sourceforge? > > No: sys.path is specified (originally) as containing a list of byte > strings; it was extended to also support path importers (or whatever > that PEP ca

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Giovanni Bajo schrieb: > +1, but I would love to see a more formal definition of what a "bugfix" is, > which would reduce the ambiguous cases, and thus reduce the number of times > the > release managers are called to pronounce. > > Other projects, for instance, describe point releases as "open f

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: > I don't regard this as the provision of a new feature but as the removal > of an unnecessary restriction (which I would prefer to call a bug). You got the definition of "bug" wrong. Primarily, a bug is a deviation from the specification. Extending the domain of an argument

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Nick Coghlan schrieb: >> But it *is* a desirable, albeit new, feature, so I'm surprised that you >> don't appear to perceive it as such for a downstream release. > > And unlike 2.2's True/False problem, it is an *environmental* feature, rather > than a programmatic one. Not sure what you mean b

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: >>> I agree it's a relatively large patch for a release candidate but if >>> prudence suggests deferring it, it should be a *definite* for 2.5.1 and >>> subsequent releases. >> >> Possibly. I remain unconvinced. >> > > But it *is* a desirable, albeit new, feature, so I'm sur

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: >> As this can't be considered a bugfix (that I can see), I'd be against it >> being >> checked into 2.5. >> > Are you suggesting that Python's inability to correctly handle Unicode > path elements isn't a bug? Not sure whether Anthony suggests it, but I do. > Or simply

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread M.-A. Lemburg
Kristján V. Jónsson wrote: > Hello All. > I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) > which allows unicode paths in sys.path and uses the unicode file api on > windows. > This is tried and tested on 2.5, and backported to 2.3 and is currently > running on client

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Raymond Hettinger
Giovanni Bajo wrote: > >+1, but I would love to see a more formal definition of what a "bugfix" is, >which would reduce the ambiguous cases, and thus reduce the number of times the >release managers are called to pronounce. > > Sorry, that is just a pipe-dream. To some degree, all bug-fixes are

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Giovanni Bajo
Guido van Rossum <[EMAIL PROTECTED]> wrote: > IMO it's the call of the release managers. Board members ought to > trust the release managers and not apply undue pressure. +1, but I would love to see a more formal definition of what a "bugfix" is, which would reduce the ambiguous cases, and thus

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread skip
Guido> IMO it's the call of the release managers. Board members ought to Guido> trust the release managers and not apply undue pressure. Indeed. Let's not go whacking people with boards. The Perl people would just laugh at us... Skip ___ Pyth

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Guido van Rossum
On 9/8/06, Steve Holden <[EMAIL PROTECTED]> wrote: > Anthony Baxter wrote: > > On Friday 08 September 2006 19:19, Steve Holden wrote: > > > >>But it *is* a desirable, albeit new, feature, so I'm surprised that you > >>don't appear to perceive it as such for a downstream release. > > > > > > Point r

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: > On Friday 08 September 2006 19:19, Steve Holden wrote: > >>But it *is* a desirable, albeit new, feature, so I'm surprised that you >>don't appear to perceive it as such for a downstream release. > > > Point releases (2.x.1 and suchlike) are absolutely not for new features

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Anthony Baxter
On Friday 08 September 2006 19:19, Steve Holden wrote: > But it *is* a desirable, albeit new, feature, so I'm surprised that you > don't appear to perceive it as such for a downstream release. Point releases (2.x.1 and suchlike) are absolutely not for new features. They're for bugfixes, only. It'

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Nick Coghlan
Steve Holden wrote: > Anthony Baxter wrote: >> On Friday 08 September 2006 18:24, Steve Holden wrote: >>> I agree it's a relatively large patch for a release candidate but if >>> prudence suggests deferring it, it should be a *definite* for 2.5.1 and >>> subsequent releases. >> >> Possibly. I remai

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: > On Friday 08 September 2006 18:24, Steve Holden wrote: > >>>As this can't be considered a bugfix (that I can see), I'd be against it >>>being checked into 2.5. >> >>Are you suggesting that Python's inability to correctly handle Unicode >>path elements isn't a bug? Or simply

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Anthony Baxter
On Friday 08 September 2006 18:24, Steve Holden wrote: > > As this can't be considered a bugfix (that I can see), I'd be against it > > being checked into 2.5. > > Are you suggesting that Python's inability to correctly handle Unicode > path elements isn't a bug? Or simply that this inability isn't

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: > On Friday 08 September 2006 02:56, Kristján V. Jónsson wrote: > >>Hello All. >>I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) >>which allows unicode paths in sys.path and uses the unicode file api on >>windows. This is tried and tested on 2.5, a

Re: [Python-Dev] Unicode Imports

2006-09-07 Thread Anthony Baxter
On Friday 08 September 2006 02:56, Kristján V. Jónsson wrote: > Hello All. > I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) > which allows unicode paths in sys.path and uses the unicode file api on > windows. This is tried and tested on 2.5, and backported to 2.3 and is

[Python-Dev] Unicode Imports

2006-09-07 Thread Kristján V . Jónsson
Hello All. I just added patch 1552880 to sourceforge.  It is a patch for 2.6 (and 2.5) which allows unicode paths in sys.path and uses the unicode file api on windows. This is tried and tested on 2.5, and backported to 2.3 and is currently running on clients in china and esewhere.  It is m

Re: [Python-Dev] unicode imports

2006-06-20 Thread Martin v. Löwis
Thomas Heller wrote: >> Is that code available somewhere still? Does it still work? > > Available as patch 1093253, I have not tried if it stil works I see. It's quite a huge change, that's probably why nobody found the time to review it, yet. > To be really useful, wide char versions of other t

Re: [Python-Dev] unicode imports

2006-06-20 Thread Thomas Heller
Martin v. Löwis schrieb: > Thomas Heller wrote: >> It should be noted that I once started to convert the import machinery >> to be fully unicode aware. As far as I can tell, a *lot* has to be changed >> to make this work. > > Is that code available somewhere still? Does it still work? Available

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Thomas Heller wrote: > It should be noted that I once started to convert the import machinery > to be fully unicode aware. As far as I can tell, a *lot* has to be changed > to make this work. Is that code available somewhere still? Does it still work? > I started with refactoring Python/import.c

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Kristján V. Jónsson wrote: > Wouldn´t it be possible then to emulate the unix way? Simply encode > any unicode paths to utf-8, process them as normal, and then decode > them just prior to the actual windows io call? That won't work. People also put path names from the ANSI code page onto sys.path

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Kristján V. Jónsson wrote: > I don't have specific information on the machines. We didn´t try > very hard to get things to work with 2.3 since we simply assumed it > would work automatically when we upgraded to a more mature 2.4. I > could try to get more info, but it would be 2.3 specific. Have

Re: [Python-Dev] unicode imports

2006-06-19 Thread Guido van Rossum
On 6/16/06, Kristján V. Jónsson <[EMAIL PROTECTED]> wrote: > Although python has had full unicode support for filenames for a long time > on selected platforms (e.g. Windows), there is one glaring deficiency: It > cannot import from paths containing unicode. I´ve tried creating folders > with chi

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
something I will have to patch in for our application. Cheers, Kristján -Original Message- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 19. júní 2006 13:46 To: Kristján V. Jónsson Cc: "Martin v. Löwis"; Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jón

Re: [Python-Dev] unicode imports

2006-06-19 Thread Nick Coghlan
Kristján V. Jónsson wrote: > Funny that no other platforms could benefit from a unicode import path. > Does that mean that windows will reign supreme? Please explain. As near as I can tell, other platforms use encoded strings with the normal (byte-based) posix file API, so the Python interpreter

Re: [Python-Dev] unicode imports

2006-06-19 Thread M.-A. Lemburg
Thomas Heller wrote: > It should be noted that I once started to convert the import machinery > to be fully unicode aware. As far as I can tell, a *lot* has to be changed > to make this work. > > I started with refactoring Python/import.c, but nobody responded to the > question > whether such a

Re: [Python-Dev] unicode imports

2006-06-19 Thread Thomas Heller
It should be noted that I once started to convert the import machinery to be fully unicode aware. As far as I can tell, a *lot* has to be changed to make this work. I started with refactoring Python/import.c, but nobody responded to the question whether such a refactoring patch would be accepted

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson wrote: > The standard install path in chinese distributions can be with a > non-ANSI path, and installing an embedded python application there > will break it. I very much doubt this.

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
trary directory. Kristján -Original Message- From: Neil Hodgson [mailto:[EMAIL PROTECTED] Sent: 17. júní 2006 04:53 To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson: > Although python has had full unicode support for filenames for

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
#x27;C:\\PyDev25\\lib', 'C:\\PyDev25\\lib\\plat-win', 'C:\\PyDev25\\lib\\lib-tk', 'C:\\PyDev25\\PCbuild8 ', 'C:\\PyDev25', 'C:\\PyDev25\\lib\\site-packages'] Default encoding: ascii Input encoding: cp850 Output encodings: cp850 cp850 -Ori

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
of source files, but I cannot write "kristján = 1". But that's for a future PEP. Kristján -Original Message- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 16. júní 2006 15:30 To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján

Re: [Python-Dev] unicode imports

2006-06-17 Thread Ronald Oussoren
On 17-jun-2006, at 6:44, Nick Coghlan wrote: > Bob Ippolito wrote: >> There's a similar issue in that if sys.prefix contains a colon, >> Python >> is also busted: >> http://python.org/sf/1507224 >> >> Of course, that's not a Windows issue, but it is everywhere else. The >> offending code in tha

Re: [Python-Dev] unicode imports

2006-06-17 Thread Martin v. Löwis
Neil Hodgson wrote: >It should be unusual for a Chinese installation to use an install > path that can not be represented in MBCS. Try encoding the install > directory into MBCS before adding it to sys.path. Indeed. Unfortunately, people apparently install an English version (because they can

Re: [Python-Dev] unicode imports

2006-06-17 Thread Martin v. Löwis
Kristján V. Jónsson wrote: > The standard install path in chinese distributions can be with a > non-ANSI path, and installing an embedded python application there will > break it. I very much doubt this. On a Chinese system, the Program Files folder likely has a non-*ASCII* name, but it will have

Re: [Python-Dev] unicode imports

2006-06-16 Thread Bob Ippolito
On Jun 16, 2006, at 9:44 PM, Nick Coghlan wrote: > Bob Ippolito wrote: >> There's a similar issue in that if sys.prefix contains a colon, >> Python is also busted: >> http://python.org/sf/1507224 >> Of course, that's not a Windows issue, but it is everywhere else. >> The offending code in tha

Re: [Python-Dev] unicode imports

2006-06-16 Thread Neil Hodgson
Kristján V. Jónsson: > Although python has had full unicode support for filenames for a long time > on selected platforms (e.g. Windows), there is one glaring deficiency: It > cannot import from paths containing unicode. I´ve tried creating folders > with chinese characters and adding them to pa

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Bob Ippolito wrote: > There's a similar issue in that if sys.prefix contains a colon, Python > is also busted: > http://python.org/sf/1507224 > > Of course, that's not a Windows issue, but it is everywhere else. The > offending code in that case is Modules/getpath.c, Since it has to do with the

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Phillip J. Eby wrote: > Actually, you would want to put it in sys.path_hooks, and then instances > would be placed in path_importer_cache automatically. If you are adding > it to the path_hooks after the fact, you should simply clear the > path_importer_cache. Simply poking stuff into the path

Re: [Python-Dev] unicode imports

2006-06-16 Thread Bob Ippolito
On Jun 16, 2006, at 9:02 AM, Phillip J. Eby wrote: > At 01:29 AM 6/17/2006 +1000, Nick Coghlan wrote: >> Kristján V. Jónsson wrote: >>> A cursory glance at import.c shows that the import mechanism is >>> fairly >>> complicated, and riddled with "char *path" thingies, and manual >>> string >>>

Re: [Python-Dev] unicode imports

2006-06-16 Thread Phillip J. Eby
At 01:29 AM 6/17/2006 +1000, Nick Coghlan wrote: >Kristján V. Jónsson wrote: > > A cursory glance at import.c shows that the import mechanism is fairly > > complicated, and riddled with "char *path" thingies, and manual string > > arithmetic. Do you have any suggestions on a clean way to unicodify

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Kristján V. Jónsson wrote: > A cursory glance at import.c shows that the import mechanism is fairly > complicated, and riddled with "char *path" thingies, and manual string > arithmetic. Do you have any suggestions on a clean way to unicodify the > import mechanism? Can you install a PEP 302 p

[Python-Dev] unicode imports

2006-06-16 Thread Kristján V . Jónsson
Greetings!   Although python has had full unicode support for filenames for a long time on selected platforms (e.g. Windows), there is one glaring deficiency:  It cannot import from paths containing unicode.  I´ve tried creating folders with chinese characters and adding them to path, to no