Re: [Python-Dev] Add a new locale codec?

2012-02-10 Thread Stephen J. Turnbull
Victor Stinner writes: If this is needed, it should be spelled os.getlocaleencoding() (or sys.getlocaleencoding()?) There is already a locale.getpreferredencoding(False) function which give your the current locale encoding. The problem is that the current locale encoding may change

Re: [Python-Dev] Add a new locale codec?

2012-02-10 Thread Martin v. Löwis
As And pointed out, this is already the behaviour of the mbcs codec under Windows. locale would be the moral (*) equivalent of that under Unix. Indeed, and that precedent should be enough reason *not* to include a locale encoding. The mbcs encoding has caused much user confusion over the

Re: [Python-Dev] Add a new locale codec?

2012-02-10 Thread Victor Stinner
2012/2/10 Martin v. Löwis mar...@v.loewis.de: As And pointed out, this is already the behaviour of the mbcs codec under Windows. locale would be the moral (*) equivalent of that under Unix. Indeed, and that precedent should be enough reason *not* to include a locale encoding. The mbcs

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Victor Stinner
I think there's a general expectation that if you encode something with one codec you will be able to decode it with the same codec. That's not necessarily true for the locale encoding. There is the same problem with the filesystem encoding (sys.getfilesystemencoding()), which is the user

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Antoine Pitrou
On Thu, 9 Feb 2012 08:43:02 +0200 Simon Cross hodgestar+python...@gmail.com wrote: On Thu, Feb 9, 2012 at 2:35 AM, Steven D'Aprano st...@pearwood.info wrote: Simon Cross wrote: I think I'm -1 on a locale encoding because it refers to different actual encodings depending on where and when

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Amaury Forgeot d'Arc
2012/2/9 Antoine Pitrou solip...@pitrou.net I think there's a general expectation that if you encode something with one codec you will be able to decode it with the same codec. That's not necessarily true for the locale encoding. As And pointed out, this is already the behaviour of the

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Victor Stinner
With the difference that mbcs cannot change during execution. It is possible to change the thread ANSI code page (CP_THREAD_ACP) at runtime, but setting the system ANSI code page (CP_ACP) requires to restart Windows. I don't even know if it is possible to change it at all, except by

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Victor Stinner
As And pointed out, this is already the behaviour of the mbcs codec under Windows. locale would be the moral (*) equivalent of that under Unix. On Windows, the ANSI code page codec will be accessible using 3 different names: locale, mbcs and the real encoding name

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Stephen J. Turnbull
Victor Stinner writes: There is the same problem [that encode-decode with the 'locale' codec doesn't roundtrip reliably] with the filesystem encoding (sys.getfilesystemencoding()), -1 on a query to the OS that pretends to be a constant. You see, it's not the same problem. The difference

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Nick Coghlan
On Fri, Feb 10, 2012 at 12:59 AM, Stephen J. Turnbull step...@xemacs.org wrote: If this is needed, it should be spelled os.getlocaleencoding() (or sys.getlocaleencoding()?) Or locale.getpreferredencoding(), even ;) FWIW, I agree with Stephen on this one, but take that with the grain of salt

Re: [Python-Dev] Add a new locale codec?

2012-02-09 Thread Victor Stinner
If this is needed, it should be spelled os.getlocaleencoding() (or sys.getlocaleencoding()?) There is already a locale.getpreferredencoding(False) function which give your the current locale encoding. The problem is that the current locale encoding may change and so you have to get the new

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Victor Stinner
2012/2/8 Simon Cross hodgestar+python...@gmail.com: Is the idea to have:  bfoo.decode(locale) be roughly equivalent to  encoding = locale.getpreferredencoding(False)  bfoo.decode(encoding) ? Yes. Whereas: bfoo.decode(sys.getfilesystemencoding()) is equivalent to encoding =

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Simon Cross
I think I'm -1 on a locale encoding because it refers to different actual encodings depending on where and when it's run, which seems surprising, and there's already a more explicit way to achieve the same effect. The documentation on .getpreferredencoding() says some scary things about needing

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread And Clover
On 2012-02-08 09:28, Simon Cross wrote: I think I'm -1 on a locale encoding because it refers to different actual encodings depending on where and when it's run, which seems surprising, and there's already a more explicit way to achieve the same effect. I'd agree that this is undesirable, and

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Victor Stinner
2012/2/8 Simon Cross hodgestar+python...@gmail.com: I think I'm -1 on a locale encoding because it refers to different actual encodings depending on where and when it's run, which seems surprising, and there's already a more explicit way to achieve the same effect. The following code is just

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Simon Cross
On Wed, Feb 8, 2012 at 3:25 PM, Victor Stinner victor.stin...@haypocalc.com wrote: Sorry, I don't understand what do you mean by weird things. The locale codec doesn't touch the locale. Sorry for being unclear. My question was about the following lines from

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Simon Cross
On Wed, Feb 8, 2012 at 3:25 PM, Victor Stinner victor.stin...@haypocalc.com wrote: The current locale is process-wide: if a thread changes the locale, all threads are affected. Some functions have to use the current locale encoding, and not the locale encoding read at startup. Examples with C

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Victor Stinner
The current locale is process-wide: if a thread changes the locale, all threads are affected. Some functions have to use the current locale encoding, and not the locale encoding read at startup. Examples with C functions: strerror(), strftime(), tzname, etc. Could a core part of Python

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Steven D'Aprano
Simon Cross wrote: I think I'm -1 on a locale encoding because it refers to different actual encodings depending on where and when it's run, which seems surprising Why is it surprising? Surely that's the whole point of a locale encoding: to use the locale encoding, whatever that happens to be

Re: [Python-Dev] Add a new locale codec?

2012-02-08 Thread Simon Cross
On Thu, Feb 9, 2012 at 2:35 AM, Steven D'Aprano st...@pearwood.info wrote: Simon Cross wrote: I think I'm -1 on a locale encoding because it refers to different actual encodings depending on where and when it's run, which seems surprising Why is it surprising? Surely that's the whole point

[Python-Dev] Add a new locale codec?

2012-02-07 Thread Victor Stinner
Hi, I added PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize() and PyUnicode_EncodeLocale() to Python 3.3 to fix bugs. I hesitate to expose this codec in Python: it can be useful is some cases, especially if you need to interact with C functions. The glib library has functions using the

Re: [Python-Dev] Add a new locale codec?

2012-02-07 Thread Simon Cross
Is the idea to have: bfoo.decode(locale) be roughly equivalent to encoding = locale.getpreferredencoding(False) bfoo.decode(encoding) ? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev