Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Paul Prescod
On 9/5/06, Marcin 'Qrczak' Kowalczyk <[EMAIL PROTECTED]> wrote: > David Hopwood <[EMAIL PROTECTED]> writes: > > > The whole idea of a default encoding is flawed. Ideally there would be > > no default; programmers should be forced to think about the issue > > on a case-by-case basis. In some cases t

Re: [Python-3000] encoding hell

2006-09-06 Thread Oleg Broytmann
On Tue, Sep 05, 2006 at 06:09:21PM -0700, Guido van Rossum wrote: > On 9/4/06, Oleg Broytmann <[EMAIL PROTECTED]> wrote: > >--- email database file --- > > [EMAIL PROTECTED] > > [EMAIL PROTECTED] > >--- / --- > > > > The program opens the file in "r+" mode, reads it line by line and > >stores the

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Oleg Broytmann
On Wed, Sep 06, 2006 at 03:08:21AM -0700, Paul Prescod wrote: > Windows users do not "tell each program separately about the > encoding." The encoding varies by file type. It makes no more sense to > have a global variable that says "all of my files are Shift-JIS" than > it does to say "all of my f

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Marcin 'Qrczak' Kowalczyk
"Paul Prescod" <[EMAIL PROTECTED]> writes: > Windows users do not "tell each program separately about the > encoding." The encoding varies by file type. There are lots of Unix file types which are based on text files and their encoding is not specified explicitly. > It makes no more sense to hav

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Paul Prescod
But how would a system-wide default encoding help with any of these situations? These situations are IN FACT caused by system-wide default encodings used by naive programmers. Python should be part of the solution, not part of the problem. On 9/6/06, Oleg Broytmann <[EMAIL PROTECTED]> wrote: > On

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Oleg Broytmann
On Wed, Sep 06, 2006 at 03:55:04AM -0700, Paul Prescod wrote: > But how would a system-wide default encoding help with any of these > situations? These situations are IN FACT caused by system-wide default > encodings used by naive programmers. Python should be part of the > solution, not part of th

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Brian Quinlan
Marcin 'Qrczak' Kowalczyk wrote: > Why would it matter? If most of their programs use UTF-8, and it's > specified by the locale, then fine. My system uses mostly ISO-8859-2, > and it's also fine, as long as there is a way for the program to get > that information. The problem is that blindly using

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Michael Urman
On 9/6/06, Oleg Broytmann <[EMAIL PROTECTED]> wrote: >These situations are caused because of the lack of metadata or clear > encoding-friendly standards. Ogg, for example, is encoding friendly - it > clearly states that tags (comments) must be in UTF-8, and all Ogg Vorbis > files I have saw wer

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread David Hopwood
Guido van Rossum wrote: > On 9/4/06, David Hopwood <[EMAIL PROTECTED]> wrote: >> Guido van Rossum wrote: >> >> > I've always said (can someone find a quote perhaps?) that there ought >> > to be a sensible default encoding for files (including but not limited >> > to stdin/out/err), perhaps influenc

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Jim Jewett
On 9/4/06, David Hopwood <[EMAIL PROTECTED]> wrote: > The issue is not simplicity of implementation; it is what will provide > the simplest usage model in the long term. If new files are encoded in X > just because most of a user's existing files are encoded in X, then how is > the user supposed t

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Paul Prescod
On 9/6/06, Marcin 'Qrczak' Kowalczyk <[EMAIL PROTECTED]> wrote: > "Paul Prescod" <[EMAIL PROTECTED]> writes: > > > Windows users do not "tell each program separately about the > > encoding." The encoding varies by file type. > > There are lots of Unix file types which are based on text files > and

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Paul Prescod
On 9/6/06, Oleg Broytmann <[EMAIL PROTECTED]> wrote: > On Wed, Sep 06, 2006 at 03:55:04AM -0700, Paul Prescod wrote: >These situations are caused because of the lack of metadata or clear > encoding-friendly standards. Ogg, for example, is encoding friendly - it > clearly states that tags (comme

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Paul Prescod
On 9/6/06, Michael Urman <[EMAIL PROTECTED]> wrote: > ... I suspect the best option is some sort of TextFile > constructor that defaults to ASCII (or has no default) but accepts an > easy way to use the "recommended" or system encoding, or any explicit > one. That's exactly what I'm asking for.

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Paul Prescod
On 9/4/06, David Hopwood <[EMAIL PROTECTED]> wrote: >... > I would prefer that there is no default. But since that is incompatible > with the existing API for open(), I accept that I'm not likely to win > that argument. First, can you outline how the proposal of no default is incompatible with the

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread Michael Urman
On 9/6/06, Paul Prescod <[EMAIL PROTECTED]> wrote: > On 9/6/06, Michael Urman <[EMAIL PROTECTED]> wrote: > > ... I suspect the best option is some sort of TextFile > > constructor that defaults to ASCII (or has no default) but accepts an > > easy way to use the "recommended" or system encoding, or

Re: [Python-3000] locale-aware strings ?

2006-09-06 Thread David Hopwood
Jim Jewett wrote: > On 9/4/06, David Hopwood <[EMAIL PROTECTED]> wrote: > >> The issue is not simplicity of implementation; it is what will provide >> the simplest usage model in the long term. If new files are encoded in X >> just because most of a user's existing files are encoded in X, then >>