On Tuesday 30 September 2008, M.-A. Lemburg wrote:
On 2008-09-30 08:00, Martin v. Löwis wrote:
Change the default file system encoding to store bytes in Unicode is
like introducing a new Python type: fake Unicode for filename hacks.
Exactly. Seems like the best solution to me, despite
On 2008-10-01 09:54, Ulrich Eckhardt wrote:
On Tuesday 30 September 2008, M.-A. Lemburg wrote:
On 2008-09-30 08:00, Martin v. Löwis wrote:
Change the default file system encoding to store bytes in Unicode is
like introducing a new Python type: fake Unicode for filename hacks.
Exactly. Seems
[EMAIL PROTECTED] wrote:
The reasoning is that a lot of software doesn't care if it's wrong for
edge cases, it's really hard to come up with something that's correct
with respect to all of those edge cases (absurdly difficult, if you need
to stay in the straightjacket of string / bytes types,
M.-A. Lemburg [EMAIL PROTECTED] wrote:
On 2008-10-01 09:54, Ulrich Eckhardt wrote:
On Tuesday 30 September 2008, M.-A. Lemburg wrote:
On 2008-09-30 08:00, Martin v. Löwis wrote:
Change the default file system encoding to store bytes in Unicode is
like introducing a new Python type: fake
On 03:54 pm, [EMAIL PROTECTED] wrote:
I'm actually sort of liking this idea. A Pathname class, for
convenience
a subtype of String, but containing the underlying binary
representation
used by the OS. Even non-unicode pathnames could be represented.
On the one hand, I agree with you -
[EMAIL PROTECTED] wrote:
I'm actually sort of liking this idea. A Pathname class, for
convenience
a subtype of String, but containing the underlying binary
representation
used by the OS. Even non-unicode pathnames could be represented.
On the one hand, I agree with you - except for
SQLite has a similar problem with NULLs, and I'm definitely sticking
paths in there, too.
I think that you can say all C libraries.
Just for the sake of nit-picking: the socket library, and the regular
POSIX stream IO library (as well as C standard unformatted IO) deal
just fine with
Bill Janssen wrote:
Perhaps PEP 355 just went too far.
That was certainly one of the major objections to it. A filesystem path
object which didn't try to combine a half-dozen different modules into
methods on a single object, but instead focused on solving a few
specific problems with using raw
Change the default file system encoding to store bytes in Unicode is like
introducing a new Python type: fake Unicode for filename hacks.
Exactly. Seems like the best solution to me, despite your polemics.
Regards,
Martin
___
Python-Dev mailing
On Tue, Sep 30, 2008 at 12:22 AM, Georg Brandl [EMAIL PROTECTED] wrote:
Victor Stinner schrieb:
Le Monday 29 September 2008 18:45:28 Georg Brandl, vous avez écrit :
If I had to choose, I'd still argue for the modified UTF-8 as filesystem
encoding (if it were UTF-8 otherwise), despite possible
On 2008-09-30 08:00, Martin v. Löwis wrote:
Change the default file system encoding to store bytes in Unicode is like
introducing a new Python type: fake Unicode for filename hacks.
Exactly. Seems like the best solution to me, despite your polemics.
Not a bad idea... have os.listdir()
Adam Olsen writes:
[1] You could argue that Unicode should add new scalars to handle all
currently invalid UTF-8 sequences.
AFAIK there are about 2^31 of these, though!
___
Python-Dev mailing list
Python-Dev@python.org
On Tue, Sep 30, 2008 at 5:24 AM, Stephen J. Turnbull [EMAIL PROTECTED] wrote:
Adam Olsen writes:
[1] You could argue that Unicode should add new scalars to handle all
currently invalid UTF-8 sequences.
AFAIK there are about 2^31 of these, though!
They've promised to never allocate above
On Mon, Sep 29, 2008 at 8:55 PM, Terry Reedy [EMAIL PROTECTED] wrote:
Le Monday 29 September 2008 19:06:01 Guido van Rossum, vous avez écrit :
I know I keep flipflopping on this one, but the more I think about it
the more I believe it is better to drop those names than to raise an
exception.
On Mon, Sep 29, 2008 at 11:00 PM, Martin v. Löwis [EMAIL PROTECTED] wrote:
Change the default file system encoding to store bytes in Unicode is like
introducing a new Python type: fake Unicode for filename hacks.
Exactly. Seems like the best solution to me, despite your polemics.
Martin, I
On Tue, 30 Sep 2008 11:50:10 pm Guido van Rossum wrote:
To avoid silent skipping, is it possible to drop 'unreadable'
names, issue a warning (instead of exception), and continue to
completion? Warning: unreadable filename skipped; see
PyWiki/UnreadableFilenames
That would be annoying as
On Tue, Sep 30, 2008 at 7:53 AM, Steven D'Aprano [EMAIL PROTECTED] wrote:
On Tue, 30 Sep 2008 11:50:10 pm Guido van Rossum wrote:
To avoid silent skipping, is it possible to drop 'unreadable'
names, issue a warning (instead of exception), and continue to
completion? Warning: unreadable
On Tue, Sep 30, 2008 at 8:20 AM, M.-A. Lemburg [EMAIL PROTECTED] wrote:
In the end, I think it's better not to be clever and just return
the filenames that cannot be decoded as bytes objects in os.listdir().
Unfortunately that's going to break most code that is using
os.listdir(), so it's
Guido van Rossum schrieb:
With the filenames decoded by UTF-8, your files named têste, ô, dossié will
be displayed and handled correctly. The others are *invalid* in the
filesystem
encoding UTF-8 and therefore would be represented by something like
u'dir\uXXffname' where XX is some private
Steven D'Aprano schrieb:
On Tue, 30 Sep 2008 11:50:10 pm Guido van Rossum wrote:
To avoid silent skipping, is it possible to drop 'unreadable'
names, issue a warning (instead of exception), and continue to
completion? Warning: unreadable filename skipped; see
PyWiki/UnreadableFilenames
On 2008-09-30 18:46, Guido van Rossum wrote:
On Tue, Sep 30, 2008 at 8:20 AM, M.-A. Lemburg [EMAIL PROTECTED] wrote:
In the end, I think it's better not to be clever and just return
the filenames that cannot be decoded as bytes objects in os.listdir().
Unfortunately that's going to break
Guido van Rossum schrieb:
On Tue, Sep 30, 2008 at 10:28 AM, Georg Brandl [EMAIL PROTECTED] wrote:
How can it *regularly* drive you crazy when the majority of fie names
[...] encoded correctly (as you assert above)?
Because Office files are a) often named with long, seemingly descriptive
On Tue, Sep 30, 2008 at 11:13 AM, Georg Brandl [EMAIL PROTECTED] wrote:
Victor Stinner schrieb:
On Windows, we might reject bytes filenames for all file operations: open(),
unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError)
Since I've seen no objections to this yet: please no.
Guido van Rossum wrote:
On Mon, Sep 29, 2008 at 11:00 PM, Martin v. Löwis [EMAIL PROTECTED] wrote:
Change the default file system encoding to store bytes in Unicode is like
introducing a new Python type: fake Unicode for filename hacks.
Exactly. Seems like the best solution to me, despite your
I didn't get an answer to my question: what is the result bytes (fake
characters) stored in unicode + real unicode? I guess that the result is
mixed bytes and characters in unicode instead of raising an error
(invalid types). So again: why introducing a new type instead of reusing
Guido van Rossum wrote:
However
the *proposed* behavior (returns bytes if the arg was bytes, and
returns str when the arg was str) is IMO sane, and no different than
the polymorphism found in len() or many builtin operations.
My concern still is that it brings the bytes type into the status
I'm not sure either way. I've heard it claim that Windows filesystem
APIs use Unicode natively. Does Python 3.0 on Windows currently
support filenames expressed as bytes?
Yes, it does (at least, os.open, os.stat support them, builtin open
doesn't).
Are they encoded first before
passing to
On Tue, Sep 30, 2008 at 12:42 PM, Terry Reedy [EMAIL PROTECTED] wrote:
Guido van Rossum wrote:
On Tue, Sep 30, 2008 at 11:13 AM, Georg Brandl [EMAIL PROTECTED] wrote:
Victor Stinner schrieb:
On Windows, we might reject bytes filenames for all file operations:
open(),
unlink(),
On Tue, Sep 30, 2008 at 1:04 PM, Martin v. Löwis [EMAIL PROTECTED] wrote:
Guido van Rossum wrote:
On Mon, Sep 29, 2008 at 11:00 PM, Martin v. Löwis [EMAIL PROTECTED]
wrote:
Change the default file system encoding to store bytes in Unicode is like
introducing a new Python type: fake Unicode
On Tue, Sep 30, 2008 at 1:12 PM, Terry Reedy [EMAIL PROTECTED] wrote:
Terry Reedy wrote:
Guido van Rossum wrote:
I'm not sure either way. I've heard it claim that Windows filesystem
APIs use Unicode natively. Does Python 3.0 on Windows currently
support filenames expressed as bytes? Are
On Tue, Sep 30, 2008 at 1:29 PM, Martin v. Löwis [EMAIL PROTECTED] wrote:
Guido van Rossum wrote:
However
the *proposed* behavior (returns bytes if the arg was bytes, and
returns str when the arg was str) is IMO sane, and no different than
the polymorphism found in len() or many builtin
On Windows, we might reject bytes filenames for all file operations: open(),
unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError)
Since I've seen no objections to this yet: please no. If we offer a
lower-level bytes filename API, it should work for all platforms.
Oh, ok. I had assumed Windows just uses a fixed encoding without the problem
of misencoded filenames.
It's the other way 'round: On Windows, Unicode file names are the
natural choice, and byte strings have limitations. In a sense, Windows
got it right - but then, they started later. Unix missed
2008/9/30 Glenn Linderman [EMAIL PROTECTED]:
So the problem is that a Unicode file system interface can't deal with
non-UTF-8 byte streams as file names.
So it seems there are four suggested approaches, all of which have aspects
that are inconvenient.
Let's not forget what happens when a
On Sep 30, 2008, at 5:40 PM, Martin v. Löwis wrote:
On Windows, we might reject bytes filenames for all file
operations: open(),
unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError)
Since I've seen no objections to this yet: please no. If we offer a
lower-level bytes filename
On 30-Sep-2008, at 23:42 , Martin v. Löwis wrote:
It's the other way 'round: On Windows, Unicode file names are the
natural choice, and byte strings have limitations. In a sense, Windows
got it right - but then, they started later. Unix missed the
opportunity
of declaring that all file APIs
My concern still is that it brings the bytes type into the status of
another character string type, which is really bad, and will require
further modifications to Python for the lifetime of 3.x.
I'd like to understand why this is really bad. I though it was by
design that the str and bytes
Yes! If there is a byte-string access method for Windows, pretty please
make it decode from UTF-8 internally and call the Unicode version of the
Windows APIs. The non-unicode windows APIs are pretty much just broken
-- Ideally, Python should never be calling those.
I don't think we will
How does windows (and Python on windows) handle NFC versus NFD issues?
That's left to the application.
Can I have two files called ümlaut.txt, one in NFD and one NFC form?
Yes, you can. It sounds confusing, but only in a theoretical way. You
never have combining characters on Windows (at
On Tue, Sep 30, 2008 at 3:21 PM, Martin v. Löwis [EMAIL PROTECTED] wrote:
My concern still is that it brings the bytes type into the status of
another character string type, which is really bad, and will require
further modifications to Python for the lifetime of 3.x.
I'd like to understand
On Sep 30, 2008, at 6:21 PM, Martin v. Löwis wrote:
IOW, Java hasn't solved the problem in the last 10 years.
Java is already really bad at being a small little language to write
cooperating tools in. I'd never even attempt to write a little
pipeline filter in Java -- I've already pretty
On 1-Oct-2008, at 00:32 , Martin v. Löwis wrote:
How does windows (and Python on windows) handle NFC versus NFD
issues?
That's left to the application.
Can I have two files called ümlaut.txt, one in NFD and one NFC
form?
Yes, you can. It sounds confusing, but only in a theoretical
On Wed, 1 Oct 2008 07:40:01 am Martin v. Löwis wrote:
On Windows, we might reject bytes filenames for all file
operations: open(), unlink(), os.path.join(), etc. (raise a
TypeError or UnicodeError)
Since I've seen no objections to this yet: please no. If we offer a
lower-level bytes
On Tue, Sep 30, 2008 at 4:08 PM, Steven D'Aprano [EMAIL PROTECTED] wrote:
On Wed, 1 Oct 2008 07:40:01 am Martin v. Löwis wrote:
On Windows, we might reject bytes filenames for all file
operations: open(), unlink(), os.path.join(), etc. (raise a
TypeError or UnicodeError)
Since I've seen
On Wed, 1 Oct 2008 09:21:37 am you wrote:
On Tue, Sep 30, 2008 at 4:08 PM, Steven D'Aprano [EMAIL PROTECTED]
wrote:
On Wed, 1 Oct 2008 07:40:01 am Martin v. Löwis wrote:
On Windows, we might reject bytes filenames for all file
operations: open(), unlink(), os.path.join(), etc. (raise a
On Tue, Sep 30, 2008 at 7:04 PM, Steven D'Aprano [EMAIL PROTECTED] wrote:
I believe on disk it uses UTF-16.
Which is made up of bytes. There may be byte sequences that are illegal
UTF-16, but that's not what Martin said. I don't understand how there
can be UTF-16 sequences which don't
Le Wednesday 01 October 2008 00:28:22 Martin v. Löwis, vous avez écrit :
I don't think we will manage to release Python 3.0 this year if that
change is to be implemented. And then, I don't think the release manager
will agree to such a delay.
The minimum change is to disallow bytes/str mix:
-
On 30 Sep, 09:22 pm, [EMAIL PROTECTED] wrote:
On Tue, Sep 30, 2008 at 1:04 PM, Martin v. Löwis [EMAIL PROTECTED]
wrote:
Guido van Rossum wrote:
On Mon, Sep 29, 2008 at 11:00 PM, Martin v. Löwis
[EMAIL PROTECTED] wrote:
Martin, I don't understand why you are in favor of storing raw bytes
On Tue, Sep 30, 2008 at 8:06 PM, [EMAIL PROTECTED] wrote:
The proposal of using U+ seems like it would have been almost the same
from such a wrapper's perspective, except (A) people using the filesystem
APIs without the benefit of such a wrapper would have been even more
screwed, and (B)
On Sep 30, 2008, at 10:06 PM, [EMAIL PROTECTED] wrote:
However, Martin, I can promise you that I will _never_ ask for any
convenience functions related to bytes as a result of this
decision. I want bytes to come back from filesystem APIs because I
intend to have a wrapper layer which
Guido van Rossum wrote:
No, that's because bytes is missing from the explicit list of
allowable types in io.open. Victor has a one-line trivial patch for
this. Could you try this though?
import _fileio
_fileio._FileIO(b'tem')
import _fileio
_fileio._FileIO(b'tem')
_fileio._FileIO(3, 'r')
On 03:32 am, [EMAIL PROTECTED] wrote:
On Sep 30, 2008, at 10:06 PM, [EMAIL PROTECTED] wrote:
Can you clarify what proposal you are supporting for Python:
Sure. Neither of your descriptions is terribly accurate, but I'll try
to explain.
1) Two sets of APIs, one returning unicode strings,
Sorry, maybe I'm just being thick here, but I don't understand how that
is possible. On the physical disk, each Windows file name must be
represented by a byte string, yes? So how is it possible that there are
Windows files with names that can't be represented as a byte string?
What have
However, Martin, I can promise you that I will _never_ ask for any
convenience functions related to bytes as a result of this decision.
:-)
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
On Mon, Sep 29, 2008 at 6:07 AM, Victor Stinner
[EMAIL PROTECTED] wrote:
The default behaviour should be to use unicode and raise an error if
conversion to unicode fails. It should also be possible to use bytes using
bytes arguments and optional arguments (for getcwd).
- listdir(unicode) -
Le Monday 29 September 2008 17:16:47 Steven Bethard, vous avez écrit :
- getcwd() - unicode
- getcwd(bytes=True) - bytes
Please let's not introduce boolean flags like this. How about
``getcwdb`` in parallel with the old ``getcwdu``?
Yeah, you're right. So i wrote a new patch:
On Mon, Sep 29, 2008 at 10:00 AM, Victor Stinner
[EMAIL PROTECTED] wrote:
Le Monday 29 September 2008 17:16:47 Steven Bethard, vous avez écrit :
- getcwd() - unicode
- getcwd(bytes=True) - bytes
Please let's not introduce boolean flags like this. How about
``getcwdb`` in parallel with
On Mon, Sep 29, 2008 at 11:06 AM, Guido van Rossum [EMAIL PROTECTED] wrote:
On Mon, Sep 29, 2008 at 9:45 AM, Georg Brandl [EMAIL PROTECTED] wrote:
This approach (changing all path-handling functions to accept either bytes
or string, but not both) is doomed in my eyes. First, there are lots of
On Sep 29, 2008, at 6:17 PM, Adam Olsen wrote:
I suspect linux will eventually take this route as well. If ext3 had
an option for UTF-8 validation I know I'd want it on. That'd move the
error to the program creating bogus file names, rather than those
trying to read, display, and manage them.
On Mon, Sep 29, 2008 at 4:29 PM, Victor Stinner
[EMAIL PROTECTED] wrote:
Le Monday 29 September 2008 19:06:01 Guido van Rossum, vous avez écrit :
- listdir(unicode) - unicode and raise an error on invalid filename
I know I keep flipflopping on this one, but the more I think about it
the
import os
import os.path
import sys
if os.path.supports_unicode_filenames:
cwd = getcwd()
else:
cwd = getcwdb()
encoding = sys.getfilesystemencoding()
for filename in os.listdir(cwd):
if os.path.supports_unicode_filenames:
text = str(filename,
On Mon, Sep 29, 2008 at 5:29 PM, Victor Stinner
[EMAIL PROTECTED] wrote:
Le Monday 29 September 2008 19:06:01 Guido van Rossum, vous avez écrit :
- listdir(unicode) - unicode and raise an error on invalid filename
I know I keep flipflopping on this one, but the more I think about it
the
Guido van Rossum writes:
On Mon, Sep 29, 2008 at 4:29 PM, Victor Stinner
[EMAIL PROTECTED] wrote:
It would be hard for a newbie programmer to understand why he's
unable to find his very important file (important r?port.doc)
using os.listdir().
*Every* failure in this scenario will
63 matches
Mail list logo
