[issue11230] "Full unicode import system" not in 3.2

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Python 3.2 is out of maintenance. Full Unicode path support was added in Python 
3.3 by issue3080.

--
nosy: +serhiy.storchaka
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[Tom Christiansen]

Tom Christiansen tchr...@perl.com added the comment:

Whoops, I meant that it appears that Python runs its identifiers through NFC.  
How that gets along with a filesystem that has quasi-NFD filenames I'm not 
sure, but it seems like it might be a variant of the case-insensitivity issue 
in filenames.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

 The issue I'm thinking about is that the Mac HSF+ filesystem

There is no issue with HFS+ normalization. The kernel normalizes filenames to 
its own variant, Python doesn't have to care about this.

When you write import hé normalized to NFC or import hé normalized to 
NFD, Python tries to open hé normalized to NFC.py: then the HFS+ filename 
does its own normalization (= hé normalized to its variant of NFD.py).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[Tom Christiansen]

Tom Christiansen tchr...@perl.com added the comment:

How does this work for modules that have filesystem names different from the 
one used for import? The issue I'm thinking about is that the Mac HSF+ 
filesystem keeps its Unicode filenames in (close to) NFD form. That means that 
a module named caf\N{LATIN SMALL LETTER E WITH ACUTE} with 4 graphemes and 4 
code points in its name winds up in the filesystem as cafe\N{COMBINING ACUTE 
ACCENT} still with 4 graphemes but now with 5 code points.

I believe (well, suspect; I have empirical evidence not proof) Python stores 
its own identifiers in NFD, so this may not be quite as much of a problem as it 
might otherwise be.  Nonetheless, I have had users complain about what HFS+ 
does with such filenames, although I am not quite sure why. I think it’s 
because they access a file with 4 chars but they need a 5-char fileglob to 
wildcard it, so touch caf\N{LATIN SMALL LETTER E WITH ACUTE} and then you 
need a wildcard of ? with an extra ? to find it. Kinda weird.

--
nosy: +tchrist

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[Ned Deily]

Changes by Ned Deily n...@acm.org:


--
nosy: +georg.brandl

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[John]

John jh45...@gmail.com added the comment:

Sorry for the long delay.

haypo wrote:
  Can you propose a sentence which is more clear about bytes/Unicode?

On this page:
http://www.python.org/download/releases/3.2/
is this line:
- countless fixes regarding bytes/string issues; among them full support for a 
bytes environment (filenames, environment variables)

How about adding to that line something like:
 on UNIX; but on Windows the path to and name of each module you import can 
contain only characters that are in the ANSI codepage that your Windows is 
using

and maybe
 (will be fixed in Python 3.3)

and maybe (or not) also something like:
 (ANSI codepage = basic latin + other characters of only your own language 
group)

-- jh

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +eric.araujo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

 Victor asked Which web page should updated/fixed?
 Answer: The Python 3.2 download page.

Sorry, but I don't see which page tells that Python 3.2 has a full Unicode 
support for import. In http://www.python.org/download/releases/3.2/, I can read 
countless fixes regarding bytes/string issues; among them full support for a 
bytes environment (filenames, environment variables).

full support for a bytes environment means that Python 3.2 has been fixed on 
UNIX to support undecodable filenames, but not that Python 3.2 supports 
unencodable filenames on Windows.

Can you propose a sentence which is more clear about bytes/Unicode?

Python 3.3 will have a full Unicode support for modules: issue #3080 is already 
fixed, and I think that #11619 can be fixed (maybe not easily).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

Short answer:

In Python 3.2, « import héhé » doesn't work on Windows, but you can have 
non-ASCII paths in sys.path.

Longer answer:

I fixed the import machinery to handle correctly non-ASCII characters in module 
*paths*. But the import machinery is unable to handle non-ASCII characters in 
module *names*: it fails if the filesystem encoding is not UTF-8 (eg. it fails 
on Windows). There is another exception: Python doesn't support (yet) non 
encodable module paths on Windows. On Windows, you can use any character in 
directory names, but Python 3.2 encodes paths to the filesystem encoding (ANSI 
code page) which is a smaller charset. In practical, this Windows specific 
limitation on module paths doesn't really matter.

I plan to fix all these issues in Python 3.3: see #3080.

--

 Could you please make it clear in documentation and web pages,
 that this feature is not working yet. 

What's New in Python 3.2 documentation has this sentence: Python’s import 
mechanism can now load modules installed in directories with non-ASCII 
characters in the path name. This solved an aggravating problem with home 
directories for users with non-ASCII characters in their usernames. which is 
correct.

Which web page should updated/fixed?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[John]

John jh45...@gmail.com added the comment:

Victor asked Which web page should updated/fixed?

Answer: The Python 3.2 download page.

But what should it say?

The main point is that people like me, who remember seeing a statement about 
this a few months ago, may expect unicode to work in every conceivable 
situation, and a prominent warning that it's not *all* fixed yet, with a link 
to details in the documentation, would save them from trying things that don't 
work.

By the way, I hadn't grasped a simple point from issue 3080: I tested on 
*English* Windows by putting a Greek character in the path to some python 
modules. But the usual situation is where a *Greek* version of Windows has some 
Greek characters in the path, and from what you just wrote, that's OK now.

-- jh

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11230] Full unicode import system not in 3.2

[John]

New submission from John jh45...@gmail.com:

A few months ago I read that in 3.2 it will be possible to import modules that 
are located on paths containing any unicode character. (more precisely, with 
chars not in the local code page)

After an hour or two trying to get this to work in 3.2rc3, I went looking for 
clues, and found these 2 messages in which Victor Stinner says this feature is 
delayed until Python 3.3:
http://bugs.python.org/issue3080#msg126514
http://bugs.python.org/issue10828#msg125787

Could you please make it clear in documentation and web pages, that this 
feature is not working yet. 

The Python 3.2 download page includes this:
countless fixes regarding bytes/string issues; among them full support for a 
bytes environment (filenames, environment variables)
and I guessed this must cover importing from any unicode path, as there was no 
mention that such importing had been abandoned for this version.

-- jh

--
assignee: docs@python
components: Documentation
messages: 128711
nosy: docs@python, jh45
priority: normal
severity: normal
status: open
title: Full unicode import system not in 3.2
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11230
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com