[issue3080] Full unicode import system

2011-03-23 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: test the fixed nosy list -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-03-21 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: As I see Victor has dropped OS/2 support from Python/import.c Perhaps file Python/dynload_os2.c should be removed also. Not sure about other dynload_* files. 340f76a6a792 just removes few lines in import.c: they can easily be

[issue3080] Full unicode import system

2011-03-21 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: Understood. Sorry. I thought Python support only Windows and posix (Linux, BSD, MacOSX etc) systems now, all other OSes are not maintained. Anyway please don't care about that. -- ___

[issue3080] Full unicode import system

2011-03-21 Thread Roundup Robot
Roundup Robot devnull@devnull added the comment: New changeset 15f9eca5e956 by Victor Stinner in branch 'default': Issue #3080: On DJGPP, case_bytes() returns -1 to signal an error if the file http://hg.python.org/cpython/rev/15f9eca5e956 -- ___

[issue3080] Full unicode import system

2011-03-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.x/builds/1599/steps/test/logs/stdio == ERROR: testImpWrapper

[issue3080] Full unicode import system

2011-03-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: mod = imp.load_module(fullname, self.file, self.filename, self.stuff) TypeError: 'NoneType' object is not iterable The problem is that imp.find_module() now returns None as the filename, but imp.load_module() doesn't support

[issue3080] Full unicode import system

2011-03-20 Thread Éric Araujo
Éric Araujo mer...@netwok.org added the comment: Attached patch fixes a typo in Doc/c-api/import.rst. You can merge it in your next commit. -- Added file: http://bugs.python.org/file21305/typo.diff ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2011-03-20 Thread Andrew Svetlov
Changes by Andrew Svetlov andrew.svet...@gmail.com: -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___ ___ Python-bugs-list

[issue3080] Full unicode import system

2011-03-20 Thread Roundup Robot
Roundup Robot devnull@devnull added the comment: New changeset 7f4a4e393058 by Victor Stinner in branch 'default': Issue #3080: imp.load_module() accepts None for the module path http://hg.python.org/cpython/rev/7f4a4e393058 -- ___ Python tracker

[issue3080] Full unicode import system

2011-03-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Ok. Python 3.3 does now support non-ASCII characters in module paths and names on Windows, but only characters encodable to the ANSI code page. To support the full Unicode range, we should remove all calls to

[issue3080] Full unicode import system

2011-03-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: c) _PyImport_LoadDynamicModule() encodes the filename for _PyImport_GetDynLoadFunc(). The prototype should be changed, but only on Windows, to accept a filename as a Unicode string. Hum, the difficult part is to use Unicode in

[issue3080] Full unicode import system

2011-03-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Ok, I think that the most important part is now implemented in Python 3.3: use Unicode for module names and paths in the import machinery. Remaing parts are specific to Windows, and so I opened a new issue: #11619. Let's close this

[issue3080] Full unicode import system

2011-03-20 Thread Roundup Robot
Roundup Robot devnull@devnull added the comment: New changeset ee4e780a6b7a by Éric Araujo in branch 'default': Fix a typo (see #3080) http://hg.python.org/cpython/rev/ee4e780a6b7a -- ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2011-03-20 Thread Andrew Svetlov
Andrew Svetlov andrew.svet...@gmail.com added the comment: As I see Victor has dropped OS/2 support from Python/import.c Perhaps file Python/dynload_os2.c should be removed also. Not sure about other dynload_* files. -- ___ Python tracker

[issue3080] Full unicode import system

2011-03-19 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: Added file: http://bugs.python.org/file21296/issue3080.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-03-19 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: -- Removed message: http://bugs.python.org/msg123994 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-03-19 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Replace open_exclusive() by fopen(name, wb) on Windows: is it correct? I reverted this change in my Mercurial branch (unicode_import). rename xxxobj = xxx to keep original names and have a short patch done catch encoding

[issue3080] Full unicode import system

2011-03-19 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: test_runpy fails on Windows on make_legacy_pyc() (of test.support), I don't know why. Gotcha: I replaced mkdir() by CreateDirectoryW(), but the directory already exists error was not ignored. Fixed by 2debe178697b. --

[issue3080] Full unicode import system

2011-03-19 Thread Roundup Robot
Roundup Robot devnull@devnull added the comment: New changeset 6c80ac44ae9c by Victor Stinner in branch 'default': Issue #3080: zipimport has a full unicode suppport http://hg.python.org/cpython/rev/6c80ac44ae9c New changeset b50a0d44545a by Victor Stinner in branch 'default': Issue #3080:

[issue3080] Full unicode import system

2011-03-19 Thread Roundup Robot
Roundup Robot devnull@devnull added the comment: New changeset f8d6f6797909 by Victor Stinner in branch 'default': Issue #3080: Fix case_ok() using case_bytes() http://hg.python.org/cpython/rev/f8d6f6797909 -- ___ Python tracker

[issue3080] Full unicode import system

2011-03-19 Thread Roundup Robot
Roundup Robot devnull@devnull added the comment: New changeset dc38c4d65cd9 by Victor Stinner in branch 'default': Issue #3080: Fix call to case_ok() in find_init_module() http://hg.python.org/cpython/rev/dc38c4d65cd9 -- ___ Python tracker

[issue3080] Full unicode import system

2011-03-14 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I finished to split the huge patch into smaller commits. You can now test the unicode_import Mercurial branch. Especially, it should be tested on Windows. I don't know if I should merge the branch as an unique commit or as multiple

[issue3080] Full unicode import system

2011-03-11 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: See also #9319: when this issue will be fixed, it will be easier to fix #9319. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2011-03-10 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Nice work! Is there a specific place for comments? Here are some of them already: - Modules/zipimport.c::make_filename: remove the limit buffer, the code could look like: pathsize = PyUnicode_GET_SIZE(prefix) +

[issue3080] Full unicode import system

2011-03-10 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Is there a specific place for comments? Yes, but my work is not done. I still have parts to commit. _PyImport_GetDynLoadFunc still takes char* arguments. Oh. This one is not easy because this function has many implementations

[issue3080] Full unicode import system

2011-03-09 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I created the features/unicode_import repository with a unicode_import branch: http://hg.python.org/features/unicode_import/ It's my huge patch splitted into small and atomic commits. --

[issue3080] Full unicode import system

2011-03-04 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: r88746: Add PyModule_NewObject() function r88747: Add PyImport_AddModuleObject() and PyImport_ExecCodeModuleObject() -- ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2011-02-23 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: This new failure is perhaps related: http://www.python.org/dev/buildbot/all/builders/AMD64%20Windows%20Server%202008%203.x/builds/572/steps/test/logs/stdio == FAIL: test_module

[issue3080] Full unicode import system

2011-02-23 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: This new failure is perhaps related: (...) test_reprlib Ah yes, yesterday, I tried to remember which test was impacted by the module change, but all tests passed on Linux. Anyway, it's now fixed by r88533. --

[issue3080] Full unicode import system

2011-02-22 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I started to commit some parts of the huge patch: r88515: Mark PyWin_FindRegisteredModule() as private r88516: Remove unused argument of _PyImport_GetDynLoadFunc() r88517 (3.3), r88518 (3.2): document encoding used by import

[issue3080] Full unicode import system

2011-02-22 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: r88519: Mark _PyImport_FindBuiltin() argument as constant r88520: Add PyModule_GetNameObject() -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2011-01-31 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: As explained in issue #10828: Python 3.2 doesn't support non-ASCII module names on Windows because module names are encoded to UTF-8 instead of the filesystem encoding (the ANSI code page). --

[issue3080] Full unicode import system

2011-01-31 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: See also #6011. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___ ___

[issue3080] Full unicode import system

2011-01-21 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: I haven't investigated in detail yet, but this is the final line showing the failing test: test_module (importlib.test.builtin.test_loader.LoaderTests) ... Segmentation fault -- ___ Python tracker

[issue3080] Full unicode import system

2011-01-21 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: except for the segfault in test_importlib. Yes, as reported in my previous comment :-) Let's update the patch for practical reasons. But I don't want to touch http://codereview.appspot.com/1874048 (based on patch version 4).

[issue3080] Full unicode import system

2011-01-21 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file20474/issue3080-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-01-21 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file20475/issue3080-4-svn.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-01-21 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: Oops, missed that post - that was indeed the problem. With that fixed, tests are all good on this system. I'll give the patch a look anyway, but I'm going to have trouble diagnosing things that don't fail on my development machine. As far as

[issue3080] Full unicode import system

2011-01-21 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: but I'm going to have trouble diagnosing things that don't fail on my development machine. On Windows, try any character not encodable into your ANSI code page (eg. Ł with cp1252) in the module path and non-ASCII characters in

[issue3080] Full unicode import system

2011-01-21 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I tried issue3080-5.patch. The whole test suite pass on Windows. It pass also on Linux with -Wd -Werror -R 3:3: (except #10971 which is unrelated to this issue). I should maybe add some unit tests for non-ASCII module paths and

[issue3080] Full unicode import system

2011-01-21 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: On Sat, Jan 22, 2011 at 3:08 AM, STINNER Victor rep...@bugs.python.org wrote: I should maybe add some unit tests for non-ASCII module paths and non-ASCII module names :-) Indeed. There are a few tests in test_runpy that could be adapted to

[issue3080] Full unicode import system

2011-01-20 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: Victor, could you please create a Reitveld review for this? The auto-review creator can't cope with the Git diffs. -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Victor, could you please create a Reitveld review for this? Yes, but not yet. I have first to cleanup the patch. -- ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2011-01-20 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: OK - I'll wait until that is ready before digging into this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Use U format to parse a module name, and %R to format a module name (to escape surrogates characters and add quotes, instead of ... '%.200s' ...). See also #8754: repr() is better than str() for other reasons, eg. to see a

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Version 4 of the patch. -- Added file: http://bugs.python.org/file20474/issue3080-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file20448/issue3080-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Same patch (version 4) generated by svn. -- Added file: http://bugs.python.org/file20475/issue3080-4-svn.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: You can review the patch with Rietveld: http://codereview.appspot.com/3972045 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Oops, there is a dummy typo in imp_init_builtin() that makes test_importlib to crash (which proves that importlib has a good coverage :-)): replace s: by U: in if (!PyArg_ParseTuple(args, s:init_builtin, name)). --

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: test_reprlib fails on Windows, because '\' in quoted '\\' in the filename on repr(module). Workaround: *** index b0dc4d7..e476941 100644 --- a/Lib/test/test_reprlib.py +++ b/Lib/test/test_reprlib.py @@ -234,7 +234,7 @@ class

[issue3080] Full unicode import system

2011-01-20 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: test_runpy fails on Windows on make_legacy_pyc() (of test.support), I don't know why. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2011-01-20 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: After applying the patch, doing a make clean and rebuild, I found that test_importlib fails with a segmentation fault, but the default test suite otherwise runs without error (that's on Linux with a UTF-8 filesystem, though). I'll see how a

[issue3080] Full unicode import system

2011-01-20 Thread Nick Coghlan
Nick Coghlan ncogh...@gmail.com added the comment: As for the more limited run, I get a clean run with -uall except for the segfault in test_importlib. I'll switch to a pydebug build and see how a verbose run of that test fares. -- ___ Python

[issue3080] Full unicode import system

2011-01-19 Thread Alexander Belopolsky
Changes by Alexander Belopolsky belopol...@users.sourceforge.net: -- nosy: +belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___ ___

[issue3080] Full unicode import system

2011-01-18 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Here is a work-in-progress patch: issue3080-3.patch. The patch is HUGE and written for Python 3.3. $ diffstat issue3080-3.patch Doc/c-api/module.rst | 24 Include/import.h | 73 + Include/moduleobject.h |2

[issue3080] Full unicode import system

2011-01-18 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: This patch changes more lines of code than my previous crazy unicode patch (msg103663, issue #8242 #8611 #9425), but it changes less files. -- ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2011-01-18 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Oh, msg103663 was not the final patch. A more recent version of my patch for #8611 / #9425 is http://codereview.appspot.com/1874048: Doc/library/sys.rst |6 Include/Python.h|4 Include/fileobject.h

[issue3080] Full unicode import system

2011-01-07 Thread Terry J. Reedy
Terry J. Reedy tjre...@udel.edu added the comment: If I edit a file with IDLE, save it, and successfully run it (perhaps to test it), then when I edit a second file that imports the first, I expect the import to work. It does not always (see #10828). Import is part of the core definition of

[issue3080] Full unicode import system

2010-12-27 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Issue #10785 prepares the work for this issue: store input filename as a unicode string, instead of a byte string, in the parser. -- ___ Python tracker rep...@bugs.python.org

[issue3080] Full unicode import system

2010-12-27 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: -- versions: +Python 3.3 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2010-12-14 Thread R. David Murray
R. David Murray rdmur...@bitdance.com added the comment: With #1342 fixed, it seems that this issue is no longer critical (Haypo describes his complicated patch as useful on Windows, but not critical. So I'm downgrading it to 'high'. Perhaps it is even 'normal'. It also seems as though it

[issue3080] Full unicode import system

2010-12-14 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Haypo describes his complicated patch as useful on Windows, but not critical Usecase on Windows: your japanese friend gives you an USB key (eg. created on Windows with code page 932) with his Python project, you cannot run it on

[issue3080] Full unicode import system

2010-12-14 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Haypo describes his complicated patch as useful on Windows, but not critical Usecase on Windows: your japanese friend gives you an USB key (eg. created on Windows with code page 932) with his Python project, you cannot run it on

[issue3080] Full unicode import system

2010-10-18 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: With #8611 and #9425, I patched a lot of functions and modules, including the NullImporter and zipimport, but not the core of the import machinery. In my import_unicode SVN branch, I patched the import machinery to manipulate

[issue3080] Full unicode import system

2010-10-18 Thread Éric Araujo
Changes by Éric Araujo mer...@netwok.org: -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___ ___ Python-bugs-list

[issue3080] Full unicode import system

2010-07-29 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I posted a patch: #9425. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2010-07-10 Thread Georg Brandl
Georg Brandl ge...@python.org added the comment: Victor is working on this. -- assignee: - haypo nosy: +georg.brandl, haypo versions: -Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080

[issue3080] Full unicode import system

2010-06-30 Thread Arfrever Frehtes Taifersar Arahesis
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2010-04-27 Thread Terry J. Reedy
Changes by Terry J. Reedy tjre...@udel.edu: -- versions: +Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3080 ___ ___ Python-bugs-list

[issue3080] Full unicode import system

2008-08-09 Thread Antoine Pitrou
Changes by Antoine Pitrou [EMAIL PROTECTED]: -- priority: - critical type: - behavior versions: +Python 3.1 -Python 3.0 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3080 ___

[issue3080] Full unicode import system

2008-06-11 Thread Amaury Forgeot d'Arc
New submission from Amaury Forgeot d'Arc [EMAIL PROTECTED]: This is the most difficult part of issue1342: On Windows, don't use the FileSystemEncoding on Windows for sys.path items. Instead, it should use the wide API to perform all system calls. Py3k shouldn't ever use the file system encoding

[issue3080] Full unicode import system

2008-06-11 Thread Benjamin Peterson
Benjamin Peterson [EMAIL PROTECTED] added the comment: I suspect importlib may help with this. -- nosy: +benjamin.peterson, brett.cannon ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3080 ___