[issue1342] Crash on Windows if Python runs from a directory with umlauts
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: Fixed as r64126, using Py_FileSystemDefaultEncoding. I close this issue, and open issue3080 to rewrite all functions in import.c with full unicode in mind. -- status: open - closed ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: Here is a quick fix, that decodes filenames using Py_FileSystemDefaultEncoding, to let the release pass. I am still working on a version that keep PyObjects* as long as possible, but it will be a major change. -- keywords: +patch priority: release blocker - critical Added file: http://bugs.python.org/file10575/win_nonascii.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Antoine Pitrou [EMAIL PROTECTED] added the comment: There are some problems under Linux too: $ pwd /home/antoine/py3k/héhé $ ./python Fatal Python error: Py_Initialize: can't initialize sys standard streams Traceback (most recent call last): File /home/antoine/py3k/pristine/Lib/encodings/__init__.py, line 32, in module TypeError: zipimporter() argument 1 must be string without null bytes, not str Abandon -- nosy: +pitrou __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Antoine Pitrou [EMAIL PROTECTED] added the comment: See #2798 for the non-Windows case, with a patch. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes [EMAIL PROTECTED] added the comment: I'm increasing the severity of the bug. It's a still a major show stopper for non-English Windows users. For example see #2780 -- priority: normal - release blocker __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Changes by Giovanni Bajo: -- nosy: +rasky __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Changes by Georg Brandl: -- keywords: -rfe __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Amaury Forgeot d'Arc added the comment: Agreed. I will try to stay with PyObjects* until really needed by a system call. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Amaury Forgeot d'Arc added the comment: Assign to myself. Among the things to do, use Py_FileSystemDefaultEncoding (=mbcs on Windows) to encode sys.path items; likewise in NullImporter_init and other functions. So many places to change, we need serious testcases. -- assignee: - amaury.forgeotdarc nosy: +amaury.forgeotdarc __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Martin v. Löwis added the comment: Please don't use the FileSystemEncoding on Windows for sys.path items. Instead, it should use the wide API to perform all system calls. Py3k shouldn't ever use the file system encoding for anything on Windows. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes added the comment: I like to move _PyExc_Init() before _PySys_Init() and set sys.prefix, exec_prefix and executable with PyUnicode_DecodeFSDefault(). Without the changes Python is seg faulting on Windows when the path contains non ASCII chars. With the patch it is failing with a fatal error which is a tiny bit nicer. Added file: http://bugs.python.org/file8741/py3k_win_nonascii.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __Index: Python/pythonrun.c === --- Python/pythonrun.c (revision 58959) +++ Python/pythonrun.c (working copy) @@ -217,6 +217,9 @@ Py_FatalError(Py_Initialize: can't initialize builtins dict); Py_INCREF(interp-builtins); + /* initialize builtin exceptions */ + _PyExc_Init(); + sysmod = _PySys_Init(); if (sysmod == NULL) Py_FatalError(Py_Initialize: can't initialize sys); @@ -239,9 +242,6 @@ _PyImport_Init(); - /* initialize builtin exceptions */ - _PyExc_Init(); - /* phase 2 of builtins */ _PyImport_FixupExtension(__builtin__, __builtin__); Index: Python/sysmodule.c === --- Python/sysmodule.c (revision 58958) +++ Python/sysmodule.c (working copy) @@ -1072,11 +1072,12 @@ SET_SYS_FROM_STRING(platform, PyUnicode_FromString(Py_GetPlatform())); SET_SYS_FROM_STRING(executable, - PyUnicode_FromString(Py_GetProgramFullPath())); + PyUnicode_DecodeFSDefault( +Py_GetProgramFullPath())); SET_SYS_FROM_STRING(prefix, - PyUnicode_FromString(Py_GetPrefix())); + PyUnicode_DecodeFSDefault(Py_GetPrefix())); SET_SYS_FROM_STRING(exec_prefix, - PyUnicode_FromString(Py_GetExecPrefix())); + PyUnicode_DecodeFSDefault(Py_GetExecPrefix())); SET_SYS_FROM_STRING(maxint, PyInt_FromLong(PyInt_GetMax())); SET_SYS_FROM_STRING(maxsize, ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Guido van Rossum added the comment: If this doesn't cause any problems on other platforms, go for it. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes added the comment: I'm setting the priority to normal. The issue isn't resolved but it's not critical for the next alpha release. By the way what's your ETA for the next alpha, Guido? -- priority: high - normal __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes added the comment: I've checked in part of the patch in r58837. It doesn't solve the problem but at least it prevents Python from seg faulting on Windows. -- keywords: +py3k, rfe priority: - high resolution: - accepted __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes added the comment: Hi Martin! Thomas Wouters said on #python that you have the Windows Fu to fix the problem. Parts of the Python API for file paths, sys.path and os.environ have to be reimplemented using the wide char API. -- nosy: +loewis __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes added the comment:
The patch fixes parts of the problem. At least Python doesn't crash any
more when run from a directory with non ASCII chars. It just fails with
an import error in initstdio().
Added file: http://bugs.python.org/file8633/py3k_more_win_fsencoding.patch
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1342
__Index: Include/unicodeobject.h
===
--- Include/unicodeobject.h (revision 58687)
+++ Include/unicodeobject.h (working copy)
@@ -155,6 +155,7 @@
# define PyUnicode_DecodeCharmap PyUnicodeUCS2_DecodeCharmap
# define PyUnicode_DecodeLatin1 PyUnicodeUCS2_DecodeLatin1
# define PyUnicode_DecodeFSDefault PyUnicodeUCS2_DecodeFSDefault
+# define PyUnicode_DecodeFSDefaultAndSize PyUnicodeUCS2_DecodeFSDefaultAndSize
# define PyUnicode_DecodeRawUnicodeEscape PyUnicodeUCS2_DecodeRawUnicodeEscape
# define PyUnicode_DecodeUTF32 PyUnicodeUCS2_DecodeUTF32
# define PyUnicode_DecodeUTF32Stateful PyUnicodeUCS2_DecodeUTF32Stateful
@@ -247,6 +248,7 @@
# define PyUnicode_DecodeCharmap PyUnicodeUCS4_DecodeCharmap
# define PyUnicode_DecodeLatin1 PyUnicodeUCS4_DecodeLatin1
# define PyUnicode_DecodeFSDefault PyUnicodeUCS4_DecodeFSDefault
+# define PyUnicode_DecodeFSDefaultAndSize PyUnicodeUCS4_DecodeFSDefaultAndSize
# define PyUnicode_DecodeRawUnicodeEscape PyUnicodeUCS4_DecodeRawUnicodeEscape
# define PyUnicode_DecodeUTF32 PyUnicodeUCS4_DecodeUTF32
# define PyUnicode_DecodeUTF32Stateful PyUnicodeUCS4_DecodeUTF32Stateful
@@ -657,6 +659,12 @@
const char *s /* encoded string */
);
+PyAPI_FUNC(PyObject*) PyUnicode_DecodeFSDefaultAndSize(
+const char *s, /* encoded string */
+Py_ssize_t size /* size */
+);
+
+
/* Return a char* holding the UTF-8 encoded value of the
Unicode object.
Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 58687)
+++ Objects/unicodeobject.c (working copy)
@@ -1263,10 +1263,14 @@
}
PyObject*
-PyUnicode_DecodeFSDefault(const char *s)
-{
+PyUnicode_DecodeFSDefault(const char *s) {
Py_ssize_t size = (Py_ssize_t)strlen(s);
+return PyUnicode_DecodeFSDefaultAndSize(s, size);
+}
+PyObject*
+PyUnicode_DecodeFSDefaultAndSize(const char *s, Py_ssize_t size)
+{
/* During the early bootstrapping process, Py_FileSystemDefaultEncoding
can be undefined. If it is case, decode using UTF-8. The following assumes
that Py_FileSystemDefaultEncoding is set to a built-in encoding during the
@@ -1274,11 +1278,11 @@
*/
if (Py_FileSystemDefaultEncoding) {
#if defined(MS_WINDOWS) defined(HAVE_USABLE_WCHAR_T)
-if (strcmp(Py_FileSystemDefaultEncoding, mbcs)) {
+if (strcmp(Py_FileSystemDefaultEncoding, mbcs) == 0) {
return PyUnicode_DecodeMBCS(s, size, replace);
}
#elif defined(__APPLE__)
-if (strcmp(Py_FileSystemDefaultEncoding, utf-8)) {
+if (strcmp(Py_FileSystemDefaultEncoding, utf-8) == 0) {
return PyUnicode_DecodeUTF8(s, size, replace);
}
#endif
Index: Python/sysmodule.c
===
--- Python/sysmodule.c (revision 58687)
+++ Python/sysmodule.c (working copy)
@@ -1081,11 +1081,11 @@
SET_SYS_FROM_STRING(platform,
PyUnicode_FromString(Py_GetPlatform()));
SET_SYS_FROM_STRING(executable,
- PyUnicode_FromString(Py_GetProgramFullPath()));
+ PyUnicode_DecodeFSDefault(Py_GetProgramFullPath()));
SET_SYS_FROM_STRING(prefix,
- PyUnicode_FromString(Py_GetPrefix()));
+ PyUnicode_DecodeFSDefault(Py_GetPrefix()));
SET_SYS_FROM_STRING(exec_prefix,
- PyUnicode_FromString(Py_GetExecPrefix()));
+ PyUnicode_DecodeFSDefault(Py_GetExecPrefix()));
SET_SYS_FROM_STRING(maxint,
PyInt_FromLong(PyInt_GetMax()));
SET_SYS_FROM_STRING(maxsize,
@@ -1150,7 +1150,7 @@
p = strchr(path, delim);
if (p == NULL)
p = strchr(path, '\0'); /* End of string */
- w = PyUnicode_FromStringAndSize(path, (Py_ssize_t) (p - path));
+ w = PyUnicode_DecodeFSDefaultAndSize(path, (Py_ssize_t) (p - path));
if (w == NULL) {
Py_DECREF(v);
return NULL;
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
Christian Heimes added the comment:
I've added a fprintf(stderr, %s, path) to makepathobject(). I suspect
that PC/getpathp.c doesn't handle non ASCII chars correctly. It's using
char instead of w_char all over the place. Could that be related to the
issue, Neal?
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
c:\testäöü\PCBuild8\win32releaseset PYTHONPATH=c:\testäöü\Lib
c:\testäöü\PCBuild8\win32releasepython
c:\testõ÷³\Lib;c:\testõ÷³\PCBuild8\win32release\python30.zip;c:\testõ÷³\DLLs;c:\
testõ÷³\lib;c:\testõ÷³\lib\plat-win;c:\testõ÷³\lib\lib-tk;c:\testõ÷³\PCBuild8\wi
n32releaseFatal Python error: Py_Initialize: can't initialize sys
standard strea
ms
object : ImportError('No module named encodings.utf_8',)
type: ImportError
refcount: 4
address : 00A43540
lost sys.stderr
This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
--
nosy: +nnorwitz
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1342
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1342] Crash on Windows if Python runs from a directory with umlauts
New submission from Christian Heimes: Python 3.0 doesn't run from a directory with umlauts and possible other non ASCII chars. I renamed my development folder from C:\dev\ to c:\test äöüß name\. Python crashes after a few moments before it can reach its shell. python30.dll!PyErr_SetObject(_object * exception=0x1e1b9888, _object * value=0x00a0b8a0) Zeile 56 + 0xb Bytes C python30.dll!PyErr_SetString(_object * exception=0x1e1b9888, const char * string=0x1e18c358) Zeile 77 + 0xd Bytes C python30.dll!find_module(char * fullname=0x0021fcc0, char * subname=0x, _object * path=0x, char * buf=0x0021fb70, unsigned int buflen=257, _iobuf * * p_fp=0x0021fb64, _object * * p_loader=0x0021fb68) Zeile 1228 + 0x10 Bytes C python30.dll!import_submodule(_object * mod=0x1e1c6a88, char * subname=0x0021fcc0, char * fullname=0x) Zeile 2313 + 0x27 Bytes C python30.dll!load_next(_object * mod=0x1e1c6a88, _object * altmod=0x1e1c6a88, char * * p_name=0x, char * buf=0x0021fcc0, int * p_buflen=0x0021fcbc) Zeile 2127 + 0x15 Bytes C python30.dll!import_module_level(char * name=0x, _object * globals=0x, _object * locals=0x1e069ec3, _object * fromlist=0x, int level=0) Zeile 1908 + 0x1a Bytes C python30.dll!PyImport_ImportModuleLevel(char * name=0x1e184b04, _object * globals=0x, _object * locals=0x, _object * fromlist=0x, int level=0) Zeile 1979 + 0x18 Bytes C python30.dll!_PyCodecRegistry_Init() Zeile 841 + 0x12 BytesC python30.dll!PyCodec_LookupError(const char * name=0x) Zeile 436 + 0xc Bytes C python30.dll!unicode_decode_call_errorhandler(const char * errors=0x, _object * * errorHandler=0x0009, const char * encoding=0x1e1979ec, const char * reason=0x, const char * * input=0x0021fe80, const char * * inend=0x0021fe84, int * startinpos=0x0021fe6c, int * endinpos=0x0021fe68, _object * * exceptionObject=0x, const char * * inptr=0x0021fe90, _object * * output=0x0021fe70, int * outpos=0x0021fe88, unsigned short * * outptr=0x0021fe74) Zeile 1384 + 0xa Bytes C python30.dll!PyUnicodeUCS2_DecodeUTF8Stateful(const char * s=0x1e1dd010, int size=48, const char * errors=0x, int * consumed=0x) Zeile 1967 + 0x47 Bytes C python30.dll!PyUnicodeUCS2_FromStringAndSize(const char * u=0x1e1dd008, int size=48) Zeile 464 + 0xb Bytes C python30.dll!PyUnicodeUCS2_FromString(const char * u=0x1e1dd008) Zeile 482 + 0x7 Bytes C python30.dll!_PySys_Init() Zeile 1084 + 0xb Bytes C python30.dll!Py_InitializeEx(int install_sigs=1) Zeile 220 C python30.dll!Py_Initialize() Zeile 292 + 0x7 Bytes C python30.dll!Py_Main(int argc=2, char * * argv=0x0001) Zeile 432 C python.exe!mainCRTStartup() Zeile 398 + 0xe Bytes C kernel32.dll!7c816fd7() -- components: Interpreter Core messages: 56841 nosy: tiran severity: normal status: open title: Crash on Windows if Python runs from a directory with umlauts type: crash versions: Python 3.0 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1342 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
