[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Antoine Pitrou [EMAIL PROTECTED] added the comment: Hirokazu, does replacing the following line (rather than changing the type of the `ch` variable): ch = *s; with ch = (unsigned char) *s; fix the crash as well? -- keywords: +patch nosy: +pitrou ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Hirokazu Yamamoto [EMAIL PROTECTED] added the comment:
With this patch? Yes, it fixed crash.
Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 65223)
+++ Objects/unicodeobject.c (working copy)
@@ -1523,7 +1523,7 @@
while (s e) {
Py_UNICODE ch;
restart:
-ch = *s;
+ch = (unsigned char)*s;
if (inShift) {
if ((ch == '-') || !B64CHAR(ch)) {
'+\xc1'.decode(utf7)
Traceback (most recent call last):
File stdin, line 1, in module
File e:\python-dev\trunk\lib\encodings\utf_7.py, line 12, in decode
return codecs.utf_7_decode(input, errors, True)
UnicodeDecodeError: 'utf7' codec can't decode bytes in position 0-1:
unexpected
# But I don't know whether this behavior is right or not
I confirmed test_unicode, test_codecs, test_codeccallbacks passed.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2242
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: VS8 and VS9 are immune to the crash, even if the exception message differ between release and debug builds. VC6 crashes, and the proposed patch fixes the problem there as well. -- nosy: +amaury.forgeotdarc ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Antoine Pitrou [EMAIL PROTECTED] added the comment: Selon Hirokazu Yamamoto [EMAIL PROTECTED]: With this patch? Yes, it fixed crash. Thanks! # But I don't know whether this behavior is right or not As the name implies, utf7 is a 7-bit coding of Unicode... bytes = 0x80 must raise an exception. The error message could be better though. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Antoine Pitrou [EMAIL PROTECTED] added the comment: Should be fixed in r65227. Please reopen if there's still a problem. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Antoine Pitrou [EMAIL PROTECTED] added the comment: On second thought, perhaps it should also be backported to 2.5, so I'm leaving the bug open. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Changes by Antoine Pitrou [EMAIL PROTECTED]: -- resolution: - accepted versions: -Python 2.6, Python 3.0 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Antoine Pitrou [EMAIL PROTECTED] added the comment: I've committed the fix for 2.5 in r65234, can somebody try it out with the failing MSVC version? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: I confirm that r65234 for 2.5 corrects the crash. (Windows XP, Visual Studio 6) ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Antoine Pitrou [EMAIL PROTECTED] added the comment: Thanks Amaury! -- resolution: accepted - fixed status: open - closed ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Chris Palmer added the comment: You could also fix the problem by using iswalnum function instead of isalnum. Sorry I didn't mention this in the original report. http://msdn2.microsoft.com/en-us/library/k84c0490(VS.71).aspx __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
New submission from Chris Palmer:
When decoding some data as UTF-7 with the optional ignore argument,
Python (I am using 2.5.2) crashes. This happens only on Windows Vista (I
also tried Py 2.5.1 on Windows XP, Ubuntu 7, and FreeBSD 6). To
reproduce, set WinDbg as your post-mortem debugger and run this code:
import os
while True:
a = os.urandom(16).decode(utf7, ignore)
In WinDbg, you will see that Python died in isalnum with a bad pointer
dereference:
(f64.13b0): Access violation - code c005 (!!! second chance !!!)
eax=7c39a550 ebx=018e6837 ecx=ffe3 edx=0003 esi=018edd66
edi=ffe3
eip=7c373977 esp=0021fc40 ebp=ffe3 iopl=0 nv up ei pl zr na
pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=
efl=00010246
*** ERROR: Symbol file could not be found. Defaulted to export symbols
for C:\Windows\system32\MSVCR71.dll -
MSVCR71!isalnum+0x35:
7c373977 0fb70448movzx eax,word ptr [eax+ecx*2]
ds:0023:7c3ba516=
0:000 kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be
wrong.
0021fc3c 1e0dd81e ffe3 00ff1030 012e MSVCR71!isalnum+0x35
python25!PyUnicode_DecodeUTF7+0x10e
It seems that a sanity check present in other Windows versions is
missing in Vista. The simplest possible test program:
#include stdafx.h
#include ctype.h
int _tmain(int argc, _TCHAR* argv[])
{
isalnum(0xff8b);
return 0;
}
causes Visual Studio 2005 to raise a debug assertion failure warning. I
guess that the assert is missing in the release build, and Python can be
tricked into providing the unsafe input to isalnum.
--
components: Interpreter Core
messages: 63303
nosy: cpalmer
severity: normal
status: open
title: Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
type: crash
versions: Python 2.5
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2242
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Hirokazu Yamamoto added the comment:
I reproduced this bug with VC6 + Win2000SP4 + following code.
'+\xc1'.decode(utf7, ignore)
and this simple patch prevented crash.
Index: Objects/unicodeobject.c
===
--- Objects/unicodeobject.c (revision 61262)
+++ Objects/unicodeobject.c (working copy)
@@ -1506,7 +1506,7 @@
e = s + size;
while (s e) {
-Py_UNICODE ch;
+char ch;
restart:
ch = *s;
Probably this is due to integer conversion, but I didn't look at logic
so much.
--
nosy: +ocean-city
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2242
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Hirokazu Yamamoto added the comment: One more thing. ignore is not needed. '+\xc1'.decode(utf7) crashed my interpreter. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2242] Decoding UTF-7 with ignore warnings crashes Python on Windows Vista
Changes by Georg Brandl: -- priority: - urgent severity: normal - urgent versions: +Python 2.6, Python 3.0 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2242 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
