[issue13377] test_codecs "Segmentation fault" on Windows

[Vinay Sajip]

New submission from Vinay Sajip :

test_codecs has started failing on Windows: I can't tell if the problem is in 
the tests.

== CPython 3.3.0a0 (default, Nov 8 2011, 22:34:51) [MSC v.1500 32 bit (Intel)]
==   Windows-7-6.1.7600 little-endian
==   c:\Users\Vinay\Projects\cpython\build\test_python_2612
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, 
dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, 
verbose=0, bytes_warning=2, quiet=0)
[1/1] test_codecs
test_badbom (test.test_codecs.UTF32Test) ... ok
test_bug1098990_a (test.test_codecs.UTF32Test) ... ok
test_bug1098990_b (test.test_codecs.UTF32Test) ... ok
test_bug1175396 (test.test_codecs.UTF32Test) ... ok
test_decoder_state (test.test_codecs.UTF32Test) ... ok
test_errors (test.test_codecs.UTF32Test) ... ok
test_handlers (test.test_codecs.UTF32Test) ... ok
test_issue8941 (test.test_codecs.UTF32Test) ... ok
test_only_one_bom (test.test_codecs.UTF32Test) ... ok
test_partial (test.test_codecs.UTF32Test) ... ok
test_readline (test.test_codecs.UTF32Test) ... ok
test_readlinequeue (test.test_codecs.UTF32Test) ... ok
test_bug1098990_a (test.test_codecs.UTF32LETest) ... ok
test_bug1098990_b (test.test_codecs.UTF32LETest) ... ok
test_bug1175396 (test.test_codecs.UTF32LETest) ... ok
test_errors (test.test_codecs.UTF32LETest) ... ok
test_issue8941 (test.test_codecs.UTF32LETest) ... ok
test_partial (test.test_codecs.UTF32LETest) ... ok
test_readline (test.test_codecs.UTF32LETest) ... ok
test_readlinequeue (test.test_codecs.UTF32LETest) ... ok
test_simple (test.test_codecs.UTF32LETest) ... ok
test_bug1098990_a (test.test_codecs.UTF32BETest) ... ok
test_bug1098990_b (test.test_codecs.UTF32BETest) ... ok
test_bug1175396 (test.test_codecs.UTF32BETest) ... ok
test_errors (test.test_codecs.UTF32BETest) ... ok
test_issue8941 (test.test_codecs.UTF32BETest) ... ok
test_partial (test.test_codecs.UTF32BETest) ... ok
test_readline (test.test_codecs.UTF32BETest) ... ok
test_readlinequeue (test.test_codecs.UTF32BETest) ... ok
test_simple (test.test_codecs.UTF32BETest) ... ok
test_badbom (test.test_codecs.UTF16Test) ... ok
test_bug1098990_a (test.test_codecs.UTF16Test) ... ok
test_bug1098990_b (test.test_codecs.UTF16Test) ... ok
test_bug1175396 (test.test_codecs.UTF16Test) ... ok
test_bug691291 (test.test_codecs.UTF16Test) ... ok
test_decoder_state (test.test_codecs.UTF16Test) ... ok
test_errors (test.test_codecs.UTF16Test) ... ok
test_handlers (test.test_codecs.UTF16Test) ... ok
test_only_one_bom (test.test_codecs.UTF16Test) ... ok
test_partial (test.test_codecs.UTF16Test) ... ok
test_readline (test.test_codecs.UTF16Test) ... ok
test_readlinequeue (test.test_codecs.UTF16Test) ... ok
test_bug1098990_a (test.test_codecs.UTF16LETest) ... ok
test_bug1098990_b (test.test_codecs.UTF16LETest) ... ok
test_bug1175396 (test.test_codecs.UTF16LETest) ... ok
test_errors (test.test_codecs.UTF16LETest) ... ok
test_nonbmp (test.test_codecs.UTF16LETest) ... ok
test_partial (test.test_codecs.UTF16LETest) ... ok
test_readline (test.test_codecs.UTF16LETest) ... ok
test_readlinequeue (test.test_codecs.UTF16LETest) ... ok
test_bug1098990_a (test.test_codecs.UTF16BETest) ... ok
test_bug1098990_b (test.test_codecs.UTF16BETest) ... ok
test_bug1175396 (test.test_codecs.UTF16BETest) ... ok
test_errors (test.test_codecs.UTF16BETest) ... ok
test_nonbmp (test.test_codecs.UTF16BETest) ... ok
test_partial (test.test_codecs.UTF16BETest) ... ok
test_readline (test.test_codecs.UTF16BETest) ... ok
test_readlinequeue (test.test_codecs.UTF16BETest) ... ok
test_bug1098990_a (test.test_codecs.UTF8Test) ... ok
test_bug1098990_b (test.test_codecs.UTF8Test) ... ok
test_bug1175396 (test.test_codecs.UTF8Test) ... ok
test_decoder_state (test.test_codecs.UTF8Test) ... ok
test_lone_surrogates (test.test_codecs.UTF8Test) ... ok
test_partial (test.test_codecs.UTF8Test) ... ok
test_readline (test.test_codecs.UTF8Test) ... ok
test_readlinequeue (test.test_codecs.UTF8Test) ... ok
test_surrogatepass_handler (test.test_codecs.UTF8Test) ... ok
test_bom (test.test_codecs.UTF8SigTest) ... ok
test_bug1098990_a (test.test_codecs.UTF8SigTest) ... ok
test_bug1098990_b (test.test_codecs.UTF8SigTest) ... ok
test_bug1175396 (test.test_codecs.UTF8SigTest) ... ok
test_bug1601501 (test.test_codecs.UTF8SigTest) ... ok
test_partial (test.test_codecs.UTF8SigTest) ... ok
test_readline (test.test_codecs.UTF8SigTest) ... ok
test_readlinequeue (test.test_codecs.UTF8SigTest) ... ok
test_stream_bare (test.test_codecs.UTF8SigTest) ... ok
test_stream_bom (test.test_codecs.UTF8SigTest) ... ok
test_bug1098990_a (test.test_codecs.CP65001Test) ... ok
test_bug1098990_b (test.test_codecs.CP65001Test) ... ok
test_bug1175396 (test.test_codecs.CP65001Test) ... ok
test_decode (test.test_codecs.CP65001Test) ... Fatal Python error: Segmentation 
fault

Current thread 0x0a20:
  File "c:\Users\Vinay\Projects\cpython\lib\linecache.py", line 81 in 
updatecache
  File "c:\Users\Vinay\Projects\cpython\lib\linecache.py", 

[issue13377] test_codecs "Segmentation fault" on Windows

[Vinay Sajip]

Vinay Sajip  added the comment:

FWIW, stack trace from VS 2008:


>   python33.dll!_PyLong_New(int size=0x0002)  Line 145 C
python33.dll!PyLong_FromLong(long ival=0x81b6)  Line 210 + 0x7 
bytesC
python33.dll!_pystat_fromstructstat(win32_stat * st=0x)  Line 
1726 + 0xc bytes  C
python33.dll!posix_do_stat(_object * self=0x1e047e3d, _object * 
args=0x, char * format=0x1e12ffc0, int (const char *, win32_stat *)* 
statfunc=0x1e047e00, char * wformat=0x, int (const wchar_t *, 
win32_stat *)* wstatfunc=0x1e043d60)  Line 1846 + 0x9 bytes  C
python33.dll!posix_stat(_object * self=0x013aa148, _object * 
args=0x02292c50)  Line 3335 + 0x1d bytes   C
python33.dll!PyCFunction_Call(_object * func=0x013aa328, _object * 
arg=0x02292c50, _object * kw=0x)  Line 81 + 0x8 bytesC
python33.dll!call_function(_object * * * pp_stack=0x0021e2fc, int 
oparg=0x)  Line 4001 + 0x9a bytes C
python33.dll!PyEval_EvalFrameEx(_frame * f=0x018c93f8, int 
throwflag=0x)  Line 2628 C
python33.dll!PyEval_EvalCodeEx(_object * _co=0x01b9cac0, _object * 
globals=0x018c93f8, _object * locals=0x, _object * * args=0x01e5a9b4, 
int argcount=0x0002, _object * * kws=0x01e5a9bc, int kwcount=0x, 
_object * * defs=0x017f665c, int defcount=0x0001, _object * 
kwdefs=0x, _object * closure=0x)  Line 3376 + 0xc bytes   C
python33.dll!fast_function(_object * func=0x, _object * * * 
pp_stack=0x0021e418, int n=0x0002, int na=0x0002, int nk=0x)  
Line 4102 + 0x39 bytesC
python33.dll!call_function(_object * * * pp_stack=0x0021e418, int 
oparg=0x)  Line 4022 + 0x10 bytes C
python33.dll!PyEval_EvalFrameEx(_frame * f=0x01e5a870, int 
throwflag=0x)  Line 2628 C
python33.dll!PyEval_EvalCodeEx(_object * _co=0x01b9c980, _object * 
globals=0x01e5a870, _object * locals=0x, _object * * args=0x01e5e45c, 
int argcount=0x0002, _object * * kws=0x01e5e464, int kwcount=0x, 
_object * * defs=0x01b9283c, int defcount=0x0001, _object * 
kwdefs=0x, _object * closure=0x)  Line 3376 + 0xc bytes   C
python33.dll!fast_function(_object * func=0x, _object * * * 
pp_stack=0x0021e534, int n=0x0002, int na=0x0002, int nk=0x)  
Line 4102 + 0x39 bytesC
python33.dll!call_function(_object * * * pp_stack=0x0021e534, int 
oparg=0x)  Line 4022 + 0x10 bytes C
python33.dll!PyEval_EvalFrameEx(_frame * f=0x01e5e310, int 
throwflag=0x)  Line 2628 C
python33.dll!PyEval_EvalCodeEx(_object * _co=0x01b9c8e0, _object * 
globals=0x01e5e310, _object * locals=0x, _object * * args=0x01e5b7d4, 
int argcount=0x0003, _object * * kws=0x01e5b7e0, int kwcount=0x, 
_object * * defs=0x01b9287c, int defcount=0x0001, _object * 
kwdefs=0x, _object * closure=0x)  Line 3376 + 0xc bytes   C
python33.dll!fast_function(_object * func=0x, _object * * * 
pp_stack=0x0021e650, int n=0x0003, int na=0x0003, int nk=0x)  
Line 4102 + 0x39 bytesC
python33.dll!call_function(_object * * * pp_stack=0x0021e650, int 
oparg=0x)  Line 4022 + 0x10 bytes C
python33.dll!PyEval_EvalFrameEx(_frame * f=0x01e5b670, int 
throwflag=0x)  Line 2628 C
python33.dll!PyEval_EvalCodeEx(_object * _co=0x01735610, _object * 
globals=0x01e5b670, _object * locals=0x, _object * * args=0x01e48cb8, 
int argcount=0x0002, _object * * kws=0x01e48cc0, int kwcount=0x, 
_object * * defs=0x0172623c, int defcount=0x0001, _object * 
kwdefs=0x, _object * closure=0x)  Line 3376 + 0xc bytes   C
python33.dll!fast_function(_object * func=0x, _object * * * 
pp_stack=0x0021e76c, int n=0x0002, int na=0x0002, int nk=0x)  
Line 4102 + 0x39 bytesC
python33.dll!call_function(_object * * * pp_stack=0x0021e76c, int 
oparg=0x)  Line 4022 + 0x10 bytes C
python33.dll!PyEval_EvalFrameEx(_frame * f=0x01e48b70, int 
throwflag=0x)  Line 2628 C
python33.dll!PyEval_EvalCodeEx(_object * _co=0x017355c0, _object * 
globals=0x01e48b70, _object * locals=0x, _object * * args=0x01e3b790, 
int argcount=0x0002, _object * * kws=0x01e3b798, int kwcount=0x, 
_object * * defs=0x0172621c, int defcount=0x0001, _object * 
kwdefs=0x, _object * closure=0x)  Line 3376 + 0xc bytes   C
python33.dll!fast_function(_object * func=0x, _object * * * 
pp_stack=0x0021e888, int n=0x0002, int na=0x0002, int nk=0x)  
Line 4102 + 0x39 bytesC
python33.dll!call_function(_object * * * pp_stack=0x0021e888, int 
oparg=0x)  Line 4022 + 0x10 bytes C
python33.dll!PyEval_EvalFrameEx(_fram

[issue13377] test_codecs "Segmentation fault" on Windows

[STINNER Victor]

STINNER Victor  added the comment:

I may be related to recent changes on codecs. My following commit fixed a 
test_codecs crash, but there is maybe another bug.

changeset:   73466:9042d71962d6
user:Victor Stinner 
date:Wed Nov 09 00:02:18 2011 +0100
files:   Objects/unicodeobject.c
description:
Fix the code page decoder

 * unicode_decode_call_errorhandler() now supports the PyUnicode_WCHAR_KIND
   kind
 * unicode_decode_call_errorhandler() calls copy_characters() instead of
   PyUnicode_CopyCharacters()

--
nosy: +haypo, loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13377] test_codecs "Segmentation fault" on Windows

[STINNER Victor]

STINNER Victor  added the comment:

The test doesn't crash on our Windows buildbots. Please try to update. If the 
problem does still exist, reopen the issue.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13377] test_codecs "Segmentation fault" on Windows

[Martin v . Löwis]

Martin v. Löwis  added the comment:

Why was the change necessary? unicode_decode_call_errorhandler shouldn't be 
called with WSTR strings (i.e. it is only intended to support ready strings).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13377] test_codecs "Segmentation fault" on Windows

[STINNER Victor]

STINNER Victor  added the comment:

> Why was the change necessary?

First because unicode_decode_call_errorhandler() was called with the wrong 
argument count:

1.90 -v, &outpos, &out))
1.91 +v, &outpos))

decode_code_page_errors() calls MultiByteToWideChar() which expects a wchar_t* 
buffer.

Compute the maximum character requires to decode the full string. I prefer to 
work on a wchar_t* string to only decode surrogate pairs and computer the 
maximum character once.

decode_code_page_errors() can also be called on a substring of a longer string, 
decode_code_page_strict() can be called for the other part.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com