[issue20024] Py_BuildValue() can call Python code with an exception set

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Committed patch fixed the issue only for tuples, but not for lists or dicts. 
See more general patch in issue26168.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM.

I have searched the sources and have not found any use of Py_BuildValue() which 
can be used for test. Py_BuildValue() can fail only due to memory error. It can 
also can fail with non-UTF8 string, but all uses of Py_BuildValue() with a 
string in a tuple proceeds only ASCII strings.

--
stage: test needed - commit review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[STINNER Victor]

STINNER Victor added the comment:

Thanks for the review Serhiy.

--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[Roundup Robot]

Roundup Robot added the comment:

New changeset 29b4eb47f65e by Victor Stinner in branch 'default':
Issue #20024: Py_BuildValue() now saves/restores the current exception before
http://hg.python.org/cpython/rev/29b4eb47f65e

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[STINNER Victor]

STINNER Victor added the comment:

parsermodule.patch was applied as 91cb83f895cf (Python 3.3) and cd952e92c180 
(3.4).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


Added file: http://bugs.python.org/file33209/parsermodule.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[STINNER Victor]

New submission from STINNER Victor:

In Python 3.4, an assertion now checks that no exception is set when arbitrary 
Python code is called. Python code may suppress the exception.

When Py_BuildValue() is used to build a tuple and an error when the creation of 
an item failed, the function may execute indirectly Python code with an 
exception set.

Attached patch works around the issue by storing the current exception in a 
variable and then restore it. It changes the behaviour if more than one 
exception is raised: at the end, its the first exception which is passed to the 
caller. I prefer to get the first exception instead of the following 
exceptions, because following exceptions may be caused by the first exception. 
See for example the parser bug below for an example of a second exception 
caused by the first exception.

--

Example with the parser module:

PyObject *err = Py_BuildValue(os, elem,
  Illegal node construct.);
PyErr_SetObject(parser_error, err);

The o is not a valid format and so a SystemError is raised, but then the 
UTF-8 decoder tries to raise a second exception, an UnicodeDecodeError, because 
the decoders is called with an invalid bytes string (elem variable, instead of 
the Illegal node construct. string, because o format didn't increment the 
argument pointer). The bug is obvious in the parser module, but the problem is 
more general than that.

Gdb traceback of the parser bug:

python: Objects/typeobject.c:741: type_call: Assertion `!PyErr_Occurred()' 
failed.

Program received signal SIGABRT, Aborted.
0x00301c0359e9 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-19.fc19.x86_64
(gdb) where
#0  0x00301c0359e9 in raise () from /lib64/libc.so.6
#1  0x00301c0370f8 in abort () from /lib64/libc.so.6
#2  0x00301c02e956 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00301c02ea02 in __assert_fail () from /lib64/libc.so.6
#4  0x004dae18 in type_call (type=0x906780 _PyExc_UnicodeDecodeError, 
args=('utf-8', b'\xc8\x87\xe0\xf7\xff\x7f', 2, 3, 'invalid continuation byte'), 
kwds=0x0)
at Objects/typeobject.c:741
#5  0x0045fb75 in PyObject_Call (func=type at remote 0x906780, 
arg=('utf-8', b'\xc8\x87\xe0\xf7\xff\x7f', 2, 3, 'invalid continuation byte'), 
kw=0x0)
at Objects/abstract.c:2067
#6  0x0045fd0f in call_function_tail (callable=type at remote 
0x906780, args=('utf-8', b'\xc8\x87\xe0\xf7\xff\x7f', 2, 3, 'invalid 
continuation byte'))
at Objects/abstract.c:2104
#7  0x0045ff94 in _PyObject_CallFunction_SizeT (callable=type at 
remote 0x906780, format=0x669e19 sy#nns) at Objects/abstract.c:2150
#8  0x0048643d in PyUnicodeDecodeError_Create (encoding=0x67e312 
utf-8, object=0x76ddcf40 \310\207\340\367\377\177, length=6, start=2, 
end=3, 
reason=0x67f1c1 invalid continuation byte) at Objects/exceptions.c:1960
#9  0x0050fe7c in make_decode_exception 
(exceptionObject=0x7fff9090, encoding=0x67e312 utf-8, 
input=0x76ddcf40 \310\207\340\367\377\177, length=6, 
startpos=2, endpos=3, reason=0x67f1c1 invalid continuation byte) at 
Objects/unicodeobject.c:4009
#10 0x00510015 in unicode_decode_call_errorhandler_writer (errors=0x0, 
errorHandler=0x7fff9098, encoding=0x67e312 utf-8, 
reason=0x67f1c1 invalid continuation byte, input=0x7fff90b8, 
inend=0x7fff90b0, startinpos=0x7fff90a8, endinpos=0x7fff90a0, 
exceptionObject=0x7fff9090, inptr=0x7fff9088, 
writer=0x7fff90c0) at Objects/unicodeobject.c:4157
#11 0x005163a8 in PyUnicode_DecodeUTF8Stateful (s=0x76ddcf42 
\340\367\377\177, size=6, errors=0x0, consumed=0x0) at 
Objects/unicodeobject.c:4798
#12 0x00506198 in PyUnicode_FromStringAndSize (u=0x76ddcf40 
\310\207\340\367\377\177, size=6) at Objects/unicodeobject.c:1840
#13 0x005da9d2 in do_mkvalue (p_format=0x7fff92e0, 
p_va=0x7fff92c8, flags=0) at Python/modsupport.c:319
#14 0x005d9e50 in do_mktuple (p_format=0x7fff92e0, 
p_va=0x7fff92c8, endchar=0, n=2, flags=0) at Python/modsupport.c:164
#15 0x005db067 in va_build_value (format=0x7750f34b os, 
va=0x7fff9310, flags=0) at Python/modsupport.c:455
#16 0x005dae8c in Py_BuildValue (format=0x7750f34b os) at 
Python/modsupport.c:410
#17 0x7750641c in build_node_children (tuple=(1467, -415088696226, 
183475025091, 3, -85006798, 915338703290779849), root=0x77f7f6b8, 
line_num=0x7fff95cc)
at /home/haypo/prog/python/default/Modules/parsermodule.c:788

--
files: py_buildvalue_failed.patch
keywords: patch
messages: 206602
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Py_BuildValue() can call Python code with an exception set
versions: Python 3.4
Added file: http://bugs.python.org/file33208/py_buildvalue_failed.patch

[issue20024] Py_BuildValue() can call Python code with an exception set

[STINNER Victor]

STINNER Victor added the comment:

parsermodule.patch: fix usage of Py_BuildValue() in the parser module.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20024] Py_BuildValue() can call Python code with an exception set

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
stage:  - test needed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20024
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com