[issue24467] bytearray pop and remove Buffer Over-read
DmitryJ added the comment: If this is the case, then issue24462 should be fixed by this patch as well. I'm sorry about missing the root cause here. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24467] bytearray pop and remove Buffer Over-read
Changes by DmitryJ ga...@tut.by: Added file: http://bugs.python.org/file39781/issue24467-3.2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24467] bytearray pop and remove Buffer Over-read
DmitryJ added the comment: Attached is a patch that fixes the reported issue. Since there are no visible side effects in Python, I could not write a test for this. -- keywords: +patch Added file: http://bugs.python.org/file39780/issue24467-2.7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24467] bytearray pop and remove Buffer Over-read
Changes by DmitryJ ga...@tut.by: Added file: http://bugs.python.org/file39783/issue24467-3.4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24467] bytearray pop and remove Buffer Over-read
Changes by DmitryJ ga...@tut.by: Added file: http://bugs.python.org/file39784/issue24467-3.5.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24467] bytearray pop and remove Buffer Over-read
Changes by DmitryJ ga...@tut.by: Added file: http://bugs.python.org/file39782/issue24467-3.3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24467] bytearray pop and remove Buffer Over-read
DmitryJ added the comment: Offending code in 2.7: https://hg.python.org/cpython/file/20c9290a5de4/Objects/bytearrayobject.c#l2381 https://hg.python.org/cpython/file/20c9290a5de4/Objects/bytearrayobject.c#l2412 Let n = 16, where = 0; memmove() then attempts to copy (n - where) = 16 bytes where it should have copied 15, since we drop one. This appears to be a typical case of off-by-one. Changing (n - where) to (n - where - 1) should fix the issue. This underfows when (where + 1) n, but this case is guarded against in bytearray_pop() and cannot occur in bytearray_remove(). The exact same memmove() invocation code is found in all 3.x branches as well. -- nosy: +dev_zzo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24467 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: Attached please find a patch against the 2.7 branch. CPython built with the patch passes the tests from the test suite. Unfortunately, as there is not much control over memory allocation, there is no 100% reliable test case that would allow for reproducing the reported issue. Some notes on the patch. I have studied possible ways of fixing the issue narrowing them to two options; the approaches considered were: a. Patch bytearray methods so they use stringlib's functions with respect to the corner case of out-of-bounds access on m = n. b. Patch fastsearch() avoiding the out-of-bounds access on m = n completely. Of these two, approach a is less invasive as changes, in theory, would be contained in bytearray() code only and should not affect any other code. Approach b fixes all possible cases, but affects other code not related to bytearray. Upon closer studying of both bytearray and stringlib code, I discovered that it might be impossible to patch bytearray code only as stringlib contains a few methods that make use of the affected fastsearch() function, see e.g. stringlib_partition() as used in bytearray_partition(). If the approach of fixing bytearray specific code only would be chosen, I have to incorporate at least some of code following the fastsearch() call in stringlib_partition(). Similar considerations apply to other bytearray methods that make use of stringlib; the amount of code duplication varies. The end result is, I chose to patch fastsearch() instead. Performance wise, the change incurs a small penalty due to one extra branch when m != n and brings considerable gain in (potentially rare) case when m = n. I would appreciate if someone could test and review the patch. NB. I stand corrected for the comment in msg245457 -- there is a note I missed in the C code. My sincere apologies to the author. -- keywords: +patch Added file: http://bugs.python.org/file39772/issue24462-2.7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24481] hotshot pack_string Heap Buffer Overflow
Changes by DmitryJ ga...@tut.by: -- nosy: +dev_zzo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24481 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: I am preparing a patch for this issue, then. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: Quick analysis tells this can be attributed to the following code (in 2.7): https://hg.python.org/cpython/file/a8e24d776e99/Objects/stringlib/fastsearch.h#l110 https://hg.python.org/cpython/file/a8e24d776e99/Objects/stringlib/fastsearch.h#l116 Suppose i = 0, then s[i+m] causes OOB access when m=n. Note only one iteration is possible in case of m=n due to loop condition of i = (w = n-m = 0). Theoretically, one can try disclosing one adjacent byte, but more likely results are nothing (or potentially invalid match result) or a potential crash in an unlucky case of s[m] hitting an unmapped page. The same code lives in 3.2 (and likely any prior 3.x release), and 3.3 seems to be affected as well. 3.4 code has a modified version, but has the same problem (ss = s + m - 1; if (!STRINGLIB_BLOOM(mask, ss[i+1])) ...). -- nosy: +dev_zzo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: From the author's page at http://effbot.org/zone/stringlib.htm Note that the above Python code may access s[n], which would result in an IndexError exception. For the CPython implementation, this is not really a problem, since CPython adds trailing NULL entries to both 8-bit and Unicode strings. Apparently, this flaw was known to the author, but was not documented in C code. A possible quick-and-dirty solution is to treat m=n as a special case and resort to memcmp() or somesuch as there is no actual need to perform multiple match tries. This should fix things for bytearray and str in case str's implementation changes from appending a trailing NUL. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
