[issue24462] bytearray.find Buffer Over-read
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
Serhiy Storchaka added the comment: The patch for issue24467 also fixed this issue in different way. In any case thank you Dmitry for your patches. -- components: +Interpreter Core resolution: - duplicate stage: patch review - resolved status: open - closed superseder: - bytearray pop and remove Buffer Over-read ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
Changes by Serhiy Storchaka storch...@gmail.com: -- assignee: - serhiy.storchaka stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: Attached please find a patch against the 2.7 branch. CPython built with the patch passes the tests from the test suite. Unfortunately, as there is not much control over memory allocation, there is no 100% reliable test case that would allow for reproducing the reported issue. Some notes on the patch. I have studied possible ways of fixing the issue narrowing them to two options; the approaches considered were: a. Patch bytearray methods so they use stringlib's functions with respect to the corner case of out-of-bounds access on m = n. b. Patch fastsearch() avoiding the out-of-bounds access on m = n completely. Of these two, approach a is less invasive as changes, in theory, would be contained in bytearray() code only and should not affect any other code. Approach b fixes all possible cases, but affects other code not related to bytearray. Upon closer studying of both bytearray and stringlib code, I discovered that it might be impossible to patch bytearray code only as stringlib contains a few methods that make use of the affected fastsearch() function, see e.g. stringlib_partition() as used in bytearray_partition(). If the approach of fixing bytearray specific code only would be chosen, I have to incorporate at least some of code following the fastsearch() call in stringlib_partition(). Similar considerations apply to other bytearray methods that make use of stringlib; the amount of code duplication varies. The end result is, I chose to patch fastsearch() instead. Performance wise, the change incurs a small penalty due to one extra branch when m != n and brings considerable gain in (potentially rare) case when m = n. I would appreciate if someone could test and review the patch. NB. I stand corrected for the comment in msg245457 -- there is a note I missed in the C code. My sincere apologies to the author. -- keywords: +patch Added file: http://bugs.python.org/file39772/issue24462-2.7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: I am preparing a patch for this issue, then. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
JohnLeitch added the comment: Given my understanding of the issue, the memcmp approach seems like a viable fix. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: Quick analysis tells this can be attributed to the following code (in 2.7): https://hg.python.org/cpython/file/a8e24d776e99/Objects/stringlib/fastsearch.h#l110 https://hg.python.org/cpython/file/a8e24d776e99/Objects/stringlib/fastsearch.h#l116 Suppose i = 0, then s[i+m] causes OOB access when m=n. Note only one iteration is possible in case of m=n due to loop condition of i = (w = n-m = 0). Theoretically, one can try disclosing one adjacent byte, but more likely results are nothing (or potentially invalid match result) or a potential crash in an unlucky case of s[m] hitting an unmapped page. The same code lives in 3.2 (and likely any prior 3.x release), and 3.3 seems to be affected as well. 3.4 code has a modified version, but has the same problem (ss = s + m - 1; if (!STRINGLIB_BLOOM(mask, ss[i+1])) ...). -- nosy: +dev_zzo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
DmitryJ added the comment: From the author's page at http://effbot.org/zone/stringlib.htm Note that the above Python code may access s[n], which would result in an IndexError exception. For the CPython implementation, this is not really a problem, since CPython adds trailing NULL entries to both 8-bit and Unicode strings. Apparently, this flaw was known to the author, but was not documented in C code. A possible quick-and-dirty solution is to treat m=n as a special case and resort to memcmp() or somesuch as there is no actual need to perform multiple match tries. This should fix things for bytearray and str in case str's implementation changes from appending a trailing NUL. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24462] bytearray.find Buffer Over-read
New submission from JohnLeitch: The bytearray.find method suffers from a buffer over-read that can be triggered by passing a string equal in length to the buffer. The result is a read off the end of the buffer, which could potentially be exploited to disclose the contents of adjacent memory. Repro: var_kcjtxvgr = bytearray([0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44,0x41,0x42,0x43,0x44]) var_kcjtxvgr.find(\x41 * 0x58) Exception: 0:000 r eax=0002 ebx=0058 ecx=071adf41 edx= esi=071f2264 edi=0057 eip=1e081cf9 esp=0027fc2c ebp=071ae000 iopl=0 nv up ei pl nz na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206 python27!stringlib_find+0x169: 1e081cf9 0fbe0c2amovsx ecx,byte ptr [edx+ebp] ds:002b:071ae000=?? 0:000 dV str = 0x071adfa8 ABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCD str_len = 0n2 sub = 0x071f2264 sub_len = 0n88 offset = 0n0 0:000 db ebp-0x10 071adff0 41 42 43 44 41 42 43 44-41 42 43 44 41 42 43 44 ABCDABCDABCDABCD 071ae000 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 071ae010 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 071ae020 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 071ae030 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 071ae040 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 071ae050 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 071ae060 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? 0:000 !analyze -v -nodb *** * * *Exception Analysis * * * *** FAULTING_IP: python27!stringlib_find+169 [c:\build27\cpython\objects\stringlib\find.h @ 22] 1e081cf9 0fbe0c2amovsx ecx,byte ptr [edx+ebp] EXCEPTION_RECORD: -- (.exr 0x) ExceptionAddress: 1e081cf9 (python27!stringlib_find+0x0169) ExceptionCode: c005 (Access violation) ExceptionFlags: NumberParameters: 2 Parameter[0]: Parameter[1]: 071ae000 Attempt to read from address 071ae000 CONTEXT: -- (.cxr 0x0;r) eax=0002 ebx=0058 ecx=071adf41 edx= esi=071f2264 edi=0057 eip=1e081cf9 esp=0027fc2c ebp=071ae000 iopl=0 nv up ei pl nz na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206 python27!stringlib_find+0x169: 1e081cf9 0fbe0c2amovsx ecx,byte ptr [edx+ebp] ds:002b:071ae000=?? FAULTING_THREAD: 1e90 DEFAULT_BUCKET_ID: INVALID_POINTER_READ PROCESS_NAME: pythonw.exe ERROR_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s. EXCEPTION_CODE: (NTSTATUS) 0xc005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s. EXCEPTION_PARAMETER1: EXCEPTION_PARAMETER2: 071ae000 READ_ADDRESS: 071ae000 FOLLOWUP_IP: python27!stringlib_find+169 [c:\build27\cpython\objects\stringlib\find.h @ 22] 1e081cf9 0fbe0c2amovsx ecx,byte ptr [edx+ebp] NTGLOBALFLAG: 200 APPLICATION_VERIFIER_FLAGS: 0 APP: pythonw.exe ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre PRIMARY_PROBLEM_CLASS: INVALID_POINTER_READ BUGCHECK_STR: APPLICATION_FAULT_INVALID_POINTER_READ LAST_CONTROL_TRANSFER: from 1e081ee5 to 1e081cf9 STACK_TEXT: 0027fc48 1e081ee5 071adfa8 071f2264 0058 python27!stringlib_find+0x169 0027fc5c 1e083ac1 071adfa8 071f2264 0058 python27!stringlib_find_slice+0x35 0027fcb4 1e083b20 0001 1e083b10 1e0aafd7 python27!bytearray_find_internal+0x81 0027fcc0 1e0aafd7 070880c8 071d7a10 07086170 python27!bytearray_find+0x10 0027fcd8 1e0edd10 07086170 071d7a10 python27!PyCFunction_Call+0x47 0027fd04 1e0f017a 0027fd5c 06cc7c80 06cc7c80 python27!call_function+0x2b0 0027fd74 1e0f1150 07060d60 06cc7c80 python27!PyEval_EvalFrameEx+0x239a 0027fda8 1e0f11b2 06cc7c80 07060d60 06ccba50 python27!PyEval_EvalCodeEx+0x690 0027fdd4 1e11707a 06cc7c80 06ccba50 06ccba50 python27!PyEval_EvalCode+0x22 0027fdec 1e1181c5 0710ee20
[issue24462] bytearray.find Buffer Over-read
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24462 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
