[issue37832] _Py_HashRandomization_Init: failed to get random numbers
New submission from Jeffrey Walton : I need to setup a Debian HURD test machine to investigate a problem I was seeing in the Crypto++ library. After setting up the machine and running an apt-get install for some build tools I noticed Python was failing: Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python It seems HURD no longer provides /dev/urandom out of the box. This is new behavior, and it was not present in the past. It is present in the ISO's from June 2019 (https://cdimage.debian.org/cdimage/ports/current-hurd-i386/iso-dvd/). Python may want to employ a fallback strategy since whatever is using Python ignores the error and trucks on. Sorry to dump this on you. - jwalton@hurd-x86:~$ ls /dev/*rand* /dev/random jwalton@hurd-x86:~$ python --version Python 2.7.16 jwalton@hurd-x86:~$ uname -a GNU hurd-x86 0.9 GNU-Mach 1.8+git20190109-486/Hurd-0.9 i686-AT386 GNU jwalton@hurd-x86:~$ apt-cache show python Package: python Architecture: hurd-i386 Version: 2.7.16-1 Multi-Arch: allowed Priority: standard Section: python Source: python-defaults Maintainer: Matthias Klose Installed-Size: 68 Provides: python-ctypes, python-email, python-importlib, python-profiler, python-wsgiref Pre-Depends: python-minimal (= 2.7.16-1) Depends: python2.7 (>= 2.7.16-1~), libpython-stdlib (= 2.7.16-1), python2 (= 2.7.16-1) ... -- components: Interpreter Core messages: 349453 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: _Py_HashRandomization_Init: failed to get random numbers type: enhancement versions: Python 2.7 ___ Python tracker <https://bugs.python.org/issue37832> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1621] Do not assume signed integer overflow behavior
Jeffrey Walton added the comment: On Tue, Sep 11, 2018 at 8:26 PM, STINNER Victor wrote: > > STINNER Victor added the comment: > >> newsize <<= 1; // The largest possible value is PY_SSIZE_T_MAX + 1. > > Previously, there was a explicitly check for error raising PyErr_NoMemory() > on overflow. Now you rely on PyMem_Malloc() to detect the overflow. I'm not > sure that it's a good idea. +1. It will probably work as expected on Solaris and other OSes that don't oversubscribe memory. It will probably fail in unexpected ways on Linux when the allocation succeeds but then the OOM killer hits a random process. Jeff -- ___ Python tracker <https://bugs.python.org/issue1621> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33145] unaligned accesses in siphash24() lead to crashes on sparc
Jeffrey Walton added the comment: I know this is a bit late but I wanted to share... OpenCSW has a build farm with Solaris machines and Sparc hardware. The farm provides x86 and Sparc machines with Solaris 9 through 11. I believe OpenCSW operates in the same spirit as GCC compile farm. They welcome open source developers and upstream maintainers to help ensure packages build and run on Solaris machines. You can read about it at https://www.opencsw.org/extend-it/signup/to-upstream-maintainers/ . If Python is performing memory access patterns as discussed in the report then it would probably benefit the project to test on a Sparc machine with Solaris 11. -- nosy: +Jeffrey.Walton ___ Python tracker <https://bugs.python.org/issue33145> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28055] pyhash's siphash24 assumes alignment of the data pointer
Jeffrey Walton added the comment: I know this is a bit late but I wanted to share... OpenCSW has a build farm with Solaris machines and Sparc hardware. The farm provides x86 and Sparc machines with Solaris 9 through 11. I believe OpenCSW operates in the same spirit as GCC compile farm. They welcome open source developers and upstream maintainers to help ensure packages build and run on Solaris machines. You can read about it at https://www.opencsw.org/extend-it/signup/to-upstream-maintainers/ . If Python is performing memory access patterns as discussed in the report then it would probably benefit the project to test on a Sparc machine with Solaris 11. -- nosy: +Jeffrey.Walton ___ Python tracker <https://bugs.python.org/issue28055> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20948] -Wformat=2 -Wformat-security findings
Jeffrey Walton added the comment: On Tue, Jul 26, 2016 at 4:31 AM, Martin Panter <rep...@bugs.python.org> wrote: > > Martin Panter added the comment: > > The Modules/main.c cases are not errors. They are just long strings defined > as static constants, rather than literals passed in directly. > > I think we can close this now. Unless people think this warning is worth > using, in which case we should find a way to work around the false positives. > Would it be possible to add some instrumentation to silence the finding? There's no sense in having multiple developers and qa research the issue. I'm guessing a percentage of developers and qa will file bug reports, so it will burn some of the python team's cycles, too. Maybe something like: #if (GCC_VERSION >= 40600) || (LLVM_CLANG_VERSION >= 10700) || (APPLE_CLANG_VERSION >= 2) # define GCC_DIAGNOSTIC_AVAILABLE 1 #endif #if GCC_DIAGNOSTIC_AVAILABLE # pragma GCC diagnostic ignored "-Wformat-security" #endif If its safe to ignore the warning, then the technique above should be safe for a C/CC/CXX/CPP files. It will not cross-pollinate because its a source file, and not a header file. Jeff -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20948> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1621] Do not assume signed integer overflow behavior
Jeffrey Walton added the comment: > Has this sort of thing been done in other projects? Yes. If you are using C, you can use safe_iop. Android uses it for safer integer operations. If you are using C++, you can use David LeBlanc's SafeInt class. Microsoft uses it for safer inter operations. Jeff -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue1621> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23928] SSL wiki page, host name matching, CN and SAN
New submission from Jeffrey Walton: The Python wiki page on SSL states (https://wiki.python.org/moin/SSL): To validate that a certificate matches requested site, you need to check commonName field in the subject of the certificate. I don't think its quite correct. Both the IETF and the CA/B Forums deprecated the use of a hostname or IP address in the commonName (CN). All hostnames and IP addresses must be listed in the subjectAlternateName (SAN), and that's where to look for them. Though deprecated, placing a name in the CN is not forbidden. In fact, RFC 6125 states the CN should be used as a last resort in Section 6.4.4: Therefore, if and only if the presented identifiers do not include a DNS-ID, SRV-ID, URI-ID, or any application-specific identifier types supported by the client, then the client MAY as a last resort check for a string whose form matches that of a fully qualified DNS domain name in a Common Name field of the subject field (i.e., a CN-ID). Following the advice on the wiki might lead to a Type II error, where an otherwise good certificate is rejected. Its not as bad as accepting a bad certificate, though (by omitting the hostname matching checks). The IETF deprecated the practice of placing a name in the CN in RFC 6125, Section 6.4.4. The CA/Browser Forum deprecated a DNS name in the CN in Baseline Requirements (BR) Section 9.2.2 Subject Common Name Field. -- assignee: docs@python components: Documentation messages: 240590 nosy: Jeffrey.Walton, docs@python priority: normal severity: normal status: open title: SSL wiki page, host name matching, CN and SAN type: enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23928 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23928] SSL wiki page, host name matching, CN and SAN
Jeffrey Walton added the comment: there's not much point in reporting bugs here about it. Oh, sorry about that. That page's contents look very outdated, by the way. Yeah, there's a few opportunities for improvement. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23928 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20958] Undefined behavior flagged by Clang 3.4 (Python 3.4.0)
New submission from Jeffrey Walton: Python 3.4.0 downloaded from website (https://www.python.org/download/releases/3.4.0/). Objects/dictobject.c:756:5: runtime error: left shift of negative value -2 Objects/funcobject.c:907:5: runtime error: left shift of negative value -2 Objects/funcobject.c:726:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:1718:5: runtime error: left shift of negative value -2 Objects/tupleobject.c:120:5: runtime error: left shift of negative value -3 Objects/typeobject.c:905:9: runtime error: left shift of negative value -3 Objects/dictobject.c:818:5: runtime error: left shift of negative value -3 Objects/methodobject.c:45:5: runtime error: left shift of negative value -3 Objects/listobject.c:178:5: runtime error: left shift of negative value -3 Modules/gcmodule.c:1703:9: runtime error: left shift of negative value -2 Modules/gcmodule.c:1693:5: runtime error: left shift of negative value -3 Objects/descrobject.c:9:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:484:13: runtime error: left shift of negative value -3 Objects/tupleobject.c:195:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:503:13: runtime error: left shift of negative value -4 Objects/exceptions.c:2205:5: runtime error: left shift of negative value -2 Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Objects/frameobject.c:736:5: runtime error: left shift of negative value -3 Objects/funcobject.c:64:5: runtime error: left shift of negative value -3 Objects/methodobject.c:149:5: runtime error: left shift of negative value -2 Objects/funcobject.c:552:5: runtime error: left shift of negative value -2 Objects/descrobject.c:1364:5: runtime error: left shift of negative value -2 Objects/cellobject.c:16:5: runtime error: left shift of negative value -3 Objects/listobject.c:2744:5: runtime error: left shift of negative value -3 Objects/listobject.c:2751:5: runtime error: left shift of negative value -2 Objects/dictobject.c:3232:5: runtime error: left shift of negative value -3 Objects/dictobject.c:2826:5: runtime error: left shift of negative value -3 Objects/exceptions.c:89:5: runtime error: left shift of negative value -2 Objects/classobject.c:68:5: runtime error: left shift of negative value -3 Objects/classobject.c:193:5: runtime error: left shift of negative value -2 Objects/tupleobject.c:1079:5: runtime error: left shift of negative value -3 Objects/genobject.c:526:5: runtime error: left shift of negative value -3 Objects/tupleobject.c:948:5: runtime error: left shift of negative value -2 Objects/genobject.c:48:5: runtime error: left shift of negative value -2 Objects/genobject.c:53:5: runtime error: left shift of negative value -3 Objects/genobject.c:58:5: runtime error: left shift of negative value -2 Objects/cellobject.c:49:5: runtime error: left shift of negative value -2 Objects/typeobject.c:1170:9: runtime error: left shift of negative value -3 Objects/exceptions.c:662:5: runtime error: left shift of negative value -2 Objects/exceptions.c:1010:5: runtime error: left shift of negative value -2 Objects/exceptions.c:513:5: runtime error: left shift of negative value -2 ./Modules/_io/fileio.c:479:5: runtime error: left shift of negative value -2 Objects/unicodeobject.c:15267:5: runtime error: left shift of negative value -3 Objects/unicodeobject.c:15128:5: runtime error: left shift of negative value -2 Objects/typeobject.c:6720:5: runtime error: left shift of negative value -2 Objects/setobject.c:944:5: runtime error: left shift of negative value -3 Objects/bytesobject.c:3006:5: runtime error: left shift of negative value -3 Objects/bytesobject.c:2869:5: runtime error: left shift of negative value -2 Objects/bytearrayobject.c:3094:5: runtime error: left shift of negative value -3 Objects/bytearrayobject.c:2959:5: runtime error: left shift of negative value -2 Objects/descrobject.c:1001:9: runtime error: left shift of negative value -3 Objects/descrobject.c:873:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:861:13: runtime error: left shift of negative value -3 Objects/typeobject.c:2865:5: runtime error: left shift of negative value -2 Objects/iterobject.c:26:5: runtime error: left shift of negative value -3 Objects/iterobject.c:33:5: runtime error: left shift of negative value -2 Objects/dictobject.c:2643:9: runtime error: left shift of negative value -2 Objects/descrobject.c:1262:9: runtime error: left shift of negative value -3 Objects/memoryobject.c:74:5: runtime error: left shift of negative value -3 Objects/memoryobject.c:639:5: runtime error: left shift of negative value -3 Objects/memoryobject.c:1060:5: runtime error: left shift of negative value -2 Objects/memoryobject.c:108:5: runtime error: left shift of negative value -2 ./Modules/_io/bufferedio.c:402:5: runtime error: left shift of negative value -2 ./Modules/_io/textio.c:1156:5: runtime error: left shift of negative value
[issue20965] Clang devguide update
New submission from Jeffrey Walton: Updated. Its a lot easier to identify gaps and update once its seen with formatting. * Added info on interpreting results * Added info on flags and CFLAGS, CXXFLAGS, CC, and CXX * Added info on Mac OS X * Asan - ASan, UBsan - UBSan * Fixed UBSan example formatting * Added 'make test 21 | asan_symbolize.py' example to text * Added section on blacklisting or ignoring findings * Added blacklist example with audioop.c * Spelling corrections -- components: Devguide files: clang.diff hgrepos: 227 keywords: patch messages: 213935 nosy: Jeffrey.Walton, ezio.melotti priority: normal severity: normal status: open title: Clang devguide update type: enhancement versions: Python 3.4 Added file: http://bugs.python.org/file34477/clang.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20965 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20965] Clang devguide update
Jeffrey Walton added the comment: Contributor license was signed, though I don't consider my self a contributor. The devs are making the real contributions. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20965 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20944] Engineering Process Improvements
New submission from Jeffrey Walton: Python's code is crisp and sharp. From a software design perspective, I don't see a lot of room for improvement. However, looking at some of the issues flagged by Clang sanitizers and existing bug reports, I think the project has a couple of small opportunities for improvement in the engineering process. The small improvements have the potential for large payoffs, and could help take a mature code base to the next level. The improvements are not sexy, and they are often overlooked. The improvements are: (1) add instrumentation to the code; and (2) add scanning and analysis to the engineering process. To improve the code and engineering process to proactively hunt obscure and hard to find bugs, I would suggest three actionable items: * ASSERTions * Clang Santizers * Coverity Scans Placing measures to proactively hunt bugs will improve the code quality, and likely allow the project to drop `-fwrapv` and friends from compiler options (http://bugs.python.org/issue1621). ASSERTions == The code uses Posix's little assert on occasion. Posix assert is useless during development because it raise SIGABRT. Additionally, the code has minimal assertions, so assertions are not being utilized effectively. I think there is an opportunity to use a big ASSERT. The big ASSERT raises a SIGTRAP, and during development it will ensure the debugger snaps on an unexpected condition. The big ASSERT raises awareness where needed and allows the developers to continue handling on a negative code path. ASSERTs should be liberally sprinkled. That is, ASSERT parameters, return values and program state. Python is setup correctly with NDEBUG on Release builds, so the ASSERTs will be removed in production. With a full compliment of ASSERTs in place, the code will essential debug itself under most circumstances. In fact, I often pitch it as self-debugging code to management. Self debugging code has three benefits. First, self debugging code is effectively a force multiplier, and it allows fewer developers to maintain a larger code base. Second, it allows non-senior members to effectively contribute to the project. Google Summer of Code interns and junior developers could be effective team members since the ASSERTs will often guide them in tracing problems. Third, ASSERTs free up time for other endeavors, like adding features to Python or watching football. Clang Sanitizers === I think the code base would benefit from regular Clang sanitizer scans. From initial scanning and testing, I believe Clang and its sanitizers has shown its value. The sanitizers perform dynamic analysis. The issues flagged are almost always real issues, and require almost no filtering due to false positives. The analysis should include the undefined behavior sanitizer (-fsanitize=undefined) and the address sanitizer (-fsanitize=address). There are more sanitizers available, and they are listed at http://clang.llvm.org/docs/UsersManual.html#controlling-code-generation. From past experience, I've used Asan (address sanitizer) and UBsan (undefined behavior sanitizer) to find tricky bugs in other projects and libraries. For example, Wei Dai's Crypto++ would fail one self test when compiled with Intel's ICC (but pass them all under Clang, Comeau, GCC and MSVC). The failed Crypto++ self test was due to ICC aggressively removing undefined behavior, and the Intel compiler/optimizer targeted [not readily apprent] undefined behavior in the underlying library code. After fixing the library code identifed by Clang UBsan, Crypto++ tested fine under all compilers. The real beauty here is: (1) Python already has a mature process, so the addition of the Clang analysis will take minimal effort; and (2) Python has a rich set of self-test, so the Clang sanitizers can *really* be effective. For areas that don't have complete self test coverage, its another task that can be delegated to non-senior members and contributors. That frees up time for senior members to do other things, like adding features to Python or watching football. Coverity Scans == I think the project could benefit from Coverity scanning and analysis. The analysis is quite good most of the time, and I believe it will complement Python's exiting engineering process. In addition, it will complement Clang analysis is Clang is adopted. Coverity is one of the high-end analysis engines available. Though Coverity is a pay-to-play service, Coverity offers a free scanning service for free projects. The URL for the free service is http://scan.coverity.com. It cost nothing to sign up for, and it costs nothing to run a scan. Minimal effort is required to prepare a binary for upload. Coverity performs static analysis, and it will flag false positives on occasion. However, its a trade-off in the effort to proactively hunt bugs. -- components: Build messages: 213728 nosy
[issue1621] Do not assume signed integer overflow behavior
Jeffrey Walton added the comment: Also see http://bugs.python.org/issue20944 for suggestions to identify the offending code. -- nosy: +Jeffrey.Walton ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1621 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20944] Engineering Process Improvements
Jeffrey Walton added the comment: On Sun, Mar 16, 2014 at 11:12 AM, R. David Murray rep...@bugs.python.org wrote: R. David Murray added the comment: We already have Coverty scan in place, and were in fact featured by them for our code quality. Currently Christian Heimes is the lead on that effort, and is monitoring the Coverty reports. Oh, my bad. I searched for the project and got 0 results. Please accept my apologies. EDIT: my dislexia got me. I searched for Pyhton, and not Python. Sorry about that. We've been working on Clang stuff as developers have had interest, and accepted a patch enabling address sanitizer to be used. That's great. Jeff -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20944 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20947] -Wstrict-overflow findings
New submission from Jeffrey Walton: $ hg id 3736bf94535c+ tip Forgive me if you were aware of these. /usr/bin/gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -I./Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/usr/local/include -Icpython/./Include -Icpython/. -c cpython/./Modules/_posixsubprocess.c -o build/temp.linux-x86_64-3.4cpython/./Modules/_posixsubprocess.o cpython/./Modules/_posixsubprocess.c: In function ‘child_exec’: cpython/./Modules/_posixsubprocess.c:491:33: warning: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Wstrict-overflow] cpython/./Modules/_posixsubprocess.c: In function ‘subprocess_fork_exec’: cpython/./Modules/_posixsubprocess.c:491:33: warning: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Wstrict-overflow] cpython/./Modules/_posixsubprocess.c:491:33: warning: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Wstrict-overflow] cpython/./Modules/_posixsubprocess.c:491:33: warning: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Wstrict-overflow] ... /usr/bin/gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -I./Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/usr/local/include -Icpython/./Include -Icpython/. -c cpython/./Modules/sha512module.c -o build/temp.linux-x86_64-3.4cpython/./Modules/sha512module.o cpython/./Modules/sha512module.c: In function ‘sha512_transform’: cpython/./Modules/sha512module.c:128:1: warning: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Wstrict-overflow] cpython/./Modules/sha512module.c:128:1: warning: assuming pointer wraparound does not occur when comparing P +- C1 with P +- C2 [-Wstrict-overflow] -- components: Build hgrepos: 224 messages: 213742 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: -Wstrict-overflow findings versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20947 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20948] -Wformat=2 -Wformat-security findings
New submission from Jeffrey Walton: $ hg id 3736bf94535c+ tip -Wformat=2 -Wformat-security are useful for detecting possible security related bugs. Compiling with the two options produced a few hits in the source code. /usr/bin/gcc -pthread -c -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -I. -IInclude -I./Include-DPy_BUILD_CORE -o Objects/unicodeobject.o cpython/./Objects/unicodeobject.c cpython/./Objects/unicodeobject.c: In function ‘unicode_fromformat_arg’: cpython/./Objects/unicodeobject.c:2527:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2531:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2535:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2538:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2542:13: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2549:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2553:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2557:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Objects/unicodeobject.c:2560:25: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] I think those are necessary for to `unicode_fromformat_arg`. /usr/bin/gcc -pthread -c -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -fno-common -Wstrict-overflow -Wformat=2 -Wformat-security -Wcast-align -Wtrampolines -I. -IInclude -I./Include-DPy_BUILD_CORE -o Modules/main.o cpython/./Modules/main.c cpython/./Modules/main.c: In function ‘usage’: cpython/./Modules/main.c:111:5: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Modules/main.c:118:9: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] cpython/./Modules/main.c:119:9: warning: format not a string literal, argument types not checked [-Wformat-nonliteral] I think the occurrences in main.c could benefit from %s to ensure the program does not accidentally leak. -- components: Build hgrepos: 225 messages: 213743 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: -Wformat=2 -Wformat-security findings versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20948 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20948] -Wformat=2 -Wformat-security findings
Jeffrey Walton added the comment:
If interested, I think the warnings can be selectively turned off:
#if defined (__GNUC__) ((__GNUC__ == 4 __GNUC_MINOR__ = 6) || (__GNUC__
= 5))
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored -Wformat-security
#endif
unicode_fromformat_arg(...)
{
...
}
#if defined (__GNUC__) ((__GNUC__ == 4 __GNUC_MINOR__ = 6) || (__GNUC__
= 5))
# pragma GCC diagnostic pop
#endif
Microsoft has a similar mechanism.
It should allow the project to compile with -Wformat-security full time while
maintinaing a quiet compile (silent is good).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20948
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20948] -Wformat=2 -Wformat-security findings
Jeffrey Walton added the comment: #if defined (__GNUC__) ((__GNUC__ == 4 __GNUC_MINOR__ = 6) || (__GNUC__ = 5)) # pragma GCC diagnostic push # pragma GCC diagnostic ignored -Wformat-security #endif My bad... -Wformat-nonliteral -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20948 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20949] Missing platform security integrations
New submission from Jeffrey Walton:
$ hg id
3736bf94535c+ tip
A standard Python build does not take a proactive approach to integrating with
platform security measures. Attepting to add the measures results in a failed
build.
For example:
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export CFLAGS=-fPIC -fstack-protector-all -D_FORTIFY_SOURCE=2
export CXXFLAGS=-fPIC -fstack-protector-all -D_FORTIFY_SOURCE=2
export LDFLAGS=-pie -Wl,-z,noexecstack -Wl,-z,noexecheap -Wl,-z,now
-Wl,-z,relro
will configure properly, but will fail to build.
The idea is to build executables with {-fPIE,-pie} and build shared objects
with {-fPIC,-shared}. Both executables and shared objects get the remaining
platform security integrations like stack protectors and NX stacks/heaps.
In the case an object file is used for both an executable and shared object, it
should be compiled with -fPIC (and linking will include -pie or -shared as
required). Its OK to use -fPIC in place of -fPIE. See
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52885 for details.
*
Examining the failed compile:
/usr/bin/gcc -pthread -shared -pie -Wl,-z,noexecstack -Wl,-z,noexecheap
-Wl,-z,now -Wl,-z,relro -pie -Wl,-z,noexecstack -Wl,-z,noexecheap -Wl,-z,now
-Wl,-z,relro -pie -Wl,-z,noexecstack -Wl,-z,noexecheap -Wl,-z,now -Wl,-z,relro
-fPIC -fstack-protector-all -D_FORTIFY_SOURCE=2
build/temp.linux-x86_64-3.4/home/jwalton/Desktop/cpython-checkout/Modules/_struct.o
-L/usr/lib/x86_64-linux-gnu -L/usr/local/lib -o
build/lib.linux-x86_64-3.4/_struct.cpython-34m.so
So, autotools tried to add both -pie (for executables) and -shared (for shared
objects). Fail.
The same problem occurs with _struct.cpython-34m.so,
_ctypes_test.cpython-34m.so, array.cpython-34m.so, cmath.cpython-34m.so,
math.cpython-34m.so, time.cpython-34m.so, _datetime.cpython-34m.so,
_random.cpython-34m.so, _bisect.cpython-34m.so, ...
*
I know I can omit -pie from CFLAGS and CXXFLAGS:
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export CFLAGS=-fPIC -fstack-protector-all -D_FORTIFY_SOURCE=2
export CXXFLAGS=-fPIC -fstack-protector-all -D_FORTIFY_SOURCE=2
export LDFLAGS=-Wl,-z,noexecstack -Wl,-z,noexecheap -Wl,-z,now -Wl,-z,relro
but then I have to manually add -pie to Makefile lines with $BUILDPYTHON (and
others, like _testembed and _freeze_importlib):
$(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
$(LINKCC) -pie $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o
$(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
...
*
Examining an executable produced by the modified Makefil with Tobias Klein's
Checksec (http://www.trapkit.de/tools/checksec.html) shows the platform
security integrations were successfully applied:
$ checksec.sh --file ./python
RELRO STACK CANARY NXPIE RPATH
RUNPATH FILE
Full RELRO Canary found NX enabledPIE enabled No RPATH No
RUNPATH ./python
*
Running `make test` with the security integrations worked as expected, and did
not result in any adverse behavior (like an abrupt shutdown).
*
It would be great if Python tested for features like ASLR for executables, and
simply added {-fPIE,-pie} as available. The same is true for the other security
offerings (_FORTIFY_SOURCE should be added to Release builds only).
--
components: Build
hgrepos: 226
messages: 213749
nosy: Jeffrey.Walton
priority: normal
severity: normal
status: open
title: Missing platform security integrations
versions: Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20949
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20949] Missing platform security integrations
Jeffrey Walton added the comment: $ checksec.sh --file ./python RELRO STACK CANARY NXPIE RPATH RUNPATH FILE Full RELRO Canary found NX enabledPIE enabled No RPATH No RUNPATH ./python Here's what a standard Python build looks like (without the platform security integrations): $ checksec.sh --file ./python RELRO STACK CANARY NXPIE RPATH RUNPATH FILE No RELRONo canary found NX enabledNo PIE No RPATH No RUNPATH ./python I believe the NX stack is coming Debian's hardening for amd64's (https://wiki.debian.org/Hardening). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20949 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18747] Re-seed OpenSSL's PRNG after fork
Jeffrey Walton added the comment: It probably is an OpenSSL bug but the declaration doesn't help us. It's not the first time Python has to work around OpenSSL, e.g. #18709. Sorry to dig up an old issue. But here's some reading on it if interested. Ben Laurire pushed a patch to mix in PID and time. The PID was already being used, so the patch adds the time. See Mixing time into the pool, http://www.mail-archive.com/openssl-dev@openssl.org/msg33012.html. And the commit: https://github.com/openssl/openssl/commit/3cd8547a2018ada88a4303067a2aa15eadc17f39. OpenSSL added a wiki page for reading on the subject: http://wiki.openssl.org/index.php/Random_fork-safety. -- nosy: +Jeffrey.Walton ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18747 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20952] OpenSSL and RDRAND
New submission from Jeffrey Walton: Some versions of OpenSSL use the RDRAND engine by default. The versions include openssl-1.0.1-beta1 through openssl-1.0.1f. RDRAND has taken some criticism because its essentially unaudited and it could be spiked like the Dual-EC generator (http://blog.cryptographyengineering.com/2013/09/the-many-flaws-of-dualecdrbg.html). If the RDRAND engine is in effect, then the application and the library (internally) will be using the generator. But some some folks don't want to use an unaudited generator. I'm not sure what the best action is to take. For reading on ways to disable the RDRAND engine, see http://seclists.org/fulldisclosure/2013/Dec/142. -- components: Extension Modules messages: 213769 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: OpenSSL and RDRAND ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20952 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20937] test_socket: buffer overflow in sock_recvmsg_guts
Jeffrey Walton added the comment: This might be relevant. It showed up while building Python 3.3.5 from sources. /usr/local/bin/clang -fsanitize=undefined -fPIC -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I./Include -I. -IInclude -I/usr/local/include -IPython-3.3.5/./Include -IPython-3.3.5/. -c Python-3.3.5/./Modules/socketmodule.c -o build/temp.linux-x86_64-3.3Python-3.3.5/./Modules/socketmodule.o Python-3.3.5/./Modules/socketmodule.c:1951:74: warning: comparison of unsigned expression 0 is always false [-Wtautological-compare] if (cmsgh == NULL || msg-msg_control == NULL || msg-msg_controllen 0) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20937 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20953] heap-buffer-overflow in obmalloc.c:987
New submission from Jeffrey Walton: This came from Python 3.3.5 downloaded from thePython download page (). The issue occurred while compiling with Clang 3.4 using the address sanitizer (-fsanitize=address) /usr/local/bin/clang -fsanitize=address -Xlinker -export-dynamic -o python Modules/python.o libpython3.3m.a -ldl -lutil /usr/local/ssl/lib/libssl.a /usr/local/ssl/lib/libcrypto.a -ldl -lm ./python -E -S -m sysconfig --generate-posix-vars = ==24064==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61904020 at pc 0x4ed4b2 bp 0x7fff80fff010 sp 0x7fff80fff008 READ of size 4 at 0x61904020 thread T0 #0 0x4ed4b1 in PyObject_Free Python-3.3.5/./Objects/obmalloc.c:987 #1 0x7a2141 in code_dealloc Python-3.3.5/./Objects/codeobject.c:359 #2 0x620c00 in PyImport_ImportFrozenModuleObject Python-3.3.5/./Python/import.c:1098 #3 0x620d5c in PyImport_ImportFrozenModule Python-3.3.5/./Python/import.c:1114 #4 0x63fd07 in import_init Python-3.3.5/./Python/pythonrun.c:206 #5 0x63f636 in _Py_InitializeEx_Private Python-3.3.5/./Python/pythonrun.c:369 #6 0x681d77 in Py_Main Python-3.3.5/./Modules/main.c:648 #7 0x4e6894 in main Python-3.3.5/././Modules/python.c:62 #8 0x2abf9a525eac in __libc_start_main /home/aurel32/eglibc/eglibc-2.13/csu/libc-start.c:244 #9 0x4e664c in _start (Python-3.3.5/./python+0x4e664c) AddressSanitizer can not describe address in more detail (wild memory access suspected). SUMMARY: AddressSanitizer: heap-buffer-overflow Python-3.3.5/./Objects/obmalloc.c:987 PyObject_Free Shadow bytes around the buggy address: 0x0c327fff87b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff87c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff87d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff87e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff87f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa =0x0c327fff8800: fa fa fa fa[fa]fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff8810: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff8820: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff8830: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff8840: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c327fff8850: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone:fb Freed heap region: fd Stack left redzone:f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return:f5 Stack use after scope: f8 Global redzone:f9 Global init order: f6 Poisoned by user: f7 ASan internal: fe ==24064==ABORTING make: *** [pybuilddir.txt] Error 1 -- components: Build messages: 213788 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: heap-buffer-overflow in obmalloc.c:987 versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20953 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20935] Cherry pick CFLAGS, add to flags for $(BUILDPYTHON) Makefile rule
New submission from Jeffrey Walton: From Python head in mercurial. When building Python under Clang's sanitizers, we provide a couple of flags to instrument binaries with the sanitizers. For example: export CC=/usr/local/bin/clang export CXX=/usr/local/bin/clang++ export CFLAGS=-g3 -fsanitize=undefined -fsanitize=address export CXXFLAGS=-g3 -fsanitize=undefined -fsanitize=address -fno-sanitize=vptr ./configure make However, `make` will fail due to some missing sanitizer libraries. The libraries are added at the link stage by Clang, but the invocation must include the -fsanitize=... flags. The recipe for $(BUILDPYTHON) in the Makefile does not include necessary CFLAGS: # Build the interpreter $(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY) $(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) The result is a failed link when building with the sanitizers. It would be great if the sanizter flags (-fsanitize=undefined -fsanitize=address -fno-sanitize=vptr) were cherry picked from the FLAGS by the build system and added to the recipe as required: $(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY) $(LINKCC) -fsanitize=undefined -fsanitize=address -fno-sanitize=vptr $(PY_LDFLAGS) $(LINKFORSHARED) ... Please consider picking up the sanitizer flags and adding them to the build rule. -- components: Build hgrepos: 220 messages: 213661 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: Cherry pick CFLAGS, add to flags for $(BUILDPYTHON) Makefile rule type: enhancement versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20935 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20935] Cherry pick CFLAGS, add to flags for $(BUILDPYTHON) Makefile rule
Jeffrey Walton added the comment: And: Modules/_testembed: Modules/_testembed.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY) $(LINKCC) -g3 -fsanitize=address $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20935 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20935] Cherry pick CFLAGS, add to flags for $(BUILDPYTHON) Makefile rule
Jeffrey Walton added the comment: And: Modules/_freeze_importlib: Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LINKCC) -g3 -fsanitize=address $(PY_LDFLAGS) -o $@ Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20935 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20936] test_strftime: enormous allocation, fails under Clang sanitizer
New submission from Jeffrey Walton: From Python head in mercurial: $ hg id 7ce22d0899e4+ tip [118/389/1] test_strftime ==11587==WARNING: AddressSanitizer failed to allocate 0x7fff bytes ==11587==AddressSanitizer's allocator is terminating the process instead of returning 0 ==11587==If you don't like this behavior set allocator_may_return_null=1 ==11587==AddressSanitizer CHECK failed: /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc:149 ((0)) != (0) (0x0, 0x0) #0 0x4d79df in __asan::AsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/asan/asan_rtl.cc:66 #1 0x4dd241 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:69 #2 0x4dbf80 in __sanitizer::AllocatorReturnNull() /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator.cc:149 #3 0x4d1a76 in malloc /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:75 #4 0x7c8140 in _buffered_init /home/jwalton/Desktop/cpython-checkout/./Modules/_io/bufferedio.c:772 #5 0x7c6888 in bufferedreader_init /home/jwalton/Desktop/cpython-checkout/./Modules/_io/bufferedio.c:1424 #6 0x5b06e8 in wrap_init /home/jwalton/Desktop/cpython-checkout/Objects/typeobject.c:5293 #7 0x4fd729 in PyObject_Call /home/jwalton/Desktop/cpython-checkout/Objects/abstract.c:2067 #8 0x6642d6 in ext_do_call /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4551 #9 0x6642d6 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2869 #10 0x670b7a in fast_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4324 #11 0x65fbc8 in call_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4252 #12 0x65fbc8 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2829 #13 0x655a7b in PyEval_EvalCodeEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:3578 #14 0x670cb5 in fast_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4334 #15 0x65fbc8 in call_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4252 #16 0x65fbc8 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2829 #17 0x670b7a in fast_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4324 #18 0x65fbc8 in call_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4252 #19 0x65fbc8 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2829 #20 0x655a7b in PyEval_EvalCodeEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:3578 #21 0x84c177 in function_call /home/jwalton/Desktop/cpython-checkout/Objects/funcobject.c:632 #22 0x4fd729 in PyObject_Call /home/jwalton/Desktop/cpython-checkout/Objects/abstract.c:2067 #23 0x6642d6 in ext_do_call /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4551 #24 0x6642d6 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2869 #25 0x655a7b in PyEval_EvalCodeEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:3578 #26 0x84c177 in function_call /home/jwalton/Desktop/cpython-checkout/Objects/funcobject.c:632 #27 0x4fd729 in PyObject_Call /home/jwalton/Desktop/cpython-checkout/Objects/abstract.c:2067 #28 0x830dcc in method_call /home/jwalton/Desktop/cpython-checkout/Objects/classobject.c:347 #29 0x4fd729 in PyObject_Call /home/jwalton/Desktop/cpython-checkout/Objects/abstract.c:2067 #30 0x5ae10f in slot_tp_call /home/jwalton/Desktop/cpython-checkout/Objects/typeobject.c:5809 #31 0x4fd729 in PyObject_Call /home/jwalton/Desktop/cpython-checkout/Objects/abstract.c:2067 #32 0x6653a0 in do_call /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4456 #33 0x6653a0 in call_function /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4254 #34 0x6653a0 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2829 #35 0x655a7b in PyEval_EvalCodeEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:3578 #36 0x84c177 in function_call /home/jwalton/Desktop/cpython-checkout/Objects/funcobject.c:632 #37 0x4fd729 in PyObject_Call /home/jwalton/Desktop/cpython-checkout/Objects/abstract.c:2067 #38 0x6642d6 in ext_do_call /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:4551 #39 0x6642d6 in PyEval_EvalFrameEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:2869 #40 0x655a7b in PyEval_EvalCodeEx /home/jwalton/Desktop/cpython-checkout/Python/ceval.c:3578 #41 0x84c177 in function_call /home/jwalton/Desktop/cpython-checkout/Objects/funcobject.c:632 #42 0x4fd729 in PyObject_Call /home
[issue20937] test_socket: buffer overflow in sock_recvmsg_guts
New submission from Jeffrey Walton: From Python head in mercurial: $ hg id 7ce22d0899e4+ tip Exporting set allocator_may_return_null=1 for Clang might tickle this issue. Without the export, this test did not fail. = ==21071==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603b99f4 at pc 0x4aafea bp 0x7fffd2318c70 sp 0x7fffd2318c20 WRITE of size 24 at 0x603b99f4 thread T0 #0 0x4aafe9 in write_msghdr /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:1395 #1 0x4aafe9 in __interceptor_recvmsg /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:1405 #2 0x2ad35b764146 in sock_recvmsg_guts cpython/./Modules/socketmodule.c:2968 #3 0x2ad35b75f83c in sock_recvmsg cpython/./Modules/socketmodule.c:3098 #4 0x6642ba in ext_do_call cpython/./Python/ceval.c:4548 #5 0x6642ba in PyEval_EvalFrameEx cpython/./Python/ceval.c:2869 #6 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #7 0x670cb5 in fast_function cpython/./Python/ceval.c:4334 #8 0x65fbc8 in call_function cpython/./Python/ceval.c:4252 #9 0x65fbc8 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2829 #10 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #11 0x670cb5 in fast_function cpython/./Python/ceval.c:4334 #12 0x65fbc8 in call_function cpython/./Python/ceval.c:4252 #13 0x65fbc8 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2829 #14 0x670b7a in fast_function cpython/./Python/ceval.c:4324 #15 0x65fbc8 in call_function cpython/./Python/ceval.c:4252 #16 0x65fbc8 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2829 #17 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #18 0x84c177 in function_call cpython/./Objects/funcobject.c:632 #19 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #20 0x6642d6 in ext_do_call cpython/./Python/ceval.c:4551 #21 0x6642d6 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2869 #22 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #23 0x84c177 in function_call cpython/./Objects/funcobject.c:632 #24 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #25 0x830dcc in method_call cpython/./Objects/classobject.c:347 #26 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #27 0x5ae10f in slot_tp_call cpython/./Objects/typeobject.c:5809 #28 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #29 0x6653a0 in do_call cpython/./Python/ceval.c:4456 #30 0x6653a0 in call_function cpython/./Python/ceval.c:4254 #31 0x6653a0 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2829 #32 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #33 0x84c177 in function_call cpython/./Objects/funcobject.c:632 #34 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #35 0x6642d6 in ext_do_call cpython/./Python/ceval.c:4551 #36 0x6642d6 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2869 #37 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #38 0x84c177 in function_call cpython/./Objects/funcobject.c:632 #39 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #40 0x830dcc in method_call cpython/./Objects/classobject.c:347 #41 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #42 0x5ae10f in slot_tp_call cpython/./Objects/typeobject.c:5809 #43 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #44 0x6653a0 in do_call cpython/./Python/ceval.c:4456 #45 0x6653a0 in call_function cpython/./Python/ceval.c:4254 #46 0x6653a0 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2829 #47 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #48 0x84c177 in function_call cpython/./Objects/funcobject.c:632 #49 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #50 0x6642d6 in ext_do_call cpython/./Python/ceval.c:4551 #51 0x6642d6 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2869 #52 0x655a7b in PyEval_EvalCodeEx cpython/./Python/ceval.c:3578 #53 0x84c177 in function_call cpython/./Objects/funcobject.c:632 #54 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #55 0x830dcc in method_call cpython/./Objects/classobject.c:347 #56 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #57 0x5ae10f in slot_tp_call cpython/./Objects/typeobject.c:5809 #58 0x4fd729 in PyObject_Call cpython/./Objects/abstract.c:2067 #59 0x6653a0 in do_call cpython/./Python/ceval.c:4456 #60 0x6653a0 in call_function cpython/./Python/ceval.c:4254 #61 0x6653a0 in PyEval_EvalFrameEx cpython/./Python/ceval.c:2829 #62 0x670b7a in fast_function cpython/./Python/ceval.c:4324 #63 0x65fbc8 in call_function cpython/./Python/ceval.c:4252 #64 0x65fbc8 in PyEval_EvalFrameEx cpython/./Python
[issue20937] test_socket: buffer overflow in sock_recvmsg_guts
Jeffrey Walton added the comment:
This does not look quite right from Modules/sockewtmodule.c.
/* Fill in an iovec for each item, and save the Py_buffer
structs to release afterwards. */
if (nitems 0 ((iovs = PyMem_New(struct iovec, nitems)) == NULL ||
(bufs = PyMem_New(Py_buffer, nitems)) == NULL)) {
PyErr_NoMemory();
goto finally;
}
for (; nbufs nitems; nbufs++) {
if (!PyArg_Parse(PySequence_Fast_GET_ITEM(fast, nbufs),
w*;recvmsg_into() argument 1 must be an iterable
of single-segment read-write buffers,
bufs[nbufs]))
goto finally;
iovs[nbufs].iov_base = bufs[nbufs].buf;
iovs[nbufs].iov_len = bufs[nbufs].len;
}
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20937
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20940] Test 239: buffer overflow in sock_recvmsg_guts
New submission from Jeffrey Walton: Test 240 also suffers from a buffer overflow on sock_recvmsg_guts. Test 240 is the test that follows 239, and 239 is [239/389/2] test_unittest. (I don't believe the message for 239 has flushed). = ==29767==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603c0ce4 at pc 0x4aafea bp 0x7fff4c426010 sp 0x7fff4c425fc0 WRITE of size 24 at 0x603c0ce4 thread T0 #0 0x4aafe9 in write_msghdr /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:1395 #1 0x4aafe9 in __interceptor_recvmsg /home/jwalton/Desktop/clang-3.4/llvm-3.4/projects/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:1405 #2 0x2b955a764166 in sock_recvmsg_guts ./Modules/socketmodule.c:2968 #3 0x2b955a75f856 in sock_recvmsg ./Modules/socketmodule.c:3098 #4 0x6642ea in ext_do_call ./Python/ceval.c:4548 #5 0x6642ea in PyEval_EvalFrameEx ./Python/ceval.c:2869 #6 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #7 0x670ce5 in fast_function ./Python/ceval.c:4334 #8 0x65fbf8 in call_function ./Python/ceval.c:4252 #9 0x65fbf8 in PyEval_EvalFrameEx ./Python/ceval.c:2829 #10 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #11 0x670ce5 in fast_function ./Python/ceval.c:4334 #12 0x65fbf8 in call_function ./Python/ceval.c:4252 #13 0x65fbf8 in PyEval_EvalFrameEx ./Python/ceval.c:2829 #14 0x670baa in fast_function ./Python/ceval.c:4324 #15 0x65fbf8 in call_function ./Python/ceval.c:4252 #16 0x65fbf8 in PyEval_EvalFrameEx ./Python/ceval.c:2829 #17 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #18 0x84c1a7 in function_call ./Objects/funcobject.c:632 #19 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #20 0x664306 in ext_do_call ./Python/ceval.c:4551 #21 0x664306 in PyEval_EvalFrameEx ./Python/ceval.c:2869 #22 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #23 0x84c1a7 in function_call ./Objects/funcobject.c:632 #24 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #25 0x830dfc in method_call ./Objects/classobject.c:347 #26 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #27 0x5ae13f in slot_tp_call ./Objects/typeobject.c:5809 #28 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #29 0x6653d0 in do_call ./Python/ceval.c:4456 #30 0x6653d0 in call_function ./Python/ceval.c:4254 #31 0x6653d0 in PyEval_EvalFrameEx ./Python/ceval.c:2829 #32 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #33 0x84c1a7 in function_call ./Objects/funcobject.c:632 #34 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #35 0x664306 in ext_do_call ./Python/ceval.c:4551 #36 0x664306 in PyEval_EvalFrameEx ./Python/ceval.c:2869 #37 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #38 0x84c1a7 in function_call ./Objects/funcobject.c:632 #39 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #40 0x830dfc in method_call ./Objects/classobject.c:347 #41 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #42 0x5ae13f in slot_tp_call ./Objects/typeobject.c:5809 #43 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #44 0x6653d0 in do_call ./Python/ceval.c:4456 #45 0x6653d0 in call_function ./Python/ceval.c:4254 #46 0x6653d0 in PyEval_EvalFrameEx ./Python/ceval.c:2829 #47 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #48 0x84c1a7 in function_call ./Objects/funcobject.c:632 #49 0x4fd729 in PyObject_Call ./Objects/abstract.c:2067 #50 0x664306 in ext_do_call ./Python/ceval.c:4551 #51 0x664306 in PyEval_EvalFrameEx ./Python/ceval.c:2869 #52 0x655aab in PyEval_EvalCodeEx ./Python/ceval.c:3578 #53 0x84c1a7 in function_call ./Objects/funcobject.c:632 [Missing remainder of trace] -- components: Tests hgrepos: 222 messages: 213683 nosy: Jeffrey.Walton priority: normal severity: normal status: open title: Test 239: buffer overflow in sock_recvmsg_guts versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20940 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20941] pytime.c:184 and pytime.c:218: runtime error, outside the range of representable values of type 'long'
New submission from Jeffrey Walton:
pytime.c:184: runtime error: value -1e+200 is outside the range of
representable values of type 'long'
and
pytime.c:218: runtime error: value -1e+200 is outside the range of
representable values of type 'long'
It appears the cast on 'intpart' is generating the finding. 'intpart' is a
double.
*sec = (time_t)intpart;
err = intpart - (double)*sec;
if (err = -1.0 || err = 1.0) {
error_time_t_overflow();
return -1;
}
Shouldn't a range test based on TIME_T_MAX with an epsilon occur first?
--
components: Tests
hgrepos: 223
messages: 213686
nosy: Jeffrey.Walton
priority: normal
severity: normal
status: open
title: pytime.c:184 and pytime.c:218: runtime error, outside the range of
representable values of type 'long'
versions: Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20941
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20935] Support building Python with Clang sanitizer rules
Jeffrey Walton added the comment: On Sat, Mar 15, 2014 at 6:34 PM, Benjamin Peterson rep...@bugs.python.org wrote: Benjamin Peterson added the comment: Just use LDFLAGS. Yeah, I tried that and broke the sanitizer: https://groups.google.com/d/msg/address-sanitizer/cu2WoD1Bwx8/zUoY9GH7oHkJ. The other fallback is to stuff it into CC and CXX. But that breaks some autotool projects. (Python is OK, though). Jeff -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20935 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20935] Support building Python with Clang sanitizer rules
Jeffrey Walton added the comment: On Sat, Mar 15, 2014 at 7:11 PM, Benjamin Peterson rep...@bugs.python.org wrote: Benjamin Peterson added the comment: CFLAGS=-g3 -fsanitize=address LDFLAGS=-fsanitize=address ./configure --with-system-expat make -j4 works for me. Oh,my bad. I thought you meant add the libraries like libclang_rt.asan_osx.a to the flags. Jeff -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20935 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4
New submission from Jeffrey Walton: Downloaded Python-3.4.0rc3. Compiled with Clang 3.4, added -fsanitzie=undefined to CFLAGS. Ran 'make' and 'make check'. Lots of issues, many are duplicates (see below). One or more of these issues might be the reason for `-fwrapv`. This fellow from objimpl.h seems to be the source of many of the issues: #define _Py_AS_GC(o) ((PyGC_Head *)(o)-1) * Modules/gcmodule.c:1718:5: runtime error: left shift of negative value -2 Objects/tupleobject.c:120:5: runtime error: left shift of negative value -3 Objects/typeobject.c:905:9: runtime error: left shift of negative value -3 Objects/dictobject.c:818:5: runtime error: left shift of negative value -3 Objects/methodobject.c:45:5: runtime error: left shift of negative value -3 Objects/listobject.c:178:5: runtime error: left shift of negative value -3 Modules/gcmodule.c:1703:9: runtime error: left shift of negative value -2 Modules/gcmodule.c:1693:5: runtime error: left shift of negative value -3 Objects/descrobject.c:9:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:484:13: runtime error: left shift of negative value -3 Objects/tupleobject.c:195:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:503:13: runtime error: left shift of negative value -4 Objects/exceptions.c:2205:5: runtime error: left shift of negative value -2 Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Objects/frameobject.c:736:5: runtime error: left shift of negative value -3 Objects/funcobject.c:64:5: runtime error: left shift of negative value -3 Objects/methodobject.c:149:5: runtime error: left shift of negative value -2 Objects/funcobject.c:552:5: runtime error: left shift of negative value -2 Objects/descrobject.c:1364:5: runtime error: left shift of negative value -2 Objects/cellobject.c:16:5: runtime error: left shift of negative value -3 Objects/listobject.c:2744:5: runtime error: left shift of negative value -3 Objects/listobject.c:2751:5: runtime error: left shift of negative value -2 Objects/dictobject.c:3232:5: runtime error: left shift of negative value -3 Objects/dictobject.c:2826:5: runtime error: left shift of negative value -3 Objects/exceptions.c:89:5: runtime error: left shift of negative value -2 Objects/classobject.c:68:5: runtime error: left shift of negative value -3 Objects/classobject.c:193:5: runtime error: left shift of negative value -2 Objects/tupleobject.c:1079:5: runtime error: left shift of negative value -3 Objects/genobject.c:526:5: runtime error: left shift of negative value -3 Objects/tupleobject.c:948:5: runtime error: left shift of negative value -2 Objects/genobject.c:48:5: runtime error: left shift of negative value -2 Objects/genobject.c:53:5: runtime error: left shift of negative value -3 Objects/genobject.c:58:5: runtime error: left shift of negative value -2 Objects/cellobject.c:49:5: runtime error: left shift of negative value -2 Objects/typeobject.c:1170:9: runtime error: left shift of negative value -3 Objects/exceptions.c:662:5: runtime error: left shift of negative value -2 Objects/exceptions.c:1010:5: runtime error: left shift of negative value -2 Objects/exceptions.c:513:5: runtime error: left shift of negative value -2 ./Modules/_io/fileio.c:479:5: runtime error: left shift of negative value -2 Objects/setobject.c:944:5: runtime error: left shift of negative value -3 Objects/unicodeobject.c:15267:5: runtime error: left shift of negative value -3 Objects/unicodeobject.c:15128:5: runtime error: left shift of negative value -2 Objects/typeobject.c:6720:5: runtime error: left shift of negative value -2 Objects/bytesobject.c:3006:5: runtime error: left shift of negative value -3 Objects/bytesobject.c:2869:5: runtime error: left shift of negative value -2 Objects/bytearrayobject.c:3094:5: runtime error: left shift of negative value -3 Objects/bytearrayobject.c:2959:5: runtime error: left shift of negative value -2 Objects/descrobject.c:1001:9: runtime error: left shift of negative value -3 Objects/descrobject.c:873:5: runtime error: left shift of negative value -2 Modules/gcmodule.c:861:13: runtime error: left shift of negative value -3 Objects/typeobject.c:2865:5: runtime error: left shift of negative value -2 Objects/dictobject.c:2643:9: runtime error: left shift of negative value -2 Objects/descrobject.c:943:5: runtime error: left shift of negative value -3 Objects/iterobject.c:26:5: runtime error: left shift of negative value -3 Objects/iterobject.c:33:5: runtime error: left shift of negative value -2 Objects/memoryobject.c:74:5: runtime error: left shift of negative value -3 Objects/memoryobject.c:639:5: runtime error: left shift of negative value -3 Objects/memoryobject.c:1060:5: runtime error: left shift of negative value -2 Objects/memoryobject.c:108:5: runtime error: left shift of negative value -2 ./Modules/_io/bufferedio.c:402:5: runtime error: left shift of negative value -2 ./Modules/_io/textio.c:1156
[issue20929] Undefined behavior flagged by Clang 3.4
Changes by Jeffrey Walton noloa...@gmail.com: Added file: http://bugs.python.org/file34426/python-3-4-make-test.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20930] Debian 7.3: This platform's pyconfig.h needs to define PY_FORMAT_LONG_LONG
Jeffrey Walton added the comment: Defining PY_FORMAT_LONG_LONG had a side effect: /usr/local/bin/clang -c -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -g3 -fsanitize=undefined -DPY_FORMAT_LONG_LONG=1 -g3 -fsanitize=undefined -DPY_FORMAT_LONG_LONG=1 -I. -IInclude -I./Include-DPy_BUILD_CORE -o Objects/unicodeobject.o Objects/unicodeobject.c Objects/unicodeobject.c:2325:15: warning: incompatible integer to pointer conversion initializing 'char *' with an expression of type 'int' [-Wint-conversion] char *f = PY_FORMAT_LONG_LONG; ^ ~~~ * export CC=/usr/local/bin/clang export CXX=/usr/local/bin/clang++ export CFLAGS=-g3 -fsanitize=undefined -DPY_FORMAT_LONG_LONG=1 export CXXFLAGS=-g3 -fsanitize=undefined -fno-sanitize=vptr -DPY_FORMAT_LONG_LONG=1 ... ./configure --disable-ipv6 make -- hgrepos: +219 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20930 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4
Jeffrey Walton added the comment: Could you check if the current default branch of mercurial fixes the problems? Checkout is complete. Working through a build now. Could you look at http://bugs.python.org/issue20930 and advise on the best way to proceed? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4
Jeffrey Walton added the comment: The big list of issues earlier has been parred down to the following after `make`: Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Objects/listobject.c:2046:22: runtime error: index 623 out of bounds for type 'PyObject *[256]' Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' `make test` still needs to be run. I'm going to open another bug report since this is a different branch. Is that OK? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4
Jeffrey Walton added the comment: What branch are you building now? Python 3.5 from `hg clone http://hg.python.org/cpython`. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4 (Python 3.4-RC3)
Jeffrey Walton added the comment: Updated title to reflect Python 3.4-RC3. -- title: Undefined behavior flagged by Clang 3.4 - Undefined behavior flagged by Clang 3.4 (Python 3.4-RC3) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4 (Python 3.4-RC3)
Jeffrey Walton added the comment: Well, that's 3.4.1 atm. My bad. I managed to download a file named Python-3.4.0rc3.tgz this morning (the tarball is still in my downloads). I'm not sure from where since http://www.python.org/download/ does not have it -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20932] Undefined behavior flagged by Clang 3.4 (Python 3.5 from hg)
New submission from Jeffrey Walton: Checked out Python-3.5 from mercurial. Compiled with Clang 3.4, added -fsanitzie=undefined to CFLAGS. Ran 'make' and 'make check'. A few issues during `make`: Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Objects/listobject.c:2046:22: runtime error: index 623 out of bounds for type 'PyObject *[256]' Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Similar issues with `make test`, with the addition of some mis-aligned pointers. A digest is below, and the tagCDataObject alignment issue dominates the issues. The tests ended early due to a segfault. $ make test ... [ 19/389/1] test_struct Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' /home/jwalton/Desktop/cpython-checkout/Objects/floatobject.c:2028: runtime error: value 3.40282e+38 is outside the range of representable values of type 'float' /home/jwalton/Desktop/cpython-checkout/Modules/_struct.c:792:15: runtime error: left shift of 72057594037927935 by 8 places cannot be represented in type 'long long' ... [ 43/389/2] test_cmd_line Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' /home/jwalton/Desktop/cpython-checkout/Modules/_ctypes/_ctypes.c:2890:10: runtime error: member access within misaligned address 0x2b54033f1cc8 for type 'CDataObject' (aka 'struct tagCDataObject'), which requires 16 byte alignment 0x2b54033f1cc8: note: pointer points here ff ff ff ff 01 00 00 00 00 00 00 00 68 06 43 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ^ test test_cmd_line failed -- multiple errors occurred; run in verbose mode for details ... [ 45/389/2] test_datetime Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' /home/jwalton/Desktop/cpython-checkout/Python/pytime.c:218: runtime error: value -1e+200 is outside the range of representable values of type 'long' /home/jwalton/Desktop/cpython-checkout/Python/pytime.c:218: runtime error: value 1e+200 is outside the range of representable values of type 'long' ... test test_traceback failed -- Traceback (most recent call last): File /home/jwalton/Desktop/cpython-checkout/Lib/test/test_traceback.py, line 155, in test_encoded_file do_test(, foo, ascii, 3) File /home/jwalton/Desktop/cpython-checkout/Lib/test/test_traceback.py, line 132, in do_test stdout = stdout.decode(output_encoding).splitlines() LookupError: unknown encoding: Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' ... [130/389/5] test_capi Objects/longobject.c:40:42: runtime error: index -3 out of bounds for type 'PyLongObject [262]' Fatal Python error: Segmentation fault Current thread 0x2b6eaf8c7b20 (most recent call first): File /home/jwalton/Desktop/cpython-checkout/Lib/test/test_capi.py, line 466 in test__testcapi File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/case.py, line 574 in run File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/case.py, line 622 in __call__ File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/suite.py, line 125 in run File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/suite.py, line 87 in __call__ File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/suite.py, line 125 in run File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/suite.py, line 87 in __call__ File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/suite.py, line 125 in run File /home/jwalton/Desktop/cpython-checkout/Lib/unittest/suite.py, line 87 in __call__ File /home/jwalton/Desktop/cpython-checkout/Lib/test/support/__init__.py, line 1584 in run File /home/jwalton/Desktop/cpython-checkout/Lib/test/support/__init__.py, line 1685 in _run_suite File /home/jwalton/Desktop/cpython-checkout/Lib/test/support/__init__.py, line 1719 in run_unittest File /home/jwalton/Desktop/cpython-checkout/Lib/test/regrtest.py, line 1277 in lambda File /home/jwalton/Desktop/cpython-checkout/Lib/test/regrtest.py, line 1278 in runtest_inner File /home/jwalton/Desktop/cpython-checkout/Lib/test/regrtest.py, line 978 in runtest File /home/jwalton/Desktop/cpython-checkout/Lib/test/regrtest.py, line 532 in main File /home/jwalton/Desktop/cpython-checkout/Lib/test/regrtest.py, line 1562 in main_in_temp_cwd File /home/jwalton/Desktop/cpython-checkout/Lib/test/regrtest.py, line 1587 in module File /home/jwalton/Desktop/cpython-checkout/Lib/runpy.py, line 86 in _run_code File /home/jwalton/Desktop/cpython-checkout/Lib/runpy.py, line 171 in _run_module_as_main Traceback (most recent call last): File /home/jwalton/Desktop/cpython
[issue20932] Undefined behavior flagged by Clang 3.4 (Python 3.5 from hg)
Changes by Jeffrey Walton noloa...@gmail.com: Added file: http://bugs.python.org/file34429/python-3.5-make-test.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20932 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4 (Python 3.4-RC3)
Jeffrey Walton added the comment: Also see http://bugs.python.org/issue20932. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20932] Undefined behavior flagged by Clang 3.4 (Python 3.5 from hg)
Jeffrey Walton added the comment:
Pulled the latest after BP cleared some more runtime errors:
$ hg summary
parent: 89662:7ce22d0899e4 tip
merge 3.3
branch: default
commit: 2 modified
update: (current)
Misaligned accesses dominate with over 11K:
$ cat python-3.5-make-test.txt | grep misaligned | wc -l
11320
*
$ make test
...
[ 21/389] test_sort
Objects/listobject.c:1973:30: runtime error: index 257 out of bounds for type
'PyObject *[256]'
...
[ 31/389] test_xml_etree
/home/jwalton/Desktop/cpython-checkout/Modules/expat/xmltok.c:1396:11: runtime
error: left shift of 1 by 31 places cannot be represented in type 'int'
/home/jwalton/Desktop/cpython-checkout/Modules/expat/xmltok.c:1398:16: runtime
error: left shift of 1 by 31 places cannot be represented in type 'int'
...
[ 33/389] test_aifc
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1587:39: runtime
error: left shift of negative value -24
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1546:19: runtime
error: left shift of negative value -24
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1513:43: runtime
error: left shift of negative value -24
...
[ 61/389] test_httplib
/home/jwalton/Desktop/cpython-checkout/Lib/socket.py:444: ResourceWarning:
unclosed ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=2049, proto=6,
laddr=('172.16.1.26', 51321), raddr=('82.94.164.164', 443)
self._sock = None
...
[ 99/389] test_audioop
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1060:20: runtime
error: left shift of negative value -70
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1061:20: runtime
error: left shift of negative value -70
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1811:9: runtime error:
left shift of negative value -22
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1811:9: runtime error:
left shift of negative value -22
...
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:422:11: runtime error:
left shift of negative value -1
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1639:19: runtime
error: left shift of negative value -69
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1639:19: runtime
error: left shift of negative value -17767
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1639:19: runtime
error: left shift of negative value -70
/home/jwalton/Desktop/cpython-checkout/Modules/audioop.c:1639:19: runtime
error: left shift of negative value -4548489
...
[109/389] test_unicode
/home/jwalton/Desktop/cpython-checkout/Modules/_ctypes/_ctypes.c:2890:10:
runtime error: member access within misaligned address 0x2ac5068c24f8 for type
'CDataObject' (aka 'struct tagCDataObject'), which requires 16 byte alignment
0x2ac5068c24f8: note: pointer points here
ff ff ff ff 01 00 00 00 00 00 00 00 78 e4 af 02 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00
^
/home/jwalton/Desktop/cpython-checkout/Modules/_ctypes/_ctypes.c:2891:10:
runtime error: member access within misaligned address 0x2ac5068c24f8 for type
'CDataObject' (aka 'struct tagCDataObject'), which requires 16 byte alignment
0x2ac5068c24f8: note: pointer points here
ff ff ff ff 01 00 00 00 00 00 00 00 78 e4 af 02 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00
^
...
/home/jwalton/Desktop/cpython-checkout/Modules/_ctypes/_ctypes.c:3862:5:
runtime error: member access within misaligned address 0x2ae9f1252e58 for type
'PyCFuncPtrObject', which requires 16 byte alignment
0x2ae9f1252e58: note: pointer points here
00 00 00 00 01 00 00 00 00 00 00 00 58 0c 8e 02 00 00 00 00 a8 2e 25 f1 e9
2a 00 00 01 00 00 00
^
/home/jwalton/Desktop/cpython-checkout/Modules/_ctypes/_ctypes.c:3863:5:
runtime error: member access within misaligned address 0x2ae9f1252e58 for type
'PyCFuncPtrObject', which requires 16 byte alignment
0x2ae9f1252e58: note: pointer points here
00 00 00 00 01 00 00 00 00 00 00 00 58 0c 8e 02 00 00 00 00 a8 2e 25 f1 e9
2a 00 00 01 00 00 00
^
...
2 tests failed:
test_faulthandler test_venv
2 tests altered the execution environment:
test___all__ test_warnings
25 tests skipped:
test_bz2 test_crypt test_curses test_dbm_gnu test_dbm_ndbm
test_devpoll test_gdb test_gzip test_idle test_kqueue test_lzma
test_msilib test_nis test_ossaudiodev test_readline test_sqlite
test_startfile test_tcl test_tk test_ttk_guionly test_ttk_textonly
test_winreg test_winsound test_zipfile64 test_zlib
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20932
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20932] Undefined behavior flagged by Clang 3.4 (Python 3.5 from hg)
Jeffrey Walton added the comment: This is just the same as #20929, I believe? I think most of the issues in the 20929 report (Python 3.4-RC3) are present in this report. But under this report, I can re-test as you check in the fixes. (Python 3.4-RC3 is fixed in time from my perspective). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20932 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20932] Undefined behavior flagged by Clang 3.4 (Python 3.5 from hg)
Jeffrey Walton added the comment: Here's another one I missed. The first is a problem due to silent truncation when casting from the double 3.40282e+38 to a float (or Clang is wrong). [ 10/389] test_struct /home/jwalton/Desktop/cpython-checkout/Objects/floatobject.c:2028: runtime error: value 3.40282e+38 is outside the range of representable values of type 'float' /home/jwalton/Desktop/cpython-checkout/Modules/_struct.c:792:15: runtime error: left shift of 72057594037927935 by 8 places cannot be represented in type 'long long' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20932 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20929] Undefined behavior flagged by Clang 3.4 (Python 3.4-RC3)
Jeffrey Walton added the comment: On Fri, Mar 14, 2014 at 10:28 PM, Benjamin Peterson rep...@bugs.python.org wrote: Benjamin Peterson added the comment: Well, that's 3.4.1 atm. Here's why I got that stale version: https://www.google.com/search?q=download+python+3.4. The first hit is RC3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20929 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
