[issue37362] test_gdb must not fail on "unexpected" messages written into stderr
Dave Malcolm added the comment: I think when I wrote this I was over-optimistically thinking that we could just add more patterns, but if it's becoming a pain, then your approach looks good to me. -- nosy: +dmalcolm ___ Python tracker <https://bugs.python.org/issue37362> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)
Dave Malcolm added the comment: Thanks! -- ___ Python tracker <https://bugs.python.org/issue9263> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9635] Add Py_BREAKPOINT and sys._breakpoint hooks
Dave Malcolm added the comment: On Fri, 2018-02-23 at 00:16 +, Cheryl Sabella wrote: > Cheryl Sabella added the comment: > > Did PEP553 make this issue obsolete? I *think* they have slightly different scope: if I'm reading it right, PEP553 is about injecting a breakpoint into the Python debugger. This proposal was about injecting a lower-level breakpoint for debugging CPython itself (for e.g. gdb to handle). The idea was to make it easier to, say, step through a particular CPython construct at the C level by injecting a breakpoint right before it: def test_something(): # lots of setup sys.c_level_breakpoint() # whatever comes next so that sys.c_level_breakpoint drops you into, say, gdb, and from there you can step through the following Python code at the C level, without having to express stepping through all the setup at the C/gdb level. Hope that makes sense. That said, I'm full-time on gcc these days, and unlikely to pursue this from the CPython side. -- ___ Python tracker <https://bugs.python.org/issue9635> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31237] test_gdb disables 25% of tests in optimized builds
Dave Malcolm added the comment: The problem is that there are so many variables: * which version of which compiler * optimization flags * which version of gdb * which CPU architecture etc (and the compiler and/or gdb could be carrying patches from downstream distributors...) All of these can affect the debugging experience; as soon as optimizations are turned on it's very hard to predict exactly how well we can capture the frame information in the debugger. Hence it makes sense to skip these tests for optimized builds, to avoid creating noise in the buildbots. Downstream distributions might want to re-enable the tests, if they have sufficient control over the compiler and debugger versions that are in use, I guess. Or maybe there's a way to express some of the above sanely? Not sure. Hope this is constructive. -- ___ Python tracker <http://bugs.python.org/issue31237> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21590] Systemtap and DTrace support
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue21590> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment: On Mon, 2014-05-19 at 14:15 +, Dave Malcolm wrote: > Dave Malcolm added the comment: > > On Mon, 2014-05-19 at 11:10 +, Bohuslav "Slavek" Kabrda wrote: > > Bohuslav "Slavek" Kabrda added the comment: > > > > Hi, > > I'd like to take this over after Dave Malcolm. I don't see any issues that > > haven't been resolved, so my question is: What else can I do to make this > > patch acceptable? > > I'm attaching a rebased version of this patch that applies to current > > default branch (FWIW, we use this patch downstream in Fedora in our > > python3-debug build and it works well) > > > > -- > > Added file: > > http://bugs.python.org/file35293/cpython-systemtap-2014-05-19.patch > > Is this attachment missing the "instrumentation.rst"? FWIW I see it within http://bugs.python.org/file26074/cpython-systemtap-2012-06-21-001.patch > There are also a couple of example scripts we ship in the RPMs, iirc. -- ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment: On Mon, 2014-05-19 at 11:10 +, Bohuslav "Slavek" Kabrda wrote: > Bohuslav "Slavek" Kabrda added the comment: > > Hi, > I'd like to take this over after Dave Malcolm. I don't see any issues that > haven't been resolved, so my question is: What else can I do to make this > patch acceptable? > I'm attaching a rebased version of this patch that applies to current default > branch (FWIW, we use this patch downstream in Fedora in our python3-debug > build and it works well) > > -- > Added file: > http://bugs.python.org/file35293/cpython-systemtap-2014-05-19.patch Is this attachment missing the "instrumentation.rst"? There are also a couple of example scripts we ship in the RPMs, iirc. -- ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20162] Test test_hash_distribution fails on RHEL 6.5 / ppc64
Dave Malcolm added the comment: On Tue, 2014-01-07 at 16:30 +, Yury V. Zaytsev wrote: > Yury V. Zaytsev added the comment: > > After lots of fiddling, I can tell you what's wrong with the macro: > apparently it's a compiler bug, visible at -O2 and disappearing at -O1. Can you reduce the suspected compiler bug to a minimal reproducer? Please then run it through the preprocessor using gcc's -E option, and then attach it to this bug. What exact version of the compiler are you using, and with what flags? -- ___ Python tracker <http://bugs.python.org/issue20162> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19901] tests fail due to unsupported SO_REUSEPORT when building Python 3.3.2-r2
Dave Malcolm added the comment: [FWIW, this looks similar to an issue I ran into on Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=913732 which was due to a mismatch between the kernel headers on the system vs the actually running kernel. I patched around it there with a downstream-only patch (our builds are done on chroots on RHEL-5 boxes, so such mismatches tend to occur)] -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue19901> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19743] test_gdb failures
Dave Malcolm added the comment: FWIW, I feel that it's worth just expecting failures with an *optimized* build: with an optimizing compiler, there's likely to always be some program counter location where the debugger is going to get confused for some variables. Given umpteen different compiler variants, debugger variants etc, this is never going to be perfect. So IMHO we should run the test on a --pydebug build, but accept that there will be some failures when optimization is on. -- ___ Python tracker <http://bugs.python.org/issue19743> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18772] Fix gdb plugin for new sets dummy object
Dave Malcolm added the comment: Antoine's patch looks reasonable to me, FWIW. -- ___ Python tracker <http://bugs.python.org/issue18772> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
Dave Malcolm added the comment: On Mon, 2013-07-29 at 14:01 +, Ronald Oussoren wrote: > Ronald Oussoren added the comment: > > The attached patch for the benchmark README.txt should document the status > quo. > > -- > Added file: http://bugs.python.org/file31073/issue-18181-full-v3.txt It looks like you attached the wrong patch: the patch you attached seems to be unrelated to benchmarking (and presumably relates to issue 18181) -- ___ Python tracker <http://bugs.python.org/issue17449> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18213] py-bt errors on backtrace with PyRun_SimpleString and friends
Dave Malcolm added the comment: pmuldoon: did you truncate the output of bt? (or did the superior gdb you're using do this behind the scenes? I know you hack on gdb itself, and this looks a superior gdb debugging an inferior gdb). The reason for this error: Python Exception (2, 'No such file or directory', ''): is that py-bt is attempting to print the line of source code that the frame is at, but the source was provided as a string, not a file, presumably due to a call to PyRun_SimpleString et al. Python sets the filename of such code to be the dummy name "", and py-bt attempts to use that as a real filename and open it, failing (unless you happen to have a file named "" in which case it tries to list it). -- assignee: -> dmalcolm title: py-bt errors on backtrace -> py-bt errors on backtrace with PyRun_SimpleString and friends ___ Python tracker <http://bugs.python.org/issue18213> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17833] test_gdb broken PPC64 Linux
Changes by Dave Malcolm : -- resolution: -> fixed stage: commit review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue17833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17833] test_gdb broken PPC64 Linux
Dave Malcolm added the comment: Can you try this candidate fix? -- Added file: http://bugs.python.org/file30007/possible-fix.patch ___ Python tracker <http://bugs.python.org/issue17833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17833] test_gdb broken PPC64 Linux
Dave Malcolm added the comment:
Looking at test_gdb.out, the issue is that the threads waiting for the GIL have
e.g. this at the top of their backtrace:
Thread 2 (Thread 0x3fffb14af200 (LWP 37119)):
#0 0x008075181ea8 in __pthread_cond_timedwait (cond=0x103ea140 ,
mutex=0x103ea170 , abstime=0x3fffb14abf68) at
pthread_cond_timedwait.c:167
#1 0x100f59f8 in PyCOND_TIMEDWAIT (cond=0x103ea140 ,
mut=0x103ea170 , us=5000) at
/home/dje/src/cpython/Python/condvar.h:103
#2 0x100f5ffc in take_gil (tstate=0x1043b020) at
/home/dje/src/cpython/Python/ceval_gil.h:224
i.e. the topmost frame has name "__pthread_cond_timedwait" (two leading
underscores.
However, the gdb hooks (in Tools/gdb/libpython.py) have:
def is_waiting_for_gil(self):
'''Is this frame waiting on the GIL?'''
# This assumes the _POSIX_THREADS version of Python/ceval_gil.h:
name = self._gdbframe.name()
if name:
return name.startswith('pthread_cond_timedwait')
i.e. no underscores, hence that name.startswith conditional doesn't match.
--
___
Python tracker
<http://bugs.python.org/issue17833>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17833] test_gdb broken PPC64 Linux
Dave Malcolm added the comment: Thanks. I can't see from that output what's going wrong. Can you apply the following patch, which (I hope) will print more detailed info where the failure happens. -- keywords: +patch Added file: http://bugs.python.org/file30005/find-more-information.patch ___ Python tracker <http://bugs.python.org/issue17833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17833] test_gdb broken PPC64 Linux
Dave Malcolm added the comment: Is this an optimized or a debug build? (aka --with-pydebug) What are the optimization flags passed to the C compiler? -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue17833> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17547] "checking whether gcc supports ParseTuple __format__... " erroneously returns yes with gcc 4.8
Dave Malcolm added the comment: On Wed, 2013-04-24 at 10:48 +, Alex Leach wrote: > Alex Leach added the comment: > > I don't think I can tell you anything you don't know already, but ... > > On Tue, 23 Apr 2013 19:38:18 +0100, Dave Malcolm > wrote: > > > BTW, is that GCC format checking code available anywhere? > > Is this what you mean? > > http://gcc.gnu.org/viewcvs/gcc/trunk/gcc/c-family/c-format.c?annotate=193304&pathrev=193304 > > I got taken straight there from the link you sent, above... [snip] > Am I right in thinking that it was an out-of-tree patch to GCC, from the > > pre-plugin days? > Sorry that I was a bit vague. GCC's __attribute((format(i, j, k))__ takes 3 parameters. The first parameter is a string code naming the mini-language expressing valid format strings and their mapping to the types that follow in the varargs. I was asking about the string code "PyArg_ParseTuple" used in the CPython configure test: the link above is the implementation of GCC's builtin format string codes: and AFAIK the only ones it "knows" about there are the ones in static const format_kind_info format_types_orig[] = in the above file (lines 808-873 of gcc/c-family/c-format.c) which are codes for printf, fprintf, various internal gcc APIs, NSString, scanf, strftime, and strfmon. i.e. I believe that the "PyArg_ParseTuple" code is from some 3rd party patch to GCC that never made it into GCC mainline. Does this code still exist anywhere? [Not that this has any bearing on this bug and the proposed patch: I'm a big fan of compiler attributes, and wouldn't want to remove them from the API.] Hope the above makes sense -- ___ Python tracker <http://bugs.python.org/issue17547> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17547] "checking whether gcc supports ParseTuple __format__... " erroneously returns yes with gcc 4.8
Dave Malcolm added the comment: BTW, is that GCC format checking code available anywhere? Am I right in thinking that it was an out-of-tree patch to GCC, from the pre-plugin days? [My cpychecker code adds some similar checking, but it doesn't use this attribute] -- ___ Python tracker <http://bugs.python.org/issue17547> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17547] "checking whether gcc supports ParseTuple __format__... " erroneously returns yes with gcc 4.8
Dave Malcolm added the comment: On Sat, 2013-04-20 at 21:25 +, Ned Deily wrote: > Ned Deily added the comment: > > Dave, any reason this shouldn't go into the imminent 2.7.5 and 3.3.2 releases? I was trying to think of one; the only reason I can think of is if there are any compilers out there that don't recognize "-Wformat", or have different behaviors for this test. That said, if "-Wformat" breaks the compile, then the configure test should fail, and thus all should be well. So I can't think of any issue (famous last words...) -- ___ Python tracker <http://bugs.python.org/issue17547> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17772] test_gdb doesn't detect a gdb built with python3.3 (or higher)
Dave Malcolm added the comment: Thanks. Does upstream gdb have a plan dcoumented somewhere for how to deal with all of the FOO-gdb.py files. Are they expected to be coded to the common Python 2/3 subset? -- ___ Python tracker <http://bugs.python.org/issue17772> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17772] test_gdb doesn't detect a gdb built with python3.3 (or higher)
Dave Malcolm added the comment: I didn't know that gdb supported embedding Python 3. Is this a set of patches you're applying downstream, or an official gdb feature? If so, it means everyone coding to the gdb API needs to somehow make their FOO-gdb.py files be Python 3-compatible. Currently the Tools/gdb/libpython.py is Python 2 compatible, and I'm sure there are going to be incompatibilities with python 3. -- ___ Python tracker <http://bugs.python.org/issue17772> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17547] "checking whether gcc supports ParseTuple __format__... " erroneously returns yes with gcc 4.8
New submission from Dave Malcolm: It appears that gcc's -Wformat warning changed from being on by default in gcc 4.7 and earlier to being off by default in gcc 4.8, needing to be enabled with -Wformat (or -Wall), if I'm reading: http://gcc.gnu.org/viewcvs/gcc?view=revision&revision=193304 correctly This breaks CPython's configure.ac test: "Check whether GCC supports PyArg_ParseTuple format" in that the warning the test is supposed to emit is guarded by -Wformat, hence with gcc 4.8 this test always emits: checking whether gcc supports ParseTuple __format__... yes despite the fact that gcc doesn't. This leads to the pyconfig.h defining HAVE_ATTRIBUTE_FORMAT_PARSETUPLE, and thus we get this error when trying later use the python build to build a C extension module where -Wformat (or -Wall) *is* supplied: In file included from /usr/include/python2.7/Python.h:126:0, from isys.c:20: /usr/include/python2.7/modsupport.h:28:1: error: 'PyArg_ParseTuple' is an unrecognized format function type [-Werror=format=] PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3); Am attaching a work-in-progress patch to supply the missing argument at configure time Note to self: am tracking this downstream as https://bugzilla.redhat.com/show_bug.cgi?id=927358 -- components: Build files: 00175-fix-configure-Wformat.patch keywords: patch messages: 185226 nosy: dmalcolm priority: normal severity: normal stage: patch review status: open title: "checking whether gcc supports ParseTuple __format__... " erroneously returns yes with gcc 4.8 type: compile error versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file29574/00175-fix-configure-Wformat.patch ___ Python tracker <http://bugs.python.org/issue17547> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17449] dev guide appears not to cover the benchmarking suite
New submission from Dave Malcolm: Does the devguide document the benchmarking suite anywhere? I can't see it anywhere in the index on http://docs.python.org/devguide/ and google doesn't seem to show anything. suggested content: * how to run the benchmarks for a Python 2 implementation * how to run the benchmarks for a Python 3 implementation Sorry if it exists and I missed it -- assignee: brett.cannon components: Devguide messages: 184421 nosy: brett.cannon, dmalcolm, ezio.melotti priority: normal severity: normal status: open title: dev guide appears not to cover the benchmarking suite ___ Python tracker <http://bugs.python.org/issue17449> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Dave Malcolm added the comment:
For reference, quoting PEP 3149:
>>> sysconfig.get_config_var('SO')
'.cpython-32mu.so'
--
nosy: +dmalcolm
___
Python tracker
<http://bugs.python.org/issue16754>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17258] multiprocessing.connection challenge implicitly uses MD5
New submission from Dave Malcolm: Within multiprocessing.connection, deliver_challenge() and answer_challenge() use hmac for a challenge/response. hmac implicitly defaults to using MD5. MD5 should no longer be used for security purposes. See e.g. http://www.kb.cert.org/vuls/id/836068 This fails in a FIPS-compliant environment (e.g. with the patches I apply to hashlib in issue 9216). There's thus a possibility of an attacker defeating the multiprocessing authenticator. I'm attaching a patch which changes multiprocessing to use a clearly identified algorithm (for the day when it needs changing again), hardcoding it as "sha256"; presumably all processes within a multiprocess program that share authkey can share the algorithm. It's not clear to me whether hmac.py should also be changed (this would seem to have tougher backwards-compat concerns). [Note to self: I'm tracking this downstream for RHEL as https://bugzilla.redhat.com/show_bug.cgi?id=879695 (this bug is currently only visible to RH employees)] -- components: Library (Lib) files: avoid-implicit-usage-of-md5-in-multiprocessing.patch keywords: patch messages: 182547 nosy: dmalcolm, sbt priority: normal severity: normal stage: patch review status: open title: multiprocessing.connection challenge implicitly uses MD5 versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file29134/avoid-implicit-usage-of-md5-in-multiprocessing.patch ___ Python tracker <http://bugs.python.org/issue17258> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17206] Py_XDECREF() expands its argument multiple times
New submission from Dave Malcolm:
When running my refcount static analyzer [1] on a large corpus of real-world C
extension code for Python, I ran into the following construct in various places:
Py_XDECREF(PyObject_CallObject(callable, args));
This is bogus code: Py_XDECREF expands its argument multiple times, so after
this goes through the C preprocessor the callable is actually called invoked up
to 4 times, leaking refs to 3 of the results - assuming that none of the calls
fail, and a non pydebug build of CPython (which would expand the argument more
times).
The raw preprocessor output for an optimized Python 2.7 looks like this:
do { if ((PyObject_CallObject(callable, args)) == ((void *)0)) ; else do {
if ( --((PyObject*)(PyObject_CallObject(callable, args)))->ob_refcnt != 0) ;
else ( (*(((PyObject*)((PyObject *)(PyObject_CallObject(callable,
args->ob_type)->tp_dealloc)((PyObject *)((PyObject
*)(PyObject_CallObject(callable, args); } while (0); } while (0);
Cleaned up (and removing some of the casts for clarity) it looks like this:
do {
if ((PyObject_CallObject(callable, args)) == ((void *)0))
;
else
do {
if (--(PyObject_CallObject(callable, args)->ob_refcnt) != 0)
;
else
(*(PyObject_CallObject(callable, args)->ob_type)->tp_dealloc)
PyObject_CallObject(callable, args);
} while (0);
} while (0);
which is clearly not what the extension author was expecting.
Should we:
(A) update the macro so that it expands its argument only once, or
(B) warn people not to write code like the above?
Similar considerations apply to the other macros, I guess, but I've seen the
above used "in the wild", I haven't yet seen it for the other macros).
Seen in the wild in:
python-alsa-1.0.25-1.fc17:
pyalsa/alsamixer.c:179
00174 | for (i = 0; i < count; i++) {
00175 | t = PyTuple_New(2);
00176 | if (t) {
00177 | PyTuple_SET_ITEM(t, 0,
PyInt_FromLong(pfd[i].fd));
00178 | PyTuple_SET_ITEM(t, 1,
PyInt_FromLong(pfd[i].events));
00179>| Py_XDECREF(PyObject_CallObject(reg, t));
00180 | Py_DECREF(t);
00181 | }
00182 | }
00183 |
00184 | Py_XDECREF(reg);
pyalsa/alsahcontrol.c:190
00185 | for (i = 0; i < count; i++) {
00186 | t = PyTuple_New(2);
00187 | if (t) {
00188 | PyTuple_SET_ITEM(t, 0,
PyInt_FromLong(pfd[i].fd));
00189 | PyTuple_SET_ITEM(t, 1,
PyInt_FromLong(pfd[i].events));
00190>| Py_XDECREF(PyObject_CallObject(reg, t));
00191 | Py_DECREF(t);
00192 | }
00193 | }
00194 |
00195 | Py_XDECREF(reg);
pyalsa/alsaseq.c:3277
03272 | for (i = 0; i < count; i++) {
03273 | t = PyTuple_New(2);
03274 | if (t) {
03275 | PyTuple_SET_ITEM(t, 0, PyInt_FromLong(pfd[i].fd));
03276 | PyTuple_SET_ITEM(t, 1, PyInt_FromLong(pfd[i].events));
03277>| Py_XDECREF(PyObject_CallObject(reg, t));
03278 | Py_DECREF(t);
03279 | }
03280 | }
03281 |
03282 | Py_XDECREF(reg);
python-4Suite-XML-1.0.2-14.fc17:
Ft/Xml/src/domlette/refcounts.c:80
00075 | /* refcount = this */
00076 | expected = 1;
00077 | }
00078 | else {
00079 | sprintf(buf, "Unexpected object type '%.200s'"
,node->ob_type->tp_name);
00080>| Py_XDECREF(PyObject_CallMethod(tester, "error", "s", buf));
00081 | return 0;
00082 | }
00083 |
00084 | sprintf(buf, "%.200s refcounts", node->ob_type->tp_name);
00085 | return do_test(tester, buf, expected, node->ob_refcnt);
[Note to self: I actually saw this because it uncovered a traceback in
cpychecker, which I fixed as:
http://git.fedorahosted.org/cgit/gcc-python-plugin.git/commit/?id=99fa820487e380b74c2eda1d97facdf2ee6d2f3a
]
[1] https://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html
--
components: Interpreter Core
messages: 182107
nosy: dmalcolm
priority: normal
severity: normal
status: open
title: Py_XDECREF() expands its argument multiple times
versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5
___
Python tracker
<http://bugs.python.org/issue17206>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13390] Hunt memory allocations in addition to reference leaks
Dave Malcolm added the comment: Minor bikeshedding/spelling nit: should "_Py_AllocedBlocks" be changed to "_Py_AllocatedBlocks" (and s/_Py_GetAllocedBlocks/_Py_GetAllocatedBlocks/)? -- ___ Python tracker <http://bugs.python.org/issue13390> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15516] exception-handling bug in PyString_Format
Changes by Dave Malcolm : -- stage: -> patch review ___ Python tracker <http://bugs.python.org/issue15516> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue16612> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15043] test_gdb is disallowed by default security settings in Fedora 17
Dave Malcolm added the comment: In my Fedora Python packages I've been applying this patch: http://pkgs.fedoraproject.org/cgit/python.git/plain/00156-gdb-autoload-safepath.patch which uses this code fragment to detect if gdb has the autoload safe path code, rather than trying to guess it from version numbers (which I *think* is failing for Stefan due to a backport of the autoload stuff in that gdb): def gdb_has_autoload_safepath(): # Recent GDBs will only auto-load scripts from certain safe # locations, so we will need to turn off this protection. # However, if the GDB doesn't have it, then the following # command will generate noise on stderr (rhbz#817072): cmd = "--eval-command=set auto-load safe-path /" p = subprocess.Popen(["gdb", "--batch", cmd], stderr=subprocess.PIPE) _, stderr = p.communicate() return '"on" or "off" expected.' not in stderr HAS_AUTOLOAD_SAFEPATH = gdb_has_autoload_safepath() (etc) -- ___ Python tracker <http://bugs.python.org/issue15043> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16191] ceval cleanup
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue16191> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15778] str(ImportError(b'foo')) fails
Dave Malcolm added the comment: (patch added) -- stage: -> patch review ___ Python tracker <http://bugs.python.org/issue15778> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15778] str(ImportError(b'foo')) fails
Changes by Dave Malcolm : -- keywords: +patch Added file: http://bugs.python.org/file26988/fix-str-of-bogus-ImportError.patch ___ Python tracker <http://bugs.python.org/issue15778> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15778] str(ImportError(b'foo')) fails
Dave Malcolm added the comment: Sorry, it's 6825fd9b00ed -- ___ Python tracker <http://bugs.python.org/issue15778> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15778] str(ImportError(b'foo')) fails
New submission from Dave Malcolm: In Python 3.2 and earlier: >>> str(ImportError(b'foo')) "b'foo'" In Python 3.3: >>> str(ImportError(b'foo')) Traceback (most recent call last): File "", line 1, in TypeError: __str__ returned non-string (type bytes) This appears to be due to commit 76272. I'm attaching a patch which fixes it, though perhaps ImportError should type-check the arguments Motivation: This leads to a failure of the test suite for "docutils" when run under Python 3.3 betas, specifically: == ERROR: test_unicode (test_error_reporting.ErrorStringTests) -- Traceback (most recent call last): File "/home/david/coding/python3.3/docutils/trunk/docutils/test3/test_error_reporting.py", line 154, in test_unicode str(ErrorString(ImportError(self.bs File "/home/david/coding/python3.3/docutils/trunk/docutils/build/lib/docutils/utils/error_reporting.py", line 124, in __str__ super(ErrorString, self).__str__()) File "/home/david/coding/python3.3/docutils/trunk/docutils/build/lib/docutils/utils/error_reporting.py", line 74, in __str__ return str(self.data) TypeError: __str__ returned non-string (type bytes) See http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/test/test_error_reporting.py?revision=7464&view=markup for the test code. Arguably docutils could just pick a different exception subclass. -- messages: 169042 nosy: brett.cannon, dmalcolm priority: normal severity: normal status: open title: str(ImportError(b'foo')) fails type: behavior versions: Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue15778> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15610] PyImport_ImportModuleEx always fails in 3.3 with "ValueError: level must be >= 0"
Dave Malcolm added the comment: On Thu, 2012-08-09 at 21:04 +, STINNER Victor wrote: > STINNER Victor added the comment: > > Oh, I didn't realize that the documentation says that the default value is -1. > http://docs.python.org/library/functions.html#__import__ > > "level specifies whether to use absolute or relative imports. The default is > -1 which indicates both absolute and relative imports will be attempted. 0 > means only perform absolute imports. Positive values for level indicate the > number of parent directories to search relative to the directory of the > module calling __import__()." That's the python 2 documentation The 3.3 docs here: http://docs.python.org/dev/library/functions.html#__import__ say "Changed in version 3.3: Negative values for level are no longer supported (which also changes the default value to 0)." > We should probably tolerate -1, or just drop the exception. -- ___ Python tracker <http://bugs.python.org/issue15610> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15610] PyImport_ImportModuleEx always fails in 3.3 with "ValueError: level must be >= 0"
Dave Malcolm added the comment: (FWIW, this was observed when compiling pygobject-3.3.4 against Python-3.3.0b1) -- ___ Python tracker <http://bugs.python.org/issue15610> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15610] PyImport_ImportModuleEx always fails in 3.3 with "ValueError: level must be >= 0"
New submission from Dave Malcolm:
I've been testing various 3rd-party python code against 3.3b1, and see
ValueError: level must be >= 0
exceptions where C code is using PyImport_ImportModuleEx.
PyImport_ImportModuleEx reads as
63 #define PyImport_ImportModuleEx(n, g, l, f) \
64 PyImport_ImportModuleLevel(n, g, l, f, -1)
within http://hg.python.org/cpython/file/aaa68dce117e/Include/import.h as of
now (2012-08-09)
Within PyImport_ImportModuleLevel there's this check:
1280 if (level < 0) {
1281 PyErr_SetString(PyExc_ValueError, "level must be >= 0");
1282 goto error;
1283 }
which thus always fires.
So it would seem that currently any usage of PyImport_ImportModuleEx will fail.
--
components: Interpreter Core
messages: 167828
nosy: dmalcolm
priority: normal
severity: normal
status: open
title: PyImport_ImportModuleEx always fails in 3.3 with "ValueError: level must
be >= 0"
type: behavior
versions: Python 3.3, Python 3.4
___
Python tracker
<http://bugs.python.org/issue15610>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9635] Add Py_BREAKPOINT and sys._breakpoint hooks
Dave Malcolm added the comment:
On Tue, 2010-11-02 at 17:25 +, Antoine Pitrou wrote:
> Antoine Pitrou added the comment:
>
> I would rename Py_BREAKPOINT to _Py_BREAKPOINT since we don't really want to
> support this. Also, why do you allow any arguments to sys._breakpoint()?
Agreed about _Py_BREAKPOINT.
The reason for allowing arguments to sys._breakpoint() is so that the
developer can pass in arbitrary objects (or collections of objects),
which can then be easily inspected from the debugger. Does that seem
sane?
Maybe the docs should read:
--
This may be of use when tracking down bugs: the breakpoint can be
guarded by Python-level conditionals, and supply Python-generated data::
if foo and bar and not baz:
sys._breakpoint(some_func(foo, bar, baz))
In the above example, if the given python conditional holds (and no
exception is raised calling "some_func"), execution will halt under
the debugger within Python/sysmodule.c:sys_breakpoint, and the result of
some_func() will be inspectable in the debugger as
((PyTupleObject*)args)[0]
static PyObject *
sys_breakpoint(PyObject *self, PyObject *args)
{
_Py_BREAKPOINT();
Py_RETURN_NONE;
}
It can also be useful to call when debugging the CPython interpreter: if
you add a call to this function immediately before the code of interest,
you can step out of sys_breakpoint and then step through subsequent
execution.
--
I thought about it making it METH_O instead (to make it easier to look
at a single object), but then you'd be forced to pass an object in when
using it, I think (though None should work).
--
___
Python tracker
<http://bugs.python.org/issue9635>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15540] Python 3.3 and numpy
Changes by Dave Malcolm : -- nosy: +teoliphant ___ Python tracker <http://bugs.python.org/issue15540> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15540] Python 3.3 and numpy
Changes by Dave Malcolm : -- nosy: +ncoghlan ___ Python tracker <http://bugs.python.org/issue15540> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15540] Python 3.3 and numpy
Changes by Dave Malcolm : -- nosy: +haypo, loewis ___ Python tracker <http://bugs.python.org/issue15540> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15540] Python 3.3 and numpy
Changes by Dave Malcolm : Added file: http://bugs.python.org/file26669/hack-out-test-against-MAX_UNICODE-from-cpython-3.3.patch ___ Python tracker <http://bugs.python.org/issue15540> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15540] Python 3.3 and numpy
New submission from Dave Malcolm: I've been trying to get numpy working with Python 3.3, and to so I had to make some changes to CPython - hence I'm posting this to the Python bug tracker. numpy pokes at the insides of PyUnicodeObject in a few places and is thus affected by the PEP 393 changes in Python 3.3 I'm attaching my latest work-in-progress patch for numpy, which mostly works (it has 3 remaining errors). AIUI, the "numpy.str_" type subclasses PyUnicodeObject but with its own custom allocator, which takes a size (this is called in PyArray_Scalar when type_num == NPY_UNICODE). unicode_new_subtype calls tp_alloc but passes in 0 for the size, so we can't use that. So I had to reimplement parts of unicode creation in-place within numpy's PyArray_Scalar, copying macros from out of cpython's unicodeobject.c The other wart is that, AIUI, numpy supports byte-swapping the values within an array, and when this is done for a unicode array, it byte-swaps the 4-byte UCS4 values. At that point, it's unlikely that the resulting 4-byte values are below 0x10, leading to various failures from inside CPython's unicode handling. So I hacked those test from out of CPython :) I'm attaching the diff I've got against cpython (clearly just a hack at this stage). I may of course be misunderstanding the insides of numpy. With these changes, the numpy test suite runs, with just these remaining errors: == ERROR: Ticket #16 -- Traceback (most recent call last): File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/core/tests/test_regression.py", line 41, in test_pickle_transposed b = pickle.load(f) EOFError == ERROR: test_power_zero (test_umath.TestPower) -- Traceback (most recent call last): File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/core/tests/test_umath.py", line 139, in test_power_zero assert_complex_equal(np.power(zero, 0+1j), cnan) RuntimeWarning: invalid value encountered in power == ERROR: Failure: ValueError (can't handle version 187 of numpy.ndarray pickle) -- Traceback (most recent call last): File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/nose-1.1.2-py3.3.egg/nose/failure.py", line 37, in runTest raise self.exc_class(self.exc_val).with_traceback(self.tb) File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/nose-1.1.2-py3.3.egg/nose/loader.py", line 232, in generate for test in g(): File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/lib/tests/test_format.py", line 429, in test_roundtrip arr2 = roundtrip(arr) File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/lib/tests/test_format.py", line 420, in roundtrip arr2 = format.read_array(f2) File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/lib/format.py", line 449, in read_array array = pickle.load(fp) ValueError: can't handle version 187 of numpy.ndarray pickle -- Ran 4776 tests in 68.189s FAILED (KNOWNFAIL=6, SKIP=1, errors=3) -- components: Interpreter Core files: get-numpy-working-with-python-3.3.patch keywords: patch messages: 167256 nosy: dmalcolm priority: normal severity: normal status: open title: Python 3.3 and numpy versions: Python 3.3 Added file: http://bugs.python.org/file26668/get-numpy-working-with-python-3.3.patch ___ Python tracker <http://bugs.python.org/issue15540> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9635] Add Py_BREAKPOINT and sys._breakpoint hooks
Dave Malcolm added the comment: Note to self: a messy way of forcing gdb to do the equivalent of a breakpoint directly from Python is: os.kill(os.getpid(), signal.SIGTRAP) -- ___ Python tracker <http://bugs.python.org/issue9635> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15516] exception-handling bug in PyString_Format
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue15516> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12605] Enhancements to gdb 7 debugging hooks
Dave Malcolm added the comment:
I believe this was due to this line:
return self._gdbframe.name().startswith('pthread_cond_timedwait')
within is_waiting_for_gil(), and your gdb returned None for
self._gdbframe.name() for your build.
I've changed it to:
name = self._gdbframe.name()
if name:
return name.startswith('pthread_cond_timedwait')
in my latest version.
I've also bulletproofed against the --without-threads case by simply
skipping the cases that fail (some tests run into this gdb error:
Cannot find new threads: generic error
unless we add LD_PRELOAD=PATH-TO-libpthread.so.1 as a workaround, but it
seems that trying to add that workaround is more risky (what path?) than
simply skipping the tests under those circumstances.
Tested successfully on a Fedora 15 x86_64 box (gdb-7.3-43.fc15.x86_64)
with these configurations:
* --with-pydebug
* --with-pydebug --enable-shared
* (no flags = optimized)
* --enable-shared (optimized)
* --with-pydebug --without-threads
(Note that because of issue #14774 I have to regenerate
_sysconfigdata.py between each configuration, otherwise it gets confused
about whether or not the build is optimized)
--
___
Python tracker
<http://bugs.python.org/issue12605>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14443] Distutils test_bdist_rpm failure
Dave Malcolm added the comment: Fixes committed to 3.2 and default: RHEL 6.3 is now green for both branches. Closing this bug out. -- resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue14443> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15110] strange Tracebacks with importlib
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue15110> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14443] Distutils test_bdist_rpm failure
Dave Malcolm added the comment: (with the patch, it also continues to work on a test Fedora 15 box) -- ___ Python tracker <http://bugs.python.org/issue14443> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14443] Distutils test_bdist_rpm failure
Dave Malcolm added the comment:
The approach Nick suggests seems reasonable, but rpm.expandMacro isn't usable:
there's no guarantee that the machine has Python bindings for rpm installed
(especially not for the version of Python that we've just built).
However, we can get at the macro by shelling out to the "rpm" command:
subprocess.getoutput('rpm --eval %{__os_install_post}')
I'm attaching a patch which does this; works on my RHEL 6.3 test box.
--
Added file: http://bugs.python.org/file26157/fix-14443.patch
___
Python tracker
<http://bugs.python.org/issue14443>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14785] Add sys._debugmallocstats()
Changes by Dave Malcolm : -- keywords: -needs review resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue14785> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment: I'm attaching an updated version of the patch; I hope there's still time before Saturday to get it into 3.3 I found a bug in the configure script in the old patch: /usr/bin/dtrace was required, even without requesting systemtap. I've fixed it by adding a new "SYSTEMTAPDEPS" to the configure.a/Makefile.pre.in A minor subtlety: within the configure.ac's --with-systemtap branch it is set up like this: SYSTEMTAPDEPS="\$(srcdir)/Python/pysystemtap.h" where the leading backslash is needed so that $(srcdir) doesn't get interpreted by the shell when running configure as "run the command named 'srcdir'". I also added removal of $(srcdir)/Python/pysystemtap.h to the "clean" target I made a slight change to the static markers themselves: they now pass a 4th argument, the PyFrameObject*, since it's possible to make use of this to inspect locals etc from systemtap, which might be of use to people. This doesn't introduce any further complexity to the ceval.c code. I moved the documentation from the devguide back to the cpython source tree, rewrote it as a HOWTO (Doc/howto/instrumentation.rst), adding some extra material (e.g. about tapsets). I also added a NEWS item. Tested on a Fedora 15 box with these configurations: * --with-pydebug (implicit --without-systemtap) * explicitly --without-systemtap * --with-pydebug --with-systemtap * --with-pydebug --enable-shared --with-systemtap * --enable-shared --with-systemtap (note that because of issue 14774 I had to rebuild _sysconfigdata.py each time, or the tests fail due to being confused about whether we were configured with --enable-shared) How is this looking? -- Added file: http://bugs.python.org/file26074/cpython-systemtap-2012-06-21-001.patch ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14443] Distutils test_bdist_rpm failure
Dave Malcolm added the comment:
I'm able to reproduce this on a RHEL 6 box, and I did some investigating. The
stray .pyc files are indeed reported by "file" as "python 2.6 byte-compiled" so
yes, it's using /usr/bin/python to byte-compile them
On RHEL 6, with redhat-rpm-config-9.0.3-33.el6.noarch,
/usr/lib/rpm/redhat/macros defines os_install_post as:
%__os_install_post\
/usr/lib/rpm/redhat/brp-compress \
%{!?__debug_package:/usr/lib/rpm/redhat/brp-strip %{__strip}} \
/usr/lib/rpm/redhat/brp-strip-static-archive %{__strip} \
/usr/lib/rpm/redhat/brp-strip-comment-note %{__strip} %{__objdump} \
/usr/lib/rpm/brp-python-bytecompile \
/usr/lib/rpm/redhat/brp-python-hardlink \
%{!?__jar_repack:/usr/lib/rpm/redhat/brp-java-repack-jars} \
%{nil}
Note how in this definition, brp-python-bytecompile is passed in without any
arguments (contrast with msg159396, which is from a later version of the
macros).
Hence it unconditionally (and erroneously) uses /usr/bin/python to byte-compile
any .py files found in the package payload.
The change to add %{__python} to the invocation of
/usr/lib/rpm/brp-python-bytecompile appears to have been in
redhat-rpm-config-9.1.0 for Fedora 13 (see
https://bugzilla.redhat.com/show_bug.cgi?id=521141), whereas RHEL 6 has the
earlier code.
It may be possible to work around this by providing an overridden definition of
__os_install_post in the specfile. We do this in the python26.spec for EPEL5;
grep for "__os_install_post" within:
http://pkgs.fedoraproject.org/gitweb/?p=python26.git;a=blob;f=python26.spec;h=6b490b9b71f42c26b7d4ec4031685fb3230c5602;hb=refs/heads/el5
--
___
Python tracker
<http://bugs.python.org/issue14443>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment: Thanks. I too am wondering where the documentation should go. We currently have several locations for documentation: (1) the man page (2) the "Doc" subdirectory aka docs.python.org (3) various text files in the source tree, such as Misc/SpecialBuilds.txt (4) the devguide (5) www.python.org, and other web sites This feature is about instrumentation of CPython, which is of use to several audiences, such as: (a) people working on CPython's internals (b) Python developers seeking to understand performance issues (c) sysadmins trying to tweak the performance of, say, a deployed web app As I understand it, the devguide is targetting (a), whereas the Doc subdirectory of the cpython sources is targetting (b) when they're writing the code, but not necessarily when tuning it, and (c) doesn't seem to get covered. So perhaps there should be a new top-level section of documentation within the main Python Doc directory covering instrumentation and performance? (e.g. "instrumentation" or "debugging"). If so, these docs could be an entry within that section. Though I don't want to block this feature on writing up a full guide to debugging/deploying Python :) Less ambitiously, we could just add it to the "howto" section (which appears to be something of a dumping ground for docs that didn't fit neatly into the other categories). Would that be acceptable in lieu of a better location? Or is this something for the python-dev mailing list? Cheers! -- ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Dave Malcolm : Removed file: http://bugs.python.org/file25615/cpython-systemtap-2012-05-16-001.patch ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Dave Malcolm : -- Removed message: http://bugs.python.org/msg160914 ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Dave Malcolm : -- Removed message: http://bugs.python.org/msg160913 ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment: (Sorry about msg160913; looks like my initial email response took about an hour to get through, and I'd subsequently commented using the webui form) -- ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment:
On Wed, 2012-05-16 at 16:41 +, Antoine Pitrou wrote:
> Antoine Pitrou added the comment:
>
> I tried the patch under Mageia 1 and got the following failure:
>
> test_systemtap skipped -- Test systemtap script did not run; stderr was:
> b"Pass 1: parsed user script and 72 library script(s) using
> 56252virt/20964res/1828shr kb, in 70usr/0sys/82real ms.\nPass 2: analyzed
> script: 1 probe(s), 1 function(s), 0 embed(s), 0 global(s) using
> 56648virt/21492res/1900shr kb, in 10usr/0sys/3real ms.\nPass 3: using cached
> /home/antoine/.systemtap/cache/15/stap_155c3565481f113c929ad94e10c2b48e_779.c\nPass
> 4: using cached
> /home/antoine/.systemtap/cache/15/stap_155c3565481f113c929ad94e10c2b48e_779.ko\nPass
> 5: starting run.\nPass 5: run completed in 0usr/0sys/3real ms.\nPass 5: run
> failed. Try again with another '--vp 1' option.\n"
Thanks. I'm attaching an updated patch which reworks test_systemtap so
that
(a) it turns up the verbosity of stap invocations from "-v" to "-vv"
(b) it tests the "hello world" stap script on a trivial binary
("true") before attempting to invoke sys.executable under stap
What does the error message look like with the extra verbosity?
The chmod invocation sounds like an issue with how Mageia have packaged
SystemTap, assuming that you're using a packaged build of systemtap.
--
Added file:
http://bugs.python.org/file25615/cpython-systemtap-2012-05-16-001.patch
___
Python tracker
<http://bugs.python.org/issue14776>
___diff -r 0f6a6f59b002 .hgignore
--- a/.hgignore Wed May 16 19:08:36 2012 +0200
+++ b/.hgignore Wed May 16 13:34:52 2012 -0400
@@ -19,6 +19,7 @@
python.exe$
reflog.txt$
tags$
+Python/pysystemtap.h
Lib/plat-mac/errors.rsrc.df.rsrc
Doc/tools/sphinx/
Doc/tools/docutils/
diff -r 0f6a6f59b002 Lib/test/test_systemtap.py
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/Lib/test/test_systemtap.pyWed May 16 13:34:52 2012 -0400
@@ -0,0 +1,205 @@
+# Verify that systemtap static probes work
+#
+import subprocess
+import sys
+import sysconfig
+import os
+import unittest
+
+from test.support import run_unittest, TESTFN, unlink
+
+if '--with-systemtap' not in sysconfig.get_config_var('CONFIG_ARGS'):
+raise unittest.SkipTest("Python was not configured --with-systemtap")
+
+try:
+_, stap_version = subprocess.Popen(["stap", "-V"],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ ).communicate()
+except OSError:
+# This is what "no stap" looks like. There may, however, be other
+# errors that manifest this way too.
+raise unittest.SkipTest("Couldn't find stap on the path")
+
+def invoke_systemtap_script(script, cmd):
+# Start a child process, probing with the given systemtap script
+# (passed as stdin to the "stap" tool)
+# The script should be a bytes instance
+# Return (stdout, stderr) pair
+
+p = subprocess.Popen(["stap", "-", '-vv', '-c', cmd],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+out, err = p.communicate(input=script)
+return out, err
+
+# Verify that stap can run a simple "hello world"-style script
+# This can fail for various reasons:
+# - missing kernel headers
+# - permissions (a non-root user needs to be in the "stapdev" group)
+TRIVIAL_STAP_SCRIPT = b'probe begin { println("hello world") exit () }'
+
+out, err = invoke_systemtap_script(TRIVIAL_STAP_SCRIPT, 'true')
+if out != b'hello world\n':
+raise unittest.SkipTest("Test systemtap script did not run; stderr was:
%s" % err)
+
+# We don't expect stderr to be empty, since we're invoking stap with "-vv":
stap
+# will (we hope) generate debugging output on stderr.
+
+def invoke_python_under_systemtap(script, pythoncode=None, pythonfile=None):
+# Start a child python process, probing with the given systemtap script
+# (passed as stdin to the "stap" tool)
+# The script should be a bytes instance
+# Return (stdout, stderr) pair
+
+if pythonfile:
+pythoncmd = '%s %s' % (sys.executable, pythonfile)
+else:
+pythoncmd = '%s -c %r' % (sys.executable, pythoncode)
+
+# The process tree of a stap invocation of a command goes through
+# something like this:
+#stap ->fork/exec(staprun; exec stapio ->f/e(-c cmd); exec staprun -r)
+# and this trip through setuid leads to LD_LIBRARY_
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment:
> I tried the patch under Mageia 1 and got the following failure:
Thanks. I'm attaching an updated patch which reworks test_systemtap so that
(a) it turns up the verbosity of stap invocations from "-v" to "-vv"
(b) it tests the "hello world" stap script on a trivial binary ("true")
before attempting to invoke sys.executable under stap
What does the error message look like with the extra verbosity?
> (I also had to chmod +s staprun - scary :-))
The chmod invocation sounds like an issue with how Mageia have packaged
SystemTap, assuming that you're using a packaged build of systemtap.
--
Added file:
http://bugs.python.org/file25613/cpython-systemtap-2012-05-16-001.patch
___
Python tracker
<http://bugs.python.org/issue14776>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8330] Failures seen in test_gdb on buildbots
Dave Malcolm added the comment: Oops; closing. Thanks. -- resolution: -> fixed status: open -> closed ___ Python tracker <http://bugs.python.org/issue8330> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Dave Malcolm : -- nosy: +fche, scox ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14785] Add sys._debugmallocstats()
Dave Malcolm added the comment: Thanks. I'm attaching an updated version of the patch, wrapping all new C entrypoints within a #ifndef Py_LIMITED_API I also moved the existing _PyObject_DebugMallocStats() entrypoint to within a #ifndef Py_LIMITED_API. As noted above, it is not documented (and these patches change it from accepting void to a FILE*). -- Added file: http://bugs.python.org/file25584/add-debug-malloc-stats-v3.patch ___ Python tracker <http://bugs.python.org/issue14785> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14785] Add sys._debugmallocstats()
Dave Malcolm added the comment: Updated version of the patch, using test.script_helper.assert_python_ok() and adding a NEWS entry -- Added file: http://bugs.python.org/file25579/add-debug-malloc-stats-v2.patch ___ Python tracker <http://bugs.python.org/issue14785> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14785] Add sys._debugmallocstats()
New submission from Dave Malcolm : I'm attaching a patch which generalizes the at-exit PYTHONMALLOCSTATS memory usage report, so that it's available in a regular build and can be triggered from Python, by calling: sys._debugmallocstats() This can be useful when debugging memory usage issues: a script can call the debug hook before and after certain activities, and the before/after amounts can be compared. The patch moves this arena-accouting code when a new arena is allocated: ++ntimes_arena_allocated; if (narenas_currently_allocated > narenas_highwater) narenas_highwater = narenas_currently_allocated; from out of the #ifdef PYMALLOC_DEBUG guard and into all PYMALLOC configurations, so that this data is available within the dump. Given how much activity happens when a new arena is allocated, I believe this doesn't impact performance. It changes _PyObject_DebugMallocStats() to take an arbitrary FILE*, rather than assuming stderr (which was handy for my original use-case of debugging a web server). This function is already marked with PyAPI_FUNC() but not documented (albeit only present in a debug build). Tested with --with-pymalloc, --without-pymalloc, and --with-pydebug FWIW, Red Hat has been using a version of this patch in RHEL 5 as of RHEL 5.6 (http://rhn.redhat.com/errata/RHSA-2011-0027.html), and also in Fedora since September 2011 with python-2.7.2-15 and python3-3.2.2-6 (for the forthcoming Fedora 17). -- files: add-debug-malloc-stats.patch keywords: needs review, patch messages: 160459 nosy: dmalcolm priority: normal severity: normal stage: patch review status: open title: Add sys._debugmallocstats() type: enhancement versions: Python 3.3 Added file: http://bugs.python.org/file25545/add-debug-malloc-stats.patch ___ Python tracker <http://bugs.python.org/issue14785> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4111] Add Systemtap/DTrace probes
Dave Malcolm added the comment: Issue #13405 covers DTrace; I've taken the liberty of also opening issue #14776 to cover SystemTap. I hope that once one of these is in the tree it will be easier to get the other one in. -- ___ Python tracker <http://bugs.python.org/issue4111> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Dave Malcolm : Added file: http://bugs.python.org/file25540/devguide-systemtap-2012-05-11-001.patch ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Dave Malcolm added the comment: Thanks Eric, Antoine and Mark. I'm attaching two new patches: a replacement patch for cpython, and a new patch for the devguide I've moved the docs to the dev guide, starting a new "Debugging and Instrumentation" section there. Changes to the cpython patch: * fixed a bug in configure.in that was enabling systemtap support even without --with-systemtap (if the devel toolchain was present) * I added an initial check to test_systemtap to skip the tests unless Python was configured --with-systemtap * pysystemtap.h is not meant to be public, so I've moved the source pysystemtap.d and generated header pysystemtap.h from Include/ to Python/. I also simplified pysystemtap.d (removed the #pragma lines, since I believe they're DTrace-specific). * I've introduced a Python/ceval_systemtap.h private header as suggested by Antoine, moving things in there to simplify the changes to Python/ceval.c Changes to the devguide docs: * removed the ".. impl-detail" as this only seems to work (and be appropriate) in the cpython Doc build, not in devguide. * added "eu-readelf -n" example from Mark The docs refer to the low-level way of doing things (using the markers directly), but don't yet spell out the higher-level way (creating a tapset). I've left this out of the patches for now to keep the patches simpler. -- Added file: http://bugs.python.org/file25539/cpython-systemtap-2012-05-11-001.patch ___ Python tracker <http://bugs.python.org/issue14776> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Dave Malcolm added the comment:
I've refreshed my SystemTap patch, and opened a new issue, issue #14776 to
cover SystemTap.
Issue #4111 was originally opened on 2008-10-12 as "Add DTrace probes", and was
generalized on 2009-12-08 to "Add Systemtap/DTrace probes". That issue was
closed on 2011-11-14 to be superceded by issue #13405 ("Add DTrace probes"),
hence I opened the SystemTap RFE as a separate issue to this one.
As noted in issue #14776: I believe that although DTrace and SystemTap are
similar, they are sufficiently different from each other that it's going to
take separate work to support one or the other (and that the maintainers of the
support for each within CPython are likely to be different people). I hope
that once one of them is merged, the other will become a lot easier to merge.
--
___
Python tracker
<http://bugs.python.org/issue13405>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
New submission from Dave Malcolm :
I'm attaching a patch which adds static markers for SystemTap for two probeable
events within CPython's bytecode interpreter, along with test cases and
documentation.
I'm hoping to get this merged for 3.3; is this PEP-worthy, or can this be done
through patch review?
Note: issue #4111 was originally opened on 2008-10-12 as "Add DTrace probes",
and was generalized on 2009-12-08 to "Add Systemtap/DTrace probes". That issue
was closed on 2011-11-14 to be superceded by issue #13405 ("Add DTrace
probes"), hence I'm opening this as a separate issue.
I believe that although DTrace and SystemTap are similar, they are sufficiently
different from each other that it's going to take separate work to support one
or the other (and that the maintainers of the support for each within CPython
are likely to be different people). I hope that once one of them is merged,
the other will become a lot easier to merge.
Potentially other markers could be added for other events (the DTrace patch in
issue #13405 has gained some), but I wanted to start small for now.
--
components: Interpreter Core
files: cpython-systemtap-2012-05-10-001.patch
keywords: needs review, patch
messages: 160374
nosy: dmalcolm
priority: normal
severity: normal
stage: patch review
status: open
title: Add SystemTap static markers
type: enhancement
versions: Python 3.3
Added file:
http://bugs.python.org/file25529/cpython-systemtap-2012-05-10-001.patch
___
Python tracker
<http://bugs.python.org/issue14776>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14774] _sysconfigdata.py doesn't support multiple build configurations
Dave Malcolm added the comment: Note to self: workaround is to rm ../../Lib/_sysconfigdata.py || make ../../Lib/_sysconfigdata.py before running my tests in either configuration, to force the file to be regenerated using what "make" thinks the settings are -- ___ Python tracker <http://bugs.python.org/issue14774> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14774] _sysconfigdata.py doesn't support multiple build configurations
New submission from Dave Malcolm :
When building from source, if I create multiple configuration directories and
build from there e.g.:
mkdir configs
cd configs
mkdir config-A
cd config-A
../../configure
make
cd ..
mkdir config-B
cd config-B
../../configure --enable-shared
make
cd ../config-A
./python -c "import sysconfig; print(sysconfig.get_config_var('CONFIG_ARGS')
then sysconfig's settings are the same for *every* config, reflecting those of
the last build (config-B above, rathern than those of config-A).
This turns out to be due to this:
./python -SE -m sysconfig --generate-posix-vars
This generates $(srcdir)/Lib/_sysconfigdata.py for whichever config was last
Is there a way of fixing this whilst keeping it a python file? Or do we need
to build from a C file, perhaps?
--
components: Build
messages: 160366
nosy: dmalcolm
priority: normal
severity: normal
status: open
title: _sysconfigdata.py doesn't support multiple build configurations
type: behavior
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue14774>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14757] INCA: Inline Caching meets Quickening in Python 3.3
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue14757> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14748] spwd.getspall() is returning LDAP (non local) users too
Dave Malcolm added the comment: Like passwd and group information, the shadow password entries are pulled through libc's Name Service Switch and modules for it, depending on configuration. See "man nsswitch.conf". Hence this is likely to be a configuration difference between the two boxes. Some notes from one of my Red Hat colleagues: * Is a module listed in /etc/nsswitch.conf so that it'll be used to look up "shadow" information? * Does the module support looking up shadow information? The libnss_ldap.so.2 stub from nss-pam-ldapd does; SSSD (at least version 1.8.3) doesn't. * Are there shadowAccount entries in the directory server? An IPA server won't have them, because IPA makes use of the directory server's built-in password policy functionality to avoid depending on clients to enforce aging policies. * Is the client performing the lookup authorized to read the shadow data from the directory server? * Does the client perform any additional access control? The daemon in nss-pam-ldapd only exposes shadow information to processes running as UID 0. etc Hope this is helpful -- ___ Python tracker <http://bugs.python.org/issue14748> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14443] Distutils test_bdist_rpm failure
Dave Malcolm added the comment:
__os_install_post is defined within /usr/lib/rpm/redhat/macros and contains
this fragment:
/usr/lib/rpm/brp-python-bytecompile %{__python}
%{?_python_bytecompile_errors_terminate_build} \
Hence it will use %{__python} as the default when byte-compiling.
%__python has the default definition of /usr/bin/python, but this can be
overridden, either in the spec file using:
%global __python some_path_to_the_python_to_use
or in the command-line invocation of rpmbuild:
rpmbuild --define="__python some_path_to_the_python_to_use"
You can use the --showrc option to rpmbuild to see all of the macro expansions
it's using (lots of output).
So if it is indeed using the wrong python executable to do the bytecompiling,
the above ought to fix it. The simplest fix would probably be for bdist_rpm to
add:
--define="__python %s" % sys.executable
to the rpmbuild invocation, I think.
--
___
Python tracker
<http://bugs.python.org/issue14443>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14443] Distutils test_bdist_rpm failure
Dave Malcolm added the comment: As a post-processing step, rpmbuild will attempt to byte-compile any .py files it encounters, and the results must be listed in the %files manifest. [1] This is done by the script brp-python-bytecompile, which uses the compileall module. However, my guess is that it's not using the correct version of python when invoking "compileall", which would explain why it's using the pre-PEP3147 location for the .pyc/.pyo files. Can you run "file" on the .pyc files and confirm which version of Python they're bytecode for? My guess is that it's bytecompiled them with /usr/bin/python, rather than your local build of python. Some notes: In older versions of RPM, brp-python-bytecompile took a single optional argument: the python interpreter to use, defaulting to /usr/bin/python. I generalized this to support multiple defaults when adding Python 3 support to Fedora: see https://bugzilla.redhat.com/show_bug.cgi?id=531117 That patch could be generalized to support /usr/local/lib. [1] In Fedora we do this using "__os_install_post", which is defined in /usr/lib/rpm/redhat/macros (from the redhat-rpm-config package), which has the invocation of /usr/lib/rpm/brp-python-bytecompile So it could be possible to override the python interpreter to use by redefining __os_install_post -- ___ Python tracker <http://bugs.python.org/issue14443> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14621] Hash function is not randomized properly
Dave Malcolm added the comment:
$ gdb --eval-command="break _PyRandom_Init" --eval-command="run"
--eval-command="print _Py_HashSecret" --eval-command="set
_Py_HashSecret.prefix=0xcdcdcd00" --eval-command="print _Py_HashSecret"
--eval-command="continue" -eval-command="continue" --args python -c
'a="\x00\xcf\x0b\x96\x19"; b="\x00\x6d\x29\x45\x18"; print(hash(a));
print(hash(b))'
On 32-bit:
$1 = {prefix = 0, suffix = 0}
$2 = {prefix = -842150656, suffix = 0}
1220138288
1220138288
On 64-bit:
$1 = {prefix = 0, suffix = 0}
$2 = {prefix = 3452816640, suffix = 0}
Continuing.
4087671194599937328
-167939011306192
--
___
Python tracker
<http://bugs.python.org/issue14621>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14621] Hash function is not randomized properly
Dave Malcolm added the comment:
Thanks for filing this bug report.
I'm not seeing the equal hashes you describe.
I'm using this recipe to hardcode a specific prefix and print the hashes using
it:
$ gdb --eval-command="break _PyRandom_Init" --eval-command="run"
--eval-command="print _Py_HashSecret" --eval-command="set
_Py_HashSecret.prefix=0xcdcdcdcd" --eval-command="print _Py_HashSecret"
--eval-command="continue" -eval-command="continue" --args python -c
"a='\x27\xfd\x5a\x18'; b='\x26\xfe\x78\xfa'; print(hash(a)); print(hash(b))"
On a 32-bit build of Python 2.7.3 (i686), if I set
_Py_HashSecret.prefix=0xcdcdcdcd, I get non-equal hashes for the data you
specify (output trimmed somewhat for conciseness):
$1 = {prefix = 0, suffix = 0}
$2 = {prefix = -842150451, suffix = 0}
Continuing.
-121255142
-1199906326
Similarly, on a 64-bit build of Python 2.7.3 (x86_64), I get non-equal hashes:
$1 = {prefix = 0, suffix = 0}
$2 = {prefix = 3452816845, suffix = 0}
-3992804574342296806
-8147489705433570838
Did I misunderstand the report? Thanks.
--
___
Python tracker
<http://bugs.python.org/issue14621>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14621] Hash function is not randomized properly
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue14621> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9960] test_structmembers fails on s390x (bigendian 64-bit): int/Py_ssize_t issue
Dave Malcolm added the comment: The originally attached patch is no good for the the 2.* branch, as it appears that _testcapimodule.c will not become "ssize_t" safe in Python 2.*; see e.g.: http://hg.python.org/cpython/rev/3ecddf168f1f Am attaching a revised patch that I'm applying downstream in Fedora's builds of 2.7.3 -- Added file: http://bugs.python.org/file25199/fix-test_structmember-on-64bit-bigendian.patch ___ Python tracker <http://bugs.python.org/issue9960> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14444] Virtualenv not portable from Python 2.7.2 to 2.7.3 (os.urandom missing)
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue1> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13963] dev guide has no mention of mechanics of patch review
Dave Malcolm added the comment: It would appear that having: [diff] git = on in ~/.hgrc breaks the rietveld integration, since "hg diff" then emits a diff that doesn't identify the hg revision number, and hence the importer can't determine the baseline. -- ___ Python tracker <http://bugs.python.org/issue13963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13963] dev guide has no mention of mechanics of patch review
Changes by Dave Malcolm : -- nosy: +loewis ___ Python tracker <http://bugs.python.org/issue13963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13963] dev guide has no mention of mechanics of patch review
Dave Malcolm added the comment: Some notes from discussion with MvL at PyCon sprint: The ideal is that: - for any patch attached to an issue: the patch is uploaded to a Rietveld instance colocated in the same db as Roundup (bugs.python.org) - if it works, than a "review" link is visible next to the patch, taking you to a patch review UI: unfortunately, it takes you to the most recent patch on the issue, rather than the patch you click on - unfortunately there are a few ways for the patch upload to fail, and if this happens, then currently there is no visible indication within Roundup. If this happens, then patch review will need to be done within Roundup itself (i.e. the comment form or via email) - how can it fail: the importer needs to determine what the baseline to which the patch is to be applied. It can figure this out for a "hg diff" (assuming that the baseline revision is known to hg.python.org/cpython), but not for git-style diffs (unless you do it on the current head of "default") - in theory the nosy list of a bug within Roundup is synchronized with that of the patch within Rietveld. Unfortunately there is currently at least one bug with this (e.g. dmalcolm isnt on the nosy of http://bugs.python.org/review/13703/show despite being on the nosy of http://bugs.python.org/issue13703): FIXME: do we have a metabug about this? See: http://psf.upfronthosting.co.za/roundup/meta/issue394 http://psf.upfronthosting.co.za/roundup/meta/issue439 Some code links: http://svn.python.org/projects/tracker/instances/python-dev/rietveld/ http://svn.python.org/projects/tracker/instances/python-dev/extensions/create_patch.py -- ___ Python tracker <http://bugs.python.org/issue13963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14234] CVE-2012-0876 (hash table collisions CPU usage DoS) for embedded copy of expat
Changes by Dave Malcolm : -- nosy: +barry ___ Python tracker <http://bugs.python.org/issue14234> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14234] CVE-2012-0876 (hash table collisions CPU usage DoS) for embedded copy of expat
New submission from Dave Malcolm : Expat 2.1.0 Beta was recently announced: http://mail.libexpat.org/pipermail/expat-discuss/2012-March/002768.html which contains (among other things) a fix for a hash-collision denial-of-service attack (CVE-2012-0876) I'm attaching a patch which minimally backports the hash-collision fix part of expat 2.1.0 to the embedded copy of expat in the CPython source tree, and which adds a call to XML_SetHashSalt() to pyexpat when creating parsers. It reuses part of the hash secret from Py_HashSecret. -- components: XML files: expat-hash-randomization.patch keywords: patch messages: 155198 nosy: dmalcolm priority: normal severity: normal status: open title: CVE-2012-0876 (hash table collisions CPU usage DoS) for embedded copy of expat versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file24762/expat-hash-randomization.patch ___ Python tracker <http://bugs.python.org/issue14234> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14103] argparser should create a bash_completion script for me.
Changes by Dave Malcolm : -- components: +Library (Lib) type: -> enhancement ___ Python tracker <http://bugs.python.org/issue14103> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Dave Malcolm added the comment: jcea: BTW, will you be at PyCon US this year? if so, can we sprint on getting the DTrace and SystemTap hooks into CPython in some form acceptable to the rest of the CPython maintainers? -- ___ Python tracker <http://bugs.python.org/issue13405> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14066] Crash in imputil.imp.find_module when a *directory* exists with the given name due to double-close of FILE*
Dave Malcolm added the comment: Duplicate of issue 7732 -- resolution: -> duplicate status: open -> closed superseder: -> imp.find_module crashes Python if there exists a directory named "__init__.py" ___ Python tracker <http://bugs.python.org/issue14066> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14066] Crash in imputil.imp.find_module when a *directory* exists with the given name due to double-close of FILE*
New submission from Dave Malcolm :
$ mkdir some_directory_ending_with_a.py
$ python -c "import imputil;
imputil.imp.find_module('some_directory_ending_with_a')"
*** glibc detected *** python: double free or corruption (!prev):
0x01589bf0 ***
Aborted
What's happening is that call_find_module opens fp, and tries to create a
PyFileObject from it:
2844 if (fp != NULL) {
>>2845 fob = PyFile_FromFile(fp, pathname, fdp->mode, fclose);
2846 if (fob == NULL) {
2847 fclose(fp);
2848 return NULL;
2849 }
2850 }
The call fails at the very end of fill_file_fields() inside the call to
dircheck() here:
180 f->f_fp = fp;
181 f = dircheck(f);
182 return (PyObject *) f;
183 }
184
but f->fp == fp
Hence when fill_file_fields returns NULL, and f is cleaned up here:
484 if (fill_file_fields(f, fp, o_name, mode, close) == NULL) {
>> 485 Py_DECREF(f);
486 Py_DECREF(o_name);
487 return NULL;
488 }
then f->fp is closed within file_dealloc()
Then, when PyFile_FromFile returns NULL, call_find_module calls fclose(fp):
2844 if (fp != NULL) {
2845 fob = PyFile_FromFile(fp, pathname, fdp->mode, fclose);
2846 if (fob == NULL) {
>>2847 fclose(fp);
2848 return NULL;
2849 }
2850 }
and it attempts to close fp for the second time.
The documentation for PyFile_FromFile:
http://docs.python.org/c-api/file.html#PyFile_FromFile
says:
"Return NULL and close the file using close on failure".
It therefore looks like call_find_module's fclose(fp) is incorrect here, and is
what leads to the double-free.
--
components: Library (Lib)
messages: 153799
nosy: dmalcolm
priority: normal
severity: normal
status: open
title: Crash in imputil.imp.find_module when a *directory* exists with the
given name due to double-close of FILE*
type: crash
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue14066>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14001] Python v2.7.2 / v3.2.2 (SimpleXMLRPCServer): DoS (excessive CPU usage) by processing malformed XMLRPC / HTTP POST request
Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker <http://bugs.python.org/issue14001> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Dave Malcolm added the comment: I'm not quite sure how that would interact with the -R command-line option for enabling randomization. The changes to the docs in the latest patch clarifies the meaning of what I've implemented (I hope). My view is that we should simply enable hash randomization by default in 3.3 At that point, PYTHONHASHRANDOMIZATION and the -R option become meaningless (and could be either removed, or silently ignored), and you have to set PYTHONHASHSEED=0 to get back the old behavior. -- ___ Python tracker <http://bugs.python.org/issue13703> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13963] dev guide has no mention of mechanics of patch review
New submission from Dave Malcolm : I've been waiting for patch review of my work on http://bugs.python.org/issue13703 only to discover that people *have* been reviewing it. It turns out that next to some of the patches in the issue tracker there's a "review" link, which takes me to http://bugs.python.org/review/13703/show and there's a whole second conversation going on about the issue there. I haven't been getting any emails about it. I've been looking through http://docs.python.org/devguide but I can't see any mention of the mechanics of patch review, or that this secondary site exists. Clearly I missed something big, but was it actually documented anywhere? http://docs.python.org/devguide/patch.html mentions uploading patches for review, and talks about an iterative process of commenting and refining a patch, but when I read it, I assumed it was referring to discussion within the issue, rather than on this secondary site. Some questions: * Do all patches go into this review site, or do I have to do something extra to get them to land there? * I have patches for both 2.6 and 3.1 - are they kept separate, or do they affect each other's "delta from patch set"? * Is there a way to enable notifications, e.g. for me to receive emails when someone comments on my patch? or to put a comment into the *issue* noting that someone commented on the patch? Thanks -- components: Devguide messages: 152813 nosy: dmalcolm, ezio.melotti priority: normal severity: normal status: open title: dev guide has no mention of mechanics of patch review ___ Python tracker <http://bugs.python.org/issue13963> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Dave Malcolm added the comment: On Mon, 2012-02-06 at 23:00 +, Marc-Andre Lemburg wrote: > Marc-Andre Lemburg added the comment: > > Alex Gaynor wrote: > > There's no need to cover any container types, because if their constituent > > types are securely hashable then they will be as well. And of course if > > the constituent types are unsecure then they're directly vulnerable. > > I wouldn't necessarily take that for granted: since container > types usually calculate their hash based on the hashes of their > elements, it's possible that a clever combination of elements > could lead to a neutralization of the the hash seed used by > the elements, thereby reenabling the original attack on the > unprotected interpreter. > > Still, because we have far more vulnerable hashable types out there, > trying to find such an attack doesn't really make practical > sense, so protecting containers is indeed not as urgent :-) FWIW, I'm still awaiting review of my patches. I don't believe Marc-Andre's concerns are a sufficient rebuttal to the approach I've taken. If anyone is aware of an attack via numeric hashing that's actually possible, please let me know (privately). I believe only specific apps could be affected, and I'm not aware of any such specific apps. -- ___ Python tracker <http://bugs.python.org/issue13703> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Dave Malcolm added the comment: > Can't randomization just be applied to integers as well? > It could, but see http://bugs.python.org/issue13703#msg151847 Would my patches be more or less likely to get reviewed with vs without an extension of randomization to integers? -- ___ Python tracker <http://bugs.python.org/issue13703> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Dave Malcolm added the comment: On Mon, 2012-02-06 at 10:20 +, Marc-Andre Lemburg wrote: > Marc-Andre Lemburg added the comment: > > STINNER Victor wrote: > > > > STINNER Victor added the comment: > > > >> In a security fix release, we shouldn't change the linkage procedures, > >> so I recommend that the LoadLibrary dance remains. > > > > So the overhead in startup time is not an issue? > > It is an issue. Not only in terms of startup time, but also msg152362 indicated that there was negligible impact on startup time when randomization is disabled. The impact when it *is* enabled is unclear, but reported there as "isn't crippling". > because randomization per default makes Python behave in > non-deterministc ways - which is not what you want from a > programming language or interpreter (unless you explicitly > tell it to behave like that). The release managers have pronounced: http://mail.python.org/pipermail/python-dev/2012-January/115892.html Quoting that email: > 1. Simple hash randomization is the way to go. We think this has the > best chance of actually fixing the problem while being fairly > straightforward such that we're comfortable putting it in a stable > release. > 2. It will be off by default in stable releases and enabled by an > envar at runtime. This will prevent code breakage from dictionary > order changing as well as people depending on the hash stability. -- ___ Python tracker <http://bugs.python.org/issue13703> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Dave Malcolm added the comment: Am attaching a backport of optin-hash-randomization-for-3.1-dmalcolm-2012-01-30-002.patch to 2.6 Randomization covers the str, unicode and buffer types; equality of hashes is preserved for these types. -- Added file: http://bugs.python.org/file24375/optin-hash-randomization-for-2.6-dmalcolm-2012-01-30-001.patch ___ Python tracker <http://bugs.python.org/issue13703> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
