I'm working on vmprof (github.com/vmprof/vmprof-python) which works for both cpython and pypy (pypy has special support, cpython is patched on-the fly)
On Sun, Feb 8, 2015 at 6:39 AM, Gregory P. Smith <g...@krypto.org> wrote: > To get at the Python thread state from a signal handler (using 2.7 as a > reference here; but i don't believe 3.4 has changed this part much) you need > to modify the interpreter to expose pystate.c's "autoTLSkey" and thread.c's > "struct key" as well as "keyhead" and "keymutex". > > From there, in your signal handler you must try to acquire the newly exposed > keymutex and do nothing if you were unable to get it. If you did acquire it > (rare not to), you can walk the keyhead list looking for autoTLSkey to find > the current valid thread state. > > I had an intern (hi Elena!) write a signal sampling based low overhead > Python CPU profiler based on that last summer. I believe there are still > bugs to shaken out (if they are even possible to fix... Armin's comments are > true: signal handler code is super limited). I am stating this here because > I want someone to pester me at PyCon if I haven't released our work as a > proof of concept by then. The important take away: From what I could figure > out, you need to modify the CPython interpreter to be more amenable to such > introspection. > > A downside of a signal based profiler: *ALL* of the EINTR mishandling bugs > within the Python interpreter, stdlib, and your own code will show up in > your application. So until those are fixed (hooray for Antoine's PEP!), it > may not be practical for use on production processes which is sort of the > entire point of a low overhead sampling profiler... > > I'd like to get a buildbot setup that runs the testsuite while a continual > barrage of signals are being generated. We really don't stress test that > stuff (as evidence by the EINTR mishandling issues that are rampant) as > non-fatal signals are so rare for most things... until they aren't. > > As a side note and encouragement: I wonder what PyPy could do for > dynamically enabled and disabled low overhead CPU profiling. (take that as a > hint that I want someone else to get extremely creative!) > > -gps > > On Sat Feb 07 2015 at 1:34:26 PM Greg Ewing <greg.ew...@canterbury.ac.nz> > wrote: >> >> Maciej Fijalkowski wrote: >> > However, you can't access thread >> > locals from signal handlers (since in some cases it mallocs, thread >> > locals are built lazily if you're inside the .so, e.g. if python is >> > built with --shared) >> >> You might be able to use Py_AddPendingCall to schedule >> what you want done outside the context of the signal >> handler. >> >> The call will be made by the main thread, though, >> so if you need to access the frame of whatever thread >> was running when the signal occured, you will have >> to track down its PyThreadState somehow and get the >> frame from there. Not sure what would be involved >> in doing that. >> >> -- >> Greg >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com