New submission from Kristján Valur Jónsson <krist...@ccpgames.com>:
One of the problems with the profiling modules provided with Python is that they are not useful in the presence of multiple threads. This is because time spent in a different thread may be falsely attributed to some random place in a thread being profiled. This patch helps fix that, by making the _lsprof.c module (the engine behind cProfile) multi-stack aware. At every entry into the profiler, a check is made to see which stack is in operation (by looking at the thread state). The previous stack is then paused and profiling commences on the new stack. Time spent on other stacks is then subtracted from the measured time on each stack. A complication arises because it is no longer possible to determine the recursion level of each function (or subcall instance) on each stack by looking at the function's entry alone. For this reason, it becomes necessary to walk the stack in cases where there are multiple stacks and multiple total recursions seen for the entries. This patch has been successfully used, with a modifiaction for stackless python, in production at CCP (the modification uses the Tasklet ID rather than the TLS pointer as a key to the stack map). To be useful, it is important that all threads in the process are set to use the same cProfile.Profiler() instance. Currently there is no easy way to do that and this patch doesn't attempt to fix that. But is is possible that an application designed for profiling would attach the profiler at each thread start point. (In the version of Stackless Python that this is used on, it is possible to enable tracing/profiling of all tasklets simultaneously) ---------- components: Extension Modules files: _lsprof.patch keywords: patch, patch messages: 113964 nosy: krisvale priority: normal severity: normal status: open title: make cProfile multi-stack aware type: feature request versions: Python 3.2 Added file: http://bugs.python.org/file18539/_lsprof.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9609> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
