[issue28909] Adding LTTng-UST tracing support
Francis Deslauriers added the comment: Hi all, It seems that, as of right now, the thing blocking this patchset from going forward is the name of the intrumentation point. Two naming approached were suggested: - Keeping PyDtrace* - Changing to PyProbe* I prefer the PyProbe option as it's a more generic name and is not misleading of the underlying tracing engine but if people prefer that we keep the PyDtrace version let's go with that. So, what should we go with? I can easily update and rebase this patchset. As an example of how this feature could be used, a colleague of mine gave a talk[1] at PyCon Canada 2017 about tracing Python applications using this patchset. He built a tool to visualize Python Logging, Python function calls and Linux syscalls all in the same view. This was done using the existing Python logger tracing of LTTng-UST, the LTTng kernel tracer and the CPython LTTng-UST instrumentation of this patchset. Here is an asciinema[2] recording used in the talk, it shows the tool in action. [1]: https://youtu.be/gKmtmPqr6H8 [2]: https://asciinema.org/a/v20Hxnoh3lpzzz3FPmF86fNDS Cheers! Francis -- ___ Python tracker <https://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
Francis Deslauriers added the comment: Hi all, Is there anything I can do to move this patch-set forward? @Charalampos, I will make sure to include this in the patch. Thank you. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
Francis Deslauriers added the comment: Here is the documentation patch. -- Added file: http://bugs.python.org/file46695/0003-Documentation-of-LTTng-UST-tracing-instrumentation.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
Francis Deslauriers added the comment: > What about `PyProbe`? Given the multitude of tools and techniques in this > space, wouldn't it be worthwhile to clarify things before adding this? I > think conflating `dtrace` and `lttng` would only lead to more confusion for > users as they really are distinct technologies. I like the PyProbe name too. PyTracepoint could be another option. Here is the tests patch. The tests are using the same test cases as the DTrace and SystemTap tests. It still uses the same name as before and the tests were added to the existing test_dtrace.py file. I will send an updated version once we have the final name. -- Added file: http://bugs.python.org/file46694/0002-LTTng-UST-tracing-tests.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
Francis Deslauriers added the comment: I am finally having the time to work in this. > A nit: the name LTTng-UST is rather unfriendly, especially when used without > the dash and in all lowercase characters. Given that we're using "dtrace" and > "systemtap", it would be simpler to just use "lttng" (drop the "-ust"). We can certainly drop the "-ust" and have the name in lowercase. > It's impossible to have DTrace and SystemTap at the same time, so it was > natural to choose to auto-detect the engine. With LTTng it becomes less > obvious what the configure options should be. > Should it be possible at all to have *both* LTTng and SystemTap compiled in > at the same time? Does this make sense? It's possible to have both SystemTap and LTTng in the same binary. LTTng-UST has a configure option `--with-sdt` to include a SystemTap SDT probe alongside each LTTng-UST tracepoint. I don't have a specific usecase in mind but I can picture a setup where a binary is instrumented with both frameworks and the users either decide to use the low overhead tracing of LTTng or the versatile runtime aggregation of SystemTap depending on the problem they are trying to diagnose. So I think keeping both configure options makes sense. > Do you get unused code warnings without your patch applied? I don't. I am getting those errors too on Linux with GCC. I will make sure to fix them in the next round. I am currently working on the tests and documentation and I hope to submit patches for review early next week. Thank you, Francis -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
Francis Deslauriers added the comment: Thanks Łukasz, I will add tests following the example of the DTrace tests and update the Doc/howto/instrumentation.rst file to include this new information. LTTng can be used on all major Linux distros (Ubuntu, Debian, Fedora, etc.) either from distribution packages or compiled from source. It runs on kernels that came out after 2.6.27. You are right, I should have explained why I changed the provider name. I changed it be as specific as possible to avoid potential name clash with other Python interpreters. I will send an updated patch that keeps the old name given that it was already released. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
Changes by Francis Deslauriers <fdesl...@gmail.com>: -- nosy: +jcea ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28909] Adding LTTng-UST tracing support
New submission from Francis Deslauriers: This patch extends the tracing infrastructure to support LTTng UserSpace Tracer. Using LTTng-UST, users would have access to a low overhead tracing done entirely from userspace. Depending on the tracing configure option used (none, --with-dtrace or --with-lttngust) macros will be expanded to probes and tracepoints of the desired tracer if any. Only the needed instrumentation will be added to the binary. This technique is used to different degrees by other projects like Qemu[1] and Node.js[2]. I attached a patch adding this feature. I tested the changes and the instrumentation on LTTng-UST and SystemTap on Ubuntu 16.04. I would appreciate if someone could test those changes on macOS and other platforms supporting USDT probes. [1]https://github.com/qemu/qemu/blob/master/configure#L4303 [2]https://github.com/nodejs/node/blob/master/configure#L811 -- files: 0001-Add-LTTng-UST-probe-and-tracepoints.patch keywords: patch messages: 282733 nosy: Francis Deslauriers priority: normal severity: normal status: open title: Adding LTTng-UST tracing support type: enhancement versions: Python 3.7 Added file: http://bugs.python.org/file45802/0001-Add-LTTng-UST-probe-and-tracepoints.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28909> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
