[issue21590] Systemtap and DTrace support

[Charalampos Stratakis]

Charalampos Stratakis added the comment:

@Łukasz

Would it be possible to review the patch?

Or is it preferable to open a new issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Charalampos Stratakis]

Charalampos Stratakis added the comment:

@Łukasz

Dug a bit more to it.

Yes it is RPM specific for that case, in the sense that we create different 
subfolders for the debug and the normal(optimized) builds under the build/ dir, 
where the Include directory does not exist. the dtrace wrapper assumes that the 
directory already exists. The issue of course can be reproduced with any 
packaging system (or ways to compile python) that follow this approach, 
although I am not aware currently if this can happen or might be happening 
anywhere else.

By creating the directory (if it doesn't exist) at the Makefile rule for 
generating pydtrace_probes.h, this can be circumvented.

Attaching a patch that fixes this issue for consideration.

--
Added file: 
http://bugs.python.org/file45393/create-Include-dir-to-properly-generate-pydtrace_probes.h-file.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Berker Peksag]

Changes by Berker Peksag :


--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Łukasz Langa]

Changes by Łukasz Langa :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Łukasz Langa]

Łukasz Langa added the comment:

Yes, let's close this.

@Charalampos, the error in your build comes from the /usr/bin/dtrace wrapper:
Traceback (most recent call last):
  File "/usr/bin/dtrace", line 440, in 
sys.exit(main())
  File "/usr/bin/dtrace", line 385, in main
providers.probe_write(s_filename, filename + suffix)
  File "/usr/bin/dtrace", line 181, in probe_write
hdr = open(header, mode='w')
FileNotFoundError: [Errno 2] No such file or directory: 
'Include/pydtrace_probes.h'

For some reason the path Include/pydtrace_probes.h is not writable. Maybe this 
is RPM-specific?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Berker Peksag]

Berker Peksag added the comment:

Thanks. SilentGhost. I went with ".. highlight:: shell-session" to simplify the 
patch a bit.

Łukasz, can we close this one and create new issues for further improvements 
now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Roundup Robot]

Roundup Robot added the comment:

New changeset d05d312161f2 by Berker Peksag in branch '3.6':
Issue #21590: Silence Sphinx warnings in instrumentation.rst
https://hg.python.org/cpython/rev/d05d312161f2

New changeset 442453fa3370 by Berker Peksag in branch 'default':
Issue #21590: Merge from 3.6
https://hg.python.org/cpython/rev/442453fa3370

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Charalampos Stratakis]

Charalampos Stratakis added the comment:

Fedora so far has been using the systemtap patch downstream from dmalcolm [0].

So for 3.6 by removing the systemtap patch (it cannot be applied anymore 
cleanly) and by enabling the --with-dtrace configure flag I get this error [1]:

make: *** [Makefile:884: Include/pydtrace_probes.h] Error 1


If however I touch the pydtrace_probes.h file I get various undefined 
references [2] and also:

make: *** [Makefile:724: Programs/_freeze_importlib] Error 1


[0] 
http://pkgs.fedoraproject.org/cgit/rpms/python3.git/tree/00055-systemtap.patch
[1] https://kojipkgs.fedoraproject.org//work/tasks/9656/16279656/build.log
[2] https://kojipkgs.fedoraproject.org//work/tasks/9670/16279670/build.log

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[SilentGhost]

SilentGhost added the comment:

Here is the patch that fixes warning in /Doc/howto/instrumentation.rst as well 
as cleans up a few nits here and there. I've removed attempt at highlighting 
using c since that doesn't work in practice.

--
keywords: +patch
nosy: +berker.peksag
stage:  -> patch review
versions: +Python 3.7
Added file: http://bugs.python.org/file45264/21590_2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[SilentGhost]

SilentGhost added the comment:

Building the documentation on Ubuntu 16.10, I started getting the following 
warnings:

/Doc/howto/instrumentation.rst:58: WARNING: Could not lex literal_block as 
"python3". Highlighting skipped.
/Doc/howto/instrumentation.rst:139: WARNING: Could not lex literal_block as 
"c". Highlighting skipped.
/Doc/howto/instrumentation.rst:211: WARNING: Could not lex literal_block as 
"c". Highlighting skipped.
/Doc/howto/instrumentation.rst:328: WARNING: Could not lex literal_block as 
"c". Highlighting skipped.

I guess the trick of using c for systemtap doesn't work anymore, not entirely 
sure what's the problem with the python3 highlighting.

--
nosy: +SilentGhost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Kubilay Kocak]

Kubilay Kocak added the comment:

@Łukasz in case you're not aware, all koobs-freebsd* bots are DTrace enabled, 
and can be tested with the custom builder. I'm on IRC (python-dev) if you need 
anything from me to help progress this

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Kubilay Kocak]

Changes by Kubilay Kocak :


--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Łukasz Langa]

Łukasz Langa added the comment:

Landed during the 3.6 beta1 sprint:
https://hg.python.org/cpython/rev/d622dbd71f2b

Thanks all for the preliminary tasks! I rebuilt the patch from first 
principles, reusing large parts of Jesús' work, as well as work by Dave and 
Nikhil.

Things deliberately left out for simplicity:
- ustack helpers, I have no way of testing them at this point since
they are Solaris-specific, don't work on macOS or Linux
- PyFrameObject * in function__entry/function__return, this is
SystemTap-specific and won't compile with DTrace
- SPARC support
- dynamic tracing
- sys module dtrace facility introspection (as described in the documentation, 
there's facilities in both Linux and macOS to do the same thing)

All of those might be added later.

I also left out instance new/delete probes for now since I had problems making 
them stable. We might still be able to squeeze them in if we're quick about it.

Dave, if you have an idea how to integrate your FrameObject pointer in a way 
that compiles for DTrace, too, this is the time to speak up :)

Jesús, if you'd like to add the ustack helper back in a separate patch and add 
to the documentation how this can be tested, that's great, too. I just didn't 
feel comfortable committing code myself that I have no way of testing, 
especially that it required more code (the offsets, including the generated .h 
here and there, etc.).

--
assignee:  -> lukasz.langa
nosy: +lukasz.langa
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[cburroughs]

Changes by cburroughs :


--
nosy: +cburroughs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Augie Fackler]

Changes by Augie Fackler :


--
nosy: +durin42

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Nikhil Benesch]

Nikhil Benesch added the comment:

Since it looks like development on this has stalled, I took the liberty of 
compiling all prior patches

https://github.com/benesch/python-dtrace-tracker

and am actively working on merging them into one superpatch. I am neither a 
Python nor a DTrace expert, so I'm happy to step aside should one of you more 
experienced folks have something ready.

A few thoughts so far:

* SPARC support? Can we drop this? It seems to add a bunch of intrusive
  inlined NOPs—that I'm not sure are even necessary.

* @jcea, is this the original DTrace patch? 
https://github.com/benesch/python-dtrace-tracker/blob/master/dtrace-solaris-oracle-24.patch

  I'm trying to determine where those NOPs came from.

Cheers!

--
nosy: +benesch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Xavier Morel]

Changes by Xavier Morel :


--
nosy: +xmorel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Bohuslav "Slavek" Kabrda]

Bohuslav "Slavek" Kabrda added the comment:

I'd really prefer IRC, if at all possible for you :)
When online, I'm usually on #fedora-python, but I can really come any channel 
of your choice.

Sorry for the delays in communication, I've got a lot on my plate right now and 
I'll probably not be very quick to respond

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Dave Malcolm]

Changes by Dave Malcolm :


--
nosy: +dmalcolm

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Jesús Cea Avión]

Jesús Cea Avión added the comment:

Bohuslav, do you have Jabber/XMPP?. Mine is "j...@jabber.org". This thing will 
require real time communication.

I rather prefer XMPP/Jabber, but I could accept IRC :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21590] Systemtap and DTrace support

[Bohuslav "Slavek" Kabrda]

New submission from Bohuslav "Slavek" Kabrda:

This is a tracking bug for development of combined systemtap and dtrace patch 
for Python. The separate patches were submitted at [1] (systemtap) and [2] 
(dtrace).
Since it was agreed that it'd be best to merge the two patches to reuse as much 
code as possible between them, I'm opening this issue to track the progress and 
discuss possible improvements and suggestions.

[1] http://bugs.python.org/issue14776
[2] http://bugs.python.org/issue13405

--
components: Interpreter Core
messages: 219219
nosy: bkabrda, jcea
priority: normal
severity: normal
status: open
title: Systemtap and DTrace support
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com