[issue4111] Add DTrace probes

[Mark Wielaard]

Changes by Mark Wielaard :


--
nosy: +mjw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Dave Malcolm]

Changes by Dave Malcolm :


--
nosy: +dmalcolm

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Jim Baker]

Changes by Jim Baker :


--
nosy: +jbaker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Ah, I misread the Apple function-return probe code. Its extra argument
is the type name of the return object or "error" if an exception was
raised, not the returned object itself. Could be useful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

James McIlree from Apple has informed me on dtrace-discuss that ustack
helpers cannot currently be built on OS X. Bummer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

We could probably merge Apple's and Sun's probes without too much
trouble. Apple simply extended function-entry to include the argcount in
addition to Sun's (filename, funcname, lineno) arguments. We could use
Apple's probe while retaining compatibility with Sun-oriented DTrace
scripts already in existence.

The two different function-returns are a bit of a problem. I recommend
starting with Sun's argument list because there are already a number of
quite useful scripts that use it in the DTraceToolkit. We could extend
Sun's argument list to add the *object pointer, but I don't actually
know of a use case. I haven't seen a script in the wild that uses it. It
seems like it would be tricky to write something in a DTrace script that
could make much use of it besides printing out the address. Maybe you
could navigate your way down to the type name, but that might be
unreliable. I suggest adding the *object pointer argument only if we can
devise a use case or if one of the Apple folks pop up with one. I don't
feel too bad making Apple devs modify a single character from their
internal scripts if they don't make them public.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

Robert> Is there any interest in my expanding the list of probes? 

Yes.  Jeff Garrett (a guy I work with) added some more DTrace probes to a
2.4 source tree at work.  I mentioned them in an earlier message.  I'll
check with him at work tomorrow and see about making sure they can be
released.

I think it's important to get Sun and Apple in sync as well.  Maybe more
important than adding new probes.  (Though if the Python community drives
the creation of new probes perhaps both will pick them up.)

Skip

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Jesús Cea Avión]

Jesús Cea Avión  added the comment:

> Is there any interest in my expanding the list of probes? 

Definitively!!!.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Is there any interest in my expanding the list of probes? Ruby has quite
a few more than function-entry and function-return, to give some
examples of what is possible:

http://dev.joyent.com/projects/ruby-dtrace/wiki/Ruby+DTrace+probes+and+arguments

I think that adding probes that correspond to PyTrace_LINE and
PyTrace_EXCEPTION would be straightforward and worthwhile. PyTrace_C_*
may also be worthwhile, but you can probably accomplish similar things
with the normal pid probes if you know the C function names (although
something like printing the name of a raised exception will probably
require a dedicated probe).

Adding probes to replicate what the LLTRACE configuration option did,
but dynamically, might be interesting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Hmm, wait a second. Never mind. The Solaris patches don't have ceval.o
on the line for compiling phelper.o, either.

If dtrace needs to resolve the symbol PyEval_EvalFrameEx in an object
file, how does it know to look in ceval.o for the phelper.d? WITH_DTRACE
is defined in ceval.c and PyEval_EvalFrameEx is in ceval.o, but I don't
understand how that relates to compiling phelper.d.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Ah, duh, of course. The problem here with PyEval_EvalFrameEx is that I
don't have ceval.o on the command line *at all* since OS X's dtrace
doesn't support -G. It doesn't appear to accept ceval.o with -h, either,
so I suppose that adding the ustack helper may simply be impossible
unless Apple adds the -G flag.

Thanks for you help John.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Ted Leung]

Ted Leung  added the comment:

On Apr 22, 2009, at 1:24 PM, Skip Montanaro wrote:

>
> Skip Montanaro  added the comment:
>
> Sorry, I've been away from this issue.  I was sort of hoping the Sun  
> and
> Apple folks would just work things out amongst themselves and  
> present us
> with a fait accompli. ;-) I'll try to mess around with this a little.

I've been in touch with some folks at Apple about this, but it's not  
at the very top of their priority list

--
Added file: http://bugs.python.org/file13741/unnamed

___
Python tracker 

___On Apr 22, 2009, at 1:24 
PM, Skip Montanaro wrote:Skip 
Montanaro s...@pobox.com> added the 
comment:Sorry, I've been away from this issue.  I was sort of 
hoping the Sun andApple folks would just work things out amongst themselves 
and present uswith a fait accompli. ;-) I'll try to mess around with this a 
little.I've been in touch with 
some folks at Apple about this, but it's not at the very top of their priority 
list
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[John Levon]

John Levon  added the comment:

Yes, my mistake. I noticed this typo in the original patch:

207 +PyObject *
208 +PyEval_EvalFrameexEx(PyFrameObject *f, int throwflag))

Can you:

 - verify that HAVE_DTRACE is indeed defined for ceval.c
 - do an nm on ceval.o and look which Eval* functions you do have

Somehow, PyEval_EvalFrameEx seems not to be ending up in the .o file. It
should be easy for you to work out why...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Skip> Perhaps not quite on-topic for this tracker item, but it bugs me
that the
Skip> mere compilation of a D script requires root privileges.

It doesn't. "dtrace -G" and "dtrace -h" (the only "mere compilation"
that dtrace does) run without root privileges. That line you quoted was
actually probing a process and thus needed root privileges.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

Sorry, I've been away from this issue.  I was sort of hoping the Sun and
Apple folks would just work things out amongst themselves and present us
with a fait accompli. ;-) I'll try to mess around with this a little.

Robert> $ sudo dtrace ...

Perhaps not quite on-topic for this tracker item, but it bugs me that the
mere compilation of a D script requires root privileges.  I guess there's
not much we can do about this other than complain to Sun and Apple though.

Skip

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

John, -Z does not appear to help:

$ sudo dtrace -Z -n 'pid$target::PyEval_EvalFrameEx:entry' -c python   
dtrace: description 'pid$target::PyEval_EvalFrameEx:entry' matched 0 probes

I'm not sure how that would help. If I'm reading the man page correctly,
that just let's dtrace continue on even when the probe specifiers do not
match anything. It still doesn't let me probe these functions.

The only modification I made to phelper.d was to add the stanza of
#defines given a few messages up. The failing line expands in the C
preprocessor to this line (expanding at_evalframe(), then startframe and
endframe):

dtrace:helper:ustack: /((uintptr_t)arg0 >=
((uintptr_t)&``PyEval_EvalFrameEx) &&  (uintptr_t)arg0 < (
(uintptr_t)&``PyEval_EvalCodeEx))/ 

This is based on the phelper.d in dtrace.diff attached to this ticket by
Skip. It is the same (modulo my #defines) as the phelper.d in
Python26-07-dtrace.diff from OpenSolaris as linked by Ted.

Hmm, I am applying this to Python 2.5.4, though. Perhaps I need to go
back to the Python25-07-dtrace.diff, which does appear to use
PyEval_EvalFrame. PyEval_EvalFrameEx is in Python 2.5, so I thought I
would try the latest and greatest.

If I were to process it with dtrace -h, do you think that I could name
it as a .c and compile it into an object file with gcc? Or does dtrace
-G do significantly more stuff behind the scenes to generate its object
file?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[John Levon]

John Levon  added the comment:

Robert, I have no idea how Mac OS does pstack helpers without generating
object files, sorry.

> no simply pid$target:a.out:: probes available.

Hmm. Try adding -Z to see if that helps.

> /Users/rkern/hg/Python-2.5.4/Include/phelper.d: line 110: relocation
> remains against user symbol D``PyEval_EvalFrameEx (offset 0x5)

This is trying to tell you that there's no such function. Indeed, this
isn't mentioned in my original patches, have you been editing it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

This is on an Intel Core 2 Duo running OS X 10.5.6. __i386 is defined.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Laszlo (Laca) Peter]

Laszlo (Laca) Peter  added the comment:

Look at these lines at the beginning of phelper.d:

#if defined(__i386)
#definestartframe PyEval_EvalFrameEx
#defineendframe PyEval_EvalCodeEx
#elif defined(__amd64)
#definePyEval_EvalFrameEx PyEval_EvalFrameExReal
#definestartframe PyEval_EvalFrameExReal
#defineendframe PyEval_EvalCodeEx
#elif defined(__sparc)
#definePyEval_EvalFrameEx PyEval_EvalFrameExReal
#definestartframe PyEval_EvalFrameEx
#defineendframe PyEval_EvalFrameExReal
#endif

You may need to adjust these if your compiler defines
something different from the __i386 or __amd64.
I guess an

#else
#error your architecture was not recognized
#endif

would be useful here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Got a bit farther. Adding this stanza to the top of phelper.d gets past
the issues in the headers:

#ifdef __APPLE__
#define _SYS_TIME_H_
#define _SYS_SELECT_H_
#define __MATH_H__
#define _OS__OSBYTEORDER_H
#define _FD_SET
#define __GNUC_VA_LIST
#endif /* __APPLE__ */


However, I now get a more legitimate dtrace compilation error that John
might be able to help us interpret:

$ dtrace -o /Users/rkern/hg/Python-2.5.4/Include/phelper.h -DPHELPER 
-I. -IInclude -I/Users/rkern/hg/Python-2.5.4/Include  -C -h -s
/Users/rkern/hg/Python-2.5.4/Include/phelper.d
dtrace: failed to compile script
/Users/rkern/hg/Python-2.5.4/Include/phelper.d: line 110: relocation
remains against user symbol D``PyEval_EvalFrameEx (offset 0x5)


I also tried running this without -DPHELPER as a regular DTrace script
rather than a ustack helper and ran into a problem that I've noticed
with any OS X Python build I've tried. I cannot seem to probe any of the
C functions in the Python interpreter. There are no simply
pid$target:a.out:: probes available. I'm wondering if that is an effect
of their being in a .framework, but I think I've been able to probe
other symbols other .frameworks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Robert Kern  added the comment:

Skip, it doesn't appear that the ustack helper is getting incorporated
into the OS X build anywhere. This rule is obviously wrong (compiling
the wrong input file with the wrong flags):

Include/phelper.h: $(srcdir)/Include/phelper.d
dtrace -o $@ $(DFLAGS) -C -h -s $(srcdir)/Python/python.d

I *think* it should be something along these lines:

Include/phelper.h: $(srcdir)/Include/phelper.d
dtrace -o $@ -DPHELPER $(DFLAGS) $(CPPFLAGS) -C -h -s
$(srcdir)/Include/phelper.d

There are some static functions in the standard system headers that get
pulled in that need to be avoided. The #define _SYS_STAT_H up at the top
avoids similar problems on Solaris. I'm working through the OS X headers
now, but I'm stuck on trying to avoid sys/_structs.h. Unfortunately, it
does not have the #ifndef _SYS__STRUCTS_H magic up at the top that would
allow me to do so, nor any convenient #ifdefs around the offending
declaration. 

Even if that would work, phelper.h is not #included anywhere. I'm not
really sure where it needs to go. On Solaris, the object file gets
linked in, but on OS X, you don't generate an object file; you just
#include the .h. Somewhere. I'm guessing where the pydtrace.h is
currently, but I don't really know. John, do you have any insights?

Also, s/DTRADEHDRS/DTRACEHDRS/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Dan Villiom Podlaski Christiansen]

Changes by Dan Villiom Podlaski Christiansen :


--
nosy: +danchr
nosy_count: 12.0 -> 13.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Chris Miles]

Changes by Chris Miles :


--
nosy: +chrismiles

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Joe Ranieri]

Changes by Joe Ranieri :


--
nosy: +sirg3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[John Levon]

John Levon  added the comment:

I haven't seen "shorter than expected" message before, sounds like a Mac
OS X specific thing.

As for never getting any probes out, this is where it gets fun.
Debugging this is very tricky indeed: it involves you dropping into the
kernel debugger and looking at the interpreter's state. Most likely, the
compiler is doing something different with the relevant Python
functions, causing the logic in the DTrace ustack probe to fail.

I don't even know how the Apple guys debug this sort of thing, but
they'd definitely need to be involved.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Jesús Cea Avión]

Changes by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Ted Leung]

Ted Leung  added the comment:

I didn't run auto(re)conf.  After I did that, all was well.  However,  
the ustack provider doesn't appear to be working correctly.   I tried  
running the py_profile.d from the DTrace toolkit, and it doesn't show  
any stack traces, and when the script starts up it says

cc1: warning: /dev/fd/5 is shorter than expected

The basic function entry/exit probes appear to be working.

John +nosy'ed himself, so perhaps he'll have some insight?

On Jan 27, 2009, at 3:39 PM, Skip Montanaro wrote:

>
> Skip Montanaro  added the comment:
>
> Ted> I tried building this on my Mac and got this;
>
> Forgive me if I'm preaching to the choir here.
>
> Did you run autoconf or autoreconf after applying the patch?  If not,
> @DTRACEOBJS@ would not be a substitutable string.  It's fairly  
> common (at
> least in the Python community) to omit modified configure scripts  
> from these
> sorts of patches because the changes to generated configure scripts  
> between
> different versions of autoconf are so massive that they dwarf the  
> actual
> functional changes in the patch, often by a couple orders of  
> magnitude.
>
> Skip
>
> ___
> Python tracker 
> 
> ___

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

Ted> I tried building this on my Mac and got this;

Forgive me if I'm preaching to the choir here.

Did you run autoconf or autoreconf after applying the patch?  If not,
@DTRACEOBJS@ would not be a substitutable string.  It's fairly common (at
least in the Python community) to omit modified configure scripts from these
sorts of patches because the changes to generated configure scripts between
different versions of autoconf are so massive that they dwarf the actual
functional changes in the patch, often by a couple orders of magnitude.

Skip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Ted Leung]

Ted Leung  added the comment:

I tried building this on my Mac and got this;

ar cr libpython2.7.a Python/_warnings.o Python/Python-ast.o Python/ 
asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/ 
compile.o Python/codecs.o Python/errors.o Python/frozen.o Python/ 
frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o  
Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/ 
graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/ 
modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o  
Python/pyarena.o Python/pyfpe.o Python/pymath.o Python/pystate.o  
Python/pythonrun.o Python/structmember.o Python/symtable.o Python/ 
sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o  
Python/pystrtod.o Python/formatter_unicode.o Python/formatter_string.o  
Python/dynload_shlib.o @DTRACEOBJS@  Python/mactoolboxglue.o Python/ 
thread.o
ar: @DTRACEOBJS@: No such file or directory
make: *** [libpython2.7.a] Error 1

my configure line was:

./configure --enable-framework --enable-toolbox-glue --with-threads -- 
enable-dtrace

On Jan 25, 2009, at 2:00 PM, Skip Montanaro wrote:

>
> Skip Montanaro  added the comment:
>
> Here's a patch against the current trunk (2.7) which compiles on my  
> Mac.  It
> adds a --with-dtrace configure option.  The code checks to see if  
> the -G
> option is understood by the dtrace command.  If so, dtrace support  
> is added
> Sun-style.  If not we do things Apple's way.
>
> The existing test cases pass on Apple except for one case in  
> test_sys which
> tries to confirm the size of a frame object.  I added an #ifdef  
> WITH_DTRACE
> around the extra slot but there is no way of telling from Python  
> code that
> this extra slot is there.  That would have to somehow be exposed to  
> the
> Python programmer so the test can be adjusted.  All tests pass when
> --with-dtrace is omitted.  I have not yet tried this on Solaris.  I  
> will try
> to get to it this week if someone doesn't beat me to it.
>
> There are as yet no new dtrace test cases so I can't confirm that  
> the added
> dtrace support actually works.  There are also no documentation  
> updates.
> Unlike most compile options this adds a significant new feature to the
> runtime environment so some documentation changes are probably  
> called for.
>
> Added file: http://bugs.python.org/file12861/dtrace.diff
>
> ___
> Python tracker 
> 
> ___

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[John Levon]

Changes by John Levon :


--
nosy: +movement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

Here's a patch against the current trunk (2.7) which compiles on my Mac.  It
adds a --with-dtrace configure option.  The code checks to see if the -G
option is understood by the dtrace command.  If so, dtrace support is added
Sun-style.  If not we do things Apple's way.

The existing test cases pass on Apple except for one case in test_sys which
tries to confirm the size of a frame object.  I added an #ifdef WITH_DTRACE
around the extra slot but there is no way of telling from Python code that
this extra slot is there.  That would have to somehow be exposed to the
Python programmer so the test can be adjusted.  All tests pass when
--with-dtrace is omitted.  I have not yet tried this on Solaris.  I will try
to get to it this week if someone doesn't beat me to it.

There are as yet no new dtrace test cases so I can't confirm that the added
dtrace support actually works.  There are also no documentation updates.
Unlike most compile options this adds a significant new feature to the
runtime environment so some documentation changes are probably called for.

Added file: http://bugs.python.org/file12861/dtrace.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Brett Cannon]

Changes by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

Laca> Please see here for discussion about the -G flag on OS X:
Laca> http://markmail.org/message/4nheqnexjr2o6mcx

Laca> If I read it correctly, on OS X, you will need to use -h instead
Laca> of -G and it won't emit an object file (dtrace.o) so you will not
Laca> need to link it.

Laca> Unfortunately, this means that the makefile will have to be
Laca> different on Solaris and OS X :(

We can worm around that using configure.  Thanks for the reference.  That
will go a long way toward getting the Sun dtrace probes working on Mac OSX.
For reference by others, here's Lee Packham's dtrace patch for PostgreSQL on
Mac OSX: 
http://blog.leenux.org.uk/wp-content/uploads/2008/10/postgresql-825-fixosxdtracediff.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Laszlo (Laca) Peter]

Laszlo (Laca) Peter  added the comment:

Please see here for discussion about the -G flag on OS X:
http://markmail.org/message/4nheqnexjr2o6mcx

If I read it correctly, on OS X, you will need to use -h
instead of -G and it won't emit an object file (dtrace.o)
so you will not need to link it.

Unfortunately, this means that the makefile will have to
be different on Solaris and OS X :(

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

me> I get an error later running dtrace which I have yet to investigate.

Apple's dtrace program doesn't support the -G flag.  When I remove it from
Makefile.pre.in and rebuild I get an error about privileges:

dtrace -o Python/dtrace.o  -C -s ./Python/python.d Python/ceval.o
dtrace: failed to initialize dtrace: DTrace requires additional privileges

I tried adding "sudo" to the dtrace command.  Then it prompts for my
password and emits this error:

sudo dtrace -o Python/dtrace.o  -C -s ./Python/python.d Python/ceval.o
Password:
dtrace: failed to compile script ./Python/python.d: line 11: extraneous 
argument 'Python/ceval.o' ($1 is not referenced)

I'm not sure what to do at this point.  I'm not really a dtrace person.
Perhaps the Apple and Sun dtrace experts can offer a way out of this little
corner.

Skip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

After applying the patch and reconfiguring I get compilation errors in
Python/ceval.c.  I suspect it's because there is a new header file,
Python/python.h.  That's probably found by the #include directive in
favor of Include/Python.h because of the Mac's case-insensitive file
system.  Changing the Makefile targets and actions so it's named
python-dtrace.h seems to work a bit better.  I get an error later
running dtrace which I have yet to investigate.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Ted Leung]

Ted Leung  added the comment:

On Jan 22, 2009, at 1:35 PM, Skip Montanaro wrote:

>
> Skip Montanaro  added the comment:
>
> So I completely dropped the ball on this.  It appears we have some
> folks from Sun and Brett surmised that Ronald Oussoren would be the
> likely person to do the heavy lifting on the Apple side of things.
> Ronald, I've made you nosy.  I will try to get the Solaris patch Ted
> referenced running against the trunk (2.7) and py3k (3.1) code bases,
> but it would be helpful if you could take a quick stab at it as well.

Great.  Actually the latest version of the patch is this one: 
.   I tried to quickly compile it on MacOS 10.5, Python 2.6.1, but  
it didn't work right out of the box.

Ted

Added file: http://bugs.python.org/file12835/unnamed

___
Python tracker 

___On Jan 22, 2009, at 1:35 PM, Skip 
Montanaro wrote:Skip Montanaro s...@pobox.com> added the comment:So I 
completely dropped the ball on this.  It appears we have somefolks 
from Sun and Brett surmised that Ronald Oussoren would be thelikely person 
to do the heavy lifting on the Apple side of things.Ronald, I've made you 
nosy.  I will try to get the Solaris patch Tedreferenced running 
against the trunk (2.7) and py3k (3.1) code bases,but it would be helpful 
if you could take a quick stab at it as 
well.Great.  Actually the latest 
version of the patch is this one: h
 
ttp://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python26-07-dtrace.diff>.
   I tried to quickly compile it on MacOS 10.5, Python 2.6.1, but it 
didn't work right out of the 
box.Ted___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro  added the comment:

So I completely dropped the ball on this.  It appears we have some
folks from Sun and Brett surmised that Ronald Oussoren would be the
likely person to do the heavy lifting on the Apple side of things.
Ronald, I've made you nosy.  I will try to get the Solaris patch Ted
referenced running against the trunk (2.7) and py3k (3.1) code bases,
but it would be helpful if you could take a quick stab at it as well.

--
assignee:  -> skip.montanaro
nosy: +ronaldoussoren
versions: +Python 2.7, Python 3.1 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Robert Kern]

Changes by Robert Kern :


--
nosy: +robert.kern

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Laszlo (Laca) Peter]

Laszlo (Laca) Peter <[EMAIL PROTECTED]> added the comment:

I'm the python package maintainer at Sun.
We would really like to get the dtrace probes upstream, let me know how
I can help.

--
nosy: +laca

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Ted Leung]

Ted Leung <[EMAIL PROTECTED]> added the comment:

And courtesy of Philip Jenvey, here I am.

I would *really* like to work to help make this happen.   Laszlo Peter
at Sun has been doing the ports of Python on Solaris, but we are not up
to 2.6 just yet.  I'm attaching a pointer to his patches against 2.5:

http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python25-07-dtrace.diff

These patches include John Levon's ustack provider, which if you care
about DTrace and Python, you want to have.

I've also had some conversations with people at Apple, and they have
agreed that they will pull DTrace probes from python.org if they got in
there.  That would solve the diverging probe problem.

I'd love to have a discussion about DTrace probe futures for CPython --
probably on python-dev.

--
nosy: +twleung

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Brett Cannon]

Brett Cannon <[EMAIL PROTECTED]> added the comment:

On Thu, Nov 13, 2008 at 08:05, Skip Montanaro <[EMAIL PROTECTED]> wrote:
>
> Skip Montanaro <[EMAIL PROTECTED]> added the comment:
>
> Brett> http://www.opensource.apple.com/darwinsource/10.5.5/python-30.1.2/
>...
>Brett> 
> http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff
>
> Thanks for the pointers.  I'll work on getting a uniform patch which
> incorporates both.  (Though some of the comments in the OpenSolaris patch
> are a bit scary.)
>
>Brett> There's another problem (I think) with DTrace probes ... if the
>Brett> Apple guys release Apple-Probe-Python.d and the Sun guys release
>Brett> Sun-Probe-Python.d (just two scripts) and we setup our probes
>Brett> differently, one or both will fail (because one may expect a
>Brett> probe for foo while another expects a probe for bar).  Kind of
>Brett> sucks, considering Sun was first and Apple chose to probe Python
>Brett> differently.
>
> Yeah, that would suck.  It would be nice if they could agree on a common set
> of probes.  I don't know that we have any contacts within the two
> development communities but if we can scare some up maybe we can get them to
> talk to each other. :-/
>

Obviously Ronald is the Apple insider to talk to. If you want Sun, you
might want to talk to Ted Leung and see who he can put you in touch
with.

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro <[EMAIL PROTECTED]> added the comment:

Brett> http://www.opensource.apple.com/darwinsource/10.5.5/python-30.1.2/
...
Brett> 
http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff

Thanks for the pointers.  I'll work on getting a uniform patch which
incorporates both.  (Though some of the comments in the OpenSolaris patch
are a bit scary.)

Brett> There's another problem (I think) with DTrace probes ... if the
Brett> Apple guys release Apple-Probe-Python.d and the Sun guys release
Brett> Sun-Probe-Python.d (just two scripts) and we setup our probes
Brett> differently, one or both will fail (because one may expect a
Brett> probe for foo while another expects a probe for bar).  Kind of
Brett> sucks, considering Sun was first and Apple chose to probe Python
Brett> differently.

Yeah, that would suck.  It would be nice if they could agree on a common set
of probes.  I don't know that we have any contacts within the two
development communities but if we can scare some up maybe we can get them to
talk to each other. :-/

Skip

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Brett Hoerner]

Brett Hoerner <[EMAIL PROTECTED]> added the comment:

On Wed, Nov 12, 2008 at 9:31 PM, Skip Montanaro <[EMAIL PROTECTED]> wrote:
> I see the reference to Apple in your original post, but can't find anything
> related to dtrace & python starting from the URL you gave.  Do you have
> something more specific?

http://www.opensource.apple.com/darwinsource/10.5.5/python-30.1.2/

That isn't Python 3.0, that's what I guess they call Python "Apple
Version 30"?  Anyways, it's their Python 2.5.1 and their "patches"
against it are actually ed scripts, contained in fix/

That's where my patch comes from, it's just changing the .ed scripts
to a patch and applying it against 2.6

> At this point Jeff's code does a fair bit more than simply tracing the
> CALL_FUNCTION opcode but needs some work with the other CALL_FUNCTION_*
> opodes.  It also has some obmalloc tracing which I've not yet tested.  I
> would have thought Apple would more heavily instrument the interpreter than
> it appears they have.

Sun has released a patch against DTrace that probes more than just
function calls also.  At least, it provides line number and some other
information, 
http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff

I couldn't figure out why it broke the compile on OS X though, and
could only get it working on Solaris.

There's another problem (I think) with DTrace probes ... if the Apple
guys release Apple-Probe-Python.d and the Sun guys release
Sun-Probe-Python.d (just two scripts) and we setup our probes
differently, one or both will fail (because one may expect a probe for
foo while another expects a probe for bar).  Kind of sucks,
considering Sun was first and Apple chose to probe Python differently.
 Now we have to pick one or make a 3rd... I could be very mistaken
here, though.

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro <[EMAIL PROTECTED]> added the comment:

Brett> They have released the changes, that's what my patch (attached to
Brett> the issue) is based on.

I see the reference to Apple in your original post, but can't find anything
related to dtrace & python starting from the URL you gave.  Do you have
something more specific?

At this point Jeff's code does a fair bit more than simply tracing the
CALL_FUNCTION opcode but needs some work with the other CALL_FUNCTION_*
opodes.  It also has some obmalloc tracing which I've not yet tested.  I
would have thought Apple would more heavily instrument the interpreter than
it appears they have.

Brett> It's working, it just needs to be cleaned up (it will fail, I
Brett> believe, for people on systems without DTrace - as I said I'm not
Brett> very familiar with Makefiles).

I took care of the configure.in/Makefile.pre.in stuff today.  The intent is
that you would configure using --enable-dtrace then not have to do anything
else to build a dtrace-aware interpreter.

Brett> You can use it now against 2.6 and probably with very few changes
Brett> against 2.5.  

We are using 2.4 at work and it works there.  I'm fairly certain we should
be able to get it working for the entire 2.x and 3.x series with only a
little effort.

(I still need to get approval to release it, but I don't think that will be
a big deal.  I've already alerted my boss and he's generally receptive to
such things.)

Skip

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Brett Hoerner]

Brett Hoerner <[EMAIL PROTECTED]> added the comment:

They have released the changes, that's what my patch (attached to the
issue) is based on.

It's working, it just needs to be cleaned up (it will fail, I believe,
for people on systems without DTrace - as I said I'm not very familiar
with Makefiles).

You can use it now against 2.6 and probably with very few changes
against 2.5.

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Skip Montanaro <[EMAIL PROTECTED]> added the comment:

It appears that Apple has dtracified their Python exeutable in Leopard.
Any chance that they can be persuaded to release a patch?  I'm working
on a patch (based on some work from a friend at work), but it seems like
Apple already has a horse in the race.  It would be nice if we could
see what they have done.

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Alexander Belopolsky]

Changes by Alexander Belopolsky <[EMAIL PROTECTED]>:


--
nosy: +belopolsky

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Skip Montanaro]

Changes by Skip Montanaro <[EMAIL PROTECTED]>:


--
nosy: +skip.montanaro

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Brett Cannon]

Brett Cannon <[EMAIL PROTECTED]> added the comment:

At one point we were told Apple would try to backport their dtrace
instrumentation. I don't know what the status of that is, but it
obviously has not happened.

--
nosy: +brett.cannon

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4111] Add DTrace probes

[Brett Hoerner]

New submission from Brett Hoerner <[EMAIL PROTECTED]>:

It would be great if the main Python distribution supported DTrace on
(Open)Solaris / FreeBSD / OS X.

I've attached a patch against 2.6 that instruments function calls.  It's
from the ed scripts included in Apple's Python distribution:
http://developer.apple.com/opensource/index.html (PSF License)

It is imperfect because I wasn't sure how to do the equivalent of IFDEF
in the Makefile.pre.in, as you can see this patch will make ceval.o
depend on pydtrace.h, which depends on having a local `dtrace` command.
 The rest of the patch properly uses IFDEFs.

--
components: Build, Installation
files: python-2.6-dtrace.diff
keywords: patch
messages: 74670
nosy: bretthoerner
severity: normal
status: open
title: Add DTrace probes
type: feature request
versions: Python 2.6
Added file: http://bugs.python.org/file11772/python-2.6-dtrace.diff

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com