Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Sat, 2008-01-12 at 11:21 +1100, Sven Dowideit wrote: > excellent stuff Andy! > > To start with the obvious :) > > Due to the lack of ustack helper, I am hoping to add a stack probe Two things: first of all DTrace could really benefit from a general purpose mechanism of hooking up custom stack helpers with the rest of the system. Frankly, the way jstack is done doesn't strike me as too generic a mechanism. Second, I would like to encourage those who do DTrace integration into Perl to take a look at what JavaScript is doing. They don't have a custom stack helper either but the set of probes they offer makes stack synthesis a trivial (albeit costly) matter. Thanks, Roman.
Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Fri, 2008-01-11 at 17:32 -0800, Bryan Cantrill wrote: > On Fri, Jan 11, 2008 at 04:39:25PM -0800, Roman Shaposhnik wrote: > > On Sat, 2008-01-12 at 11:21 +1100, Sven Dowideit wrote: > > > excellent stuff Andy! > > > > > > To start with the obvious :) > > > > > > Due to the lack of ustack helper, I am hoping to add a stack probe > > > > Two things: first of all DTrace could really benefit from a > > general purpose mechanism of hooking up custom stack helpers > > with the rest of the system. Frankly, the way jstack is done > > doesn't strike me as too generic a mechanism. > > Do you actually understand how it's implemented? Not fully, but I've spent a great deal of time staring at jhelper.d ;-) Seriously, though -- my experience comes mostly from reading the source. And the source code for DTrace can get pretty tricky. If you have any suggestion for me as to what would be a good intro into stack helpers -- I'd appreciate that. > It's actually quite > general -- witness the presence of ustack helpers for both Python and > PHP. (Indeed, the underlying mechanism is _so_ general that for some > time we weren't sure if we had fallen prey to false generality.) I guess we are talking about two different kinds of generality here. What I have in mind is the generality that would allow DTrace to be able to offload some of the heavylifting to the userspace. I DO understand the ramifications of that and I know full well that pushing DTrace into userspace might be perceived as the deviation from the original design. To some extent it might even reduce the generality that you seem to be referring to. > > Second, I would like to encourage those who do DTrace integration > > into Perl to take a look at what JavaScript is doing. They don't > > have a custom stack helper either but the set of probes they offer > > makes stack synthesis a trivial (albeit costly) matter. > > While the JavaScript approach is better than nothing, it is _not_ > preferable to the ustack approach Agreed. Still it is WAY better than nothing ;-) Thanks, Roman.
Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Sat, 2008-01-12 at 01:30 +, John Levon wrote: > On Fri, Jan 11, 2008 at 04:39:25PM -0800, Roman Shaposhnik wrote: > > > > Due to the lack of ustack helper, I am hoping to add a stack probe > > > > Two things: first of all DTrace could really benefit from a > > general purpose mechanism of hooking up custom stack helpers > > with the rest of the system. Frankly, the way jstack is done > > doesn't strike me as too generic a mechanism. > > Do you have something in mind? Discalimer: I can't really claim to be an expert in DTrace, my main familiarity with it comes from working on a tool that sometimes pushes DTrace to its limits and that way I'm forced to explore how these limits could be overcome. If what I have to say below is a wrong perception of the technology I'd love to be educated by the subject experts. My main beef with stack helpers is that they have to be executed inside the kernel and are subject to the usual DTrace restrictions. I would like to see a possibility of offloading more work to user space in a generic fashion. For example, we have a libcollector.so facility that is capable of reconstructing stacks under the most challenging of situations (inside the function preamble, etc.) it is way more accurate in what it reports back than ustack() it is also capable of more stack unwinding than jstack(). Of course, it runs in user space. We do have some preliminary ideas on how such functionality can be hooked to the rest of DTrace machinery. If there's a significant portion of the DTrace community that is interested in extending DTrace functionality in a way that would make it more reliant on userspace -- perhaps it is a good thing to have on the DTrace conference agenda. Thanks, Roman.
Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Jan 11, 2008, at 6:26 PM, Eric Schrock wrote:
On Fri, Jan 11, 2008 at 06:23:14PM -0800, Roman Shaposhnik wrote:
I guess we are talking about two different kinds of generality
here.
What I have in mind is the generality that would allow DTrace to be
able to offload some of the heavylifting to the userspace. I DO
understand the ramifications of that and I know full well that
pushing
DTrace into userspace might be perceived as the deviation from the
original design. To some extent it might even reduce the generality
that you seem to be referring to.
The issue is not one of design, but of the constraints on the problem.
If we are talking about *u*stack, than I disagree with the way you're
setting up constraints. The *u*stack is by definition a userspace
related
operation. Not only that, but calling ustack() is declared void for a
reason.
Calling it changes the output of the DTrace but has no way of affecting
the rest of the DTrace script's internal logic.
DTrace actions must be executed in probe context.
Please don't redefine the problem. We are not talking about arbitrary
actions here.
Heavy lifting can be
done via post-processing in userland (symbol translation, system()
actions, etc), but the data gathering must be done in arbitrary
context.
What data gathering? Could you, please, be more specific? What data
there is that can't be leveraged from the userspace? And again, I'm
talking specifically about ustack() here.
How would you call arbitrary userspace code from, say, an interrupt
handler?
I'm not suggesting that at all. And in fact for Project D-Light we
devised
a mechanism for getting much more accurate stacktraces from userspace
compared to what DTrace is capable of. E.g.:
self uint32_t stkIdx;
collector$1:::ustack
{
printf("0 %d %d %u\n", tid, self->stkIdx++, arg0<<32|
(arg1&0x));
}
syscall::read:return
/pid == $1/
{
fname = (self->fd == 0 ) ? "" : fnames[self->fd];
printf("1 %d %d %d %d \"%s\" %d %d\n",
cpu,
tid,
timestamp,
0,
fname,
arg1,
self->stkIdx);
raise(29);
}
The awkwardness here comes from the fact that we have to:
* make libcollector available in the address space of the process
* use a raise(29) in order to poke userspace instead of
something more
DTrace specific.
Otherwise it works very nice. We get all the benefits of userspace
code:
we can match dlopen/dlclose events and such and our Java stack are
better than jstack().
Thanks,
Roman.
Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Jan 12, 2008, at 12:26 PM, Adam Leventhal wrote: On Fri, Jan 11, 2008 at 06:12:57PM -0800, Roman Shaposhnik wrote: My main beef with stack helpers is that they have to be executed inside the kernel and are subject to the usual DTrace restrictions. I would like to see a possibility of offloading more work to user space in a generic fashion. For example, we have a libcollector.so facility that is capable of reconstructing stacks under the most challenging of situations (inside the function preamble, etc.) it is way more accurate in what it reports back than ustack() it is also capable of more stack unwinding than jstack(). Of course, it runs in user space. All respect to libcollector.so, but the conditions under which DTrace must gather a stack are far more constrained than those of a user-land program. The DTrace code from probe context can't do I/O or block -- it can only load and store. Even looping is a bit of a stretch. I don't disagree at all! And I'm not trying to say that *stack() should be replaced. I'm merely trying to explore whether the best of both worlds approach is doable. What we've done for Project D-Light clearly shows that for some applications it is doable. At this point I really would like to explore whether something like that could be generalized. See my earlier email to Eric. I understand his argument perfectly well. But I'm still not convinced that it makes the idea void. If convincing me is perceived as a waste of time -- that's ok. I guess I should get back to you when I earn my Solaris kernel guru degree :-) If on the other hand, I'm not explaining what I want clearly -- I can try again. Thanks, Roman.
