Re: [PATCH 0/7] utrace/ptrace
This is the new iteration of Roland's utrace patch, this time with rewrite-ptrace-via-utrace + cleanups in utrace core. Tested-by: CAI Qian caiq...@redhat.com
Re: [PATCH 0/7] utrace/ptrace
On 12/22, Andrew Morton wrote: On Fri, 18 Dec 2009 02:11:16 +0100 Oleg Nesterov o...@redhat.com wrote: It allows for multiple separate tracing engines to work in parallel without interfering with each other. Higher-level tracing facilities can be implemented as loadable kernel modules using this layer. That's a bit brief. Do you have a nicer sales brochure? What are these separate tracing engines and what is their merge status and why would we want any of them, for what purpose? etc. IOW: give us a reason! First of all, utrace makes other things possible. gdbstub, nondestructive core dump, uprobes, kmview, hopefully more. I didn't look at these projects closely, perhaps other people can tell more. As for their merge status, until utrace itself is merged it is very hard to develop them out of tree. To me, even seccomp is the good example why utrace is useful. seccomp is simple, but it needs hooks in arch/ hot pathes. Contrary, utrace-based implementation is more flexible, simple, and it is completely hidden behind utrace. In my opinion, ptrace-utrace is another example. Once CONFIG_UTRACE goes away, we can remove almost all ptrace-related code from core kernel (and kill task_struct-ptrace/etc members). ftrace/etc is excellent in many ways, but even if we need the simple passive tracing it is not enough sometimes. And we have nothing else except ptrace currently. But ptrace is so horrible and unfixeable, and it has so many limitations. In fact, even the simple things like stop/ continue this thread/process are not trivial using ptrace, gdb/strace have to do a lot of hacks to overcome ptrace's limitations, and some of these hacks falls into mostly works, but that is all category. Of course, I can't promise we will have the new gdb which explores utrace facilities soon, but I think at leat utrace gives a chance. Well. I had a lot of technical discussions with Roland about utrace, but I never asked him why he created this thing ;) To me, utrace looks like vfs. Currently we have the single and very poor filesystem, ptrace. Until we add the appropriate layer, we can't expect the further improvements is this area. Oleg.
Re: [PATCH 0/7] utrace/ptrace
Well. I had a lot of technical discussions with Roland about utrace, but I never asked him why he created this thing ;) To me, utrace looks like vfs. Currently we have the single and very poor filesystem, ptrace. Until we add the appropriate layer, we can't expect the further improvements is this area. I think that is an excellent analogy, and it's one I've used before. Oleg and I have had our hands pretty full just with the infrastructure layer and with ptrace. Having this layer in the kernel is what makes it tractable for a lot of other people to collaborate on new features in this space, and that's what we want to enable and accelerate. Some of those on the CC list have worked and are working on such things, and I hope they will pipe up about those. Given the date, I suspect we might not see much from anybody on this (or anything) until January. Myself, I expect to be largely offline for the rest of the year. As Oleg mentioned, I have a cleanup/reimplementation of seccomp using utrace. That is quite a trivial use--it demonstrates how easy the utrace API makes it to do things like that, in contrast to previous solutions with arch-specific assembly hacking and so forth. I can dust that patch off and post it if anybody cares. Some other features based on utrace have been floating around for some time, posted here before. Those include uprobes, kmview, and the gdb stub. I don't which of those are quite ready for merging, but honing and polishing them gets quite a lot more doable with utrace in the tree instead of out. Thanks, Roland
Re: [PATCH 0/7] utrace/ptrace
Do you have an estimate or better numbers how the overhead of seccomp-over-utrace compares to the current in-tree seccomp? I never measured it. I would estimate that any difference one way or another is in the noise. The point of seccomp is to run a process that almost never makes any system calls. The only effects of utrace for that use are on the system call path itself, and the essential effects there (i.e. taking the tracing path vs the hot path) are the same as what the old seccomp implementation does. If you have some example uses of seccomp or something that can serve as a benchmark for it, I would be glad to measure the difference. Thanks, Roland
Re: [PATCH 0/7] utrace/ptrace
On Fri, 18 Dec 2009 02:11:16 +0100 Oleg Nesterov o...@redhat.com wrote: Hello. This is the new iteration of Roland's utrace patch, this time with rewrite-ptrace-via-utrace + cleanups in utrace core. So... should we merge this? I'll confess that I've rather forgotten why we might want this. It allows for multiple separate tracing engines to work in parallel without interfering with each other. Higher-level tracing facilities can be implemented as loadable kernel modules using this layer. That's a bit brief. Do you have a nicer sales brochure? What are these separate tracing engines and what is their merge status and why would we want any of them, for what purpose? etc. IOW: give us a reason!
Re: [PATCH 0/7] utrace-ptrace V1
5/7 belongs first and I've already merged it as prerequisite to utrace. We can send that upstream without delay. I hope it can get queued quickly regardless of the review delays for the utrace and ptrace work. All the other preparatory patches are just to introduce PT_PTRACED as the distinction between the obsolete hooks for old ptrace and the remaining ptrace-specific kludges (unsafe_exec, tracer_task, and the interference with SIGCHLD/wait semantics). IMHO it's pretty questionable to do that rather than test those statically such that under CONFIG_UTRACE the old hooks are compiled away entirely (either via #ifdef or via things that reduce to if (0)). But moreover, this is fritter in the details of coexistence with the old implementation or sequencing of phasing it out. I really have no idea what the acceptable path for that is going to be at all. In the past, upstream reactions have ranged from utrace never! to no options, have only the utrace-based ptrace exist at all. I don't know that anyone is positively in favor of conditionally having two ptrace implementations, except perhaps as a compromise position for those who would prefer us to jump in the lake and never propose utrace again. I'm not at all sure that there isn't any one of the people with de facto veto power who will be dead-set against ever having both in the source at the same time. I don't think we can answer that except in the actual upstream review. So if this is v1 for upstream review, then take this path or whatever other makes for the necessary fritter being easiest to read (which is usually perceived upstream to mean least patch text) and get on with it. Thanks, Roland
Re: [PATCH 0/7] utrace-ptrace V1
On 10/27, Roland McGrath wrote: 5/7 belongs first and I've already merged it as prerequisite to utrace. We can send that upstream without delay. I hope it can get queued quickly regardless of the review delays for the utrace and ptrace work. Agreed, I'll send it to Andrew. All the other preparatory patches are just to introduce PT_PTRACED as the distinction between the obsolete hooks for old ptrace and the remaining ptrace-specific kludges (unsafe_exec, tracer_task, and the interference with SIGCHLD/wait semantics). Yes. And, although you didn't say this, I completely agree: this is dirty hack. IMHO it's pretty questionable to do that rather than test those statically such that under CONFIG_UTRACE the old hooks are compiled away entirely (either via #ifdef or via things that reduce to if (0)). Agreed! Hopefully we can do this later. As you understand, the goal is to make the first series as small as possible, where small means the number of changes outside of ptrace.c. But moreover, this is fritter in the details of coexistence with the old implementation or sequencing of phasing it out. I really have no idea what the acceptable path for that is going to be at all. In the past, upstream reactions have ranged from utrace never! to no options, have only the utrace-based ptrace exist at all. Yes. CONFIG_UTRACE should go away, but when this will happen? We have to fix !HAVE_ARCH_TRACEHOOK arches first and to ensure bobody in arch/ plays with ptrace internals. I don't know that anyone is positively in favor of conditionally having two ptrace implementations, At least, we don't have CONFIG_UTRACE_PTRACE. I don't think we can answer that except in the actual upstream review. Yep. So if this is v1 for upstream review, Yes, I hope so. Oleg.
Re: [PATCH 0/7] utrace-ptrace V1
On 10/26, Oleg Nesterov wrote: For early review. Patches 1-5 can be sent upstream right now. The last 2 patches add the new implementation. utrace.patch can come before or after, I think after is better. Forgot to mention, this series is against Linus's tree. Oleg.
