Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
On Fri, 2007-02-16 at 02:19 -0800, Zachary Amsden wrote: > Doesn't stop_machine_run already take care of getting you out of all > kernel threads? So you can only be sleeping, not preempted, in which > case, this might not be an issue? No, stop_machine only protects against preempt-disabled regions. I thought about making that particularly hammer bigger, but noone had needed it yet. It would simply require a runtime way of disabling further preemption and flushing preempted threads (AFAICT there can be no deadlock issues with this). Rusty. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Eric W. Biederman wrote: > Have machine_kexec_prepare fail. > > I think your machine description or paravirt_ops or whatever it is needs > to hook both machine_kexec_prepare and machine_kexec. > > I know there actually has been some work to get kexec actually working under > Xen but I don't know where that has gone. Works, modulo bugs. Shouldn't be hard to get work on top of paravirt_ops. The current 2.6.18 code uses function pointers already because the dom0 code looks very different from domU code. It is one of the items on my todo list. I'll probably look into it once I'm done with the earlyprintk and console mess ... cheers, Gerd -- Gerd Hoffmann <[EMAIL PROTECTED]> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Keir Fraser wrote: On 16/2/07 10:19, "Zachary Amsden" <[EMAIL PROTECTED]> wrote: Doesn't stop_machine_run already take care of getting you out of all kernel threads? So you can only be sleeping, not preempted, in which case, this might not be an issue? It ensures that no (non-stopmachine) threads are running on any of the other CPUs, but it doesn't guarantee that all threads are at any form of quiescent point. They can certainly have private references to machine addresses. Otherwise, what would be the point of the process freezer? :-) Yeah, I was over-hoping on that one. At least thought it was worth suggesting as a starting point. I had similar issues considering loading VMI as a module at runtime. Zach - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
On 16/2/07 10:19, "Zachary Amsden" <[EMAIL PROTECTED]> wrote: > Doesn't stop_machine_run already take care of getting you out of all > kernel threads? So you can only be sleeping, not preempted, in which > case, this might not be an issue? It ensures that no (non-stopmachine) threads are running on any of the other CPUs, but it doesn't guarantee that all threads are at any form of quiescent point. They can certainly have private references to machine addresses. Otherwise, what would be the point of the process freezer? :-) -- Keir - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Keir Fraser wrote: On 16/2/07 07:25, "Jeremy Fitzhardinge" <[EMAIL PROTECTED]> wrote: Oh, so that's why it doesn't break when CONFIG_PREEMPT=y. In which case that preempt_disable() I spotted is wrong-and-unneeded. Why doesn't Xen work with preemption?? I've forgotten the details. Ian? Keir? Steven? Maybe it can be done. It breaks guest save/restore for us currently because threads can be sleeping with machine addresses in local storage (registers, stack). There are a few ways to achieve an acceptable solution: 1. Put processes in the freezer when we suspend. This should avoid any thread being in a critical section with machine addresses in its hand. We haven't yet investigated the performance impact of freezing processes, particularly on the downtime of live relocation. 2. Allow CONFIG_PREEMPT to be compiled in, but disable it at runtime. We could do this by, for example, reserving a bit in preempt_count() so that most preemption checks do not touch any more cache lines. I guess it would need a bit of fixing up (e.g., so that in_atomic() would not be always asserted). Even better for us would be to allow switching between involuntary and voluntary preemption at runtime. It looks as though the hook points for these two techniques are not usually compiled in at the same time, however. Doesn't stop_machine_run already take care of getting you out of all kernel threads? So you can only be sleeping, not preempted, in which case, this might not be an issue? Zach - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Keir Fraser wrote: On 16/2/07 07:25, Jeremy Fitzhardinge [EMAIL PROTECTED] wrote: Oh, so that's why it doesn't break when CONFIG_PREEMPT=y. In which case that preempt_disable() I spotted is wrong-and-unneeded. Why doesn't Xen work with preemption?? I've forgotten the details. Ian? Keir? Steven? Maybe it can be done. It breaks guest save/restore for us currently because threads can be sleeping with machine addresses in local storage (registers, stack). There are a few ways to achieve an acceptable solution: 1. Put processes in the freezer when we suspend. This should avoid any thread being in a critical section with machine addresses in its hand. We haven't yet investigated the performance impact of freezing processes, particularly on the downtime of live relocation. 2. Allow CONFIG_PREEMPT to be compiled in, but disable it at runtime. We could do this by, for example, reserving a bit in preempt_count() so that most preemption checks do not touch any more cache lines. I guess it would need a bit of fixing up (e.g., so that in_atomic() would not be always asserted). Even better for us would be to allow switching between involuntary and voluntary preemption at runtime. It looks as though the hook points for these two techniques are not usually compiled in at the same time, however. Doesn't stop_machine_run already take care of getting you out of all kernel threads? So you can only be sleeping, not preempted, in which case, this might not be an issue? Zach - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
On 16/2/07 10:19, Zachary Amsden [EMAIL PROTECTED] wrote: Doesn't stop_machine_run already take care of getting you out of all kernel threads? So you can only be sleeping, not preempted, in which case, this might not be an issue? It ensures that no (non-stopmachine) threads are running on any of the other CPUs, but it doesn't guarantee that all threads are at any form of quiescent point. They can certainly have private references to machine addresses. Otherwise, what would be the point of the process freezer? :-) -- Keir - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Keir Fraser wrote: On 16/2/07 10:19, Zachary Amsden [EMAIL PROTECTED] wrote: Doesn't stop_machine_run already take care of getting you out of all kernel threads? So you can only be sleeping, not preempted, in which case, this might not be an issue? It ensures that no (non-stopmachine) threads are running on any of the other CPUs, but it doesn't guarantee that all threads are at any form of quiescent point. They can certainly have private references to machine addresses. Otherwise, what would be the point of the process freezer? :-) Yeah, I was over-hoping on that one. At least thought it was worth suggesting as a starting point. I had similar issues considering loading VMI as a module at runtime. Zach - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Eric W. Biederman wrote: Have machine_kexec_prepare fail. I think your machine description or paravirt_ops or whatever it is needs to hook both machine_kexec_prepare and machine_kexec. I know there actually has been some work to get kexec actually working under Xen but I don't know where that has gone. Works, modulo bugs. Shouldn't be hard to get work on top of paravirt_ops. The current 2.6.18 code uses function pointers already because the dom0 code looks very different from domU code. It is one of the items on my todo list. I'll probably look into it once I'm done with the earlyprintk and console mess ... cheers, Gerd -- Gerd Hoffmann [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
On Fri, 2007-02-16 at 02:19 -0800, Zachary Amsden wrote: Doesn't stop_machine_run already take care of getting you out of all kernel threads? So you can only be sleeping, not preempted, in which case, this might not be an issue? No, stop_machine only protects against preempt-disabled regions. I thought about making that particularly hammer bigger, but noone had needed it yet. It would simply require a runtime way of disabling further preemption and flushing preempted threads (AFAICT there can be no deadlock issues with this). Rusty. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Jeremy Fitzhardinge wrote: Dan Hecht wrote: I assume you plan to eventually get all this stuff working but just want to prevent configurations that the Xen paravirt-ops isn't ready for at the moment? Instead can you do it this way: config XEN depends on PARAVIRT && !PREEMPT && HZ_100 && !DOUBLEFAULT && !KEXEC That's a bit simpler code-wise, but it does make it pretty complex to get everything just-so to even see the CONFIG_XEN option. Yes, but that is what you need to do to compile Xen - logically, to build a Xen kernel, you need to have a kernel configuration which can enable Xen - not limit the configuration of a kernel because Xen has been enabled. This is because with paravirt-ops, Xen compiled kernels may not actually run on Xen, so you can't arbitrarily drop features because you assume Xen is there. One of these has an easy solution - doublefault. You don't need to install the doublefault gate if you don't want it, and the hypervisor doesn't need to freak out if you install it, it can just ignore the gate entirely and claim #DF is not supported. Zach - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] Re: [patch 14/21] Xen-paravirt: Add XEN config options and disable unsupported config options.
Jeremy Fitzhardinge wrote: Dan Hecht wrote: I assume you plan to eventually get all this stuff working but just want to prevent configurations that the Xen paravirt-ops isn't ready for at the moment? Instead can you do it this way: config XEN depends on PARAVIRT !PREEMPT HZ_100 !DOUBLEFAULT !KEXEC That's a bit simpler code-wise, but it does make it pretty complex to get everything just-so to even see the CONFIG_XEN option. Yes, but that is what you need to do to compile Xen - logically, to build a Xen kernel, you need to have a kernel configuration which can enable Xen - not limit the configuration of a kernel because Xen has been enabled. This is because with paravirt-ops, Xen compiled kernels may not actually run on Xen, so you can't arbitrarily drop features because you assume Xen is there. One of these has an easy solution - doublefault. You don't need to install the doublefault gate if you don't want it, and the hypervisor doesn't need to freak out if you install it, it can just ignore the gate entirely and claim #DF is not supported. Zach - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
