Re: Tiny cpusets -- cpusets for small systems?

2008-02-25 Thread Max Krasnyanskiy
Paul Jackson wrote: So. I see cpusets as a higher level API/mechanism and cpu_isolated_map as lower level mechanism that actually makes kernel aware of what's isolated what's not. Kind of like sched domain/cpuset relationship. ie cpusets affect sched domains but scheduler does not use cpusets

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-25 Thread Max Krasnyanskiy
Hi Peter, Sorry for delay in reply. Please, wrap your emails at 78 - most mailers can do this. Done. On Fri, 2008-02-22 at 14:05 -0800, Max Krasnyanskiy wrote: Peter Zijlstra wrote: On Thu, 2008-02-21 at 18:38 -0800, Max Krasnyanskiy wrote: List of commits cpuisol: Make cpu

Re: [RFC] Genirq and CPU isolation

2008-02-25 Thread Max Krasnyanskiy
Randy Dunlap wrote: On Fri, 22 Feb 2008 22:19:18 -0800 Max Krasnyansky wrote: Hi Max, diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 438a014..e74db94 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -488,6 +491,26 @@ void free_irq(unsigned int irq, void *dev_id) }

Re: [RFC] Genirq and CPU isolation

2008-02-25 Thread Max Krasnyanskiy
Randy Dunlap wrote: On Fri, 22 Feb 2008 22:19:18 -0800 Max Krasnyansky wrote: Hi Max, diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 438a014..e74db94 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -488,6 +491,26 @@ void free_irq(unsigned int irq, void *dev_id) }

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-25 Thread Max Krasnyanskiy
Hi Peter, Sorry for delay in reply. Please, wrap your emails at 78 - most mailers can do this. Done. On Fri, 2008-02-22 at 14:05 -0800, Max Krasnyanskiy wrote: Peter Zijlstra wrote: On Thu, 2008-02-21 at 18:38 -0800, Max Krasnyanskiy wrote: List of commits cpuisol: Make cpu

Re: Tiny cpusets -- cpusets for small systems?

2008-02-25 Thread Max Krasnyanskiy
Paul Jackson wrote: So. I see cpusets as a higher level API/mechanism and cpu_isolated_map as lower level mechanism that actually makes kernel aware of what's isolated what's not. Kind of like sched domain/cpuset relationship. ie cpusets affect sched domains but scheduler does not use cpusets

Re: Module loading/unloading and "The Stop Machine"

2008-02-22 Thread Max Krasnyanskiy
Hi Andi, Max Krasnyanskiy <[EMAIL PROTECTED]> writes: static struct module *load_module(void __user *umod, unsigned long len, const char __user *uargs) { ... /* Now sew it into the lists so we can get lockdep an

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Fri, 2008-02-22 at 08:38 -0500, Mark Hounschell wrote: List of commits cpuisol: Make cpu isolation configrable and export isolated map cpu_isolated_map was a bad hack when it was introduced, I feel we should deprecate it and fully integrate the functionality into

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Mark Hounschell wrote: Peter Zijlstra wrote: On Thu, 2008-02-21 at 18:38 -0800, Max Krasnyanskiy wrote: As you suggested I'm sending CPU isolation patches for review/inclusion into sched-devel tree. They are against 2.6.25-rc2. You can also pull them from my GIT tree at git

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Thu, 2008-02-21 at 18:38 -0800, Max Krasnyanskiy wrote: As you suggested I'm sending CPU isolation patches for review/inclusion into sched-devel tree. They are against 2.6.25-rc2. You can also pull them from my GIT tree at git://git.kernel.org/pub/scm/linux

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Dmitry Adamushko wrote: Hi Max, [ ... ] Last patch to the stop machine is potentially unsafe and is marked as experimental. Unfortunately it's currently the only option that allows dynamic module insertion/removal for above scenarios. I'm puzzled by the following part (can be a

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Dmitry Adamushko wrote: Hi Max, [ ... ] Last patch to the stop machine is potentially unsafe and is marked as experimental. Unfortunately it's currently the only option that allows dynamic module insertion/removal for above scenarios. I'm puzzled by the following part (can be a

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Thu, 2008-02-21 at 18:38 -0800, Max Krasnyanskiy wrote: As you suggested I'm sending CPU isolation patches for review/inclusion into sched-devel tree. They are against 2.6.25-rc2. You can also pull them from my GIT tree at git://git.kernel.org/pub/scm/linux

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Mark Hounschell wrote: Peter Zijlstra wrote: On Thu, 2008-02-21 at 18:38 -0800, Max Krasnyanskiy wrote: As you suggested I'm sending CPU isolation patches for review/inclusion into sched-devel tree. They are against 2.6.25-rc2. You can also pull them from my GIT tree at git

Re: [PATCH sched-devel 0/7] CPU isolation extensions

2008-02-22 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Fri, 2008-02-22 at 08:38 -0500, Mark Hounschell wrote: List of commits cpuisol: Make cpu isolation configrable and export isolated map cpu_isolated_map was a bad hack when it was introduced, I feel we should deprecate it and fully integrate the functionality into

Re: Module loading/unloading and The Stop Machine

2008-02-22 Thread Max Krasnyanskiy
Hi Andi, Max Krasnyanskiy [EMAIL PROTECTED] writes: static struct module *load_module(void __user *umod, unsigned long len, const char __user *uargs) { ... /* Now sew it into the lists so we can get lockdep and oops

[PATCH sched-devel 0/7] CPU isolation extensions

2008-02-21 Thread Max Krasnyanskiy
Ingo, As you suggested I'm sending CPU isolation patches for review/inclusion into sched-devel tree. They are against 2.6.25-rc2. You can also pull them from my GIT tree at git://git.kernel.org/pub/scm/linux/kernel/git/maxk/cpuisol-2.6.git master Diffstat:

Re: Module loading/unloading and "The Stop Machine"

2008-02-21 Thread Max Krasnyanskiy
Tejun Heo wrote: Max Krasnyanskiy wrote: Tejun Heo wrote: Max Krasnyanskiy wrote: Thanks for the info. I guess I missed that from the code. In any case that seems like a pretty heavy refcounting mechanism. In a sense that every time something is loaded or unloaded entire machine freezes

Re: Module loading/unloading and "The Stop Machine"

2008-02-21 Thread Max Krasnyanskiy
Tejun Heo wrote: Max Krasnyanskiy wrote: Thanks for the info. I guess I missed that from the code. In any case that seems like a pretty heavy refcounting mechanism. In a sense that every time something is loaded or unloaded entire machine freezes, potentially for several milliseconds. Normally

Re: Module loading/unloading and "The Stop Machine"

2008-02-21 Thread Max Krasnyanskiy
Hi Tejun, Max Krasnyansky wrote: I was hopping you could answer a couple of questions about module loading/unloading and the stop machine. There was a recent discussion on LKML about CPU isolation patches I'm working on. One of the patches makes stop machine ignore the isolated CPUs. People

CPU isolation extensions for the stable 2.6.24.y series

2008-02-21 Thread Max Krasnyanskiy
I got quite a few positive responses from end-users about CPU isolation extensions that I posted awhile ago. For people who are interested in using latest CPU isolation stuff on top of the stable 2.6.24.y tree I created a separate GIT tree that is available at

CPU isolation extensions for the stable 2.6.24.y series

2008-02-21 Thread Max Krasnyanskiy
I got quite a few positive responses from end-users about CPU isolation extensions that I posted awhile ago. For people who are interested in using latest CPU isolation stuff on top of the stable 2.6.24.y tree I created a separate GIT tree that is available at

Re: Module loading/unloading and The Stop Machine

2008-02-21 Thread Max Krasnyanskiy
Tejun Heo wrote: Max Krasnyanskiy wrote: Thanks for the info. I guess I missed that from the code. In any case that seems like a pretty heavy refcounting mechanism. In a sense that every time something is loaded or unloaded entire machine freezes, potentially for several milliseconds. Normally

Re: Module loading/unloading and The Stop Machine

2008-02-21 Thread Max Krasnyanskiy
Hi Tejun, Max Krasnyansky wrote: I was hopping you could answer a couple of questions about module loading/unloading and the stop machine. There was a recent discussion on LKML about CPU isolation patches I'm working on. One of the patches makes stop machine ignore the isolated CPUs. People

Re: Module loading/unloading and The Stop Machine

2008-02-21 Thread Max Krasnyanskiy
Tejun Heo wrote: Max Krasnyanskiy wrote: Tejun Heo wrote: Max Krasnyanskiy wrote: Thanks for the info. I guess I missed that from the code. In any case that seems like a pretty heavy refcounting mechanism. In a sense that every time something is loaded or unloaded entire machine freezes

[PATCH sched-devel 0/7] CPU isolation extensions

2008-02-21 Thread Max Krasnyanskiy
Ingo, As you suggested I'm sending CPU isolation patches for review/inclusion into sched-devel tree. They are against 2.6.25-rc2. You can also pull them from my GIT tree at git://git.kernel.org/pub/scm/linux/kernel/git/maxk/cpuisol-2.6.git master Diffstat:

Re: Module loading/unloading and "The Stop Machine"

2008-02-08 Thread Max Krasnyanskiy
Max Krasnyansky wrote: Hi Rusty, I was hopping you could answer a couple of questions about module loading/unloading and the stop machine. There was a recent discussion on LKML about CPU isolation patches I'm working on. One of the patches makes stop machine ignore the isolated CPUs. People

Re: Module loading/unloading and The Stop Machine

2008-02-08 Thread Max Krasnyanskiy
Max Krasnyansky wrote: Hi Rusty, I was hopping you could answer a couple of questions about module loading/unloading and the stop machine. There was a recent discussion on LKML about CPU isolation patches I'm working on. One of the patches makes stop machine ignore the isolated CPUs. People

Re: cpuisol: CPU isolation extensions (take 2)

2008-02-06 Thread Max Krasnyanskiy
CC'ing linux-rt-users because I think my explanation below may be interesting for the RT folks. Mark Hounschell wrote: Max Krasnyanskiy wrote: With CPU isolation it's very easy to achieve single digit usec worst case and around 200 nsec average response times on off-the-shelf multi

Re: RT scheduler config, suggestions and questions

2008-02-06 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Wed, 2008-02-06 at 07:36 +0100, Peter Zijlstra wrote: btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread. It is. Single setting per user is not enough. Unless a use has a single RT task. ? Ah,

Re: RT scheduler config, suggestions and questions

2008-02-06 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Tue, 2008-02-05 at 15:37 -0800, Max Krasnyanskiy wrote: Folks, I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG: sched_rt_period sched_rt_ratio I do not believe that is correct. I know that we do not want to expose

Re: RT scheduler config, suggestions and questions

2008-02-06 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Tue, 2008-02-05 at 15:37 -0800, Max Krasnyanskiy wrote: Folks, I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG: sched_rt_period sched_rt_ratio I do not believe that is correct. I know that we do not want to expose

Re: RT scheduler config, suggestions and questions

2008-02-06 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Wed, 2008-02-06 at 07:36 +0100, Peter Zijlstra wrote: btw I can see watchdog being very useful to catch hard-RT tasks that exceed the deadline. But's it gotta be per thread. It is. Single setting per user is not enough. Unless a use has a single RT task. ? Ah,

RT scheduler config, suggestions and questions

2008-02-05 Thread Max Krasnyanskiy
Folks, I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG: sched_rt_period sched_rt_ratio I do not believe that is correct. I know that we do not want to expose scheduler knobs in general but theses are not the heuristic kind of knobs. There is

cpuisol: CPU isolation extensions (take 2)

2008-02-05 Thread Max Krasnyanskiy
It seems that git-send-email for some reasons did not send an introductory email. So I'm sending it manually. Sorry if you get it twice. --- Following patch series extends CPU isolation support. Yes, most people want to virtuallize CPUs these days and I want to isolate them :) . The

cpuisol: CPU isolation extensions (take 2)

2008-02-05 Thread Max Krasnyanskiy
It seems that git-send-email for some reasons did not send an introductory email. So I'm sending it manually. Sorry if you get it twice. --- Following patch series extends CPU isolation support. Yes, most people want to virtuallize CPUs these days and I want to isolate them :) . The

RT scheduler config, suggestions and questions

2008-02-05 Thread Max Krasnyanskiy
Folks, I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG: sched_rt_period sched_rt_ratio I do not believe that is correct. I know that we do not want to expose scheduler knobs in general but theses are not the heuristic kind of knobs. There is

CPU isolation and workqueues [was Re: [CPUISOL] CPU isolation extensions]

2008-02-04 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Mon, 2008-01-28 at 14:00 -0500, Steven Rostedt wrote: On Mon, 28 Jan 2008, Max Krasnyanskiy wrote: [PATCH] [CPUISOL] Support for workqueue isolation The thing about workqueues is that they should only be woken on a CPU if something on that CPU accessed them. IOW

CPU hotplug and IRQ affinity with 2.6.24-rt1

2008-02-04 Thread Max Krasnyanskiy
This is just an FYI. As part of the "Isolated CPU extensions" thread Daniel suggest for me to check out latest RT kernels. So I did or at least tried to and immediately spotted a couple of issues. The machine I'm running it on is: HP xw9300, Dual Opteron, NUMA It looks like with -rt

Re: Integrating cpusets and cpu isolation [was Re: [CPUISOL] CPU isolation extensions]

2008-02-04 Thread Max Krasnyanskiy
Paul Jackson wrote: Max wrote: Looks like I failed to explain what I'm trying to achieve. So let me try again. Well done. I read through that, expecting to disagree or at least to not understand at some point, and got all the way through nodding my head in agreement. Good. Whether the

CPU hotplug and IRQ affinity with 2.6.24-rt1

2008-02-04 Thread Max Krasnyanskiy
This is just an FYI. As part of the Isolated CPU extensions thread Daniel suggest for me to check out latest RT kernels. So I did or at least tried to and immediately spotted a couple of issues. The machine I'm running it on is: HP xw9300, Dual Opteron, NUMA It looks like with -rt

Re: Integrating cpusets and cpu isolation [was Re: [CPUISOL] CPU isolation extensions]

2008-02-04 Thread Max Krasnyanskiy
Paul Jackson wrote: Max wrote: Looks like I failed to explain what I'm trying to achieve. So let me try again. Well done. I read through that, expecting to disagree or at least to not understand at some point, and got all the way through nodding my head in agreement. Good. Whether the

CPU isolation and workqueues [was Re: [CPUISOL] CPU isolation extensions]

2008-02-04 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Mon, 2008-01-28 at 14:00 -0500, Steven Rostedt wrote: On Mon, 28 Jan 2008, Max Krasnyanskiy wrote: [PATCH] [CPUISOL] Support for workqueue isolation The thing about workqueues is that they should only be woken on a CPU if something on that CPU accessed them. IOW

Re: [CPUISOL] CPU isolation extensions

2008-01-31 Thread Max Krasnyanskiy
Hi Mark, [EMAIL PROTECTED] wrote: Following patch series extends CPU isolation support. Yes, most people want to virtuallize CPUs these days and I want to isolate them :). The primary idea here is to be able to use some CPU cores as dedicated engines for running user-space code with minimal

Integrating cpusets and cpu isolation [was Re: [CPUISOL] CPU isolation extensions]

2008-01-31 Thread Max Krasnyanskiy
Paul Jackson wrote: Max wrote: So far it seems that extending cpu_isolated_map is more natural way of propagating this notion to the rest of the kernel. Since it's very similar to the cpu_online_map concept and it's easy to integrated with the code that already uses it. If it were just

Re: [CPUISOL] CPU isolation extensions

2008-01-31 Thread Max Krasnyanskiy
Hi Mark, [EMAIL PROTECTED] wrote: Following patch series extends CPU isolation support. Yes, most people want to virtuallize CPUs these days and I want to isolate them :). The primary idea here is to be able to use some CPU cores as dedicated engines for running user-space code with minimal

Integrating cpusets and cpu isolation [was Re: [CPUISOL] CPU isolation extensions]

2008-01-31 Thread Max Krasnyanskiy
Paul Jackson wrote: Max wrote: So far it seems that extending cpu_isolated_map is more natural way of propagating this notion to the rest of the kernel. Since it's very similar to the cpu_online_map concept and it's easy to integrated with the code that already uses it. If it were just

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Daniel Walker wrote: On Mon, 2008-01-28 at 10:32 -0800, Max Krasnyanskiy wrote: Just this patches. RT patches cannot achieve what I needed. Even RTAI/Xenomai can't do that. For example I have separate tasks with hard deadlines that must be enforced in 50usec kind of range and basically

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Paul Jackson wrote: Max wrote: So far it seems that extending cpu_isolated_map is more natural way of propagating this notion to the rest of the kernel. Since it's very similar to the cpu_online_map concept and it's easy to integrated with the code that already uses it. If it were just

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Mon, 2008-01-28 at 14:00 -0500, Steven Rostedt wrote: On Mon, 28 Jan 2008, Max Krasnyanskiy wrote: [PATCH] [CPUISOL] Support for workqueue isolation The thing about workqueues is that they should only be woken on a CPU if something on that CPU accessed them. IOW

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Mon, 2008-01-28 at 11:34 -0500, Steven Rostedt wrote: On Mon, Jan 28, 2008 at 08:59:10AM -0600, Paul Jackson wrote: Thanks for the CC, Peter. Thanks from me too. Max wrote: We've had scheduler support for CPU isolation ever since O(1) scheduler went it. I'd like to

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Steven Rostedt wrote: On Mon, Jan 28, 2008 at 08:59:10AM -0600, Paul Jackson wrote: Thanks for the CC, Peter. Thanks from me too. Max wrote: We've had scheduler support for CPU isolation ever since O(1) scheduler went it. I'd like to extend it further to avoid kernel activity on those CPUs

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Paul Jackson wrote: Thanks for the CC, Peter. Ingo - see question at end of message. Max wrote: We've had scheduler support for CPU isolation ever since O(1) scheduler went it. I'd like to extend it further to avoid kernel activity on those CPUs as much as possible. I recently added the

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Hi Peter, Peter Zijlstra wrote: [ You really ought to CC people :-) ] I was not sure who though :) Do we have a mailing list for scheduler development btw ? Or it's just folks that you included in CC ? Some of the latest scheduler patches brake things that I'm doing and I'd like to make them

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Mon, 2008-01-28 at 11:34 -0500, Steven Rostedt wrote: On Mon, Jan 28, 2008 at 08:59:10AM -0600, Paul Jackson wrote: Thanks for the CC, Peter. Thanks from me too. Max wrote: We've had scheduler support for CPU isolation ever since O(1) scheduler went it. I'd like to

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Peter Zijlstra wrote: On Mon, 2008-01-28 at 14:00 -0500, Steven Rostedt wrote: On Mon, 28 Jan 2008, Max Krasnyanskiy wrote: [PATCH] [CPUISOL] Support for workqueue isolation The thing about workqueues is that they should only be woken on a CPU if something on that CPU accessed them. IOW

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Paul Jackson wrote: Max wrote: So far it seems that extending cpu_isolated_map is more natural way of propagating this notion to the rest of the kernel. Since it's very similar to the cpu_online_map concept and it's easy to integrated with the code that already uses it. If it were just

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Daniel Walker wrote: On Mon, 2008-01-28 at 10:32 -0800, Max Krasnyanskiy wrote: Just this patches. RT patches cannot achieve what I needed. Even RTAI/Xenomai can't do that. For example I have separate tasks with hard deadlines that must be enforced in 50usec kind of range and basically

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Steven Rostedt wrote: On Mon, Jan 28, 2008 at 08:59:10AM -0600, Paul Jackson wrote: Thanks for the CC, Peter. Thanks from me too. Max wrote: We've had scheduler support for CPU isolation ever since O(1) scheduler went it. I'd like to extend it further to avoid kernel activity on those CPUs

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Paul Jackson wrote: Thanks for the CC, Peter. Ingo - see question at end of message. Max wrote: We've had scheduler support for CPU isolation ever since O(1) scheduler went it. I'd like to extend it further to avoid kernel activity on those CPUs as much as possible. I recently added the

Re: [CPUISOL] CPU isolation extensions

2008-01-28 Thread Max Krasnyanskiy
Hi Peter, Peter Zijlstra wrote: [ You really ought to CC people :-) ] I was not sure who though :) Do we have a mailing list for scheduler development btw ? Or it's just folks that you included in CC ? Some of the latest scheduler patches brake things that I'm doing and I'd like to make them