Re: [patch 1/2] sched: fix idle load balancing in softirqd context
On Fri, 16 Feb 2007 18:03:35 -0800 "Siddha, Suresh B" <[EMAIL PROTECTED]> wrote: > + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu); argh. Please, do int cpu = smp_processor_id(); int idle_at_tick = idle_cpu(cpu); - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 1/2] sched: fix idle load balancing in softirqd context
On Fri, 16 Feb 2007 18:03:35 -0800 Siddha, Suresh B [EMAIL PROTECTED] wrote: + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu); argh. Please, do int cpu = smp_processor_id(); int idle_at_tick = idle_cpu(cpu); - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 1/2] sched: fix idle load balancing in softirqd context
On Sat, Feb 17, 2007 at 09:42:16AM -0500, Steven Rostedt wrote: > On Fri, 16 Feb 2007, Siddha, Suresh B wrote: > > - int cpu = smp_processor_id(); > > + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu); > > struct rq *rq = cpu_rq(cpu); > > > > But I would recommend that the idle_at_tick should be on a separate line. Ok. > I'll try to look deeper at your patches tomorrow. I've also found that I'm > having some latency problems in -rt that I think might be related to > migration. There is one more issue, I have observed in -rt tree. raise_softirq_irqoff() is unconditionally doing the wakeup_softirqd() But unless the CONFIG_PREEMPT_SOFTIRQS is configured, do_softirq() will process the softirq's like TIMER_SOFTIRQ and SCHED_SOFTIRQ in the process context. So it looks like we are unnecessarily waking the softirqd's corresponding to those softirq's. thanks, suresh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 1/2] sched: fix idle load balancing in softirqd context
On Sat, Feb 17, 2007 at 09:42:16AM -0500, Steven Rostedt wrote: On Fri, 16 Feb 2007, Siddha, Suresh B wrote: - int cpu = smp_processor_id(); + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu); struct rq *rq = cpu_rq(cpu); But I would recommend that the idle_at_tick should be on a separate line. Ok. I'll try to look deeper at your patches tomorrow. I've also found that I'm having some latency problems in -rt that I think might be related to migration. There is one more issue, I have observed in -rt tree. raise_softirq_irqoff() is unconditionally doing the wakeup_softirqd() But unless the CONFIG_PREEMPT_SOFTIRQS is configured, do_softirq() will process the softirq's like TIMER_SOFTIRQ and SCHED_SOFTIRQ in the process context. So it looks like we are unnecessarily waking the softirqd's corresponding to those softirq's. thanks, suresh - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 1/2] sched: fix idle load balancing in softirqd context
On Fri, 16 Feb 2007, Siddha, Suresh B wrote:
> Periodic load balancing in recent kernels happen in the softirq.
> In certain -rt configurations, these softirqs are handled in softirqd
> context. And hence the check for idle processor was always returning
> busy (as nr_running > 1).
>
> This patch captures the idle information at the tick and passes this info
> to softirq context through an element 'idle_at_tick' in rq.
I haven't had the time yet to look too detailed at this patch.
>
> Signed-off-by: Suresh Siddha <[EMAIL PROTECTED]>
> ---
>
> {
> unsigned long long now = sched_clock();
> struct task_struct *p = current;
> - int cpu = smp_processor_id();
> + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
> struct rq *rq = cpu_rq(cpu);
>
But I would recommend that the idle_at_tick should be on a separate line.
I'll try to look deeper at your patches tomorrow. I've also found that I'm
having some latency problems in -rt that I think might be related to
migration.
-- Steve
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 1/2] sched: fix idle load balancing in softirqd context
On Fri, 16 Feb 2007, Siddha, Suresh B wrote:
Periodic load balancing in recent kernels happen in the softirq.
In certain -rt configurations, these softirqs are handled in softirqd
context. And hence the check for idle processor was always returning
busy (as nr_running 1).
This patch captures the idle information at the tick and passes this info
to softirq context through an element 'idle_at_tick' in rq.
I haven't had the time yet to look too detailed at this patch.
Signed-off-by: Suresh Siddha [EMAIL PROTECTED]
---
{
unsigned long long now = sched_clock();
struct task_struct *p = current;
- int cpu = smp_processor_id();
+ int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
struct rq *rq = cpu_rq(cpu);
But I would recommend that the idle_at_tick should be on a separate line.
I'll try to look deeper at your patches tomorrow. I've also found that I'm
having some latency problems in -rt that I think might be related to
migration.
-- Steve
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[patch 1/2] sched: fix idle load balancing in softirqd context
Periodic load balancing in recent kernels happen in the softirq.
In certain -rt configurations, these softirqs are handled in softirqd
context. And hence the check for idle processor was always returning
busy (as nr_running > 1).
This patch captures the idle information at the tick and passes this info
to softirq context through an element 'idle_at_tick' in rq.
Signed-off-by: Suresh Siddha <[EMAIL PROTECTED]>
---
diff -pNru linux-2.6.20.x86_64/kernel/sched.c linux/kernel/sched.c
--- linux-2.6.20.x86_64/kernel/sched.c 2007-02-10 03:41:27.0 -0800
+++ linux/kernel/sched.c2007-02-16 16:22:44.0 -0800
@@ -240,6 +240,7 @@ struct rq {
unsigned long raw_weighted_load;
#ifdef CONFIG_SMP
unsigned long cpu_load[3];
+ unsigned char idle_at_tick;
#endif
unsigned long long nr_switches;
@@ -3350,12 +3351,7 @@ static void run_rebalance_domains(struct
struct rq *this_rq = cpu_rq(this_cpu);
unsigned long interval;
struct sched_domain *sd;
- /*
-* We are idle if there are no processes running. This
-* is valid even if we are the idle process (SMT).
-*/
- enum idle_type idle = !this_rq->nr_running ?
- SCHED_IDLE : NOT_IDLE;
+ enum idle_type idle = this_rq->idle_at_tick ? SCHED_IDLE : NOT_IDLE;
/* Earliest time when we have to call run_rebalance_domains again */
unsigned long next_balance = jiffies + 60*HZ;
@@ -3651,20 +3647,21 @@ void scheduler_tick(void)
{
unsigned long long now = sched_clock();
struct task_struct *p = current;
- int cpu = smp_processor_id();
+ int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
struct rq *rq = cpu_rq(cpu);
BUG_ON(!irqs_disabled());
update_cpu_clock(p, rq, now);
- if (p == rq->idle)
+ if (idle_at_tick)
/* Task on the idle queue */
wake_priority_sleeper(rq);
else
task_running_tick(rq, p);
#ifdef CONFIG_SMP
update_load(rq);
+ rq->idle_at_tick = idle_at_tick;
if (time_after_eq(jiffies, rq->next_balance))
raise_softirq(SCHED_SOFTIRQ);
#endif
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[patch 1/2] sched: fix idle load balancing in softirqd context
Periodic load balancing in recent kernels happen in the softirq.
In certain -rt configurations, these softirqs are handled in softirqd
context. And hence the check for idle processor was always returning
busy (as nr_running 1).
This patch captures the idle information at the tick and passes this info
to softirq context through an element 'idle_at_tick' in rq.
Signed-off-by: Suresh Siddha [EMAIL PROTECTED]
---
diff -pNru linux-2.6.20.x86_64/kernel/sched.c linux/kernel/sched.c
--- linux-2.6.20.x86_64/kernel/sched.c 2007-02-10 03:41:27.0 -0800
+++ linux/kernel/sched.c2007-02-16 16:22:44.0 -0800
@@ -240,6 +240,7 @@ struct rq {
unsigned long raw_weighted_load;
#ifdef CONFIG_SMP
unsigned long cpu_load[3];
+ unsigned char idle_at_tick;
#endif
unsigned long long nr_switches;
@@ -3350,12 +3351,7 @@ static void run_rebalance_domains(struct
struct rq *this_rq = cpu_rq(this_cpu);
unsigned long interval;
struct sched_domain *sd;
- /*
-* We are idle if there are no processes running. This
-* is valid even if we are the idle process (SMT).
-*/
- enum idle_type idle = !this_rq-nr_running ?
- SCHED_IDLE : NOT_IDLE;
+ enum idle_type idle = this_rq-idle_at_tick ? SCHED_IDLE : NOT_IDLE;
/* Earliest time when we have to call run_rebalance_domains again */
unsigned long next_balance = jiffies + 60*HZ;
@@ -3651,20 +3647,21 @@ void scheduler_tick(void)
{
unsigned long long now = sched_clock();
struct task_struct *p = current;
- int cpu = smp_processor_id();
+ int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
struct rq *rq = cpu_rq(cpu);
BUG_ON(!irqs_disabled());
update_cpu_clock(p, rq, now);
- if (p == rq-idle)
+ if (idle_at_tick)
/* Task on the idle queue */
wake_priority_sleeper(rq);
else
task_running_tick(rq, p);
#ifdef CONFIG_SMP
update_load(rq);
+ rq-idle_at_tick = idle_at_tick;
if (time_after_eq(jiffies, rq-next_balance))
raise_softirq(SCHED_SOFTIRQ);
#endif
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
