Re: [patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-21 Thread Andrew Morton
On Fri, 16 Feb 2007 18:03:35 -0800
"Siddha, Suresh B" <[EMAIL PROTECTED]> wrote:

> + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);

argh.  Please, do

int cpu = smp_processor_id();
int idle_at_tick = idle_cpu(cpu);
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-21 Thread Andrew Morton
On Fri, 16 Feb 2007 18:03:35 -0800
Siddha, Suresh B [EMAIL PROTECTED] wrote:

 + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);

argh.  Please, do

int cpu = smp_processor_id();
int idle_at_tick = idle_cpu(cpu);
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-20 Thread Siddha, Suresh B
On Sat, Feb 17, 2007 at 09:42:16AM -0500, Steven Rostedt wrote:
> On Fri, 16 Feb 2007, Siddha, Suresh B wrote:
> > -   int cpu = smp_processor_id();
> > +   int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
> > struct rq *rq = cpu_rq(cpu);
> >
> 
> But I would recommend that the idle_at_tick should be on a separate line.

Ok.

> I'll try to look deeper at your patches tomorrow. I've also found that I'm
> having some latency problems in -rt that I think might be related to
> migration.

There is one more issue, I have observed in -rt tree.

raise_softirq_irqoff() is unconditionally doing the wakeup_softirqd()
But unless the CONFIG_PREEMPT_SOFTIRQS is configured, do_softirq()
will process the softirq's like TIMER_SOFTIRQ and SCHED_SOFTIRQ in the
process context.  So it looks like we are unnecessarily waking the
softirqd's corresponding to those softirq's.

thanks,
suresh
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-20 Thread Siddha, Suresh B
On Sat, Feb 17, 2007 at 09:42:16AM -0500, Steven Rostedt wrote:
 On Fri, 16 Feb 2007, Siddha, Suresh B wrote:
  -   int cpu = smp_processor_id();
  +   int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
  struct rq *rq = cpu_rq(cpu);
 
 
 But I would recommend that the idle_at_tick should be on a separate line.

Ok.

 I'll try to look deeper at your patches tomorrow. I've also found that I'm
 having some latency problems in -rt that I think might be related to
 migration.

There is one more issue, I have observed in -rt tree.

raise_softirq_irqoff() is unconditionally doing the wakeup_softirqd()
But unless the CONFIG_PREEMPT_SOFTIRQS is configured, do_softirq()
will process the softirq's like TIMER_SOFTIRQ and SCHED_SOFTIRQ in the
process context.  So it looks like we are unnecessarily waking the
softirqd's corresponding to those softirq's.

thanks,
suresh
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-17 Thread Steven Rostedt


On Fri, 16 Feb 2007, Siddha, Suresh B wrote:

> Periodic load balancing in recent kernels happen in the softirq.
> In certain -rt configurations, these softirqs are handled in softirqd
> context. And hence the check for idle processor was always returning
> busy (as nr_running > 1).
>
> This patch captures the idle information at the tick and passes this info
> to softirq context through an element 'idle_at_tick' in rq.

I haven't had the time yet to look too detailed at this patch.

>
> Signed-off-by: Suresh Siddha <[EMAIL PROTECTED]>
> ---
>

>  {
>   unsigned long long now = sched_clock();
>   struct task_struct *p = current;
> - int cpu = smp_processor_id();
> + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
>   struct rq *rq = cpu_rq(cpu);
>

But I would recommend that the idle_at_tick should be on a separate line.

I'll try to look deeper at your patches tomorrow. I've also found that I'm
having some latency problems in -rt that I think might be related to
migration.

-- Steve

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-17 Thread Steven Rostedt


On Fri, 16 Feb 2007, Siddha, Suresh B wrote:

 Periodic load balancing in recent kernels happen in the softirq.
 In certain -rt configurations, these softirqs are handled in softirqd
 context. And hence the check for idle processor was always returning
 busy (as nr_running  1).

 This patch captures the idle information at the tick and passes this info
 to softirq context through an element 'idle_at_tick' in rq.

I haven't had the time yet to look too detailed at this patch.


 Signed-off-by: Suresh Siddha [EMAIL PROTECTED]
 ---


  {
   unsigned long long now = sched_clock();
   struct task_struct *p = current;
 - int cpu = smp_processor_id();
 + int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
   struct rq *rq = cpu_rq(cpu);


But I would recommend that the idle_at_tick should be on a separate line.

I'll try to look deeper at your patches tomorrow. I've also found that I'm
having some latency problems in -rt that I think might be related to
migration.

-- Steve

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-16 Thread Siddha, Suresh B
Periodic load balancing in recent kernels happen in the softirq.
In certain -rt configurations, these softirqs are handled in softirqd
context. And hence the check for idle processor was always returning
busy (as nr_running > 1).

This patch captures the idle information at the tick and passes this info
to softirq context through an element 'idle_at_tick' in rq.

Signed-off-by: Suresh Siddha <[EMAIL PROTECTED]>
---

diff -pNru linux-2.6.20.x86_64/kernel/sched.c linux/kernel/sched.c
--- linux-2.6.20.x86_64/kernel/sched.c  2007-02-10 03:41:27.0 -0800
+++ linux/kernel/sched.c2007-02-16 16:22:44.0 -0800
@@ -240,6 +240,7 @@ struct rq {
unsigned long raw_weighted_load;
 #ifdef CONFIG_SMP
unsigned long cpu_load[3];
+   unsigned char idle_at_tick;
 #endif
unsigned long long nr_switches;
 
@@ -3350,12 +3351,7 @@ static void run_rebalance_domains(struct
struct rq *this_rq = cpu_rq(this_cpu);
unsigned long interval;
struct sched_domain *sd;
-   /*
-* We are idle if there are no processes running. This
-* is valid even if we are the idle process (SMT).
-*/
-   enum idle_type idle = !this_rq->nr_running ?
-   SCHED_IDLE : NOT_IDLE;
+   enum idle_type idle = this_rq->idle_at_tick ? SCHED_IDLE : NOT_IDLE;
/* Earliest time when we have to call run_rebalance_domains again */
unsigned long next_balance = jiffies + 60*HZ;
 
@@ -3651,20 +3647,21 @@ void scheduler_tick(void)
 {
unsigned long long now = sched_clock();
struct task_struct *p = current;
-   int cpu = smp_processor_id();
+   int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
struct rq *rq = cpu_rq(cpu);
 
BUG_ON(!irqs_disabled());
 
update_cpu_clock(p, rq, now);
 
-   if (p == rq->idle)
+   if (idle_at_tick)
/* Task on the idle queue */
wake_priority_sleeper(rq);
else
task_running_tick(rq, p);
 #ifdef CONFIG_SMP
update_load(rq);
+   rq->idle_at_tick = idle_at_tick;
if (time_after_eq(jiffies, rq->next_balance))
raise_softirq(SCHED_SOFTIRQ);
 #endif
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[patch 1/2] sched: fix idle load balancing in softirqd context

2007-02-16 Thread Siddha, Suresh B
Periodic load balancing in recent kernels happen in the softirq.
In certain -rt configurations, these softirqs are handled in softirqd
context. And hence the check for idle processor was always returning
busy (as nr_running  1).

This patch captures the idle information at the tick and passes this info
to softirq context through an element 'idle_at_tick' in rq.

Signed-off-by: Suresh Siddha [EMAIL PROTECTED]
---

diff -pNru linux-2.6.20.x86_64/kernel/sched.c linux/kernel/sched.c
--- linux-2.6.20.x86_64/kernel/sched.c  2007-02-10 03:41:27.0 -0800
+++ linux/kernel/sched.c2007-02-16 16:22:44.0 -0800
@@ -240,6 +240,7 @@ struct rq {
unsigned long raw_weighted_load;
 #ifdef CONFIG_SMP
unsigned long cpu_load[3];
+   unsigned char idle_at_tick;
 #endif
unsigned long long nr_switches;
 
@@ -3350,12 +3351,7 @@ static void run_rebalance_domains(struct
struct rq *this_rq = cpu_rq(this_cpu);
unsigned long interval;
struct sched_domain *sd;
-   /*
-* We are idle if there are no processes running. This
-* is valid even if we are the idle process (SMT).
-*/
-   enum idle_type idle = !this_rq-nr_running ?
-   SCHED_IDLE : NOT_IDLE;
+   enum idle_type idle = this_rq-idle_at_tick ? SCHED_IDLE : NOT_IDLE;
/* Earliest time when we have to call run_rebalance_domains again */
unsigned long next_balance = jiffies + 60*HZ;
 
@@ -3651,20 +3647,21 @@ void scheduler_tick(void)
 {
unsigned long long now = sched_clock();
struct task_struct *p = current;
-   int cpu = smp_processor_id();
+   int cpu = smp_processor_id(), idle_at_tick = idle_cpu(cpu);
struct rq *rq = cpu_rq(cpu);
 
BUG_ON(!irqs_disabled());
 
update_cpu_clock(p, rq, now);
 
-   if (p == rq-idle)
+   if (idle_at_tick)
/* Task on the idle queue */
wake_priority_sleeper(rq);
else
task_running_tick(rq, p);
 #ifdef CONFIG_SMP
update_load(rq);
+   rq-idle_at_tick = idle_at_tick;
if (time_after_eq(jiffies, rq-next_balance))
raise_softirq(SCHED_SOFTIRQ);
 #endif
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/