Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-23 Thread Vinayak Menon
On 5/23/2018 6:47 PM, Johannes Weiner wrote: > On Wed, May 09, 2018 at 04:33:24PM +0530, Vinayak Menon wrote: >> On 5/8/2018 2:31 AM, Johannes Weiner wrote: >>> + /* Kick the stats aggregation worker if it's gone to sleep */ >>> + if (!delayed_work_pending(&group->clock_work)) >> This causes a

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-23 Thread Johannes Weiner
On Wed, May 09, 2018 at 04:33:24PM +0530, Vinayak Menon wrote: > On 5/8/2018 2:31 AM, Johannes Weiner wrote: > > + /* Kick the stats aggregation worker if it's gone to sleep */ > > + if (!delayed_work_pending(&group->clock_work)) > > This causes a crash when the work is scheduled before system

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-14 Thread Peter Zijlstra
On Thu, May 10, 2018 at 09:41:32AM -0400, Johannes Weiner wrote: > So there is a reason I'm tracking productivity states per-cpu and not > globally. Consider the following example periods on two CPUs: > > CPU 0 > Task 1: | EXECUTING | memstalled | > Task 2: | runqueued | EXECUTING | > >

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-10 Thread Johannes Weiner
On Wed, May 09, 2018 at 12:21:00PM +0200, Peter Zijlstra wrote: > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > + local_irq_disable(); > > + rq = this_rq(); > > + raw_spin_lock(&rq->lock); > > + rq_pin_lock(rq, &rf); > > Given that churn in sched.h, you seen rq_lock(

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-10 Thread Johannes Weiner
On Wed, May 09, 2018 at 12:14:54PM +0200, Peter Zijlstra wrote: > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > > index 15750c222ca2..1658477466d5 100644 > > --- a/kernel/sched/sched.h > > +++ b/kernel/sched/sched.h [

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-10 Thread Johannes Weiner
On Wed, May 09, 2018 at 12:05:51PM +0200, Peter Zijlstra wrote: > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > + u64 some[NR_PSI_RESOURCES] = { 0, }; > > + u64 full[NR_PSI_RESOURCES] = { 0, }; > > > + some[r] /= max(nonidle_total, 1UL); > > + full[r]

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-10 Thread Johannes Weiner
On Wed, May 09, 2018 at 12:04:55PM +0200, Peter Zijlstra wrote: > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > +static void psi_clock(struct work_struct *work) > > +{ > > + u64 some[NR_PSI_RESOURCES] = { 0, }; > > + u64 full[NR_PSI_RESOURCES] = { 0, }; > > + unsigned l

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-10 Thread Johannes Weiner
On Wed, May 09, 2018 at 11:59:38AM +0200, Peter Zijlstra wrote: > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > diff --git a/include/linux/psi_types.h b/include/linux/psi_types.h > > new file mode 100644 > > index ..b22b0ffc729d > > --- /dev/null > > +++ b/include

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-10 Thread Johannes Weiner
On Wed, May 09, 2018 at 01:38:49PM +0200, Peter Zijlstra wrote: > On Wed, May 09, 2018 at 12:46:18PM +0200, Peter Zijlstra wrote: > > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > > > > @@ -2038,6 +2038,7 @@ try_to_wake_up(struct task_struct *p, unsigned int > > > state, in

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Wed, May 09, 2018 at 12:46:18PM +0200, Peter Zijlstra wrote: > On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > > > @@ -2038,6 +2038,7 @@ try_to_wake_up(struct task_struct *p, unsigned int > > state, int wake_flags) > > cpu = select_task_rq(p, p->wake_cpu, SD_BALANCE_WAKE

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Vinayak Menon
On 5/8/2018 2:31 AM, Johannes Weiner wrote: > +static void psi_group_update(struct psi_group *group, int cpu, u64 now, > + unsigned int clear, unsigned int set) > +{ > + enum psi_state state = PSI_NONE; > + struct psi_group_cpu *groupc; > + unsigned int *tasks;

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > @@ -28,10 +28,14 @@ static inline int sched_info_on(void) > return 1; > #elif defined(CONFIG_TASK_DELAY_ACCT) > extern int delayacct_on; > - return delayacct_on; > -#else > - return 0; > + if (delayacct_on)

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > @@ -2038,6 +2038,7 @@ try_to_wake_up(struct task_struct *p, unsigned int > state, int wake_flags) > cpu = select_task_rq(p, p->wake_cpu, SD_BALANCE_WAKE, wake_flags); > if (task_cpu(p) != cpu) { > wake_fl

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > +static void psi_clock(struct work_struct *work) > +{ > + dwork = to_delayed_work(work); > + group = container_of(dwork, struct psi_group, clock_work); > + > + > + /* Keep the clock ticking only when there is action */

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > +/** > + * psi_memstall_enter - mark the beginning of a memory stall section > + * @flags: flags to handle nested sections > + * > + * Marks the calling task as being stalled due to a lack of memory, > + * such as waiting for a refa

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index 15750c222ca2..1658477466d5 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -919,6 +921,8 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues)

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > + u64 some[NR_PSI_RESOURCES] = { 0, }; > + u64 full[NR_PSI_RESOURCES] = { 0, }; > + some[r] /= max(nonidle_total, 1UL); > + full[r] /= max(nonidle_total, 1UL); That's a bare 64bit divide.. that typi

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > +static void psi_clock(struct work_struct *work) > +{ > + u64 some[NR_PSI_RESOURCES] = { 0, }; > + u64 full[NR_PSI_RESOURCES] = { 0, }; > + unsigned long nonidle_total = 0; > + unsigned long missed_periods; > + s

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-09 Thread Peter Zijlstra
On Mon, May 07, 2018 at 05:01:34PM -0400, Johannes Weiner wrote: > diff --git a/include/linux/psi_types.h b/include/linux/psi_types.h > new file mode 100644 > index ..b22b0ffc729d > --- /dev/null > +++ b/include/linux/psi_types.h > @@ -0,0 +1,84 @@ > +#ifndef _LINUX_PSI_TYPES_H > +#defi

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-08 Thread Johannes Weiner
On Mon, May 07, 2018 at 05:42:36PM -0700, Randy Dunlap wrote: > On 05/07/2018 02:01 PM, Johannes Weiner wrote: > > + * The ratio is tracked in decaying time averages over 10s, 1m, 5m > > + * windows. Cumluative stall times are tracked and exported as well to > >Cumulative > > > +

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-08 Thread Johannes Weiner
On Tue, May 08, 2018 at 11:04:09AM +0800, kbuild test robot wrote: >118#else /* CONFIG_PSI */ >119static inline void psi_enqueue(struct task_struct *p, u64 now) >120{ >121} >122static inline void psi_dequeue(struct task_struct *p, u64 now)

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-07 Thread kbuild test robot
Hi Johannes, I love your patch! Yet something to improve: [auto build test ERROR on linus/master] [also build test ERROR on v4.17-rc4] [cannot apply to next-20180507] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-07 Thread kbuild test robot
Hi Johannes, I love your patch! Yet something to improve: [auto build test ERROR on linus/master] [also build test ERROR on v4.17-rc4] [cannot apply to next-20180507] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0

Re: [PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-07 Thread Randy Dunlap
On 05/07/2018 02:01 PM, Johannes Weiner wrote: > > Signed-off-by: Johannes Weiner > --- > Documentation/accounting/psi.txt | 73 ++ > include/linux/psi.h | 27 ++ > include/linux/psi_types.h| 84 ++ > include/linux/sched.h| 10 + > include/linux/sched

[PATCH 6/7] psi: pressure stall information for CPU, memory, and IO

2018-05-07 Thread Johannes Weiner
When systems are overcommitted and resources become contended, it's hard to tell exactly the impact this has on workload productivity, or how close the system is to lockups and OOM kills. In particular, when machines work multiple jobs concurrently, the impact of overcommit in terms of latency and