Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Davidlohr Bueso
On Wed, 07 Nov 2018, Davidlohr Bueso wrote: I have not looked at how filesystems tune the batch size, but it would certainly be worth looking into methinks. nm this part, percpu_counter_batch is not tunable. It would still probably be acceptable (famous last words) to at least move the bottlen

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Davidlohr Bueso
On Thu, 08 Nov 2018, Dave Chinner wrote: If only we had percpu counters that had a fixed, extremely low read overhead that doesn't care about the number of CPUs in the machine Oh, wait, we do: percpu_counters.[ch]. This all seems like a counter implementation deficiency to me, not an inter

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Dave Chinner
On Wed, Nov 07, 2018 at 11:03:06AM +0100, Miklos Szeredi wrote: > On Wed, Nov 7, 2018 at 12:48 AM, Andrew Morton > wrote: > > On Mon, 29 Oct 2018 23:04:45 + Daniel Colascione > > wrote: > > > >> On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: > >> > This patch introduces a new /proc

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Vito Caputo
On Wed, Nov 07, 2018 at 11:03:06AM +0100, Miklos Szeredi wrote: > On Wed, Nov 7, 2018 at 12:48 AM, Andrew Morton > wrote: > > On Mon, 29 Oct 2018 23:04:45 + Daniel Colascione > > wrote: > > > >> On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: > >> > This patch introduces a new /proc

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Waiman Long
On 11/06/2018 10:32 PM, Davidlohr Bueso wrote: > On Tue, 06 Nov 2018, Andrew Morton wrote: > >> It would be interesting to know precisely which stat fields the >> database-which-shall-not-be-named is looking for.  Then we could cook >> up a very whizzy way of getting at the info. > > The ctxt field

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Daniel Colascione
On Wed, Nov 7, 2018 at 3:54 PM, Miklos Szeredi wrote: > On Wed, Nov 7, 2018 at 4:42 PM, Daniel Colascione wrote: > >> configuration!" is something I've heard more than once. Who's to say >> that sysfs is for exposing /proc/pid/stat, > > Patch is about /proc/stat not /proc/PID/stat. Please revise

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Miklos Szeredi
On Wed, Nov 7, 2018 at 4:42 PM, Daniel Colascione wrote: > configuration!" is something I've heard more than once. Who's to say > that sysfs is for exposing /proc/pid/stat, Patch is about /proc/stat not /proc/PID/stat. Please revise your arguments based on that. Thanks, Miklos

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Daniel Colascione
On Wed, Nov 7, 2018 at 10:03 AM, Miklos Szeredi wrote: > On Wed, Nov 7, 2018 at 12:48 AM, Andrew Morton > wrote: >> On Mon, 29 Oct 2018 23:04:45 + Daniel Colascione >> wrote: >> >>> On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: >>> > This patch introduces a new /proc/stat2 file t

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Miklos Szeredi
On Wed, Nov 7, 2018 at 12:48 AM, Andrew Morton wrote: > On Mon, 29 Oct 2018 23:04:45 + Daniel Colascione > wrote: > >> On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: >> > This patch introduces a new /proc/stat2 file that is identical to the >> > regular 'stat' except that it zeroes

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-06 Thread Davidlohr Bueso
On Tue, 06 Nov 2018, Andrew Morton wrote: It would be interesting to know precisely which stat fields the database-which-shall-not-be-named is looking for. Then we could cook up a very whizzy way of getting at the info. The ctxt field, afaik. In any case they have been able to work around the

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-06 Thread Andrew Morton
On Mon, 29 Oct 2018 23:04:45 + Daniel Colascione wrote: > On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: > > This patch introduces a new /proc/stat2 file that is identical to the > > regular 'stat' except that it zeroes all hard irq statistics. The new > > file is a drop in replacem

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-30 Thread Davidlohr Bueso
On Tue, 30 Oct 2018, Vito Caputo wrote: If you create /proc/stat2 to omit interrupts, do we then create /proc/stat3 to omit CPUs when just interrupts are of interest to the application running on a 256-cpu machine? Be real, this is a bogus argument. As mentioned, stat2 is named as such because

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-30 Thread Vito Caputo
On Tue, Oct 30, 2018 at 11:57:56AM -0700, Davidlohr Bueso wrote: > On Mon, 29 Oct 2018, Vito Caputo wrote: > > > I'm definitely not in favor of just adding another stat file that is the > > same format as the existing one with the intrs zeroed out. It's a dirty > > hack; fine for your local needs

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-30 Thread Davidlohr Bueso
On Mon, 29 Oct 2018, Vito Caputo wrote: I'm definitely not in favor of just adding another stat file that is the same format as the existing one with the intrs zeroed out. It's a dirty hack; fine for your local needs but too gross for upstream IMHO. I suspect very few users of /proc/stat actu

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Vito Caputo
On Mon, Oct 29, 2018 at 11:04:45PM +, Daniel Colascione wrote: > On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: > > This patch introduces a new /proc/stat2 file that is identical to the > > regular 'stat' except that it zeroes all hard irq statistics. The new > > file is a drop in rep

Re: Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Alexey Dobriyan
On Mon, Oct 29, 2018 at 11:40:47PM +, Daniel Colascione wrote: > On Mon, Oct 29, 2018 at 11:34 PM, Alexey Dobriyan wrote: > >> I'd much rather move to a model in which userspace *explicitly* tells > >> the kernel which fields it wants, with the kernel replying with just > >> those particular f

Re: Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Daniel Colascione
On Mon, Oct 29, 2018 at 11:34 PM, Alexey Dobriyan wrote: >> I'd much rather move to a model in which userspace *explicitly* tells >> the kernel which fields it wants, with the kernel replying with just >> those particular fields, maybe in their raw binary representations. >> The ASCII-text bag-of-

Re: Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Alexey Dobriyan
> I'd much rather move to a model in which userspace *explicitly* tells > the kernel which fields it wants, with the kernel replying with just > those particular fields, maybe in their raw binary representations. > The ASCII-text bag-of-everything files would remain available for > ad-hoc and non-p

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Daniel Colascione
On Mon, Oct 29, 2018 at 7:25 PM, Davidlohr Bueso wrote: > This patch introduces a new /proc/stat2 file that is identical to the > regular 'stat' except that it zeroes all hard irq statistics. The new > file is a drop in replacement to stat for users that need performance. For a while now, I've be

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Vito Caputo
On Mon, Oct 29, 2018 at 05:35:15PM -0400, Waiman Long wrote: > On 10/29/2018 05:23 PM, Vito Caputo wrote: > > On Mon, Oct 29, 2018 at 04:59:03PM -0400, Waiman Long wrote: > >> On 10/29/2018 04:38 PM, Davidlohr Bueso wrote: > >>> On Mon, 29 Oct 2018, Waiman Long wrote: > >>> > BTW, since you ar

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Waiman Long
On 10/29/2018 05:23 PM, Vito Caputo wrote: > On Mon, Oct 29, 2018 at 04:59:03PM -0400, Waiman Long wrote: >> On 10/29/2018 04:38 PM, Davidlohr Bueso wrote: >>> On Mon, 29 Oct 2018, Waiman Long wrote: >>> BTW, since you are making stat2 compatible with stat, will that be easier from the us

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Vito Caputo
On Mon, Oct 29, 2018 at 04:59:03PM -0400, Waiman Long wrote: > On 10/29/2018 04:38 PM, Davidlohr Bueso wrote: > > On Mon, 29 Oct 2018, Waiman Long wrote: > > > >> BTW, since you are making stat2 compatible with stat, will that be > >> easier from the user API perspective if we use a sysctl paramete

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Waiman Long
On 10/29/2018 03:25 PM, Davidlohr Bueso wrote: > A recent report from a large database vendor which I shall not name > shows concerns about poor performance when consuming /proc/stat info. > Particularly kstat_irq() pops up in the profiles and most time is > being spent there. The overall system i

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Waiman Long
On 10/29/2018 04:38 PM, Davidlohr Bueso wrote: > On Mon, 29 Oct 2018, Waiman Long wrote: > >> BTW, since you are making stat2 compatible with stat, will that be >> easier from the user API perspective if we use a sysctl parameter to >> turn on and off IRQs reporting for /proc/stat, for example? > >

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Davidlohr Bueso
On Mon, 29 Oct 2018, Waiman Long wrote: BTW, since you are making stat2 compatible with stat, will that be easier from the user API perspective if we use a sysctl parameter to turn on and off IRQs reporting for /proc/stat, for example? For one /proc/stat is also common for debugging envs (ie:

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Waiman Long
On 10/29/2018 04:00 PM, Davidlohr Bueso wrote: > On Mon, 29 Oct 2018, Waiman Long wrote: > >> I am wondering if /proc/stat_noirqs will be a more descriptive name of >> the intent of this new procfs file or we should just go with the more >> generic stat2 name. > > The reason why I went with '2' ins

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Alexey Dobriyan
> I am wondering if /proc/stat_noirqs will be a more descriptive name of > the intent of this new procfs file or we should just go with the more > generic stat2 name. What would you do if someone asks for /proc/stat without CPU numbers?

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Davidlohr Bueso
On Mon, 29 Oct 2018, Waiman Long wrote: I am wondering if /proc/stat_noirqs will be a more descriptive name of the intent of this new procfs file or we should just go with the more generic stat2 name. The reason why I went with '2' instead of a more rescriptive name was that I think of the cal

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Waiman Long
On 10/29/2018 03:25 PM, Davidlohr Bueso wrote: > A recent report from a large database vendor which I shall not name > shows concerns about poor performance when consuming /proc/stat info. > Particularly kstat_irq() pops up in the profiles and most time is > being spent there. The overall system i

[PATCH] fs/proc: introduce /proc/stat2 file

2018-10-29 Thread Davidlohr Bueso
A recent report from a large database vendor which I shall not name shows concerns about poor performance when consuming /proc/stat info. Particularly kstat_irq() pops up in the profiles and most time is being spent there. The overall system is under a lot of irqs and almost 1k cores, thus this co