Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-06-04 Thread Amit Kucheria
On Tue, May 5, 2020 at 3:07 AM David Rientjes wrote: > > On Mon, 4 May 2020, Emanuele Giuseppe Esposito wrote: > > > There is currently no common way for Linux kernel subsystems to expose > > statistics to userspace shared throughout the Linux kernel; subsystems > > have to take care of gathering

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-14 Thread Paolo Bonzini
On 14/05/20 19:35, Jonathan Adams wrote: >> In general for statsfs we took a more explicit approach where each >> addend in a sum is a separate stats_fs_source. In this version of the >> patches it's also a directory, but we'll take your feedback and add both >> the ability to hide directories (fi

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-14 Thread Jonathan Adams
On Mon, May 11, 2020 at 10:34 AM Paolo Bonzini wrote: > > Hi Jonathan, I think the remaining sticky point is this one: Apologies it took a couple days for me to respond; I wanted to finish evaluating our current usage to make sure I had a full picture; I'll summarize our state at the bottom. > O

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-11 Thread Paolo Bonzini
Hi Jonathan, I think the remaining sticky point is this one: On 11/05/20 19:02, Jonathan Adams wrote: > I think I'd characterize this slightly differently; we have a set of > statistics which are essentially "in parallel": > > - a variety of statistics, N CPUs they're available for, or > - a

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-11 Thread Jonathan Adams
On Fri, May 8, 2020 at 2:44 AM Paolo Bonzini wrote: > > [Answering for Emanuele because he's not available until Monday] > > On 07/05/20 19:45, Jonathan Adams wrote: > > This is good work. As David Rientjes mentioned, I'm currently investigating > > a similar project, based on a google-internal d

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-11 Thread Emanuele Giuseppe Esposito
On 5/8/20 11:44 AM, Paolo Bonzini wrote: > So in general I'd say the sources/values model holds up. We certainly > want to: > > - switch immediately to callbacks instead of the type constants (so that > core statsfs code only does signed/unsigned) > > - add a field to distinguish cumulative an

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-08 Thread Paolo Bonzini
[Answering for Emanuele because he's not available until Monday] On 07/05/20 19:45, Jonathan Adams wrote: > This is good work. As David Rientjes mentioned, I'm currently investigating > a similar project, based on a google-internal debugfs-based FS we call > "metricfs". It's > designed in a slig

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-07 Thread Jonathan Adams
On Mon, May 4, 2020 at 4:05 AM Emanuele Giuseppe Esposito wrote: ... > Statsfs offers a generic and stable API, allowing any kind of > directory/file organization and supporting multiple kind of aggregations > (not only sum, but also average, max, min and count_zero) and data types > (all unsigned

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-05 Thread Christian Borntraeger
Adding Stefan Raspl, who has done a lot of kvm_stat work in the past. On 05.05.20 19:21, Paolo Bonzini wrote: > On 05/05/20 19:07, David Rientjes wrote: >>> I am totally in favor of having a binary format, but it should be >>> introduced as a separate series on top of this one---and preferably by

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-05 Thread Paolo Bonzini
On 05/05/20 19:07, David Rientjes wrote: >> I am totally in favor of having a binary format, but it should be >> introduced as a separate series on top of this one---and preferably by >> someone who has already put some thought into the problem (which >> Emanuele and I have not, beyond ensuring tha

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-05 Thread David Rientjes
On Tue, 5 May 2020, Paolo Bonzini wrote: > >>> Since this is becoming a generic API (good!!), maybe we can discuss > >>> possible ways to optimize gathering of stats in mass? > >> Sure, the idea of a binary format was considered from the beginning in > >> [1], and it can be done either together wi

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-05 Thread Paolo Bonzini
On 05/05/20 18:53, Jim Mattson wrote: >>> Since this is becoming a generic API (good!!), maybe we can discuss >>> possible ways to optimize gathering of stats in mass? >> Sure, the idea of a binary format was considered from the beginning in >> [1], and it can be done either together with the curre

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-05 Thread Jim Mattson
On Tue, May 5, 2020 at 2:18 AM Emanuele Giuseppe Esposito wrote: > > > > On 5/4/20 11:37 PM, David Rientjes wrote: > > Since this is becoming a generic API (good!!), maybe we can discuss > > possible ways to optimize gathering of stats in mass? > > Sure, the idea of a binary format was considered

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-05 Thread Emanuele Giuseppe Esposito
On 5/4/20 11:37 PM, David Rientjes wrote: On Mon, 4 May 2020, Emanuele Giuseppe Esposito wrote: In this patch series I introduce statsfs, a synthetic ram-based virtual filesystem that takes care of gathering and displaying statistics for the Linux kernel subsystems. This is exciting, we

Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-04 Thread David Rientjes
On Mon, 4 May 2020, Emanuele Giuseppe Esposito wrote: > There is currently no common way for Linux kernel subsystems to expose > statistics to userspace shared throughout the Linux kernel; subsystems > have to take care of gathering and displaying statistics by themselves, > for example in the for

[PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics

2020-05-04 Thread Emanuele Giuseppe Esposito
There is currently no common way for Linux kernel subsystems to expose statistics to userspace shared throughout the Linux kernel; subsystems have to take care of gathering and displaying statistics by themselves, for example in the form of files in debugfs. For example KVM has its own code section