On Thu, Apr 19, 2018 at 12:43:19PM -0700, Andrew Morton wrote: > On Thu, 19 Apr 2018 13:09:29 -0400 Waiman Long <long...@redhat.com> wrote: > > > It was found that reading /proc/stat could be time consuming on > > systems with a lot of irqs. For example, reading /proc/stat in a > > certain 2-socket Skylake server took about 4.6ms because it had over > > 5k irqs. In that particular case, the majority of the CPU cycles for > > reading /proc/stat was spent in the kstat_irqs() function. Therefore, > > application performance can be impacted if the application reads > > /proc/stat rather frequently. > > > > The "intr" line within /proc/stat contains a sum total of all the irqs > > that have happened followed by a list of irq counts for each individual > > irq number. In many cases, the first number is good enough. The > > individual irq counts may not provide that much more information. > > > > In order to avoid this kind of performance issue, all these individual > > irq counts are now separated into a new /proc/stat_irqs file. The > > sum total irq count will stay in /proc/stat and be duplicated in > > /proc/stat_irqs. Applications that need to look up individual irq counts > > will now have to look into /proc/stat_irqs instead of /proc/stat. > > > > (cc /proc maintainer) > > It's a non-backward-compatible change. For something which has > existing for so long, it would be a mighty task to demonstrate that no > existing userspace will be disrupted by this change. > > So we need to think again. A new interface which omits the per-IRQ > stats might be acceptable.
Here is profile of open+read+close /proc/stat
30% is taking mutex only to print "0".
+ 98.80% 0.04% a.out [kernel.vmlinux] [k] entry_SYSCALL_64
▒
+ 98.75% 0.10% a.out [kernel.vmlinux] [k] do_syscall_64
▒
+ 95.56% 0.04% a.out libc-2.25.so [.] __GI___libc_read
◆
+ 95.09% 0.01% a.out [kernel.vmlinux] [k] sys_read
▒
+ 95.04% 0.03% a.out [kernel.vmlinux] [k] vfs_read
▒
+ 94.98% 0.05% a.out [kernel.vmlinux] [k] proc_reg_read
▒
+ 94.98% 0.00% a.out [kernel.vmlinux] [k] __vfs_read
▒
+ 94.92% 0.06% a.out [kernel.vmlinux] [k] seq_read
▒
+ 94.52% 3.65% a.out [kernel.vmlinux] [k] show_stat
▒
+ 48.62% 2.59% a.out [kernel.vmlinux] [k] kstat_irqs_usr
▒
+ 33.52% 9.55% a.out [kernel.vmlinux] [k] seq_put_decimal_ull
▒
+ 19.63% 19.59% a.out [kernel.vmlinux] [k] memcpy_erms
▒
+ 17.34% 9.53% a.out [kernel.vmlinux] [k] kstat_irqs
▒
- 15.45% 15.43% a.out [kernel.vmlinux] [k] mutex_lock
▒
15.43% __GI___libc_read
▒
entry_SYSCALL_64
▒
do_syscall_64
▒
sys_read
▒
vfs_read
▒
__vfs_read
▒
proc_reg_read
▒
- seq_read
▒
- 15.41% show_stat
▒
kstat_irqs_usr
▒
mutex_lock
▒
+ 13.32% 13.27% a.out [kernel.vmlinux] [k] mutex_unlock
▒
+ 4.60% 1.35% a.out [kernel.vmlinux] [k] cpumask_next
▒
+ 3.03% 3.03% a.out [kernel.vmlinux] [k] __radix_tree_lookup
▒
+ 2.96% 0.08% a.out [kernel.vmlinux] [k] seq_printf
▒
+ 2.92% 0.02% a.out libc-2.25.so [.] __GI___libc_open
▒
+ 2.89% 0.07% a.out [kernel.vmlinux] [k] seq_vprintf
▒
+ 2.81% 0.70% a.out [kernel.vmlinux] [k] vsnprintf
▒
+ 2.66% 2.66% a.out [kernel.vmlinux] [k] _find_next_bit
▒
+ 2.42% 1.36% a.out [kernel.vmlinux] [k] num_to_str
▒
+ 2.41% 0.19% a.out [kernel.vmlinux] [k] get_idle_time
▒
+ 2.39% 0.02% a.out [kernel.vmlinux] [k] do_sys_open
