Re: Interrupts hover above 40% when idle on Dell Latitude E7450

[Scott Cheloha]

On Sun, Jan 30, 2022 at 10:41:34AM -0500, Ryan Kavanagh wrote:
> On Sun, Jan 30, 2022 at 12:39:02AM -0600, Scott Cheloha wrote:
> > > btrace -e 'profile:hz:100 { @[kstack] = count(); }' > /tmp/btrace.out
> > > 
> > > for ten seconds and ran the output through
> > > 
> > > https://github.com/brendangregg/FlameGraph/raw/master/stackcollapse-bpftrace.pl
> > > https://github.com/brendangregg/FlameGraph/raw/master/flamegraph.pl
> > > 
> > > The output of stackcollapse-bpftrace.pl and flamegraph.pl are attached
> > > as btrace.collapsed and btrace.svg.
> > 
> > The flamegraph suggests that you spent 10% of that time servicing
> > ichiic(4) interrupts from idle.
> > 
> > That could be a fluke though.
> 
> In case it was a fluke, I've regenerated the flamegraph on 7.0
> GENERIC.MP#293 amd64 using 10 seconds of output on an idle machine.
> Please see attached.
> 
> > What does the main systat view look like in the interrupt column?
> > 
> > $ systat 1
> 
> Again on #293:
> 
> Interrupts(range after idling for a few seconds)
>  247 total(235-260)
>  200 clock(200-200)
>   21 ipi  (16-23)
>1 acpi0(0-1)
>6 inteldrm (5-7)
>  azalia1  (0-0)
>   11 iwm0 (10-16)
>  ehci0(0-0)
>1 ahci0(0-1)
>1 ichiic0  (0-1)
>6 pckbc0   (0-0)
>  pckbc0   (0-0)

Based on these numbers and the similar-looking flamegraph I'd say
you're spending a relatively large amount of time handling ichiic(4)
interrupts.

I don't know anything about that device but my guess is that it is
slow if you're spending that much time in x86_bus_space_io_read_1()
and its _write_1() counterpart.

Someone else is going to have to weigh in on what might be the cause
and solution.

Thank you providing the traces.

Re: Interrupts hover above 40% when idle on Dell Latitude E7450

[Scott Cheloha]

On Mon, Jan 10, 2022 at 10:36:41AM -0500, Ryan Kavanagh wrote:
> >Synopsis:high interrupt processing time on Dell Latitude E7450
> >Category:kernel amd64
> >Environment:
>   System  : OpenBSD 7.0
>   Details : OpenBSD 7.0-current (GENERIC.MP) #242: Sat Jan  8 
> 12:33:38 MST 2022
>
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> 
>   Architecture: OpenBSD.amd64
>   Machine : amd64
> >Description:
> 
> I installed OpenBSD 7.0 on a Dell Latitude E7450. The installer had me
> using GENERIC, and interrupts hovered above 40%-45% when idle. I
> upgraded to 7.0-current with the same issue. The laptop ran hot,
> presumably as a result of the high interrupt processing time.
> 
> Here is the output of vmstat -i under 7.0-current / GENERIC:
> 
> interrupt   total rate
> irq0/clock 885333   99
> irq96/acpi0  17890
> irq144/inteldrm0   117024   13
> irq114/em0  124141
> irq176/azalia1   16680
> irq115/iwm0416586   47
> irq102/ehci0  2210
> irq103/ahci0   124340   14
> irq104/ichiic0   17700
> irq145/pckbc0   177101
> irq146/pckbc0   445885
> Total 1623443  183
> 
> Stuart Henderson suggested in the misc thread [0] that I try GENERIC.MP.
> After switching, top shows that CPU0 is spending roughly the same amount
> of time processing interrupts, while CPU1 is free to do other stuff.
> Representative capture:
> 
> CPU0 states: 18.6% user,  0.0% nice,  3.0% sys,  0.2% spin, 44.3% intr,
> 33.9% idle
> CPU1 states: 50.3% user,  0.0% nice,  5.0% sys,  1.6% spin,  0.0% intr,
> 43.1% idle
> 
> `systat vm` shows an Int% hovering around 20% (I imagine it's the mean
> of the two interrupt rates shown by top), with lows in the low 10% and
> highs in the low 30%.
> 
> My laptop does not seem to get as hot as quickly under GENERIC.MP, but
> it still gets pretty hot: hw.sensors.cpu0.temp0=86.00 degC after 5
> minutes when watching a video on Youtube in firefox.
> 
> vmstat -i under GENERIC.MP after being up ~35 minutes with light load:
> 
> interrupt   total rate
> irq0/clock 860124  399
> irq0/ipi   714019  331
> irq96/acpi0   4360
> irq144/inteldrm033945   15
> irq114/em0   79653
> irq176/azalia1  10
> irq115/iwm0 73471   34
> irq102/ehci0   730
> irq103/ahci044676   20
> irq104/ichiic04310
> irq145/pckbc064412
> irq146/pckbc031781
> Total 1744760  810
> 
> I ran
> 
> btrace -e 'profile:hz:100 { @[kstack] = count(); }' > /tmp/btrace.out
> 
> for ten seconds and ran the output through
> 
> https://github.com/brendangregg/FlameGraph/raw/master/stackcollapse-bpftrace.pl
> https://github.com/brendangregg/FlameGraph/raw/master/flamegraph.pl
> 
> The output of stackcollapse-bpftrace.pl and flamegraph.pl are attached
> as btrace.collapsed and btrace.svg.

The flamegraph suggests that you spent 10% of that time servicing
ichiic(4) interrupts from idle.

That could be a fluke though.

What does the main systat view look like in the interrupt column?

$ systat 1