Re: [PATCH 0/4 v6] sched,time: reduce nohz_full syscall overhead 40%

[Thomas Gleixner]

On Wed, 10 Feb 2016, r...@redhat.com wrote:

> (v6: make VIRT_CPU_ACCOUNTING_GEN jiffy granularity)
> 
> Running with nohz_full introduces a fair amount of overhead.
> Specifically, various things that are usually done from the
> timer interrupt are now done at syscall, irq, and guest
> entry and exit times.
> 
> However, some of the code that is called every single time
> has only ever worked at jiffy resolution. The code in
> __acct_update_integrals was also doing some unnecessary
> calculations.
> 
> Getting rid of the unnecessary calculations, without
> changing any of the functionality in __acct_update_integrals
> gets us about an 11% win.
> 
> Not calling the time statistics updating code more than
> once per jiffy, like is done on housekeeping CPUs and on
> all the CPUs of a non-nohz_full system, shaves off a
> further 30%.

Reviewed-by: Thomas Gleixner 

Re: [PATCH 0/4 v6] sched,time: reduce nohz_full syscall overhead 40%

[Thomas Gleixner]

On Wed, 10 Feb 2016, r...@redhat.com wrote:

> (v6: make VIRT_CPU_ACCOUNTING_GEN jiffy granularity)
> 
> Running with nohz_full introduces a fair amount of overhead.
> Specifically, various things that are usually done from the
> timer interrupt are now done at syscall, irq, and guest
> entry and exit times.
> 
> However, some of the code that is called every single time
> has only ever worked at jiffy resolution. The code in
> __acct_update_integrals was also doing some unnecessary
> calculations.
> 
> Getting rid of the unnecessary calculations, without
> changing any of the functionality in __acct_update_integrals
> gets us about an 11% win.
> 
> Not calling the time statistics updating code more than
> once per jiffy, like is done on housekeeping CPUs and on
> all the CPUs of a non-nohz_full system, shaves off a
> further 30%.

Reviewed-by: Thomas Gleixner 

Re: [PATCH 0/4 v6] sched,time: reduce nohz_full syscall overhead 40%

[Mike Galbraith]

Hi Rik,

On Wed, 2016-02-10 at 20:08 -0500, r...@redhat.com wrote:

> I tested this series with a microbenchmark calling
> an invalid syscall number ten million times in a row,
> on a nohz_full cpu.
> 
> Run times for the microbenchmark:
> 
> 4.4 3.8 seconds
> 4.5-rc1 3.7 seconds
> 4.5-rc1 + first patch   3.3 seconds
> 4.5-rc1 + first 3 patches   3.1 seconds
> 4.5-rc1 + all patches   2.3 seconds
> 
>Same test on a non-NOHZ_FULL, non-housekeeping CPU:
> all kernels 1.86 seconds

I tested 10M stat(".", ) calls, and saw a win of ~20% on a
nohz_full cpu.  Below are nopreempt vs nohz_full+patches overhead
numbers from my box.
avg
4.4.1-nopreempt0m1.652s   0m1.633s   0m1.635s   1.640   1.000

nohz_full + patches
nohz_full inactive 0m1.642s   0m1.631s   0m1.629s   1.634.996
housekeeper CPU0m2.013s   0m2.012s   0m2.033s   2.019   1.231
nohz_full CPU  0m2.247s   0m2.233s   0m2.239s   2.239   1.365

It still ain't free ;-) but between this set, and all the other work
that has gone in ~recently, it looks one hell of a lot better.  That's
not too scary a pricetag.

-Mike

Re: [PATCH 0/4 v6] sched,time: reduce nohz_full syscall overhead 40%

[Mike Galbraith]

Hi Rik,

On Wed, 2016-02-10 at 20:08 -0500, r...@redhat.com wrote:

> I tested this series with a microbenchmark calling
> an invalid syscall number ten million times in a row,
> on a nohz_full cpu.
> 
> Run times for the microbenchmark:
> 
> 4.4 3.8 seconds
> 4.5-rc1 3.7 seconds
> 4.5-rc1 + first patch   3.3 seconds
> 4.5-rc1 + first 3 patches   3.1 seconds
> 4.5-rc1 + all patches   2.3 seconds
> 
>Same test on a non-NOHZ_FULL, non-housekeeping CPU:
> all kernels 1.86 seconds

I tested 10M stat(".", ) calls, and saw a win of ~20% on a
nohz_full cpu.  Below are nopreempt vs nohz_full+patches overhead
numbers from my box.
avg
4.4.1-nopreempt0m1.652s   0m1.633s   0m1.635s   1.640   1.000

nohz_full + patches
nohz_full inactive 0m1.642s   0m1.631s   0m1.629s   1.634.996
housekeeper CPU0m2.013s   0m2.012s   0m2.033s   2.019   1.231
nohz_full CPU  0m2.247s   0m2.233s   0m2.239s   2.239   1.365

It still ain't free ;-) but between this set, and all the other work
that has gone in ~recently, it looks one hell of a lot better.  That's
not too scary a pricetag.

-Mike