Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-12 Thread Sebastian Andrzej Siewior
On 2020-11-12 13:39:02 [+0100], Daniel Wagner wrote:
> With the current version signaltest + your test patch and 'taskset -c1'
> the results looks good again, around 230us (running since 2 hours). I've
> tested first without taskset and it took about an half hour to hit
> 350us. So pinning the threads on one CPU fixes it.

okay. So case closed.

> I think we change signaltest to use the correct affinity on
> default. Also, I see that sigwaittest has some code for it, but it, but
> it would be a good idea to set the defaults so that out of the box the
> test does the right thing.

Sounds reasonable. Having tasks jumping from one CPU to another may lead
to higher latencies.

> I'm sorry about dragging you into this problem.
I feared that something in lazy preempt or signal stack is broken. But
it appears not to be the case :)

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-12 Thread Daniel Wagner
On Tue, Nov 10, 2020 at 07:05:18PM +0100, Sebastian Andrzej Siewior wrote:
> On 2020-11-09 15:37:03 [+0100], Daniel Wagner wrote:
> > > I've been staring at the code of signaltest on Friday and I might need
> > > to stare longer to figure out what it does.
> > 
> > I hear you. Anyway, I gave the current head a run with lazy preemption
> > disabled as you asked for.
> 
> I just sent a few patches your way regarding signaltest. It should help
> you with tracing. I've been playing with it on a juno box and I didn't
> see anything odd. My max value was below 200us. I added a few tracing
> bits. With sched, signal and hrtimer events you should be able to see
> what delays the RT thread. I didn't see anything odd.

With the current version signaltest + your test patch and 'taskset -c1'
the results looks good again, around 230us (running since 2 hours). I've
tested first without taskset and it took about an half hour to hit
350us. So pinning the threads on one CPU fixes it.

I think we change signaltest to use the correct affinity on
default. Also, I see that sigwaittest has some code for it, but it, but
it would be a good idea to set the defaults so that out of the box the
test does the right thing.

I'm sorry about dragging you into this problem.


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-11 Thread Daniel Wagner
Sorry for the late response, I had to reinstall my system after a FS
corruption...

On Mon, Nov 09, 2020 at 05:31:43PM +0100, Sebastian Andrzej Siewior wrote:
> > These test run only very short with hackbench as worlkload (5 minutes).
> > Though I running these tests now for more than year with v4.4-rt and
> > some times the newer -rt releases and I've never seen the latency
> > numbers above 200us unless something was broken. Given that 5 minutes is
> > not really long, I'll let those test run for longer to see if I get the
> > same results when they run for one hour.

- 5.9.0-rc8-rt12, ca 5h
  T: 0 (11626) P:80 C:15092432 Min: 17 Act:   34 Avg:   43 Max: 226

- 5.9.0-rc8-rt13, ca 1.5h
  T: 0 (24661) P:80 C:5581936 Min: 21 Act:   35 Avg:   45 Max: 250

- 5.9.0-rc8-rt14, ca 1h
  T: 0 (  942) P:80 C:6522320 Min: 20 Act:   27 Avg:   44 Max: 352

This matches with the 5 minutes runs. -rt13 was still okay and -rt14
is clearly worse.

> > 5.10.0-rc2-rt4 vs 5.10.0-rc2-rt4(lazy preemption disabled)
> >
> >   0_cyclicdeadline t2-max-latency   pass/pass
> > 274.00/ 61.00 349.18%
>
> So the value went from 274us to 61us after disabling lazy-preempt?

Yes, that was all I changed. I want to redo this measurement. It
really looks a bit bogus. Though, one thing after the other :)

Daniel


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-10 Thread Sebastian Andrzej Siewior
On 2020-11-09 15:37:03 [+0100], Daniel Wagner wrote:
> > I've been staring at the code of signaltest on Friday and I might need
> > to stare longer to figure out what it does.
> 
> I hear you. Anyway, I gave the current head a run with lazy preemption
> disabled as you asked for.

I just sent a few patches your way regarding signaltest. It should help
you with tracing. I've been playing with it on a juno box and I didn't
see anything odd. My max value was below 200us. I added a few tracing
bits. With sched, signal and hrtimer events you should be able to see
what delays the RT thread. I didn't see anything odd.

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-09 Thread Sebastian Andrzej Siewior
On 2020-11-09 15:37:03 [+0100], Daniel Wagner wrote:
> > this looks odd. So rt1 has 415, rt2 has 399 and rt3 has 420 so lets say
> > it is the same. And then rt4 should reduce it to 340. The only part that
> > could have some influence is the are the highmem/kmap patches. But for
> > ARM64 these are still a nop and in both cases kmap_atomic() disables
> > migrate & page-fault.
> > 
> > Are you sure those numbers always reproducible and not something that
> > goes wrong and sometimes it is captured at 300us and sometimes 400us.
> 
> These test run only very short with hackbench as worlkload (5 minutes).
> Though I running these tests now for more than year with v4.4-rt and
> some times the newer -rt releases and I've never seen the latency
> numbers above 200us unless something was broken. Given that 5 minutes is
> not really long, I'll let those test run for longer to see if I get the
> same results when they run for one hour.

oki.

> > I've been staring at the code of signaltest on Friday and I might need
> > to stare longer to figure out what it does.
> 
> I hear you. Anyway, I gave the current head a run with lazy preemption
> disabled as you asked for.

…
> 5.10.0-rc2-rt4 vs 5.10.0-rc2-rt4(lazy preemption disabled)
> 
>   0_cyclicdeadline t2-max-latency   pass/pass274.00/  
>61.00 349.18%

So the value went from 274us to 61us after disabling lazy-preempt?

> cyclicdeadline seems heavily affected by the change.

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-09 Thread Daniel Wagner
On Mon, Nov 09, 2020 at 01:47:18PM +0100, Sebastian Andrzej Siewior wrote:
> So it is the new migrate-disable code? If you have stable 100us you
> should be able bisect it within the few commits between rt13 and rt14.

Okay, I'll start a bissect in this range.

> this looks odd. So rt1 has 415, rt2 has 399 and rt3 has 420 so lets say
> it is the same. And then rt4 should reduce it to 340. The only part that
> could have some influence is the are the highmem/kmap patches. But for
> ARM64 these are still a nop and in both cases kmap_atomic() disables
> migrate & page-fault.
> 
> Are you sure those numbers always reproducible and not something that
> goes wrong and sometimes it is captured at 300us and sometimes 400us.

These test run only very short with hackbench as worlkload (5 minutes).
Though I running these tests now for more than year with v4.4-rt and
some times the newer -rt releases and I've never seen the latency
numbers above 200us unless something was broken. Given that 5 minutes is
not really long, I'll let those test run for longer to see if I get the
same results when they run for one hour.

> I've been staring at the code of signaltest on Friday and I might need
> to stare longer to figure out what it does.

I hear you. Anyway, I gave the current head a run with lazy preemption
disabled as you asked for.

I had to add two ifdefs to get it compiling first:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 3fce6bbbeb5b..5a58ead3cf00 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1800,7 +1800,9 @@ void migrate_disable(void)
preempt_disable();
this_rq()->nr_pinned++;
p->migration_disabled = 1;
+#ifdef CONFIG_PREEMPT_LAZY
preempt_lazy_disable();
+#endif
preempt_enable();
 }
 EXPORT_SYMBOL_GPL(migrate_disable);
@@ -1829,7 +1831,9 @@ void migrate_enable(void)
barrier();
p->migration_disabled = 0;
this_rq()->nr_pinned--;
+#ifdef CONFIG_PREEMPT_LAZY
preempt_lazy_enable();
+#endif
preempt_enable();
 
trace_sched_migrate_enable_tp(p);


5.10.0-rc2-rt4 vs 5.10.0-rc2-rt4(lazy preemption disabled)

  0_cyclicdeadline t2-max-latency   pass/pass274.00/
 61.00 349.18%
  0_cyclicdeadline t2-avg-latency   pass/pass217.00/
 19.001042.11%
  0_cyclicdeadline t2-min-latency   pass/pass 11.00/
  1.001000.00%
  0_cyclicdeadline t1-max-latency   pass/pass113.00/
132.00 -14.39%
  0_cyclicdeadline t1-avg-latency   pass/pass 21.00/
 24.00 -12.50%
  0_cyclicdeadline t1-min-latency   pass/pass  1.00/
  1.00   0.00%
  0_cyclicdeadline t0-max-latency   pass/pass258.00/
110.00 134.55%
  0_cyclicdeadline t0-avg-latency   pass/pass140.00/
 19.00 636.84%
  0_cyclicdeadline t0-min-latency   pass/pass  5.00/
  1.00 400.00%
  0_cyclictest t3-max-latency   pass/pass 90.00/
118.00 -23.73%
  0_cyclictest t3-avg-latency   pass/pass 33.00/
 30.00  10.00%
  0_cyclictest t3-min-latency   pass/pass 11.00/
 11.00   0.00%
  0_cyclictest t2-max-latency   pass/pass 93.00/
 96.00  -3.12%
  0_cyclictest t2-avg-latency   pass/pass 28.00/
 29.00  -3.45%
  0_cyclictest t2-min-latency   pass/pass 11.00/
 11.00   0.00%
  0_cyclictest t1-max-latency   pass/pass125.00/
138.00  -9.42%
  0_cyclictest t1-avg-latency   pass/pass 30.00/
 29.00   3.45%
  0_cyclictest t1-min-latency   pass/pass 11.00/
 11.00   0.00%
  0_cyclictest t0-max-latency   pass/pass 95.00/
 97.00  -2.06%
  0_cyclictest t0-avg-latency   pass/pass 30.00/
 30.00   0.00%
  0_cyclictest t0-min-latency   pass/pass 11.00/
 11.00   0.00%
  0_pi-stress  pi-stressfail/fail  0.00/
  0.00   0.00%
  0_pmqtestt7-6-max-latency pass/pass 69.00/
 76.00  -9.21%
  0_pmqtestt7-6-avg-latency pass/pass 20.00/
 19.00   5.26%
  0_pmqtestt7-6-min-latency pass/pass 15.00/
 14.00   7.14%
  0_pmqtestt5-4-max-latency pass/pass 90.00/
 95.00  -5.26%
  0_pmqtestt5-4-avg-latency pass/pass 20.00/
 19.00   5.26%
  0_pmqtestt5-4-min-latency pass/pass 15.00/
 14.00   7.14%
  0_pmqtestt3-2-max-latency pass/p

Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-09 Thread Sebastian Andrzej Siewior
On 2020-11-06 17:14:13 [+0100], Daniel Wagner wrote:
> On Fri, Nov 06, 2020 at 11:54:47AM +0100, Sebastian Andrzej Siewior wrote:
> > > rpi3signaltest  5.9.0-rc8-rt12
> > >   813   0_signaltest t0-max-latency  : fail 214.00
> > > rpi3signaltest  5.9.0-rc8-rt12
> > >   874   0_signaltest t0-max-latency  : fail 217.00
> > > rpi3signaltest  5.9.0-rt16
> > >   963   0_signaltest t0-max-latency  : fail 321.00
> > 
> > Here, rt 13,14,15 would be interesting so we could narrow down the
> > ~100us.
> > v5.9-rc8-rt14 got new migrate-disable but I wouldn't expect it to cause
> > it. The other changes look also harmless (like the rtmutex redo which
> > should be a 0 change but then it mighe behave differently in regard to
> > workqueue in some corner cases).
> 
> rpi3signaltest  5.9.0-rc8-rt13
>   1196  0_signaltest t0-max-latency  : fail 207.00
>   1196  0_signaltest t0-avg-latency  : pass  46.00
>   1196  0_signaltest t0-min-latency  : pass  22.00
> rpi3signaltest  5.9.0-rc8-rt14
>   1197  0_signaltest t0-max-latency  : fail 301.00
>   1197  0_signaltest t0-avg-latency  : pass  47.00
>   1197  0_signaltest t0-min-latency  : pass  20.00
> rpi3signaltest  5.9.0-rt15
>   1198  0_signaltest t0-max-latency  : fail 323.00
>   1198  0_signaltest t0-avg-latency  : pass  47.00
>   1198  0_signaltest t0-min-latency  : pass  21.00

So it is the new migrate-disable code? If you have stable 100us you
should be able bisect it within the few commits between rt13 and rt14.

> > > rpi3signaltest  5.9.1-rt19
> > >   1038  0_signaltest t0-max-latency  : fail 341.00
> > > rpi3signaltest  5.9.1-rt20
> > >   1079  0_signaltest t0-max-latency  : fail 318.00
> >
> > So I have nothing to explain 20us improvement.
> 
> I think 20us is in the range of the standard deviation for this test. So
> I don't think you should be concerned too much about it as long I don't
> have proper statistical numbers.
> 
> One thing I also see is that the average was pretty constant at 47us for
> 5.9-rt and for 5.10-rt series it's around 55us. So something makes the
> whole operation slightly more expensive.
> 
> > > rpi3signaltest  5.10.0-rc1-rt1
> > >   1118  0_signaltest t0-max-latency  : fail 415.00
> > > rpi3signaltest  5.10.0-rc2-rt4
> > >   1163  0_signaltest t0-max-latency  : fail 340.00
> > 
> > -rt2 gained new kmap code.
> > -rt3 received an update of the above
> 
> rpi3signaltest  5.10.0-rc1-rt2
>   1199  0_signaltest t0-max-latency  : fail 399.00
>   1199  0_signaltest t0-avg-latency  : pass  55.00
>   1199  0_signaltest t0-min-latency  : pass  25.00
> rpi3signaltest  5.10.0-rc2-rt3
>   1200  0_signaltest t0-max-latency  : fail 420.00
>   1200  0_signaltest t0-avg-latency  : pass  55.00
>   1200  0_signaltest t0-min-latency  : pass  25.00

this looks odd. So rt1 has 415, rt2 has 399 and rt3 has 420 so lets say
it is the same. And then rt4 should reduce it to 340. The only part that
could have some influence is the are the highmem/kmap patches. But for
ARM64 these are still a nop and in both cases kmap_atomic() disables
migrate & page-fault.

Are you sure those numbers always reproducible and not something that
goes wrong and sometimes it is captured at 300us and sometimes 400us.

I've been staring at the code of signaltest on Friday and I might need
to stare longer to figure out what it does.

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-06 Thread Daniel Wagner
On Fri, Nov 06, 2020 at 11:54:47AM +0100, Sebastian Andrzej Siewior wrote:
> > rpi3signaltest  5.9.0-rc8-rt12
> >   813   0_signaltest t0-max-latency  : fail 214.00
> > rpi3signaltest  5.9.0-rc8-rt12
> >   874   0_signaltest t0-max-latency  : fail 217.00
> > rpi3signaltest  5.9.0-rt16
> >   963   0_signaltest t0-max-latency  : fail 321.00
> 
> Here, rt 13,14,15 would be interesting so we could narrow down the
> ~100us.
> v5.9-rc8-rt14 got new migrate-disable but I wouldn't expect it to cause
> it. The other changes look also harmless (like the rtmutex redo which
> should be a 0 change but then it mighe behave differently in regard to
> workqueue in some corner cases).

rpi3signaltest  5.9.0-rc8-rt13
  1196  0_signaltest t0-max-latency  : fail 207.00
  1196  0_signaltest t0-avg-latency  : pass  46.00
  1196  0_signaltest t0-min-latency  : pass  22.00
rpi3signaltest  5.9.0-rc8-rt14
  1197  0_signaltest t0-max-latency  : fail 301.00
  1197  0_signaltest t0-avg-latency  : pass  47.00
  1197  0_signaltest t0-min-latency  : pass  20.00
rpi3signaltest  5.9.0-rt15
  1198  0_signaltest t0-max-latency  : fail 323.00
  1198  0_signaltest t0-avg-latency  : pass  47.00
  1198  0_signaltest t0-min-latency  : pass  21.00

> > rpi3signaltest  5.9.1-rt19
> >   1038  0_signaltest t0-max-latency  : fail 341.00
> > rpi3signaltest  5.9.1-rt20
> >   1079  0_signaltest t0-max-latency  : fail 318.00
>
> So I have nothing to explain 20us improvement.

I think 20us is in the range of the standard deviation for this test. So
I don't think you should be concerned too much about it as long I don't
have proper statistical numbers.

One thing I also see is that the average was pretty constant at 47us for
5.9-rt and for 5.10-rt series it's around 55us. So something makes the
whole operation slightly more expensive.

> > rpi3signaltest  5.10.0-rc1-rt1
> >   1118  0_signaltest t0-max-latency  : fail 415.00
> > rpi3signaltest  5.10.0-rc2-rt4
> >   1163  0_signaltest t0-max-latency  : fail 340.00
> 
> -rt2 gained new kmap code.
> -rt3 received an update of the above

rpi3signaltest  5.10.0-rc1-rt2
  1199  0_signaltest t0-max-latency  : fail 399.00
  1199  0_signaltest t0-avg-latency  : pass  55.00
  1199  0_signaltest t0-min-latency  : pass  25.00
rpi3signaltest  5.10.0-rc2-rt3
  1200  0_signaltest t0-max-latency  : fail 420.00
  1200  0_signaltest t0-avg-latency  : pass  55.00
  1200  0_signaltest t0-min-latency  : pass  25.00

> But all this is only signal right?

Correct. I've observed this only for signaltest and sigwaittest.

> Nothing on the cyclictest front?

Correct, cyclictest doesn't show any regression.

> If lazy-preempt broke in a way then it should be only noticed by
> cyclictest. You can however disable lazy-preempt just to be sure.

Sure, will do a full run on Monday.


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-06 Thread Sebastian Andrzej Siewior
On 2020-11-04 17:06:50 [+0100], Daniel Wagner wrote:
> On Wed, Nov 04, 2020 at 02:09:30PM +0100, Sebastian Andrzej Siewior wrote:
> > Could you figure out if the arm64 thingy started with -rt4 or was
> > already in rt3?
> 
> I wrote a quick and dirty script to extract the data from my logs to see
> if the regression might be older then I remembered. I filtered out the
> obviously wrong configured runs (e.g !RT). It looks like the first
> recorded outlier is around 5.9.0-rt16. Does this already help or do you
> want me to bissect it down?

> rpi3signaltest  5.9.0-rc8-rt12
>   813   0_signaltest t0-max-latency  : fail 214.00
> rpi3signaltest  5.9.0-rc8-rt12
>   874   0_signaltest t0-max-latency  : fail 217.00
> rpi3signaltest  5.9.0-rt16
>   963   0_signaltest t0-max-latency  : fail 321.00

Here, rt 13,14,15 would be interesting so we could narrow down the
~100us.
v5.9-rc8-rt14 got new migrate-disable but I wouldn't expect it to cause
it. The other changes look also harmless (like the rtmutex redo which
should be a 0 change but then it mighe behave differently in regard to
workqueue in some corner cases).

> rpi3signaltest  5.9.1-rt19
>   1038  0_signaltest t0-max-latency  : fail 341.00
> rpi3signaltest  5.9.1-rt20
>   1079  0_signaltest t0-max-latency  : fail 318.00

Looking from my announcement:
|  - Tiny update to the rtmutex patches (make __read_rt_trylock()
|static).

this could improve things but not that much.

|  - The test_lockup module failed to compile. Reported by Fernando
|Lopez-Lezcano.

unrelated.

|  - The `kcompactd' daemon together with MEMCG could have accessed
|per-CPU variables in preemtible context.

an additional lock…

|  - The patch for the crash in the block layer (previously reported by
|David Runge) has been replaced with another set of patches which
|were submitted upstream.

looks also innocent.

So I have nothing to explain 20us improvement.

> rpi3signaltest  5.10.0-rc1-rt1
>   1118  0_signaltest t0-max-latency  : fail 415.00
> rpi3signaltest  5.10.0-rc2-rt4
>   1163  0_signaltest t0-max-latency  : fail 340.00

-rt2 gained new kmap code.
-rt3 received an update of the above

This is the only outstanding thing between rt1 and rt4.

But all this is only signal right? Nothing on the cyclictest front? If
lazy-preempt broke in a way then it should be only noticed by
cyclictest. You can however disable lazy-preempt just to be sure.

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-04 Thread Daniel Wagner
On Wed, Nov 04, 2020 at 02:09:30PM +0100, Sebastian Andrzej Siewior wrote:
> Could you figure out if the arm64 thingy started with -rt4 or was
> already in rt3?

I wrote a quick and dirty script to extract the data from my logs to see
if the regression might be older then I remembered. I filtered out the
obviously wrong configured runs (e.g !RT). It looks like the first
recorded outlier is around 5.9.0-rt16. Does this already help or do you
want me to bissect it down?

rpi3signaltest  5.4.59-rt36
  382   0_signaltest t0-max-latency  : fail 220.00
  382   0_signaltest t0-avg-latency  : pass  47.00
  382   0_signaltest t0-min-latency  : pass  20.00
rpi3signaltest  5.6.19-rt12
  368   0_signaltest t0-max-latency  : fail 221.00
  368   0_signaltest t0-avg-latency  : pass  45.00
  368   0_signaltest t0-min-latency  : pass  21.00
rpi3signaltest  5.9.0-rc8-rt12
  813   0_signaltest t0-max-latency  : fail 214.00
  813   0_signaltest t0-avg-latency  : pass  45.00
  813   0_signaltest t0-min-latency  : pass  20.00
rpi3signaltest  5.9.0-rc8-rt12
  874   0_signaltest t0-max-latency  : fail 217.00
  874   0_signaltest t0-avg-latency  : pass  45.00
  874   0_signaltest t0-min-latency  : pass  20.00
rpi3signaltest  5.9.0-rt16
  963   0_signaltest t0-max-latency  : fail 321.00
  963   0_signaltest t0-avg-latency  : pass  47.00
  963   0_signaltest t0-min-latency  : pass  21.00
rpi3signaltest  5.9.1-rt19
  1038  0_signaltest t0-max-latency  : fail 341.00
  1038  0_signaltest t0-avg-latency  : pass  46.00
  1038  0_signaltest t0-min-latency  : pass  20.00
rpi3signaltest  5.9.1-rt20
  1079  0_signaltest t0-max-latency  : fail 318.00
  1079  0_signaltest t0-avg-latency  : pass  47.00
  1079  0_signaltest t0-min-latency  : pass  21.00
rpi3signaltest  5.10.0-rc1-rt1
  1118  0_signaltest t0-max-latency  : fail 415.00
  1118  0_signaltest t0-avg-latency  : pass  53.00
  1118  0_signaltest t0-min-latency  : pass  23.00
rpi3signaltest  5.10.0-rc2-rt4
  1163  0_signaltest t0-max-latency  : fail 340.00
  1163  0_signaltest t0-avg-latency  : pass  53.00
  1163  0_signaltest t0-min-latency  : pass  24.00


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-04 Thread Sebastian Andrzej Siewior
On 2020-11-04 13:47:46 [+0100], Daniel Wagner wrote:
> On Wed, Nov 04, 2020 at 12:19:48PM +0100, Daniel Wagner wrote:
> > Yes, Just fired up signaltest 5 times for arm64 and x86_64 with the
> > latest release. Keep you posted.
> 
> arm64
>   1184  0_signaltest t0-max-latency  : fail 386.00
>   1185  0_signaltest t0-max-latency  : fail 417.00
>   1186  0_signaltest t0-max-latency  : fail 350.00
>   1187  0_signaltest t0-max-latency  : fail 360.00
>   1188  0_signaltest t0-max-latency  : fail 339.00
> 
> I noticed that also the last view 5.9-rt releases have higher values.
> For example, version 5.9.0-rc8-rt12 has only 217us.
> 
> x86_64
>   1189  0_signaltest t0-max-latency  : fail  50.00
>   1190  0_signaltest t0-max-latency  : pass  46.00
>   1191  0_signaltest t0-max-latency  : pass  45.00
>   1192  0_signaltest t0-max-latency  : pass  47.00
>   1193  0_signaltest t0-max-latency  : fail  52.00
> 
> Same thing for version 5.9.0-rc8-rt12, the max value was 40us.
> 
> I'll work on getting these reports more useful, the performance trend
> seems to be an interesting metric.

Could you figure out if the arm64 thingy started with -rt4 or was
already in rt3?

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-04 Thread Daniel Wagner
On Wed, Nov 04, 2020 at 12:19:48PM +0100, Daniel Wagner wrote:
> Yes, Just fired up signaltest 5 times for arm64 and x86_64 with the
> latest release. Keep you posted.

arm64
  1184  0_signaltest t0-max-latency  : fail 386.00
  1185  0_signaltest t0-max-latency  : fail 417.00
  1186  0_signaltest t0-max-latency  : fail 350.00
  1187  0_signaltest t0-max-latency  : fail 360.00
  1188  0_signaltest t0-max-latency  : fail 339.00

I noticed that also the last view 5.9-rt releases have higher values.
For example, version 5.9.0-rc8-rt12 has only 217us.

x86_64
  1189  0_signaltest t0-max-latency  : fail  50.00
  1190  0_signaltest t0-max-latency  : pass  46.00
  1191  0_signaltest t0-max-latency  : pass  45.00
  1192  0_signaltest t0-max-latency  : pass  47.00
  1193  0_signaltest t0-max-latency  : fail  52.00

Same thing for version 5.9.0-rc8-rt12, the max value was 40us.

I'll work on getting these reports more useful, the performance trend
seems to be an interesting metric.


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-04 Thread Daniel Wagner
On Wed, Nov 04, 2020 at 11:46:17AM +0100, Sebastian Andrzej Siewior wrote:
> How reproducible are these numbers? If these numbers increase between
> rt3 and rt4 then we have a hand full patches to look at.

Usually signaltest generated reproducible results.

I did see those higher numbers also for v5.10-rc1-rt1 (forgot to post
those). For arm64 the max latency was even higher, around 450us. But I
don't have a lot of data so I wouldn't jump to any conclusion.

> just like that?

Yes, Just fired up signaltest 5 times for arm64 and x86_64 with the
latest release. Keep you posted.


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-04 Thread Sebastian Andrzej Siewior
On 2020-11-04 11:38:09 [+0100], Daniel Wagner wrote:
> On Tue, Nov 03, 2020 at 08:57:31PM +0100, Sebastian Andrzej Siewior wrote:
> > I'm pleased to announce the v5.10-rc2-rt4 patch set.
> 
> All tests on passed in my lab. On arm64 and arm I saw slightly higher
> max latency values for signaltest and sigwaittest. Usually they are
> below 200us but currently I see up to 350us.

How reproducible are these numbers? If these numbers increase between
rt3 and rt4 then we have a hand full patches to look at.
 
> BTW, x86_64 also showed slightly higher numbers for signaltest for the
> v5.10-rc1-rt1 release. These are gone now.

just like that?

Sebastian


Re: [ANNOUNCE] v5.10-rc2-rt4

2020-11-04 Thread Daniel Wagner
On Tue, Nov 03, 2020 at 08:57:31PM +0100, Sebastian Andrzej Siewior wrote:
> I'm pleased to announce the v5.10-rc2-rt4 patch set.

All tests on passed in my lab. On arm64 and arm I saw slightly higher
max latency values for signaltest and sigwaittest. Usually they are
below 200us but currently I see up to 350us.

BTW, x86_64 also showed slightly higher numbers for signaltest for the
v5.10-rc1-rt1 release. These are gone now.