Re: ✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo Achim!

On Thu, 26 Jan 2017 19:30:12 +0100
Achim Gratz  wrote:

> If you had a non-quantized signal to start with you wouldn't need to
> add noise in the first place.

In the digital world all measurements are quantized.  All you can do is
make the quant really small.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgpptQF72XPKG.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Achim Gratz]

Kurt Roeckx writes:
> On Tue, Jan 24, 2017 at 10:38:50PM +0100, Achim Gratz wrote:
>> sys_fuzz (which whitens the quantization noise on the clock reading)
>> doesn't make a difference or you might not.  But presumably Dave Mills
>> didn't put it in there just because he was trying to add useless code.
>
> So it seems to me that this is the wrong order, you first add the
> noise and then do the quantization, not the other way around.

If you had a non-quantized signal to start with you wouldn't need to add
noise in the first place.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo All!

To follow up on my own mistake...

On Wed, 25 Jan 2017 16:52:58 -0800
"Gary E. Miller"  wrote:

> Xeon: fuzz=0.000202, tick=0.000202
> skylake:  fuzz=0.000109, tick=0.000109
> RasPi:fuzz=0.003,tick=0.003
> RasPi2:   fuzz=0.001823, tick=0.001823
> RasPi3:   fuzz=0.000937, tick=0.000937
> mac mini: fuzz=0.0005,   tick=0.001

My eye slipped. sys_tick is greater than sys_fuzz.

> Anyone got the reverse case?

Does anyone ever see sys_fuzz > sys_tick?

Hmm, measure_tick_fuzz() in ntpd/ntp_proto.c ensures sys_fuzz <= sys_tick.

So that is answered.  Can't happen.

systime.c has this odd code:

if (sys_tick > sys_fuzz)
quant = sys_tick;
else
quant = 1e-6;

Not sure why that last line is not quant = sys_fuzz.  I distrust
magic constants.  Not sure it even matters.  Prolly for legacy
adjtime() that only support micro Seconds?  Not gonna worry about it
if it is just old Macs.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgp501VIeGTJJ.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo Kurt!

On Wed, 25 Jan 2017 00:01:43 +0100
Kurt Roeckx  wrote:

> On Tue, Jan 24, 2017 at 10:38:50PM +0100, Achim Gratz wrote:
> > sys_fuzz (which whitens the quantization noise on the clock reading)
> > doesn't make a difference or you might not.  But presumably Dave
> > Mills didn't put it in there just because he was trying to add
> > useless code.  
> 
> So it seems to me that this is the wrong order, you first add the
> noise and then do the quantization, not the other way around.

The quantization noise is already in the time read from the system..

clock_gettime() has a resolution of one nano Second, but depending on the
host, it increments in steps that are a lot bigger.  ntpd tries
to determine that minimum increment and store it in sys_tick.

Annoying there was no way to read that variable that I could find.

Now there is a way, just pushed to git head, new option:

ntpq -c 'rv 0 tick'

Results are in milli Seconds.  Here is what I get on a Xeon:

spidey ntpsec # ntpq -c 'rv 0 tick fuzz'
fuzz=0.000202, tick=0.000202

Tick seems to equal fuzz.  About 202 nano Seconds.

Table time:

Xeon:   fuzz=0.000202, tick=0.000202
skylake:fuzz=0.000109, tick=0.000109
RasPi:  fuzz=0.003,tick=0.003
RasPi2: fuzz=0.001823, tick=0.001823
RasPi3: fuzz=0.000937, tick=0.000937
mac mini:   fuzz=0.0005,   tick=0.001

Pretty much sys_fuzz == sys_tick.  Except that mac!

Looks like ntpd is built to handle a lot of complexity that no longer exists
in my test platforms.  Except that mac...

looks like sys_tick can be less than sys_fuzz.  Anyone got the reverse
case?

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgp1aJMOFVhqQ.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Kurt Roeckx]

On Tue, Jan 24, 2017 at 10:38:50PM +0100, Achim Gratz wrote:
> sys_fuzz (which whitens the quantization noise on the clock reading)
> doesn't make a difference or you might not.  But presumably Dave Mills
> didn't put it in there just because he was trying to add useless code.

So it seems to me that this is the wrong order, you first add the
noise and then do the quantization, not the other way around.


Kurt

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo Achim!

On Wed, 25 Jan 2017 18:36:13 +0100
Achim Gratz  wrote:

> Gary E. Miller writes:
> > Dr. Mills did not put it there.  This was added in 2011 by Dave
> > hart. See my previous email that probably crossed by yours in
> > flight.  
> 
> The way I read things, Dave Hart's patches were to ascertain
> monotonicity on top of the already existing fuzzing.

Yes.  I was saying that sys_fuzz was new then, and the algorithm
had a serious complexity upgrade then.  So what happened before 2011
not very relevant anymore.

> > It proves that sys_fuzz does nothing on a RasPi.  At least
> > undetectable over 4 days.  I give you it proves nothing about other
> > systems.  
> 
> The fuzz we are talking about is added to outgoing timestamps, so when
> it did something it wouldn't be discernible on the server, but on the
> client receiving that timestamp.  In other words, you are looking at
> the wrong end of the data (I think).

No, the fuzz we are talking about is added to everything.  One
place the fuzz is added is in libntp/systime.c in normalize_time()
which is called from get_systime().  Everyplace ntpd gets a time it
now uses get_systime().  Except two oddball drivers: jjy and oncore.



> > I'll once again raise the challenge to someone else to prove it
> > actually does something, somewhere.  No point involving deities or
> > more bike shedding, give me data!  
> 
> You can make such requests ad inifinitum, but it won't help.  Unless
> you know what exactly the problem was that fuzzing solves, you won't
> be able to pick out that set of data that shows you whether it is or
> is not still needed.

And by your own argument, you are not adding anything either, until you
find a real problem space.

> > Now knowing more about bug 2037 I'm also confident that the
> > characteristics of the RasPi clock where not those being fixed in
> > bug 2037.  
> 
> That bug was about monotonicity of the timestamps, not the fuzzig
> itself, IIRC.

Yes, but it caused the creation of sys_fuzz and the current fuzzing
scheme.

> Again, you need to know what problem this code was solving (before the
> patches from Dave Hart, which were solving a different and probably
> more specific problem most likely) in order to know if the
> assumptions that made it necessary or useful still hold.

I thought bug 2037 , and the bug 2037 fixed, explained that nicely.  Do
you know something else not there?

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgpc79gDUysaL.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Achim Gratz]

Gary E. Miller writes:
> Dr. Mills did not put it there.  This was added in 2011 by Dave hart.
> See my previous email that probably crossed by yours in flight.

The way I read things, Dave Hart's patches were to ascertain
monotonicity on top of the already existing fuzzing.

> It proves that sys_fuzz does nothing on a RasPi.  At least undetectable
> over 4 days.  I give you it proves nothing about other systems.

The fuzz we are talking about is added to outgoing timestamps, so when
it did something it wouldn't be discernible on the server, but on the
client receiving that timestamp.  In other words, you are looking at the
wrong end of the data (I think).

> I'll once again raise the challenge to someone else to prove it
> actually does something, somewhere.  No point involving deities or
> more bike shedding, give me data!

You can make such requests ad inifinitum, but it won't help.  Unless you
know what exactly the problem was that fuzzing solves, you won't be able
to pick out that set of data that shows you whether it is or is not
still needed.

> Now knowing more about bug 2037 I'm also confident that the
> characteristics of the RasPi clock where not those being fixed in bug
> 2037.

That bug was about monotonicity of the timestamps, not the fuzzig
itself, IIRC.

Again, you need to know what problem this code was solving (before the
patches from Dave Hart, which were solving a different and probably more
specific problem most likely) in order to know if the assumptions that
made it necessary or useful still hold.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo Eric!

On Tue, 24 Jan 2017 16:42:42 -0500
"Eric S. Raymond"  wrote:

> Achim Gratz :
> > You might have enough other noise in any particular system so the
> > sys_fuzz (which whitens the quantization noise on the clock reading)
> > doesn't make a difference or you might not.  But presumably Dave
> > Mills didn't put it in there just because he was trying to add
> > useless code.  
> 
> That's what bothers me, that I don't know what we was thinking.

Grr, you motivated me to actually research this.
 
Dr. Mills was in the discussion, but it is not exactly primordial.
sys_fuzz was added in 2011:

commit b24b87c768265963557e1abb9a859490f929c057
Author: Dave Hart 
Date:   Sat Dec 17 02:30:42 2011 -0500

[Bug 2037] Fuzzed non-interpolated clock may decrease.


Bug 2037:

http://bugs.ntp.org/show_bug.cgi?id=2037

It seems most of the bug is about making the clock monotonic and
increasing, not about anything else.

The main point of the fuzz seems to be:

"This strategy preserves the nonce value of the low-order bits of
timestamps on the wire,"

To my mind, as long as the random fuzzing is below sys_fuzz, and it
is, then for my purposes it is too small to worry about.  OTOH, if
it gets in the way of something in the future I'd toss it out.  Just
preserving the nonce aspect.

> Mark, is it time to try to open contact with Mills yet?

I'm not sure this is the issue, but why wait?  He should know about
the current lives of his spawn.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgpgBoJdxjDdw.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo Eric!

On Tue, 24 Jan 2017 15:44:17 -0500
"Eric S. Raymond" <e...@thyrsus.com> wrote:

> Gary E. Miller <g...@rellim.com>:
> > Otherwise sys_fuzz * ntp_random() is more poinltess noise in the
> > code we can rip out.  
> 
> I would consider this good news, especially since that code is in the
> critical section.

Worth keeping an eye out for, but not a priority.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgpLCQd1IWPlD.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

Re: ✘sys_fuzz * ntp_random()

[Eric S. Raymond]

Gary E. Miller <g...@rellim.com>:
> Otherwise sys_fuzz * ntp_random() is more poinltess noise in the code we
> can rip out.

I would consider this good news, especially since that code is in the critical
section.
-- 
http://www.catb.org/~esr/;>Eric S. Raymond


signature.asc
Description: PGP signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel

✘sys_fuzz * ntp_random()

[Gary E. Miller]

Yo All!

Last week we had a discussion on sys_fuzz and the value of adding
random noise to some measurements.  The code defi2nes sys_fuzz asL

"* The sys_fuzz variable measures the minimum time to read the system
 * clock, regardless of its precision."

Rondomness of half the sys_fuzz is then added to some values, like this:

fuzz = ntp_random() * 2. / FRAC * sys_fuzz

Makes no sense to me.  Adding randomness helps when you have hysteresis,
stiction, friction, lash and some other things, but none of those apply
to NTP.

So I took two identical RasPi's and forced sys_fuzz to zero on one of
them.  Four days later I can see no difference in the results from
those two.

Can anyone find a case were adding the randomness can be proved to help?

We already have all the guesses we need, anyone have any data?

Otherwise sys_fuzz * ntp_random() is more poinltess noise in the code we
can rip out.

RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com  Tel:+1 541 382 8588

Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin


pgpvZRNVXTDI0.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel