Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
On 07/02/18 01:16, Bob kb8tq wrote: >> MSF disciplined oscillator?! I don't trust these receivers to any better >> than about the 20ms mark, so such a disciplined oscillator would have quite >> a long integration time! > > Once upon a time, that *was* how people did disciplined oscillators. Part of > the > answer to “how?” is that their target accuracies were not as tight as what we > now > think of as normal. I was under the impression the Radio Four carrier (198kHz ex 200kHz) was the old "frequency standard of choice" in the UK, prior to GPS. I don't recall anyone using MSF for much other than "what time is it, to maybe the nearest second?". Though I do recall John Becker of EPE magazine did a nifty little millennium countdown clock using MSF as a base. -- Phil. phil...@philpem.me.uk http://www.philpem.me.uk/ ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
> Hal Murray wrote: > If the two signals are not encoded identically, there should be an > interesting signal when one of the transmitters is off and the other is on. > Has anybody looked for that sort of pattern? > Is there a map of the dead spots? Any time-nuts live in/near one? Yes. Here is a screenshot of roughly equal strength JJY and WWVB as received in New Zealand around 10 PM local time on a KiwiSDR. Due to the timing reversal of the pulses from each station this results in solid carrier during the data bit times (no matter the bit combination: 00, 01, 10, 11) as the signals are added. The marker pulses every 10 seconds give a 0.6 sec gap when both stations are at reduced-carrier. And the double marker at the minute boundary gives a 1.6 sec gap with a 0.4 sec pulse in the middle. I thought this was sort of amusing. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Challenge accepted. This graph is probably not too useful, for the simple reason that when the receiver is spitting out mostly noise, the averages are going to be massively affected. [image: Inline images 1] On 7 February 2018 at 01:03, Hal Murray wrote: > > deirdre@gmail.com said: > > MSF disciplined oscillator?! I don't trust these receivers to any better > > than about the 20ms mark, so such a disciplined oscillator would have > quite > > a long integration time! > > It would be interesting to see if you can find any pattern in your > histogram > plots. Say, time of day. > > What happens if you average over 10 or 100 samples? > > > -- > These are my opinions. I hate spam. > > > > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/ > mailman/listinfo/time-nuts > and follow the instructions there. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message , "Deirdre O'Byrne" writes: >I've updated my paper, which now contains the attached graph. (I did a >linear regression analysis to see what the correction for the receivers >should be, and I applied receiver 2's correction to both receivers to >generate this graph). Yes, these cheap "clock-receivers" vary a lot and they are usually also very temperature sensitive. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
No doubt! But I'm trying to remain as inexpensive as possible. That it might be possie to get 5ms (300 carrier periods!) from an off-the-shelf consumer-grade component not designed for accuracy is pretty cool IMO. On 7 Feb 2018 14:31, "Bob kb8tq" wrote: Hi Back in the era of VLF disciplined oscillators, carrier phase was the preferred approach. Getting that to work with 100% AM modulation took some effort …. Bob > On Feb 7, 2018, at 2:13 AM, Poul-Henning Kamp wrote: > > > In message , "Deirdre O'Byrne" writes: > >> MSF disciplined oscillator?! I don't trust these receivers to any better >> than about the 20ms mark, so such a disciplined oscillator would have quite >> a long integration time! > > It's actually more complicated and better than that. > > The low-pass filter dominates, so the falling flank at second N > depends on the pulsewidth at second N-1. > > I can't remember the numbers I got when I "sorted" DCF77 pulses depending > on the previous pulse being short or long, but it was a fair bit better > than 20ms. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/ mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Hi Back in the era of VLF disciplined oscillators, carrier phase was the preferred approach. Getting that to work with 100% AM modulation took some effort …. Bob > On Feb 7, 2018, at 2:13 AM, Poul-Henning Kamp wrote: > > > In message > , > "Deirdre O'Byrne" writes: > >> MSF disciplined oscillator?! I don't trust these receivers to any better >> than about the 20ms mark, so such a disciplined oscillator would have quite >> a long integration time! > > It's actually more complicated and better than that. > > The low-pass filter dominates, so the falling flank at second N > depends on the pulsewidth at second N-1. > > I can't remember the numbers I got when I "sorted" DCF77 pulses depending > on the previous pulse being short or long, but it was a fair bit better > than 20ms. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message , "Deirdre O'Byrne" writes: >MSF disciplined oscillator?! I don't trust these receivers to any better >than about the 20ms mark, so such a disciplined oscillator would have quite >a long integration time! It's actually more complicated and better than that. The low-pass filter dominates, so the falling flank at second N depends on the pulsewidth at second N-1. I can't remember the numbers I got when I "sorted" DCF77 pulses depending on the previous pulse being short or long, but it was a fair bit better than 20ms. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
> > > Splitting the MSF received signal into 100ms chunks, all of the seconds > apart from the start-of-minute marker are of the form 0AB1. Using "x" > to represent a 100ms chunk whose value could not be determined, I notice > that many of the received seconds were of the form "0AB1x111" or "0AB11x11" > etc - i.e. there was only one 100ms chunk within the second whose value > could not be reliably determined and whose value was non-critical. > Ooops - apologies - this referred to another algorithm I investigated, and makes no sense in the context of this algorithm. Still I believe a blame algorithm would recover a lot of lost data. It would require a shift register 1,200 wide - with each slot representing the value during a 100ms interval from achosen edge. Thanks. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
> > > Did MSF finally go to a BPSK signal format? I heard they were considering > that. > I don't know, though I've ordered a SDR which should be able to receive the raw signal. I don't see anything about it in the documentation, though. Regards, Deirdre. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Hi > On Feb 6, 2018, at 3:48 PM, Deirdre O'Byrne wrote: > > Tom, > > Thanks for the feedback! > > On 6 February 2018 at 20:29, Tom Van Baak wrote: > >> >> 2) Not all decoding errors are equal. Since this is a time code instead of >> arbitrary binary data you can use the internal structure of the data to >> your benefit. >> > > As I said to Poul-Henning, that is the next level of error detection, which > also has application in error correcting some of the "almost-right" signals. > > >> 3) A side-effect of your data set is that you can track performance of the >> oscillator inside the logic analyzer: convert the 700k GPS timestamps into >> interval, find and replace the 4 glitch lines with 2 lines of 1.000150, and >> then use Stable32 or TimeLab to plot. I used a 10 minute running average to >> reduce the 50 us quantization noise. Note the mean frequency of your >> timebase is 152 ppm low. > > > I made it out to be 152.2ppm, which is kinda disappointing. But the signal > analyser cost very little, and you get what you pay for. > > I have not yet wrapped my head around how to create ADEV plots, so thanks > for your work on that - it's interesting to see that (presumed) initial > thermal effect. > > >> Over 8 days this results in a cumulative sampling error of 105 seconds. If >> your decoding algorithms are relative instead of absolute this won't be a >> problem. OTOH, you may be able to use your decoding process to detect this >> drift and then compensate for it in software. You have the beginnings of a >> MSF-Disciplined-Oscillator project. >> > > MSF disciplined oscillator?! I don't trust these receivers to any better > than about the 20ms mark, so such a disciplined oscillator would have quite > a long integration time! Once upon a time, that *was* how people did disciplined oscillators. Part of the answer to “how?” is that their target accuracies were not as tight as what we now think of as normal. Bob > > Thanks again. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
deirdre@gmail.com said: > MSF disciplined oscillator?! I don't trust these receivers to any better > than about the 20ms mark, so such a disciplined oscillator would have quite > a long integration time! It would be interesting to see if you can find any pattern in your histogram plots. Say, time of day. What happens if you average over 10 or 100 samples? -- These are my opinions. I hate spam. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Tom, Thanks for the feedback! On 6 February 2018 at 20:29, Tom Van Baak wrote: > > 2) Not all decoding errors are equal. Since this is a time code instead of > arbitrary binary data you can use the internal structure of the data to > your benefit. > As I said to Poul-Henning, that is the next level of error detection, which also has application in error correcting some of the "almost-right" signals. > 3) A side-effect of your data set is that you can track performance of the > oscillator inside the logic analyzer: convert the 700k GPS timestamps into > interval, find and replace the 4 glitch lines with 2 lines of 1.000150, and > then use Stable32 or TimeLab to plot. I used a 10 minute running average to > reduce the 50 us quantization noise. Note the mean frequency of your > timebase is 152 ppm low. I made it out to be 152.2ppm, which is kinda disappointing. But the signal analyser cost very little, and you get what you pay for. I have not yet wrapped my head around how to create ADEV plots, so thanks for your work on that - it's interesting to see that (presumed) initial thermal effect. > Over 8 days this results in a cumulative sampling error of 105 seconds. If > your decoding algorithms are relative instead of absolute this won't be a > problem. OTOH, you may be able to use your decoding process to detect this > drift and then compensate for it in software. You have the beginnings of a > MSF-Disciplined-Oscillator project. > MSF disciplined oscillator?! I don't trust these receivers to any better than about the 20ms mark, so such a disciplined oscillator would have quite a long integration time! Thanks again. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message <20180206225742.67030406...@ip-64-139-1-69.sjc.megapath.net>, Hal Murray writes: >> Since MSF *is* on 60 KHz, you do indeed get dead spots. > >If the two signals are not encoded identically, there should be an >interesting signal when one of the transmitters is off and the other is on. >Has anybody looked for that sort of pattern? I have seen signs of that in my data in the shape of phase-shifts, and that sort of made me concentrate on DCF & LORAN. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Hi If you look at the papers for the “new” WWVB format, there are plots of where the MSF issues are likely to be the greatest. Since both signals are phase and amplitude shifted by propagation effects, you will not get stationary nulls. You simply get zones where the reception is tough. Bob > On Feb 6, 2018, at 5:57 PM, Hal Murray wrote: > >> Since MSF *is* on 60 KHz, you do indeed get dead spots. > > If the two signals are not encoded identically, there should be an > interesting signal when one of the transmitters is off and the other is on. > Has anybody looked for that sort of pattern? > > Is there a map of the dead spots? Any time-nuts live in/near one? > > > > -- > These are my opinions. I hate spam. > > > > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
> Since MSF *is* on 60 KHz, you do indeed get dead spots. If the two signals are not encoded identically, there should be an interesting signal when one of the transmitters is off and the other is on. Has anybody looked for that sort of pattern? Is there a map of the dead spots? Any time-nuts live in/near one? -- These are my opinions. I hate spam. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Hi > On Feb 6, 2018, at 4:19 PM, Poul-Henning Kamp wrote: > > > In message > , > "Deirdre O'Byrne" writes: > >> With a blame algorithm in place it should be possible to recover these >> signals. > > Yes, easily. > > At distance MSF is significantly harder to receive than DCF77. > > One of the reasons is that USA also operates two 60kHz transmitters > also very precisely on frequency, so there are areas of the world > where the three signals cancel and areas where they reinforce > each other. I believe we only have one transmitter on the air at 60 KHz in the US. The Japanese do indeed operate multiple transmitters on the same frequency ( 40 KHz). There have been a number of proposals to set up a second US transmitter. The last time I noticed them beating the drum for one, it was going to be at 40 KHz rather than 60 KHz. The proposal pretty much died yet again …. Since MSF *is* on 60 KHz, you do indeed get dead spots. There probably is an interesting plot of locations that have issues with both the 40 KHz and 60 KHz transmissions due to simply being in the wrong place. Bob > > I tried to model this many years ago, but I don't trust the result, > somebody with better HF-propagation chops than me should look at it. > > In addition to that problem, switch-mode designers seems to just > *love* 60 kHz, and at least here in Denmark there is a lot more > "hash" around 60 kHz than 77.5 kHz. > > Finally, the modulation scheme of MSF is a bit on the overengineered > side, which makes pulse discrimination needlessly hard - as you have > also found out. > > The big advantage of the blame algorithm is that since it is so > tolerant of missing pulses, you can be throw everything away which > isn't 100% clearcut. > > If you look at the top of the dcf77.c file, you can see how I did > that for DCF77, but the complex modulation of MSF needs a much > more complex state engine there. > > Finally, many of the small "clock-receivers", like the one you use, > are optimised for battery-life and therefore they use very resonant > filters, often crystal-filters, and heavy low-pass after demodulation, > and that trows away a LOT of information which would be useful to > have to discriminate the pulses. > > If you go for the SDR approach, you will have much more information > available, and can use much more well-behaved filters to detect the > pulses, and one added advantage of carrier-tracking is that the > power-modulation is carrier-synchronous, which makes them much > easier to spot. > > So really: Get yourself an 1MSPS ADC chip and go that route instead. > > (In theory, certain modern sound-cards should be usable for this if > you can rip out their low-pass filters. Havn't tried.) > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Deirdre, Great discussion on my favorite topic. I am the guy on the other side of the lake that curses MSFs interference with WWVB. I did indeed cheat by using the GPS time and 1 second tick to recreate the WWVB timecode bits to remove the psk shifts in the received signal here on the east coast. This allowed phase tracking receivers to correctly work again. Did MSF finally go to a BPSK signal format? I heard they were considering that. regards Paul WB8TSL On Tue, Feb 6, 2018 at 4:26 PM, Poul-Henning Kamp wrote: > > In message , Bob kb8tq > writes: > > >If you want to get even more “nutty", look at the “seed” that you likely > already have > >for the computation. In this day and age, you probably know what day / > month / year it is. > > So, some of us think of that as cheating :-) > > >Since you might not (say) know the hour, you have a +/- 1 day sort of > tolerance on that. It rolls > >into month and year in some cases. The seed adds complexity, but probably > makes > >things more robust. > > I tried it, and it gave surprisingly little benefit. > > Unless very fast initial aquisition is your goal (why?!) you get a > more robust result by not "cheating", since in real life at some > point your RTC chip will contain bogus values. > > If you go the SDR route and decode DCF77 and MSF (and 162kHz France, > WWV/B, the japanese signal at 40kHz and the russian at 200/3 kHz for > that matter) in parallel, it is perfectly fair to expect them all > to have the same date (modulus timezones). > > And yes, I would really *love* to se a colaborative project that > produced an "all-world VLF timecode SDR-receiver"... > > >One cute thing is that this stuff is (in general) not very compute > intensive. If data past the > >minute tick is being looked at, you probably can afford to run multiple > parallel solutions (even > >on a < $5 MCU). > > The NTPns ran on a Soekris4501 and I was never able to measure a > difference in power having the DCF77 blame code running or not. > > After all, it's only sixty trival patterns to match once a second... > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/ > mailman/listinfo/time-nuts > and follow the instructions there. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message , Bob kb8tq writes: >If you want to get even more “nutty", look at the “seed” that you likely >already have >for the computation. In this day and age, you probably know what day / month / >year it is. So, some of us think of that as cheating :-) >Since you might not (say) know the hour, you have a +/- 1 day sort of >tolerance on that. It rolls >into month and year in some cases. The seed adds complexity, but probably makes >things more robust. I tried it, and it gave surprisingly little benefit. Unless very fast initial aquisition is your goal (why?!) you get a more robust result by not "cheating", since in real life at some point your RTC chip will contain bogus values. If you go the SDR route and decode DCF77 and MSF (and 162kHz France, WWV/B, the japanese signal at 40kHz and the russian at 200/3 kHz for that matter) in parallel, it is perfectly fair to expect them all to have the same date (modulus timezones). And yes, I would really *love* to se a colaborative project that produced an "all-world VLF timecode SDR-receiver"... >One cute thing is that this stuff is (in general) not very compute intensive. >If data past the >minute tick is being looked at, you probably can afford to run multiple >parallel solutions (even >on a < $5 MCU). The NTPns ran on a Soekris4501 and I was never able to measure a difference in power having the DCF77 blame code running or not. After all, it's only sixty trival patterns to match once a second... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message , "Deirdre O'Byrne" writes: >With a blame algorithm in place it should be possible to recover these signals. Yes, easily. At distance MSF is significantly harder to receive than DCF77. One of the reasons is that USA also operates two 60kHz transmitters also very precisely on frequency, so there are areas of the world where the three signals cancel and areas where they reinforce each other. I tried to model this many years ago, but I don't trust the result, somebody with better HF-propagation chops than me should look at it. In addition to that problem, switch-mode designers seems to just *love* 60 kHz, and at least here in Denmark there is a lot more "hash" around 60 kHz than 77.5 kHz. Finally, the modulation scheme of MSF is a bit on the overengineered side, which makes pulse discrimination needlessly hard - as you have also found out. The big advantage of the blame algorithm is that since it is so tolerant of missing pulses, you can be throw everything away which isn't 100% clearcut. If you look at the top of the dcf77.c file, you can see how I did that for DCF77, but the complex modulation of MSF needs a much more complex state engine there. Finally, many of the small "clock-receivers", like the one you use, are optimised for battery-life and therefore they use very resonant filters, often crystal-filters, and heavy low-pass after demodulation, and that trows away a LOT of information which would be useful to have to discriminate the pulses. If you go for the SDR approach, you will have much more information available, and can use much more well-behaved filters to detect the pulses, and one added advantage of carrier-tracking is that the power-modulation is carrier-synchronous, which makes them much easier to spot. So really: Get yourself an 1MSPS ADC chip and go that route instead. (In theory, certain modern sound-cards should be usable for this if you can rip out their low-pass filters. Havn't tried.) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Hi If you want to get even more “nutty", look at the “seed” that you likely already have for the computation. In this day and age, you probably know what day / month / year it is. Since you might not (say) know the hour, you have a +/- 1 day sort of tolerance on that. It rolls into month and year in some cases. The seed adds complexity, but probably makes things more robust. If the purpose is to “always be right” then retaining a seed probably improves things. The flip side is (of course) “what if I’ve been lied to?”. That applies with or without a seed. Heading off into a situation where you never (re)lock could be one result. How long do you go before you decide to try something else? One cute thing is that this stuff is (in general) not very compute intensive. If data past the minute tick is being looked at, you probably can afford to run multiple parallel solutions (even on a < $5 MCU). Lots of zigs and zags …. Bob > On Feb 6, 2018, at 2:37 PM, Poul-Henning Kamp wrote: > > > In message > > , "Deirdre O'Byrne" writes: > >> I've been trying to see if I could design a decoding algorithm that >> would be more noise-tolerant than the algorithms I've seen out in >> the wild. > > You can: I baptised it "the blame algoritm". > > The trick is not to try to accept pulses as valid but to try to > throw out pulses which are impossible. > > Imagine you have a 120 second long shift-register, and you feed > your received pulses into it. > > Then try brute force, for every one of the newest 60 positions if > that can be the start of a minute or not, by testing all the > constraints you can think of, and there are surprisingly many. > > Some are obvious, the bits encoding the hour cannot contain "39", > but that is a remarkable weak filter that seldom kicks in. > > A much stronger filter is that the bits encoding the hour must be > the same as in the previous minute *unless* minutes were 59 in the > previous minute *and* zero in this minute. > > If you count it up, that is a strong and very peculiar relationship > on all the hour-bits and all the minutes-bits and if even one single > of them are wrong, you can definitively discard that theory for the > start of the minute. > > A similar thing holds for the date bits, the time in the > previous minute must be 23:59:59 and in this 00:00:00 for > there to be any difference between the dates, and even > then, only a small number of possible changes in the date > bits are valid. > > If you look in http://phk.freebsd.dk/phkrel/NTPns.20080902.tgz > you will find a file called dcf77_blame.c with my code, > here is a couple of the simpler tests: > >/* LSB of minutes must be different from previous minute */ >j = ip->shiftprev[(offset + 21) % 60]; >if (j * ip->shiftreg[(offset + 21) % 60] > 0) >FAIL((why, " 0")); > >/* > * If the LSB of minutes was '1' in previous minute > * the next higher bit must have changed, if it was > * a '0' it must not. > */ >if (j * >ip->shiftreg[(offset + 22) % 60] * >ip->shiftprev[(offset + 22) % 60] > 0) >FAIL((why, " 1")); > > When using this algorithm, missing pulses is almost a > non-issue up to around 40% of them missing, and even > in an enviroment like that, it is not uncommon to see > the algorithm lock on to the minute in about 34 seconds > and know the full time in less than 3 minutes. > > If you make your pulse width discriminator *really* selective, > which you might as well, you can "blacklist" disproved > minute positions for the next many minutes as the > risk of a '1' and '0' being confused is close to zero. > > That will get you minute lock, even with 70%-90% missing > pulses, in a matter of minutes if you use a longer > shift register. > > I did a parallel prototype for MSF, but I didn't need it > so I never completed it, not sure I have it around any > more. > > I should really write an article about that code... > > Poul-Henning > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
That's the next level of error detection. Unfortunately I notice with MSF and these cheap receivers it can be difficult to determine what the binary values being transmitted during the second were, which was my motivation for the analysis I did. I notice that there is one significant difference between DCF and MSF - the former does not entirely turn off the carrier during the second, whereas the latter does. I suspect that is the reason why with MSF receivers it can so often be difficult to determine what the binary values transmitted were, as the receiver AGC circuitry possibly has a difficult time keeping up with drastically changing received power levels, which subsequently give rise to noise in the received signal. Splitting the MSF received signal into 100ms chunks, all of the seconds apart from the start-of-minute marker are of the form 0AB1. Using "x" to represent a 100ms chunk whose value could not be determined, I notice that many of the received seconds were of the form "0AB1x111" or "0AB11x11" etc - i.e. there was only one 100ms chunk within the second whose value could not be reliably determined and whose value was non-critical. With a blame algorithm in place it should be possible to recover these signals. On 6 February 2018 at 19:37, Poul-Henning Kamp wrote: > > In message gmail.com> > , "Deirdre O'Byrne" writes: > > >I've been trying to see if I could design a decoding algorithm that > >would be more noise-tolerant than the algorithms I've seen out in > >the wild. > > You can: I baptised it "the blame algoritm". > > The trick is not to try to accept pulses as valid but to try to > throw out pulses which are impossible. > > Imagine you have a 120 second long shift-register, and you feed > your received pulses into it. > > Then try brute force, for every one of the newest 60 positions if > that can be the start of a minute or not, by testing all the > constraints you can think of, and there are surprisingly many. > > Some are obvious, the bits encoding the hour cannot contain "39", > but that is a remarkable weak filter that seldom kicks in. > > A much stronger filter is that the bits encoding the hour must be > the same as in the previous minute *unless* minutes were 59 in the > previous minute *and* zero in this minute. > > If you count it up, that is a strong and very peculiar relationship > on all the hour-bits and all the minutes-bits and if even one single > of them are wrong, you can definitively discard that theory for the > start of the minute. > > A similar thing holds for the date bits, the time in the > previous minute must be 23:59:59 and in this 00:00:00 for > there to be any difference between the dates, and even > then, only a small number of possible changes in the date > bits are valid. > > If you look in http://phk.freebsd.dk/phkrel/NTPns.20080902.tgz > you will find a file called dcf77_blame.c with my code, > here is a couple of the simpler tests: > > /* LSB of minutes must be different from previous minute */ > j = ip->shiftprev[(offset + 21) % 60]; > if (j * ip->shiftreg[(offset + 21) % 60] > 0) > FAIL((why, " 0")); > > /* > * If the LSB of minutes was '1' in previous minute > * the next higher bit must have changed, if it was > * a '0' it must not. > */ > if (j * > ip->shiftreg[(offset + 22) % 60] * > ip->shiftprev[(offset + 22) % 60] > 0) > FAIL((why, " 1")); > > When using this algorithm, missing pulses is almost a > non-issue up to around 40% of them missing, and even > in an enviroment like that, it is not uncommon to see > the algorithm lock on to the minute in about 34 seconds > and know the full time in less than 3 minutes. > > If you make your pulse width discriminator *really* selective, > which you might as well, you can "blacklist" disproved > minute positions for the next many minutes as the > risk of a '1' and '0' being confused is close to zero. > > That will get you minute lock, even with 70%-90% missing > pulses, in a matter of minutes if you use a longer > shift register. > > I did a parallel prototype for MSF, but I didn't need it > so I never completed it, not sure I have it around any > more. > > I should really write an article about that code... > > Poul-Henning > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message , Ruslan Nabioullin writes: >On Tue, Feb 6, 2018 at 2:37 PM, Poul-Henning Kamp wrote: >> If you look in http://phk.freebsd.dk/phkrel/NTPns.20080902.tgz >> you will find a file called dcf77_blame.c with my code, >How difficult would it be to complete these modules and integrate them >with the rest of NTP, as NTP decoder modules? So instead of an AM HF >receiver, the setup for these signals would be: Well, NTPns *is* a NTP server program, but a very special one. If you really want to receive VLF for NTP purposes, you should hook a 1M sample/sec ADC to your antenna and implement _all_ the demodulation in software. (The BeagleBoneBlack with its two PRU units would be close to a perfect platform for this) The more stable your sample frequency your ADC has, the easier the software will be to write. You can carrier-track DCF77, 162kHz France, MSF and Loran-C in one go, by simply averaging the raw antennasignal in a long enough circular buffer, then multiply by a suitable SIN/COS complex signal. For signals on kHz spacing (50, 162, 198 any other you care for) you can make a single one millisecond (ie: 1000 samples) circular buffer, and extract all the phases from that buffer, by multiplying by suitable SIN/COS and averaging the result to DC. For 77.5 kHz you will need a 2 millisecond buffer. If you want to recover the phase modulation on 162 kHz you can use the phase from the first buffer as your reference. (See: http://phk.freebsd.dk/loran-c/SW) If you want to recover the PRNG phase on DCF77 you will need to do the full early/prompt/late correlation, but that is also pretty cheap as you can pre-generate the sequence to compare with at compile time. Alternatively make a full-second buffer and average into that with polarity suitably swapped based on AM pulse width, and do the early/prompt/later about once a minute or so in high-level software. Loran-C is the same basic story, I did that on a 32bit ARM some years ago. (See: http://phk.freebsd.dk/AducLoran/) So much code to write and so little time... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
rnabioul...@gmail.com said: > How difficult would it be to complete these modules and integrate them with > the rest of NTP, as NTP decoder modules? So instead of an AM HF receiver, > the setup for these signals would be: The simple way to do that is to use the shared-memory interface. No changes to ntpd required. -- These are my opinions. I hate spam. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
I used this cheap (but good for the price!) signal analyser - http://www.ebay.ie/itm/Hobby-Components-UK-USB-24M-8CH-24MHz-Logic-Analyser/161309221423?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649 I used sigrok to do the recording - https://www.sigrok.org/ Let me know if you collect some data - I'd be interested in seeing your results! On 6 February 2018 at 19:36, Adrian Godwin wrote: > Hi Deirdre, > > I'd like to repeat your measurement at a different location (eastern > england). > > What did you use to capture the data and write it as a vcd file ? > > -adrian > > > On Tue, Feb 6, 2018 at 6:40 PM, Deirdre O'Byrne > wrote: > > > OK so it's not the microsecond or nanosecond stuff that much of this list > > is about, but I've been running an experiment for the past few days > > gathering data on how well (or otherwise) a pair of cheap EM2S radio > > receiver modules receive the MSF radio signal. I've been trying to see > if I > > could design a decoding algorithm that would be more noise-tolerant than > > the algorithms I've seen out in the wild. > > > > Details are on my github, and the results are presented in a paper - > > > > https://github.com/deirdreobyrne/MSF-EM2S/blob/master/paper.pdf > > ___ > > time-nuts mailing list -- time-nuts@febo.com > > To unsubscribe, go to https://www.febo.com/cgi-bin/ > > mailman/listinfo/time-nuts > > and follow the instructions there. > > > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/ > mailman/listinfo/time-nuts > and follow the instructions there. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
On Tue, Feb 6, 2018 at 2:37 PM, Poul-Henning Kamp wrote: > If you look in http://phk.freebsd.dk/phkrel/NTPns.20080902.tgz > you will find a file called dcf77_blame.c with my code, > here is a couple of the simpler tests: > > /* LSB of minutes must be different from previous minute */ > j = ip->shiftprev[(offset + 21) % 60]; > if (j * ip->shiftreg[(offset + 21) % 60] > 0) > FAIL((why, " 0")); > > /* > * If the LSB of minutes was '1' in previous minute > * the next higher bit must have changed, if it was > * a '0' it must not. > */ > if (j * > ip->shiftreg[(offset + 22) % 60] * > ip->shiftprev[(offset + 22) % 60] > 0) > FAIL((why, " 1")); > ... > I did a parallel prototype for MSF, but I didn't need it > so I never completed it, not sure I have it around any > more. > > I should really write an article about that code... How difficult would it be to complete these modules and integrate them with the rest of NTP, as NTP decoder modules? So instead of an AM HF receiver, the setup for these signals would be: LF receiver set to CW demodulation set to appropriate parameters -> NTP timekeeping system sound card One of my organization's projects consists of robust public time transfer via NTP over the Internet, based on a combination of various on-site standards (rackmount OCXO, rubidium, and/or cesium standards) and external signals (incl. WWV and CHU, using preamplifier -> preselector -> analog parametric demodulator -> sound card, controlled by ionospheric prediction daemon software on GNU/Linux via GPIB), the nodes being geographically dispersed throughout the US and Canada. It's probable that I will end up relocating to Western Europe (coincidentally, the Republic of Ireland!) in the moderate future, and therefore it would be nice if these LF or HF signals were to be supported, for use as fallbacks to the standard GNSS sources (each site typically will have one military and one industrial civilian rackmount GPS receiver). -Ruslan -- Ruslan Nabioullin Wittgenstein Laboratories rnabioul...@gmail.com (508) 523-8535 50 Louise Dr. Hollis, NH 03049 ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
In message , "Deirdre O'Byrne" writes: >I've been trying to see if I could design a decoding algorithm that >would be more noise-tolerant than the algorithms I've seen out in >the wild. You can: I baptised it "the blame algoritm". The trick is not to try to accept pulses as valid but to try to throw out pulses which are impossible. Imagine you have a 120 second long shift-register, and you feed your received pulses into it. Then try brute force, for every one of the newest 60 positions if that can be the start of a minute or not, by testing all the constraints you can think of, and there are surprisingly many. Some are obvious, the bits encoding the hour cannot contain "39", but that is a remarkable weak filter that seldom kicks in. A much stronger filter is that the bits encoding the hour must be the same as in the previous minute *unless* minutes were 59 in the previous minute *and* zero in this minute. If you count it up, that is a strong and very peculiar relationship on all the hour-bits and all the minutes-bits and if even one single of them are wrong, you can definitively discard that theory for the start of the minute. A similar thing holds for the date bits, the time in the previous minute must be 23:59:59 and in this 00:00:00 for there to be any difference between the dates, and even then, only a small number of possible changes in the date bits are valid. If you look in http://phk.freebsd.dk/phkrel/NTPns.20080902.tgz you will find a file called dcf77_blame.c with my code, here is a couple of the simpler tests: /* LSB of minutes must be different from previous minute */ j = ip->shiftprev[(offset + 21) % 60]; if (j * ip->shiftreg[(offset + 21) % 60] > 0) FAIL((why, " 0")); /* * If the LSB of minutes was '1' in previous minute * the next higher bit must have changed, if it was * a '0' it must not. */ if (j * ip->shiftreg[(offset + 22) % 60] * ip->shiftprev[(offset + 22) % 60] > 0) FAIL((why, " 1")); When using this algorithm, missing pulses is almost a non-issue up to around 40% of them missing, and even in an enviroment like that, it is not uncommon to see the algorithm lock on to the minute in about 34 seconds and know the full time in less than 3 minutes. If you make your pulse width discriminator *really* selective, which you might as well, you can "blacklist" disproved minute positions for the next many minutes as the risk of a '1' and '0' being confused is close to zero. That will get you minute lock, even with 70%-90% missing pulses, in a matter of minutes if you use a longer shift register. I did a parallel prototype for MSF, but I didn't need it so I never completed it, not sure I have it around any more. I should really write an article about that code... Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] Receiving the MSF time signal on cheap radio modules
Hi Deirdre, I'd like to repeat your measurement at a different location (eastern england). What did you use to capture the data and write it as a vcd file ? -adrian On Tue, Feb 6, 2018 at 6:40 PM, Deirdre O'Byrne wrote: > OK so it's not the microsecond or nanosecond stuff that much of this list > is about, but I've been running an experiment for the past few days > gathering data on how well (or otherwise) a pair of cheap EM2S radio > receiver modules receive the MSF radio signal. I've been trying to see if I > could design a decoding algorithm that would be more noise-tolerant than > the algorithms I've seen out in the wild. > > Details are on my github, and the results are presented in a paper - > > https://github.com/deirdreobyrne/MSF-EM2S/blob/master/paper.pdf > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/ > mailman/listinfo/time-nuts > and follow the instructions there. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
[time-nuts] Receiving the MSF time signal on cheap radio modules
OK so it's not the microsecond or nanosecond stuff that much of this list is about, but I've been running an experiment for the past few days gathering data on how well (or otherwise) a pair of cheap EM2S radio receiver modules receive the MSF radio signal. I've been trying to see if I could design a decoding algorithm that would be more noise-tolerant than the algorithms I've seen out in the wild. Details are on my github, and the results are presented in a paper - https://github.com/deirdreobyrne/MSF-EM2S/blob/master/paper.pdf ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
