Re: Clock Skew Due to CPU Frequency Management? (was Re: Dual-core system will not create NTP peers)
On Tue, Feb 19, 2008 at 07:37:32AM -0600, Moshe Yudkowsky wrote: I had a very kind private communication from a list member. Let me reformulate my question. On the AMD64 dual-core platform, the system clock has far too much drift even though the hardware clock seems well behaved. I don't seem to be experiencing the double-speed clock problem that I see references to on the net as of a few years ago, but I am experiencing some other odd behavior. The most likely culprit, as far as I can tell from research, is some interaction between the hardware's ability to adjust CPU clock speed to save power/control temperature (which then influences the system clock's theory of what a CPU tick means in terms of actual time passing) with the dual-core mode. But that seems rather wacky. I find it hard to believe that a kernel release wouldn't work on dual core CPUs. But if the problem really is that's the case, I would like to know what boot-time settings I can use to test this hypothesis. For that matter I'd like to know why the kernel itself isn't picking up on this theoretical problem -- ISTR reading about an enhanced CPU clock mode that could solve this problem, but I can't find references to it or how to build a kernel or modules from source that would include this capability. I've spent a substantial amount of time on this problem, and I can't come to any conclusion. I have to wonder if it's a problem with NTP itself, but if so it's a problem that only manifests itself on the AMD64 box, although I continue to fiddle with NTP settings. Someone here at work was having trouble getting NTP working on their system a few weeks ago. Disabling spread spectrum in the BIOS solved the problem. Of course as far as I know disabling spread spectrum violate emmisions certifications in europe, but I could be wrong about that since I don't actually live there (anymore at least). Spread spectrum clocking really screws with the timings NTP depends on. -- Len Sorensen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Clock Skew Due to CPU Frequency Management? (was Re: Dual-core system will not create NTP peers)
Len, Thanks for writing with the spread-spectrum idea. I don't know if you saw the final email in this thread, but it turns out that (as far as I can tell) adjtimexconfig made a bad estimate when it was installed, and put a TICK value of 9750 in /etc/defaults/adjtimex, which of course works out to a clock that's always 2.5% slow. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe Poor Mexico -- so far from G-d and so near the United States. -- President Porfirio Diaz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
On Wed, 2008-02-20 at 22:16 -0600, Moshe Yudkowsky wrote: I have discovered the source of the problem: the file /etc/default/adjtimex had a TICK value of 9750. That's 2.5% less than the standard value of 1 and accounts exactly for the clock skew I observed. Oh, I wish I'd remembered that file. I had that problem way back in 2003 when I switched from an old K7 system to a dual Opteron. I used the same hard drive and everything. I didn't reinstall. The old value was very different from the new value. Something to keep in mind when upgrading a motherboard. -s -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
Stephen Olander-Waters wrote: On Wed, 2008-02-20 at 22:16 -0600, Moshe Yudkowsky wrote: I have discovered the source of the problem: the file /etc/default/adjtimex had a TICK value of 9750. That's 2.5% less than the standard value of 1 and accounts exactly for the clock skew I observed. Oh, I wish I'd remembered that file. I had that problem way back in 2003 when I switched from an old K7 system to a dual Opteron. I used the same hard drive and everything. I didn't reinstall. The old value was very different from the new value. Something to keep in mind when upgrading a motherboard. In this case, I built a system from scratch, using 1's and 0's made out of flint, and it's a mystery where that TICK value came from. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe It's not just a mistake -- it's an adventure! -- Babs Bunny -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
C Wakefield wrote: Moshe: I'll bet it's the bios. Which version are you running? Asus had an update on november 20th, 2007 to 1302. I just double-checked: I was running 1302. I've reflashed the BIOS anyway. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe It's a sobering thought, for example, to realize that by the time he was my age, Mozart had been dead for two years. -- Tom Lehrer -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
Keep the replies on the list. On Wed, Feb 20, 2008 at 04:54:43AM -0600, Moshe Yudkowsky wrote: Running the system overnight, and doing an ntpdate every five minutes, I notice that I very consistently take a +7.5s offset -- seems like exactly 2.5% to the limit of my ability to measure. So the system clock runs 2.5% slow.. what could that mean? Don't know. Why do you run ntpdate? The ntp docs say that ntpdate will be removed soon since ntp has a -q option that works better. Why not just run ntp continuously? -- Do you have more than one computer turned on locally? What type of internet connection do you have? If the internet is only inermittant, you could set up a local ntp server (if its clock is more reliable). Or, if you happen to have a GPS, you could set that up as a time source. I don't understand why the rtc can be (or is) a problem on the amd64 boards; its a major pain in the butt. My old 486 has a very accurate clock, being off only about 1 second per week. Whenever I have it on my network (as in, not on-the-shelf spare), I use it as a time server when my dial-up isn't dialed-up. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers'
Moshe, What part of keep the replies on the list didn't you understand. Reply to the list. On Wed, Feb 20, 2008 at 08:04:36AM -0600, Moshe Yudkowsky wrote: Douglas A. Tutty wrote: Keep the replies on the list. On Wed, Feb 20, 2008 at 04:54:43AM -0600, Moshe Yudkowsky wrote: Running the system overnight, and doing an ntpdate every five minutes, I notice that I very consistently take a +7.5s offset -- seems like exactly 2.5% to the limit of my ability to measure. Why do you run ntpdate? The ntp docs say that ntpdate will be removed soon since ntp has a -q option that works better. ntpd has the -g option to over-ride the 1000s limitation and the -q option to allow the time to be set initially in one big jump (just like ntpdate) instead of skewing for several days. Why not just run ntp continuously? I am running ntpdate because ntp will not sync when the clock skew is so large. ntpdate provided me with debugging information about what my problem is. Try ntp. Read the ntp docs package. They say after a suitable period of mourning, the ntpdate program may be retired. After all, the ntp program with iburst, -q and -g or -x if necessary gives a better algorithm, error correction, and debugging than ntpdate. Do you have more than one computer turned on locally? What type of internet connection do you have? If the internet is only inermittant, you could set up a local ntp server (if its clock is more reliable). Or, if you happen to have a GPS, you could set that up as a time source. I have a lot of other computers, including an old AMD AthlonXP that keeps sync without any problems. Well, then with the right ntp options, the time on your amd64 dual cpu box will always be in sync with your other boxes. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers'
Douglas A. Tutty wrote: Moshe, What part of keep the replies on the list didn't you understand. Reply to the list. That'd be the part where I misread the statement. My apologies. Try ntp. Read the ntp docs package. They say after a suitable period of mourning, the ntpdate program may be retired. After all, the ntp program with iburst, -q and -g or -x if necessary gives a better algorithm, error correction, and debugging than ntpdate. ntpd simply never forms a peer relationship even though I've tried many different options. The -q option mimics ntpdate. The -x option increases the slew threshold; what I need is more rapid, not less rapid, synchronization. If I'm losing 2.5% of my clock, a slew rate of 0.5 ms/s isn't going to keep up. I've tried similar settings to -x through the tinker option in the configuration file, but I will give it another shot. The -g option does work; it's a Debian default configuration option (/etc/defaults/ntp). I get a step adjustment the first time I start ntpd. Unfortunately, after that initial step, I never get any subsequent synchronization and ntpd never forms a peer relationship. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe Live fast. Die young. Leave no documentation. -- Programmer's Creed, as told by Mike Bakula -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
I have discovered the source of the problem: the file /etc/default/adjtimex had a TICK value of 9750. That's 2.5% less than the standard value of 1 and accounts exactly for the clock skew I observed. I adjusted the tick back to a reasonable value (see below) and my clock problems appear to be solved. I have no idea how the file obtained this value. I suspect that when adjtimex was installed, the package used /usr/sbin/adjtimexconfig and that it somehow obtained completely incorrect values. To adjust tick and frequency for maximum accuracy, I highly recommend Larry Dolittle's ntpclient package at http://doolittle.icarus.com/ntpclient/. In particular, the HOWTO file not only explains the debug output of his tools but also gave me invaluable information that I could use to resolve my own problem. Math notes to future readers, since tick can be confusing: the tick is how many microseconds to add to the current clock reading. Ticks occur 100 times per what the system thinks is a second, and if the value of a tick is set to 1, then it will add 100 * 10,000 = 1,000,000 microseconds to the system clock each second. Because the value of the tick in my system was set to 9750, each second I added only 975,000 microseconds to the system clock, for a deficit of 2.5%. See the adjustimex man page as well as Larry's HOWTO page. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe Only a fool fights in a burning house. -- Ancient Klingon Proverb -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
On Mon, Feb 18, 2008 at 11:18:16PM -0600, Moshe Yudkowsky wrote: I've built a dual-core AMD64 system, but I cannot get ntpd to work. I'm using debian unstable, 2.6.24-1-amd64, which is apparently really x86_64. ntpdate sever_name will reset the system clock. ntp, when started, will also set the system clock if needed. ntpd then never synchronizes to another ntp as a peer, which means there are no further adjustments to the time. The system clock falls behind very quickly, on the order of minutes per hour. This is not a firewall problem: all the other Linux, XP and MacOS boxes synchronize without any problems. Please note that I have removed all 'restrict' statements from /etc/ntp.conf -- and I can't even sync to a local server that easily syncs over the 'net. I've tried a few (well, perhaps all) the recipes I could find on the web; for example, booting with no_timer_check and/or noapic. Unfortunately, if there's a documentation repository that explains these flags I have yet to find it -- and this does not fix the problem. QUESTION: Other than running ntpdate as a cron job every five minutes, is there some way to get ntpd to work correctly? For example, is there a kernel module that needs to be rebuilt with different flags? can you run ntpdc -p this is what I get remote local st poll reach delay offsetdisp === *damogran.nerdbo 192.168.11.102 1024 377 0.03798 0.000461 0.10612 =202.174.42.5192.168.11.103 1024 377 0.09567 0.005525 0.09669 =core.narx.net 192.168.11.102 1024 377 0.06985 0.004480 0.11482 =203.80.162.195 192.168.11.102 1024 377 0.01834 0.003523 0.09671 -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe From stupidity there is always something to be learned, but it's always the same thing: don't be stupid. -- Robert M. Adams -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- I know the human being and fish can coexist peacefully. - George W. Bush 09/29/2000 Saginaw, MI signature.asc Description: Digital signature
Clock Skew Due to CPU Frequency Management? (was Re: Dual-core system will not create NTP peers)
I had a very kind private communication from a list member. Let me reformulate my question. On the AMD64 dual-core platform, the system clock has far too much drift even though the hardware clock seems well behaved. I don't seem to be experiencing the double-speed clock problem that I see references to on the net as of a few years ago, but I am experiencing some other odd behavior. The most likely culprit, as far as I can tell from research, is some interaction between the hardware's ability to adjust CPU clock speed to save power/control temperature (which then influences the system clock's theory of what a CPU tick means in terms of actual time passing) with the dual-core mode. But that seems rather wacky. I find it hard to believe that a kernel release wouldn't work on dual core CPUs. But if the problem really is that's the case, I would like to know what boot-time settings I can use to test this hypothesis. For that matter I'd like to know why the kernel itself isn't picking up on this theoretical problem -- ISTR reading about an enhanced CPU clock mode that could solve this problem, but I can't find references to it or how to build a kernel or modules from source that would include this capability. I've spent a substantial amount of time on this problem, and I can't come to any conclusion. I have to wonder if it's a problem with NTP itself, but if so it's a problem that only manifests itself on the AMD64 box, although I continue to fiddle with NTP settings. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe If, after hearing my songs, just one human being is inspired to say something nasty to a friend, it will all have been worthwhile. -- Tom Lehrer -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
C Wakefield wrote: What's the hardware? Could be a bios bug. Chris, The motherboard is an Asus M2N-SLI Deluxe. The CPU is an AMD64 6500 x2, 2.8 GHz clock. I'm not aware of any issues with this board. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe Dynamite resolves a lot of narrative complications. -- Ryoki Inoue -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
Alex Samad wrote: ntpdc -p After leaving this run for a couple of hours, I get this set of information, which is actually rather horrifying even to my untrained eye: # ntpq -c as ; ntpdc -c peers -c sysinfo ind assID status conf reach auth condition last_event cnt === 1 37850 9024 yes yes nonereject reachable 2 2 37851 9024 yes yes nonereject reachable 2 3 37852 9024 yes yes nonereject reachable 2 4 37853 9024 yes yes nonereject reachable 2 5 37854 9024 yes yes nonereject reachable 2 6 37855 9024 yes yes nonereject reachable 2 remote local st poll reach delay offsetdisp === =mirror.web-ster 172.28.54.1622 64 377 0.06209 243.41326 0.04036 =patbox.patrickd 172.28.54.1622 64 377 0.06819 237.67744 0.05783 =druid.storyinme 172.28.54.1622 64 277 0.03696 239.99789 0.04701 =ntp-1.gw.uiuc.e 172.28.54.1622 64 377 0.01086 242.87614 0.04802 =ntp-2.gw.uiuc.e 172.28.54.1622 64 377 0.01221 244.02863 0.03761 =64.73.32.134172.28.54.1622 64 377 0.01271 240.80815 0.05812 system peer: 0.0.0.0 system peer mode: unspec leap indicator: 11 stratum: 16 precision:-20 root distance:0.0 s root dispersion: 0.14473 s reference ID: [83.84.69.80] reference time: . Thu, Feb 7 2036 0:28:16.000 system flags: auth monitor ntp kernel stats jitter: 0.332382 s stability:0.000 ppm broadcastdelay: 0.003998 s authdelay:0.00 s I'm attempting the use between four and seven ntp servers method out of the US pool of ntp.org, as well as a couple of local machines, one of which is marked prefer. I'm using the Debian stable NTP, not that it's helping. By way of contrast, here's what this same dump looks like on a single-processor AMD AthlonXP on the same local network: # ntpq -c as ; ntpdc -c peers -c sysinfo ind assID status conf reach auth condition last_event cnt === 1 41784 9614 yes yes none sys.peer reachable 1 remote local st poll reach delay offsetdisp === *ntp-1.gw.uiuc.e 172.28.54.1602 256 377 0.01498 -0.007047 0.11313 system peer: ntp-1.gw.uiuc.edu system peer mode: client leap indicator: 00 stratum: 3 precision:-20 root distance:0.07011 s root dispersion: 0.05965 s reference ID: [130.126.24.24] reference time: cb657ba1.e7d162e6 Tue, Feb 19 2008 10:00:33.905 system flags: auth monitor ntp kernel stats jitter: 0.004196 s stability:0.000 ppm broadcastdelay: 0.003998 s authdelay:0.00 s In other words, it's not the firewall. And since it does sync up once it's not a problem with the conf file. It's something else, which I haven't been able to find in well over 10 days... -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe The aim of the middle manager is slightly more honest than the nobility, since former desire only to impress the oppressors while not depressing those who avoid oppression -- Larry Tomko (ca 1992) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
What's the hardware? Could be a bios bug. Chris W. OnFebruary 18, 2008 09:18:16 pm Moshe Yudkowsky wrote: I've built a dual-core AMD64 system, but I cannot get ntpd to work. I'm using debian unstable, 2.6.24-1-amd64, which is apparently really x86_64. ntpdate sever_name will reset the system clock. ntp, when started, will also set the system clock if needed. ntpd then never synchronizes to another ntp as a peer, which means there are no further adjustments to the time. The system clock falls behind very quickly, on the order of minutes per hour. This is not a firewall problem: all the other Linux, XP and MacOS boxes synchronize without any problems. Please note that I have removed all 'restrict' statements from /etc/ntp.conf -- and I can't even sync to a local server that easily syncs over the 'net. I've tried a few (well, perhaps all) the recipes I could find on the web; for example, booting with no_timer_check and/or noapic. Unfortunately, if there's a documentation repository that explains these flags I have yet to find it -- and this does not fix the problem. QUESTION: Other than running ntpdate as a cron job every five minutes, is there some way to get ntpd to work correctly? For example, is there a kernel module that needs to be rebuilt with different flags? -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe From stupidity there is always something to be learned, but it's always the same thing: don't be stupid. -- Robert M. Adams -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
On Tue, Feb 19, 2008 at 10:37:17AM -0600, Moshe Yudkowsky wrote: C Wakefield wrote: What's the hardware? Could be a bios bug. Chris, The motherboard is an Asus M2N-SLI Deluxe. The CPU is an AMD64 6500 x2, 2.8 GHz clock. I'm not aware of any issues with this board. I'm running on that MB now, AMD64 Athlon 3800+ single CPU. ntp is running just fine on Debian Etch amd64. Doug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
OnFebruary 19, 2008 08:58:23 pm Moshe Yudkowsky wrote: Douglas A. Tutty wrote: On Tue, Feb 19, 2008 at 10:37:17AM -0600, Moshe Yudkowsky wrote: The motherboard is an Asus M2N-SLI Deluxe. The CPU is an AMD64 6500 x2, 2.8 GHz clock. I'm not aware of any issues with this board. That was an error on my part: it's a 5600, not 6500. I'm running on that MB now, AMD64 Athlon 3800+ single CPU. ntp is running just fine on Debian Etch amd64. Excellent data point, thanks! If I'm reading Google correctly, the problem lies with the dual CPU. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe There is something fundamentally wrong with a country [USSR] where the citizens want to buy your underwear. -- Paul Thereaux Moshe: I'll bet it's the bios. Which version are you running? Asus had an update on november 20th, 2007 to 1302. source: http://support.asus.com/download/download.aspx?SLanguage=en-us Chris W. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Dual-core system will not create NTP peers
Douglas A. Tutty wrote: On Tue, Feb 19, 2008 at 10:37:17AM -0600, Moshe Yudkowsky wrote: The motherboard is an Asus M2N-SLI Deluxe. The CPU is an AMD64 6500 x2, 2.8 GHz clock. I'm not aware of any issues with this board. That was an error on my part: it's a 5600, not 6500. I'm running on that MB now, AMD64 Athlon 3800+ single CPU. ntp is running just fine on Debian Etch amd64. Excellent data point, thanks! If I'm reading Google correctly, the problem lies with the dual CPU. -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe There is something fundamentally wrong with a country [USSR] where the citizens want to buy your underwear. -- Paul Thereaux -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Dual-core system will not create NTP peers
I've built a dual-core AMD64 system, but I cannot get ntpd to work. I'm using debian unstable, 2.6.24-1-amd64, which is apparently really x86_64. ntpdate sever_name will reset the system clock. ntp, when started, will also set the system clock if needed. ntpd then never synchronizes to another ntp as a peer, which means there are no further adjustments to the time. The system clock falls behind very quickly, on the order of minutes per hour. This is not a firewall problem: all the other Linux, XP and MacOS boxes synchronize without any problems. Please note that I have removed all 'restrict' statements from /etc/ntp.conf -- and I can't even sync to a local server that easily syncs over the 'net. I've tried a few (well, perhaps all) the recipes I could find on the web; for example, booting with no_timer_check and/or noapic. Unfortunately, if there's a documentation repository that explains these flags I have yet to find it -- and this does not fix the problem. QUESTION: Other than running ntpdate as a cron job every five minutes, is there some way to get ntpd to work correctly? For example, is there a kernel module that needs to be rebuilt with different flags? -- Moshe Yudkowsky * [EMAIL PROTECTED] * www.pobox.com/~moshe From stupidity there is always something to be learned, but it's always the same thing: don't be stupid. -- Robert M. Adams -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
