I hear you and completely understand how reliable that setup is, but evaluating the facts: Host to VM time sync was definitely disabled - always - standard practice Host BIOS clock is set to UTC - again standard practice Host OS (VMWARE v5.5U2 - Dell cut) also syncs to the same internet ntp servers as PDC - purely for it's own purposes BDC and all members always sync from PDC, including AD-integrated Synology NAS
Someone raised a point however regarding VMTools maybe screwing up We have recently upgraded all sites/vm's to stable vmtools v10.3.5 I have not seen any other sites with time issues since the upgrade. I'll investigate this path, had not considered this, keeping my mind open However, the fact still remains: *I still find it strange that since changing the PDC to sync to sync ONLY from 3.au.pool.ntp.org <http://3.au.pool.ntp.org/>, that the problem went away* *This one point still bothers me* Like most of you, I am an old war dog in this industry - 41 years now.. Not competing, just saying.. I have learned not to jump to conclusions too often I am open to any plausible explanations Kindly, ROY ADAMS* | *P 07 3040 5010 | Web: http://www.racs.com.au/ | Wiki: https://ex.racs.com.au:444/ | eMail: mailto:r...@racs.com.au <r...@racs.com.au> Please never upgrade to the latest Windows 10 - You don’t need the hassle, and I don’t need the work. If you think it's expensive to hire a professional to do the job, wait until you hire an amateur - Red Adair. Life is a journey through a series of adventures.. Live them, love them, hate them, but never give up on your dreams, desires, and goals. On Fri, 1 Feb 2019 at 18:14, Chris Barnes <chris.p.bar...@gmail.com> wrote: > Yup, I've been running a stratum 2 server for the AU region and I was > going to say exactly this. If there was a pool member behaving so badly for > this long then the NTP Pool project would have pulled that server. > > For a server to be included in the pool it has to maintain a score above > 10, maximum score is 20. As soon as a problem is detected it's score starts > dropping very quickly. The worst score is -100 and it's a long slow > recovery to go from -100 back up to +10 to be included in the pool again. > > > On Fri., 1 Feb. 2019, 1:32 pm Rob Thomas, <xro...@gmail.com> wrote: > >> As someone who runs a few Stratum 1 and 2 servers, pool.ntp.org checks >> my servers every 15 minutes or so, and if they're more than 50ms out, >> they're de-prioritized, until they're removed. It's all completely >> automatic. >> >> Example: https://www.ntppool.org/scores/203.4.240.10 >> >> I think this is more an issue on your end, honestly. >> >> --Rob >> >> >> On Fri, 1 Feb 2019 at 12:17, Roy Adams <r...@racs.com.au> wrote: >> >>> Single PDC site... Hit the PDC every 30 mins or so - forward, back, >>> forward, back etc >>> then of course started changing time on all domain members shortly after >>> - each time. >>> Problem is it was not instant for all members and the AD-integrated >>> Synology NAS.... >>> Backups broke, complaints from 20 users randomly every 30 mins until >>> isolated. >>> >>> Cluster 3.au.pool.ntp.org has been fine since 3.39pm Brisbane time >>> yesterday. >>> I'll just ignore the 0. for now and wait for someone @ ntp.org to spot >>> it I think >>> >>> Could just be specific to win2008r2Sp1 - who knows. >>> AU Admins, you have been warned :) >>> Enuf of my time wasted on it >>> >>> Thanks for all the comments and PM's >>> I have actually picked up a lot of tips from you all - many thanks >>> >>> >>> Kindly, >>> >>> ROY ADAMS* | *P 07 3040 5010 | Web: http://www.racs.com.au/ | Wiki: >>> https://ex.racs.com.au:444/ | eMail: mailto:r...@racs.com.au >>> <r...@racs.com.au> >>> Please never upgrade to the latest Windows 10 - You don’t need the >>> hassle, and I don’t need the work. >>> If you think it's expensive to hire a professional to do the job, wait >>> until you hire an amateur - Red Adair. >>> Life is a journey through a series of adventures.. Live them, love them, >>> hate them, but never give up on your dreams, desires, and goals. >>> >>> >>> >>> On Fri, 1 Feb 2019 at 00:54, Joseph Daly <jd...@arrowstreetcapital.com> >>> wrote: >>> >>>> One small thing and this is probably just the wording of the email. >>>> >>>> >>>> >>>> >>>> >>>> *I always use the below config for domain controllers:* >>>> >>>> *sc config W32Time start= auto & net start W32Time* >>>> >>>> *w32tm /config /manualpeerlist:"0.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=> >>>> 2.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__2.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PRGGsc1Vf_jVqorVPZnTpw7JvXoa49lzKAVZTXF0gUs&e=> >>>> 3.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>" >>>> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait* >>>> >>>> >>>> >>>> All DCs or just your PDC emulator? Usually the other DCs sync from the >>>> PDC emulator. >>>> >>>> >>>> >>>> >>>> >>>> *From:* Roy Adams <r...@racs.com.au> >>>> *Sent:* Thursday, January 31, 2019 1:33 AM >>>> *To:* Patch Management Mailing List < >>>> patchmanagem...@listserv.patchmanagement.org> >>>> *Subject:* Re:[patchmanagement] [AusNOG] ntp server issues today... >>>> strange one... clutching at straws but just in case >>>> >>>> >>>> >>>> Thanks for the PM's offering ideas >>>> >>>> I am tempted to set it back to 0. to debug the offending ntp pool IP, >>>> but it was breaking all the backups among other things due to AD sync being >>>> more than 5 mins out. >>>> >>>> >>>> >>>> I always use the below config for domain controllers: >>>> >>>> sc config W32Time start= auto & net start W32Time >>>> >>>> w32tm /config /manualpeerlist:"0.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=> >>>> 2.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__2.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PRGGsc1Vf_jVqorVPZnTpw7JvXoa49lzKAVZTXF0gUs&e=> >>>> 3.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>" >>>> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait >>>> >>>> >>>> >>>> One of the replies noted that linux sanity checks by getting ntp time >>>> from 4 servers - I wish MS were that smart. >>>> >>>> Clearly MS are not using all the configured servers, I suspect they are >>>> purely for failover like a DNS client. >>>> >>>> >>>> >>>> I have just changed this site to: >>>> >>>> w32tm /config /manualpeerlist:"3.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__3.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=B3Zd-DIP0O9MfiOMufSpJ84RI0He4iXoMtlIv_CdbB0&e=>" >>>> /syncfromflags:manual /reliable:yes /update & w32tm /resync /nowait >>>> >>>> So far so good.. still stable >>>> >>>> >>>> >>>> All Domain members of course sync to the DC >>>> >>>> I am not seeing this on any other sites.. all sites are cookie cutter >>>> for me >>>> >>>> >>>> >>>> >>>> >>>> event logs confirm ONLY the change... not the server IP :( >>>> >>>> The system time has changed to 2019-01-31T01:47:11.254922100Z from >>>> 2019-01-31T02:18:29.514800000Z. >>>> >>>> The system time has changed to 2019-01-31T01:47:11.254000000Z from >>>> 2019-01-31T01:47:11.254922100Z. >>>> >>>> The system time has changed to 2019-01-31T03:43:51.747000000Z from >>>> 2019-01-31T03:12:32.312621000Z. >>>> >>>> The system time has changed to 2019-01-31T03:36:17.703840400Z from >>>> 2019-01-31T04:07:36.105000000Z. >>>> >>>> The system time has changed to 2019-01-31T03:36:17.703000000Z from >>>> 2019-01-31T03:36:17.703840400Z. >>>> >>>> The system time has changed to 2019-01-31T05:41:23.075000000Z from >>>> 2019-01-31T05:10:04.617935900Z. >>>> >>>> The system time has changed to 2019-01-31T06:01:12.107000000Z from >>>> 2019-01-31T06:01:12.107000000Z. >>>> >>>> The system time has changed to 2019-01-31T05:30:09.707385800Z from >>>> 2019-01-31T06:01:28.112628100Z. >>>> >>>> The system time has changed to 2019-01-31T05:30:09.707000000Z from >>>> 2019-01-31T05:30:09.707385800Z. >>>> >>>> The system time has changed to 2019-01-31T05:39:51.770000000Z from >>>> 2019-01-31T05:39:51.770276000Z. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Kindly, >>>> >>>> >>>> >>>> ROY ADAMS* | *P 07 3040 5010 | Web: http://www.racs.com.au/ >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.racs.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=ZRKXJDH4ciRXiwcDhbLIwFHIvgqzrytOtvGja-WyEso&e=> >>>> | Wiki: https://ex.racs.com.au:444/ >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__ex.racs.com.au-3A444_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=DsV_izPAHTAwk-h02V5W_v5P98BrMy1Ul7Kol0HTTmc&e=> >>>> | >>>> eMail: mailto:r...@racs.com.au <r...@racs.com.au> >>>> >>>> Please never upgrade to the latest Windows 10 - You don’t need the >>>> hassle, and I don’t need the work. >>>> If you think it's expensive to hire a professional to do the job, wait >>>> until you hire an amateur - Red Adair. >>>> >>>> Life is a journey through a series of adventures.. Live them, love >>>> them, hate them, but never give up on your dreams, desires, and goals. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Thu, 31 Jan 2019 at 16:13, Nick Stallman <n...@agentpoint.com> >>>> wrote: >>>> >>>> Do you know which server specifically? The ntp pools hand out random >>>> NTP server IPs, it's not a fixed server. >>>> >>>> I'm not a Windows server admin, but this would likely be why Linux >>>> connects to ~4 NTP servers so it can disregard dodgy servers. >>>> >>>> On 31/1/19 5:09 pm, Roy Adams wrote: >>>> >>>> Hi All, I have a domain controller *seemingly* receiving bad time info >>>> today from 0.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=U499vWTuQrHOdlAPlmDHrA-rgZbLYU7PaXpE2Kd48eM&e=> >>>> >>>> Issuing this confirmed the time was flapping forward 30 mins, then 30 >>>> mins later back to normal: >>>> >>>> w32tm /query /status >>>> >>>> It confirmed the above ntp server as the server that supplied the bad >>>> (then good, then bad, then good etc) time >>>> >>>> I have now changed the DC to pull instead from 3.au.pool.ntp.org >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__au.pool.ntp.org&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=vF1MfzyyrPVr436Gt8h40rxV1qMJ68GEs4Gu9MqWD-k&e=> >>>> . >>>> >>>> 1 hour has passed and so far so good. >>>> >>>> >>>> >>>> Cannot say I have ever seen anything like this... >>>> >>>> It's only occurring on one site on a windows2008r2sp1 domain controller. >>>> >>>> The DC in turn relays this updated time to all domain members of course. >>>> >>>> Anyone else had time issues on any sites today in Aus? >>>> >>>> >>>> >>>> >>>> >>>> Kindly, >>>> >>>> >>>> >>>> ROY ADAMS* | *P 07 3040 5010 | Web: http://www.racs.com.au/ >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.racs.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=ZRKXJDH4ciRXiwcDhbLIwFHIvgqzrytOtvGja-WyEso&e=> >>>> | Wiki: https://ex.racs.com.au:444/ >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__ex.racs.com.au-3A444_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=DsV_izPAHTAwk-h02V5W_v5P98BrMy1Ul7Kol0HTTmc&e=> >>>> | >>>> eMail: mailto:r...@racs.com.au <r...@racs.com.au> >>>> >>>> Please never upgrade to the latest Windows 10 - You don’t need the >>>> hassle, and I don’t need the work. >>>> If you think it's expensive to hire a professional to do the job, wait >>>> until you hire an amateur - Red Adair. >>>> >>>> Life is a journey through a series of adventures.. Live them, love >>>> them, hate them, but never give up on your dreams, desires, and goals. >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> >>>> AusNOG mailing list >>>> >>>> AusNOG@lists.ausnog.net >>>> >>>> http://lists.ausnog.net/mailman/listinfo/ausnog >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ausnog.net_mailman_listinfo_ausnog&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=FeOndl6vwYUzDu74O11bqvYM6U3YN9aOiq9rAI3KKvw&e=> >>>> >>>> -- >>>> >>>> *Nick Stallman* >>>> >>>> *Technical Director* >>>> >>>> n...@agentpoint.com >>>> >>>> 02 8039 6820 <0280396820> >>>> >>>> www.agentpoint.com.au >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=> >>>> >>>> >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=> >>>> >>>> >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=> >>>> >>>> Level 3, 100 Harris Street, Pyrmont NSW 2009 >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=> >>>> >>>> >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=> >>>> >>>> >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.agentpoint.com.au_&d=DwMFaQ&c=CI4-aYV4fOpCXKvKTF2ntOZntzAcNsg3IKFmdux4fpc&r=ZO0rs3RFhaNJvARM24Iy1y9-0qhOC5NjmySIqVusRM4&m=P1VICLtys7ExKCosSmwoCaMSdezGSjskIV3o0GAwJZs&s=PN87Mw56MVmSkWBGo0FRbjQ1ii1ML1UB8SR17fFvyIQ&e=> >>>> >>>> Arrowstreet Capital,LP-DISCLAIMER: >>>> ============================== >>>> This email message and its attachments are being sent by Arrowstreet >>>> Capital, Limited Partnership and are confidential and proprietary. If you >>>> are not the intended recipient, please notify us immediately by replying to >>>> this message and destroy all copies of this message and any attachments. >>>> Thank you. >>>> >>> _______________________________________________ >>> AusNOG mailing list >>> AusNOG@lists.ausnog.net >>> http://lists.ausnog.net/mailman/listinfo/ausnog >>> >> _______________________________________________ >> AusNOG mailing list >> AusNOG@lists.ausnog.net >> http://lists.ausnog.net/mailman/listinfo/ausnog >> > _______________________________________________ > AusNOG mailing list > AusNOG@lists.ausnog.net > http://lists.ausnog.net/mailman/listinfo/ausnog >
_______________________________________________ AusNOG mailing list AusNOG@lists.ausnog.net http://lists.ausnog.net/mailman/listinfo/ausnog