Brad is mostly correct below. The instructions on the Server web content explicitly state that the content must not be spidered. Nobody wants that data to be scraped and used long-term. It's not really about the load that would place on the server. The blacklist plugin we run detects spidering attempts and will add the offending IP to the blacklist.
We have enough trouble with people hardcoding IPs in their ntp.conf files. We've been discussing how often NTP should re-query DNS names too. If you are looking for a pool of servers, it's far better to use the "pool" directive and point at the appropriate pool, instead of trying to do the job yourself by going thru the Servers entries. Having said that, there are people who do not want to add their servers to the Pool, but they are perfectly happy to add their servers to the Servers web on support.ntp.org . It's clear we need to see how to make all of this better. For example, what can be done to automate the checking of server entries in the Servers web, so we can "do something" about machines that are listed there but are no longer functioning? H Brad Knowles writes: > > --===============5561077762517319938== > Content-Type: multipart/signed; boundary="Apple-Mail=_B18C9047-F716-4EF2-BAFE > -8760411A197A"; protocol="application/pgp-signature"; micalg=pgp-sha256 > > > --Apple-Mail=_B18C9047-F716-4EF2-BAFE-8760411A197A > Content-Transfer-Encoding: quoted-printable > Content-Type: text/plain; > charset=utf-8 > > On Aug 24, 2016, at 4:17 PM, [email protected] wrote: > > > In setting up a couple new pool servers recently, I noted there were a = > lot of dead servers on the stratum two list at = > support.ntp.org/bin/view/Servers/StratumTwoTimeServers > > It was a bit time consuming, and frustrating, to try and find good = > servers (and I'm in the US, where there are plenty of servers). Lots of = > back and forth, testing them manually with ping and 'ntpdate -q=E2=80=99 > > Yup. It=E2=80=99s a manually maintained list in a wiki, and each entry = > is under the control of the person(s) who supposedly own/run that = > resource. > > > I tried to contact the webmaster over there to see if they wanted = > assistance in cleaning up the list, and never got any responses, just a = > "hey, your email has been put in the moderation queue=E2=80=9D. > > Speaking as the postmaster and listmaster for ntp.org, you have my = > apologies. I won=E2=80=99t make excuses or try to explain how we got = > where we are, I will just say that we are trying to do better and we are = > actively working to try to improve our support for the community. > > However, since that list was originally created by the community, and = > each entry should remain under the control of the person(s) who own/run = > that resource, IMO it=E2=80=99s not really appropriate for you to be = > trying to actively monitor any of those systems. At least, not without = > their express prior approval. > > > Moreover, the system running the website for support.ntp.org is ancient = > and weak, and any attempt to do any kind of =E2=80=9Cspidering=E2=80=9D = > is likely to cause undue load. Perhaps to the point of making it fall = > over and die. Fixing that is one of the things on our long list of = > =E2=80=9CTODO=E2=80=9D items. > > That said, even if the hardware, operating system, web server software, = > etc=E2=80=A6 were all very current and highly robust, I still do not = > believe that it is appropriate for people to be spidering or mirroring = > the content from our site. At least, not without our express prior = > approval. > > If I need to clean up our robots.txt and the landing page of the wiki to = > make this more clear, then I will do so. > > > I imagine this becomes a frustration for many that are trying to = > contribute to the pool, and so I wanted to do something about it. > > Here=E2=80=99s the trick =E2=80=94 the pool.ntp.org project doesn=E2=80=99= > t actually have any relation whatsoever to the list of NTP servers you = > identified. They are wholly and totally separate and distinct. We are = > happy to provide space for pool.ntp.org within the ntp.org domain and to = > provide whatever other support we can, but they are actually two = > separate projects. > > If you want to improve the pool by monitoring things, then you should = > talk to the people who are running pool.ntp.org and currently doing the = > monitoring of the servers within the pool. > > OTOH, if you want to improve the quality of the list of NTP services = > listed at the URL you provided, then you should be coming to talk to the = > people who maintain support.ntp.org. > > > So I built this: http://ntp.exactlywww.com/ntp/ > > It's primary purpose is to assist pool server operators in finding = > reliable servers to configure their own ntpd. However, it has the side = > benefit of allowing the stratum two folks to keep tabs on their servers. = > I've exposed the data via a rudimentary API (details on the same page), = > which can be queried using whatever service/programming language floats = > your boat. In fact, with something like NodePing, you could configure a = > status check to ensure that the API response always says "up: 1=E2=80=9D. > > Google didn=E2=80=99t ask you to monitor their servers, and neither did = > we. > > Nor did we ask you to take the content from our site and repackage and = > distribute it in a format that is more to your liking. > > > I=E2=80=99m not necessarily opposed to the goal you=E2=80=99re trying to = > achieve, but I think you=E2=80=99ve gone about it the wrong way and = > without talking to the right people. > > Come talk to us. We=E2=80=99ll be happy to work with you to see if = > there is a way we can make this sort of thing happen in an officially = > approved manner. > > Maybe we can work out a way so that the list of NTP servers in question = > is officially maintained through a real database and not just a wiki, = > and we can send out notices to all the currently listed maintainers so = > that we can get their official buy-in on moving the registry over and = > being remotely monitored. > > But you really need to come talk to us. > > Send e-mail to [email protected] and I=E2=80=99ll be monitoring the = > moderation queue, so that we can get your messages approved and sent = > through quickly. > > -- > Brad Knowles <[email protected]> > > > --Apple-Mail=_B18C9047-F716-4EF2-BAFE-8760411A197A > Content-Transfer-Encoding: 7bit > Content-Disposition: attachment; > filename=signature.asc > Content-Type: application/pgp-signature; > name=signature.asc > Content-Description: Message signed with OpenPGP using GPGMail > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.30 > > iQIcBAEBCAAGBQJXvnSaAAoJEKOyg2+mWTCTGTEP/2oz36UZBs8pDewG4jTmIlNp > sco5ddBsCtOH8O+5kuAlTr6DTy2Fgr/V6WtTf0PAtVDCswVuaWiabA3IUebAJFFR > 1fgR+FGVtfYeby3h8lj3gE63S9hlTSty6EZJGyhi7NKHc5fuVDbcoDZme8XvUIXN > RY/TL1WtBuVqMGY9SQtD83DmtXqfWO/swVTYd5qQDQAux6jJ3v7yjsBC15rKYq25 > AvhW7Tz6ry71xBgS/VgVk6ygyw4ADJvy8KobOuN1HeR2AO1FfhcxYfynn6ZlPpaT > f+IN76zglJCVAb0pVk03Z0U4KvtTCal851Fwfm61VJbMzdHeidYwnwFvxe8mDQBm > JQKrsNNWoCyh2CIIF0IRnP36dHu8hyhJXG6TsMUOWeQoqsdno9hyQ61dwITsgh7f > 6qWV4Dg0sIzqsFbLb5/btsy+s6JRJvB0Vc46NGAzB+K/CAKpIWj88MDUX5bF6Lg/ > ++/kfHeW/enwTTohjycxZa4baDTBAlBLrB0Jv/JxYRKYtibaOjDaZnWyZSeSLJvT > QD62QvUNpHZCD/FD2u/5GLZ3cTt5ulEXmpUeUw27wF8ldBKkeZl7Rc2x/EwwcBCW > sJvVEZ2zG8t09YgYvXn0z2NXeI2MX0s3mOPiq6xn9EduE1t+7yx+C9aQEyscjJZR > 6neX9FhJ/8tOvZLPZyan > =fkoX > -----END PGP SIGNATURE----- > > --Apple-Mail=_B18C9047-F716-4EF2-BAFE-8760411A197A-- > > --===============5561077762517319938== > Content-Type: text/plain; charset="us-ascii" > MIME-Version: 1.0 > Content-Transfer-Encoding: 7bit > Content-Disposition: inline > > _______________________________________________ > pool mailing list > [email protected] > http://lists.ntp.org/listinfo/pool > --===============5561077762517319938==-- > _______________________________________________ pool mailing list [email protected] http://lists.ntp.org/listinfo/pool
