Re: [atlas] Tagging probes which are using the ISP's DNS server?
Hello, I am currently out of the office until January 2nd. If this is an urgent message please reach out to my manager Robert Kisteleki or myself directly at +31 643419539. Happy holidays :) Guy Meyer On 6 Oct 2022, at 12:32, Simon Brandt via ripe-atlas wrote: > Hi, > > Since a lot of probes use RFC1918 DNS resolvers (like home DSL/Cable routers > etc.) you can't tell, if an ISP-resolver or Public-resolver is actually used. > > Another thing I noticed is, that some eyeball providers stopped provisioning > their own DNS resolvers. Instead, they assing public resolvers like > Cloudflare to their customers. > > If the distinction isn't to difficult to implement, I would prefer these > three types as system tags: > > Inside-AS DNS > Outside-AS DNS > RFC1918 DNS > > Best Regards, > Simon > > > On 6 October 2022 09:23:15 UTC, Robert Kisteleki wrote: > Hello, > > This seems to be an interesting question. > > We can certainly apply some (system) tags for probes that have the popular > resolvers 8.8.8.8, 9.9.9.9 and so on in the resolver configuration. This > would allow users like you to easily filter for, or filter out, probes that > do this. > > One complication is that in many cases probes (an by extension, the users) > have such a public resolver *in addition to* whatever else they use - which > complicates the semantics of what resolver was actually used. But I guess one > can accept that as a fact and still consider the feature to be useful. > > As an extension, we can, if that's deemed useful, tag other resolvers, along > the lines of: > * resolvers on private IPs (ie. on-net in the home?) > * mixed private-and-quadX > * mixed private-and-public > > If we go this far, a probe could have multiple tags, eg. uses- + > uses-private + mixed-private-and-quad. This may be overdoing it... > > We'd be curious about what you think. > > Regards, > Robert > > > On 2022-10-06 03:38, Max Grobecker wrote: > Hi, > > a few days ago I wanted to debug a name resolution problem of one of our > domains. > For this reason, I wanted to test if probes inside a specific ASN are having > difficulties to resolve a specific name (because only customers of this ISP > were complaining). > This lead to very mixed results, mostly because some of the selected probes > did queries to a public DNS service like Google, Quad9 and so on. > The problem existed only with the provider's DNS servers for some reason. > > > It did take some time to make a script which tried to filter out these > probes, so I wondered if anyone else had the same use-case and problem. > Is there a way to automatically tag probes, which are (seemingly) using the > ISP's own DNS servers, or, at least, not a well-known public service? > This could be done maybe by querying a special DNS name which returns the IP > address from where the query was received (like "whoami.akamai.net"). > By comparing the ASN of the probe and the ASN of the IP address returned by > the DNS query, one could determine, if the ISP's servers are used. > This would also be true for people running their own recursor, but this could > be filtered as well very easy. > If an ISP is using multiple ASN, this could be a problem. Maybe there's an > easy solution for this as well. > > Probes which pass this test, could then be tagged with "DNS-using-ISP-server" > or something like that and explicitly be selected for specific DNS resolution > tests. > > > Greetings, > Max > > > -- > ripe-atlas mailing list > ripe-atlas@ripe.net > https://lists.ripe.net/mailman/listinfo/ripe-atlas > -- > ripe-atlas mailing list > ripe-atlas@ripe.net > https://lists.ripe.net/mailman/listinfo/ripe-atlas -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
Hello, I am currently out of the office until January 2nd. If this is an urgent message please reach out to my manager Robert Kisteleki or myself directly at +31 643419539. Happy holidays :) Guy Meyer On 6 Oct 2022, at 19:11, Karsten Thomann via ripe-atlas wrote: > On Thursday, 6 October 2022 19:42:58 EEST netravnen+ripel...@gmail.com wrote: > On Thu, 6 Oct 2022 at 13:32, Simon Brandt via ripe-atlas > > wrote: > If the distinction isn't to difficult to implement, I would prefer these > three types as system tags: > > Inside-AS DNS > Outside-AS DNS > RFC1918 DNS > > Agreed, these three tags would IMO satisfy *most* use cases. Certainly > mine, too. > I'm now curious how that would work with dual-stack probes using different > providers for v4 and v6... > > My probe uses my local Provider for v4 and HE for IPv6 (static /48 network > with reverse dns delegation would never be possible from my local provider). > According to that logic, using my local router as dns server for the probe, > could set all three tags depending on the used dns server. > As an example depending on the transport protocol used for dns: > - v4 uses RFC1918 IP as resolver IP and inside AS DNS Server > - v6 uses public IP of subnet and resolves via a different AS if the query is > sent over v6 > It could also be argued, that the v6 case is still Inside-AS DNS, but it > should be clear how the tags are determined. > > Or is this case in the meantime special enough to ignore it due to the rising > native v6 rollout by providers? > > > > -- > ripe-atlas mailing list > ripe-atlas@ripe.net > https://lists.ripe.net/mailman/listinfo/ripe-atlas -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
On 10 Oct 2022, at 13:42, Robert Kisteleki wrote: Here's a quick-and-dirty answer to this: what the tags could look like if we applied them. This is about the ~12K connected probes, which collectively have ~23K resolver entries. Note: in this experiment the "more specific tags win", i.e. if quad1 is applied, then outside-asn tag is not. Thanks for this quick and useful report, Robert. What I take from it is that - (a) for most of the categories of interest, the one which (most closely) matches a given probe may readily be determined, and - (b) only one category seems ambiguous with regard to Max's original question, since "inside ASN" doesn't necessarily correspond to "using the ISP's DNS server". I'm wondering whether this ambiguity matters in the context of Max's question, and reckon that his opinion on this is a significant one. For my own network, I have always the option of reconfiguring to use ULA/RFC1918 (both private) instead of GLA/RFC1918 (inside-ASN/private). Thanks again, Niall (hat off) -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
I believe it would be informative if we assembled some statistics about the current state, ie. how many probes would get what kind of tags if we applied them today. Perhaps Max already has some up his sleeve? Here's a quick-and-dirty answer to this: what the tags could look like if we applied them. This is about the ~12K connected probes, which collectively have ~23K resolver entries. Note: in this experiment the "more specific tags win", i.e. if quad1 is applied, then outside-asn tag is not. 6718 system-resolv-v4-private 4502 system-resolv-v4-inside-asn 2834 system-resolv-v4-google 1247 system-resolv-v4-outside-asn 1020 system-resolv-v4-loopback 933 system-resolv-v4-quad1 338 system-resolv-v4-quad9 31 system-resolv-v4-1-24 1745 system-resolv-v6-inside-asn 968 system-resolv-v6-ula 560 system-resolv-v6-loopback 473 system-resolv-v6-quad9 461 system-resolv-v6-google 335 system-resolv-v6-outside-asn 319 system-resolv-v6-linklocal 187 system-resolv-v6-quad1 Cheers, Robert -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
Inside-AS DNS Outside-AS DNS RFC1918 DNS We probably need a "localhost DNS" too I would still argue that the "big" public resolvers warrant their own tags. Apart from that, should there be separate tags for RFC1918 and ULA? Paraphrasing a quote from may decades ago: "probe DNS tagging is like math - a simple idea but it can get complicated" :-) We could separate IPv4 and IPv6 tags in this context. Which is clearer but also more complicated to process. Still, for each af, probes could get multiple of these tags. I believe it would be informative if we assembled some statistics about the current state, ie. how many probes would get what kind of tags if we applied them today. Perhaps Max already has some up his sleeve? Regards, Robert -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
On 6 Oct 2022, at 13:32, Simon Brandt via ripe-atlas wrote: If the distinction isn't to difficult to implement, I would prefer these three types as system tags: Inside-AS DNS Outside-AS DNS RFC1918 DNS At least these are necessary, but I'm not sure that they are sufficient. For example, "Inside-AS" masks the distinction between on-site and ISP-provided resolvers, which are distinct on my domestic network (RFC1918 + GLA) and still (I believe) on the campus network where I once had a day job (Gv4 + GLA). Apart from that, should there be separate tags for RFC1918 and ULA? €0,02 Niall -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
On Thursday, 6 October 2022 19:42:58 EEST netravnen+ripel...@gmail.com wrote: > On Thu, 6 Oct 2022 at 13:32, Simon Brandt via ripe-atlas > > wrote: > > If the distinction isn't to difficult to implement, I would prefer these > > three types as system tags: > > > > Inside-AS DNS > > Outside-AS DNS > > RFC1918 DNS > > Agreed, these three tags would IMO satisfy *most* use cases. Certainly > mine, too. I'm now curious how that would work with dual-stack probes using different providers for v4 and v6... My probe uses my local Provider for v4 and HE for IPv6 (static /48 network with reverse dns delegation would never be possible from my local provider). According to that logic, using my local router as dns server for the probe, could set all three tags depending on the used dns server. As an example depending on the transport protocol used for dns: - v4 uses RFC1918 IP as resolver IP and inside AS DNS Server - v6 uses public IP of subnet and resolves via a different AS if the query is sent over v6 It could also be argued, that the v6 case is still Inside-AS DNS, but it should be clear how the tags are determined. Or is this case in the meantime special enough to ignore it due to the rising native v6 rollout by providers? -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
On Thu, 6 Oct 2022 at 13:32, Simon Brandt via ripe-atlas wrote: > If the distinction isn't to difficult to implement, I would prefer these > three types as system tags: > > Inside-AS DNS > Outside-AS DNS > RFC1918 DNS Agreed, these three tags would IMO satisfy *most* use cases. Certainly mine, too. -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
Hi, Since a lot of probes use RFC1918 DNS resolvers (like home DSL/Cable routers etc.) you can't tell, if an ISP-resolver or Public-resolver is actually used. Another thing I noticed is, that some eyeball providers stopped provisioning their own DNS resolvers. Instead, they assing public resolvers like Cloudflare to their customers. If the distinction isn't to difficult to implement, I would prefer these three types as system tags: Inside-AS DNS Outside-AS DNS RFC1918 DNS Best Regards, Simon On 6 October 2022 09:23:15 UTC, Robert Kisteleki wrote: >Hello, > >This seems to be an interesting question. > >We can certainly apply some (system) tags for probes that have the popular >resolvers 8.8.8.8, 9.9.9.9 and so on in the resolver configuration. This would >allow users like you to easily filter for, or filter out, probes that do this. > >One complication is that in many cases probes (an by extension, the users) >have such a public resolver *in addition to* whatever else they use - which >complicates the semantics of what resolver was actually used. But I guess one >can accept that as a fact and still consider the feature to be useful. > >As an extension, we can, if that's deemed useful, tag other resolvers, along >the lines of: >* resolvers on private IPs (ie. on-net in the home?) >* mixed private-and-quadX >* mixed private-and-public > >If we go this far, a probe could have multiple tags, eg. uses- + >uses-private + mixed-private-and-quad. This may be overdoing it... > >We'd be curious about what you think. > >Regards, >Robert > > >On 2022-10-06 03:38, Max Grobecker wrote: >> Hi, >> >> a few days ago I wanted to debug a name resolution problem of one of our >> domains. >> For this reason, I wanted to test if probes inside a specific ASN are having >> difficulties to resolve a specific name (because only customers of this ISP >> were complaining). >> This lead to very mixed results, mostly because some of the selected probes >> did queries to a public DNS service like Google, Quad9 and so on. >> The problem existed only with the provider's DNS servers for some reason. >> >> >> It did take some time to make a script which tried to filter out these >> probes, so I wondered if anyone else had the same use-case and problem. >> Is there a way to automatically tag probes, which are (seemingly) using the >> ISP's own DNS servers, or, at least, not a well-known public service? >> This could be done maybe by querying a special DNS name which returns the IP >> address from where the query was received (like "whoami.akamai.net"). >> By comparing the ASN of the probe and the ASN of the IP address returned by >> the DNS query, one could determine, if the ISP's servers are used. >> This would also be true for people running their own recursor, but this >> could be filtered as well very easy. >> If an ISP is using multiple ASN, this could be a problem. Maybe there's an >> easy solution for this as well. >> >> Probes which pass this test, could then be tagged with >> "DNS-using-ISP-server" or something like that and explicitly be selected for >> specific DNS resolution tests. >> >> >> Greetings, >> Max >> > >-- >ripe-atlas mailing list >ripe-atlas@ripe.net >https://lists.ripe.net/mailman/listinfo/ripe-atlas -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
Max Grobecker writes: > This could be done maybe by querying a special DNS name which returns > the IP address from where the query was received (like > "whoami.akamai.net"). By comparing the ASN of the probe and the ASN > of the IP address returned by the DNS query, one could determine, if > the ISP's servers are used. There should be no need for a new service. The SOS queries already provides the necessary raw data. You can see resolver addresses in the probe's "SOS History". Someone "just" has to process the data and produce a "Resolver-in-same-AS" tag. > This would also be true for people running their own recursor, but > this could be filtered as well very easy. How? Reject resolvers which are only used by a single probe? Or did you have something smarter in mind? If not, I fear it would produce a large number of false positives. Many ISPs will have a relatively large resolver to probe ratio (when counting resolver addresses visible to authoritative servers). > If an ISP is using multiple ASN, this could be a problem. Maybe > there's an easy solution for this as well. Geoff Huston has tried to analyze this as part of open resolver measurements: https://www.potaroo.net/ispcol/2019-09/centrality.html Doing a "same CC and not well-kown public resolver" might do it. Bjørn -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
Hello, This seems to be an interesting question. We can certainly apply some (system) tags for probes that have the popular resolvers 8.8.8.8, 9.9.9.9 and so on in the resolver configuration. This would allow users like you to easily filter for, or filter out, probes that do this. One complication is that in many cases probes (an by extension, the users) have such a public resolver *in addition to* whatever else they use - which complicates the semantics of what resolver was actually used. But I guess one can accept that as a fact and still consider the feature to be useful. As an extension, we can, if that's deemed useful, tag other resolvers, along the lines of: * resolvers on private IPs (ie. on-net in the home?) * mixed private-and-quadX * mixed private-and-public If we go this far, a probe could have multiple tags, eg. uses- + uses-private + mixed-private-and-quad. This may be overdoing it... We'd be curious about what you think. Regards, Robert On 2022-10-06 03:38, Max Grobecker wrote: Hi, a few days ago I wanted to debug a name resolution problem of one of our domains. For this reason, I wanted to test if probes inside a specific ASN are having difficulties to resolve a specific name (because only customers of this ISP were complaining). This lead to very mixed results, mostly because some of the selected probes did queries to a public DNS service like Google, Quad9 and so on. The problem existed only with the provider's DNS servers for some reason. It did take some time to make a script which tried to filter out these probes, so I wondered if anyone else had the same use-case and problem. Is there a way to automatically tag probes, which are (seemingly) using the ISP's own DNS servers, or, at least, not a well-known public service? This could be done maybe by querying a special DNS name which returns the IP address from where the query was received (like "whoami.akamai.net"). By comparing the ASN of the probe and the ASN of the IP address returned by the DNS query, one could determine, if the ISP's servers are used. This would also be true for people running their own recursor, but this could be filtered as well very easy. If an ISP is using multiple ASN, this could be a problem. Maybe there's an easy solution for this as well. Probes which pass this test, could then be tagged with "DNS-using-ISP-server" or something like that and explicitly be selected for specific DNS resolution tests. Greetings, Max -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
On 10/6/22 01:35, Johan ter Beest wrote: Although you are completely free to choose whichever DNS server you think is best for your probe, we generally advise to use the same one that you assign to the rest of the network that the probe is connected to. Thank you. To avoid any misunderstanding: When you say "rest of the network" you mean the LAN/WAN-router, not the ISP's AS, correct? :-) Daniel AJ -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
Hi Daniel, On 06/10/2022 07:41, Daniel AJ Sokolov wrote: On this topic: Is there an Atlas-preferred setting for this? Should the probe I host ideally use the ISP's DNS, or ideally a DNS from a global player? Or ideally an academic/non-profit DNS? Although you are completely free to choose whichever DNS server you think is best for your probe, we generally advise to use the same one that you assign to the rest of the network that the probe is connected to. Cheers, Johan ter Beest RIPE Atlas Team BR Daniel AJ On 10/5/22 18:38, Max Grobecker wrote: Hi, a few days ago I wanted to debug a name resolution problem of one of our domains. For this reason, I wanted to test if probes inside a specific ASN are having difficulties to resolve a specific name (because only customers of this ISP were complaining). This lead to very mixed results, mostly because some of the selected probes did queries to a public DNS service like Google, Quad9 and so on. The problem existed only with the provider's DNS servers for some reason. It did take some time to make a script which tried to filter out these probes, so I wondered if anyone else had the same use-case and problem. Is there a way to automatically tag probes, which are (seemingly) using the ISP's own DNS servers, or, at least, not a well-known public service? This could be done maybe by querying a special DNS name which returns the IP address from where the query was received (like "whoami.akamai.net"). By comparing the ASN of the probe and the ASN of the IP address returned by the DNS query, one could determine, if the ISP's servers are used. This would also be true for people running their own recursor, but this could be filtered as well very easy. If an ISP is using multiple ASN, this could be a problem. Maybe there's an easy solution for this as well. Probes which pass this test, could then be tagged with "DNS-using-ISP-server" or something like that and explicitly be selected for specific DNS resolution tests. Greetings, Max -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
Re: [atlas] Tagging probes which are using the ISP's DNS server?
On this topic: Is there an Atlas-preferred setting for this? Should the probe I host ideally use the ISP's DNS, or ideally a DNS from a global player? Or ideally an academic/non-profit DNS? BR Daniel AJ On 10/5/22 18:38, Max Grobecker wrote: Hi, a few days ago I wanted to debug a name resolution problem of one of our domains. For this reason, I wanted to test if probes inside a specific ASN are having difficulties to resolve a specific name (because only customers of this ISP were complaining). This lead to very mixed results, mostly because some of the selected probes did queries to a public DNS service like Google, Quad9 and so on. The problem existed only with the provider's DNS servers for some reason. It did take some time to make a script which tried to filter out these probes, so I wondered if anyone else had the same use-case and problem. Is there a way to automatically tag probes, which are (seemingly) using the ISP's own DNS servers, or, at least, not a well-known public service? This could be done maybe by querying a special DNS name which returns the IP address from where the query was received (like "whoami.akamai.net"). By comparing the ASN of the probe and the ASN of the IP address returned by the DNS query, one could determine, if the ISP's servers are used. This would also be true for people running their own recursor, but this could be filtered as well very easy. If an ISP is using multiple ASN, this could be a problem. Maybe there's an easy solution for this as well. Probes which pass this test, could then be tagged with "DNS-using-ISP-server" or something like that and explicitly be selected for specific DNS resolution tests. Greetings, Max -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
[atlas] Tagging probes which are using the ISP's DNS server?
Hi, a few days ago I wanted to debug a name resolution problem of one of our domains. For this reason, I wanted to test if probes inside a specific ASN are having difficulties to resolve a specific name (because only customers of this ISP were complaining). This lead to very mixed results, mostly because some of the selected probes did queries to a public DNS service like Google, Quad9 and so on. The problem existed only with the provider's DNS servers for some reason. It did take some time to make a script which tried to filter out these probes, so I wondered if anyone else had the same use-case and problem. Is there a way to automatically tag probes, which are (seemingly) using the ISP's own DNS servers, or, at least, not a well-known public service? This could be done maybe by querying a special DNS name which returns the IP address from where the query was received (like "whoami.akamai.net"). By comparing the ASN of the probe and the ASN of the IP address returned by the DNS query, one could determine, if the ISP's servers are used. This would also be true for people running their own recursor, but this could be filtered as well very easy. If an ISP is using multiple ASN, this could be a problem. Maybe there's an easy solution for this as well. Probes which pass this test, could then be tagged with "DNS-using-ISP-server" or something like that and explicitly be selected for specific DNS resolution tests. Greetings, Max -- ripe-atlas mailing list ripe-atlas@ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas
