Re: [atlas] Case sensitive DNS queries from the probe

[Philip Homburg]

Hi Wilfried,

Quick mental counting suggests that there are 3 different options:
1) if resolve-on-probe is not set, the target is resolved by the Atlas
backend and the probe only gets the IP address. The (glibc) stub
resolver on CentOS does not do any case mangling.
2) if resolve-on-probe is set then the probe resolves the target using
the stub resolver in libevent. This stub resolver mangles the case.
3) for DNS measurements, the domain name parameter is used as is.
However the target of the measurement is subject to case 1 or 2.

Philip

Re: [atlas] Case sensitive DNS queries from the probe

[Wilfried Woeber]

Thanks Philip,

interesting!

But this is maybe just one side of the coin. For UDMs, where the target(s)
are specified as FQDN, would the various interfaces preserve the case
that would eventually be shipped to the stub resolver, or is this done
"automatically" anyway.

Apologies if I am missing something :-)

Thanks,
Wilfried

On 2015-08-06 14:25, Philip Homburg wrote:
> Hi Wilfried,
> 
> On 2015/08/06 12:44 , Wilfried Woeber wrote:
>> Is there any intelligence available about the percentage (or type) of 
>> responders
>> answering with the "correct" camel case?
> 
> I'm running measurement 1666409 which asks for 'WwW.rIpE.nEt.'. In
> addition it sets the DO flag.
> 
> Looking at a download of 'latest', I got the following results:
> Total number of probes: 8267
> Total number of perfect probes: 6780
> Total number of probes with bad Qname: 320
> Total number of probes with FORMERR: 447
> Total number of probes with REFUSED: 74
> Total number of probes with SERVFAIL: 15
> Total number of probes with no answers: 5
> Total number of probes with EDNS0 in Answer: 86
> Total number of probes with no result (timeout): 609
> 
> So 8267 probes are active and 6780 returned the correct results for all
> of their resolvers.
> 
> The other statistics are per resolver. So a single probe may trigger
> more than one error. I'm surprised by the high number of FORMERRs.
> Finding EDNS0 records in the Answer section is also fascinating.
> 
> From other statistics, we find that about 300 probes cannot perform HTTP
> measurements due to DNS lookup failures.
> 
> Philip
> 
> 

Re: [atlas] Case sensitive DNS queries from the probe

[Fearghas Mckay]

Emile

> On 7 Aug 2015, at 05:26, Emile Aben  wrote:
> 
> How about having a single overview page per probe with various things
> that could be wrong with the probe?

That would be truly excellent Emile!

:-)

f

Re: [atlas] Case sensitive DNS queries from the probe

[Emile Aben]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Fearghas, Colin, others,

On 06/08/15 17:05, Philip Homburg wrote:
> On 2015/08/06 16:56 , Colin Johnston wrote:
>> would be great to have this info, happy to help to debug problems
>> with dns connectivity if my isp is affected as well on mass
>> scale, i hope not :)

How about having a single overview page per probe with various things
that could be wrong with the probe?
Some things that could be on that page:
- - DNS mangling (could include a detailed description of what was wrong
exactly per resolver used)
- - IPv6 configured but not working
- - path MTU issues
- - SSL measurement results different from all other probes (hints at an
MiTM)
etc.

Monthly probe reports could have a link to that page in case something
is wrong.

cheers,
Emile


-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBAgAGBQJVxDOJAAoJEKxthF6wloMOrGsP/26w+jtoqnIWDRtbS7F8Gp1E
iBM7qSXJ2BrC7WNYnqObLcQHnFw2OTOf/lOSwu/O/9pBS7PcWGctqJdulSl7l3hU
w938XyJv3yS1eQnxTNZ2/83Lj+vMHl5p/WA740aPTVmCDLP3TKvbh4Ziwk331Czh
ZTRVObt1eOerV6hsMsUYBMCAMQp0PMIJ15RZSIqmDwVCYhOxuoKYVjrofwvcM9E+
2oM5MNT4mWQQatrfqjNSEehTV4b4TJkPUQuJ3El5qtzdO65LQeSCkrMR326cTNrq
QOTK2Ry3E3+UWDvc/XeomJR2RPZEgOafdgjwRAIawF9uNS4CtTqrWd6Fuci0V84v
wCdUbXTc52zJi+pPc1lrtLfk1RxdPyYSlX+dzvtY8OI1PLApglZDUfUIRYcWM2jb
aovrbDKb14iCtFjkeuObsWGJcWxoAXxqG/Pis695cmGley+uJyGP+HCLvBWohXKa
t3irnu2uyBxYSz3a7pGD/BsolRczSBP3IzRE3hYUeh717t7pYLGAosv8r0zQBLsE
qxM1p/9hJ5aOK/CEYGckqGDEHzmMt8LMlcsUgSrYKfRCvuFy5afvpSXFKlYrhZDt
EB8rl73MXJ9tyVJhxy3tK8Vhm1u5O0N7XvWiK6mSAEMt+boMT6wacX2rm4Mtw73Z
rQhH6BID9VU5q/ccIupL
=4Hz9
-END PGP SIGNATURE-

Re: [atlas] Case sensitive DNS queries from the probe

[Philip Homburg]

On 2015/08/06 16:56 , Colin Johnston wrote:
> would be great to have this info, happy to help to debug problems with dns 
> connectivity if my isp is affected as well on mass scale, i hope not :)

You should be able to get a list of probes that mangle case in DNS from
the probe archive API. https://atlas.ripe.net/docs/rest/#probe-archive

Philip

Re: [atlas] Case sensitive DNS queries from the probe

[Colin Johnston]

would be great to have this info, happy to help to debug problems with dns 
connectivity if my isp is affected as well on mass scale, i hope not :)

Colin

> On 6 Aug 2015, at 14:12, Fearghas Mckay  wrote:
> 
> 
>> On 6 Aug 2015, at 13:57, Philip Homburg  wrote:
>> 
>> For the DNS issue, we have a system tag 'Resolver Mangles Case'. But
>> that depends on the probe host looking at the probe page.
>> 
> 
> and knowing that things like that might be published there :-)
> 
>> It is not clear when and how often we should mail probe hosts. Though in
>> theory we could add these kinds of issues to the monthly probe status
>> report.
> 
> Doing this would be useful IMO, I certainly glance over them for the overview 
> of how things are behaving. 
> 
>   f
> 
> 

Re: [atlas] Case sensitive DNS queries from the probe

[Fearghas Mckay]

> On 6 Aug 2015, at 13:57, Philip Homburg  wrote:
> 
> For the DNS issue, we have a system tag 'Resolver Mangles Case'. But
> that depends on the probe host looking at the probe page.
> 

and knowing that things like that might be published there :-)

> It is not clear when and how often we should mail probe hosts. Though in
> theory we could add these kinds of issues to the monthly probe status
> report.

Doing this would be useful IMO, I certainly glance over them for the overview 
of how things are behaving. 

f

Re: [atlas] Case sensitive DNS queries from the probe

[Philip Homburg]

On 2015/08/06 14:43 , Fearghas Mckay wrote:
> Is there a process to tell probe owners that they have issues like that ? 

For the DNS issue, we have a system tag 'Resolver Mangles Case'. But
that depends on the probe host looking at the probe page.

It is not clear when and how often we should mail probe hosts. Though in
theory we could add these kinds of issues to the monthly probe status
report.

Philip

Re: [atlas] Case sensitive DNS queries from the probe

[Fearghas Mckay]

Phillip

> On 6 Aug 2015, at 13:25, Philip Homburg  wrote:
> 
> From other statistics, we find that about 300 probes cannot perform HTTP
> measurements due to DNS lookup failures.

Is there a process to tell probe owners that they have issues like that ? 

f

Re: [atlas] Case sensitive DNS queries from the probe

[Philip Homburg]

Hi Wilfried,

On 2015/08/06 12:44 , Wilfried Woeber wrote:
> Is there any intelligence available about the percentage (or type) of 
> responders
> answering with the "correct" camel case?

I'm running measurement 1666409 which asks for 'WwW.rIpE.nEt.'. In
addition it sets the DO flag.

Looking at a download of 'latest', I got the following results:
Total number of probes: 8267
Total number of perfect probes: 6780
Total number of probes with bad Qname: 320
Total number of probes with FORMERR: 447
Total number of probes with REFUSED: 74
Total number of probes with SERVFAIL: 15
Total number of probes with no answers: 5
Total number of probes with EDNS0 in Answer: 86
Total number of probes with no result (timeout): 609

So 8267 probes are active and 6780 returned the correct results for all
of their resolvers.

The other statistics are per resolver. So a single probe may trigger
more than one error. I'm surprised by the high number of FORMERRs.
Finding EDNS0 records in the Answer section is also fascinating.

>From other statistics, we find that about 300 probes cannot perform HTTP
measurements due to DNS lookup failures.

Philip

Re: [atlas] Case sensitive DNS queries from the probe

[Wilfried Woeber]

Hi Philip,

thanks for the explanation!

Is there any intelligence available about the percentage (or type) of responders
answering with the "correct" camel case?

Thanks,
Wilfried 

On 2015-07-29 15:21, Philip Homburg wrote:
> On 2015/07/29 15:13 , Yang Yu wrote:
>  > Looking through DNS queries log I saw tons of queries from the probe
>> for case sensitive names. Can someone please enlighten me what the
>> possible use cases are?
>>  
>> query: wWw.FacebOOK.COm IN A
> 
> Many years ago there was a proposal for stub resolvers to perform case
> mangling in DNS requests to obtain increased protection against spoofing
> attacks.
> 
> This never became popular but is implemented in the stub resolver in
> libevent, which is used by the measurement code on the probes.
> 
> Then net effect is that probes will send these types of queries whenever
> the measurement code needs to look up a DNS name to obtain a target's IP
> address.
> 
> Philip

Re: [atlas] Case sensitive DNS queries from the probe

[Philip Homburg]

On 2015/07/29 15:13 , Yang Yu wrote:
 > Looking through DNS queries log I saw tons of queries from the probe
> for case sensitive names. Can someone please enlighten me what the
> possible use cases are?
>  
> query: wWw.FacebOOK.COm IN A

Many years ago there was a proposal for stub resolvers to perform case
mangling in DNS requests to obtain increased protection against spoofing
attacks.

This never became popular but is implemented in the stub resolver in
libevent, which is used by the measurement code on the probes.

Then net effect is that probes will send these types of queries whenever
the measurement code needs to look up a DNS name to obtain a target's IP
address.

Philip

Re: [atlas] Case sensitive DNS queries from the probe

[Stephane Bortzmeyer]

On Wed, Jul 29, 2015 at 10:13:57PM +0900,
 Yang Yu  wrote 
 a message of 18 lines which said:

> Looking through DNS queries log I saw tons of queries from the probe
> for case sensitive names. Can someone please enlighten me what the
> possible use cases are?

Probably a test of