[Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Jeffrey Walton
I believe wget has a security flaw in its certificate hostname matching code. In the attached server certificate, the hostname is provided via a Subject Alt Name (SAN). The only SAN entry is a DNS name for "*.com". Also attached is the default CA, which was used to sign the server's certificate.

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Darshit Shah
Hi Jeffrey, Thanks for pointing this out! I am no expert in security or SSL for that matter. However, this does seem like a huge security flaw. I'll try and set up a test case as soon as I can using the materials provided by you. It would be even more helpful if someone could pitch in with more h

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Daniel Kahn Gillmor
Hi Jeffrey-- On 03/18/2014 01:43 AM, Jeffrey Walton wrote: > I believe wget has a security flaw in its certificate hostname matching code. > > In the attached server certificate, the hostname is provided via a > Subject Alt Name (SAN). The only SAN entry is a DNS name for "*.com". > Also attached

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Daniel Stenberg
On Tue, 18 Mar 2014, Darshit Shah wrote: I'll try and set up a test case as soon as I can using the materials provided by you. It would be even more helpful if someone could pitch in with more help since: 1. This is not my domain and I don't understand it much. 2. I'm keeping really busy with

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Ángel González
I don't think wget should be checking correct hostname scope of the certificate. I mean, it'd be ok to have some general rule as "noone can use a certificate for *.whatever or *." [1] but embedding the Public Suffix List seems overkill. And the implementation should probably be performed at open

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Tim Rühsen
Hi Jeffrey, thanks for pointing this out. BTW, to reproduce the issue I used a GnuTLS compiled/linked version of Wget: $ wget -d --ca-certificate=ca-rsa-cert.pem --private-key=ca-rsa-key-plain.pem https://example.com:8443 2014-03-18 21:48:04 (1.88 GB/s) - Read error at byte 5116 (The TLS connec

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Jeffrey Walton
Hi Tim, On Tue, Mar 18, 2014 at 5:31 PM, Tim Rühsen wrote: > ... > BTW, to reproduce the issue I used a GnuTLS compiled/linked version of Wget: > > $ wget -d --ca-certificate=ca-rsa-cert.pem --private-key=ca-rsa-key-plain.pem > https://example.com:8443 > 2014-03-18 21:48:04 (1.88 GB/s) - Read err

Re: [Bug-wget] Overly permissive hostname matching

2014-03-18 Thread Daniel Kahn Gillmor
On 03/18/2014 05:31 PM, Tim Rühsen wrote: > $ wget -d --ca-certificate=ca-rsa-cert.pem --private-key=ca-rsa-key-plain.pem > https://example.com:8443 > 2014-03-18 21:48:04 (1.88 GB/s) - Read error at byte 5116 (The TLS connection > was non-properly terminated.).Retrying. > > There seems to be a p

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Tim Ruehsen
On Tuesday 18 March 2014 20:05:07 Daniel Kahn Gillmor wrote: > On 03/18/2014 05:31 PM, Tim Rühsen wrote: > > IHMO, the Public Suffix List (PSL) should not only be used to verify > > cookies but also be used for certificate hostname checking. > > > > Libraries as GnuTLS should offer an API for this

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Stenberg
On Tue, 18 Mar 2014, Ángel González wrote: Daniel, how does cURL check correctness of the certificate hostname suffix? It insists on at least two dots. So yes, "*.apple" will cause problems for us too. I view the public suffix list as one of the worst kludges in networking history and whil

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Kahn Gillmor
On 03/19/2014 10:38 AM, Daniel Stenberg wrote: > On Tue, 18 Mar 2014, Ángel González wrote: > >> Daniel, how does cURL check correctness of the certificate hostname >> suffix? > > It insists on at least two dots. So yes, "*.apple" will cause problems > for us too. There are also errors in the op

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Stenberg
On Wed, 19 Mar 2014, Daniel Kahn Gillmor wrote: It insists on at least two dots. So yes, "*.apple" will cause problems for us too. There are also errors in the opposite direction: it sounds like curl will accept a cert for *.co.uk, right? Exactly, due to the lack of public suffix awareness!

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Kahn Gillmor
On 03/19/2014 06:19 AM, Tim Ruehsen wrote: > As a programmer, I want to have control. E.g. the option to load from a > different file, or to switch off loading. Why ? e.g. for testing purposes, or > simply imagine a "swiss army knife" client for experts - maybe they want to > have control via CL

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 10:59 AM, Daniel Kahn Gillmor wrote: > On 03/19/2014 06:19 AM, Tim Ruehsen wrote: >> As a programmer, I want to have control. E.g. the option to load from a >> different file, or to switch off loading. Why ? e.g. for testing purposes, or >> simply imagine a "swiss army knif

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Stenberg
On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin with "!" That sounds wrong: A rule may begin with a "!" (exclamation mark). If it does, it is labelled as a "exception rule" and then treated as if the exclamation mark is not present. -- / daniel.haxx.se

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Kahn Gillmor
On 03/19/2014 11:26 AM, Jeffrey Walton wrote: > wget "http://publicsuffix.org/list/effective_tld_names.dat"; -O $MOZILLA_LIST I recommend using the following HTTPS URL instead, so that you have some level of cryptographic verification of the data before loading it: https://hg.mozilla.org/mozilla

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:38 AM, Daniel Kahn Gillmor wrote: > On 03/19/2014 11:26 AM, Jeffrey Walton wrote: > >> wget "http://publicsuffix.org/list/effective_tld_names.dat"; -O $MOZILLA_LIST > > I recommend using the following HTTPS URL instead, so that you have some > level of cryptographic veri

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:42 AM, Jeffrey Walton wrote: > On Wed, Mar 19, 2014 at 11:38 AM, Daniel Kahn Gillmor > wrote: >> On 03/19/2014 11:26 AM, Jeffrey Walton wrote: >> >>> wget "http://publicsuffix.org/list/effective_tld_names.dat"; -O $MOZILLA_LIST >> >> I recommend using the following HTTP

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:37 AM, Jeffrey Walton wrote: > On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg wrote: >> On Wed, 19 Mar 2014, Jeffrey Walton wrote: >> >>> # Remove lines that begin with "!" >> >> >> That sounds wrong: >> >> A rule may begin with a "!" (exclamation mark). If it does

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg wrote: > On Wed, 19 Mar 2014, Jeffrey Walton wrote: > >> # Remove lines that begin with "!" > > > That sounds wrong: > > A rule may begin with a "!" (exclamation mark). If it does, it is labelled > as a "exception rule" and then treated as if t

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:38 AM, Daniel Kahn Gillmor wrote: > On 03/19/2014 11:26 AM, Jeffrey Walton wrote: > >> wget "http://publicsuffix.org/list/effective_tld_names.dat"; -O $MOZILLA_LIST > > I recommend using the following HTTPS URL instead, so that you have some > level of cryptographic veri

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 11:45 AM, Jeffrey Walton wrote: > On Wed, Mar 19, 2014 at 11:37 AM, Jeffrey Walton wrote: >> On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg wrote: >>> On Wed, 19 Mar 2014, Jeffrey Walton wrote: >>> # Remove lines that begin with "!" >>> >>> >>> That sounds wrong:

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Daniel Kahn Gillmor
On 03/19/2014 11:55 AM, Jeffrey Walton wrote: > Also, be careful of where you are pulling the list from. I got burned > by pulling a list that was not being updated > (https://bugzilla.mozilla.org/show_bug.cgi?id=968064). i've been similarly burned before too, but i settled on the mxr address i ju

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Ángel González
On 19/03/14 16:37, Jeffrey Walton wrote: On Wed, Mar 19, 2014 at 11:30 AM, Daniel Stenberg wrote: On Wed, 19 Mar 2014, Jeffrey Walton wrote: # Remove lines that begin with "!" That sounds wrong: A rule may begin with a "!" (exclamation mark). If it does, it is labelled as a "exceptio

Re: [Bug-wget] Overly permissive hostname matching

2014-03-19 Thread Jeffrey Walton
On Wed, Mar 19, 2014 at 3:03 PM, Ángel González wrote: > On 19/03/14 16:37, Jeffrey Walton wrote: >> >> ... > Also note that by removing the "*." from the beginning of the lines*, you > are acepting more hosts than > you should, such as a certificate for *.com.bd (represented as *.bd in the > PSL)

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Tim Rühsen
Am Mittwoch, 19. März 2014, 10:59:05 schrieb Daniel Kahn Gillmor: > I'm imagining a C library API that has a public suffix list context > object that can do efficient lookups (however we define the lookups), > and the library would bundle a pre-compiled context, based on the > currently-known publi

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 5:52 PM, Tim Rühsen wrote: > Am Mittwoch, 19. März 2014, 10:59:05 schrieb Daniel Kahn Gillmor: >> I'm imagining a C library API that has a public suffix list context >> object that can do efficient lookups (however we define the lookups), >> and the library would bundle a p

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Daniel Stenberg
On Thu, 20 Mar 2014, Tim Rühsen wrote: I broke out the public suffix code together and created a first go (really very quick, distcheck fails - couldn't figure out this evening). https://github.com/rockdaboot/libpsl Ok, I'll be the first to rain on the parade. Sorry but it seems fit to do th

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 6:11 PM, Daniel Stenberg wrote: > On Thu, 20 Mar 2014, Tim Rühsen wrote: > >> I broke out the public suffix code together and created a first go (really >> very quick, distcheck fails - couldn't figure out this evening). >> >> https://github.com/rockdaboot/libpsl > > > Ok,

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 20/03/14 23:11, Daniel Stenberg wrote: You do realize that with a *GPL license on the thing, you won't get adopted by OpenSSL, curl and possibly others? I can't prevent you of course and the decision is yours to make, but I'd prefer a BSD style license as then I could really consider basing

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 20/03/14 22:52, Tim Rühsen wrote: I broke out the public suffix code together and created a first go (really very quick, distcheck fails - couldn't figure out this evening). https://github.com/rockdaboot/libpsl The first step was a psl_is_tld() function. There is a test case for some major t

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 20/03/14 23:16, Jeffrey Walton wrote: I can't prevent you of course and the decision is yours to make, but I'd prefer a BSD style license as then I could really consider basing future enhancements of curl on this effort. Does GNU have a permissive license? I know permissive does not meet al

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 7:11 PM, Ángel González wrote: > On 20/03/14 23:16, Jeffrey Walton wrote: >> >> >>> I can't prevent you of course and the decision is yours to make, but I'd >>> prefer a BSD style license as then I could really consider basing future >>> enhancements of curl on this effort.

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Daniel Stenberg
On Fri, 21 Mar 2014, Ángel González wrote: The LGPL would be an option. Not for curl though and probably not to other BSD/MIT licensed projects... -- / daniel.haxx.se

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Ángel González
On 21/03/14 00:21, Daniel Stenberg wrote: On Fri, 21 Mar 2014, Ángel González wrote: The LGPL would be an option. Not for curl though and probably not to other BSD/MIT licensed projects... That's a good point. Jeff wrote: Isn't copyright assigned to GNU or FSF? No. By licensing somet

Re: [Bug-wget] Overly permissive hostname matching

2014-03-20 Thread Jeffrey Walton
On Thu, Mar 20, 2014 at 8:12 PM, Ángel González wrote: > On 21/03/14 00:21, Daniel Stenberg wrote: >> >> ... >> (Sorry, I don't know. I'm not a lawyer, so my solution is usually to >> avoid GPL code all together). > > That's a solution. Although it's a sad result from usage of a license > intended

Re: [Bug-wget] Overly permissive hostname matching

2014-03-21 Thread Tim Ruehsen
On Thursday 20 March 2014 23:11:31 Daniel Stenberg wrote: > On Thu, 20 Mar 2014, Tim Rühsen wrote: > > I broke out the public suffix code together and created a first go (really > > very quick, distcheck fails - couldn't figure out this evening). > > > > https://github.com/rockdaboot/libpsl > > O

Re: [Bug-wget] Overly permissive hostname matching

2014-03-21 Thread Tim Ruehsen
On Thursday 20 March 2014 17:58:05 Jeffrey Walton wrote: > On Thu, Mar 20, 2014 at 5:52 PM, Tim Rühsen wrote: > I had a sidebar with one of the OpenSSL devs because OpenSSL is > cutting in hostname matching in version 1.0.2. > > He shared a link to a IETF working group on the subject: > https://w

Re: [Bug-wget] Overly permissive hostname matching

2014-03-21 Thread Tim Ruehsen
On Thursday 20 March 2014 23:43:08 Ángel González wrote: > On 20/03/14 22:52, Tim Rühsen wrote: > > I broke out the public suffix code together and created a first go (really > > very quick, distcheck fails - couldn't figure out this evening). > > > > https://github.com/rockdaboot/libpsl > > > >

Re: [Bug-wget] Overly permissive hostname matching

2014-03-21 Thread Ángel González
On 18/03/14 16:00, Jeffrey Walton wrote: What if a certificate is issued by a trusted CA that *does* match part of the public suffix list (perhaps because the CA has determined tha tthe application has rightful control over the entire zone)? In practice we know four things. First, no one authorit