[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: i believe new work will be done via the new issue. marking this closed. if there is something not covered by issue38576 that remains, please open a new issue for it. new discussion on this long issue is easy to get lost in. -- resolution: ->

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Ned Deily added the comment: What is the status of this issue? Now that Issue38576 has been opened to cover the host address part, can this issue be closed or downgraded? Should Issue38576 be a release blocker? -- ___ Python tracker

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Riccardo Schirone added the comment: I have created https://bugs.python.org/issue38576 to address CVE-2019-18348. @gregory.p.smith if you have particular complains about these CVEs feel free to let me know (even privately). I think the security impact of these flaws is: an application that

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: Can you please open a separate issue for CVE-2019-18348? It is easier to track that way. (META: In general I think the CVE process is being abused and that these really did not deserve that treatment. https://lwn.net/Articles/801157/ is good reading

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Riccardo Schirone added the comment: CVE-2019-18348 has been assigned to the issue explained in https://bugs.python.org/issue30458#msg347282 . Maybe a separate bug for it would be better though. CVE-2019-18348 is about injecting CRLF in HTTP requests through the *host* part of a URL.

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Jason R. Coombs : -- pull_requests: +15900 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/16321 ___ Python tracker ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Ned Deily added the comment: With the breaking out of the portential and/or actual regression (e.g. invalid requests can no longer be crafted) into Issue38216, itself a potential release blocker, we are still left here with the as-yet unresolved issue identified above in msg34728 (e.g. not

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Jason R. Coombs added the comment: I've created issue38216 to address the (perceived) regression. -- ___ Python tracker ___ ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Jason R. Coombs added the comment: > Should we open a separate issue to track (fixing) the regression? Yes, I think so. The ticket I referenced mainly addresses an incompatibility that was introduced with Python 3.0, so is much less urgent than the one introduced more recently, so I believe

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Larry Hastings added the comment: Should we open a separate issue to track fixing the regression? -- ___ Python tracker ___ ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Ned Deily added the comment: If I understand Jason's message correctly, the changes for Issue30458 introduced a regression in 3.7.4 and will introduce the same regression in other branches as they are released, including 3.5.8 whose rc1 is now in testing. 3.7.5rc1 is scheduled to be tagged

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Jason R. Coombs added the comment: This change caused a regression or two captured in issue36274. Essentially, by blocking invalid requests, it's now not possible for a system intentionally to generate invalid requests for testing purposes. As these point releases of Python start making it

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: I'm not a fan of CVE numbers in general, people have been creating too many of those. But that also means I just don't care if someone does. Having a CVE entry is not a way to claim something is important. This issue is still open and can be used to

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Riccardo Schirone added the comment: Will the flaw outlined in https://bugs.python.org/issue30458#msg347282 be fixed in python itself? If so, I think a CVE for python should be requested to MITRE (I can request one, in that case). Moreover, does it make sense to create a new bug to track

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Larry Hastings added the comment: New changeset afe3a4975cf93c97e5d6eb8800e48f368011d37a by larryhastings (Miro Hrončok) in branch '3.5': bpo-30458: Disallow control chars in http URLs. (GH-12755) (#13207) https://github.com/python/cpython/commit/afe3a4975cf93c97e5d6eb8800e48f368011d37a

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Riccardo Schirone added the comment: > > A second problem comes into the game. Some C libraries like glibc strip the > > end of the hostname (strip at the first newline character) and so HTTP > > Header injection is still possible is this case: > >

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Karthikeyan Singaravelan added the comment: Okay, the url variable against which the regex check is made is not the full url but the path. The HTTPConnection class sets self.host [0] in the constructor which is used to send the Host header. Perhaps the regex check could be done for the host

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: The commit b7378d77289c911ca6a0c0afaf513879002df7d5 is incomplete: it doesn't seem to check for control characters in the "host" part of the URL, only in the "path" part of the URL. Example: --- try: from urllib import request as urllib_request except

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: Note for myself: Python 2 urllib.urlopen(url) always quotes the URL and so is not vulnerable to HTTP Header Injection (at least, not to this issue ;-)). -- ___ Python tracker

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Roundup Robot : -- pull_requests: +13655 pull_request: https://github.com/python/cpython/pull/13771 ___ Python tracker ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Sihoon Lee : -- pull_requests: +13545 pull_request: https://github.com/python/cpython/pull/12524 ___ Python tracker ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Stéphane Wirtel : -- pull_requests: +13546 pull_request: https://github.com/python/cpython/pull/11768 ___ Python tracker ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: Assigning to Larry to decide if he wants to merge that PR into 3.5 or not. -- assignee: -> larry nosy: +larry ___ Python tracker ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Gregory P. Smith : -- versions: -Python 3.6, Python 3.7 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Gregory P. Smith : -- versions: -Python 2.7 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: New changeset bb8071a4cae5ab3fe321481dd3d73662ffb26052 by Victor Stinner in branch '2.7': bpo-30458: Disallow control chars in http URLs (GH-12755) (GH-13154) (GH-13315) https://github.com/python/cpython/commit/bb8071a4cae5ab3fe321481dd3d73662ffb26052

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: I backported the fix from Python 3.7 to Python 2.7: PR 13315. Please review it carefully, I had to make multiple changes to adapt the fix to Python 2: * non-ASCII characters are explicitly rejected * urllib doesn't reject control characters: they are quoted

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by STINNER Victor : -- pull_requests: +13225 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Charalampos Stratakis added the comment: A small clarification on the differences of those two CVE's. CVE-2019-9740: CLRF sequences are not properly handled in python built-in modules urllib/urllib2 in the query part of the url parameter of urlopen() function CVE-2019-9947: CLRF sequences

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Miro Hrončok : -- pull_requests: +13118 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Ned Deily added the comment: New changeset c50d437e942d4c4c45c8cd76329b05340c02eb31 by Ned Deily (Miro Hrončok) in branch '3.6': bpo-30458: Disallow control chars in http URLs. (GH-12755) (GH-13155) https://github.com/python/cpython/commit/c50d437e942d4c4c45c8cd76329b05340c02eb31 --

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Gregory P. Smith : -- versions: -Python 3.7 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: New changeset 7e200e0763f5b71c199aaf98bd5588f291585619 by Gregory P. Smith (Miro Hrončok) in branch '3.7': bpo-30458: Disallow control chars in http URLs. (GH-12755) (GH-13154)

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Miro Hrončok : -- pull_requests: +13072 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Miro Hrončok : -- pull_requests: +13071 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Kubilay Kocak : -- nosy: +koobs ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Miro Hrončok added the comment: I'll work on 3.7 backport. -- nosy: +hroncok ___ Python tracker ___ ___ Python-bugs-list mailing

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Karthikeyan Singaravelan added the comment: > One thing to note though is that they claim URLs with spaces embedded in them > are apparently somewhat common in the world, we might want to relax our check > to not include space (\x20) in the rejected characters for that reason. Guess I

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: A note from the urllib3 fixes to this: They chose to go the route of auto-%-encoding the offending characters in URLs instead. I do not think the stdlib should do this. One thing to note though is that they claim URLs with spaces embedded in them are

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Karthikeyan Singaravelan added the comment: IMO it does qualify as a security issue. In case of urllib to be lenient and can be exploited it's good to document like tarfile and xml modules that have a warning about untrusted data potentially causing issues and perhaps link to a url

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: New changeset b7378d77289c911ca6a0c0afaf513879002df7d5 by Gregory P. Smith in branch 'master': bpo-30458: Use InvalidURL instead of ValueError. (GH-13044) https://github.com/python/cpython/commit/b7378d77289c911ca6a0c0afaf513879002df7d5 --

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Gregory P. Smith : -- pull_requests: +12964 stage: backport needed -> patch review ___ Python tracker ___ ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Gregory P. Smith : -- stage: patch review -> backport needed ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

miss-islington added the comment: New changeset 2fc936ed24cf04ed32f6015a8aa78c8ea40da66b by Miss Islington (bot) (Xtreak) in branch 'master': bpo-30458: Disable https related urllib tests on a build without ssl (GH-13032)

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Karthikeyan Singaravelan : -- pull_requests: +12953 stage: backport needed -> patch review ___ Python tracker ___ ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Karthikeyan Singaravelan added the comment: Sorry, I will toggle back the issue status. Not sure why bpo didn't warn in this case. -- assignee: gregory.p.smith -> stage: patch review -> backport needed versions: -Python 3.8 ___ Python tracker

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Karthikeyan Singaravelan added the comment: This causes buildbot failure (AMD64 FreeBSD 10-STABLE Non-Debug 3.x and AMD64 Debian root 3.x). I tried debugging and it's reproducible on my mac machine that has python not built with ssl and not reproducible on Ubuntu machine built with ssl.

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by Gregory P. Smith : -- assignee: gregory.p.smith -> stage: patch review -> backport needed ___ Python tracker ___ ___

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: backports to older releases will need to be done manually and take care depending on how much of a concern tightening the existing abusive lenient behavior of the http.client API to enforce what characters are allowed in URLs is to stable releases. I

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: New changeset c4e671eec20dfcb29b18596a89ef075f826c9f96 by Gregory P. Smith in branch 'master': bpo-30458: Disallow control chars in http URLs. (GH-12755) https://github.com/python/cpython/commit/c4e671eec20dfcb29b18596a89ef075f826c9f96 --

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: > urllib3 now vendors a copy of the rfc3986 library: > https://pypi.org/project/rfc3986/ There are multiple Python projects to validate URI: * https://github.com/python-hyper/rfc3986/ -> https://pypi.org/project/rfc3986/ * https://github.com/dgerber/rfc3987

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: "wave Hi! I've noticed that CVE-2019-11236 has been assigned to the CRLF injection issue described here. It seems that the library has been patched in GitHub, but no new release has been made to pypi. (...)" This urllib3 change:

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: It seems like a change has been pushed into urllib3 to fix this issue, but that there is an issue with international URLs and that maybe RFC 3986 should be updated. RFC 3986: "Uniform Resource Identifier (URI): Generic Syntax" (January 2005)

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: > Will this break something in the world other than our own test_xmlrpc test? > Probably. Do they have a right to complain about it? Not one we need listen > to. I understand. But. Can we consider that for old Python versions like Python 2.7 and 3.5?

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Gregory P. Smith added the comment: > *Maybe* we need to provide a way to allow to pass junk characters in an URL? > (disable URL validation) We should not do this in our http protocol stack code. Anyone who _wants_ that is already intentionally violating the http protocol which defeats

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: Since this issue has a long history and previously attempts to fix it failed, it seems like the Internet is a black or white world, more like a scale of gray... *Maybe* we need to provide a way to allow to pass junk characters in an URL? (disable URL

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: > According to the following message, urllib3 is also vulnerable to HTTP Header > Injection: (...) And the issue has been reported to urllib3: https://github.com/urllib3/urllib3/issues/1553 Copy of the first message: """ At

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

STINNER Victor added the comment: bpo-36276 has been marked as a duplicate of this issue. According to the following message, urllib3 is also vulnerable to HTTP Header Injection: https://bugs.python.org/issue36276#msg337837 Copy of Alvin Chang's msg337837: """ I am also seeing the same

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Martin Panter added the comment: Gregory, I haven’t tried recent Python code, but I expect the problem with percent decoding is still there. If you did try my example, what results did you see? Be aware that these techniques only work if the OS co-operates and connects to localhost when you

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Karthikeyan Singaravelan added the comment: As @gregory.p.smith noted in GitHub [0] this fixes only protocol level bugs. There are some parsing ambiguities in urllib that are potential security issues still to be fixed. issue20271 - urllib.urlparse('http://benign.com\[attacker.com]')

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

Change by STINNER Victor : -- title: [CVE-2019-9740][CVE-2019-9947][security] CRLF Injection in httplib -> [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699) ___ Python tracker