[issue22165] Empty response from http.server when directory listing contains invalid unicode
Roundup Robot added the comment: New changeset 1bc41bbbe02d by Ned Deily in branch '3.4': Issue #22165: Skip test_undecodable_filename on OS X prior to 10.5. https://hg.python.org/cpython/rev/1bc41bbbe02d New changeset 85258e08b69b by Ned Deily in branch 'default': Issue #22165: merge from 3.4 https://hg.python.org/cpython/rev/85258e08b69b -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Changes by Senthil Kumaran sent...@uthcode.com: -- resolution: - fixed stage: - resolved ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Changes by Senthil Kumaran sent...@uthcode.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Senthil Kumaran added the comment:
Looks like we hit with an encoding issue, which is due to way os.fsdecode() and
os.listdir() decode the filenames.
support.TESTFN_UNDECODABLE
b'@test_99678_tmp\xe7w\xf0'
dir_list = os.listdir(self.tempdir)
dir_list
['@test_99678_tmp%E7w%F0.txt', 'test']
filename = os.fsdecode(support.TESTFN_UNDECODABLE) + '.txt'
filename
'@test_99678_tmp\udce7w\udcf0.txt'
==
FAIL: test_undecodable_filename (test.test_httpservers.SimpleHTTPServerTestCase)
--
Traceback (most recent call last):
File
/Users/buildbot/buildarea/3.4.murray-snowleopard/build/Lib/test/test_httpservers.py,
line 282, in test_undecodable_filename
.encode('utf-8', 'surrogateescape'), body)
AssertionError: b'href=%40test_62069_tmp%ED%B3%A7w%ED%B3%B0.txt' not found in
b'!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;\nhtml\nhead\nmeta
http-equiv=Content-Type content=text/html; charset=utf-8\ntitleDirectory
listing for tmp0asrs9ei//title\n/head\nbody\nh1Directory listing for
tmp0asrs9ei//h1\nhr\nul\nlia
href=%40test_62069_tmp%25E7w%25F0.txt@test_62069_tmp%E7w%F0.txt/a/li\nlia
href=testtest/a/li\n/ul\nhr\n/body\n/html\n'
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22165
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Changes by Senthil Kumaran sent...@uthcode.com: -- status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Roundup Robot added the comment: New changeset a894b629bbea by Serhiy Storchaka in branch '3.4': Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales. http://hg.python.org/cpython/rev/a894b629bbea New changeset 7cdc941d5180 by Serhiy Storchaka in branch 'default': Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales. http://hg.python.org/cpython/rev/7cdc941d5180 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Serhiy Storchaka added the comment: Oh, I missed that os.listdir() on Mac returns really strange result. Thank you Senthil. Here is a patch which try to workaround this. I'm not sure that it is enough. May be we should fix os.listdir(). Or conclude that this issue can't be fixed on Mac OS. -- components: +Macintosh nosy: +haypo, hynek, ned.deily, ronaldoussoren resolution: fixed - stage: resolved - Added file: http://bugs.python.org/file36390/test_undecodable_filename.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Ronald Oussoren added the comment: OSX returns a strange value in os.listdir because the HFS+ filesystem itself has unicode filenames and transforms byte strings that are assumed to contain UTF-8 into something the filesystem can handle (and seems to replace bytes that aren't valid UTF-8 into a percent-encoded value). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Serhiy Storchaka added the comment: Well, then the workaround should work. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Roundup Robot added the comment: New changeset b05d4f3ee190 by Serhiy Storchaka in branch '3.4': Issue #22165: Fixed test_undecodable_filename on Mac OS. http://hg.python.org/cpython/rev/b05d4f3ee190 New changeset 58e0d2c3ead8 by Serhiy Storchaka in branch 'default': Issue #22165: Fixed test_undecodable_filename on Mac OS. http://hg.python.org/cpython/rev/58e0d2c3ead8 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Senthil Kumaran added the comment: Attached patch looks good to me. If an unittest can be provided for this situation in test_httpservers.py it will be comprehensive and will be good to go. -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Serhiy Storchaka added the comment: Here is a patch with a test. -- assignee: - serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Changes by Serhiy Storchaka storch...@gmail.com: Added file: http://bugs.python.org/file36388/issue22165_2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Roundup Robot added the comment: New changeset f180a9156cc8 by Serhiy Storchaka in branch '3.4': Issue #22165: SimpleHTTPRequestHandler now supports undecodable file names. http://hg.python.org/cpython/rev/f180a9156cc8 New changeset 3153a400b739 by Serhiy Storchaka in branch 'default': Issue #22165: SimpleHTTPRequestHandler now supports undecodable file names. http://hg.python.org/cpython/rev/3153a400b739 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Serhiy Storchaka added the comment: Thank you for the review Senthil. -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
New submission from Josh Lee:
While SimpleHTTPServer from Python2 would happily spit out whatever bytes were
in the directory listing, Python3's http.server logs an error and closes the
connection without responding to the HTTP request.
$ mkdir $'\xff'
$ ls
\377/
$ python3 -m http.server
Serving HTTP on 0.0.0.0 port 8000 ...
Exception happened during processing of request from ('74.125.59.145', 19648)
Traceback (most recent call last):
File /home/josh/local/lib/python3.5/socketserver.py, line 321, in
_handle_request_noblock
self.process_request(request, client_address)
File /home/josh/local/lib/python3.5/socketserver.py, line 347, in
process_request
self.finish_request(request, client_address)
File /home/josh/local/lib/python3.5/socketserver.py, line 360, in
finish_request
self.RequestHandlerClass(request, client_address, self)
File /home/josh/local/lib/python3.5/socketserver.py, line 681, in __init__
self.handle()
File /home/josh/local/lib/python3.5/http/server.py, line 398, in handle
self.handle_one_request()
File /home/josh/local/lib/python3.5/http/server.py, line 386, in
handle_one_request
method()
File /home/josh/local/lib/python3.5/http/server.py, line 677, in do_GET
f = self.send_head()
File /home/josh/local/lib/python3.5/http/server.py, line 716, in send_head
return self.list_directory(path)
File /home/josh/local/lib/python3.5/http/server.py, line 772, in
list_directory
% (urllib.parse.quote(linkname), html.escape(displayname)))
File /home/josh/local/lib/python3.5/urllib/parse.py, line 688, in quote
string = string.encode(encoding, errors)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position
0: surrogates not allowed
--
components: Library (Lib)
messages: 225016
nosy: durin42, jleedev
priority: normal
severity: normal
status: open
title: Empty response from http.server when directory listing contains invalid
unicode
versions: Python 3.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22165
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
R. David Murray added the comment: It should return a server error, I think. -- nosy: +r.david.murray stage: - needs patch type: - behavior versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Augie Fackler added the comment: Why not treat the filename as opaque bytes, and let the client fetch it anyway? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
R. David Murray added the comment: Because http traffic is supposed to be either latin-1 or whatever charset is specified (at least, to my understanding that is the case), so sending incorrectly encoded data seems wrong. On the other hand, we support unix files systems not having well defined charsets, so extending this to directory listings in http isn't crazy. That does raise the question, though, of passing the bytes through python3's string model without breaking anything, so some careful thought may be required. I haven't looked at the details, though, so it might well be pretty simple. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Serhiy Storchaka added the comment: Here is a patch which fixes handling of undecodable paths in SimpleHTTPRequestHandler. -- keywords: +patch nosy: +serhiy.storchaka stage: needs patch - patch review Added file: http://bugs.python.org/file36304/issue22165.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22165] Empty response from http.server when directory listing contains invalid unicode
Changes by Demian Brecht demianbre...@gmail.com: -- nosy: +demian.brecht ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22165 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
