[issue1159051] Handle corrupted gzip files with unexpected EOF

Wummel added the comment: Here is a new test script that works with simple strings and no file objects. It reproduces the error by cutting off the last two bytes of the GZIP data. The resulting struct error is due to the read() methods missing a check that the requested amount of data is actuall

[issue1159051] Handle corrupted gzip files with unexpected EOF

Changes by Serhiy Storchaka : -- assignee: -> serhiy.storchaka nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: At the moment gzip can raise two errors on unexpected EOF: struct.error from struct.unpack() or TypeError from ord(). Both bz2 and lzma raise EOFError in such cases. The proposed patch converts both truncated gzip errors to EOFError as for bz2 and lzma. Add

[issue1159051] Handle corrupted gzip files with unexpected EOF

Nadeem Vawda added the comment: I've reviewed the patch and posted some comments on Rietveld. > I doubt about backward compatibility. It's obvious that struct.error and > TypeError are unintentional, and EOFError is purposed for this case. However > users can catch undocumented but de facto e

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: Here is an updated patch addressing Nadeem Vawda's comments. Thank you. -- Added file: http://bugs.python.org/file28795/gzip_eof-3.4_2.patch ___ Python tracker

[issue1159051] Handle corrupted gzip files with unexpected EOF

Nadeem Vawda added the comment: The updated patch looks good to me. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsu

[issue1159051] Handle corrupted gzip files with unexpected EOF

Roundup Robot added the comment: New changeset 174332b89a0d by Serhiy Storchaka in branch '3.2': Issue #1159051: GzipFile now raises EOFError when reading a corrupted file http://hg.python.org/cpython/rev/174332b89a0d New changeset 87171e88847b by Serhiy Storchaka in branch '3.3': Issue #1159051

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: Actually previous patch doesn't fix original problem, it only ensure that GzipFile consistent with BZ2File and LZMAFile. To fix original problem we need other patch, and this patch looks as new feature for 3.4. Here is a sample patch for LZMAFile. BZ2File pa

[issue1159051] Handle corrupted gzip files with unexpected EOF

Nadeem Vawda added the comment: I think the new behavior should be controlled by a constructor flag, maybe named "defer_errors". I don't like the idea of adding the flag to read(), since that makes us diverge from the standard file interface. Making a distinction between size<0 and size=None seem

[issue1159051] Handle corrupted gzip files with unexpected EOF

Matthias Klose added the comment: this change breaks a test case in the bzr testsuite; will try to get to it next week. See https://launchpad.net/bugs/1116079 -- nosy: +doko, larry priority: normal -> release blocker ___ Python tracker

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: tuned_gzip does dangerous things, it overloads private methods of GzipFile. >From Bazaar 2.3 Release Notes: * Stop using ``bzrlib.tuned_gzip.GzipFile``. It is incompatible with python-2.7 and was only used for Knit format repositories, which haven't been

[issue1159051] Handle corrupted gzip files with unexpected EOF

Changes by Matthias Klose : -- priority: release blocker -> normal ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscr

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: I will be offline some time. Feel free to revert these changes in 2.7-3.3 if it is necessary. -- ___ Python tracker ___ ___

[issue1159051] Handle corrupted gzip files with unexpected EOF

Matthias Klose added the comment: another test case failure with this patch: https://launchpad.net/ubuntu/+archive/test-rebuild-20130329/+build/4416983 reproducible with feedparser 5.1.3 from pypi, on x86 (but not x86_64). ERROR: test_gzip_struct_error (__main__.TestCompression) ---

[issue1159051] Handle corrupted gzip files with unexpected EOF

Serhiy Storchaka added the comment: In both cases broken applications use the undocumented implementation details. But such changes are extremely strong for bugfix release and they should not be done without a special need. I propose to revert these changes in 2.7, 3.2 and 3.3 (possibly leavin

[issue1159051] Handle corrupted gzip files with unexpected EOF

Roundup Robot added the comment: New changeset abc780332b60 by Benjamin Peterson in branch '2.7': backout 214d8909513d for regressions (#1159051) http://hg.python.org/cpython/rev/abc780332b60 -- ___ Python tracker

[issue1159051] Handle corrupted gzip files with unexpected EOF

Roundup Robot added the comment: New changeset 854ba6f414a8 by Georg Brandl in branch '3.2': Issue #1159051: Back out a fix for handling corrupted gzip files that http://hg.python.org/cpython/rev/854ba6f414a8 -- ___ Python tracker

[issue1159051] Handle corrupted gzip files with unexpected EOF

Changes by Georg Brandl : -- priority: release blocker -> normal ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue1159051] Handle corrupted gzip files with unexpected EOF

Changes by Georg Brandl : -- versions: -Python 2.7, Python 3.2, Python 3.3 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue1159051] Handle corrupted gzip files with unexpected EOF

Roundup Robot added the comment: New changeset 9c2831fe84e9 by Georg Brandl in branch '3.3': Back out patch for #1159051, which caused backwards compatibility problems. http://hg.python.org/cpython/rev/9c2831fe84e9 New changeset 5400e8fbc1de by Georg Brandl in branch 'default': null-merge revers

[issue1159051] Handle corrupted gzip files with unexpected EOF

Changes by Mark Lawrence : -- stage: -> needs patch type: -> behavior versions: +Python 2.7, Python 3.1, Python 3.2 ___ Python tracker ___ ___

[issue1159051] Handle corrupted gzip files with unexpected EOF

Daniel Diniz added the comment: Confirmed on trunk with test_gzip_error.py: struct.error: unpack requires a string argument of length 4 -- nosy: +ajaksu2 ___ Python tracker _

[issue1159051] Handle corrupted gzip files with unexpected EOF

Yoav Weiss added the comment: What is the reason that the currently submitted patch is not good enough and current stage is "needs patch"? The current patch seem to solve this issue, which is a very common one when dealing with gzip files coming from the Internet. In any case, an indication on