[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Both. Other programs, and in python scripts when raise() is removed in zipfile.py. Unless your results are different. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment:
Ethan,
I'd refer you to msg92309...
And
When testing with WinZip it looks like this:
No errors detected in compressed data of C:\Downloads\test.zip.
Testing ...
Testing test\OK
Testing test\test2.txt OK
Testing test1.txtOK
Then in python:
Python 3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import zipfile
>>> zf = zipfile.ZipFile('test.zip')
>>> namelist = zf.namelist()
>>> namelist
['test/', 'test/test2.txt', 'test1.txt']
>>> for af in namelist:
... zf.read(af)
...
Traceback (most recent call last):
File "", line 2, in
File "c:\Python34\lib\zipfile.py", line 1117, in read
with self.open(name, "r", pwd) as fp:
File "c:\Python34\lib\zipfile.py", line 1180, in open
% (zinfo.orig_filename, fname))
zipfile.BadZipFile: File name in directory 'test\\' and header b'test/' differ.
So, based on that everything is already converted to forward slashes for the
extraction.
--
___
Python tracker
<http://bugs.python.org/issue6839>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Extraction works fine, the issue was that raise() was creating an exception, and stoping the whole extraction process. When replaced with a warning, everything works fine. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Jim, The problems documented here are related to two cases (both apparently arriving from world of windows): 1. two relative paths with inverted slash in one of them (test\test2.txt vs test/test2.txt) 2. relative path vs absolute path (windows\temp\test.txt vs c:\windows\temp\test.txt) The extraction part seems to be doing a good job at writing the files into sane locations. IMHO, there's no point in trying to replace slashes or otherwise "normalize", as this would fix the cases where the presence of an inverted slashes should be noted in debug output. By the same token stripping the drive letter from the absolute path part would just deprive us from noticing such intricacies in these special zip files. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Is there anything else that you need me to provide? -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Jim, I've got some test cases where the zlib_forward_slash.patch doesn't cut it. That was the reason for trying a broader approach with filename_mismatch patches. -- Added file: http://bugs.python.org/file35120/zipfile_276_filename_mismatch_v3.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: I just looked through 2.7.6 version of zipfile, and the the error handling there is either through using raise() or print(). So, inline with the guidance provided for 2.7.6, perhapswe should stick with print() instead of warning.warn(). I'll post that a bit later. test.zip up there is the test case for this change. Is there any other test case needed? -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file35114/zipfile_340_filename_mismatch_v2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: 3.4.0 pathc with stacklevel=2 -- Added file: http://bugs.python.org/file35117/zipfile_340_filename_mismatch_v3.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Attached is a patch with warnings against 2.7.6 (this one should be good to go) -- Added file: http://bugs.python.org/file35115/zipfile_276_filename_mismatch_v2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file35113/zipfile_276_filename_mismatch_v2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Attached is a patch with warnings against 3.4.0 -- Added file: http://bugs.python.org/file35114/zipfile_340_filename_mismatch_v2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Attached is a patch with warnings against 2.7.6 -- Added file: http://bugs.python.org/file35113/zipfile_276_filename_mismatch_v2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file33666/zipfile_stupid3.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file35103/zipfile_340_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file35104/zipfile_276_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Once again patch against 2.7.6 -- Added file: http://bugs.python.org/file35104/zipfile_276_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: update -- Added file: http://bugs.python.org/file35103/zipfile_340_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file35101/zipfile_276_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file35102/zipfile_340_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Patch against zipfile 3.4.0 attached. -- Added file: http://bugs.python.org/file35102/zipfile_340_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Patch against 2.7.6 attached. -- Added file: http://bugs.python.org/file35101/zipfile_276_filename_mismatch.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: In any event, I think that zipfile_stupid3.patch would be the best trivial fix to this issue. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Terry, I apologize about the second change of headers, somehow I must have used the submission form to post the comment from a tab that had the old content, and the headers didn't refresh there. I assure you that it was not my intention to change them again. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: I've also tested with WinZip, and Windows Explorer, on windows. Both extract the contents of test.zip without a warning (just like 7zip on Windows did). This behavior counts as Denial Of Service if the zipfile Library is used to extract files, besides lots of formats use ZIP as an envelope; DOCX, APK, JAR, EPUB come to mind. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: If I got a file scanner in my mail gateway implemented with this, one can easily avoid getting the contents of zip-files scanned. Is that enough of a security impact? -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Also, this behavior is present on all platforms and all versions of Python (zipfile Library), so maybe the headers should be adjusted there too. I'm not saying that this is necessarily a big freaking hole, but by using this, one can prevent files from being extracted using this simple trick. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: For the version headers, I've added the versions featuring the broken behavior. That's all. I'm not saying that this is I'm extracting malware from the Central Quarantine files, and the vendor's implementation is broken and is causing this issue for me on every single file inside the archive. Let's say, I've got a wrapper script that feeds the contents of a zip file to be scanned with this, because of this behavior, the wrapper will error out... Customers will say your product sucks, etc. Does this really take an act of god to fix this? -- versions: +Python 3.1, Python 3.2, Python 3.3 ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Gentlemen, Is there's any way this fix can be included in any version? Currently, the fact that the exception is thrown makes extracting some zip files impossible with this library, and rolling your own is a bit painful. (either using a wrapper around 7zip to handle those or just provide cloned/patched versions for every major python version). This ridiculous behavior is really not consistent with other ZIP implementations (7zip just ignores the mismatch). Thank you for your time and effort. -- versions: +Python 3.1, Python 3.2, Python 3.5 ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: It might not be a regular "security" issue, but it is not extracting some files that it should. There's a possible scenario, where it can be a security issue. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file33662/zipfile_stupid.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file33663/zipfile_stupid2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Excellent, please see my third attempt. -- Added file: http://bugs.python.org/file33666/zipfile_stupid3.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Can we get this simple "fix" implemented in time for the next 2.7.x release?! Thank you! -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: This one has the parentheses for print, so that it works in python 3.x. Also, the default fallback behavior in this case is to use the filename from the zips' directory (the first path in the warning). -- Added file: http://bugs.python.org/file33663/zipfile_stupid2.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: I'm in a similar situation, my test file raises this: File name in directory "windows\TEMP\\test123.txt" and header "C:\windows\TEMP\\test123.txt" differ. It turns out that I can't find any cross platform procedures for processing the paths between the different platforms. And there are other things like doing it in portable way; os.path.split() nor os.path.basename() won't touch windows paths on un*x, etc... So, I'd like to propose an easy way, just allow the process to extract the files (and print a warning message) rather that just raising an exception (raise BadZipfile,...) and stopping the extraction altogether. -- versions: +Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file33662/zipfile_stupid.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Changes by Adam Polkosnik : Removed file: http://bugs.python.org/file32160/zipfile.py.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: Just tested my patch on mac, and it appears that it didn't work on OSX (and likely on other unix platforms too). Conclusion... os.path.basename() will not do anything to windows paths when running on unix. I'm thinking that instead of bailing at 'File name in directory "%s" and header "%s" differ.', the library should just print a warning, and continue. -- ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6839] zipfile can't extract file
Adam Polkosnik added the comment: I've got bitten by a different variation of this bug. In my case the issue can be summarized by: zipfile.BadZipfile: File name in directory "Windows\TEMP\test.tmp" and header "C:\Windows\TEMP\test.tmp" differ. Attached is a patch for Python27/lib/zipfile.py. I understand that it might not be the best approach, but at least we just compare the filenames without caring much about those pesky paths preceding them. -- nosy: +apolkosnik versions: +Python 2.7 Added file: http://bugs.python.org/file32160/zipfile.py.patch ___ Python tracker <http://bugs.python.org/issue6839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
