[issue6839] zipfile can't extract file

Changes by Alan McIntyre alan.mcint...@gmail.com: -- nosy: -alanmcintyre ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6839 ___ ___ Python-bugs

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre alan.mcint...@gmail.com added the comment: I'd be glad to do some code reviews or something in exchange for the time of somebody with commit rights. :-) If anybody is interested in getting this change committed, please let me know and I'll check that the patch is still valid

[issue11981] dupe self.fp.tell() in zipfile.ZipFile.writestr

Alan McIntyre alan.mcint...@gmail.com added the comment: I also can't see any file operations that might occur between the two .tell() calls, and a full test pass (including test_zipfile64) on the py3k development branch doesn't turn up any problems on Linux (2.6.38, x86_64) for me, so I

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre alan.mcint...@gmail.com added the comment: I re-checked testzip-patch3.diff since some time has passed since I last commented on it, and it still seems to work ok (the small test_zipfile.py block failed to apply, but that's easy enough to do manually). Passes full test run

[issue1757072] Zipfile robustness

Alan McIntyre alan.mcint...@gmail.com added the comment: So far I haven't had the opportunity to sit down and write up a lenient zipfile handling patch; my apologies to those that could really use one. If somebody does propose a patch, I'll be glad to test and review it. I suppose I would

[issue1710703] zipfile.ZipFile behavior inconsistent.

Changes by Alan McIntyre alan.mcint...@gmail.com: Removed file: http://bugs.python.org/file9144/empty-zipfile.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1710703

[issue1710703] zipfile.ZipFile behavior inconsistent.

Alan McIntyre alan.mcint...@gmail.com added the comment: My apologies if Georg was waiting on me to say, Yes. :-) I've attached an updated patch that has the NEWS/doc changes Antoine mentioned. I also just checked that the tests still pass on Linux against the current trunk

[issue1710703] zipfile.ZipFile behavior inconsistent.

Changes by Alan McIntyre alan.mcint...@gmail.com: Removed file: http://bugs.python.org/file18534/zipfile_empty2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1710703

[issue4844] ZipFile doesn't range check in _EndRecData()

Alan McIntyre alan.mcint...@gmail.com added the comment: I had to look up the abbreviation (Easier to Ask Forgiveness than Permission), but that does sound like a good idea. Thanks for mentioning it. :-) -- ___ Python tracker rep...@bugs.python.org

[issue1757072] Zipfile robustness

Alan McIntyre alan.mcint...@gmail.com added the comment: Maybe ZipFile should have an option to handle archives in a non-strict mode, in which it would raise warnings or just completely ignore a small set of minor violations of the spec. That way people that want behavior that's

[issue9720] zipfile writes incorrect local file header for large files in zip64

Alan McIntyre alan.mcint...@gmail.com added the comment: Here's an updated patch for the py3k trunk with tests. This pretty much doubles the runtime of test_zipfile64.py. The patch also removes some unnecessary code from the existing test_zipfile64 tests. Note: It looks like writestr

[issue9720] zipfile writes incorrect local file header for large files in zip64

Alan McIntyre alan.mcint...@gmail.com added the comment: Thanks for the patch, Craig; I should have some time later today or tomorrow to do a review. Did you have a patch for the test suite(s) as well? If not, I can just make sure your test case is covered in test_zipfile64

[issue4844] ZipFile doesn't range check in _EndRecData()

Alan McIntyre alan.mcint...@gmail.com added the comment: I wrote a test for this and tried out the patch on the Python3 trunk, and it seems to work ok. I've attached an updated patch that includes the test. It probably wouldn't hurt to go look for other places where a struct is being

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre alan.mcint...@gmail.com added the comment: Here's an updated patch against the py3k trunk (testzip-patch3.diff) that passes all tests (including test_zipfile64) on Linux on a 64-bit machine. I can backport this to 2.x, but I'll wait until somebody indicates a need for it before

[issue1710703] zipfile.ZipFile behavior inconsistent.

Alan McIntyre alan.mcint...@gmail.com added the comment: Apparenty _EndRecData64 needed the same kind of check that _EndRecData has when trying to seek to the end-of-archive record. So I added that, and everything seems to work correctly now. All tests pass on my 64-bit Linux box (including

[issue1710703] zipfile.ZipFile behavior inconsistent.

Alan McIntyre alan.mcint...@gmail.com added the comment: Sure thing; I'll see if I can have a look within the next week or so. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1710703

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre alan.mcint...@gmail.com added the comment: Ok, I'll see if I can update that in the next week or so. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1739648

[issue1773632] Remove references to _xmlrpclib from xmlrpclib.py

Alan McIntyre alan.mcint...@gmail.com added the comment: Both 2.6 and 2.7 are in the maintenance-only stage at this point, aren't they? I personally don't think this important enough to worry about for 2.x. -- ___ Python tracker rep

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre alan.mcint...@gmail.com added the comment: Attached is a patch that makes the zipfile module check the result of testzip when run as __main__ as well as in test_zipfile.py, and adds some tests in test_zipfile64.py. The changes to test_zipfile64.py increased its runtime from ~671

[issue6839] zipfile can't extract file

Alan McIntyre alan.mcint...@gmail.com added the comment: FileRoller doesn't complain about the mismatched slashes either. Where did the ZIP come from, by the way? I seem to recall that there have been other instances in which ZIP applications were more forgiving than the zipfile module. How

[issue6839] zipfile can't extract file

Alan McIntyre alan.mcint...@gmail.com added the comment: Sorry about the confusion--I think I confused myself by looking at the bit about CRC checksums in the Info-ZIP Unicode Path Extra Field section before I posted. I meant to say that the central directory name looks preferred over the per

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre alan.mcint...@gmail.com added the comment: It would appear that the change to testzip has already been made: http://mail.python.org/pipermail/python-checkins/2008-August/072892.html http://mail.python.org/pipermail/python-3000-checkins/2008-August/004310.html I can add some tests

[issue3535] zipfile has problem reading zip files over 2GB

Alan McIntyre [EMAIL PROTECTED] added the comment: Your patch seems like a better way to detect whether a file is written as Zip64, and it seems to be able to properly handle extracting a 2GB file from a 2GB archive, so I'd vote to include it. I tested it with r66233, using a file made from

[issue3535] zipfile has problem reading zip files over 2GB

Alan McIntyre [EMAIL PROTECTED] added the comment: No, I don't have commit access at the moment. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3535

[issue3317] duplicate lines in zipfile.py

Alan McIntyre [EMAIL PROTECTED] added the comment: Thanks for fixing this, Amaury. I ran the test_zipfile64 and test_zipfile tests on Linux and OS X, and they pass. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3317

[issue3317] duplicate lines in zipfile.py

Alan McIntyre [EMAIL PROTECTED] added the comment: The patch seems to work just fine for me, all tests pass (including test_zipfile64) on an Intel Mac. I'd vote to go ahead and revert the magicXXX variables back to their original names--I'm sure Twisted isn't the only project out

[issue3317] duplicate lines in zipfile.py

Alan McIntyre [EMAIL PROTECTED] added the comment: I don't see a patch attached, but the duplicated code does need removing. If you can attach a patch I'll try it out. As much as I dislike the string names (magicXXX seemed much more descriptive), I suppose they're publicly available

[issue1622] zipfile hangs on certain zip files

Alan McIntyre [EMAIL PROTECTED] added the comment: Sure, I'll look at it later today or over the weekend. I should probably break out the parts that apply to other issues and update those patches as well. __ Tracker [EMAIL PROTECTED] http://bugs.python.org

[issue1622] zipfile hangs on certain zip files

Alan McIntyre [EMAIL PROTECTED] added the comment: Is there anything else that needs to be addressed before this can be committed? At the moment I don't know of anything, just wanted to make sure somebody wasn't waiting on me. As a reminder, issues #1526 and #1746 should be closed

[issue1503] test_xmlrpc is still flakey

Alan McIntyre [EMAIL PROTECTED] added the comment: It's my fault the xmlrpc tests try to use non-blocking sockets. That got added because sometimes failing tests would just sit there with the server blocking until the entire test process got killed for running too long. There are some tests

[issue2138] Factorial

Alan McIntyre added the comment: It seems like most of the methods on integers are for two-argument operations (add, mul, div, etc.), while a lot of single-argument operations are in the math module. If this gets added would it fit better as a function in the math module? I have to say

[issue2138] Factorial

Alan McIntyre added the comment: Except that hypot is not a one-liner, if you want to get edge cases right. Ah, true; thanks for pointing that out. Should there be some upper limit on the argument math.factorial would take? At the moment I can't think of any reason for picking a given limit

[issue1757072] Zipfile robustness

Alan McIntyre added the comment: It would seem that such a zip file is not consistent with the spec (http://www.pkware.com/documents/casestudies/APPNOTE.TXT). My first reaction is that we shouldn't accept behavior outside the spec unless it's something that's done by many popular ZIP

[issue2004] tarfile extractall() allows local attacker to overwrite files while extracting

Alan McIntyre added the comment: I noticed that in the trunk, ZipFile._extract_member, at line 865, still uses 777 (the default of os.makedirs) to create directories. I attached a patch for it. A quick grep shows that tarfile still uses the default permissions for os.makedirs and mkdir

[issue1698917] ZipFile.printdir fix (2.6)

Alan McIntyre added the comment: I think you applied the wrong patch; the referenced revision doesn't include the tests from zipfile-printdir-2.diff. _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1698917

[issue1746] ZIP files with archive comments longer than 4k not recognized as valid by zipfile module

Alan McIntyre added the comment: A fix for this is included in the patch for issue 1622. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1746 __ ___ Python-bugs-list mailing list

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: Here's an updated patch (zipfile-unsigned-fixes2.diff) that contains Eric's fixes. I also changed the structure for the Zip64 extension data to be unsigned. I think this should cover all the deficiencies caused by the improper use of unsigned values. Note

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: I just noticed that my changes for issue 1526 are included in this patch. Eric, if you have time could you have a look at that issue and see if you think I addressed it properly? If not I can back them out into a separate patch for that issue

[issue1526] DeprecationWarning in zipfile.py while zipping 113000 files

Alan McIntyre added the comment: I just noticed that my changes for this issue are included in the patch for issue 1622; if that gets committed then this issue should be closed. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1526

[issue1710703] zipfile.ZipFile behavior inconsistent.

Alan McIntyre added the comment: Here's a quick patch that covers the issues mentioned in my post from 2007-05-09. Added file: http://bugs.python.org/file9144/empty-zipfile.diff _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1710703

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: Currently the extra field data is only parsed when it contains Zip64 extended information. It could probably be broken up into a list of (id, data) pairs at a minimum for all data types, since the spec says that's the structure that should be used. I don't

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: Thanks for the reminder, Eric; I'll include that and post the updated patch. As a side note, on OS X, running regrtest with -uall or -ulargefile still skips test_zipfile4 for some reason. I'll have a look at that before submitting the next version of the patch

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: Here's the first draft of a patch (zipfile-unsigned-fixes.diff) that does a few things: - Interpret fields as unsigned unless required (CRC, etc.) - Corrects reading of ZIP files with large archive comments - Replaces hard-coded structure sizes with module-level

[issue1011893] Problems importing packages in ZIP file

Alan McIntyre added the comment: I just tried it on 2.5 and 2.6, since I didn't have the earlier versions available. If I find some time I'll see if I can try it on 2.3 2.4; maybe that will shed some light on whether it's been fixed or if is something specific to Windows that could still

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: Well I can't promise it will be swift, since my winter vacation ended today, but I'll keep on top of it as best I can. :) __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1622

[issue1526] DeprecationWarning in zipfile.py while zipping 113000 files

Alan McIntyre added the comment: Here's a patch that just uses the mod 64k approach. If I get time to look at some other implementations, and find a better way to handle it, I'll submit an update. Otherwise, maybe on bug day people can try it out with a variety of archiving utilities to see

[issue1622] zipfile hangs on certain zip files

Alan McIntyre added the comment: Based on the ZIP spec (I'm using the one here: http://www.pkware.com/documents/casestudies/APPNOTE.TXT), I'm inclined to agree. There's a general note that says All fields unless otherwise noted are unsigned and stored in Intel low-byte:high-byte, low-word:high

[issue1011893] Problems importing packages in ZIP file

Alan McIntyre added the comment: I just tried out this test with the trunk and 2.5.2 from svn (on OS X, not sure if that might make a difference), and didn't get the reported error. Since the issue has apparently been fixed at some point, is there any reason not to close this issue

[issue774221] 2.3c1: zipfile.py confused by garbage at the very end of zip

Alan McIntyre added the comment: Since there's no reply about where ZIP files with appended 'garbage' might come from, should this be closed? I agree with amk; it doesn't seem wise to ignore incorrectly formatted files without a good reason. On a related note, it seems that correctly formatted

[issue1746] ZIP files with archive comments longer than 4k not recognized as valid by zipfile module

New submission from Alan McIntyre: The current behavior of zipfile._EndRecData is to look in the last 4k of data in a file if it appears it might have an archive comment; this results in flagging proper ZIP files with comments longer than ~4074 bytes as not a ZIP file. I plan on posting a patch

[issue1526] DeprecationWarning in zipfile.py while zipping 113000 files

Alan McIntyre added the comment: Oh thanks, I meant to ask whether or not the run time was too long, but forgot. Only running when -ulargefile is enabled seems fine to me. I can tweak the patch for that if you'd like; moving it to test_zipfile64 should do that, right

[issue1526] DeprecationWarning in zipfile.py while zipping 113000 files

Alan McIntyre added the comment: The reported warning was being produced when writing the end of central directory record, in ZipFile.close(). Based on a little experiment with 70k test text files, the default archiver in OS X appears to just use the number of files mod 64k in the end

[issue1739648] zipfile.testzip() using progressive file reads

Alan McIntyre added the comment: This sort of change definitely needs to be made to avoid reading huge files into memory for testzip(). I've attached a modified patch as zipfile_testzip_amcintyre.diff; it didn't seem appropriate to have a module-level variable for the size of the test read

[issue683910] zipfile should have a tarfile-like API

Alan McIntyre added the comment: I just posted a patch on issue 467924 (http://bugs.python.org/issue467924) that adds extract and extractall methods. Recommend closing this issue unless the patch over there is insufficient for some reason. -- nosy: +alanmcintyre

[issue467924] Improve the ZipFile Interface

Alan McIntyre added the comment: I attached a patch with the following changes (as zipfile_extract.diff): (1) Add a note to the docs (under writestr) about how the compression is selected if a ZipInfo is passed as the zinfo_or_arcname parameter. If anybody thinks it's a good idea to add

[issue467924] Improve the ZipFile Interface

Alan McIntyre added the comment: Are the method renames/additions suggested in the original issue worth doing? When I first started using this module, I found the documentation easy and thorough enough to understand how to use it, so I would vote for just leaving the ZipFile interface the way

[issue1760357] ZipFile.write fails with bad modification time

Alan McIntyre added the comment: At the moment I don't have a Windows machine available, but on my Mac, time.localtime doesn't seem to mind interpreting negative input values. So I doubt that forcing timestamps to be non-negative is the way to fix this. I poked around a bit in the current 2.6

[issue467924] Improve the ZipFile Interface

Alan McIntyre added the comment: There was another issue that also asked for an extract feature, and if I recall correctly I said I'd try to work on it (I think I have some code somewhere for it but I'll have to look). Tonight or tomorrow I will see if I can find that other issue and let you

[issue1381] cmath is numerically unsound

Alan McIntyre added the comment: I have to review a few complex math topics for some of my current course work, so I wouldn't mind taking a look into this. I can't promise I'll have the time required to make all of cmath correct (assuming it's as unsound as claimed), but I'll do what I can

[issue1201] Error in array concept

Alan McIntyre added the comment: The last character is n-1, but the section you quote says the *right edge* of the last character (not the last character itself) has index n; this seems correct in the context of the mnemonic scheme. -- nosy: +alanmcintyre

[issue1134] Parsing a simple script eats all of your memory

Alan McIntyre added the comment: Confirmed that this happens on Mac OS X with a fresh build of py3k from svn. -- nosy: +alanmcintyre __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1134

Subscription donations to PSF

or otherwise communicate this interest to the PSF so they have a feel for how many people would take advantage of the donation option if they offered it. For those not familiar with the PSF, plenty of info can be found here: http://python.org/psf/ Thanks! Alan McIntyre -- http

Re: changing __call__ on demand

I tried this: class test(object): ... def __call1(self): ... print 1 ... __call__ = __call1 ... t = test() t() 1 Is that what you were looking for? -- Alan McIntyre ESRG LLC http://www.esrgtech.com Stefan Behnel wrote: Hi! This somewhat puzzles me: Python 2.4 (#1, Feb 3 2005

Re: changing __call__ on demand

Thanks; I didn't read close enough. :) -- Alan McIntyre ESRG LLC http://www.esrgtech.com Michael Hoffman wrote: Alan McIntyre wrote: class test(object): ...def __call1(self): ...print 1 ...__call__ = __call1 Is that what you were looking for? That still only allows him to have one

Re: listerator clonage

of. Hope this helps, Alan McIntyre ESRG LLC http://www.esrgtech.com Cyril BAZIN wrote: Hello, I want to build a function which return values which appear two or more times in a list: So, I decided to write a little example which doesn't work: #l = [1, 7, 3, 4, 3, 2, 1] #i = iter(l) #for x in i: #j

Re: Lambda

, Alan McIntyre ESRG LLC http://www.esrgtech.com -- http://mail.python.org/mailman/listinfo/python-list

Re: Collapsing a list into a list of changes

Thanks to everybody that responded; I appreciate all the input, even if I didn't respond to all of it individually. :) -- http://mail.python.org/mailman/listinfo/python-list

Re: Collapsing a list into a list of changes

Steve, Yeah, in this particular application the ordering and reoccurrence of a value in a non-contiguous way does matter; if those two things weren't required I think the method you suggested would be a good way to remove the duplicates. Thanks! Coates, Steve (ACHE) wrote: It's not _exactly_

Re: Collapsing a list into a list of changes

Tony, Actually I only want to remove a certain kind of duplication; if an item occurs twice - say like this: [1,1,1,2,2,2,1,1,1], then I need to keep the order and occurrence of the individual values: [1,2,1]. Using a dict as you proposed loses the order of occurrence, as well as multiple

Re: Collapsing a list into a list of changes

Alex, Wow, that method turns out to be the fastest so far in a simple benchmark on Python2.3 (on my machine, of course, YMMV); it takes 14% less time than the one that I deemed most straightforward. :) Thanks, Alan Alex Martelli wrote: H, what role does the enumeration play here? I don't

Re: Trouble converting hex to decimal?

Earl, Try this: ord('\x00') 0 or: import struct struct.unpack('b', '\x00') (0,) If you're needing to pull values out of multiple bytes (shorts, longs, floats, etc.), have a look at the struct module. Here's an example: struct.unpack('f', '\x00\x00(B') (42.0,) Hope this helps, Alan Earl

Re: Collapsing a list into a list of changes

Your first example is along the lines of what I was thinking when I said elegant. :) I was looking for something that I could drop into one or two lines of code; I may not do that if I'm writing code that will have to be maintained, but it's still nice to know how to do it. Thanks :) Alan

Re: Collapsing a list into a list of changes

I think you're right; sometimes I'm susceptible to the I can do that in one line of code temptation. :) Since this current bit of code will probably end up in something that's going to be maintained, I will probably stick with the straightforward method just to be nice to the maintainer

Re: Collapsing a list into a list of changes

Jack, I'm not using 2.4 yet; still back in 2.3x. :) Thanks for the examples, though - they are clear enough that I will probably use them when I upgrade. Thanks, Alan Jack Diederich wrote: If you are using python2.4, import itertools as it [x[0] for (x) in

Re: empty classes as c structs?

Christopher, I've found myself doing the same thing. You could do something like this: blah = type('Struct', (), {})() blah.some_field = x I think I'd only do this if I needed to construct objects at runtime based on information that I don't have at compile time, since the two lines of code for

Re: string issue

) print ips_new Or this: ips_new = [ip for ip in ips if '255' not in ip] print ips_new Hope this helps, Alan McIntyre http://www.esrgtech.com rbt wrote: Either I'm crazy and I'm missing the obvious here or there is something wrong with this code. Element 5 of this list says it doesn't contain

Re: string issue

Wow, that's cool; I'd never seen that before. :) Thanks, Steve.. Steve Holden wrote: You are modifying the list as you iterate over it. Instead, iterate over a copy by using: for ip in ips[:]: ... regards Steve -- http://mail.python.org/mailman/listinfo/python-list