Re: [Python-Dev] Mysterious Python pyc file corruption problems
On Thu, 16 May 2013 11:42:30 -0400 Barry Warsaw ba...@python.org wrote: On May 16, 2013, at 08:33 AM, Nick Coghlan wrote: Personally, I would be suspicious of developmental web services doing auto-reloading while an installer is recompiling the world. I don't have enough context to be sure how plausible that is as a possible explanation, though. It's possible that some Python written system service is getting invoked during the bytecompile-on-installation phase. But now that we've found cases of this problem with Python 3.3, I'm less inclined to follow that line of reasoning since it implements atomic rename. Please try to reproduce it by adding e.g. some sleep() calls in the middle of the writing routine. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
Hi all, How about using the shared-or-exclusive advisory file locks (with flock() or fcntl())? It may only work on Posix though. A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] HAVE_FSTAT?
Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks unconditional? It would avoid having to maintain unused fallback paths. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HAVE_FSTAT?
2013/5/17 Antoine Pitrou solip...@pitrou.net: Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks unconditional? It would avoid having to maintain unused fallback paths. +1 (Maybe Snakebite has such an exotic system, though?) :) -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HAVE_FSTAT?
Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif Are there other guards for similarly common libc functions? If so, perhaps each one should be removed in a series of change sets, one per guard. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HAVE_FSTAT?
On Fri, 17 May 2013 09:15:29 -0500 Skip Montanaro s...@pobox.com wrote: Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif Are there other guards for similarly common libc functions? I don't think so. Someone should take a look though :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-05-10 - 2013-05-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open3966 ( +3) closed 25805 (+47) total 29771 (+50) Open issues with patches: 1776 Issues opened (32) == #17487: wave.Wave_read.getparams should be more user friendly http://bugs.python.org/issue17487 reopened by Claudiu.Popa #17807: Generator cleanup without tp_del http://bugs.python.org/issue17807 reopened by pitrou #17905: Add check for locale.h http://bugs.python.org/issue17905 reopened by pitrou #17951: TypeError during gdb backtracing http://bugs.python.org/issue17951 opened by Catalin.Patulea #17953: sys.modules cannot be reassigned http://bugs.python.org/issue17953 opened by Valentin.Lorentz #17955: Minor updates to Functional HOWTO http://bugs.python.org/issue17955 opened by akuchling #17956: add ScheduledExecutor http://bugs.python.org/issue17956 opened by neologix #17957: remove outdated (and unexcellent) paragraph in whatsnew http://bugs.python.org/issue17957 opened by tshepang #17960: Clarify the required behaviour of locals() http://bugs.python.org/issue17960 opened by ncoghlan #17961: Use enum names as values in enum.Enum convenience API http://bugs.python.org/issue17961 opened by ncoghlan #17963: Deprecate the frame hack for implicitly getting module details http://bugs.python.org/issue17963 opened by ncoghlan #17967: urllib2.open failed to access a url when a perent directory of http://bugs.python.org/issue17967 opened by foxkiller #17969: multiprocessing crash on exit http://bugs.python.org/issue17969 opened by kristjan.jonsson #17970: Mutlithread XML parsing cause segfault http://bugs.python.org/issue17970 opened by mrDoctorWho0.. #17972: inspect module docs omits many functions http://bugs.python.org/issue17972 opened by s7v7nisla...@gmail.com #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 opened by pitrou #17975: libpython3.so conflicts between $VERSIONs http://bugs.python.org/issue17975 opened by prlw1 #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 opened by jasujm #17978: Python crashes if Py_Initialize/Py_Finalize are called multipl http://bugs.python.org/issue17978 opened by romuloceccon #17979: Cannot build 2.7 with --enable-unicode=no http://bugs.python.org/issue17979 opened by amaury.forgeotdarc #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 opened by fweimer #17984: io and _pyio modules require the _io module http://bugs.python.org/issue17984 opened by serhiy.storchaka #17985: multiprocessing Queue.qsize() and Queue.empty() with different http://bugs.python.org/issue17985 opened by aod #17986: Alternative async subprocesses (pep 3145) http://bugs.python.org/issue17986 opened by sbt #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 opened by fdrake #17988: ElementTree.Element != ElementTree._ElementInterface http://bugs.python.org/issue17988 opened by jwilk #17989: ElementTree.Element broken attribute setting http://bugs.python.org/issue17989 opened by jwilk #17991: ctypes.c_char gives a misleading error when passed a one-chara http://bugs.python.org/issue17991 opened by Steven.Barker #17994: Change necessary in platform.py to support IronPython http://bugs.python.org/issue17994 opened by icordasc #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 opened by giampaolo.rodola #17997: ssl.match_hostname(): sub string wildcard should not match IDN http://bugs.python.org/issue17997 opened by christian.heimes #17998: internal error in regular expression engine http://bugs.python.org/issue17998 opened by jdemeyer Most recent 15 issues with no replies (15) == #17998: internal error in regular expression engine http://bugs.python.org/issue17998 #17997: ssl.match_hostname(): sub string wildcard should not match IDN http://bugs.python.org/issue17997 #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 #17994: Change necessary in platform.py to support IronPython http://bugs.python.org/issue17994 #17991: ctypes.c_char gives a misleading error when passed a one-chara http://bugs.python.org/issue17991 #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 #17986: Alternative async subprocesses (pep 3145) http://bugs.python.org/issue17986 #17975: libpython3.so conflicts between $VERSIONs http://bugs.python.org/issue17975 #17942: IDLE Debugger: names, values misaligned http://bugs.python.org/issue17942 #17933: test_ftp failure / ftplib error formatting issue http://bugs.python.org/issue17933 #17924: Deprecate stat.S_IF* integer constants
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On May 16, 2013, at 02:19 PM, Guido van Rossum wrote: Now consider the following scenario. It involves *three* processes. - Two unrelated processes both start and want to import the same module. - They both see the .pyc file is missing/corrupt and decide to write it. - The first process finishing writing the file, writing the correct header. - Now a third process wants to import the module, sees the valid header, and starts reading the file. - However, while this is going on, the second process gets ready to write the file. - The second process truncates the file, writes the dummy header, and then stalls. - At this point the third process (which thought it was reading a valid file) sees an unexpected EOF because the file has been truncated. Now, this would explain the EOFError, but not necessarily the ValueError with unknown type code. However, it looks like marshal doesn't always check for EOF immediately (sometimes it calls getc() without checking the result, and sometimes it doesn't check the error state after calling r_string()), so I think all the errors are actually explainable from this scenario. Thanks for this, it's a very interesting scenario. I think this isn't a complete explanation of what's going on though. I've spoken with our defect analyst and looked at a bunch of the bug reports, and as far as we can tell, the corruptions are permanent. Users generally have to take manual action to delete the .pyc files and re-create them. One thing I hadn't realized until now is that until Python 3.4, py_compile.py doesn't write the pyc files atomically, and in fact this is the mechanism we're using to create the pyc files at package installation time. That could explain why we're still seeing these issues even in Python 3.3. I've also uncovered a bug from 2010 reported in Debian[1] about pyc file corruptions that happened when the byte-compilation driver program exited before its workers[2] could complete. We're definitely seeing issues post-landing of this fix, so I need to do some more analysis to see if that fix was enough. If it wasn't, and we're not doing atomic renames, than that could explain the permanent corruptions. Cheers, -Barry [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=590224 [2] the workers each call calling `$PYTHON -m py_compile - py-filenames` signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On May 16, 2013, at 11:48 PM, Tres Seaver wrote: I can confirm at least that I have seen this problem within the last two weeks on Ubuntu boxes unrelated to the thw Debian / Ubuntu build infrastruction. Hi Tres. If you see this happen, *please* get in touch with me, preferably before you fix it ;). I'd like to do some additional analysis on a broken system in semi-realtime. Cheers, - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJRllovAAoJEBJutWOnSwa/+kgP/iyqwiKJvSszzPITSSU8lOps Ll75gWXyyrLyQ/NBp4neo+d7hsf/GqLT3wn6dc7dcq23khqo4c6tYlmdVaZ5QJGk qu9BjSPdKLdDQFt3k3MyHpjJKwOa6Fn/JmCyZnoPd2zST/RZUsL6kCBOosG6FGTU QJUeEM/GIv5wQ60tfdVmI3zBtYYkjfrCX6uHcw2xFt1JHAKKgH/rNh72t2q3YKKG yqVAbyxQyjxEkS7IQmacPPSy7sEbT5GD8xR9F/P6w5h05DaTqbau9haF2kyxhPvI wEP0UQZWsZ/QmMNaS9w/WcWx31ASCNR4NIYqdWYd5KHQqs7Y1Vf6sFW1hNhfPFR9 0C4wJhublK1ewKYf5AjHeLOEVddN1xNGqNZr/7FouOqNlDYwRObde8J0FONWzHkl GV6qqxvKnq4FI4Y+EtUaRS52j3NkMhD22bpkTf2EJij1LV80AsHeB01o/ZBUjTqu jmgJ2QLS34h3gGIU9+OgE9rtzTACdI783SQ5827hiCVaE62IhTgrkLGBNSv1WFxp Onc+E2YteEwHtYMsBK1ck8YQkbgC/XYpSAVZGoW8xbXy5/+33fQRUZPj9zXNBW2F Jzgo+mGqPC5SIKkaiZY22GYERC6bvwCsoV9o+DpwmCUhwT/2ATb7IAp+8Vembjit JqMBWyvdEg4uzjnDuAYZ =93Og -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On May 17, 2013, at 12:10 AM, Thomas Wouters wrote: The 'unknown type codes' can also be explained if the two processes writing to the .pyc files are *different Python versions*. As you may recall, at Google we used to use modified Python interpreters that used '.pyc-2.2', '.pyc-2.4', etc, for the pyc extension. That was because otherwise different Python versions would keep overwriting the .pyc files of shared Python modules, and at Google scale it caused all manner of problems... I guess Ubuntu is approaching Google scale ;-) I'd like to think so. :) But I don't think this is part of the equation. For Python 2 on Debian/Ubuntu, we use an elaborate symlink farm to keep all pyc files in Python-version-specific directories. The stdlib files are already segregated, but the symlink farm takes care of package installs. Note that the symlinks are to the .py files, not the pyc files. Fortunately we almost don't care about this anymore. We dropped 2.6 in Ubuntu a while ago and we'll very likely drop 2.6 in Debian Jessie. We don't care about any Python 3s earlier than 3.2, and getting rid of the symlink farm was the primary motivator for PEP 3147. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On May 16, 2013, at 05:30 PM, Brett Cannon wrote: Just so people know, this is how we used to do it. In importlib we write the entire file to a temp file and then to an atomic rename. Yep, and I suspect that our fix, even if we don't completely identify the root cause, will be to change py_compile.py to do atomic renames. Whether that would be an appropriate fix for 3.2, 3.3 and 2.7 is a different discussion. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that asking for reports on other systems is part of the reason you posted, but I don't remember seeing any others yet. Right. :) It's harder to dig out similar problems in Debian[1] but it's pretty clear that there have been *some* similar reports in Debian. Ubuntu and Debian share almost all their Python infrastructure. It would definitely be interesting to whether Fedora/RedHat or any other Linux distros have seen similar problems. I don't know how Fedora/RH does package installation. In Debian/Ubuntu, we do not ship pyc files, but instead they are generated in post-installation scripts, which boil down to calls to `$PYTHON -m py_compile - filenames`. Do failures only occur during compileall process? (or whatever substitute you use). No, they are all post-installation failures in unrelated packages that try to import pure-Python modules. AFAICT, the post-installation byte-compilation scripts are not erroring. Doing a post-compilation verification step might be interesting, but I bet backporting atomic renames to py_compile.py will fix the problem, or at least band-aid over it. ;) -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On Fri, 2013-05-17 at 12:42 -0400, Barry Warsaw wrote: On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that asking for reports on other systems is part of the reason you posted, but I don't remember seeing any others yet. Right. :) It's harder to dig out similar problems in Debian[1] but it's pretty clear that there have been *some* similar reports in Debian. Ubuntu and Debian share almost all their Python infrastructure. It would definitely be interesting to whether Fedora/RedHat or any other Linux distros have seen similar problems. FWIW I don't recall seeing such problems on Fedora/RH, though that could be due to... I don't know how Fedora/RH does package installation. In Debian/Ubuntu, we do not ship pyc files, but instead they are generated in post-installation scripts, which boil down to calls to `$PYTHON -m py_compile - filenames`. Fedora/RH pregenerate the .pyc files during rpm creation, and they exist as part of the rpm payload. Dave ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On May 17, 2013, at 01:19 PM, David Malcolm wrote: Fedora/RH pregenerate the .pyc files during rpm creation, and they exist as part of the rpm payload. Good to know, thanks. Do you use `$PYTHON -m py_compile` to generate the pyc files at build time? -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/17/2013 12:26 PM, Barry Warsaw wrote: On May 16, 2013, at 11:48 PM, Tres Seaver wrote: I can confirm at least that I have seen this problem within the last two weeks on Ubuntu boxes unrelated to the thw Debian / Ubuntu build infrastruction. Hi Tres. If you see this happen, *please* get in touch with me, preferably before you fix it ;). I'd like to do some additional analysis on a broken system in semi-realtime. Wilco (although I don't know for sure what provoked it: my memory is that it was while running 'tox' or 'detox' for ZODB). Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGWfCAACgkQ+gerLs4ltQ5YcQCguzlxAP8InrLEgdGx7JiK0as4 z9MAnR53bubpntt+272Y0BNYlEO8YcdI =LSAR -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On 5/17/2013 12:42 PM, Barry Warsaw wrote: On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: Do failures only occur during compileall process? (or whatever substitute you use). No, they are all post-installation failures in unrelated packages that try to import pure-Python modules. What I mean is, is the corruption (not the detection of corruption) only happening during mass compilation of the stdlib? When user imports a single non-stdlib file he has written the first time, does that ever get corrupted. AFAICT, the post-installation byte-compilation scripts are not erroring. I THINK that you are answering my question by saying that corruption only happens during installation mass compilation. Doing a post-compilation verification step might be interesting, but I bet backporting atomic renames to py_compile.py will fix the problem, or at least band-aid over it. ;) I intended to suggest that py_compile be changed to do that. Then Brett said it already had for 3.4. I see no reason why not to backport, but maybe someone else will. The main design use of marshal is to produce .pyc files that consist of a prefix and marshalled codeobject. Perhaps marshal.dump(s) should be extended to take a third prefix='' parameter that would be prepended to the result as produced today. .dump first does .dumps, though inline. I assume that .dumps constructs a string by starting with [], appending pieces, and joining. At least, any composite object dump would. The change would amount to starting with [prefix] instead of []. Then py_compile would amount to pyc = pyc-prefix marshal.dump(codeobject, file, pyc) Terry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On May 17, 2013, at 03:02 PM, Terry Jan Reedy wrote: What I mean is, is the corruption (not the detection of corruption) only happening during mass compilation of the stdlib? When user imports a single non-stdlib file he has written the first time, does that ever get corrupted. It's not limited to the stdlib, but yes, as far as we can tell, it happens during package installation on an end-user system, when the trigger script mass byte-compiles a package's Python source files. I intended to suggest that py_compile be changed to do that. Then Brett said it already had for 3.4. I see no reason why not to backport, but maybe someone else will. I tend to agree. The main design use of marshal is to produce .pyc files that consist of a prefix and marshalled codeobject. Perhaps marshal.dump(s) should be extended to take a third prefix='' parameter that would be prepended to the result as produced today. .dump first does .dumps, though inline. I assume that .dumps constructs a string by starting with [], appending pieces, and joining. At least, any composite object dump would. The change would amount to starting with [prefix] instead of []. Then py_compile would amount to pyc = pyc-prefix marshal.dump(codeobject, file, pyc) That wouldn't be a backportable change, and right now I'm trying to stay focused on fixing this specific problem. ;) -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On Fri, 2013-05-17 at 14:23 -0400, Barry Warsaw wrote: On May 17, 2013, at 01:19 PM, David Malcolm wrote: Fedora/RH pregenerate the .pyc files during rpm creation, and they exist as part of the rpm payload. Good to know, thanks. Do you use `$PYTHON -m py_compile` to generate the pyc files at build time? We use compileall.compiledir() most of the time, but occasionally use py_compile.compile() Specifically, for python 2, the core rpm-build package has a script: /usr/lib/rpm/brp-python-bytecompile run automatically in a postprocessing phase after the upstream source has installed to a DESTDIR, and this invokes compileall.compiledir() on all .py files in the package payload, with various logic to segment the different parts of the filesystem to be bytecompiled by the appropriate python binary (since we have duplicate .py files for different python runtimes). This is all done sequentially, so I'd be surprised if different pythons splatted on each other's .pyc files at this time. In addition, python3-devel contains a: /etc/rpm/macros.pybytecompile which defines a py_byte_compile() macro, which can be used for overriding these rules (IIRC), and this does use pycompile.compile() Hope this is helpful Dave ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
Am 15.05.2013 22:58, schrieb Barry Warsaw: I am looking into a particularly vexing Python problem on Ubuntu that manifests in several different ways. I think the problem is the same one described in http://bugs.python.org/issue13146 and I sent a message on the subject to the ubuntu-devel list: https://lists.ubuntu.com/archives/ubuntu-devel/2013-May/037129.html please consider that Ubuntu does have some other upgrade issues, when files in the dpkg database (/var/lib/dpkg/info/*) are corrupted, or just are files having null bytes. So these and the pyc issues share in common that these files are written after a package is unpacked. I'm not saying that the problem might be with the pyc writing, but we do see other file corruption as well. Matthias ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2
On Thu, 16 May 2013 13:24:36 +0200 Charles-François Natali cf.nat...@gmail.com wrote: 2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax modules. Details can be found in the changelogs: It seems that I'm the main culprit of this releases. Well, when I look at the changelogs, what strikes me more is that you're the author of *many* fixes, and also a lot of new features/improvements. So I wouldn't feel bad if I were you, this kind of things happens (and it certainly did to me). Seconded. Thanks Serhiy for your contributions :=) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mysterious Python pyc file corruption problems
On Sat, May 18, 2013 at 3:19 AM, David Malcolm dmalc...@redhat.com wrote: On Fri, 2013-05-17 at 12:42 -0400, Barry Warsaw wrote: On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that asking for reports on other systems is part of the reason you posted, but I don't remember seeing any others yet. Right. :) It's harder to dig out similar problems in Debian[1] but it's pretty clear that there have been *some* similar reports in Debian. Ubuntu and Debian share almost all their Python infrastructure. It would definitely be interesting to whether Fedora/RedHat or any other Linux distros have seen similar problems. FWIW I don't recall seeing such problems on Fedora/RH, though that could be due to... I don't know how Fedora/RH does package installation. In Debian/Ubuntu, we do not ship pyc files, but instead they are generated in post-installation scripts, which boil down to calls to `$PYTHON -m py_compile - filenames`. Fedora/RH pregenerate the .pyc files during rpm creation, and they exist as part of the rpm payload. So in effect, we'd always be doing an atomic rename (and file conflicts between RPMs wouldn't be allowed in the first place). Combined with Brett's info that even 3.3 doesn't use atomic renames for pre-compilation (only for implicit compilation), I consider that strong evidence in favour of Guido's theory that Debian are getting write conflicts somewhere. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2
On Sat, May 18, 2013 at 7:59 AM, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 16 May 2013 13:24:36 +0200 Charles-François Natali cf.nat...@gmail.com wrote: 2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax modules. Details can be found in the changelogs: It seems that I'm the main culprit of this releases. Well, when I look at the changelogs, what strikes me more is that you're the author of *many* fixes, and also a lot of new features/improvements. So I wouldn't feel bad if I were you, this kind of things happens (and it certainly did to me). Seconded. Thanks Serhiy for your contributions :=) Indeed! Any need for quick releases to address regressions is always a collective failure - for them to happen, the error has to be in something not checked by our test suite, and the code change has to be one where nobody monitoring python-checkins spotted a potential issue. Hopefully the fixes for these regressions also came with new test cases (although that is obviously difficult for upstream regressions like those in the PyOpenSSL release bundled with the original Windows binaries). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2
Am 18.05.2013 05:21, schrieb Nick Coghlan: On Sat, May 18, 2013 at 7:59 AM, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 16 May 2013 13:24:36 +0200 Charles-François Natali cf.nat...@gmail.com wrote: 2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax modules. Details can be found in the changelogs: It seems that I'm the main culprit of this releases. Well, when I look at the changelogs, what strikes me more is that you're the author of *many* fixes, and also a lot of new features/improvements. So I wouldn't feel bad if I were you, this kind of things happens (and it certainly did to me). Seconded. Thanks Serhiy for your contributions :=) Indeed! Any need for quick releases to address regressions is always a collective failure - for them to happen, the error has to be in something not checked by our test suite, and the code change has to be one where nobody monitoring python-checkins spotted a potential issue. Hopefully the fixes for these regressions also came with new test cases (although that is obviously difficult for upstream regressions like those in the PyOpenSSL release bundled with the original Windows binaries). Exactly. Thanks Serhiy for making us improve the test suite :) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
