Re: [Python-Dev] Marking GC details as CPython-only
Le Wed, 13 Feb 2013 20:50:51 +, Richard Oudkerk a écrit : > On 13/02/2013 7:25pm, Antoine Pitrou wrote: > > I think resurrecting objects from __del__ is crazy, so IMO what you > > suggest is fine. > > You mean like subprocess.Popen.__del__? I quite agree. Ouch. I didn't know about that. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] build from source on mac uses clang, python.org binary uses gcc
Hi All, I've run into "some issues" installing lxml for python 3.3 on my mac: One of the stumbling blocks I've hit is that I built python 3.3 from source (./configure && make && make altinstall), and it used clang: buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe Python 3.3.0 (default, Jan 23 2013, 09:56:03) [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> ...which makes lxml pretty unhappy. However, when I install the binary Python 3.3 from python.org, it's built with GCC: buzzkill:virtualenvs chris$ python3.3 Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> Why the difference? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build from source on mac uses clang, python.org binary uses gcc (missing links)
On 14/02/2013 09:18, Chris Withers wrote: Hi All, I've run into "some issues" installing lxml for python 3.3 on my mac: (forgot the links) https://mailman-mail5.webfaction.com/pipermail/lxml/2013-February/006730.html https://mailman-mail5.webfaction.com/pipermail/lxml/2013-February/006731.html cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] efficient string concatenation (yep, from 2004)
Hi Greg, On Wed, Feb 13, 2013 at 10:17 PM, Greg Ewing wrote: > If it's that unreliable, why was it ever implemented > in the first place? I was young and loved hacks and python-dev felt that it was a good idea at the time (http://mail.python.org/pipermail/python-dev/2004-August/046686.html). A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build from source on mac uses clang, python.org binary uses gcc
On 14 Feb, 2013, at 10:18, Chris Withers wrote: > Hi All, > > I've run into "some issues" installing lxml for python 3.3 on my mac: > > > One of the stumbling blocks I've hit is that I built python 3.3 from source > (./configure && make && make altinstall), and it used clang: > > buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe > Python 3.3.0 (default, Jan 23 2013, 09:56:03) > [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> > > ...which makes lxml pretty unhappy. Why is that? Clang is a working C and C++ compiler. I've noticed in the past that lxml's automatic compilation of libxml2 and libxslt is dodgy on OSX, but haven't had time yet to work on a patch that calculates the right compiler flags instead of hardcoding some guesses in setup.py. Also, libxml2 won't compile on OSX 10.8 at all unless you use the version in the repository (an incompatibility with the pthread headers on 10.8). You get clang instead of GCC when configure detects that clang is the only usable compiler on OSX, where usable means that the compiler is present and is not llvm-gcc. Llvm-gcc contains bugs that miscompile at least parts of the unicode implementation in CPython 3.3 and word on the street is that these bugs won't get fixed unless they are also present in clang. > > However, when I install the binary Python 3.3 from python.org, it's built > with GCC: > > buzzkill:virtualenvs chris$ python3.3 > Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> > > Why the difference? The binary was created on an older release of OSX where gcc is a valid compiler choice. Distutils should still pick clang when it detects that gcc is actually llvm-gcc though, doesn't it do that on your system? BTW. Which OSX release are you using? And which variant of the OSX installer? Ronald P.S. Isn't this a question for python-list? > > cheers, > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting >- http://www.simplistix.co.uk > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Status of the PEP 433?
Hi, I read again all emails related to PEP 433. There are different opposition to this change and I tried to make a summary. My question is: how can I move this PEP forward (accept, defer or reject it)? Can I accept it myself, or do we need a global agreement, or should someone else decide for this PEP (who? Antoine maybe)? -- Most complains concern issues if cloexec is set by default. I don't understand why so many people feel concerned because my PEP proposes *to not change anything to the default behaviour*: cloexec will not be set by default, as it was always the case since Python 1. You have to explicitly use -e command line option, PYTHONCLOEXEC=1 environment variable, or call sys.setdefaultcloexec(True). If you *explicitly* change the default value of the cloexec parameter, you must expect to change the behaviour of your application and maybe discover "bugs" in your application (application incompatible with cloexec set by default). (I don't think that any library would like to play with sys.setdefaultcloexec(), since most developers agree that it might break applications.) -- Ok, let's see all oppositions, presented as a FAQ. (I hope that I didn't misuse email quotes.) Q: the PEP does not solve file descriptor inherance after fork() A: Correct, it only solves issues with exec(), as explained in the PEP. Q: the PEP changes the POSIX behaviour. A: Antoine replied that "Python is not POSIX", Nick wrote "While many of Python's APIs are heavily inspired by POSIX, it still isn't POSIX" (he was talking about integer division). Perl sets cloexec flag by default since its first version, something like 25 years ago, except for explicit POSIX::* calls. Ruby 2 will also set cloexec flag by default. Q: The PEP may break applications. A: Most developers agree that it is (very) unlikely. If file descriptor inherance matters, subprocess must be used (because it rocks!) with pass_fds. If subprocess is used without pass_fds, applications stops working since python 3.2 (since python 3.0 on Windows). pass_fds will clear the close-on-exec flag on the file descriptors with the PEP. If an application breaks because it relies on file descriptor inherance, it's easy to detect it (EBADF error) and simple to fix it (ex: just call sys.setdefaultcloexec(False) at startup, or set explicitly cloexec parameter on the few functions causing trouble). Q: Performance overhead if cloexec is set by default (extra syscalls). A: The PEP now contains a benchmark: the overhead *on a microbenchmark* is between 1 and 3%. I bet that nobody will be able to notice a difference on a real application. Q: The default value of cloexec must not be configurable: "libraries cannot rely on it and have to make file descriptors cloexec on an individual basis, since the default flag can be disabled". A: IMO inherance of file descriptor doesn't matter in most cases, so only a few function must be modified to explicitly set or clear cloexec flag. Said differently: libraries should not care of the default value of the cloexec parameter, which should only be configured by applications (not libraries). Q: The default value of cloexec must not be configurable: "just by looking at this code, you have no way to know whether the file descriptor will be inherited by the child process." A: Guido wrote "Honestly, what happened to the idea that we're all adults here? The likelihood that someone is clueless enough to misuse the flag and yet clueful enough to find out how to change it seems remote -- and they pretty much deserve what they get." -- Another alternative (not listed in the PEP yet) is to leave the posix module (its name depends on the platform: posix, nt or ce) unchanged, and only modify the os module to add cloexec parameter. I implemented this alternative in the "posixmod" branch of the PEP 433 repository. It gives a better traceback on error because the cloexec flag is set in Python, so it's possible to know if the error comes from the creation of the file descriptor or setting the flag in the traceback. (Does it really matter? I never seen an error on setting or clearing the flag.) The problem is that it only concernes posix and os modules: the PEP does also modify socket, io and select modules, which don't have "low" and "high" level modules. So it's possible possible to create a socket using the POSIX behaviour. (Does it matter, it was said that Python is not POSIX.) I'm not convinced that we should do something special for posix/os, I prefer the actual version of the PEP (modify also posix). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Question regarding: Lib/_markupbase.py
Sorry, I just thought that: if '>' in rawdata[j:] would do a search, that is, that the implementation of "in" would just reuse/call the implementation of "find" and that the position returned would be used as: -1: not in != -1: in which seemed to me like the easy implementation of "in". That's why I was wondering why to search twice. Now I realize that it doesn't work the way I thought. Thank you for showing me and sorry for the confusion. Best regards, Guido (another Guido ;-) ) - Ursprüngliche Message - Von: Antoine Pitrou An: python-dev@python.org CC: Gesendet: 11:22 Dienstag, 12.Februar 2013 Betreff: Re: [Python-Dev] Question regarding: Lib/_markupbase.py Le Mon, 11 Feb 2013 11:02:04 -0800, Guido van Rossum a écrit : > Warning: see http://bugs.python.org/issue17170. Depending on the > length of the string being scanned and the probability of finding the > specific character, the proposed change could actually be a > *pessimization*. OTOH if the character occurs many times, the slice > will actually cause O(N**2) behavior. So yes, it depends greatly on > the distribution of the input data. That said, the savings are still puny unless you spend your time calling str.find(). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/just_another_developer%40yahoo.de ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of the PEP 433?
Victor Stinner writes: > > Q: The PEP may break applications. > A: Most developers agree that it is (very) unlikely. If file > descriptor inherance matters, subprocess must be used (because it > rocks!) with pass_fds. But that doesn't cover the case for programs that don't fork and really just want to exec another program. when using subprocess, you'll always have two processes running. > If subprocess is used without pass_fds, > applications stops working since python 3.2 (since python 3.0 on > Windows). pass_fds will clear the close-on-exec flag on the file > descriptors with the PEP. If an application breaks because it relies > on file descriptor inherance, it's easy to detect it (EBADF error) and the application may open other files/sockets and the unused file descriptor would be reused. there's no EBADF in that case. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build from source on mac uses clang, python.org binary uses gcc
On 14/02/2013 12:15, Ronald Oussoren wrote: buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe Python 3.3.0 (default, Jan 23 2013, 09:56:03) [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin Type "help", "copyright", "credits" or "license" for more information. ...which makes lxml pretty unhappy. Why is that? I'm not lxml's maintainer, I'd suggest asking Stefan... The binary was created on an older release of OSX where gcc is a valid compiler choice. Distutils should still pick clang when it detects that gcc is actually llvm-gcc though, doesn't it do that on your system? No. The gcc-based Python 3.3.0 from python.org uses gcc to compile lxml. BTW. Which OSX release are you using? And which variant of the OSX installer? 10.7.5. If by "OSX Installer" you mean that for Python 3.3.0, then it was the 64-bit one from python.org. P.S. Isn't this a question for python-list? I think questions about compiling python and how python chooses to compile its extensions, and weird things on particular operating systems related to that, are fair game for python-dev. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build from source on mac uses clang, python.org binary uses gcc
In article <511d350b.1080...@simplistix.co.uk>, Chris Withers wrote: > On 14/02/2013 12:15, Ronald Oussoren wrote: > > > >> buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe > >> Python 3.3.0 (default, Jan 23 2013, 09:56:03) > >> [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin > >> Type "help", "copyright", "credits" or "license" for more information. > > > >> > >> ...which makes lxml pretty unhappy. > > > > Why is that? > > I'm not lxml's maintainer, I'd suggest asking Stefan... > > > The binary was created on an older release of OSX where gcc is a valid > > compiler choice. Distutils > > should still pick clang when it detects that gcc is actually llvm-gcc > > though, doesn't it do that on your system? > > No. The gcc-based Python 3.3.0 from python.org uses gcc to compile lxml. Hmm, that's odd. It should be using clang unless on 10.7 with a current Xcode 4 installed unless something is overriding the compiler selection, e.g. setting CC=clang. (Also, I note you appear to be using an out-of-date clang; Xcode 4.6 and its Command Line Tools are current: $ clang --version Apple LLVM version 4.2 (clang-425.0.24) (based on LLVM 3.2svn Because of the relative newness of clang in the OS X world, it's important to keep up-to-date.) But, installing lxml 3.1.0 (from the tarball you cite) with the Python 3.3.0 64-bit/32-b it install seems to work fine for me. At least, it didn't get any import errors. Looking at your traceback (from https://mailman-mail5.webfaction.com/pipermail/lxml/2013-February/006730. html) appears to show that the lxml is picking up libxml2 and libxslt from /Library/Frameworks/Python.framework/Versions/7.2/lib and or /Library/Frameworks/Python.framework/Versions/7.2/lib. That almost certainly is the root of your problems. I'm not sure what those are but possibly EPD Python distributions. Your 3.3 builds should not be trying to used them to satisfy libraries. Perhaps there are links in /usr/local/*. Removing those links or files in /usr/local should allow you to build lxml with the Apple-supplied libxml2 and libxslt. If you want newer versions or the libs, either build them for 64-/32-bit archs or use a third-party package manager like Homebrew or MacPorts to supply them (and/or supply a Python 3.3). -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build from source on mac uses clang, python.org binary uses gcc
On 14 Feb, 2013, at 20:03, Chris Withers wrote: > On 14/02/2013 12:15, Ronald Oussoren wrote: >> >>> buzzkill:virtualenvs chris$ /src/Python-3.3.0/python.exe >>> Python 3.3.0 (default, Jan 23 2013, 09:56:03) >>> [GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin >>> Type "help", "copyright", "credits" or "license" for more information. >> >>> >>> ...which makes lxml pretty unhappy. >> >> Why is that? > > I'm not lxml's maintainer, I'd suggest asking Stefan... I should have been more clear: what error does lxml give when compiling? Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
