Re: [Python-Dev] FYI: Python 2.7.1 + gcc 4.6 (experimental) probable optimizer problem
> Does that version of gcc emit any warnings during compilation? Yes, there are a few: http://cci.lbl.gov/~rwgk/tmp/gcc_trunk_168695_fc14_py271/ This is with: g++ (GCC) 4.6.0 20110112 (experimental) Ralf___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] FYI: Python 2.7.1 + gcc 4.6 (experimental) probable optimizer problem
I just wanted to share an observation in case Python developers are interested: Python 2.7.1 doesn't build with the current gcc 4.6 svn. Note that gcc 4.6 is now in "bug-fix only" mode. Some details: Fedora 14 64-bit. The first time I noticed the problem was in Nov or early Dec 2010; I'm pretty sure it worked in Oct maybe still early Nov. Python configured simply with ./configure g++ (GCC) 4.6.0 20101206 (experimental) % make /bin/sh: line 1: 41686 Segmentation fault (core dumped) CC='gcc -pthread' LDSHARED='gcc -pthread -shared ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py build make: *** [sharedmods] Error 139 g++ (GCC) 4.6.0 20110108 (experimental) % make XXX lineno: 743, opcode: 0 Traceback (most recent call last): File "/net/theta/raid1/rwgk/junk/Python-2.7.1/Lib/site.py", line 62, in import os File "/net/theta/raid1/rwgk/junk/Python-2.7.1/Lib/os.py", line 743, in def urandom(n): SystemError: unknown opcode make: *** [sharedmods] Error 1 make finishes OK if I configure --with-pydebug. Therefore my guess is that there is an optimizer bug in the current gcc 4.6 that's only triggered by a specific construct in Python. (A lot of other stuff builds and runs fine.) BTW: I've been doing gcc pre-release testing regularly for many year, starting with gcc 3.3. This is the first time I see the Python build fail persistently for several weeks. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Real segmentation fault handler
FWIW: I didn't have much luck translating segfaults into exceptions. It (seemed) to work on some platforms, but not others; this was in the context of C++. In my experience, it is more useful to generate Python and C stack traces and bail out. I also do this for floating-point exceptions. The handlers are installed at runtime from a low-level extension module: http://cctbx.svn.sourceforge.net/viewvc/cctbx/trunk/boost_adaptbx/meta_ext.cpp?view=markup Example output is below. It works under Linux and partially under Mac OS X. Ralf % boost_adaptbx.segmentation_fault Now dereferencing null-pointer ... show_stack(1): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/segmentation_fault.py(10) run show_stack(2): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/segmentation_fault.py(14) libc backtrace (18 frames, most recent call last): /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python [0x4118e9] /lib64/libc.so.6(__libc_start_main+0xf4) [0x363241e074] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(Py_Main+0x935) [0x4123c5] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_SimpleFileExFlags+0x1a0) [0x4a8860] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_FileExFlags+0x10e) [0x4a85ce] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCode+0x32) [0x487402] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCodeEx+0x81f) [0x4873bf] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x6bc1) [0x486541] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x2bb9) [0x482539] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyObject_Call+0x13) [0x415ae3] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2ba7c6f7] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0)+0x28) [0x2ba87148] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::function0::operator()() const+0x19e) [0x2ba8816e] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2ba7fef8] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0x7d) [0x2ba7fb5d] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so(boost::python::objects::caller_py_function_impl > >::operator()(_object*, _object*)+0x29) [0x2b8470a9] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so [0x2b843790] /lib64/libc.so.6 [0x3632430f30] Segmentation fault (Python and libc call stacks above) % boost_adaptbx.divide_by_zero Now dividing by zero (in C++) ... show_stack(1): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/divide_by_zero.py(10) run show_stack(2): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/divide_by_zero.py(14) libc backtrace (18 frames, most recent call last): /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python [0x4118e9] /lib64/libc.so.6(__libc_start_main+0xf4) [0x363241e074] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(Py_Main+0x935) [0x4123c5] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_SimpleFileExFlags+0x1a0) [0x4a8860] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_FileExFlags+0x10e) [0x4a85ce] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCode+0x32) [0x487402] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCodeEx+0x81f) [0x4873bf] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x6bc1) [0x486541] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x2bb9) [0x482539] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyObject_Call+0x13) [0x415ae3] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2ba7c6f7] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0)+0x28) [0x2ba87148] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::function0::operator()() const+0x19e) [0x2ba8816e] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2ba7fef8] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0x7d) [0x2ba7fb5d] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so(boost::python::objects::caller_py_function_impl > >::operator()(_object*, _object*)+0x12a) [0x2b84759a] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so [0x2b8437a4] /lib64/libc.so.6 [0x3632430f30] Floating-point error (Python and libc call stacks above) - Original Message From: Victor Stinner <[EMAIL PROTECTED]> To: python-dev@python.org Sent: Monday, September 29, 2008 4:05:53 PM Subject: [Python-Dev] Real segmentation fault handler Hi, I would like to be able to catch SIGSEGV in my Python code! So I started to hack Python trunk to support this feature. Th
Re: [Python-Dev] Different float formatting on Windows and Linux
> The tests for float.__format__ are breaking on Windows, because of this > issue: http://bugs.python.org/issue1600. Basically, Windows is using 3 > digits for exponents < 100, and Linux (and at least MacOS) are using 2. Yes, this is very annoying and I once lost of lot of time because of the Windows difference. > I think the options are: > 1: Do nothing. Adapt the tests to deal with the differences. > 2: Force 3 characters for exponents < 100. > 3: Force 2 characters for exponents < 100. I'd go for 3. Rationale: this change will mostly affect scientific code, which is mostly developed and used on Unix systems. Thanks for taking care if this nuisance! Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.5.1c1 pickle problem
Hi Raymond, Thanks for the detailed explanation! > I'm not sure what your code was doing where the bugfix would cause > breakage. If its __getitem__() override returned a meaningful value > for each element in obj.keys(), then it should have worked fine. Of > course, if it was raising an exception or triggering a side-effect, > then one could argue that the bugfix was working as intended by > allowing the subclasser to affect how the base class goes about its > business. It wasn't actually my own code, but that of a colleague. His __getitem__() override is below. It fails because __getitem__() is called with the wrong type of argument (plain int instead of a list of atoms). I'm not sure it presents a convincing use case. I would have used composition instead of inheritance in the same situation. I don't feel competent to join the discussion about the changes in the pickling behavior, but I feel quite strongly that such a change should not be introduced in a bug-fix release. Ralf def __getitem__(self, atoms): LIST = [] for atom in atoms: attr = 'index' if hasattr(atom, 'unique_index'): attr = 'unique_index' LIST.append(getattr(atom, attr)) key1 = tuple(LIST) LIST.reverse() key2 = tuple(LIST) rc = None try: rc = dict.__getitem__(self, key1) except KeyError, e: pass try: rc = dict.__getitem__(self, key2) except KeyError, e: pass return rc __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 2.5.1c1 pickle problem
When pickling instances of an object derived from dict, the pickle in Python 2.5.1c1 calls the object's __getitem__() method. In contrast, earlier versions of Python incl. 2.5 don't call that method. Below is a minimal example with outputs. Is the difference in behavior an oversight or new feature? I couldn't find anything directly related in the release notes. The difference in behavior breaks some of our code. Thanks! Ralf class user(dict): def __getitem__(self, key): print "GETITEM", key if (__name__ == "__main__"): import sys print sys.version u = user() u[1] = 2 import pickle pickle.dumps(u) print "Done." 2.5.1c1 (r251c1:54692, Apr 11 2007, 16:15:52) [GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] GETITEM 1 Done. 2.5 (r25:51908, Apr 11 2007, 16:11:19) [GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] Done. 2.4.2 (#1, Feb 12 2006, 03:45:41) [GCC 4.1.0 20060210 (Red Hat 4.1.0-0.24)] Done. 2.3.4 (#1, Oct 26 2004, 16:45:38) [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] Done. 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] Done. No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started. http://mobile.yahoo.com/mail ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5.1 plans
It would be nice if this simple fix could be included (main branch and 2.5.1): https://sourceforge.net/tracker/?func=detail&aid=1598181&group_id=5470&atid=105470 [ 1598181 ] subprocess.py: O(N**2) bottleneck I submitted the trivial fix almost two months ago, but apparently nobody feels responsible... Ralf - Original Message From: Neal Norwitz <[EMAIL PROTECTED]> To: Python Dev Sent: Wednesday, January 3, 2007 10:59:15 PM Subject: Re: [Python-Dev] 2.5.1 plans The current schedule looks like it's shaping up to be: Wed, Jan 24 for 2.5.1c1 Wed Jan 31 for 2.5.1 It would be great if you could comment on some of the bug reports below. I think several already have patches/suggested fixes. These looks like they would be nice to fix:: http://python.org/sf/1579370 Segfault provoked by generators and exceptions http://python.org/sf/1377858 segfaults when using __del__ and weakrefs There is one outstanding issue I was notified of in private mail:: http://python.org/sf/1568240 Tix is not included in 2.5 for Windows It's not clear to me if this should be fixed, but it's got a high priority:: http://python.org/sf/1467929 %-formatting and dicts n __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- "Martin v. L�wis" <[EMAIL PROTECTED]> wrote: > Ralf W. Grosse-Kunstleve wrote: > > If there is a consenus, I'd create a new exception > ImportErrorNoModule(name) > > that is used consistently from all places. This would ensure uniformity of > the > > message in the future. > > A correction proposal should only be given if it is likely correct. It is not a proposal, just a "note". Maybe a better alternative would be ImportError: No module name foo Reminder: To resolve import problems consult the section on "Packages" at http://www.python.org/doc/tut/ > There can be many reasons why an import could fail: there might be > no read permission for the file, The warning in 2.5b1 doesn't fire in this case: % ls -l junk.py -- 1 rwgk cci 16 Jun 28 05:01 junk.py % python Python 2.5b1 (r25b1:47027, Jun 26 2006, 02:59:25) [GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import junk Traceback (most recent call last): File "", line 1, in ImportError: No module named junk >>> > or the PYTHONPATH might be setup > incorrectly. That's impossible to detect. > IOW, a hint about a missing __init__.py should only be given if > a directory with the name of module was found, but lacked an > __init__.py (i.e. in the cases where currently a warning is > produced). I am thinking you'd need to build up a buffer of potential warnings while trying to resolve an import. If the import succeeds the buffer is discarded, if it fails it is added to the exception message, or the warnings are "flushed" right before the ImportError is raised. Does that sound right? How would this interact with threading (it seems you'd need a separate buffer for each thread)? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- "Delaney, Timothy (Tim)" <[EMAIL PROTECTED]> wrote: > Michael Hudson wrote: > > > Benji York <[EMAIL PROTECTED]> writes: > > > >> Nick Coghlan wrote: > >>> Perhaps ImportWarning should default to being ignored, the same way > >>> PendingDeprecationWarning does? > >>> > >>> Then -Wd would become 'the one obvious way' to debug import problems > >> > >> +1 > > > > I'm not sure what this would achieve -- people who don't know enough > > about Python to add __init__.py files aren't going to know enough to > > make suppressed-by-default warnings not suppressed. > > The change was prompted by developers (specifically, Google developers). > Developers should be able to put -Wd in their automated build scripts. > > > The more I think about it, the more I like the idea of saying > > something when an import fails only because of a missing __init__.py > > file. I guess I should try to implement it... > > This is by far and away my preference as well (stating which directories > may have been importable if they had __init__.py in the exception) but > it was shot down in the original discussion. I guess it is probably quite tricky to implement. Note the list of files with the "No module named" message I posted earlier. Somehow you'd have to keep track of all potential directories in all these different contexts. I think a combination of pointing to the documentation and mentioning -Wd would cover all situations. Most people just need a reminder. That's easy to achieve with a new ImportErrorNoModule(name) exception. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- "Martin v. L�wis" <[EMAIL PROTECTED]> wrote: > So spend some of the money to come up with an alternate solution for > 2.5b2. With a potential damage of a million dollars, it shouldn't be > too difficult to provide a patch by tomorrow, right? My share is only 10 man hours, payed for by the US government at a scientist salary. :-) A simple patch with a start is attached. Example: % ./python Python 2.5b1 (r25b1:47027, Jun 26 2006, 03:15:33) [GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import foo Traceback (most recent call last): File "", line 1, in ImportError: No module named foo Note that subdirectories are searched for imports only if they contain an __init__.py file. See the section on "Packages" in the Python tutorial for details (http://www.python.org/doc/tut/). >>> The "No module named" message is repeated in these files (2.5b1 tree): ./Demo/imputil/knee.py ./Lib/ihooks.py ./Lib/modulefinder.py ./Lib/xmlcore/etree/ElementTree.py ./Lib/runpy.py ./Lib/imputil.py If there is a consenus, I'd create a new exception ImportErrorNoModule(name) that is used consistently from all places. This would ensure uniformity of the message in the future. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com import_patch Description: 467797280-import_patch ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 6/25/06, Fred L. Drake, Jr. <[EMAIL PROTECTED]> wrote: > > On Sunday 25 June 2006 20:48, Greg Ewing wrote: > > > BTW, when that was being discussed, did anyone consider > > > allowing a directory to be given a .py suffix as an > > > alternative way to mark it as a package? > > :-) > > I'd certainly be a lot happier with that than with the current behavior. > > Silly little warnings about perfectly good data-only directories are just > > silly. > > And silly whining about warnings for silly name conflicts are just as silly. > :-) I cannot smile here. I anticipate real damage in terms of $$$. To see what lead me to the subject of this thread, please take a quick look here, which is the result of running (most) of our unit tests: http://cci.lbl.gov/~rwgk/tmp/py25b1_ImortWarning_flood I can work around it, sure. Everybody can work around it, of course. But consider that one hour of a professional person is at least $100 with benefits etc. included. (If that sounds high, I know people charging much more than that; also consider that the going rate for a car mechanic in the bay area is $90, as you probably know.) Now say you have 1000 groups of developers having to work around the warning (I bet you have more). There will be discussions, alternatives will be tried and discarded, etc. Say that eats about 10 man hours per group before the issue is settled, which again is a very conservative estimate I believe. That makes a total of $100 in damages, at least. Is that warning really worth a millon dollar? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > I think it is safe to say that Twisted is more widely used than anything > Google has yet released. Twisted also has a reasonably plausible > technical reason to dislike this change. Google has a bunch of engineers > who, apparently, cannot remember to create an empty __init__.py file in > some directories sometimes. Simply adding a note to the ImportError message would solve this problem "just in time": >>> import mypackage.foo Traceback (most recent call last): File "", line 1, in ? ImportError: No module named mypackage.foo Note that subdirectories are searched for imports only if they contain an __init__.py file: http://www.python.org/doc/essays/packages.html __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyRange_New() alternative?
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Scott David Daniels wrote: > > I am not sure about your compiler, but if I remember the standard > > correctly, the following code shouldn't complain: > > > > PyObject_CallFunction((PyObject*) (void *) &PyRange_Type, > > "lll", start, start+len*step, step) > > You remember the standard incorrectly. There are three standards to consider: C89/90, C99, C++97/98. Here you can find the opinion of one of the authors of the C++ standard in this matter: http://mail.python.org/pipermail/c++-sig/2005-December/009869.html __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > I don't know. Whether a warning is a problem is a matter of attitude, also. Our users will think our applications are broken if they see warnings like that. It is not professional. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- "Martin v. L�wis" <[EMAIL PROTECTED]> wrote: > The specific question was > > "Is there a way to set the warning options via an environment variable?" > > This has nothing to do with beta1; the warnings module was introduced > many releases ago, along with all the mechanics to disable warnings. Due to the new ImportWarning first introduced in 2.5b1 the question of disabling warnings is becoming much more pressing (I am assuming that I am not again the only person on the planet to have this problem). > I guess you misunderstood. Yes. > I propose you put warnings.simplefilter() > into your code. The warnings was introduced before 2.2.1 IIRC, so this > should work on all releases you want to support (but have no effect on > installations where the warning isn't generated). Where would I put the warnings.simplefilter()? I have hundreds of scripts and __init__.py files. I just came accross this situation (simplified): % cd boost/boost % python2.5 >>> import math __main__:1: ImportWarning: Not importing directory 'math': missing __init__.py This is because there is a subdirectory math in boost/boost, something that I cannot change. The PYTHONPATH is not set at all in this case. I.e. I get the ImportWarning just because my current working directory happens to contain a subdirectory which matches one of the Python modules in the standard library. Isn't this going to cause widespread problems? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyRange_New() alternative?
--- Scott David Daniels <[EMAIL PROTECTED]> wrote: > Ralf W. Grosse-Kunstleve wrote: > > Thanks! This does the trick for me: > > > > #if PY_VERSION_HEX >= 0x0203 > > PyObject_CallFunction( > > (PyObject*) &PyRange_Type, "lll", start, start+len*step, step) > > #else > > PyRange_New(start, len, step, 1) > > #endif > > > > I am compiling the code above with a C++ compiler (in the context of > > Boost.Python). Newer g++ versions unfortunatly produce a warning if -Wall > is > > specified: > > > > warning: dereferencing type-punned pointer will break strict-aliasing rules > > I am not sure about your compiler, but if I remember the standard > correctly, the following code shouldn't complain: > > PyObject_CallFunction((PyObject*) (void *) &PyRange_Type, > "lll", start, start+len*step, step) Thanks for the suggestion! I just tried: g++ (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-20) g++ (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3) g++ (GCC) 4.0.0 20050519 (Red Hat 4.0.0-8) with -Wall -Wno-sign-compare. These compilers don't issue the "will break strict-aliasing rules" warning with or without the intermediate (void *). However, I also tried: g++ (GCC) 4.1.0 20060304 (Red Hat 4.1.0-3) which issues the warning without the (void *), but not with the (void *). I am not an expert of the C/C++ language details, but the intermediate cast seems to be a great local alternative to the global -fno-strict-aliasing flag. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyRange_New() alternative?
--- Tim Peters <[EMAIL PROTECTED]> wrote: > [Ralf W. Grosse-Kunstleve] > > Thanks! This does the trick for me: > > > > #if PY_VERSION_HEX >= 0x0203 > > PyObject_CallFunction( > > (PyObject*) &PyRange_Type, "lll", start, start+len*step, step) > > Note that this is extremely lax about possible overflow in the > arithmetic. For that reason it can't be recommend for general use. > > > #else > > PyRange_New(start, len, step, 1) > > #endif > > > > I've tested this with Python 2.2.3, 2.3.4, 2.4.3, 2.5b1. Python 2.2.3 > (RedHat > > WS 3) compiles the PyRange_Type call, but there is a runtime error: > > > > TypeError: cannot create 'xrange' instances > > Sorry, I didn't follow that. The only mention of PyRange_Type in the > #if'ed code above is in a block that looks like it should be entirely > ignored in a 2.2.3 Python (provided you're using the right header > files and the C compiler isn't broken). First I tried the PyRange_Type code with Python 2.2.3 and no #ifdef. I resorted to the #ifdef and the old PyRange_New() call only because it didn't work. > Compile all of Python that way, and you'll probably see more of those > than you can count ;-) Python is normally compiled with, and is > _intended_ to be compiled with, > > -fno-strict-aliasing > > If you didn't do that, try it. I missed this. Thanks for pointing it out. > Since it was never documented to begin with, it was a "use at your own > risk" thing anyway. As you're currently it's only known user > throughout all of history :-), if you do all the work of > rehabilitating it, I'd be at best a weak -1 anyway: one of the > problems with PyRange_New was that its signature was wildly different > than the builtin range()'s. That made it a poor API for "surprise, > surprise!" reasons alone. That was a mistake, and I'd rather > inconvenience you than pass that mistake on to our precious children > ;-) I agree, although I find it hard to believe I am that unique. I'll google for PyRange_New in a couple of years to see how many more people got stranded. :-) Although it will bias the measure of my uniqueness, I still believe you should tell people in the documentation what to do, e.g. PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", start, stop, step) which avoids showing the sloppy start+len*step hack. BTW: by the time my children pick up programming nobody under 30 will use C Python anymore. ;-) __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyRange_New() alternative?
--- Bob Ippolito <[EMAIL PROTECTED]> wrote: > > I am sure I can get this to work with some digging, but I am > > posting here to > > highlight a communication problem. I feel if a function is removed the > > alternative should be made obvious in the associated documentation; in > > particular if there is no existing documentation for the alternative. > > He means something like this: > PyObject_CallFunction(PyRange_Type, "llli", ...) Thanks! This does the trick for me: #if PY_VERSION_HEX >= 0x0203 PyObject_CallFunction( (PyObject*) &PyRange_Type, "lll", start, start+len*step, step) #else PyRange_New(start, len, step, 1) #endif I've tested this with Python 2.2.3, 2.3.4, 2.4.3, 2.5b1. Python 2.2.3 (RedHat WS 3) compiles the PyRange_Type call, but there is a runtime error: TypeError: cannot create 'xrange' instances I am compiling the code above with a C++ compiler (in the context of Boost.Python). Newer g++ versions unfortunatly produce a warning if -Wall is specified: warning: dereferencing type-punned pointer will break strict-aliasing rules This refers to the (PyObject*) &PyRange_Type cast. I believe the warning is bogus, but people still get upset about it (google the C++-SIG archive). Is there a chance that PyRange_New() could be resurrected, with the fragment above (plus additional overflow check for start+len*step) as the implementation? That would fix the problems of the old implementation, there would be no reason to have the cast in C++, no frustrated end-users, and one change less to document. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyRange_New() alternative?
--- Georg Brandl <[EMAIL PROTECTED]> wrote: > Well, PyRange_New *was* undocumented, Yes, that was clear. > However, it would perhaps be helpful to add a note to the whatsnew document > for users like yourself. Andrew, does that make sense? I am worried about using an alternative that is *again* not documented. There is no mention of "PyRange" in the Python C API documentation, not even "range". Unless I am the only user of PyRange_New() in the whole wide world a few extra lines in the "changes" document will prevent recurring questions. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyRange_New() alternative?
--- Georg Brandl <[EMAIL PROTECTED]> wrote: > Ralf W. Grosse-Kunstleve wrote: > > http://docs.python.org/dev/whatsnew/ports.html says: > > > > The PyRange_New() function was removed. It was never documented, never > used > > in the core code, and had dangerously lax error checking. > > > > I use this function (don't remember how I found it; this was years ago), > > therefore my code doesn't compile with 2.5b1 (it did OK before with 2.5a2). > Is > > there an alternative spelling for PyRange_New()? > > You can call PyRange_Type with the appropriate parameters. Thanks a lot for the hint! However, I cannot find any documentation for PyRange_*. I tried this page... http://docs.python.org/api/genindex.html and google. Did I miss something? I am sure I can get this to work with some digging, but I am posting here to highlight a communication problem. I feel if a function is removed the alternative should be made obvious in the associated documentation; in particular if there is no existing documentation for the alternative. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Ralf W. Grosse-Kunstleve wrote: > > Is there a way to set the warning options via an environment variable? > > This is off-topic for python-dev, What is the channel I should use? (I am testing a beta 1.) > but: why don't switch off the warnings > in the code? We support installation from sources with the native Python if available. Any Python >= 2.2.1 works. It would be frustrating if we had to give up on this just because of a warning designed for newcomers. In our applications we typically address this type of problem with informative exceptions. For example, if a Boost.Python wrapped C++ object doesn't support pickling: RuntimeError: Pickling of "cctbx_sgtbx_ext.space_group_symbols" instances is not enabled (http://www.boost.org/libs/python/doc/v2/pickle.html) Something like this could help newcomers just the same without impacting experienced users with large, complex package structures. E.g.: >>> import mypackage.foo Traceback (most recent call last): File "", line 1, in ? ImportError: No module named mypackage.foo Note that subdirectories are searched for imports only if they contain an __init__.py file: http://www.python.org/doc/essays/packages.html __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ImportWarning flood
--- Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 6/21/06, Ralf W. Grosse-Kunstleve <[EMAIL PROTECTED]> wrote: > > I am getting tons of "ImportWarning: Not importing directory". See below > for > > examples. It is impractical for me to reorganize our directory structure. > I'd > > be busy for a week or more and people would probably scream at me because > all > > the paths have changed. Are there other options to get rid of the warnings? > > Check out the -W command line option and the warnings module. These > document how to suppress warnings. Thanks! This does the trick: python -W'ignore:Not importing directory' But this doesn't: python -W'ignore:Not importing directory:ImportWarning' I tried a bunch of variations without success. A few examples here would be very valuable: http://docs.python.org/lib/warning-filter.html Also, the magic incantation to silence the warnings would be very helpful here: http://docs.python.org/dev/whatsnew/other-lang.html Is there a way to set the warning options via an environment variable? Otherwise I am forced to use a wrapper or aliases. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] ImportWarning flood
http://docs.python.org/dev/whatsnew/other-lang.html says: > One error that Python programmers sometimes make is forgetting to > include an __init__.py module in a package directory. Debugging this > mistake can be confusing, and usually requires running Python with the > -v switch to log all the paths searched. In Python 2.5, a new > ImportWarning warning is raised when an import would have picked up a > directory as a package but no __init__.py was found. I am getting tons of "ImportWarning: Not importing directory". See below for examples. It is impractical for me to reorganize our directory structure. I'd be busy for a week or more and people would probably scream at me because all the paths have changed. Are there other options to get rid of the warnings? Thanks! Ralf /net/rosie/scratch1/rwgk/dist/libtbx/libtbx/command_line/scons.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py from libtbx.utils import Sorry /net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/cctbx/math': missing __init__.py from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil /net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx/math': missing __init__.py from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil scons: Reading SConscript files ... /net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/__init__.py:112: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/CVS': missing __init__.py file, path, desc = imp.find_module(self.name, smpath) /net/rosie/scratch1/rwgk/dist/phenix/phenix/__init__.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py try: import libtbx.forward_compatibility /net/rosie/scratch1/rwgk/dist/phenix/phenix/refinement/__init__.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/iotbx': missing __init__.py import iotbx.phil /net/rosie/scratch1/rwgk/dist/iotbx/iotbx/phil.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/cctbx': missing __init__.py from cctbx import sgtbx /net/rosie/scratch1/rwgk/dist/cctbx/cctbx/array_family/flex.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx': missing __init__.py import scitbx.array_family.flex /net/rosie/scratch1/rwgk/dist/scitbx/scitbx/array_family/flex.py:2: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/boost': missing __init__.py import boost.optional /net/rosie/scratch1/rwgk/dist/libtbx/libtbx/utils.py:226: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/mmtbx': missing __init__.py try: module = __import__(module_path) etc. etc. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PyRange_New() alternative?
http://docs.python.org/dev/whatsnew/ports.html says: The PyRange_New() function was removed. It was never documented, never used in the core code, and had dangerously lax error checking. I use this function (don't remember how I found it; this was years ago), therefore my code doesn't compile with 2.5b1 (it did OK before with 2.5a2). Is there an alternative spelling for PyRange_New()? Thank you in advance! Ralf __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PY_SSIZE_T_MIN?
Congratulations to the Python 2.5a1 release! I started adjusting Boost.Python to work with this new release and it is going very well. I noticed this #define in pyport.h: #define PY_SSIZE_T_MAX ((Py_ssize_t)(((size_t)-1)>>1)) However, I couldn't find a corresponding PY_SSIZE_T_MIN which would come in handy to adjust old code using INT_MIN (from limits.h). Are there arguments against defining PY_SSIZE_T_MIN? Or is this just an oversight? Objects/longobject.c uses: return -PY_SSIZE_T_MAX-1; To maximize consistency this would seem ideal to me: pyport.h: #define PY_SSIZE_T_MIN (-PY_SSIZE_T_MAX-1) longobject.c: return PY_SSIZE_T_MIN; Cheers, Ralf __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft proposal: Implicit self in Python 3.0
--- Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > Please try the code below to see the performance impact. > > oh, please. do you seriously think that if you don't have to type "self" > yourself, Python will suddenly be able to turn all instance variables into > local function variables without any performance overhead ? Yes, exactly, since you can make the local namespace vs. instance attribute distinction in C. Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft proposal: Implicit self in Python 3.0
--- Thomas Wouters <[EMAIL PROTECTED]> wrote: > The main difference isn't the lookup of 'self', it's the attribute retrieval > of 'x' from 'self'. I see. Thanks! If you put 'self' into a special category (with corresponding C code), couldn't you use the same "indexing" tricks as for local variables and make 'self.x' just as fast as 'x'? Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft proposal: Implicit self in Python 3.0
--- Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 1/6/06, Armin Rigo <[EMAIL PROTECTED]> wrote: > > On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote: > > > There are three different peculiarity in Python 2.x > > > in respect of 'self' method argument: > > > > Yuk! This has been discussed again and again already. *Please* move > > this discussion to comp.lang.python. > > Yes please. This is a death penalty, even without your explicit rejection. No professional person has the time to sift through that volume of noise. > This won't change. The outside world is trying to tell you something; and they are not all dumb as you suggest in your other posting. I suggest creating [EMAIL PROTECTED] Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft proposal: Implicit self in Python 3.0
--- Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Ralf W. Grosse-Kunstleve wrote: > > > ***Please*** make Python more selfish. Note that this is also an obvious > avenue > > for significant performance increases. If self is implicit you don't have > to do > > the dictionary lookup for "self" all the time as is the case now. > > what dictionary lookup ? IIUC, "self" is first looked up in the local dictionary. Please try the code below to see the performance impact. As an alternative to a copy/paste exercise try this: wget http://cci.lbl.gov/~rwgk/python/self_lookup.py python self_lookup.py The output with Python 2.4.1/Fedora3/Opteron is: loop_function: 13.10 loop_class: 17.11 Cheers, Ralf import time def loop_function(x, n): result = 0 for i in xrange(n): result += x return result class loop_class: def __init__(self, x, n): self.x = x self.n = n def __call__(self): result = 0 for i in xrange(self.n): result += self.x return result def run(x=3.1415, n=10**8): t0 = time.time() loop_function(x=x, n=n) print "loop_function: %.2f" % (time.time() - t0) t0 = time.time() loop_class(x=x, n=n)() print " loop_class: %.2f" % (time.time() - t0) if (__name__ == "__main__"): run() __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft proposal: Implicit self in Python 3.0
--- Alexander Kozlovsky <[EMAIL PROTECTED]> wrote: > What do you think about this? I (who writes Python code for a living) love it! See also: http://cci.lbl.gov/~rwgk/python/adopt_init_args_2005_07_02.html ***Please*** make Python more selfish. Note that this is also an obvious avenue for significant performance increases. If self is implicit you don't have to do the dictionary lookup for "self" all the time as is the case now. Good luck with your battle! Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python + Visual C++ 8.0?
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Out of curiosity: can you please try invoking this enum_print to stdout > with your VS2005-built boost module? I see it uses fprintf, so I would > expect it to crash. After beating on this for half an hour I am coming to the conclusion that there is no way the enum_print function is invoked. No matter what I try, Python always calls enum_str if available. If I set the enum_str slot to 0, it calls enum_repr. If I set that slot to 0 as well it calls the tp_str method of PyInt_Type, since the Boost.Python enum type inherits from it. I have no clue how I can crash the VC7/VC8 mix doing "normal" things. Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python + Visual C++ 8.0?
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Well, yes: the areas are > - memory management > - stdio > - locales > for the C library; for the C++, I'm not so sure, but I think one of the > areas is > - static members of class templates, in particular in STL containers Thanks for the insight! For Boost.Python users the situation is actually not bad at all: - there is not a single malloc() in the Boost.Python sources - there is only one free(), which is specific to gcc; i.e. this code is not seen by Visual C++. - there is only one FILE* that is seen by Visual C++. It appears in the signature of a function assigned to the tp_print slot. As far as I can tell this is the only soft spot. I'll lobby for having the print function removed since it is optional and both tp_str and tp_repr are defined anyway. - nobody should be mixing C++ libraries compiled with different compilers on any platform; it doesn't get much more dangerous than this. Since Python is pure C "static members of class templates" should therefore be a non-issue. This leaves only "stdio" and "locales" to be considered. We are strictly avoiding both in C++ to maximize portability. I/O is intrinsically slow and therefore there is no point in coding it in C++, especially since Python's I/O is very fast anyway. I am not sure about locales because this is not very interesting at all for scientific code development. Under the assumption that the one tp_print function in the Boost.Python sources is removed, I am arriving at the conclusion that it is more than pure luck that all our tests work. People writing C++ extensions view the compiled layer as low-level number-crunching back-end. It is quite natural not to find I/O and use of locales in such code. And even if there is I/O, chances are it is done via std::cout rather than a raw FILE* obtained from Python via difficult to memorize incantations (I don't even know how to do it). Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python + Visual C++ 8.0?
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > P.P.S. You do know that this configuration (extension compiled > with VS2005, Python compiled wit VS.NET2003) is not supported, > right? Thanks to Adal's help I got all our C++ extensions (about 50) to work with VC8. I am using Python 2.4.2 compiled with VC7.1. We have more than 150 unit tests which all pass. Our applications also work under Windows 2000 even if the platform has never seen any Visual Studio installation. Also, in over two years I never had problems mixing VC6 Python with VC7.1 extensions; our binary installers worked on any Windows 2000 or XP system I've ever tried. Since we are using Boost.Python, I believe we are heavily exercising the compiler, the C/C++ libraries, and the DLL machinery. However, we don't have any home-grown C++ GUI code. Could it be that problems due to mixing objects from different compiler versions are restricted to certain areas, like GUI libraries? Cheers, Ralf __ Yahoo! for Good - Make a difference this year. http://brand.yahoo.com/cybergivingweek2005/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python + Visual C++ 8.0?
--- Adal Chiriliuc <[EMAIL PROTECTED]> wrote: > Then you need to run mt.exe to embedd the manifest: > mt.exe /outputresource:"cctbx_math_ext.pyd;#2" /manifest > cctbx_math_ext.pyd.manifest That is the magic trick! After applying the mt command to all our extensions most of our unit tests work even with the VC7.1 compiled Python. I am very optimistic I can get all our tests to work with a few C++ adjustments (related to the new way STL iterators are implemented). Thanks a lot! I could not have figured this out myself in a million years. Cheers, Ralf __ Yahoo! for Good - Make a difference this year. http://brand.yahoo.com/cybergivingweek2005/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python + Visual C++ 8.0?
--- Adal Chiriliuc <[EMAIL PROTECTED]> wrote: > Python uses LoadLibraryEx with the LOAD_WITH_ALTERED_SEARCH_PATH flag > which means that DLLs used by the extension will be searched > IN THE EXTENSION FOLDER and not on PATH. > > Try putting msvcp80.dll right next to your extension DLL. I tried that first since that's the way we use VC 7.1 extensions with older Python versions (compiled with VC 6). > It is a little strange that it is not loaded directly from the Windows > side by side folder (%WINDIR%\WinSxS). You should check if the manifest > is embedded correctly inside the DLL. Sorry, the manifests are new to me. How can I check if the manifest is correctly embedded? FWIW: I already tried copying the manifest into the directory with the extensions. In case it matters, here are the compiler and liker switches I am using (commands issued by scons): cl /nologo /D_CRT_SECURE_NO_DEPRECATE /wd4996 /Zm800 /MD /GR /EHsc /DBOOST_DISABLE_THREADS /DNDEBUG /Ox -DBOOST_PYTHON_MAX_BASES=2 -DBOOST_PYTHON_SOURCE link /nologo /incremental:no /dll /out:lib\cctbx_math_ext.pyd /implib:lib\cctbx_math_ext.lib /LIBPATH:lib Am I missing some magic new switch? > And like Martin said, mixing Python and extensions compiled with > different compilers is a bad idea. If that's really the case it will mean a lot of work. Mixing VC6 Python and VC7.x extensions never gave us any trouble. Thanks! Cheers, Ralf __ Yahoo! for Good - Make a difference this year. http://brand.yahoo.com/cybergivingweek2005/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python + Visual C++ 8.0?
--- "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Ralf W. Grosse-Kunstleve wrote: > > I am using a Visual Studio 2005 Professional installation. I also ran > > vcredist_x86.exe. Moving msvc[mpr]80.dll to a directory on PATH didn't > > help. However, standalone executables work OK without any gymnastics. > > Does anyone know what the problem could be with the extensions? > > Can't check right now - but could it be that the standalone executables > get an extra copy of this library in their binary directory, as part > of the build process? No. > > A quick attempt to compile Python from scratch using Visual C++ 8.0 > > produced a python.exe, but it doesn't run (the debug / send report / > > don't send report box pops up). Has someone tried this before? > > Yes; a patch to fix this problem has been checked into the trunk > (but I have no plans for backporting it to the 2.4 branch, as 2.4 > will be compiled with VC7.1 "forever" - i.e. until its final release). OK. > Regards, > Martin > > P.S. I currently also plan to build Python 2.5 with VC 7.1. > P.P.S. You do know that this configuration (extension compiled > with VS2005, Python compiled wit VS.NET2003) is not supported, > right? I wasn't sure about that. In the transition from VC 6 to VC 7.x we didn't have compatibility problems. I was hoping this trend continues ... If that's not the case we will need two Python 2.5 installers, one compiled with VC 7.1, one with VC 8.0. I'll try to build SVN Python with VC 8.0. If that succeeds, is there an equivalent of "make install" that will give me a directory structure resembling what I get from the binary installer? Thanks! Cheers, Ralf __ Yahoo! DSL Something to write home about. Just $16.99/mo. or less. dsl.yahoo.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python + Visual C++ 8.0?
We have a bunch of C++ extensions (Boost.Python) that work fine under Windows when compiled with Visual C++ 7.1. With a few minor tweaks all extensions also compile with Visual C++ 8.0, but trying to "import" any of the extensions fails with this message (shown in a pop-up box): This application has failed to start because MSVCP80.dll was not found. Re-installing the application may fix this problem. I am using a Visual Studio 2005 Professional installation. I also ran vcredist_x86.exe. Moving msvc[mpr]80.dll to a directory on PATH didn't help. However, standalone executables work OK without any gymnastics. Does anyone know what the problem could be with the extensions? A quick attempt to compile Python from scratch using Visual C++ 8.0 produced a python.exe, but it doesn't run (the debug / send report / don't send report box pops up). Has someone tried this before? Cheers, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
--- David Abrahams <[EMAIL PROTECTED]> wrote: > Christoph Ludwig <[EMAIL PROTECTED]> writes: > > > I do not claim the 2 TUs test will cover all possible scenarios. I am not > even > > sure this decision should be left to an automated test. Because if the test > > breaks for some reason then the user is left with a linker error that is > > time-consuming to track down. > > However, at least by the usual hierarchy of values, the sort of > runtime error that results from the current needless linking with C++ > on ELF/Linux is even worse. Indeed. A few months ago the current configure behavior lead to a major loss of our time, probably a whole week between 4-5 people. The problem was that a Python compiled under RH 8.0 was used to build and run new C++ extensions under Fedroa Core 3. Some extensions ran OK, others crashed without warning after running to a certain point. It was very confusing. To avoid this situation in the future, we permanently added a test to our setup scripts, comparing the result of ldd python | grep libstdc++ to the corresponding output for extension modules. Cheers, Ralf Sell on Yahoo! Auctions no fees. Bid on great items. http://auctions.yahoo.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
--- David Abrahams <[EMAIL PROTECTED]> wrote: > Yes, all the tests are passing that way. > > > (On ELF based Linux/x86, at least.) That leaves me wondering > > > > * when is --with-cxx really necessary? > > I think it's plausible that if you set sys.dlopenflags to share > symbols it *might* end up being necessary, but IIRC Ralf does use > sys.dlopenflags with a standard build of Python (no > --with-cxx)... right, Ralf? Yes, I am using sys.setdlopenflags like this: if (sys.platform == "linux2"): sys.setdlopenflags(0x100|0x2) /usr/include/bits/dlfcn.h:#define RTLD_GLOBAL 0x00100 /usr/include/bits/dlfcn.h:#define RTLD_NOW 0x2 Note that the default Python 2.4.1 installation links python with g++. I've never had any problems with this configuration under any Linux version, at least: Redhat 7.3, 8.0, 9.0, WS3, SuSE 9.2, Fedora Core 3, and some other versions I am not sure about. Specifically for this posting I've installed Python 2.4.1 --without-cxx. All of our 50 Boost.Python extensions still work without a problem. Cheers, Ralf Sell on Yahoo! Auctions no fees. Bid on great items. http://auctions.yahoo.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __autoinit__
For those who didn't like my proposal a week ago, please have another look: http://mail.python.org/pipermail/python-list/2005-July/289446.html Please reply only to python-list. Cheers, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reducing self.x=x; self.y=y; self.z=z boilerplate code
Aahz wrote: > This is off-topic for python-dev. Please take it to comp.lang.python. > (It's not immediately obvious that this is off-topic, I know, but please > take my word for it.) Done: http://mail.python.org/pipermail/python-list/2005-July/288292.html Sorry for creating so much noise here. Cheers, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] reducing self.x=x; self.y=y; self.z=z boilerplate code
Josiah Carlson wrote: > Now, don't get me wrong, definining __slots__ can be a pain in the > tookus, but with a proper metaclass, that metaclass can define the > __slots__ attribute based on the argument list for __init__(). > > There you go. Where? The meta-class idea sounds interesting. Could you work it out? > A syntax change is wholly unnecessary. I wonder why everybody gets so agitated about a syntax enhancement proposal. I am not proposing a syntax change! I know enhancing the syntax is work, but shouldn't a syntax leading to less code clutter be the higher value? IMO a long-term gain counts for much more than avoiding a one-time investment implementing a useful feature. Take, for example, the syntax enhancement supporting "import foo as bar". I could just as easily write: import foo bar = foo del foo "A syntax change is wholly unnecessary." Why was it important enough anyway? It was a good enhancement because it is clearly more expressive and reduces clutter. In my experience the self.x=x; self.y=y etc. problem arises much more often than the problem solved by "import foo as bar". IMO a *built-in* solution is much more important in practice. Why does everybody have to reinvent the adopt_init_args/initialize wheel, or have to sheepishly type self.x=x...? I am not wedded to the .x idea, but counter-proposals should be at least approximately as powerful as what I propose. > class grouping: > def __init__(self, _x, _y, _z): > initialize(self, locals()) The "_x" alternative looks interesting but is problematic: it doesn't have a special meaning so far, but it is also not a syntax error. You are also not getting rid of the odd (from a learner's viewpoint) requirement to stick in "locals()". Remember where Python comes from: it goes back to a teaching language, enabling non-geeks to write programs. "initialize(self, locals())" doesn't fit that bill, especially if you have to import initialize first. In contrast def __init__(self, .x, .y, .z): almost looks like ^^ self.x and should therefore be easy to learn and remember. I'd also be happy with def __init__(self, self.x, self.y, self.z): which wouldn't be too different from unpacking tuples (and is currently a syntax error). However, remember, elegance = power/length. .x above has the same power as self.x, but is shorter, therefore more elegant. :) Hoping-that-this-leads-to-some-consensus-for-a-*built-in*-solution-ly yours, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reducing self.x=x; self.y=y; self.z=z boilerplate code
Jp Calderone wrote: > If you use vars(self).update(locals()), it even looks halfway > pleasant ;) I'm not sure what python-dev's current opinion of > vars(obj) is though (I'm hoping someone'll tell me). http://docs.python.org/lib/built-in-funcs.html#l2h-76 is pretty clear: vars([object]) Without arguments, return a dictionary corresponding to the current local symbol table. With a module, class or class instance object as argument (or anything else that has a __dict__ attribute), returns a dictionary corresponding to the object's symbol table. The returned dictionary should not be modified: the effects on the corresponding symbol table are undefined. > Of course, both of these fall over for __slots__'ful classes. It'd be > nice if there were a general way to deal with attributes of an > instance, regardless of the implementation details of its memory > layout. I agree. Ideally I'd like this class grouping: __slots__ = True def __init__(self, .x, .y, .z): pass to be equivalent to: class grouping: __slots__ = ["x", "y", "z"] def __init__(self, x, y, z): self.x = x del x self.y = y del y self.z = z del z Cheers, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reducing self.x=x; self.y=y; self.z=z boilerplate code
I am happy to see that others agree we need something better than self.x=x; self.y=y; self.z=z. Phillip J. Eby wrote: >class grouping: >def __init__(self, x, y, z): >initialize(self, locals()) Been there (older code): http://phenix-online.org/cctbx_sources/scitbx/scitbx/python_utils/misc.py I don't like it because - I do have to remember how to import adopt_init_args/initialize. - I also have to remember the locals() part (unpythonic boilerplate again). - I get both self.x and x. This lead to subtle bugs a few times when I accidentally assigned to x instead of self.x or vice versa in the wrong place). - It is sure to be less efficient than the .x support I propose. I'd be happy if - adopt_init_args/initialize became a (efficiently implemented) Python built-in. - and the locals() part is not needed. However, IMO the .x solution is still far better because I often want to do something like this: class grouping: def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again): pass With the adopt_init_args/initialize solution you'd have to write: class grouping: def __init__(self, keep_this, and_this, but_not_this, but_this_again): initialize(self, locals(), exclude=["but_not_this"]) Unpythonic boilerplate again (the but_not_this duplication). Cheers, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] reducing self.x=x; self.y=y; self.z=z boilerplate code
Hi, I often find myself writing: class grouping: def __init__(self, x, y, z): self.x = x self.y = y self.z = z I hate it, and every time I show this to a Python newcomer I get that skeptic look. How about this for a change? class grouping: def __init__(self, .x, .y, .z): pass This is supposed to work the same way as: def __init__(self, x, y, z): self.x = x del x self.y = y del y self.z = z del z Currently the .x syntax leads to: def __init__(self, .x, .y, .z): ^ SyntaxError: invalid syntax I.e. it seems to me that there shouldn't be any backward compatibility issues. I'll write a PEP if I hear a few voices of support. (Otherwise I'll stick to my "adopt_init_args" workaround: http://phenix-online.org/cctbx_sources/libtbx/libtbx/introspection.py which does a similar job but doesn't look as elegant and is also quite inefficient). Cheers, Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] proposal+patch: sys.gettickeraccumulation()
Brett C. wrote: > Ralf W. Grosse-Kunstleve wrote: > > Brett C. wrote: > > > > >I have been mulling over this proposal and I think I am finally > > >settling on +0 as long as it can be shown that it does not hurt > > >performance at all (yes, it shouldn't but this is the eval loop we are > > >talking about and that thing is a touchy beast). > > > > > > Is there a standard benchmark I could use to demonstrate the > > impact of the two new additions on the runtime? > > Thanks! > > =) Parrotbench and PyBench would be enough for me to sign off on any > performance hit. There is also pystone. I have done my homework now, based on parrotbench. Timings are below. The most important conclusion from my viewpoint is that my original goal can be achieved with a small patch and with a runtime penalty that is in the noise. However, mysterious runtime behavior of Objects/longobject.c lead me to change my original patch, and to the unanswered question "could small rearrangements in longobject.c potentially boost performance by 45% on some platforms?" Please don't be put off by the size of this message. My patch is essentially just one line in ceval.c (_Py_TickerAccumulation++)! To restate my original goal: I am looking for a simple way to answer the question: How much of a speedup can I expect if I reimplement a piece of Python code in C or C++? Note that I am not asking how much time is spent in Python including all the extensions that come with it, but how much time is spent in the Python bytecode interpreter loop. To clarify, look at the timings for two tests in parrotbench: b1 5.8287 time30049474 ticks0.1940 us/tick b2 1.5944 time 482584 ticks3.3039 us/tick Each of the tests was run with range(10) in b.py to increase accuracy. The first column shows time.time() differences, the second the number of bytecodes processed in ceval.c, and the last column shows micro-seconds per tick. Why is the us/tick value for b2 17 times larger than that for b1? The answer is that b2 makes heavy use of long integers, and that a lot of time is spent in Objects/longobject.c doing long-integer arithmetic. Compared to b1, relatively little time is spent interpreting bytecodes. Back to the original question: how much sense does it make to reimplement b1 or b2 in C? Simply looking at the us/tick values is telling me that I can expect much more of a performance boost by reimplementing b1 rather than b2. This is because b2 spends a lot of time in C already. A similar situation arises if extensions like Numeric are used. Computationally intensive operations like matrix multiplication and matrix inversion are already done at C speed. The us/tick let us quickly estimate how disappointed we would be after moving Python code to C or C++. We get this estimate without having to do detailed source code analysis, and before we have wasted our time doing the recoding (we have been through this a couple of times ...). The timings below show that the patched Python which counts the number of interpreted bytecodes is in the worst case 0.6% slower than the original Python, but some tests even run faster at face value: $gcc296_compiled/original/python -O b_timings.py | grep reporter b0 16.8831 time b1 5.9553 time b2 1.5914 time b3 10.8149 time b4 5.2528 time b5 11.6437 time b6 11.0248 time b7 27.6960 time all 90.9276 time $gcc296_compiled/work/python -O b_timings.py | grep reporter b0 16.9193 time50979585 ticks0.3319 us/tick b1 5.8287 time30049474 ticks0.1940 us/tick b2 1.5944 time 482584 ticks3.3039 us/tick b3 10.7931 time43577404 ticks0.2477 us/tick b4 5.2865 time18463044 ticks0.2863 us/tick b5 11.6086 time17065750 ticks0.6802 us/tick b6 10.0992 time6464 ticks0.1683 us/tick b7 27.6830 time 1928644 ticks 14.3536 us/tick all 89.8760 time 222548399 ticks0.4038 us/tick The b7 test is the same as b2 but with xrange(1) instead of xrange(1000), and with all print statements removed. Ratios (rounded to 3 decimals): 16.9193/16.8831=1.002 5.8287/5.9553 =0.979 1.5944/1.5914 =1.002 10.7931/10.8149=0.998 5.2865/5.2528 =1.006 11.6086/11.6437=0.997 10.0992/11.0248=0.916 27.6830/27.6960=1.000 Therefore I'd argue that the runtime penalty for the one additional long long increment in ceval.c (_Py_TickerAccumulation++) is in the noise. The timings were collected on a 2.8GHz Dual Xeon, Redhat WS 3. Python was compiled under Red Hat 7.3 with gcc 2.96. See below why. My latest patch can be found at the end of this posting. It can be applied t
