[issue11653] Problems with some tests using -j2
Changes by Ned Deily n...@acm.org: -- status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Ned Deily n...@acm.org added the comment: Antoine is correct in principle, however LD_LIBRARY_PATH does not work on OS X. To test without installing, you would need to use DYLD_LIBRARY_PATH instead. Also keep in mind that, unlike most other systems, OS X binaries embed absolute paths to dynamic libraries. As a learning exercise for myself, I constructed this example. I added a printf to the errno module, a module that is included in the shared library, to help see what's happening: $ cd /tmp/a/py3k $ ./configure --prefix=/tmp/a/local --enable-shared LDFLAGS=-L/opt/local/lib CPPFLAGS=-I/opt/local/include make $ file ./libpython3.3m.dylib ./libpython3.3m.dylib: Mach-O 64-bit dynamically linked shared library x86_64 $ otool -L ./python ./python: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 550.42.0) /tmp/a/local/lib/libpython3.3m.dylib (compatibility version 3.3.0, current version 3.3.0) [...] # note, make install not yet run so lib not available in configured path $ ls /tmp/a/local/lib/libpython3.3m.dylib ls: /tmp/a/local/lib/libpython3.3m.dylib: No such file or directory $ ./python -c 'pass' dyld: Library not loaded: /tmp/a/local/lib/libpython3.3m.dylib Referenced from: /private/tmp/a/py3k/./python Reason: image not found Trace/BPT trap # override path so that uninstalled shared lib is found in build dir $ DYLD_LIBRARY_PATH=. ./python -c 'pass' errno module version 0 $ make install $ ls /tmp/a/local/lib/libpython3.3m.dylib /tmp/a/local/lib/libpython3.3m.dylib* # now lib is installed and found $ ./python -c 'pass' errno module version 0 # edit message in errno module to version 1 $ make # have not done a new install so old shared lib is found $ ./python -c 'pass' errno module version 0 # enable dynamic loader printing of loads $ export DYLD_PRINT_LIBRARIES=1 $ ./python -c 'pass' dyld: loaded: /private/tmp/a/py3k/./python dyld: loaded: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation dyld: loaded: /tmp/a/local/lib/libpython3.3m.dylib [...] errno module version 0 # override loads from build directory - works correctly $ DYLD_LIBRARY_PATH=. ./python -c 'pass' dyld: loaded: /private/tmp/a/py3k/./python dyld: loaded: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation dyld: loaded: /private/tmp/a/py3k/libpython3.3m.dylib errno module version 1 # install updated lib $ make install # also works correctly $ ./python -c 'pass' dyld: loaded: /private/tmp/a/py3k/./python dyld: loaded: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation dyld: loaded: /tmp/a/local/lib/libpython3.3m.dylib [...] errno module version 1 So the moral of the story is: if you use --enable-shared, you need to be careful to either make install or to use DYLD_LIBRARY_PATH=/path/to/build_directory. Skip, are you OK with closing this issue? -- resolution: - works for me stage: - committed/rejected status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Skip Montanaro s...@pobox.com added the comment: Ned Skip, what parameters are you using with ./configure ? Pretty vanilla. Install in my directory tree, get libraries from MacPorts: --prefix=/Users/skip/local --enable-shared LDFLAGS=-L/opt/local/lib CPPFLAGS=-I/opt/local/include I thought you had fixed the --enable-shared linkage problems, but taking it out seems to solve this issue. I'm guessing nothing has been checked in yet to solve that problem. S -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Antoine Pitrou pit...@free.fr added the comment: Pretty vanilla. Install in my directory tree, get libraries from MacPorts: --prefix=/Users/skip/local --enable-shared LDFLAGS=-L/opt/local/lib CPPFLAGS=-I/opt/local/include I thought you had fixed the --enable-shared linkage problems, but taking it out seems to solve this issue. I'm guessing nothing has been checked in yet to solve that problem. It's not --enable-shared, it's that you have to set LD_LIBRARY_PATH=., otherwise your new ./python.exe will use some other Python 3.3 shared library installed on your system. So you're basically testing an old Python with the latest test suite :) We do have a buildbot testing shared builds btw: http://www.python.org/dev/buildbot/all/buildslaves/bolen-ubuntu -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
New submission from Skip Montanaro s...@pobox.com: At Antoine's behest, I tried running ./python.exe -m test -j2 in my cpython sandbox and saw several test case failures which didn't appear when I executed a simple make test He suggested I add the -W flag and try again. I did, but I don't see any difference in the output, so either I'm still doing something wrong or the -W flag is somehow mishandled. As far as I can tell the failing tests were not re-run in verbose mode. script command output attached. I ran these tests on Mac OS X Leopard (10.5.8). S -- components: Tests files: typescript messages: 131896 nosy: skip.montanaro priority: normal severity: normal status: open title: Problems with some tests using -j2 versions: Python 3.3 Added file: http://bugs.python.org/file21357/typescript ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +ezio.melotti, r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Brett Cannon br...@python.org added the comment: It should have been '-w', not '-W'. -- nosy: +brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Skip Montanaro s...@pobox.com added the comment: Thanks. That worked better. Here is the tail end of the output showing the verbose test output. -- Added file: http://bugs.python.org/file21362/output ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Antoine Pitrou pit...@free.fr added the comment: Brett, -W exists too and it seems it failed working here. Skip, can you please try make distclean and then rebuild from scratch? -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Éric Araujo mer...@netwok.org added the comment: I think the pydoc failure is due to haypo’s patch in #3080. Another patch (by me) in #8754 has a fix. -- nosy: +eric.araujo, haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Skip Montanaro s...@pobox.com added the comment: Antoine Skip, can you please try make distclean and then rebuild from Antoine scratch? I did. The last output was after make distclean ./configure ... my usual suspects ... make ./python.exe -j2 -w I will try one more time later this evening with a capital W. I have a train to catch first. Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Antoine Pitrou pit...@free.fr added the comment: Antoine Skip, can you please try make distclean and then rebuild from Antoine scratch? I did. The last output was after Oops, sorry. I will try one more time later this evening with a capital W. I have a train to catch first. Well, if -W didn't work once, it probably won't work twice. Not sure you want to bother. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Roundup Robot devnull@devnull added the comment: New changeset 9aa6097131ef by Antoine Pitrou in branch '3.2': Issue #11653: fix -W with -j in regrtest http://hg.python.org/cpython/rev/9aa6097131ef New changeset c381b35e4f31 by Antoine Pitrou in branch 'default': Issue #11653: fix -W with -j in regrtest. http://hg.python.org/cpython/rev/c381b35e4f31 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Skip Montanaro s...@pobox.com added the comment: Antoine New changeset 9aa6097131ef by Antoine Pitrou in branch '3.2': Antoine Issue #11653: fix -W with -j in regrtest Antoine http://hg.python.org/cpython/rev/9aa6097131ef Antoine New changeset c381b35e4f31 by Antoine Pitrou in branch 'default': Antoine Issue #11653: fix -W with -j in regrtest. Antoine http://hg.python.org/cpython/rev/c381b35e4f31 Saw these checkins and tried again. I definitely see different output and it seems to rerun failed tests immediately in verbose mode. For example: [238/352] test_pydoc Re-running test test_pydoc in verbose mode test_badimport (test.test_pydoc.PydocDocTest) ... FAIL test_help_output_redirect (test.test_pydoc.PydocDocTest) ... ok test_html_doc (test.test_pydoc.PydocDocTest) ... ok test_input_strip (test.test_pydoc.PydocDocTest) ... ok test_issue8225 (test.test_pydoc.PydocDocTest) ... ok test_not_here (test.test_pydoc.PydocDocTest) ... ok test_stripid (test.test_pydoc.PydocDocTest) ... ok test_text_doc (test.test_pydoc.PydocDocTest) ... ok test_class (test.test_pydoc.TestDescriptions) ... ok test_module (test.test_pydoc.TestDescriptions) ... ok test_server (test.test_pydoc.PydocServerTest) ... ok test_content_type_err (test.test_pydoc.PydocUrlHandlerTest) ... ok test_url_requests (test.test_pydoc.PydocUrlHandlerTest) ... ok However, I also see lots of tests crashing in imp. Here's one example: test test_ttk_textonly crashed -- Traceback (most recent call last): File /Users/skip/src/hgpython/cpython/Lib/test/regrtest.py, line 988, in runtest_inner the_package = __import__(abstest, globals(), locals(), []) File /Users/skip/src/hgpython/cpython/Lib/test/test_ttk_textonly.py, line 7, in module from tkinter.test import runtktests File /Users/skip/src/hgpython/cpython/Lib/importlib/_bootstrap.py, line 437, in load_module return self._load_module(fullname) File /Users/skip/src/hgpython/cpython/Lib/importlib/_bootstrap.py, line 141, in decorated return fxn(self, module, *args, **kwargs) File /Users/skip/src/hgpython/cpython/Lib/importlib/_bootstrap.py, line 330, in _load_module code_object = self.get_code(name) File /Users/skip/src/hgpython/cpython/Lib/importlib/_bootstrap.py, line 407, in get_code imp._fix_co_filename(found, source_path) AttributeError: 'module' object has no attribute '_fix_co_filename' Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Brett Cannon br...@python.org added the comment: The import failures are because you didn't rebuild (I added a new function to imp). Re-build and the import issues will go away. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Skip Montanaro s...@pobox.com added the comment: Brett The import failures are because you didn't rebuild (I added a new Brett function to imp). Re-build and the import issues will go away. An easy mistake to make. I'm used to running make test which would have guaranteed python.exe was up-to-date. I just executed hg pull -u make ./python.exe -m test -j2 -W and I am seeing the same AttributeError exceptions. So: make distclean ./configure ... make ./python.exe -m test -j2 -W and I still get the AttributeError exceptions. According to hg id I am using this: changeset: 68889:f09f7ab40ce6 tag: tip user:Raymond Hettinger pyt...@rcn.com date:Wed Mar 23 20:33:30 2011 -0700 summary: Add tests for _source to importable and exec'able. Is that what I should be using? Skip -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Ned Deily n...@acm.org added the comment: Skip, what parameters are you using with ./configure ? -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11653] Problems with some tests using -j2
Brett Cannon br...@python.org added the comment: You have a build problem, Skip. On my system and at least all of the stable buildbots for 3.3 there are no test failures from imp._fix_co_filename(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11653 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
