[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Gregory P. Smith added the comment: confirmed that this is present in 3.7 rev 72c34cf6dd as well. -- nosy: +ned.deily priority: high -> release blocker ___ Python tracker ___

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Gregory P. Smith added the comment: This is consistent for me, not random. run -m test.regrtest in -v mode and it passes. remove the -v and it fails. run Lib/test/test_pkg.py directly and it fails with the details mentioned here. this prevents CI builds from succeeding for me now.

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

pms.coder added the comment: I have the same issue on Debian 9: == CPython 3.8.0a0 (heads/master:874809e, Sep 6 2018, 23:31:00) [GCC 6.3.0 20170516] == Linux-4.9.0-6-amd64-x86_64-with-glibc2.17 little-endian == cwd: /home/xxx/xxx/cpython/git/cpython/build/test_python_55266 == CPU count: 4 ==

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Nick Coghlan added the comment: My guess as to why we're only seeing this for parallel test cases is taht for sequential tests, the implicit import inside open() is unlikely to happen while test_pkg is running, whereas for parallel tests, test_pkg will run in a relatively pristine process,

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Lorenz Mende added the comment: @jeremy: nice work The fix works for me reproducable, tests run normal now. Why dont you create a PR? Can someone explain why this issue only came up with parallel tests? -- ___ Python tracker

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Jeremy Kloth added the comment: Since my buildbot has been infected with this bug, I took some time to hunt it out. It turns out that issue is caused by an internal import triggered by the open() function (at least on Windows). A recent change to the interpreter (commit 9e4994d) changed

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Brett Cannon added the comment: Maybe I'm missing something, but who is racing whom in this case? All the examples people have shared are simply running the test module directly which means there's no parallelism in the execution of the test runner with other tests. Does unittest.main()

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Lorenz Mende added the comment: I was able to reproduce the issue with Win 10, current cpython @master. All sequential test runs are good. With parallel execution a race condition comes up, as Victor already mentioned. I was able to track the issue to the teardown function of the TestCase In

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Nick Coghlan added the comment: Given that importlib is essentially just doing "listdir", it would be interesting to know how the following behaves in a tight loop on affected systems: # Write a file f = open(os.path.join(dirname, "testname.py"), "w") f.write("text\n")

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

STINNER Victor added the comment: Failure on AMD64 Windows8.1 Non-Debug 3.x buildbot: https://buildbot.python.org/all/#/builders/12/builds/1223 ERROR: test_4 (test.test_pkg.TestPkg) FAIL: test_7 (test.test_pkg.TestPkg) -- ___ Python tracker

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

STINNER Victor added the comment: Brett: > I can't reproduce with master on macOS. Vladimir Matveev: > I've tried to repro this on Mac, Windows box and Windows VM - works fine for > all cases. Well, it's a race condition :-( It seems hard to reproduce, but it exists ;-) --

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

STINNER Victor added the comment: Recent failure on AMD64 Debian root 3.x: https://buildbot.python.org/all/#/builders/27/builds/1433 0:07:40 load avg: 2.19 [278/418/1] test_pkg failed -- running: test_tools (1 min 18 sec) test_1 (test.test_pkg.TestPkg) ... ok test_2 (test.test_pkg.TestPkg)

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Vladimir Matveev added the comment: I've tried to repro this on Mac, Windows box and Windows VM - works fine for all cases. -- nosy: +v2m ___ Python tracker ___

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Brett Cannon added the comment: I can't reproduce with master on macOS. -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

STINNER Victor added the comment: I removed Windows experts from the nosy list, since the issue is related to importlib, no Windows. -- components: -Windows nosy: +barry, brett.cannon, ncoghlan -paul.moore, steve.dower, tim.golden, xtreak, zach.ware

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Karthikeyan Singaravelan added the comment: Almost always fails 8/10 times. cpython git:(master) $ ./python -m unittest -v test.test_pkg test_1 (test.test_pkg.TestPkg) ... ok test_2 (test.test_pkg.TestPkg) ... ok test_3 (test.test_pkg.TestPkg) ... ok test_4 (test.test_pkg.TestPkg) ... ERROR

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

STINNER Victor added the comment: I'm also able to reproduce the bug on Python 3.6. Note: -X utf8 option is unrelated to this issue. -- versions: +Python 3.6 ___ Python tracker

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

STINNER Victor added the comment: The test creates the following files: /tmp/tmpsobc8gzf/t7.py /tmp/tmpsobc8gzf/t7 /tmp/tmpsobc8gzf/t7/__init__.py /tmp/tmpsobc8gzf/t7/sub.py /tmp/tmpsobc8gzf/t7/sub /tmp/tmpsobc8gzf/t7/sub/__init__.py /tmp/tmpsobc8gzf/t7/sub/.py /tmp/tmpsobc8gzf/t7/sub/subsub

[issue34200] importlib: python -m test test_pkg -m test_7 fails randomly

Change by STINNER Victor : -- title: python -X utf8 -m test test_pkg fails -> importlib: python -m test test_pkg -m test_7 fails randomly ___ Python tracker ___