[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x buildbots:

https://buildbot.python.org/all/#/builders/154/builds/104

It seems that it has some problems with (unclosed?) file descriptors:

test_terminate_dead (test.test_subprocess.Win32ProcessTestCase) ... skipped 
'Windows specific tests'
==
FAIL: test_close_fds (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/test_subprocess.py", 
line 2378, in test_close_fds
"Some fds were left open")
AssertionError: {3} is not false : Some fds were left open
==
FAIL: test_close_fds_after_preexec (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/test_subprocess.py", 
line 2653, in test_close_fds_after_preexec
self.assertNotIn(fd, remaining_fds)
AssertionError: 3 unexpectedly found in {0, 1, 2, 3}
==
FAIL: test_pass_fds (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/support/__init__.py", 
line 615, in wrapper
return func(*args, **kw)
  File 
"/home/buildbot/buildarea/3.x.einat-ubuntu/build/Lib/test/test_subprocess.py", 
line 2503, in test_pass_fds
"fd to be closed passed")
AssertionError: {4} is not false : fd to be closed passed
--
Ran 285 tests in 68.129s
FAILED (failures=3, skipped=28)
1 test failed again:
test_subprocess

--
components: Tests
messages: 323001
nosy: pablogsal
priority: normal
severity: normal
status: open
title: test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This also happens on x86-64 High Sierra 3.x buildbot:

https://buildbot.python.org/all/#/builders/145/builds/265

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

More failures on AMD64 Debian PGO 3.x:

https://buildbot.python.org/all/#/builders/47/builds/1362

test_terminate (test.test_subprocess.Win32ProcessTestCase) ... skipped 'Windows 
specific tests'
test test_subprocess failed
test_terminate_dead (test.test_subprocess.Win32ProcessTestCase) ... skipped 
'Windows specific tests'
==
FAIL: test_close_fds (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
"/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_subprocess.py",
 line 2378, in test_close_fds
"Some fds were left open")
AssertionError: {3} is not false : Some fds were left open
==
FAIL: test_close_fds_after_preexec (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
"/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_subprocess.py",
 line 2653, in test_close_fds_after_preexec
self.assertNotIn(fd, remaining_fds)
AssertionError: 3 unexpectedly found in {0, 1, 2, 3}
==
FAIL: test_pass_fds (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
"/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/support/__init__.py",
 line 615, in wrapper
return func(*args, **kw)
  File 
"/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_subprocess.py",
 line 2503, in test_pass_fds
"fd to be closed passed")
AssertionError: {4} is not false : fd to be closed passed
--
Ran 285 tests in 27.876s
FAILED (failures=3, skipped=28)
1 test failed again:
test_subprocess

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Some of VSTS Mac OS and Linux builds also have the same failure 

Sample Linux : 
https://python.visualstudio.com/cpython/_build/results?buildId=21154&view=logs
Sample Mac OS : 
https://python.visualstudio.com/cpython/_build/results?buildId=21155&view=logs

Test case failure reported in Arch : https://bugs.python.org/issue34327

Thanks

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Same issue with my builds : 

* Linux -> 
https://python.visualstudio.com/cpython/_build/results?buildId=21273&view=logs
* macOS -> 
https://python.visualstudio.com/cpython/_build/results?buildId=21274&view=logs

--
nosy: +matrixise, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

same problem here:
linux: 
https://python.visualstudio.com/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_build/index?buildId=21241
macOS: 
https://python.visualstudio.com/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_build/index?buildId=21242

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

When I try to find the issue with git bisect, I have one commit.

72ec3193b5118a2ccc8be8bf03d7b74691c6a264 is the first bad commit
commit 72ec3193b5118a2ccc8be8bf03d7b74691c6a264
Author: Victor Stinner 
Date:   Thu Aug 2 19:34:20 2018 +0200

bpo-34170: Cleanup pymain_run_filename() (GH-8631)

* Inline pymain_run_file() and pymain_open_filename() into
  pymain_run_filename()
* Created pymain_run_stdin() which is pymain_run_filename() with
  filename=NULL
* Rename pymain_run_filename() to pymain_run_file()

:04 04 6e49f5791bc63c3bf67a3674ccc00a412b5f089a 
1b93e169af3afb07daaf06e2247a3eae6a88cac5 M  Modules

with ./python -m test test_subprocess -v

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

I continue to work on this issue

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

I think there is an issue with pymain_run_file, in the new code of Victor, a 
file descriptor is not closed but I don't know why.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Maybe we can strace a failing test and cross check the opens and the closes to 
see which one is not being closed. Or monitor /proc/PID/fd while the test runs. 
I can investigate later today myself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Pablo, the main issue comes after this commit, you can strace the call 
for example 

./python -m unittest test.test_subprocess.POSIXProcessTestCase.test_pass_fds

on this test function and you will see than there is a opened file descriptor.

I tried to rewrite the patch of victor and see what was the difference, and 
there is no issue with pymain_run_stdin but I have the issue with 
pymain_run_file.

after that, I don't know how to find the leaked file.

Any idea?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +8155
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d8078626770a8d358eb83d7928c12d75ff4e821a by Victor Stinner in 
branch 'master':
bpo-34170: Fix pymain_run_file() (GH-8660)
https://github.com/python/cpython/commit/d8078626770a8d358eb83d7928c12d75ff4e821a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Thanks Victor, I don't know enough the API C-Python :/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34326] test_subprocess.POSIXProcessTestCase fails in AMD64 Ubuntu 3.x

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com