[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[miss-islington]

miss-islington  added the comment:


New changeset faadc52e755cdb316a53f3db5aa11cb97f1c4b87 by Miss Islington (bot) 
in branch '3.9':
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271)
https://github.com/python/cpython/commit/faadc52e755cdb316a53f3db5aa11cb97f1c4b87


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[miss-islington]

miss-islington  added the comment:


New changeset 8a4557240b98c322d611bfbba3ea51eac3fb841a by Miss Islington (bot) 
in branch '3.8':
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271)
https://github.com/python/cpython/commit/8a4557240b98c322d611bfbba3ea51eac3fb841a


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Steve Dower]

Change by Steve Dower :


--
assignee:  -> steve.dower
nosy:  -miss-islington
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 9.0 -> 10.0
pull_requests: +22169
pull_request: https://github.com/python/cpython/pull/23273

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[miss-islington]

Change by miss-islington :


--
pull_requests: +22170
pull_request: https://github.com/python/cpython/pull/23274

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Steve Dower]

Steve Dower  added the comment:


New changeset 9b6934230c35e24d8582ea8c58456fa8eab72ae2 by Steve Dower in branch 
'master':
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271)
https://github.com/python/cpython/commit/9b6934230c35e24d8582ea8c58456fa8eab72ae2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +22168
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23271

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[honglei jiang]

honglei jiang  added the comment:

Still missing _testinternalcapi module for Python3.9rc2
like https://bugs.python.org/issue36635

C:\Users\Admin>c:\python39\python.exe -m test test_deque
0:00:00 Run tests sequentially
0:00:00 [1/1] test_deque
test test_deque failed -- Traceback (most recent call last):
  File "c:\python39\lib\test\test_deque.py", line 781, in test_sizeof
check(deque(), basesize + blocksize)
  File "c:\python39\lib\test\support\__init__.py", line 1484, in check_sizeof
import _testinternalcapi
ModuleNotFoundError: No module named '_testinternalcapi'

test_deque failed

== Tests result: FAILURE ==

1 test failed:
test_deque

Total duration: 1.1 sec
Tests result: FAILURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[honglei jiang]

honglei jiang  added the comment:

c:\Python39>python -m test -j4
== CPython 3.9.0b2 (tags/v3.9.0b2:0b1a1c8, Jun 8 2020, 22:55:31) [MSC v.1924 64 
bit (AMD64)]
== Windows-10-10.0.18362-SP0 little-endian
== cwd: C:\Users\jhong\AppData\Local\Temp\test_python_5960
== CPU count: 4
== encodings: locale=cp936, FS=utf-8
0:00:00 Run tests in parallel using 4 child processes

== Tests result: FAILURE ==

358 tests OK.

24 tests failed:
test__locale test_array test_buffer test_capi test_deque
test_exceptions test_flufl test_fstring test_grammar
test_itertools test_locale test_memoryio test_ordered_dict
test_parser test_pickle test_positional_only_arg
test_string_literals test_struct test_syntax test_sys
test_traceback test_urllib test_urllib2 test_xml_etree_c

43 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_curses
test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl
test_fork1 test_gdb test_grp test_ioctl test_kqueue
test_multiprocessing_fork test_multiprocessing_forkserver test_nis
test_openpty test_ossaudiodev test_pipes test_poll test_posix
test_pty test_pwd test_readline test_resource test_smtpnet
test_socketserver test_spwd test_syslog test_threadsignals
test_timeout test_tix test_tk test_ttk_guionly test_urllib2net
test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net
test_xxtestfuzz test_zipfile64

Total duration: 6 min 44 sec
Tests result: FAILURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Ned Deily]

Ned Deily  added the comment:

> The Windows installer also runs pip and IDLE tests before being published 
> [...]

That's good and reminds me that I forgot to mention that for the macOS 
installer, besides the already mentioned tests, we also download, install, and 
smoke test a third-party wheel from PyPI using the bundled pip and we also do a 
visual inspection of the release's changelog by launching through IDLE the 
docset bundled with each installer. If the Python binaries included with the 
installer are built for multiple CPU architectures (macOS universal binaries), 
we would also attempt to ensure that the test suite is run at least once on 
each architecture (as of several months ago, we no longer regularly distribute 
such binaries but there's a good chance we may need to do so again in the 
not-too-distant future, for example, if Apple introduces macOS machines based 
on ARM CPUs).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Steve Dower]

Steve Dower  added the comment:

There are non-binding CI tests run with Github Actions (post-merge) that test 
from a layout, and last I checked they were all passing. Nobody is really 
monitoring it but me though, as it isn't a buildbot. 

The Windows installer also runs pip and IDLE tests before being published, 
though not a full test suite.

Honestly, it's pretty quick to generate an install layout for Windows now I 
have the PC/layout script. We could do it for every test run if we wanted, 
though then a number of build-tree only tests would be skipped.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Ned Deily]

Ned Deily  added the comment:

> So it seems that only Linux has "installed" builbot tests. I suppose the next 
> step is adding some macOS and Windows ones.

Having more buildbots test from "installed" locations, rather than the build 
directory, is good.

But that's not the whole issue here. Those "installed" tests are only testing 
that a Python runs from an installed location on the same system it was built.  
The Windows and macOS installers use various packaging methods to provide 
complete Python installations, including binaries, that can be installed on 
other systems using various OS-native install methods familiar to users of each 
platform. And the binaries are generally built in such a way to support running 
on multiple release versions of its operating system. For example, the current 
Python installers for macOS from python.org are usable on macOS 10.9 through 
10.15. To produce the various installer packages, there are other layers of 
tooling involved, say, on top of "make install" in the Unixy build process for 
macOS; something similar is true for Windows releases.

FWIW, every macOS installer is test installed on a vanilla system, the whole 
standard test suite is run, and an IDLE smoke test is run to ensure there are 
no noticeable regressions before handing the build artifacts off to the release 
manager. Every installer is tested on at least one supported macOS version and 
usually more than one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

We have a bunch of buildbots that test the installed Python. For instance, if 
you search for "installed" in 
https://buildbot.python.org/all/#/builders?tags=%2B3.x we have:

aarch64 Fedora Rawhide Clang Installed 3.x  
aarch64 Fedora Stable Clang Installed 3.x   
AMD64 Fedora Rawhide Clang Installed 3.x
AMD64 Fedora Stable Clang Installed 3.x
PPC64LE Fedora Rawhide Clang Installed 3.x
PPC64LE Fedora Stable Clang Installed 3.x
s390x Fedora Clang Installed 3.x
s390x Fedora Rawhide Clang Installed 3.x
x86 Gentoo Installed with X

So it seems that only Linux has "installed" builbot tests. I suppose the next 
step is adding some macOS and Windows ones.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40754] Test installers before releasing (ModuleNotFoundErrors)

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I verified that running tests for installed 64-bit 3.9.0b1 on Win10 is somewhat 
a joke.  I got 22 failures.  Multiple tests failed on missing 
_testinternalcapi, and at least 1 on _xxtestfuzz.  Several tests had multiple 
failures and would need to be run individually and verbosely to see why.  For 
instance, test_itertools tries to run test.support.__init__.checksizeof 4 
times, and the latter imports _testinternalcapi, resulting is 4 failures.

Release managers: I think installers should be run and the suite run with the 
result.  This is not the first time a file has been omitted.  One needed by 
IDLE was once omitted, I believe on Windows.  There is now a test that reads 
it, but is useless unless run.  (And I should add checks for all the other 'not 
.py' files.)

honglei: Thank you for both posts.  Next time, please remove the quoted message 
when replying by email.  Once posted on the web page, below the quoted message, 
the quote becomes redundant noise.

--
nosy: +lukasz.langa, ned.deily, pablogsal, terry.reedy
title: ModuleNotFoundError: No module named '_testinternalcapi' under Win10 -> 
Test installers before releasing (ModuleNotFoundErrors)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com