[issue46524] test_peg_generator takes 8 minutes on Windows

Gregory P. Smith added the comment: Thanks to Kumar for contributing Windows compiler flags side of this (the point of this issue). -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

STINNER Victor added the comment: Duration of the "Tests" step of https://github.com/python/cpython/pull/30890 * GHA win32: 14 min 11 sec (test_peg_generator: 8 min 16 sec) * GHA win64: 21 min 2 sec (test_peg_generator: 16 min 38 sec) * Azure win32: 9 min 34 sec (test_peg_generator: 5 min 30

[issue46524] test_peg_generator takes 8 minutes on Windows

Gregory P. Smith added the comment: test_peg_generator is significantly less of the long tail on optimized builds now. CI extremely noisy performance wise (times vary by 2-3x without any differences anways) so I can't easily judge if this made a notable difference in windows CI latency.

[issue46524] test_peg_generator takes 8 minutes on Windows

miss-islington added the comment: New changeset e8258608c28c65680253d0ca6167430e34c2fd87 by Miss Islington (bot) in branch '3.9': [3.9] [3.10] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (GH-31015) (GH-31089) (GH-31093)

[issue46524] test_peg_generator takes 8 minutes on Windows

Change by miss-islington : -- nosy: +miss-islington nosy_count: 10.0 -> 11.0 pull_requests: +29279 pull_request: https://github.com/python/cpython/pull/31093 ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

Gregory P. Smith added the comment: New changeset f5ebec4d3e1199ec38b88920cfde8e460e5120dd by Gregory P. Smith in branch '3.10': [3.10] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (GH-31015) (GH-31089)

[issue46524] test_peg_generator takes 8 minutes on Windows

Change by Gregory P. Smith : -- pull_requests: +29274 pull_request: https://github.com/python/cpython/pull/31089 ___ Python tracker ___

[issue46524] test_peg_generator takes 8 minutes on Windows

Gregory P. Smith added the comment: New changeset 164a017e13ee96bd1ea1ae79f5ac9e25fe83994e by Gregory P. Smith in branch 'main': bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (#31015)

[issue46524] test_peg_generator takes 8 minutes on Windows

Change by Gregory P. Smith : -- pull_requests: +29197 pull_request: https://github.com/python/cpython/pull/31015 ___ Python tracker ___

[issue46524] test_peg_generator takes 8 minutes on Windows

Gregory P. Smith added the comment: If a decent parallelism CI systems are not available from github (they seem stuck at 2-3 threads per https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners), an alternative approach could be to shard across multiple

[issue46524] test_peg_generator takes 8 minutes on Windows

Change by Kumar Aditya : -- keywords: +patch pull_requests: +29196 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31017 ___ Python tracker ___

[issue46524] test_peg_generator takes 8 minutes on Windows

Gregory P. Smith added the comment: re: slow tests in the first half of the list. the same total amount of time is going to be spent regardless. In our test suite on a modern fast 16 thread system, all but 10 tests are completed in parallel within the first 30 seconds. The remaining ~10

[issue46524] test_peg_generator takes 8 minutes on Windows

Karthikeyan Singaravelan added the comment: See also https://bugs.python.org/issue46576 and https://github.com/python/cpython/pull/31015 -- nosy: +xtreak ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

Terry J. Reedy added the comment: Do all of the tests use all of the slowly built extensions, or could the test file be split into multiple files run separately, each faster? -- nosy: +terry.reedy ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

Eric Snow added the comment: On Tue, Jan 25, 2022 at 4:14 PM STINNER Victor wrote: > Currently, most CI run "make buildbottest" which uses -r option of > libregrtest: randomize tests order. How hard would it be to first randomize the list and then move the slow tests up to a random position

[issue46524] test_peg_generator takes 8 minutes on Windows

Mark Shannon added the comment: It's plenty slow on linux as well. I like the idea of starting the slower tests first. The long tail of slow tests is annoying when running `make -j12 test`. -- nosy: +Mark.Shannon ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

Kumar Aditya added the comment: Another solution would be to shard the tests on windows i.e. run tests on two different jobs concurrently, edgedb does this. See https://github.com/edgedb/edgedb/actions/runs/1746736086 -- ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

STINNER Victor added the comment: > Would it be possible to skip these tests when no changes to the parser > related code are made to speed up tests? Maybe, some CIs could export an environment variable which contains the list of modified files, and then each file would be able to decide

[issue46524] test_peg_generator takes 8 minutes on Windows

Kumar Aditya added the comment: Would it be possible to skip these tests when no changes to the parser related code are made to speed up tests? -- nosy: +kumaraditya303 ___ Python tracker

[issue46524] test_peg_generator takes 8 minutes on Windows

STINNER Victor added the comment: > In which case it would be generally helpful to start the longest-running > tests first. Currently, most CI run "make buildbottest" which uses -r option of libregrtest: randomize tests order. -- ___ Python

[issue46524] test_peg_generator takes 8 minutes on Windows

Tim Peters added the comment: As a general thing, I expect people on Windows always run the tests with multiple processes. In which case it would be generally helpful to start the longest-running tests first. As is, because of its late-in-the-alphabet name, "test_peg_generator" gets started

[issue46524] test_peg_generator takes 8 minutes on Windows

Pablo Galindo Salgado added the comment: The test needs to build a lot of C extensions with different parsers, and that compilation is what takes most of the time. I don't think we should skip these tests by default on Windows, as it gives us valuable information (that the parser features

[issue46524] test_peg_generator takes 8 minutes on Windows

Change by Tim Peters : -- nosy: +tim.peters ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue46524] test_peg_generator takes 8 minutes on Windows

New submission from STINNER Victor : test_peg_generator takes between 5 and 16 minutes on the Windows CI run on Python pull requests. Example of timings on my PR https://github.com/python/cpython/pull/30890 * GitHub Action win32: 8 min 16 sec * GitHub Action win64: 16 min 38 sec * Azure