Branch: refs/heads/smoke-me/jkeenan/t-harness-revamp-20261008
Home: https://github.com/Perl/perl5
Commit: 1e24ecf3d90e82e2f526eccc5bb3abbcb93c619e
https://github.com/Perl/perl5/commit/1e24ecf3d90e82e2f526eccc5bb3abbcb93c619e
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/TEST
M t/harness
M t/porting/test_testlist.t
Log Message:
-----------
t/TEST: Change /dump_?tests/ to 'dump_manifested_tests'
Currently, the '-dumptests' switch in t/TEST only reports tests
listed in the MANIFEST. In preparation for the addition/restoration of
tests generated via 'make' to the list of tests run by t/TEST, we need
to name things more precisely.
t/harness: For consistency with t/TEST, rename '@tests' and similar to
'@manifested_tests,' i.e., tests listed in MANIFEST.
Commit: 034a7bedc9e9667e2bb46a4409e1afcece9374e9
https://github.com/Perl/perl5/commit/034a7bedc9e9667e2bb46a4409e1afcece9374e9
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Rather than work on global @ARGV, clone and rename it
Currently, @ARGV starts out with a mixture of command-line switches and files to
be tested. We then do lots of pushing onto it and re-writing elements.
This means that during debugging saying x \@ARGV doesn't show us what we
originally passed to the program. Instead, assign to a lexically-scoped
variable. Rename some other variables similarly.
Bring other aspects of t/TEST up to at least 2005-level best practices,
e.g., use lexical file and directory handles; use 3-argument 'open'.
Use 3-argument 'open'.
Commit: 086529718d8cbc47687e7f0c6be5c55d04777ceb
https://github.com/Perl/perl5/commit/086529718d8cbc47687e7f0c6be5c55d04777ceb
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Add generated test files to list to be tested
Commit: 25e83b441f2599138a89517ffc5e6fcc603928da
https://github.com/Perl/perl5/commit/25e83b441f2599138a89517ffc5e6fcc603928da
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Corrections to t/TEST only
Leaving t/harness unchanged.
In t/harness, to change @manifested_tests (back) to @tests (making
similar changes to dump_manifested_tests) causes test failures in
t/porting/test_testlist.t.
So this commit makes changes in t/TEST suggested during code review but,
for the time being, leaves t/harness untouched.
On the one hand, that enables t/porting/test_testlist.t to PASS. It
also keeps the generated tests from running when you are running
t/porting/test_testlist.t
On the other hand, it means that certain identifiers within t/harness
are misleadingly named. As Tony pointed out in code review, the list of
tests accumulated during t/harness are not gleaned from the MANIFEST
and, hence, do not really qualify as "manifested" tests.
During development of GH #24918 I was trying to leave
t/porting/test_testlist.t as untouched as possible. That was because
that test program, introduced in 2023, had as its objective:
# Test that t/TEST and t/harness test the same files, and that all the
# test files (.t files) listed in MANIFEST are tested by both.
To achieve that objective, test_testlist.t relies on the assumption that
each of t/TEST and t/harness have a command-line switch named
`-dumptests` that will produced the same results.
Commit: 1bfe8dd35ee4ae4fd878eea67542dc00b0fa6788
https://github.com/Perl/perl5/commit/1bfe8dd35ee4ae4fd878eea67542dc00b0fa6788
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
M t/porting/test_testlist.t
Log Message:
-----------
test_testlist.t: Temporarily skip all tests
So as to accommodate reversion of 'manifested_tests' in t/harness to
just 'tests' ... and that's so as not to get in our way when
implementing generated tests in t/harness.
Commit: d63eb4f1c2a53f1a9466d1933169114cb6dc64e4
https://github.com/Perl/perl5/commit/d63eb4f1c2a53f1a9466d1933169114cb6dc64e4
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Restore name of dump_manifested_tests() to dump_tests()
There's nothing within this subroutine that is specific to
manifested_tests. If we simply call it dump_tests() we can use it
within t/harness without redefining it.
Commit: 36bb9710d52d2733a46b0f73204e759d8649d9b5
https://github.com/Perl/perl5/commit/36bb9710d52d2733a46b0f73204e759d8649d9b5
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
Uncuddle some else's
Add inline comments where we're pulling in subroutines defined in
t/TEST.
Some other small tidying.
Commit: e88ae090be200bee2e2960dd92721fa3dce47a1e
https://github.com/Perl/perl5/commit/e88ae090be200bee2e2960dd92721fa3dce47a1e
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
Move declaration of variable closer to where used
Commit: 1dd3bf0b86b931e4da6837368dbd0a6fa4aec04d
https://github.com/Perl/perl5/commit/1dd3bf0b86b931e4da6837368dbd0a6fa4aec04d
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
M t/porting/test_testlist.t
Log Message:
-----------
Reactivate (simpler) t/porting/test_testlist.t
At this point, we're comparing the manifested tests found by t/TEST to
match all the tests found by t/harness. We have not yet implemented
testing of generated tests in t/harness, so we're not yet concerned with
the number or identity of generated tests found by t/TEST.
Commit: 3c259046bf244c1322a660645088e52ec6402c99
https://github.com/Perl/perl5/commit/3c259046bf244c1322a660645088e52ec6402c99
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
t/harness: Locate generated tests and test them
Commit: 5fa3bc1407d79122601125be3a9ae3da53561710
https://github.com/Perl/perl5/commit/5fa3bc1407d79122601125be3a9ae3da53561710
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
Don't mix specific tests with generated tests
Commit: eee1abfa75f3e56e1c39c3e55acc22983729b602
https://github.com/Perl/perl5/commit/eee1abfa75f3e56e1c39c3e55acc22983729b602
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Pause test_testlist.t temporarily
Commit: 9d8468f7f0568583b0615b3fb7b6f12284bb766a
https://github.com/Perl/perl5/commit/9d8468f7f0568583b0615b3fb7b6f12284bb766a
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Reposition identification of generated tests
Commit: 3beedcf6f145aa7f88ab289794236a308a937034
https://github.com/Perl/perl5/commit/3beedcf6f145aa7f88ab289794236a308a937034
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
_compute_tests_and_ordering: Improve encapsulation
%total_time was used only within _compute_tests_and_ordering() and
sort_by_execution_order. The latter, in turn, was/is only called within
_compute_tests_and_ordering(). Bring declaration of %total_time inside
_compute_tests_and_ordering(). Transform sort_by_execution_order()
into a lexical sub declared within _compute_tests_and_ordering().
Commit: b12802470cc9581a4856fc2a2e41c8f3149a6ce7
https://github.com/Perl/perl5/commit/b12802470cc9581a4856fc2a2e41c8f3149a6ce7
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
test_testlib.t: Add testing subroutines and use them
Add 3 subroutines which can be used to test a given list of test files
via t/TEST, t/harness and ./runtests choose, respectively.
Add 2 subroutines used to create lookup tables within the 3
aforementioned subroutines.
Rearrange the file so that subroutines are defined beneath the test code
and superseded tests are placed in an END block. (We'll get back to
them later.)
Commit: dd5ddd947080d9789d90738c1209ded692566b66
https://github.com/Perl/perl5/commit/dd5ddd947080d9789d90738c1209ded692566b66
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Test generated tests for dist/Devel-PPPort
Using recently developed testing subroutines. Add inline comments;
deleted commented-out code.
Commit: 5ada024e2a2afff30d4051c3e5c760863d8cca6f
https://github.com/Perl/perl5/commit/5ada024e2a2afff30d4051c3e5c760863d8cca6f
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
test_testlist.t: Restore pre-5.43.4 tests
Suitably tweaked.
Commit: 1f5ed22b48bb80b6fbd32c1c7f65e4a4ef55c482
https://github.com/Perl/perl5/commit/1f5ed22b48bb80b6fbd32c1c7f65e4a4ef55c482
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Skip two instances of 'test_via_runtests'
Experimentation showed that while the test suite ran as expected when invoked
with 'make test', 'make test_harness', etc., we experienced indefinite hangs
during 't/porting/test_testlist.t' whenever we called the following:
$ make test_porting
# what's "inside" 'make test_porting':
$ TEST_ARGS='porting/*.t lib/diagnostics.t' TESTFILE=harness ./runtests
choose
$ TEST_ARGS='porting/test_testlist.t' TESTFILE=TEST ./runtests choose
$ TEST_ARGS='porting/*.t' TESTFILE=TEST ./runtests choose
Examination of the output of 'ps auxwww | grep testlist' showed that
additional processes were being spawned.
But the following did *not* hang:
$ TESTFILE=harness ./runtests choose
$ TESTFILE=TEST ./runtests choose
It is not clear why passing "TEST_ARGS=[something which expands to include
porting/test_testlist.t]" to "./runtests choose" behaves in this way. But as
that is not a way most users would trigger these tests, we can defer further
investigation for now via 'todo_skip'.
Commit: fc00efc0dcbb4ca9a6ec1b7911d12c5eec97d33b
https://github.com/Perl/perl5/commit/fc00efc0dcbb4ca9a6ec1b7911d12c5eec97d33b
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Avoid uninitialized value warning
Commit: b32d53bd071609a667d86fe376b1df0b44f3b1a9
https://github.com/Perl/perl5/commit/b32d53bd071609a667d86fe376b1df0b44f3b1a9
Author: James E Keenan <[email protected]>
Date: 2026-10-11 (Sun, 11 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
test_testlist.t: Skip 4 tests on Win32
We need to determine cause of test failures on that platform.
Compare: https://github.com/Perl/perl5/compare/8ea6a5dd5a76...b32d53bd0716
To unsubscribe from these emails, change your notification settings at
https://github.com/Perl/perl5/settings/notifications