On Thu, Jul 27, 2000 at 04:21:07AM +0100, Hugo wrote:
> First up, are all perl-qa messages going to bootstrap as well? If so,
> I don't need to be on both lists.
No. I'm posting the initial few RFCs on bootstrap to get people's
attention. All discussion should go to perl-qa only. Sorry if I
didn't make this explicit.
> In <[EMAIL PROTECTED]>, Michael G Schwern writes:
> :In One Sentence
> :---------------
> :
> :All patches to perl must have an associated testing patch.
>
> Can you explain more about how you'll test documentation?
Damn, I ALWAYS get caught in hyperbole!
I'd like to test the things the documentation states about Perl as
well as the code examples in the docs, rather than spelling and
formatting issues (that can be for somebody else to tackle, and
another RFC).
Formatting and typo doc patches probably wouldn't need a testing
patch.
> Many patches also come into existence because some regression test
> is already failing. This applies in particular to configuration
> issues, but also elsewhere. Do these need new tests?
Probably not.
But once things settle down and we have a completely passing
regression suite (stop snickering) nearly all regression failures will
occur because of a proposed code patch. In this case, the patch is
rejected and sent back to the author for revision.
Situations where a regression test fails would arise from changes
outside of Perl. Some OS made some library change or something. In
that case you'd have a patch to fix an existing test and no a new test
would not be necessary.
We could have some way to let patch authors say that a test already
exists and which one it is.
> It is currently an (apparent) no-no to add tests to perl that fail.
> While I can understand the desire to avoid distressing end users
> with fully anticipated test failures, I think we need a better
> solution to this - when a problem is identified, the _first_ thing
> that should appear is the new test that identifies the problem by
> failing. Just because we're not sure how to fix the problem yet,
> or haven't had the tuits yet, is no reason not to add the failing
> test.
On stable (maintenance) Perl releases the answer is clear. EVERYTHING
PASSES. Something fails, we don't release.
For devel perls, its murky what the right thing to do is. You're
definately right about needing to add the tests, failing or passing.
It will obviously be necessary to release devel versions of Perl with
tests that fail. At a minimum we can have a formal way to specify
that we expect a test to fail. "On these OS's, with these libraries and
these configs this test is expected to fail. See bug #23234" or
something like that. Only written explicitly in a way Perl can
understand.
This issue is big enough to warrent a seperate discussion/RFC.
--
Michael G Schwern http://www.pobox.com/~schwern/ [EMAIL PROTECTED]
Just Another Stupid Consultant Perl6 Kwalitee Ashuranse
But why? It's such a well designed cesspool of C++ code. Why wouldn't
you want to hack mozilla?
-- Ziggy
