Jani Taskinen schrieb:
What you risk is that not each and every test is prepared for being run
with every version - although, maybe, in theory it should be. This is
It should not be theory for regression tests? If new release does not
pass the old tests but the old versions still do, then it's quite likely
the new version is buggy? Now we have different versions of same tests
in each branch (in the worst cases) and thus the behaviour might really
be different between them when it should not be.
I am with you on the idea of a central repository and the arguments
behind your two questions.
Though, you know, I'm lazy and I fear you really find issues that need
to be fixed :-) .
For a transition period there's likely to be more work and the number of
test failures is likely to go up. That is nothing to really worry about
as long as you manage to educate users that it is not a quality defect
of PHP as such but a temporary matter of an different and improved
testing approach.
There is one thing I fear, although it is desired to do. Many extensions
link external libraries. If you throw all tests in one place and run all
tests with all PHP versions, I strongly assume you'll get more reports
on test failures. That is because the likeliness of someone out there
running new tests designed for the latest version of an external library
against an old library will increase.
This part I didn't quite understand..isn't this same issue with the
current situation as well?
Its only a diffuse feeling of mine, I could well be wrong.
In any case I am not trying to argue against your proposal. Just the
opposite.
Let's assume the worst case status quo of different versions of the same
test in each branch. The one in an old branch has been written with the
versions of the external library in mind that has been current when the
branch was current. The one in a current branch has been written against
the latest version of the external library.
I continue to assume that users of the old PHP branch run rather old
systems whereas users of a current PHP branch run on current systems.
Therefore only few people may have tried to run the old test against the
latest library or the other way around. It is just a feeling that your
proposal will implicitly cause some combinations of PHP version x test x
external library version to be tested that have not been checked before.
Your "one test for all" proposal is likely to unveil some "different
versions of the same test" and "external library is not BC" issues.
That's good. But its gonna be work...
Ulf
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
