Jani Taskinen schrieb:
Having tests in multiple branches is PITA. Hasn't anyone considered that
the best way would be to move all tests into their own repository
(directory..whatever :) in SVN..? Considering they are supposed to be
used for testing against regressions and BC breaks, they should always
be runnable using any PHP version?
It is doable, likely desired but it is going to be a bit of work.
What you save is the work of having to update multiple branches manually.
What you risk is that not each and every test is prepared for being run
with every version - although, maybe, in theory it should be. This is
certainly something that can be solved by adding some more magic to
run-tests.php, e.g. "==SKIP-VERSION==" as suggested by Hannes -
practicalities.
Also, you may end up shipping the same huge set of tests for every PHP
version regardless if all tests you ship are compatible with that
version. Again, some magic should solve that - practicalities.
There is one thing I fear, although it is desired to do. Many extensions
link external libraries. If you throw all tests in one place and run all
tests with all PHP versions, I strongly assume you'll get more reports
on test failures. That is because the likeliness of someone out there
running new tests designed for the latest version of an external library
against an old library will increase.
Yes, you want to know about any such incompatibility. Yes, it helps you
making your software better.
But it may mean significantly more work in particular if you follow the
common paragidma of "test ample must always be green at any time on
every system".
What may cure the extra work problem is to be more strict on compatible
versions. I am aware how much people love BC. But there's a point where
keeping BC should be left as an exercise to those asking for BC.
All in all, I like the idea but I fear extra work.
Ulf
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
