On Tue, Sep 13, 2011 at 8:23 PM, Stas Malyshev <smalys...@sugarcrm.com> wrote: > Hi! > > On 9/13/11 11:04 AM, Ferenc Kovacs wrote: >> >> as I mentioned before, make test (at least it should) passes the -n -c >> argument to run-tests.php >> see >> http://svn.php.net/viewvc/php/php-src/trunk/Makefile.global?view=markup#l104 >> which means if you run make test that can produce different result, >> than simply running run-tests.php > > That needs to be fixed, make test and run-tests.php should run exactly the > same code and produce identical results. > >> I'm pretty sure that I could screw up 99% of test results with a >> properly adjusted php.ini. > > This needs to be fixed too - tests should have values they depend on either > in command line or in INI section. >
agree. I think that it is relevant to mention, that we discussed on irc that maybe we should create a wrapper script for running the testsuite. so we could create a make-test.sh(and a make-test.bat), and we could move the logic from the Makefile there. so if you run make test, that would only execute the shell script, which would execute the run-tests.php with the proper arguments. this would also mean that if you run ./run-tests or make test, you would get the same result, and the advanced users could still use run-tests.php if they want, but we would have a good baseline. this would also solve the problem, that on some platforms the run-tests.php won't work with the default configuration (AFAIR Christian Weiske, Gustavo Lopes and Alexey Shein all run into the issue, that they php.ini from their distribution didn't had the 'E' in variables_order, so the $_ENV superglobal wasn't populated which caused the run-tests.php to ignore the environment variables :/). btw: most features which is available through arguments of run-tests.php also available through environmental variables for make test, but AFAIK there is no documentation, so I think that most people who uses run-tests.php directly isn't really need it, but run-tests.php has a --help option, while make test has not. -- Ferenc Kovács @Tyr43l - http://tyrael.hu -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
