Edit report at https://bugs.php.net/bug.php?id=36398&edit=1
ID: 36398
Updated by: c...@php.net
Reported by: nickj-phpbugs at nickj dot org
Summary: "make test" should be fully scriptable [Do not
prompt for email address]
-Status: Open
+Status: Closed
Type: Feature/Change Request
Package: Testing related
Operating System: Linux
PHP Version: 5.1.2
-Assigned To:
+Assigned To: cmb
Block user comment: N
Private report: N
New Comment:
Thanks for the patch, and sorry that it has been missed, but â¦
> Though, I don't know if we really want php-qa list to get filled
> with results of the tests executed automatically every X minutes.
No, we really don't want that. Instead it would be more helpful,
if individual test fails would be investigated by the testers, and
reported as separate tickets, ideally accompanied by a respective
pull request.
Previous Comments:
------------------------------------------------------------------------
[2008-08-07 06:58:48] nickj-phpbugs at nickj dot org
Yes, it would be nice if after calling for a patch, if the submitted patch
could be either accepted or rejected, rather than ignored for 2.5 years ;-)
I gather run-tests.php may be rewritten sometime (I think I saw it somewhere as
a possible Google SoC project), and if this happens including functionality to
send the results using an email address specified on the command line could be
nice to include. Presumably then someone (or some bit of software) can collate
the output and help to pinpoint consistent problems, or when things break on
particular platforms, and give a summary of the health of PHP on different
platforms, and so forth.
------------------------------------------------------------------------
[2008-08-05 12:21:28] Bjorn dot Wiberg at its dot uu dot se
Shouldn't the patch be incorporated into run-tests.php? As it has been around
for a long time.
------------------------------------------------------------------------
[2006-02-17 06:53:53] nickj-phpbugs at nickj dot org
No worries! There is a patch now available for this at:
http://www.files.nickj.org/php/run-tests-patch.txt
Also, the run-test.php script is not currently E_STRICT clean. I have included
in the patch some simple changes I had to make to get it to run with E_STRICT
enabled.
Note that there are 3 remaining things I did not know how to fix for clean
E_STRICT output, so even with this patch run-tests.php is still not totally
E_STRICT clean. The three remaining things were:
1) Was this error:
Error! type: 8; File: /root/php-5.1-dev/php5.1-200602150330/run-tests.php;
Line: 93; Message: ob_end_clean(): failed to delete buffer. No buffer to delete.
... which was generated by this line under E_STRICT:
while(@ob_end_clean());
2) Was the use of date() in this line:
$output_file = $CUR_DIR . '/php_test_results_' . date('Ymd_Hi') . '.txt';
Which gives this error under E_STRICT:
Error! type: 2048; File: /root/php-5.1-dev/php5.1-200602150330/run-tests.php;
Line: 246; Message: date(): It is not safe to rely on the system's timezone
settings. Please use the date.timezone setting, the TZ environment variable or
the date_default_timezone_set() function. In case you used any of those methods
and you are still getting this warning, you most likely misspelled the timezone
identifier. We selected 'Australia/Melbourne' for 'EST/11.0/DST' instead
Not sure how to say "just use the system default" nowadays, given that we
probably shouldn't hard-code a timezone, as users from anywhere could be
running this test script.
3) Was the "select_stream" call in the "system_with_timeout" function, which
generates this error under E_STRICT:
Error! type: 2; File: /root/php-5.1-dev/php5.1-200602150330/run-tests.php;
Line: 893; Message: stream_select(): 230 is not a valid stream resource
------------------------------------------------------------------------
[2006-02-16 07:35:06] tony2...@php.net
Thanks, I've just fixed the typo.
As for the request - I'm pretty sure you can add it to run-tests.php and send a
patch for us to review.
Though, I don't know if we really want php-qa list to get filled with results
of the tests executed automatically every X minutes.
------------------------------------------------------------------------
[2006-02-16 06:19:59] nickj-phpbugs at nickj dot org
Thank you - that sort of helps, and I realise now that I should have phrased my
request better (so I have reopened, and rephrased it to hopefully be clearer).
What I mostly wanted was to run the tests, and then email the results off,
without prompting.
I can enable/disable the prompt now with:
TEST_PHP_ARGS="-q" make test
And I can see the available options with:
TEST_PHP_EXECUTABLE="/root/tmp/php-5.1-dev/php5.1-200602150330/sapi/cli/php"
sapi/cli/php run-tests.php --help
However, unless I am mistaken, I can't see any option to specify the email
address to use, such as for example: TEST_PHP_ARGS="--email-results
--use-email=m...@email.com" make test
My underlying assumption here is that you folks want and use the output of
"make test" in some way. If that's not the case, then of course, don't make it
an option. However, if you are using this, then wouldn't it be good to be able
to script it so that it could automatically email off the results of make test,
without having to prompt the user? Then you could (for example) have the build
farm automatically email the qa.php.net list with the results of "make test"
after every build of every snapshot. Maybe you already do this in some other
way, but if not, it seems like it could be useful to me.
You could also update http://qa.php.net/running-tests.php with this
information, so that people could "set and forget" to run the tests and email
the results, without prompting them.
P.s. there is a very small typo in the help for "make test". It says "-q
Quite, no user interaction"; but you want it to say "Quiet", not "Quite".
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
https://bugs.php.net/bug.php?id=36398
--
Edit this bug report at https://bugs.php.net/bug.php?id=36398&edit=1
