Hi Stas,
> -----Original Message-----
> From: Stanislav Malyshev [mailto:smalys...@gmail.com]
> Sent: Wednesday, February 28, 2018 9:26 PM
> To: PHP Internals <internals@lists.php.net>
> Subject: [PHP-DEV] run-tests.php exit code
>
> Hi!
>
> When running tests with run-tests.php, if the tests fail, the script will
> exit with
> non-zero exit code, but only if REPORT_EXIT_STATUS is set. This was the case
> since 2002 when this capability has been introduced.
>
> I think it would be nice if we reversed the default and made the script use
> exit
> code by default, unless NO_EXIT_STATUS is set. All usages of this script that
> I
> know (including our own CI suite) use this setting, and I don't think there's
> a
> good case for not using it. I think it makes sense to default to the common
> use
> case. So:
>
> 1. Does it make sense to change the default?
I've stumbled upon this a couple of times, too, especially when writing new
scripts for non core test automation. IMO, given today there's much more
automation than ever, defaulting to the exit code translation makes sense. Test
fails should not be subtle by default at any levels.
> 2. Do we want RFC for it?
IMHO not really deserving an RFC.
> 3. Can we put it in 7.1 or do we want to wait for 7.3?
>
If someone doesn't use it on CI already, that's probably just a blessed
ignorance as the tests runs would always pass. Depends if we want the change to
be going more soft for some non core CI tests and avoid confusion on manual
test runs as in some cases an additional output could be produced. Given it's
in many cases harder to investigate failures on CI, perhaps 7.3 would be safe
to put it with the corresponding documentation, etc.
Regarding the option itself, we'd probably not need to introduce a new one.
Instead, a piece like below could be put at the top
If (getenv("REPORT_EXIT_STATUS") === false) putenv("REPORT_EXIT_STATUS=1");
Regards
Anatol
