Revolyssup opened a new pull request, #12462: URL: https://github.com/apache/apisix/pull/12462
The test here https://github.com/apache/apisix/actions/runs/16511586539/job/46694453675 failed due to invalid configuration but in case of Bailout, the tests is considered successful. This change asserts and finds for the bailed out test and fails CI if any test was bailed out. ## Further enquiries The doc says "By default the test scaffold treats NGINX server startup failures as fatal errors in running the tests. The must_die section, however, turns such a failure into a normal test checkup." If you have just one test then it holds true. But when even one test passes and subsequent test has invalid config then TEST IS CONSIDERED PASS. WHen invalid configuration was passed in Test 1 (As expected the Bailout is considered total test failure) Test run interrupted! Test Summary Report ------------------- t/core/ctx.t (Wstat: 65280 (exited 255) Tests: 0 Failed: 0) Non-zero exit status: 255 Files=1, Tests=0, 31 wallclock secs ( 0.01 usr 0.00 sys + 0.19 cusr 0.17 csys = 0.37 CPU) Result: FAIL When invalid configuration was passed in Test 2(or any other test after one passed test): (The test pass even after bailout) Bailout called. Further testing stopped: t/core/ctx.t TEST 2: http header - Cannot start nginx using command "/usr/bin/openresty -p /home/ashish/dev/apisix/t/servroot/ -c /home/ashish/dev/apisix/t/servroot/conf/nginx.conf > /dev/null" (status code 256). t/core/ctx.t .. ok Test run interrupted! All tests successful. Files=1, Tests=6, 31 wallclock secs ( 0.02 usr 0.01 sys + 0.27 cusr 0.22 csys = 0.52 CPU) Result: PASS All tests successful. Files=1, Tests=6, 31 wallclock secs ( 0.02 usr 0.01 sys + 0.27 cusr 0.22 csys = 0.52 CPU) Result: PASS FAILED--Further testing stopped: t/core/ctx.t TEST 2: http header - Cannot start nginx using command "/usr/bin/openresty -p /home/ashish/dev/apisix/t/servroot/ -c /home/ashish/dev/apisix/t/servroot/conf/nginx.conf > /dev/null" (status code 256). ### Checklist - [ ] I have explained the need for this PR and the problem it solves - [ ] I have explained the changes or the new features added to this PR - [ ] I have added tests corresponding to this change - [ ] I have updated the documentation to reflect this change - [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first) <!-- Note 1. Mark the PR as draft until it's ready to be reviewed. 2. Always add/update tests for any changes unless you have a good reason. 3. Always update the documentation to reflect the changes made in the PR. 4. Make a new commit to resolve conversations instead of `push -f`. 5. To resolve merge conflicts, merge master instead of rebasing. 6. Use "request review" to notify the reviewer after making changes. 7. Only a reviewer can mark a conversation as resolved. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@apisix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
