Currently, StressNewTest will turn green WITHOUT RUNNING ANY TESTS if it thinks (often wrongly) that more than 25 test files were touched.
This is both dishonest (it can give a false green) AND totally unnecessary (concourse already applies a timeout to StressNewTest, currently set at 6 hours, so there is no reason not to at least try to run all changed tests, they may easily complete in under 6 hours even if many files). *** I PROPOSE that we should remove this 25-file-free-pass loophole. *** As always, if anyone is having trouble getting their PR to pass StressNew, please bring it up on the dev list and we can discuss the appropriate solution on a case-by-case basis (e.g. increasing timeout, fixing the logic that identifies changes test files, splitting into multiple PRs, etc).
