GitHub shows red warnings for PRs that didn't pass all the tests. You should never merge PRs which are red, or not current anymore (this could also be a red status indicator). If the failing tests can't be resolved quickly it might be worth splitting the tests between stable / unstable and have different status indicators for these tests. Then people can see easily if the PR is breaking the stable tests.
Jörn On Wed, Sep 27, 2017 at 8:12 PM, Gautam <gautamn...@gmail.com> wrote: > Hi Chris, > > There could be possibility that someone might have merged the changes > without having all checks running, or overlooked the result. Currently we > don't have any mechanism where git farm can reject such merge where CI/Unit > test fails. I will try to explore any such possibilities. Meanwhile I would > rather disable those fail tests with a git hub issue so that build can be > clean for now. We can revisit those issue later. > > > > > On Wed, Sep 27, 2017 at 11:02 AM, kellen sunderland < > kellen.sunderl...@gmail.com> wrote: > >> I share in your frustration Chris. I've also spent a fair amount of time in >> the past few days digging through console logs to try and see if there was >> anything actionable. I haven't noticed any tests that were failing >> consistently, maybe you can post an issue with some specific tests? For me >> the larger issue is Sanity Check failures and segfaults at the end of test >> runs (after all tests pass). I'm assuming that people are working on the >> Sanity Check issues. If there's anything external contributors can do >> please let us know. >> >> On Wed, Sep 27, 2017 at 5:48 PM, Chris Olivier <cjolivie...@gmail.com> >> wrote: >> >> > By the way, I am not referring to a few tests that are known to fail >> 1%-10% >> > or so of the time (ie test_batchnorm_training) and are being actively >> > worked on. I am referring to tests that fail 100% of the time and are >> still >> > merged into master, and thus propagate to all branches when sync'd from >> > master. >> > >> > On Wed, Sep 27, 2017 at 8:43 AM, Chris Olivier <cjolivie...@gmail.com> >> > wrote: >> > >> > > How are so many broken unit tests getting into master? Is stuff being >> > > merged without passing CI/unit testing? I have been trying to get >> three >> > > PR's to build for over a week now. Each time it's some broken test or >> > > another that has nothing to do with my code changes. It's extremely >> > > frustrating -- I waste whole days on this, trying to figure out why my >> > code >> > > is breaking strange things only to realize later it's broken in all >> > > branches. >> > > >> > >> > > > > -- > Best Regards, > Gautam Kumar
