There's a couple of things we can look at: We can take human element out of it, running the tests on a proposed patch can and probably should be done automatically. I heard there were plans for this, I hope they materialize sooner rather than later. I think this is one of Nathans projects
We are running the tests now on a nightly basis which is great, however nobody seems to be looking at the results otherwise we would have figured out the tests were failing 3 weeks ago and either disabled the test or fixed the problem. I kinda feel that looking at the tests on a daily basis should be on someones daily morning checklist, perhaps Nathan and/or Dalai can find someone for this task? as for commit messages, we have good guidance there [1], having people try a little harder will probably go a long way there. --Ray [1] https://wiki.blender.org/wiki/Style_Guide/Commit_Messages On 2020-01-02 7:04 p.m., Sebastián Barschkis wrote: > Hi Ray, > > you’re right, the Cycles tests need to be updated. They need to make use of > the new fluid modifier. > Sergey pointed this out to me right after (within 12h) I landed the commits, > i.e. the issue is known. > > So yes, blame it on me, I overlooked this part. I will look into this as soon > as possible. > > I agree that this is not ideal. But apart from better commit messages, how > can we do better at this time? > > Happy new year and best wishes, > Sebastián > >> On 2. Jan 2020, at 22:31, Ray Molenkamp <[email protected]> wrote: >> >> All, >> >> Hate to be a heckler for running the unit tests, but please: >> >> When you land and/or review something big, RUN THE UNIT TESTS! >> >> When the monster 10 patch mantaflow patch landed, it broke >> 6 cycles unit tests on all platforms, 5 with different renders >> than the reference images and one full on blender crash [1] >> >> bda4a284d20164fec2433f7c40f49fc903319400 [2] fixed the render >> differences and turned the crash into a render difference [3] >> >> Unrelated side note: whats with the less than helpful >> commit message on this commit? it may as well have been >> committed with the message 'something fluid' or 'Tuesday' >> would have just as enlightening for other developers. >> >> To summarize: >> >> - The person submitting the patches has not run the tests >> - The reviewers have not run the tests >> - Less than optimal commit messages >> - 18!! days after landing, there is still a failing test >> >> Holiday season or not: I think we can and should do better than this >> >> --Ray >> >> [1] https://i.imgur.com/LE3baOg.png >> [2] https://developer.blender.org/rBbda4a284d20164fec2433f7c40f49fc903319400 >> [3] https://i.imgur.com/5we0hEv.png >> >> >> _______________________________________________ >> Bf-committers mailing list >> [email protected] >> https://lists.blender.org/mailman/listinfo/bf-committers > _______________________________________________ > Bf-committers mailing list > [email protected] > https://lists.blender.org/mailman/listinfo/bf-committers _______________________________________________ Bf-committers mailing list [email protected] https://lists.blender.org/mailman/listinfo/bf-committers
