2018-03-05 14:14 GMT+01:00 Stephane Ducasse <[email protected]>: > Hi pavel > > when I'm back can you explain to me because I did not get it :).
:) it is simple. If you want to ignore issues that the slice is not adding, you need to know which of them to ignore. That's why the original monkey run all the validations twice - first time to collect a list of failing tests in the fresh unchanged image and then with the slice or configuration loaded. So it doubled the issue validation time. There are several alternative strategies like to cache the failing test results for every build and use them for validations but the best strategy is simply to keep the amount of failing tests in the clean image on the zero level and force people to keep the system clean. The original monkey hasn't exposed the list of ignored tests. So it was possible that some test was ignored because of a temporal network issue but for the second time it failed for o good reason and you even didn't know. Cheers, -- Pavel > > Stef > > > On Mon, Mar 5, 2018 at 11:07 AM, Pavel Krivanek > <[email protected]> wrote: >> 2018-03-05 10:54 GMT+01:00 Marcus Denker <[email protected]>: >>> >>> >>>> On 5 Mar 2018, at 10:27, Alistair Grant <[email protected]> wrote: >>>> >>>> Hi Marcus, >>>> >>>> On 5 March 2018 at 09:23, Marcus Denker <[email protected]> wrote: >>>>> >>>>> >>>>>> On 5 Mar 2018, at 09:16, Alistair Grant <[email protected]> wrote: >>>>>> >>>>>> Hi Esteban & Marcus, >>>>>> >>>>>> I'm getting repeated validation failures for: >>>>>> >>>>>> https://pharo.manuscript.com/f/cases/21431 >>>>>> >>>>>> >>>>>> It's the same set of tests that fail each time, and as far as I can >>>>>> tell they have nothing to do with the patch I submitted. >>>>>> >>>>>> Do you know if this is happening on other tests? >>>>> >>>>> I saw that Saturday but decided to wait till Monday (weekends are >>>>> important..). >>>>> >>>>> So: no, I have *no* idea what happened. From one CI run to the next, >>>>> suddenly around 160 tests related to Calypso started failing due to a >>>>> missing method. >>>>> >>>>> Now starting from sometime today, this problem stoped. The last failing >>>>> PR checks >>>>> fail due to different reasons… >>>>> >>>>> And I have no idea why. >>>>> >>>>> (And yes, we al know that >>>>> 1) the PR checks need more compute power, too slow >>>>> 2) we *need* to track down the reason why still *a lot* of times the PR >>>>> fails >>>>> even though it should not. >>>>> >>>>> The problem is that just keeping a build alive of this kind is a full >>>>> time job.. that >>>>> we have nobody doing, so many many people do as much as they can and we >>>>> hope it will get better….) >>>> >>>> Thanks for the update. >>>> >>> >>> Oh, and it was completely unrelated. Your change is for Pharo6... >>> >>>> I took a look at the failures and it appears that >>>> >>>> BehaviorTest>>testBehaviorRespectsPolymorphismWithTraitBehavior >>>> ClassDescriptionTest>>testClassDescriptionRespectsPolymorphismWithTraitDescription >>>> ClassTest>>testClassRespectsPolymorphismWithTrait >>>> >>>> are all failing due to changes in Fuel - methods were changed from >>>> traits to local methods. >>>> >>> Yes, the problem is that the monkey (the contribution checker) fails as soon >>> as there are errors even in the main image. >>> >>> The last Pharo6 has these tests failing, so now all contribution checks for >>> Pharo6 fail. >>> >>> What needs to be done? >>> >>> -> your change can be accepted as we know it does not fail more fixes >>> -> then we need fix the tests in Pharo6 >>> -> in a perfect world we would update the slice checker to only fail for >>> now test failing… (it used to be lille that…). >> >> I must say that It made the validation two times slower, fragile and >> led to the hiding of problems instead of solving them... >> >> -- Pavel >> >>> >>> As I said: this is a full time job… >>> >>> Marcus >> >
