Dmitriy, thanks for pushing this! We have to bring the number of the failing tests to zero. I hope that the community gets behind this effort.
D. On Mon, Feb 12, 2018 at 8:26 AM, Dmitry Pavlov <dpavlov....@gmail.com> wrote: > Hi Folks, > > I want to resurrect this process and start to fix tests. Currently approx. > 60-80 tests are failed each run on master. > > As one from the first point I suggest to use TC investigations for brief > research of test failures. > > If you can help with research please create and assign investigation on TC > (on test fail select dropdown menu, then 'Investigate / Mute' > then 'Investigated by: me' & 'Resolve: auto/manually'; Runs > https://ci.ignite.apache.org/viewType.html?buildTypeId= > IgniteTests24Java8_RunAll&branch_IgniteTests24Java8=%3Cdefault%3E > ). Also help with existing fixes review/merge would be very appreciated. > > Issue creation is still desired, but investigation will help us to identify > tests failures which are under research and failures not researched for > now. > > Sincerely, > Dmitriy Pavlov > > вт, 7 нояб. 2017 г. в 0:06, Dmitry Pavlov <dpavlov....@gmail.com>: > > > "Ignored" means that the test is disabled from the code, using annotation > > @Ignore. Not sure it works for our JUnit3 style Java tests. It is used > for > > .NET. > > "Muted" test is disabled in the interface TeamCity, but test code is > > actually executed (which is why hanging tests needs to be failed in > > addition to mute). > > > > The fact that the run all now became green means all the tests that fall > > at this point are flaky. If it is about 200 non-muted flaky tests having > > the probability of passing 98%, then the whole chain will pass only with > > the probability 0.98 ^ 200 = 1.7% > > > > I suggest continue to identify, fix, or mute such flaky tests to get > > stable green RunAll for correct code. > > > > пн, 6 нояб. 2017 г. в 22:57, Denis Magda <dma...@apache.org>: > > > >> Dmitriy, > >> > >> Now it’s time to fix a pretty decent number of muted and ignored tests. > >> BTW, what’s the difference between “ignored” and “muted”? I’ve seen the > >> former before. > >> > >> — > >> Denis > >> > >> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <dpavlov....@gmail.com> > >> wrote: > >> > > >> > Hi Igniters, > >> > > >> > I am happy to report that the activity of monitoring and fixing the > >> tests > >> > brought the first significant victory! > >> > > >> > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) > >> is > >> > completely green: > >> https://ci.ignite.apache.org/viewLog.html?buildId=930953 > >> > > >> > This is an reason for being pride, but not a reason to stop this > >> activity. > >> > Unfortunately, there are still a number of tests with unpredictable > >> > failures (flaky, 234 tests), and a number of tests have been muted. > Full > >> > project scope is available in the article > >> > > >> https://cwiki.apache.org/confluence/display/IGNITE/ > Make+Teamcity+Green+Again > >> > > >> > In any case, thank to all of you who helped and continues to help the > >> > project to correct the tests and bugs. > >> > > >> > Sincerely, > >> > Pavlov Dmitry > >> > > >> > > >> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <ptupit...@apache.org>: > >> > > >> >>> Green Again > >> >>> Again > >> >> As if it ever was green :) > >> >> > >> >> Of course +1 on this, let me know if you see any .NET-related > failures. > >> >> > >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков < > >> nizhikov....@gmail.com> > >> >> wrote: > >> >> > >> >>> Hello, Igniters. > >> >>> > >> >>> Also ready to help to #MakeTeamcityGreenAgain ! > >> >>> > >> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < > >> >>> daradu...@gmail.com> написал: > >> >>> > >> >>>> Hi guys. > >> >>>> > >> >>>> I vote for #MakeTeamcityGreenAgain. :-) > >> >>>> > >> >>>> FYI: it had been described and supported previously[1] > >> >>>> > >> >>>> After the completion of my current task I will try to help with > this > >> >>>> activity. > >> >>>> > >> >>>> [1] > >> >>>> http://apache-ignite-developers.2346864.n4.nabble. > >> >>>> com/Test-failures-td14353.html > >> >>>> > >> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <a...@apache.org>: > >> >>>> > >> >>>>> Nikolay, > >> >>>>> > >> >>>>> That's also a big problem for me, as reviewer, to accept changes > and > >> >>>> merge > >> >>>>> them to master. > >> >>>>> > >> >>>>> Dmitriy, > >> >>>>> > >> >>>>> Currently we have some contributions *from community* blocked by > red > >> >>>>> Teamcity. > >> >>>>> > >> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < > >> >>>> nizhikov....@gmail.com> > >> >>>>> wrote: > >> >>>>> > >> >>>>>> +1 to Dmitry Pavlov from me. > >> >>>>>> > >> >>>>>> Green team city would be very helpful. > >> >>>>>> > >> >>>>>> > >> >>>>>>> If you are a newcomer, please share your feeling: are you > >> >> confused > >> >>> by > >> >>>>>> test > >> >>>>>>> failures on Teamcity? > >> >>>>>> > >> >>>>>> I'm working on my very first ignite issue > >> >>>>>> and it's not easy to understand TC build results. > >> >>>>>> > >> >>>>>> > >> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: > >> >>>>>> > >> >>>>>> I think the right question to ask is why do we have RED tests to > >> >>> begin > >> >>>>>>> with. In my view, TC should be green. If there are tests that do > >> >> not > >> >>>>> pass, > >> >>>>>>> we should disable them and file blocker tickets for the next > >> >>> releases. > >> >>>>>>> > >> >>>>>>> D. > >> >>>>>>> > >> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < > >> >>> dpavlov....@gmail.com > >> >>>>> > >> >>>>>>> wrote: > >> >>>>>>> > >> >>>>>>> Hi Igniters, > >> >>>>>>>> > >> >>>>>>>> We all know that there are a lot of open tasks in Ignite > project. > >> >>> To > >> >>>> be > >> >>>>>>>> able to handle all these tasks, we need to grow our community. > >> >> And > >> >>> to > >> >>>>>>>> achieve that, we should pay special attention to new community > >> >>>> members. > >> >>>>>>>> > >> >>>>>>>> As recent newcomer I’ve faced the following problem: There is > no > >> >>> way > >> >>>> to > >> >>>>>>>> find out by yourself whether your pull request changes are > >> >> correct > >> >>> or > >> >>>>>>>> not. > >> >>>>>>>> Contributor can not distinguish pull request test failures and > >> >>> master > >> >>>>>>>> test > >> >>>>>>>> failures. Experienced Ignite contributor can estimate the > damage, > >> >>> but > >> >>>>> how > >> >>>>>>>> it can be done by new community member? > >> >>>>>>>> > >> >>>>>>>> Once failing test is introduced to master it is multiplexed to > >> >> all > >> >>>>> later > >> >>>>>>>> PRs. As a result, we have tons of failures and suites timeouts, > >> >>> which > >> >>>>>>>> waste > >> >>>>>>>> Teamcity agents time. > >> >>>>>>>> > >> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m > >> >> sure > >> >>>> that > >> >>>>>>>> there are a lot of experienced developers in the community who > >> >>>>> understand > >> >>>>>>>> issues with tests very quickly. Please contact me if you are > >> >> ready > >> >>> to > >> >>>>>>>> help > >> >>>>>>>> the community to fix current test issues. > >> >>>>>>>> > >> >>>>>>>> I would appreciate any ideas and sharing your vision how we can > >> >>>> achieve > >> >>>>>>>> green teamcity. > >> >>>>>>>> > >> >>>>>>>> If you are a newcomer, please share your feeling: are you > >> >> confused > >> >>> by > >> >>>>>>>> test > >> >>>>>>>> failures on Teamcity? > >> >>>>>>>> > >> >>>>>>>> Sincerely, > >> >>>>>>>> > >> >>>>>>>> Dmitriy Pavlov > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> Best Regards, Vyacheslav D. > >> >>>> > >> >>> > >> >> > >> > >> >
