Hi, as I wrote in one of the previous e-mails in this thread: "I think the notification of the involved commiter could be done as part of creating of the issue to fix the tests. The person involved can be assigned as an asignee. "
To clarify, by involved commiter I meant either author of the test or the person who is maintaining that part of the codebase. Hopefully this clarifies this. Tibor Dňa št 25. 1. 2024, 9:22 Francisco Javier Tirado Sarti <[email protected]> napísal(a): > I agree with Josef, I realize the SpringBoot listener example test was > disabled while fixing an issue related to it. If not, the test would have > remained disabled. > > On Thu, Jan 25, 2024 at 5:51 AM Jozef Marko <[email protected]> > wrote: > > > Thank you for this discussion. I think disabling flaky tests in given way > > is reasonable. > > > > I have one related question, once similar flaky test is disabled, and a > > ticket is reported. Who will be assigned to this ticket? Will it be the > > test author? > > > > My point is, that usually only test author and few other engineers > working > > on the same component understand completely the given test, but probably > > they do not check the codebase daily if the test is still enabled. > > > > So, will be these people notified? Otherwise, most of these tickets will > > be probably closed after the proposed period without any test update/fix. > > > > Jozef Marko > > Software Developer > > [email protected] > > > > -----Original Message----- > > From: Toni Rikkola <[email protected]> > > Sent: Wednesday, January 24, 2024 3:58 PM > > To: [email protected] > > Subject: [EXTERNAL] Re: [PROPOSAL] Workflow for ignoring long failing > tests > > > > The reason I mentioned this was that we had this same decision made 8 > > years ago and shared it to the entire team. > > It stated that anyone can disable any flaky test and make a ticket for > it. > > It was even more strict, you just had to have one flaky failure. > > > > So to me it looks like we are just stating the same thing again hoping > for > > a different outcome. > > > > But yes, to make it clear for everyone we should write it down and we can > > see how it goes and act on it later if needed. > > > > Toni > > > > On Wed, Jan 24, 2024 at 2:09 PM Tibor Zimányi <[email protected]> > wrote: > > > > > I think we don't need to enforce it or gatekeep this any way. The > > > thing is, if we have the agreement that we want to have this official > > > workflow, it should fix the problem you are mentioning Toni, that > > > people are afraid to do so. If it will be officially stated, that > > > failing tests can be disabled in the code, people should just do it. > > > They can always point to this potential agreement. The best thing > > > would be to even document it on Confluence. I will do that if we have > an > > agreement on this. > > > > > > Tibor > > > > > > Dňa st 24. 1. 2024, 12:40 Toni Rikkola <[email protected]> > napísal(a): > > > > > > > Not informing the author was my mistake on that PR. I should have > > > > asked > > > the > > > > original author to review it. > > > > > > > > But the problem is not that we lack this procedure. The KIE team > > > > agreed > > > on > > > > this in 2015. It is that maybe we lack the strength to do it, > > > > especially when it is someone else's area of code. This needs some > > > > sort of > > > gatekeeper > > > > or supervisor and time frames when the build quality and if these > > > problems > > > > were acted on are validated and measured. > > > > > > > > Meanwhile I offer myself to work on these and disable them, if > > > > anyone > > > wants > > > > to point me to some flaky test I can act on them. I could even make > > > > some system for measurement. We can then, or now, decide what we > > > > then do with that data. > > > > > > > > Toni > > > > > > > > On Tue, Jan 23, 2024 at 8:04 PM Tibor Zimányi <[email protected]> > > > wrote: > > > > > > > > > I think the notification of the involved commiter could be done as > > > > > part > > > > of > > > > > creating of the issue to fix the tests. The person involved can be > > > > assigned > > > > > as an asignee. > > > > > > > > > > T. > > > > > > > > > > Dňa ut 23. 1. 2024, 18:50 Francisco Javier Tirado Sarti < > > > > > [email protected]> > > > > > napísal(a): > > > > > > > > > > > Ok, but I think we first need to establish a way (my apologies > > > > > > is > > > that > > > > > way > > > > > > is already there) to notify committerthat a test that they were > > > > involved > > > > > > with is failing > > > > > > That way, we will avoid disable tests like this one > > > > > > INVALID URI REMOVED > > > > > > apache_incubator-2Dkie-2Dkogito-2Dexamples_issues_1831&d=DwIFaQ& > > > > > > c=jf_iaSHvJObTbx-siA1ZOg&r=Tvtr6trQuIEn20iSHtFwcpynBt8ViCybz4kYp > > > > > > jmDDUQ&m=C6gAvciwgXz7uJX__01AVZ3YHxSotIhuktwbnBnNuTvnLRU-NHMRqOd > > > > > > NENKhsqKe&s=8p3WYlpcXfcLl1n4BnrSpBWNIdPIUhslTPsb41eE4IM&e= , > > > > > which > > > > > > are quite important to ensure Springboot messaging is working. > > > > > > > > > > > > On Tue, Jan 23, 2024 at 6:38 PM ricardo zanini fernandes < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > On Tue, Jan 23, 2024 at 1:03 PM Martin Cimbálek > > > > > > > <[email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > Regards, > > > > > > > > Martin Cimbalek > > > > > > > > > > > > > > > > On Tue, Jan 23, 2024 at 3:36 PM Tibor Zimányi < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > I want to propose a workflow for situations, when some > > > > > > > > > tests > > > are > > > > > > > failing > > > > > > > > > for a longer time. In such cases, my proposed workflow is: > > > > > > > > > - If a test or a set of tests is failing for two days > > > (nightlies > > > > or > > > > > > PR > > > > > > > > > checks), ignore those tests in the codebase, so they are > > > > > > > > > not > > > > > > executed. > > > > > > > > > - File an issue in kie-issues repository, reporting the > > > > > > > > > test > > > > > > failures. > > > > > > > > > - If the issue is not resolved for half a year, delete > > > > > > > > > those > > > > tests > > > > > > from > > > > > > > > the > > > > > > > > > codebase. > > > > > > > > > > > > > > > > > > What do you think please? This should make sure all > > > > > > > > > failures > > > that > > > > > > don't > > > > > > > > get > > > > > > > > > fixed immediately after they occur get logged in the issue > > > > tracker, > > > > > > so > > > > > > > > they > > > > > > > > > can be appropriately handled and don't block unrelated PR > > > checks > > > > > and > > > > > > > > > builds. It will also make sure that tests that are not > > > > > > > > > fixed > > > for > > > > a > > > > > > long > > > > > > > > > time (therefore they could be perceived as not important) > > > > > > > > > are > > > > not a > > > > > > > > > maintenance burden for the future. > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > Tibor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
