I agree with Josef, I realize the SpringBoot listener example test was disabled while fixing an issue related to it. If not, the test would have remained disabled.
On Thu, Jan 25, 2024 at 5:51 AM Jozef Marko <jozef.ma...@ibm.com.invalid> wrote: > Thank you for this discussion. I think disabling flaky tests in given way > is reasonable. > > I have one related question, once similar flaky test is disabled, and a > ticket is reported. Who will be assigned to this ticket? Will it be the > test author? > > My point is, that usually only test author and few other engineers working > on the same component understand completely the given test, but probably > they do not check the codebase daily if the test is still enabled. > > So, will be these people notified? Otherwise, most of these tickets will > be probably closed after the proposed period without any test update/fix. > > Jozef Marko > Software Developer > jozef.ma...@ibm.com > > -----Original Message----- > From: Toni Rikkola <trikk...@redhat.com> > Sent: Wednesday, January 24, 2024 3:58 PM > To: dev@kie.apache.org > Subject: [EXTERNAL] Re: [PROPOSAL] Workflow for ignoring long failing tests > > The reason I mentioned this was that we had this same decision made 8 > years ago and shared it to the entire team. > It stated that anyone can disable any flaky test and make a ticket for it. > It was even more strict, you just had to have one flaky failure. > > So to me it looks like we are just stating the same thing again hoping for > a different outcome. > > But yes, to make it clear for everyone we should write it down and we can > see how it goes and act on it later if needed. > > Toni > > On Wed, Jan 24, 2024 at 2:09 PM Tibor Zimányi <tzima...@apache.org> wrote: > > > I think we don't need to enforce it or gatekeep this any way. The > > thing is, if we have the agreement that we want to have this official > > workflow, it should fix the problem you are mentioning Toni, that > > people are afraid to do so. If it will be officially stated, that > > failing tests can be disabled in the code, people should just do it. > > They can always point to this potential agreement. The best thing > > would be to even document it on Confluence. I will do that if we have an > agreement on this. > > > > Tibor > > > > Dňa st 24. 1. 2024, 12:40 Toni Rikkola <trikk...@redhat.com> napísal(a): > > > > > Not informing the author was my mistake on that PR. I should have > > > asked > > the > > > original author to review it. > > > > > > But the problem is not that we lack this procedure. The KIE team > > > agreed > > on > > > this in 2015. It is that maybe we lack the strength to do it, > > > especially when it is someone else's area of code. This needs some > > > sort of > > gatekeeper > > > or supervisor and time frames when the build quality and if these > > problems > > > were acted on are validated and measured. > > > > > > Meanwhile I offer myself to work on these and disable them, if > > > anyone > > wants > > > to point me to some flaky test I can act on them. I could even make > > > some system for measurement. We can then, or now, decide what we > > > then do with that data. > > > > > > Toni > > > > > > On Tue, Jan 23, 2024 at 8:04 PM Tibor Zimányi <tzima...@apache.org> > > wrote: > > > > > > > I think the notification of the involved commiter could be done as > > > > part > > > of > > > > creating of the issue to fix the tests. The person involved can be > > > assigned > > > > as an asignee. > > > > > > > > T. > > > > > > > > Dňa ut 23. 1. 2024, 18:50 Francisco Javier Tirado Sarti < > > > > ftira...@redhat.com> > > > > napísal(a): > > > > > > > > > Ok, but I think we first need to establish a way (my apologies > > > > > is > > that > > > > way > > > > > is already there) to notify committerthat a test that they were > > > involved > > > > > with is failing > > > > > That way, we will avoid disable tests like this one > > > > > INVALID URI REMOVED > > > > > apache_incubator-2Dkie-2Dkogito-2Dexamples_issues_1831&d=DwIFaQ& > > > > > c=jf_iaSHvJObTbx-siA1ZOg&r=Tvtr6trQuIEn20iSHtFwcpynBt8ViCybz4kYp > > > > > jmDDUQ&m=C6gAvciwgXz7uJX__01AVZ3YHxSotIhuktwbnBnNuTvnLRU-NHMRqOd > > > > > NENKhsqKe&s=8p3WYlpcXfcLl1n4BnrSpBWNIdPIUhslTPsb41eE4IM&e= , > > > > which > > > > > are quite important to ensure Springboot messaging is working. > > > > > > > > > > On Tue, Jan 23, 2024 at 6:38 PM ricardo zanini fernandes < > > > > > ricardozan...@gmail.com> wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > On Tue, Jan 23, 2024 at 1:03 PM Martin Cimbálek > > > > > > <cim...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > Regards, > > > > > > > Martin Cimbalek > > > > > > > > > > > > > > On Tue, Jan 23, 2024 at 3:36 PM Tibor Zimányi < > > tzima...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > I want to propose a workflow for situations, when some > > > > > > > > tests > > are > > > > > > failing > > > > > > > > for a longer time. In such cases, my proposed workflow is: > > > > > > > > - If a test or a set of tests is failing for two days > > (nightlies > > > or > > > > > PR > > > > > > > > checks), ignore those tests in the codebase, so they are > > > > > > > > not > > > > > executed. > > > > > > > > - File an issue in kie-issues repository, reporting the > > > > > > > > test > > > > > failures. > > > > > > > > - If the issue is not resolved for half a year, delete > > > > > > > > those > > > tests > > > > > from > > > > > > > the > > > > > > > > codebase. > > > > > > > > > > > > > > > > What do you think please? This should make sure all > > > > > > > > failures > > that > > > > > don't > > > > > > > get > > > > > > > > fixed immediately after they occur get logged in the issue > > > tracker, > > > > > so > > > > > > > they > > > > > > > > can be appropriately handled and don't block unrelated PR > > checks > > > > and > > > > > > > > builds. It will also make sure that tests that are not > > > > > > > > fixed > > for > > > a > > > > > long > > > > > > > > time (therefore they could be perceived as not important) > > > > > > > > are > > > not a > > > > > > > > maintenance burden for the future. > > > > > > > > > > > > > > > > Best regards, > > > > > > > > Tibor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > For additional commands, e-mail: dev-h...@kie.apache.org > >
