+1 I support removing the retries for the many reasons already mentioned. - Michael
On Tue, Jul 12, 2022 at 12:35 AM Matteo Merli <mme...@apache.org> wrote: > > +1 > > The retry that repeats the whole maven job is often hiding the real > test failures that were in the 1st failed job. > -- > Matteo Merli > <mme...@apache.org> > > On Mon, Jul 11, 2022 at 9:06 PM Anon Hxy <anonhx...@gmail.com> wrote: > > > > Hi Boschi > > > > I support this plan and I think we need take some effort to make the Pulsar > > CI more stable. > > > > Thanks, > > Xiaoyu Hou > > > > Nicolò Boschi <boschi1...@gmail.com> 于2022年7月11日周一 22:14写道： > > > > > Hi all, > > > > > > I'd like to start a discussion about the stability of Pulsar CI. > > > > > > It is common that some tests suite in our CI times out. This is because > > > when a test fails the entire suite is retried from the beginning (max 3 > > > times). (example: > > > https://github.com/apache/pulsar/runs/7281063499?check_suite_focus=true) > > > > > > The command-line retries may sound helpful in making the CI green for a > > > given pull but they actually hide test failures (that may be flakies or > > > real issues!!). > > > > > > Another issue is that you can't easily see the failed test and most of the > > > time the quickest solution is just to blindly restart the failed jobs. > > > This > > > is not the correct behaviour and it will make the CI less stable over > > > time. > > > > > > The plan would be: > > > - Remove the retries (see https://github.com/apache/pulsar/pull/16524) > > > - Create issue for flaky tests > > > - Fix them / move to quarantine > > > > > > WDYT? > > > > > > Thanks, > > > Nicolò Boschi > > >
