+1 Penghui
On Tue, Jul 12, 2022 at 1:47 PM Michael Marshall <mmarsh...@apache.org> wrote: > +1 I support removing the retries for the many reasons already mentioned. > > - Michael > > On Tue, Jul 12, 2022 at 12:35 AM Matteo Merli <mme...@apache.org> wrote: > > > > +1 > > > > The retry that repeats the whole maven job is often hiding the real > > test failures that were in the 1st failed job. > > -- > > Matteo Merli > > <mme...@apache.org> > > > > On Mon, Jul 11, 2022 at 9:06 PM Anon Hxy <anonhx...@gmail.com> wrote: > > > > > > Hi Boschi > > > > > > I support this plan and I think we need take some effort to make the > Pulsar > > > CI more stable. > > > > > > Thanks, > > > Xiaoyu Hou > > > > > > Nicolò Boschi <boschi1...@gmail.com> 于2022年7月11日周一 22:14写道： > > > > > > > Hi all, > > > > > > > > I'd like to start a discussion about the stability of Pulsar CI. > > > > > > > > It is common that some tests suite in our CI times out. This is > because > > > > when a test fails the entire suite is retried from the beginning > (max 3 > > > > times). (example: > > > > > https://github.com/apache/pulsar/runs/7281063499?check_suite_focus=true) > > > > > > > > The command-line retries may sound helpful in making the CI green > for a > > > > given pull but they actually hide test failures (that may be flakies > or > > > > real issues!!). > > > > > > > > Another issue is that you can't easily see the failed test and most > of the > > > > time the quickest solution is just to blindly restart the failed > jobs. This > > > > is not the correct behaviour and it will make the CI less stable > over time. > > > > > > > > The plan would be: > > > > - Remove the retries (see > https://github.com/apache/pulsar/pull/16524) > > > > - Create issue for flaky tests > > > > - Fix them / move to quarantine > > > > > > > > WDYT? > > > > > > > > Thanks, > > > > Nicolò Boschi > > > > >
