I agree with moving the tests out of the flaky test group. I just checking some new PRs
https://github.com/apache/pulsar/pull/17195 https://github.com/apache/pulsar/pull/17201 https://github.com/apache/pulsar/pull/17193 https://github.com/apache/pulsar/pull/17204 The `Pulsar CI / CI - Unit - Brokers - Flaky` looks more stable than other groups such as `Pulsar CI / CI - System - Pulsar Connectors - Thread` `Pulsar CI / CI - Unit - Brokers - Broker Group 2` `Pulsar CI / CI - Unit - Brokers - Broker Group 1` It looks like we can change the test group for now and make it required. To a new test group, or move them to `Pulsar CI / CI - Unit - Brokers - * ` Thanks, Penghui On Mon, Aug 22, 2022 at 4:33 PM tison <wander4...@gmail.com> wrote: > While agree to require more tests to pass, here are my two coins: > > 1. This group is named "flaky tests" so I regard it as flaky tests > literally. NOT require these tests to pass could be by design. Besides, > IIRC some developers keep investigating tests in the flaky test group, try > to make them stable, and move out of the flaky test group. This seems the > desired approach to resolve flaky tests. > > 2. Instead of barely "require broker-flaky" test group, do you have a list > of tests that fail frequently? Otherwise, we just go back to the situation > where we want to exclude it from the required status - it's quite unstable. > > Best, > tison. > > > mattison chao <mattisonc...@apache.org> 于2022年8月22日周一 15:56写道： > > > Hi all > > > > Recently, some tests in the `broker-flaky` test group always failed, but > > since it doesn't block CI, no one cared for a long time. > > > > This behaviour causes some test scenarios to go unchecked and risk > > introducing some regressions, and I think we need to make this test group > > set required. > > > > e.g. https://github.com/apache/pulsar/pull/17163 > > > > WDYT? > > > > Best, > > Mattison > > >
