codelipenghui commented on pull request #10148: URL: https://github.com/apache/pulsar/pull/10148#issuecomment-814091933
> the problem is that currently those tests are very flaky and we are ignoring them anyway. Having a test that is failing and if it fails we simply ignore it it is useless. In order to not lose them I suggest to create "blocker" issues for the next release and to fix them one at a time. If we have a panel that tells that a test failed with a disclaimer "do not worry about this test", then what is the meaning of that test ? it is simply a waste of resources to run it and more noise for the reviewers and for the contributors. The flaky test does not mean it is a useless test, it can fail with A problem and B problem, A and B introduced the `flaky`. But the new change might cause C problem. If we always ignore the flaky test, the new break change might be merged into the master branch. And if the test is very flaky, my point is we need to fix it. If it has not failed frequently before, but frequently now, this is most likely caused by some concurrent merge. A similar situation happened before, most of them are caused by the concurrent merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
