Flaky tests

2019-02-27 Thread Stanislav Kozlovski
Hey there Flink community, I work on a fellow open-source project - Apache Kafka - and there we have been fighting flaky tests a lot. We run Java 8 and Java 11 builds on every Pull Request and due to test flakiness, almost all of them turn out red with 1 or 2 tests (completely unrelated to the

Re: Flaky tests

2019-02-27 Thread Chesnay Schepler
people still check which tests are failing to ensure they are unrelated, and track them regardless. On 26.02.2019 17:28, Stanislav Kozlovski wrote: Hey there Flink community, I work on a fellow open-source project - Apache Kafka - and there we have been fighting flaky tests a lot. We run Jav

Re: Flaky tests

2019-02-27 Thread Aljoscha Krettek
this. > However to truly solve this one must look at the original cause to prevent > new flaky tests from being added. > From what I remember, many of our tests were flaky because they relied on > timings (e.g. lets Thread.sleep for X and assume Y has happened) or had > similar

Re: Flaky tests

2019-02-28 Thread Ufuk Celebi
flaky tests. Best, Ufuk On Wed, Feb 27, 2019 at 3:04 PM Aljoscha Krettek wrote: > > I agree with Chesnay, and I would like to add that the most important step > towards fixing flakiness is awareness and willingness. As soon as you accept > flakiness and start working around it (as y

Re: Flaky tests

2019-02-28 Thread Till Rohrmann
my recent PR > experience was still close to what Stanislav describes). > > @Robert: Do we have standard labels that we apply to tickets that > report a flaky test? I think this would be helpful to make sure that > we have a good overview of the state of flaky tests. > > Best,

Re: Flaky tests

2019-02-28 Thread Ufuk Celebi
t > > report a flaky test? I think this would be helpful to make sure that > > we have a good overview of the state of flaky tests. > > > > Best, > > > > Ufuk > > > > On Wed, Feb 27, 2019 at 3:04 PM Aljoscha Krettek > > wrote: > > > >

[jira] [Created] (FLINK-31920) Flaky tests

2023-04-24 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-31920: -- Summary: Flaky tests Key: FLINK-31920 URL: https://issues.apache.org/jira/browse/FLINK-31920 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-34407) Flaky tests causing workflow timeout

2024-02-07 Thread Aleksandr Pilipenko (Jira)
Aleksandr Pilipenko created FLINK-34407: --- Summary: Flaky tests causing workflow timeout Key: FLINK-34407 URL: https://issues.apache.org/jira/browse/FLINK-34407 Project: Flink Issue

[jira] [Created] (FLINK-29611) Fix flaky tests in CoBroadcastWithNonKeyedOperatorTest

2022-10-12 Thread Sopan Phaltankar (Jira)
Sopan Phaltankar created FLINK-29611: Summary: Fix flaky tests in CoBroadcastWithNonKeyedOperatorTest Key: FLINK-29611 URL: https://issues.apache.org/jira/browse/FLINK-29611 Project: Flink

[jira] [Created] (FLINK-29789) Fix flaky tests in CheckpointCoordinatorTest

2022-10-27 Thread Sopan Phaltankar (Jira)
Sopan Phaltankar created FLINK-29789: Summary: Fix flaky tests in CheckpointCoordinatorTest Key: FLINK-29789 URL: https://issues.apache.org/jira/browse/FLINK-29789 Project: Flink Issue