Hi, I think there's one thing that makes us unhappy, especially for new contributors to the project: the health of our CI. We probably see it green about as often as a solar eclipse. :-)
It wastes time and makes it harder and more frustrating to check CI results for patches and to plan releases with confidence. We've also developed a kind of broken window mentality towards CI failures: we tend to ignore them, even though some of them may actually highlight real issues. I've already done some work to reduce the number of failing tests, and now I'm going to invest a bit more time in this area. My current goal is to get the 6.0 CI running without test failures, at least for a while. 1. Fix JMH test reporting (done). For a few days, our CI was failing because of a JMH test failure that actually highlighted a real issue. JMH test failures are now classified and reported as JUnit test failures, without aborting the whole build: https://issues.apache.org/jira/browse/CASSANDRA-21757 2. Work through the existing test failures. I'll go through the tests that are currently failing or have been reported as flaky and fix them, or reach out to the relevant committers for help. For example: https://issues.apache.org/jira/browse/CASSANDRA-21769 Another area we can improve in parallel is CI execution time. The current turnaround time for 6.0 CI runs (8–9 hours) is too long, and we're increasingly tempted to avoid or minimise CI runs for patches because of the cost. It also creates a barrier for contributors who don't have enough hardware to run a large test suite locally. It also quickly consumes our donated CI credits. Here are three areas I'd like to focus on: 1. Optimise test scopes. Reduce the number of duplicate tests that don't provide additional coverage, particularly in the Python dtests. 2. Reduce the number of JDK versions we need to support and test. As we agreed in our JDK upgrade policy some time ago, we can propagate support for newer JDKs to older Cassandra releases, allowing us to drop support for older JDKs in trunk. 3. Optimise test execution itself. In particular, improve test and test-class startup and shutdown times. See also: https://issues.apache.org/jira/browse/CASSANDRA-21383 for more details/ideas. The ultimate goal is to make CI more reliable, faster, and less expensive to run, so that we can spend less time dealing with CI failures and more time developing Cassandra with CI as a proper safety net. -- Dmitry
