On reducing circle ci usage during dev while iterating, not with the
intention to replace the pre-commit CI (yet), we could do away with
testing only dtests, jvm-dtests, units and cqlsh for a _single_
configuration imo. That would greatly reduce usage. I hacked it quickly
here for illustration purposes:
https://app.circleci.com/pipelines/github/bereng/cassandra/1164/workflows/3a47c9ef-6456-4190-b5a5-aea2aff641f1
The good thing is that we have the tooling to dial in whatever we decide
atm.
Changing pre-commit is a different discussion, to which I agree btw. But
the above could save time and $ big time during dev and be done and
merged in a matter of days imo.
I can open a DISCUSS thread if we feel it's worth it.
On 15/2/24 10:24, Mick Semb Wever wrote:
Mick and Ekaterina (and everyone really) - any thoughts on what
test coverage, if any, we should commit to for this new
configuration? Acknowledging that we already have /a lot/ of CI
that we run.
Branimir in this patch has already done some basic cleanup of test
variations, so this is not a duplication of the pipeline. It's a
significant improvement.
I'm ok with cassandra_latest being committed and added to the
pipeline, *if* the authors genuinely believe there's significant time
and effort saved in doing so.
How many broken tests are we talking about ?
Are they consistently broken or flaky ?
Are they ticketed up and 5.0-rc blockers ?
Having to deal with flakies and broken tests is an unfortunate reality
to having a pipeline of 170k tests.
Despite real frustrations I don't believe the broken windows analogy
is appropriate here – it's more of a leave the campground cleaner…That
being said, knowingly introducing a few broken tests is not that
either, but still having to deal with a handful of consistently
breaking tests for a short period of time is not the same cognitive
burden as flakies. There are currently other broken tests in 5.0:
VectorUpdateDeleteTest, upgrade_through_versions_test; are these
compounding to the frustrations ?
It's also been questioned about why we don't just enable settings we
recommend. These are settings we recommend for new clusters. Our
existing cassandra.yaml needs to be tailored for existing clusters
being upgraded, where we are very conservative about changing defaults.
