I spoke with Mick offline about this a bit but wanted to relay it here for posterity:
** Why are cdc and compression unit tests run separately? > In the case of cdc, I erred on the side of caution in impl assuming that most people would not be using it so any degradation in performance for the non-cdc case would be net negative for the majority of users. As such, the CDC mutation allocation and segment management codepaths are separate from the regular. Were we to only run tests with cdc enabled, we'd never exercise the code-paths for what would presumably be the majority use-case of the CommitLog. Which I say would be bad. :) This amounts to checking a bool in Mutation a couple times and synchronizing on an object during new segment creation and deletion, so it may be worth considering unifying the commit log allocator paths on the CDC path and remove the need for separate testing in 5.0. There are quite a few different users running CDC out in the world today without significant issue, and some robust system level benchmarking and micro-benchmarking of CommitLog behavior at boundary conditions should establish further confidence in the acceptable nature of a change like that. On Mon, Apr 6, 2020 at 3:20 AM Mick Semb Wever <m...@apache.org> wrote: > It's been three weeks since the last update¹. What follows is a quick > state of affairs and a few questions for anyone to jump in and help > with. > > The ASF CI is running at > https://ci-cassandra.apache.org/view/branches/ and will soon have 36 > builds agents in total. Thanks to Instaclustr, Amazon, iland, and > DataStax. The previous `builds.apache.org` is now retired for all > Cassandra builds. > > > ** Broken Builds > > Tests results continue to improve. Last week at one point we saw trunk > down to 24 test failures for the whole pipeline. That's including > three runs of unit tests (normal, cdc, compression) and three runs of > dtests (normal, novnode, offheap). Because of that it was easy to > see the breakage CASSANDRA-15684. Breakages happen, but to be able to > spot them so quickly is a win! > > > ** Emails on Broken Builds > > Currently you will only get a broken build email if you break the > artifacts stage of the pipeline. > > Pipeline stages that are now consistently passing and are candidates > for adding broken build email notifications to are: > - Cassandra-2.2-jvm-dtest > - Cassandra-2.2-long-test > - Cassandra-3.0-jvm-dtest > - Cassandra-3.0-long-test > - Cassandra-3.0-test > - Cassandra-3.11-jvm-dtest > - Cassandra-3.11-long-test > - Cassandra-3.11-test > - Cassandra-3.11-stress-test > - Cassandra-3.11-test-compression > - Cassandra-trunk-fqltool-test > - Cassandra-trunk-stress-test > - Cassandra-trunk-test > - Cassandra-trunk-test-compression > - Cassandra-trunk-test-burn > > Given no objection, I will add broken email notification to these, > starting with the smaller stages that have been longest stable. > Please make sure that you are not filtering out emails from > jenk...@builds.apache.org so you get to know you broke it before > someone has to tell you. > > > ** Our ci-cassandra Jenkins versus CircleCI > > Thanks to David Capwell, who spent some time comparing ci-cassandra to > CircleCI. > > His findings were… > - There's a reasonable match across the unit, stress, fql, and > jvm-dtest tests. > - CircleCI does not run the cdc unit, burn, long unit, cqlsh tests, > or offheap dtests. > - ci-cassandra does not have JDK 11 test runs, or the upgrade > (jvm-dtest and dtest) tests. > - The cqlsh tests: which only ci-cassandra runs; are broken in trunk > with the recent python3 upgrade. > > > ** Why are cdc and compression unit tests run separately? > > Why do we duplicate unit test runs for cdc and compression? > Is there any ML or ticket to provide history on this? > Can we not instead just enable these settings in the normal unit run? > > > regards, > Mick > > > ¹) > https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
