Complicated question unfortunately — and something we’re actively working on improving:
Cassci is no longer being offered/run by Datastax and so we've need to come up with a new solution, and what that ultimately is is still a WIP — it’s loss was very huge obviously and a testament to the awesome resource and effort that was put into providing it to the community for all those years. - Short Term/Current: Tests (both dtests and unit tests) are being run via the ASF Jenkins (https://builds.apache.org) - but that solution isn’t hugely helpful as it’s resource constrained. - Short-Medium Term: we hope to get a fully baked CircleCI solution to get reliable fast test runs. - Long Term: Actively being discussed but I’m optimistic that we can get something awesome for the project with some stable combination of CircleCI + ASF Jenkins, and once we do I’m sure this will change any long term plans. For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/cassandra/tree/trunk/test/unit/org/apache/cassandra): Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/… looks like the last successful job to finish was #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/389/testReport/). There are currently a total of 6 tests (all from CompressedInputStreamTest) failing on trunk via ASF Jenkins. These specific test failures are environmental. The only *unit* test on trunk that I currently know to be flaky is org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054) For Distributed Tests (DTests) (a.k.a the Python ones - https://github.com/apache/cassandra-dtest): The situation is a great deal more complicated due to the length of time and number of resources executing all of the dtests take (and executing the tests across the various configurations)... There are 4 dtest jobs on ASF Jenkins for trunk: https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/ https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-large/ https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-novnode/ https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-offheap/ It looks like you’ll need to go back to run #353 (https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/353/testReport/) to see the test results as the last 2 jobs that were triggered failed to execute. Depending on the environment variables set tests are executed or skipped — so you’ll see different tests being run on the no-vnode job/off-heap job/regular dtest job (or some tests might be run multiple times) More recently we’ve been woking on getting CircleCI running. Some sample runs from my personal fork can be seen at https://circleci.com/gh/mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid account to get more CircleCI resources (with 100 containers we can actually build the project, run all of the unit tests, and run all of the dtests in roughly 28 minutes!). I’m actively working to determine out exactly can (and cannot) be executed reliably, routinely, and easily by anyone with just a simple free CircleCI account. I’m also working on getting scheduled CircleCI daily runs setup against trunk/3.0 — more on both of those when we’ve got that story fully baked.. Hope this answers your question! There are quite a few dtests currently failing and as Jeff mentioned I’ve created JIRAs for a lot of them already so any help (no matter how trivial or annoying it might be or seem) to get everything green again. best, kjellman On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia <chovatia.jayd...@gmail.com<mailto:chovatia.jayd...@gmail.com>> wrote: Is there a way to check which tests are failing in trunk currently? Previously this URL <http://cassci.datastax.com/> was giving such results but is no longer working. Jaydeep On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa <jji...@gmail.com<mailto:jji...@gmail.com>> wrote: In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help. If you haven't been paying attention to JIRA, you likely didn't notice that Josh went through and triage/categorized a bunch of issues by adding components, and Michael took the time to open a bunch of JIRAs for failing tests. How many is a bunch? Something like 35 or so just for tests currently failing on trunk. If you're a regular contributor, you already know that dtests are flakey - it'd be great if a few of us can go through and fix a few. Even incremental improvements are improvements. Here's an easy search to find them: https://issues.apache.org/jira/secure/IssueNavigator. jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+ component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+ DESC%2C+created+ASC&mode=hide If you're a new contributor, fixing tests is often a good way to learn a new part of the codebase. Many of these are dtests, which live in a different repo ( https://github.com/apache/cassandra-dtest ) and are in python, but have no fear, the repo has instructions for setting up and running dtests( https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md ) Normal contribution workflow applies: self-assign the ticket if you want to work on it, click on 'start progress' to indicate that you're working on it, mark it 'patch available' when you've uploaded code to be reviewed (in a github branch, or as a standalone patch file attached to the JIRA). If you have questions, feel free to email the dev list (that's what it's here for). Many thanks will be given, - Jeff
