Hi, At least some of those failures are real. I don't think we should release 3.10 until the real failures are addressed. As I said earlier one of them is a wrong answer bug that is not going to be fixed in 3.10.
Can we just ignore failures because we think they don't mean anything? Who is going to check which of the 60 failures is real? These tests were passing just fine at the beginning of December and then commits happened and now the tests are failing. That is exactly what their for. They are good tests. I don't think it matters if the failures are "real" today because those are valid tests and they don't test anything if they fail for spurious reasons. They are a critical part of the Cassandra infrastructure as much as the storage engine or network code. In my opinion the tests need to be fixed and people need to fix them as they break them and we need to figure out how to get from people breaking them and it going unnoticed to they break it and then fix it in a time frame that fits the release schedule. My personal opinion is that releases are a reward for finishing the job. Releasing without finishing the job creates the wrong incentive structure for the community. If you break something you are no longer the person that blocked the release you are just one of several people breaking things without consequence. I think that rapid feedback and triaging combined with releases blocked by the stuff individual contributors have broken is the way to more consistent releases both schedule wise and quality wise. Regarding delaying 3.10? Who exactly is the consumer that is chomping at the bit to get another release? One that doesn't reliably upgrade from a previous version? Ariel On Tue, Jan 10, 2017, at 08:13 AM, Josh McKenzie wrote: > First, I think we need to clarify if we're blocking on just testall + > dtest > or blocking on *all test jobs*. > > If the latter, upgrade tests are the elephant in the room: > http://cassci.datastax.com/view/cassandra-3.11/job/cassandra-3.11_dtest_upgrade/lastCompletedBuild/testReport/ > > Do we have confidence that the reported failures are all test problems > and > not w/Cassandra itself? If so, is that documented somewhere? > > On Mon, Jan 9, 2017 at 7:33 PM, Nate McCall <zznat...@gmail.com> wrote: > > > I'm not sure I understand the culmination of the past couple of threads on > > this. > > > > With a situation like: > > http://cassci.datastax.com/view/cassandra-3.11/job/cassandra-3.11_dtest/ > > lastCompletedBuild/testReport/ > > > > We have some sense of stability on what might be flaky tests(?). > > Again, I'm not sure what our criteria is specifically. > > > > Basically, it feels like we are in a stalemate right now. How do we > > move forward? > > > > -Nate > >
