Re: [PROPOSAL] Remove `make elixir` from `make check`

2019-12-12 Thread Joan Touzet
Jay, consider this a +1 to remove those if you can get to them before I do. If this makes the Elixir tests useful again, then it's a good compromise. -Joan On 2019-12-13 2:18 a.m., Jay Doane wrote: I've been trying to merge into master for the last 24+ hours, and have restarted the build about

Re: [PROPOSAL] Remove `make elixir` from `make check`

2019-12-12 Thread Jay Doane
I've been trying to merge into master for the last 24+ hours, and have restarted the build about 10 times so far, but it has yet to pass. I kept track of failures, and these came up numerous times: 1) test GET /dbname/_design_docs (PartitionDDocTest) test/elixir/test/partition_ddoc_test.exs

Re: [PROPOSAL] Remove `make elixir` from `make check`

2019-12-12 Thread Joan Touzet
Hi Adam, As long as someone is working on fixing the Elixir tests, fine. They *are* failing significantly more often on Jenkins than on Travis, for what it's worth. All our work to build a much better setup for Jenkins may be lost if people don't fix these tests promptly. Would a suitable

Re: [PROPOSAL] Drop Erlang 19 support in CouchDB 3.0

2019-12-12 Thread Adam Kocoloski
+1 > On Dec 11, 2019, at 8:35 PM, Joan Touzet wrote: > > Hello everyone, > > I'm working this week with Paul Davis on our new Jenkins CI > infrastructure, which is coming along nicely. One of the changes I'm > planning to make is that our PR tests will run against only 3 versions > of Erlang:

Re: [PROPOSAL] Remove `make elixir` from `make check`

2019-12-12 Thread Adam Kocoloski
Hi Joan, I’ve seen the Elixir suite implicated more frequently as well. I haven’t done the analysis to see if the failures are concentrated in one or two flakes or if they’re more evenly distributed. If it’s a small number of flaky tests I think we have time to fix/disable them rather than kno

[PROPOSAL] Remove `make elixir` from `make check`

2019-12-12 Thread Joan Touzet
Hi again, As I've been looking more closely at the CI suite for the Jenkins transition, I've noticed that our Elixir test cases are actually the most likely to fail. In 6 consecutive CI runs, 5 runs failed due to failures in the Elixir suite. (The 6th failed due to a JS test failure.) We sta

Re: Script for displaying test errors and timing information

2019-12-12 Thread Joan Touzet
+1. I think this is useful information to collect on every CI run and store alongside the run, though keeping in mind we only keep the last 7-10 runs in Jenkins nowadays. If we need it longer, we should stash the output with the log stasher. This could allow graphing if someone wants to build

Re: [PROPOSAL] Drop Erlang 19 support in CouchDB 3.0

2019-12-12 Thread Joan Touzet
Sure...do you mean just in the release notes, or some tangible change to rebar.config.script / configure? -Joan On 2019-12-12 4:06 p.m., Paul Davis wrote: +1 The only thought that comes to mind is that it might be useful to differ in some of our error messages between versions. AFAIK, there's

Re: [PROPOSAL] Drop Erlang 19 support in CouchDB 3.0

2019-12-12 Thread Paul Davis
+1 The only thought that comes to mind is that it might be useful to differ in some of our error messages between versions. AFAIK, there's nothing from 19.x that currently would prevent someone from using it. We just don't have the resources to cover all of the tests and packaging for it. Which is

Script for displaying test errors and timing information

2019-12-12 Thread Paul Davis
Hey all, I was poking around at Jenkins the other day trying to get a good idea of how much time we're spending in various parts of the build. It occurred to me that one good way to at least investigate our eunit test suite is to parse all of the generated surefire reports. I spent an hour or so y

Re: [PROPOSAL] Drop Erlang 19 support in CouchDB 3.0

2019-12-12 Thread Dave Cottlehuber
On Thu, 12 Dec 2019, at 01:35, Joan Touzet wrote: > Hello everyone, > > I'm working this week with Paul Davis on our new Jenkins CI > infrastructure, which is coming along nicely. One of the changes I'm > planning to make is that our PR tests will run against only 3 versions > of Erlang: > > 1. T