Jay, consider this a +1 to remove those if you can get to them before I do.
If this makes the Elixir tests useful again, then it's a good compromise. -Joan On 2019-12-13 2:18 a.m., Jay Doane wrote:
I've been trying to merge into master for the last 24+ hours, and have restarted the build about 10 times so far, but it has yet to pass. I kept track of failures, and these came up numerous times: 1) test GET /dbname/_design_docs (PartitionDDocTest) test/elixir/test/partition_ddoc_test.exs:160 ** (RuntimeError) timed out after 10294 ms code: retry_until( stacktrace: (couchdbtest) test/elixir/lib/couch/db_test.ex:301: Couch.DBTest.retry_until/4 test/elixir/test/partition_ddoc_test.exs:163: (test) 1) test All Docs tests (AllDocsTest) test/elixir/test/all_docs_test.exs:15 ** (RuntimeError) timed out after 30097 ms code: retry_until(fn -> stacktrace: (couchdbtest) test/elixir/lib/couch/db_test.ex:301: Couch.DBTest.retry_until/4 test/elixir/test/all_docs_test.exs:75: (test) which both seem to be timing out. I'm +1 to removing these tests temporarily. Jay On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet <woh...@apache.org> wrote:Hi again, As I've been looking more closely at the CI suite for the Jenkins transition, I've noticed that our Elixir test cases are actually the most likely to fail. In 6 consecutive CI runs, 5 runs failed due to failures in the Elixir suite. (The 6th failed due to a JS test failure.) We started the Elixir effort to retire the JS suite. We reached a decision some months ago to put it into `make check` so that people would pay attention to its output, and work to fix those tests, accelerating our chances to get rid of the JS suite. Unfortunately, that's not materialised. Our Elixir test porters seem to have stopped their work for a while now, and no one is systematically addressing the failures in that suite. I've also heard other developers mention (via IRC) that some of the test cases hold invalid assumptions about how CouchDB works, especially with the Erlang-based clustering code. It sounds to me like the effort needs a full code review. With 3.0 around the corner, I want people to be able to trust the output of `make check` when downloading the tarball. If there is no objection, when I merge the Erlang version / CI changes on Monday, I will also comment out the call to `make elixir` as part of make check. When the Elixir porting team is more confident in the reliability and completeness of their work, and we can successfully retire the JS suite, we can reconsider. -Joan "really wanting to see green, but only seeing red" Touzet
