I've been trying to merge into master for the last 24+ hours, and have
restarted the build about 10 times so far, but it has yet to pass.
I kept track of failures, and these came up numerous times:
1) test GET /dbname/_design_docs (PartitionDDocTest)
test/elixir/test/partition_ddoc_test.exs:160
** (RuntimeError) timed out after 10294 ms
code: retry_until(
stacktrace:
(couchdbtest) test/elixir/lib/couch/db_test.ex:301:
Couch.DBTest.retry_until/4
test/elixir/test/partition_ddoc_test.exs:163: (test)
1) test All Docs tests (AllDocsTest)
test/elixir/test/all_docs_test.exs:15
** (RuntimeError) timed out after 30097 ms
code: retry_until(fn ->
stacktrace:
(couchdbtest) test/elixir/lib/couch/db_test.ex:301:
Couch.DBTest.retry_until/4
test/elixir/test/all_docs_test.exs:75: (test)
which both seem to be timing out.
I'm +1 to removing these tests temporarily.
Jay
On Thu, Dec 12, 2019 at 1:39 PM Joan Touzet <woh...@apache.org> wrote:
> Hi again,
>
> As I've been looking more closely at the CI suite for the Jenkins
> transition, I've noticed that our Elixir test cases are actually the
> most likely to fail. In 6 consecutive CI runs, 5 runs failed due to
> failures in the Elixir suite. (The 6th failed due to a JS test failure.)
>
> We started the Elixir effort to retire the JS suite. We reached a
> decision some months ago to put it into `make check` so that people
> would pay attention to its output, and work to fix those tests,
> accelerating our chances to get rid of the JS suite.
>
> Unfortunately, that's not materialised. Our Elixir test porters seem to
> have stopped their work for a while now, and no one is systematically
> addressing the failures in that suite. I've also heard other developers
> mention (via IRC) that some of the test cases hold invalid assumptions
> about how CouchDB works, especially with the Erlang-based clustering
> code. It sounds to me like the effort needs a full code review.
>
> With 3.0 around the corner, I want people to be able to trust the output
> of `make check` when downloading the tarball. If there is no objection,
> when I merge the Erlang version / CI changes on Monday, I will also
> comment out the call to `make elixir` as part of make check.
>
> When the Elixir porting team is more confident in the reliability and
> completeness of their work, and we can successfully retire the JS suite,
> we can reconsider.
>
> -Joan "really wanting to see green, but only seeing red" Touzet
>
