Maxim Nikulin <maniku...@gmail.com> writes:
> To be clear, I do not ask for any changes. It is great to have some
> tests even in the current form. I just have never tried ert before, so I
> have some questions.
Note that I have no experience with tests since I never did programming
professionally. I wrote the tests in the patch simply by looking at
other examples in the file.
> Am I right that just one failure will be reported if a change in the
> regexp causes problems with several test inputs? I am afraid, it is
> rather inconvenient since this tests are rather for quality of
> heuristics than exact requirements. To find proper balance of accuracy
> and speed/regexp complexity, it is better to have all failures at once.
> I am looking up for a feature like EXPECT_EQ (delayed failure) in
> addition to strict ASSERT_EQ in googletest or at least like
> TestCase.subTest in puthon unittest. For this particular case even
> parametrized tests are enough (googletest, pytest).
I cannot find anything like delayed failure in the info node for ERT and
in the source code. There are should, should-not, should-error, and
skip-unless asserts.
> I have tried implement something similar to illustrate my expectations.
> I think, for experienced lisp programmers the code looks ugly
Using a macro is certainly more efficient than my copy-paste in the
patch :) Though you may probably use more backquoting (see Backquote in
Elisp manual) when writing macros. That would look more readable.
> I do not know if it is possible to implement "might" (in addition to
> "should") that using restart or some other technique will prevent
> immediate abandoning of the test but will mark whole test as failed at
> the end.
It is indeed possible and maybe even welcome in emacs-devel.
> Actually I hope to get response that I am trying to reinvent a wheel and
> org or emacs has an established way to write tests feeding the same
> function with list of cases
Well... Example from ERT info page:
(ert-deftest ert-test-mismatch ()
(should (eql (cl-mismatch "" "") nil))
(should (eql (cl-mismatch "" "a") 0))
(should (eql (cl-mismatch "a" "a") nil))
(should (eql (cl-mismatch "ab" "a") 1))
(should (eql (cl-mismatch "Aa" "aA") 0))
(should (eql (cl-mismatch '(a b c) '(a b d)) 2)))
> ... and to get all failures in a reasonably readable report.
When running ert interactively, things should get a little more
manageable. See Running Tests Interactively node of ERT manual.
Hope it helps.
Best,
Ihor
