On 19.01.2024 13:50, Ihor Radchenko wrote:
gerard.vermeu...@posteo.net writes:
This patch addresses a number of glitches discovered during
interactive
testing and contains possible fixes (sub-patch test-ob-maxima fixes
batch testing on my system, but not interactive testing).
Thanks for your interest in improving Org mode tests!
The test-ob-maxima patch may be due to a different Maxima version, but
I do not think so. In batch testing test-ob-maxima passes contrary to
interactive mode.
On my side, Maxima tests pass both interactively and non-interactively.
I use Maxima 5.47.0
What is your version?
The same: maxima --version
Maxima 5.47.0
In batch testing without the patch I get:
Test ob-maxima/batch+verbatim condition:
(ert-test-failed
((should
(equal (org-babel-execute-src-block)
"(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n
gamma(z + 1)"))
:form
(equal
"(linenum:0,\n(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n
gamma(z + 1)"
"(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n
gamma(z + 1)")
:value nil :explanation
(arrays-of-different-length 104 92
"(linenum:0,\n(assume(z >
0),integrate(exp(-t)*t^z,t,0,inf))\n
gamma(z + 1)"
"(assume(z >
0),integrate(exp(-t)*t^z,t,0,inf))\n
gamma(z + 1)"
first-mismatch-at 1)))
FAILED 162/1252 ob-maxima/batch+verbatim (0.694916 sec) at
../lisp/test-ob
-maxima.el:71
I do not know where the difference comes from. I am not a Maxima user
and not inclined to track this down unless you tell me it is important.
The test-ob-python patch unsets org-src-preserve-indentation which I
have set.
The test-ob patch also unsets org-src-preserve-indentation.
The test-org patch also unsets org-src-preserve-indentation. This
reduces the number of failures to 2 while testing test-org
interactively.
It is generally not expected that Org tests will pass when you try to
run them on Emacs instance with your personal config. This is because
you can have arbitrary customizations, hooks, or advices that change
the
Org mode defaults.
let-binding a couple of variables cannot fix this general problem for
all scenarios. So, I do not see any reason to accept patches that make
the tests run with personal config of a single user.
Before I made the patch, I had reverted changes that worked
interactively
in my personal config but caused failures with make test.
Because of the Maxima test failure (also in batch), I was not sure that
all
tests pass on a reference system.
What might be useful, however, is making sure that tests do not fail
interactively with a clean Emacs config (make repro).
Check out testing/README file.
I have a super-weird system where make repro opens Emacs with
as default-directory my home directory. So I have to make symbolic
links from my home to the testing and lisp directories.
But make test works.
Because of testing my
0001-org-babel-demarcate-block-split-using-element-API.patch
interactively in my personal config (before make test which takes
quite some time), I figured out that it would guess to suppress false
failures.
(ert-deftest ob-maxima/string-input ()
"Test of string input"
(org-test-at-id "b5842ed4-8e8b-4b18-a1c9-cef006b6a6c8"
(org-babel-next-src-block 2)
- (should (equal "- sin(x)" (org-babel-execute-src-block)))))
+ (should (equal '(("(linenum:0," "") ("-" "sin(x)"))
+ (org-babel-execute-src-block)))))
The test in question is testing the output of
#+begin_src maxima :var fun="sin(x)" :var q=2 :results silent
print(diff(fun, x, q))$
#+end_src
It is certainly not expected that such code block returns "linenum:0".
I would say that it is an unexplained difference, I still get (with
"linenum:0") :
#+header: :wrap "src text -n"
#+name: lst:maxima-example
#+begin_src maxima -n :batch batch :cmdline --quiet :exports both
:results raw
rat(1/(x+1) + x/(x-1));
#+end_src
#+name: lst:maxima-example-results
#+results: lst:maxima-example
#+begin_src text -n
(%i1) (linenum:0,
(%i1) rat(1/(x+1)+x/(x-1))
2
x + 2 x - 1
(%o1)/R/ ------------
2
x - 1
#+end_src
Tests in Org mode are verifying that Org mode behaves correctly. It
makes no sense to change the test when Org mode result is not correct -
we should instead fix Org mode.
In the majority of cases, if the test is failing, it is not something
wrong with the test, but something wrong with Org mode code that test
is
verifying.
(ert-deftest test-ob/does-not-replace-a-block-with-the-results ()
(org-test-with-temp-text "#+NAME: foo
diff --git a/testing/lisp/test-org-element.el
b/testing/lisp/test-org-element.el
index ca7d77e28..f0958ff59 100644
--- a/testing/lisp/test-org-element.el
+++ b/testing/lisp/test-org-element.el
@@ -4457,7 +4457,7 @@ Text
'(paragraph nil " Two spaces\n\n \n Two spaces"))
'(paragraph nil "Two spaces\n\n\nTwo spaces")))
(should
- (equal
+ (equal ;; Fails in test. Why? (equal ...) form works in ielm.
It would help if you provided the error ERT displayed when the test
failed.
I cannot reproduce it. Now it works in interactive testing (personal
config
as when I first saw this).
If you ask me, I am still willing to collect the 2 sub-patches with the
let bindings which improve testing interactively in personal configs
somewhat. If not, it is OK (I understand now that it is not important).
Regards -- Gerard
