I have a very minor batch of test improvement patches. Lately I've been trying to run all of our tests using Guix. This is a nice usage of Guix as it means in theory I can run all of our org-babel tests (git-annex, java, etc...) without having to have them clutter up my everyday PATH.
I have been running into some problems and I can't successfully run all the tests quite yet. I think it's been a while since some of these tests have been run. I don't belive anyone is currently running all the tests regularly. My current situation looks like this: Ran 1602 tests, 1570 results as expected, 28 unexpected, 4 skipped 11 expected failures I've got 2 failures in ob-maxima All the Julia tests fail One ruby test fails All the scheme tests fail One org-attach-git test fails I have yet to try running the csharp tests using mono. The default compiler is "dotnet" which isn't available on Guix as the FSF doesn't like it or something. Not sure exactly what's wrong with dotnet from a freedom perspective. Might be something about bootstrapping it. I imagine that most of those failures have to do with my system setup, not the tests themselves. More investigation is needed. These commits mostly pass CI Morgan. Recent commits to the tests have broken them with LANG set to non-english and on Emacs 28. I might get around to fixing that eventually. The build log and guix build definition are available upon request.
>From 989d6ebe50ed2a7c44dc21ac8110eefbfa308609 Mon Sep 17 00:00:00 2001 From: Morgan Smith <[email protected]> Date: Sat, 11 Apr 2026 10:22:56 -0400 Subject: [PATCH 1/4] Testing: Fix various compile warnings * testing/lisp/test-ob-clojure.el: Declare `org-babel-clojure-backend'. Require ob-clojure. * testing/lisp/test-ob.el (test-ob/noweb-expansion): Pass the correct number of arguments to format. * testing/lisp/test-org-clock.el: Require test-duplicates-detector. * testing/lisp/test-org-element.el (test-org-element/babel-call-parser): Wrap useless `eq' clause with `should' to make it useful. * testing/lisp/test-org.el: Require org-colview. --- testing/lisp/test-ob-clojure.el | 2 ++ testing/lisp/test-ob.el | 2 +- testing/lisp/test-org-clock.el | 1 + testing/lisp/test-org-element.el | 9 ++++----- testing/lisp/test-org.el | 1 + 5 files changed, 9 insertions(+), 6 deletions(-) diff --git a/testing/lisp/test-ob-clojure.el b/testing/lisp/test-ob-clojure.el index 78aa2f0e3..b70271eae 100644 --- a/testing/lisp/test-ob-clojure.el +++ b/testing/lisp/test-ob-clojure.el @@ -25,8 +25,10 @@ ;;; Code: (require 'org-test "../testing/org-test") +(defvar org-babel-clojure-backend) (unless (and (featurep 'ob-clojure) org-babel-clojure-backend) (signal 'missing-test-dependency '("Support for Clojure code blocks"))) +(require 'ob-clojure) ;; tangle (ert-deftest ob-clojure/org-babel-tangle () diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el index 3644a511b..81df263d2 100644 --- a/testing/lisp/test-ob.el +++ b/testing/lisp/test-ob.el @@ -1014,7 +1014,7 @@ test-ob/noweb-expansion (format "prefix;; [[file:%s::inner][inner]] prefix1 prefix;; inner ends here" - file file) + file) (org-babel-expand-noweb-references nil nil :eval)))))) (ert-deftest test-ob/splitting-variable-lists-in-references () diff --git a/testing/lisp/test-org-clock.el b/testing/lisp/test-org-clock.el index 0aee027d1..6d7ff3649 100644 --- a/testing/lisp/test-org-clock.el +++ b/testing/lisp/test-org-clock.el @@ -14,6 +14,7 @@ ;;; Code: (require 'org-test "../testing/org-test") +(require 'test-duplicates-detector "../testing/lisp/test-duplicates-detector") (require 'org-duration) (require 'org-clock) diff --git a/testing/lisp/test-org-element.el b/testing/lisp/test-org-element.el index cc058fff9..4628867ae 100644 --- a/testing/lisp/test-org-element.el +++ b/testing/lisp/test-org-element.el @@ -935,11 +935,10 @@ test-org-element/babel-call-parser (eq 'babel-call (org-test-with-temp-text "#+CALL: test()" (org-element-type (org-element-at-point))))) - (should - (org-test-with-temp-text "#+CALL: test\n\n" - (eq 'babel-call - (org-element-type (org-element-at-point))) - (equal "test" (org-element-property :call (org-element-at-point))))) + (org-test-with-temp-text "#+CALL: test\n\n" + (should (eq 'babel-call + (org-element-type (org-element-at-point)))) + (should (equal "test" (org-element-property :call (org-element-at-point))))) ;; Ignore case. (should (eq 'babel-call diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el index 949e75089..1d7b4e161 100644 --- a/testing/lisp/test-org.el +++ b/testing/lisp/test-org.el @@ -29,6 +29,7 @@ (eval-when-compile (require 'cl-lib)) (eval-when-compile (require 'org-macs)) ;For `org-with-gensyms'. (require 'org) +(require 'org-colview) (require 'org-inlinetask) (require 'org-refile) (require 'org-agenda) base-commit: e186e41778a46b8d12015db5a1c79aab3ee33565 -- 2.54.0
>From a69a703cfe1256e1e0b4ffdfda58b0493ecbf751 Mon Sep 17 00:00:00 2001 From: Morgan Smith <[email protected]> Date: Wed, 22 Jul 2026 13:10:36 -0400 Subject: [PATCH 2/4] Testing: Don't set `org-babel-temporary-directory' in tests This causes test failures in build systems that don't have "/tmp" available as a temporary directory (like GNU Guix). Also the other org-babel tests don't seem to require this. * testing/lisp/test-ob-R.el (ob-session-async-R-simple-session-async-value) (ob-session-async-R-simple-session-async-output) (ob-session-async-R-named-output ob-session-async-R-named-value) (ob-session-async-R-output-drawer ob-session-async-R-value-drawer) (ob-session-R-result-output ob-session-R-result-value) (ob-R-nested-list): * testing/lisp/test-ob-julia.el (test-ob-julia/async-simple-session-output) (test-ob-julia/async-named-output, test-ob-julia/async-output-drawer): * testing/lisp/test-ob-python.el (test-ob-python/async-simple-session-output) (test-ob-python/inline-session-output) (test-ob-python/async-inline-session-output) (test-ob-python/async-named-output) (test-ob-python/async-output-drawer): Don't set `org-babel-temporary-directory'. --- testing/lisp/test-ob-R.el | 11 +---------- testing/lisp/test-ob-julia.el | 5 +---- testing/lisp/test-ob-python.el | 11 +++-------- 3 files changed, 5 insertions(+), 22 deletions(-) diff --git a/testing/lisp/test-ob-R.el b/testing/lisp/test-ob-R.el index f7b563df9..eb0ff8c56 100644 --- a/testing/lisp/test-ob-R.el +++ b/testing/lisp/test-ob-R.el @@ -185,7 +185,6 @@ test-ob-r/NA-blank (ert-deftest ob-session-async-R-simple-session-async-value () (let (ess-ask-for-ess-directory ess-history-file - (org-babel-temporary-directory "/tmp") (org-confirm-babel-evaluate nil)) (org-test-with-temp-text "#+begin_src R :session R :async yes\n Sys.sleep(.1)\n paste(\"Yep!\")\n#+end_src\n" @@ -200,7 +199,6 @@ ob-session-async-R-simple-session-async-value (ert-deftest ob-session-async-R-simple-session-async-output () (let (ess-ask-for-ess-directory ess-history-file - (org-babel-temporary-directory "/tmp") (org-confirm-babel-evaluate nil) ;; Workaround for Emacs 27. See https://orgmode.org/list/87ilduqrem.fsf@localhost (ess-r-post-run-hook (lambda () (ess-command (ess-calculate-width 9999))))) @@ -217,7 +215,6 @@ ob-session-async-R-simple-session-async-output (ert-deftest ob-session-async-R-named-output () (let (ess-ask-for-ess-directory ess-history-file - (org-babel-temporary-directory "/tmp") org-confirm-babel-evaluate (src-block "#+begin_src R :async :session R :results output\n 1:5\n#+end_src") (results-before "\n\n#+NAME: foobar\n#+RESULTS:\n: [1] 1") @@ -235,7 +232,6 @@ ob-session-async-R-named-value (let (ess-ask-for-ess-directory ess-history-file org-confirm-babel-evaluate - (org-babel-temporary-directory "/tmp") (src-block "#+begin_src R :async :session R :results value\n paste(\"Yep!\")\n#+end_src") (results-before "\n\n#+NAME: foobar\n#+RESULTS:\n: [1] 1") (results-after "\n\n#+NAME: foobar\n#+RESULTS:\n: Yep!\n")) @@ -250,7 +246,6 @@ ob-session-async-R-output-drawer (let (ess-ask-for-ess-directory ess-history-file org-confirm-babel-evaluate - (org-babel-temporary-directory "/tmp") (src-block "#+begin_src R :async :session R :results output drawer\n 1:5\n#+end_src") (result "\n\n#+RESULTS:\n:results:\n[1] 1 2 3 4 5\n:end:\n") ;; Workaround for Emacs 27. See https://orgmode.org/list/87ilduqrem.fsf@localhost @@ -266,7 +261,6 @@ ob-session-async-R-value-drawer (let (ess-ask-for-ess-directory ess-history-file org-confirm-babel-evaluate - (org-babel-temporary-directory "/tmp") (src-block "#+begin_src R :async :session R :results value drawer\n 1:3\n#+end_src") (result "\n\n#+RESULTS:\n:results:\n| 1 |\n| 2 |\n| 3 |\n:end:\n")) (org-test-with-temp-text @@ -281,7 +275,6 @@ ob-session-R-result-output (let (ess-ask-for-ess-directory ess-history-file org-confirm-babel-evaluate - (org-babel-temporary-directory "/tmp") (src-block "#+begin_src R :session R :results output \n 1:3\n#+end_src") (result "\n\n#+RESULTS:\n: [1] 1 2 3\n" )) (org-test-with-temp-text @@ -294,8 +287,7 @@ ob-session-R-result-output (ert-deftest ob-session-R-result-value () (let (ess-ask-for-ess-directory ess-history-file - org-confirm-babel-evaluate - (org-babel-temporary-directory "/tmp")) + org-confirm-babel-evaluate) (org-test-with-temp-text "#+begin_src R :session R :results value \n 1:50\n#+end_src" (should @@ -308,7 +300,6 @@ ob-R-nested-list (let (ess-ask-for-ess-directory ess-history-file org-confirm-babel-evaluate - (org-babel-temporary-directory "/tmp") (text " #+NAME: example-list - simple diff --git a/testing/lisp/test-ob-julia.el b/testing/lisp/test-ob-julia.el index 34d433a00..95a0e6cb5 100644 --- a/testing/lisp/test-ob-julia.el +++ b/testing/lisp/test-ob-julia.el @@ -219,8 +219,7 @@ test-ob-julia/session-value-sleep (org-babel-execute-src-block))))) (ert-deftest test-ob-julia/async-simple-session-output () - (let ((org-babel-temporary-directory temporary-file-directory) - (org-confirm-babel-evaluate nil)) + (let ((org-confirm-babel-evaluate nil)) (org-test-with-temp-text "#+begin_src julia :session :async yes :results output import time @@ -237,7 +236,6 @@ test-ob-julia/async-simple-session-output (ert-deftest test-ob-julia/async-named-output () (let (org-confirm-babel-evaluate - (org-babel-temporary-directory temporary-file-directory) (src-block "#+begin_src julia :async :session :results output print(\"Yep!\") #+end_src") @@ -261,7 +259,6 @@ test-ob-julia/async-named-output (ert-deftest test-ob-julia/async-output-drawer () (let (org-confirm-babel-evaluate - (org-babel-temporary-directory temporary-file-directory) (src-block "#+begin_src julia :async :session :results output drawer print(list(range(3))) #+end_src") diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el index 073bac0a4..3a38a6e8e 100644 --- a/testing/lisp/test-ob-python.el +++ b/testing/lisp/test-ob-python.el @@ -239,8 +239,7 @@ test-ob-python/async-simple-session-output ;; Disable the test on older Emacs as built-in python.el sometimes ;; fail to initialize session. (skip-unless (version<= "28" emacs-version)) - (let ((org-babel-temporary-directory temporary-file-directory) - (org-confirm-babel-evaluate nil)) + (let ((org-confirm-babel-evaluate nil)) (org-test-with-temp-text "#+begin_src python :session :async yes :results output import time @@ -259,8 +258,7 @@ test-ob-python/inline-session-output ;; Disable the test on older Emacs as built-in python.el sometimes ;; fail to initialize session. (skip-unless (version<= "28" emacs-version)) - (let ((org-babel-temporary-directory temporary-file-directory) - (org-confirm-babel-evaluate nil) + (let ((org-confirm-babel-evaluate nil) (org-babel-inline-result-wrap "=%s=")) (org-test-with-temp-text "src_python[:session :results output]{print(1+1)}" @@ -270,8 +268,7 @@ test-ob-python/async-inline-session-output ;; Disable the test on older Emacs as built-in python.el sometimes ;; fail to initialize session. (skip-unless (version<= "28" emacs-version)) - (let ((org-babel-temporary-directory temporary-file-directory) - (org-confirm-babel-evaluate nil) + (let ((org-confirm-babel-evaluate nil) (org-babel-inline-result-wrap "=%s=") (test-line "src_python[:session :async yes :results output]{print(1+1)}")) (org-test-with-temp-text @@ -291,7 +288,6 @@ test-ob-python/async-named-output ;; fail to initialize session. (skip-unless (version<= "28" emacs-version)) (let (org-confirm-babel-evaluate - (org-babel-temporary-directory temporary-file-directory) (src-block "#+begin_src python :async :session :results output print(\"Yep!\") #+end_src") @@ -318,7 +314,6 @@ test-ob-python/async-output-drawer ;; fail to initialize session. (skip-unless (version<= "28" emacs-version)) (let (org-confirm-babel-evaluate - (org-babel-temporary-directory temporary-file-directory) (src-block "#+begin_src python :async :session :results output drawer print(list(range(3))) #+end_src") -- 2.54.0
>From 421cc872d185e44dd2d01c8d860ef53121d46f14 Mon Sep 17 00:00:00 2001 From: Morgan Smith <[email protected]> Date: Wed, 22 Jul 2026 13:31:12 -0400 Subject: [PATCH 3/4] Testing: Fix detection of git-annex executable * testing/lisp/test-org-attach-git.el: Use `executable-find' instead of `org-test-for-executable' as git-annex version 10.20250721 errors when passed the flag "--version". --- testing/lisp/test-org-attach-git.el | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/testing/lisp/test-org-attach-git.el b/testing/lisp/test-org-attach-git.el index 9e262e500..68ae2834a 100644 --- a/testing/lisp/test-org-attach-git.el +++ b/testing/lisp/test-org-attach-git.el @@ -26,7 +26,11 @@ (require 'org-test "../testing/org-test") -(org-test-for-executable "git-annex") +;; `org-test-for-executable' won't work since git-annex errors when +;; called like "git-annex --version" +(unless (executable-find "git-annex") + (signal 'missing-test-dependency (list "git-annex"))) + (require 'org-attach-git) (require 'cl-lib) -- 2.54.0
>From e4c4cf854e076ebc17beccb001af71410771f7f1 Mon Sep 17 00:00:00 2001 From: Morgan Smith <[email protected]> Date: Wed, 22 Jul 2026 14:45:42 -0400 Subject: [PATCH 4/4] Testing: ob-haskell/ghci-info: Fix test * testing/lisp/test-ob-haskell-ghci.el (ob-haskell/ghci-info): Update to regex to also accept the output I receive from ghc version 9.10.2 --- testing/lisp/test-ob-haskell-ghci.el | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/testing/lisp/test-ob-haskell-ghci.el b/testing/lisp/test-ob-haskell-ghci.el index 70a8a842e..a458e6541 100644 --- a/testing/lisp/test-ob-haskell-ghci.el +++ b/testing/lisp/test-ob-haskell-ghci.el @@ -463,7 +463,7 @@ ob-haskell/ghci-type (ert-deftest ob-haskell/ghci-info () "The ghci meta command ':info' ." (should (string-match-p - "repeat :: a -> \\[a\\][ \t]+-- Defined in ‘GHC.List’" + "repeat :: a -> \\[a\\][ \t]+-- Defined in ‘GHC.\\(Internal.\\)?List’" (test-ob-haskell-ghci ":results output" ":info repeat")))) -- 2.54.0
