Hi Org Mode maintainers, I encountered what appears to be an issue with noweb reference expansion during tangling. I'm not sure if this is a known limitation, expected behavior, or if I'm misunderstanding how it should work, but I've prepared some patches that seem to fix the issue.
The Problem: When using `:noweb tangle`, nested noweb references don't get recursively expanded. For example: #+begin_src c :tangle example.c :noweb tangle <<ref1>> <<ref2>> #+end_src #+begin_src c :noweb-ref ref1 // block A content #+end_src #+begin_src c :noweb-ref ref2 :noweb tangle // block B content <<ref3>> #+end_src #+begin_src c :noweb-ref ref3 // block C content #+end_src Expected: The tangled output should contain "// block C content" Actual: The tangled output contains the unexpanded reference "<<ref3>>" Interestingly, changing `:noweb tangle` to `:noweb yes` in the ref2 block makes it work, but that would also expand the reference during export, which I want to avoid. The Root Cause: The issue seems to be in `org-babel-expand-noweb-references`. When it recursively expands nested references, it hardcodes the context to `:eval`, so blocks with `:noweb tangle` don't get expanded even during tangling. The Solution: I've attached two patches: 1. Tests that demonstrate the issue and validate the fix 2. A minimal implementation that adds a context parameter to track whether we're in `:tangle`, `:export`, or `:eval` context The fix maintains backward compatibility (the context parameter is optional and defaults to `:eval`). Before I submit this as a proper patch series, I wanted to check: - Is this the expected behavior, or is it indeed a bug? - If it's a bug, does this approach seem reasonable? - Are there any other considerations I should account for? The patches are available on my branch: `fix-noweb-tangle-recursive-expansion` Thanks for your time and for maintaining this excellent tool! Best regards, Dominic Meiser
From 659c01dbfe06379c64e6785805eff4955c7c3134 Mon Sep 17 00:00:00 2001 From: Dominic Meiser <[email protected]> Date: Sat, 6 Sep 2025 09:12:50 -0600 Subject: [PATCH 1/2] testing: Add tests for noweb tangle recursive expansion * testing/lisp/test-ob-tangle.el (ob-tangle/noweb-tangle-recursive-expansion): Add test to verify that :noweb tangle recursively expands nested noweb references during tangling. (ob-tangle/noweb-tangle-vs-export-contexts): Add test to verify that noweb expansion correctly respects different contexts (:tangle vs :export vs :eval) when determining whether to expand nested references. The first test demonstrates that when a source block with :noweb tangle contains references to other blocks that themselves contain noweb references, all references should be properly expanded in the tangled output, respecting the context-specific noweb settings. The second test ensures that the context-passing mechanism works correctly by verifying that blocks with different :noweb settings expand appropriately based on the expansion context. --- testing/lisp/test-ob-tangle.el | 75 ++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el index cd6876370..25fde1e39 100644 --- a/testing/lisp/test-ob-tangle.el +++ b/testing/lisp/test-ob-tangle.el @@ -764,6 +764,81 @@ This is to ensure that we properly resolve the buffer name." ;; Clean up the tangled file with the filename from org-test-with-temp-text-in-file (delete-file tangle-filename))))) +(ert-deftest ob-tangle/noweb-tangle-recursive-expansion () + "Test that :noweb tangle recursively expands nested noweb references." + (let ((file (make-temp-file "org-tangle-"))) + (unwind-protect + (progn + (org-test-with-temp-text-in-file + (format " +#+begin_src c :tangle %s :noweb tangle +// some code +<<noweb-ref1>> +<<noweb-ref2>> +#+end_src + +#+begin_src c :noweb-ref noweb-ref1 +// code from source block A +#+end_src + +#+begin_src c :noweb-ref noweb-ref2 :noweb tangle +// code from source block B +<<noweb-ref3>> +#+end_src + +#+begin_src c :noweb-ref noweb-ref3 +// code from source block C +#+end_src +" file) + (let ((org-babel-noweb-error-all-langs nil) + (org-babel-noweb-error-langs nil)) + (org-babel-tangle))) + (let ((tangled-content (with-temp-buffer + (insert-file-contents file) + (buffer-string)))) + ;; The tangled output should contain the content from block C + ;; (not the unexpanded <<noweb-ref3>> reference) + (should (string-match-p "// code from source block C" tangled-content)) + ;; The tangled output should NOT contain the unexpanded reference + (should-not (string-match-p "<<noweb-ref3>>" tangled-content)))) + (delete-file file)))) + +(ert-deftest ob-tangle/noweb-tangle-vs-export-contexts () + "Test that noweb expansion respects different contexts during tangle vs export." + (let ((tangle-file (make-temp-file "org-tangle-"))) + (unwind-protect + (progn + (org-test-with-temp-text-in-file + (format " +#+begin_src c :tangle %s :noweb yes +// tangled code +<<tangle-only>> +<<no-export>> +#+end_src + +#+begin_src c :noweb-ref tangle-only :noweb tangle +// visible during tangle +#+end_src + +#+begin_src c :noweb-ref no-export :noweb no-export +// visible during eval but not export +#+end_src +" tangle-file) + ;; Test tangling + (let ((org-babel-noweb-error-all-langs nil) + (org-babel-noweb-error-langs nil)) + (org-babel-tangle))) + + ;; Check tangled content + (let ((tangled-content (with-temp-buffer + (insert-file-contents tangle-file) + (buffer-string)))) + ;; Should have tangle-only content + (should (string-match-p "// visible during tangle" tangled-content)) + ;; Should have no-export content since :noweb no-export allows tangle context + (should (string-match-p "// visible during eval but not export" tangled-content)))) + (delete-file tangle-file)))) + (provide 'test-ob-tangle) ;;; test-ob-tangle.el ends here -- 2.50.1
From ae598b2fc36a162e2391934427407a8077a6c9f4 Mon Sep 17 00:00:00 2001 From: Dominic Meiser <[email protected]> Date: Sat, 6 Sep 2025 09:45:39 -0600 Subject: [PATCH 2/2] ob-core: Fix noweb tangle recursive expansion * lisp/ob-core.el (org-babel-expand-noweb-references): Add optional CONTEXT parameter to specify the expansion context (:tangle, :export, or :eval). Use the context when recursively expanding nested noweb references instead of hardcoding :eval. Improve documentation to explain the context parameter and its importance for recursive expansion. * lisp/ob-tangle.el (org-babel-tangle-single-block): Pass :tangle context to org-babel-expand-noweb-references. This fixes the issue where :noweb tangle would not recursively expand nested noweb references. Previously, when expanding noweb references, nested blocks would only be expanded if they had :noweb yes or :noweb eval, not :noweb tangle, because the expansion context was hardcoded to :eval in the recursive calls. The CONTEXT parameter defaults to :eval when not specified, maintaining backward compatibility. The context determines which :noweb header argument values are honored during recursive expansion: - :tangle context honors "tangle", "yes", "no-export", etc. - :export context honors "yes", "strip-tangle" - :eval context honors "eval", "yes", "no-export", etc. Reported-by: dmeiser --- lisp/ob-core.el | 24 ++++++++++++++++++++---- lisp/ob-tangle.el | 3 ++- testing/lisp/test-ob-tangle.el | 2 +- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 1402827e4..2976ab9bd 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -3137,7 +3137,7 @@ CONTEXT may be one of :tangle, :export or :eval." (defvar org-babel-expand-noweb-references--cache-buffer nil "Cons (BUFFER . MODIFIED-TICK) for cached noweb references. See `org-babel-expand-noweb-references--cache'.") -(defun org-babel-expand-noweb-references (&optional info parent-buffer) +(defun org-babel-expand-noweb-references (&optional info parent-buffer context) "Expand Noweb references in the body of the current source code block. When optional argument INFO is non-nil, use the block defined by INFO @@ -3146,6 +3146,20 @@ instead. The block is assumed to be located in PARENT-BUFFER or current buffer \(when PARENT-BUFFER is nil). +When CONTEXT is specified, use it for noweb expansion context. +CONTEXT may be one of :tangle, :export, or :eval (defaults to :eval). + +The context determines which noweb header arguments are honored when +recursively expanding nested references: +- :tangle context: expands blocks with :noweb tangle, :noweb yes, etc. +- :export context: expands blocks with :noweb export, :noweb yes, etc. +- :eval context: expands blocks with :noweb eval, :noweb yes, etc. + +This is important for recursive expansion: when a block with :noweb tangle +references another block that also contains noweb references, those nested +references should only be expanded if the referenced block's :noweb setting +permits expansion in the tangle context. + For example the following reference would be replaced with the body of the source-code block named `example-block'. @@ -3184,7 +3198,8 @@ block but are passed literally to the \"example-block\"." (not (equal (cdr v) "no")))))) (noweb-re (format "\\(.*?\\)\\(%s\\)" (with-current-buffer parent-buffer - (org-babel-noweb-wrap))))) + (org-babel-noweb-wrap)))) + (context (if context context :eval))) (unless (equal (cons parent-buffer (with-current-buffer parent-buffer (buffer-chars-modified-tick))) @@ -3207,8 +3222,9 @@ block but are passed literally to the \"example-block\"." (expand-body (i) ;; Expand body of code represented by block info I. - `(let ((b (if (org-babel-noweb-p (nth 2 ,i) :eval) - (org-babel-expand-noweb-references ,i) + `(let ((b (if (org-babel-noweb-p (nth 2 ,i) context) + (org-babel-expand-noweb-references + ,i parent-buffer context) (nth 1 ,i)))) (if (not comment) b (let ((cs (org-babel-tangle-comment-links ,i))) diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el index 4c224743b..24c8f876b 100644 --- a/lisp/ob-tangle.el +++ b/lisp/ob-tangle.el @@ -581,7 +581,8 @@ non-nil, return the full association list to be used by (let ((body (if (org-babel-noweb-p params :tangle) (if (string= "strip-tangle" (cdr (assq :noweb (nth 2 info)))) (replace-regexp-in-string (org-babel-noweb-wrap) "" (nth 1 info)) - (org-babel-expand-noweb-references info)) + (org-babel-expand-noweb-references + info nil :tangle)) (nth 1 info)))) (with-temp-buffer (insert diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el index 25fde1e39..be668918a 100644 --- a/testing/lisp/test-ob-tangle.el +++ b/testing/lisp/test-ob-tangle.el @@ -828,7 +828,7 @@ This is to ensure that we properly resolve the buffer name." (let ((org-babel-noweb-error-all-langs nil) (org-babel-noweb-error-langs nil)) (org-babel-tangle))) - + ;; Check tangled content (let ((tangled-content (with-temp-buffer (insert-file-contents tangle-file) -- 2.50.1
