Ihor Radchenko <yanta...@posteo.net> writes:
>> #+BEGIN_SRC emacs-lisp
>> (list (list 1 2) (list "/" "<>") 'hline (list 3 4) (list 5 6))
>> #+END_SRC
>>
>> Org Babel outputs
>> ...
>> with the second element of the list
>>
>> (list "/" "<>")
>>
>> swallowed, without a word.
>>
>> Why would Org Babel do this?
>>
>> And, how can one output tables with column groups?
>
> This is because of how `orgtbl-to-generic' is implemented. It is taking
> pieces from the full ox.el exporter, hard-coding certain things. For
> example, it always removes special table lines:
>
> ;; Since we are going to export using a low-level mechanism,
> ;; ignore special column and special rows manually.
I refactored `orgtbl-to-generic', so that it does not have to duplicate
`org-export-as'. Now, things should be more consistent with the normal
export.
May you try the attached tentative patch set?
>From f1ff68920c417343fda4c5a6450567d703ccf9b6 Mon Sep 17 00:00:00 2001
Message-ID: <f1ff68920c417343fda4c5a6450567d703ccf9b6.1720440129.git.yanta...@posteo.net>
From: Ihor Radchenko <yanta...@posteo.net>
Date: Mon, 8 Jul 2024 13:52:32 +0200
Subject: [PATCH 1/2] ox: New custom option to disable macro replacement
* lisp/ox.el (org-export-replace-macros): New custom option
controlling macro replacement.
(org-export--annotate-info): Honor it, except when processing inline
code block results and their {{{results...}}} macro.
* etc/ORG-NEWS (Allow disabling macro replacement during export):
Announce the new option.
* doc/org-manual.org (Macro Replacement):
(Summary of the export process): Document the new option.
---
doc/org-manual.org | 11 +++++++----
etc/ORG-NEWS | 9 +++++++++
lisp/ox.el | 13 +++++++++++--
3 files changed, 27 insertions(+), 6 deletions(-)
diff --git a/doc/org-manual.org b/doc/org-manual.org
index d30c18e0c..5f5104f91 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -12682,9 +12682,11 @@ ** Macro Replacement
#+cindex: @samp{MACRO}, keyword
#+vindex: org-export-global-macros
-Macros replace text snippets during export. Macros are defined
-globally in ~org-export-global-macros~, or document-wise with the
-following syntax:
+#+vindex: org-export-replace-macros
+Macros replace text snippets during export[fn::The macro replacement
+can be disabled by setting ~org-export-replace-macros~ to nil (default
+is t).]. Macros are defined globally in ~org-export-global-macros~,
+or document-wise with the following syntax:
: #+MACRO: name replacement text; $1, $2 are arguments
@@ -16702,7 +16704,8 @@ *** Summary of the export process
3. Remove commented subtrees in the whole buffer (see [[*Comment
Lines]]);
-4. Replace macros in the whole buffer (see [[*Macro Replacement]]);
+4. Replace macros in the whole buffer (see [[*Macro Replacement]]),
+ unless ~org-export-replace-macros~ is nil;
5. When ~org-export-use-babel~ is non-nil (default), process code
blocks:
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 0c3b14128..dcd324115 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -43,6 +43,15 @@ or newer.
# adding new customizations, or changing the interpretation of the
# existing customizations.
+*** Allow disabling macro replacement during export
+
+New custom option ~org-export-replace-macros~ controls whether Org
+mode replaces macros in the buffer before export. Set it to nil to
+disable macro replacement.
+
+This variable has no effect on the ={{{results...}}}= macros for inline
+code block results.
+
*** Allow headline/olp target in ~org-capture-templates~ to be a function/variable
The variable ~org-capture-templates~ accepts a target specification as
diff --git a/lisp/ox.el b/lisp/ox.el
index 6fa21be90..902c9f089 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -860,6 +860,14 @@ (defcustom org-export-expand-links t
:package-version '(Org . "9.7")
:type 'boolean)
+(defcustom org-export-replace-macros t
+ "When non-nil, replace macros before export.
+This variable does not affect {{{results}}} macros when processing
+code block results."
+ :group 'org-export-general
+ :package-version '(Org . "9.8")
+ :type 'boolean)
+
(defcustom org-export-snippet-translation-alist nil
"Alist between export snippets backends and exporter backends.
@@ -3048,8 +3056,9 @@ (defun org-export--annotate-info (backend info &optional subtreep visible-only e
(org-export-backend-name backend))
(org-export-expand-include-keyword nil nil nil nil (plist-get info :expand-links))
(org-export--delete-comment-trees)
- (org-macro-initialize-templates org-export-global-macros)
- (org-macro-replace-all org-macro-templates parsed-keywords)
+ (when org-export-replace-macros
+ (org-macro-initialize-templates org-export-global-macros)
+ (org-macro-replace-all org-macro-templates parsed-keywords))
;; Refresh buffer properties and radio targets after previous
;; potentially invasive changes.
(org-set-regexps-and-options)
--
2.45.2
>From 3e0a5164661153d41a89ce984c9282273c54a3de Mon Sep 17 00:00:00 2001
Message-ID: <3e0a5164661153d41a89ce984c9282273c54a3de.1720440129.git.yanta...@posteo.net>
In-Reply-To: <f1ff68920c417343fda4c5a6450567d703ccf9b6.1720440129.git.yanta...@posteo.net>
References: <f1ff68920c417343fda4c5a6450567d703ccf9b6.1720440129.git.yanta...@posteo.net>
From: Ihor Radchenko <yanta...@posteo.net>
Date: Mon, 8 Jul 2024 13:54:14 +0200
Subject: [PATCH 2/2] orgtbl-to-generic: Retain special rows in code block
table output
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
* lisp/org-table.el (orgtbl--skip):
(orgtbl--skipcols): New helper functions.
(orgtbl-to-generic): Use `org-export-as' machinery to setup table
export instead of duplicating ox.el internals. Retain special rows in
tables when exporting to Org. Org export is used by ob-core to format
the code block output and will thus retain special rows.
* testing/lisp/test-org-table.el (test-org-table/to-generic): Adjust
test to expect special rows to be exported.
* etc/ORG-NEWS (~orgtbl-to-generic~ retains special rows when
exporting to Org): Announce the breaking change.
Reported-by: Rudolf Adamkovič <rud...@adamkovic.org>
Link: https://orgmode.org/list/87r0ch6q9h.fsf@localhost
---
etc/ORG-NEWS | 11 +++
lisp/org-table.el | 151 +++++++++++++++++----------------
testing/lisp/test-org-table.el | 2 +-
3 files changed, 89 insertions(+), 75 deletions(-)
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index dcd324115..9780b53a6 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -110,6 +110,17 @@ to dynamically generate the content of the resulting ~<head>~ tag in
the resulting HTML document.
** Miscellaneous
+*** ~orgtbl-to-generic~ retains special rows when exporting to Org
+
+Previously, special table rows were unconditionally removed when
+export to Org. Now, the defaults follow what ox-org does - to retain
+special rows by default. See [[*=ox-org= now exports special table rows
+by default]].
+
+To retain the old behaviour, add ~:with-special-rows nil~ to PARAMS argument:
+
+: (orgtbl-to-generic table '(:with-special-rows nil)
+
*** Trailing =-= is now allowed in plain links
Previously, plain links like
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 4a2623b55..641f2b5f3 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -5654,6 +5654,42 @@ (defun orgtbl-insert-radio-table ()
(insert txt)
(goto-char pos)))
+(defun orgtbl--skip (ast _ info)
+ "Extract first X table rows from AST.
+X is taken from :skip property in INFO plist.
+Return the modified AST."
+ (when-let ((skip (plist-get info :skip)))
+ (unless (wholenump skip) (user-error "Wrong :skip value"))
+ (let ((n 0))
+ (org-element-map ast 'table-row
+ (lambda (row)
+ (if (>= n skip) t
+ (org-element-extract row)
+ (cl-incf n)
+ nil))
+ nil t)))
+ ast)
+
+(defun orgtbl--skipcols (ast _ info)
+ "Extract first X table columns from AST.
+X is taken from :skipcols property in INFO plist.
+Special columns are always ignored.
+Return the modified AST."
+ (when-let ((skipcols (plist-get info :skipcols)))
+ (unless (consp skipcols) (user-error "Wrong :skipcols value"))
+ (org-element-map ast 'table
+ (lambda (table)
+ (let ((specialp (org-export-table-has-special-column-p table)))
+ (dolist (row (org-element-contents table))
+ (when (eq (org-element-property :type row) 'standard)
+ (let ((c 1))
+ (dolist (cell (nthcdr (if specialp 1 0)
+ (org-element-contents row)))
+ (when (memq c skipcols)
+ (org-element-extract cell))
+ (cl-incf c)))))))))
+ ast)
+
;;;###autoload
(defun orgtbl-to-generic (table params)
"Convert the `orgtbl-mode' TABLE to some other format.
@@ -5665,7 +5701,8 @@ (defun orgtbl-to-generic (table params)
line. PARAMS is a property list of parameters that can
influence the conversion.
-Valid parameters are:
+Valid parameters are all the export options understood by the export
+backend and also:
:backend, :raw
@@ -5774,84 +5811,50 @@ (defun orgtbl-to-generic (table params)
;; regular backend has a transcoder for them. We
;; provide one so they are not ignored, but displayed
;; as-is instead.
- (macro . (lambda (m c i) (org-element-macro-interpreter m nil))))))
- data info)
+ (macro . (lambda (m c i) (org-element-macro-interpreter m nil)))
+ ;; Only export the actual table. Do nothing with the
+ ;; containing section regardless what backend think about
+ ;; it. (It is somewhat like BODY-ONLY argument in
+ ;; `org-export-as', but skips not only transcoding the
+ ;; full document, but also section containing the table.
+ (section . (lambda (_ contents _) contents))))))
;; Store TABLE as Org syntax in DATA. Tolerate non-string cells.
;; Initialize communication channel in INFO.
(with-temp-buffer
+ (let ((standard-output (current-buffer)))
+ (dolist (e table)
+ (cond ((eq e 'hline) (princ "|--\n"))
+ ((consp e)
+ (princ "| ") (dolist (c e) (princ c) (princ " |"))
+ (princ "\n")))))
(let ((org-inhibit-startup t)) (org-mode))
- (org-fold-core-ignore-modifications
- (let ((standard-output (current-buffer))
- (org-element-use-cache nil))
- (dolist (e table)
- (cond ((eq e 'hline) (princ "|--\n"))
- ((consp e)
- (princ "| ") (dolist (c e) (princ c) (princ " |"))
- (princ "\n")))))
- (org-element-cache-reset)
- ;; Add backend specific filters, but not user-defined ones. In
- ;; particular, make sure to call parse-tree filters on the
- ;; table.
- (setq info
- (let ((org-export-filters-alist nil))
- (org-export-install-filters
- (org-combine-plists
- (org-export-get-environment backend nil params)
- `(:back-end ,(org-export-get-backend backend))))))
- (setq data
- (org-export-filter-apply-functions
- (plist-get info :filter-parse-tree)
- (org-element-map (org-element-parse-buffer) 'table
- #'identity nil t)
- info))
+ (defvar org-export-before-processing-functions) ; ox.el
+ (defvar org-export-process-citations) ; ox.el
+ (defvar org-export-expand-links) ; ox.el
+ (defvar org-export-filter-parse-tree-functions) ; ox.el
+ (defvar org-export-filters-alist) ; ox.el
+ (declare-function
+ org-export-as "ox"
+ (backend &optional subtreep visible-only body-only ext-plist))
+ ;; We disable the usual pre-processing and post-processing,
+ ;; i.e., hooks, Babel code evaluation, and macro expansion.
+ ;; Only backend specific filters are retained.
+ (let ((org-export-before-processing-functions nil)
+ (org-export-replace-macros nil)
+ (org-export-use-babel nil)
+ (org-export-before-parsing-functions nil)
+ (org-export-process-citations nil)
+ (org-export-expand-links nil)
+ (org-export-filter-parse-tree-functions
+ '(orgtbl--skip orgtbl--skipcols))
+ (org-export-filters-alist
+ '((:filter-parse-tree . org-export-filter-parse-tree-functions))))
+ (when (or (not backend) (plist-get params :raw)) (require 'ox-org))
(when (and backend (symbolp backend) (not (org-export-get-backend backend)))
- (user-error "Unknown :backend value"))))
- (when (or (not backend) (plist-get info :raw)) (require 'ox-org))
- ;; Handle :skip parameter.
- (let ((skip (plist-get info :skip)))
- (when skip
- (unless (wholenump skip) (user-error "Wrong :skip value"))
- (let ((n 0))
- (org-element-map data 'table-row
- (lambda (row)
- (if (>= n skip) t
- (org-element-extract row)
- (cl-incf n)
- nil))
- nil t))))
- ;; Handle :skipcols parameter.
- (let ((skipcols (plist-get info :skipcols)))
- (when skipcols
- (unless (consp skipcols) (user-error "Wrong :skipcols value"))
- (org-element-map data 'table
- (lambda (table)
- (let ((specialp (org-export-table-has-special-column-p table)))
- (dolist (row (org-element-contents table))
- (when (eq (org-element-property :type row) 'standard)
- (let ((c 1))
- (dolist (cell (nthcdr (if specialp 1 0)
- (org-element-contents row)))
- (when (memq c skipcols)
- (org-element-extract cell))
- (cl-incf c))))))))))
- ;; Since we are going to export using a low-level mechanism,
- ;; ignore special column and special rows manually.
- (let ((special? (org-export-table-has-special-column-p data))
- ignore)
- (org-element-map data (if special? '(table-cell table-row) 'table-row)
- (lambda (datum)
- (when (if (org-element-type-p datum 'table-row)
- (org-export-table-row-is-special-p datum nil)
- (org-export-first-sibling-p datum nil))
- (push datum ignore))))
- (setq info (plist-put info :ignore-list ignore)))
- ;; We use a low-level mechanism to export DATA so as to skip all
- ;; usual pre-processing and post-processing, i.e., hooks, Babel
- ;; code evaluation, include keywords and macro expansion. Only
- ;; backend specific filters are retained.
- (let ((output (org-export-data-with-backend data custom-backend info)))
- ;; Remove final newline.
- (if (org-string-nw-p output) (substring-no-properties output 0 -1) ""))))
+ (user-error "Unknown :backend value: %S" backend))
+ (let ((output (org-export-as custom-backend nil nil 'body-only params)))
+ ;; Remove final newline.
+ (if (org-string-nw-p output) (substring-no-properties output 0 -1) ""))))))
(defun org-table--generic-apply (value name &optional with-cons &rest args)
(cond ((null value) nil)
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index df63a65fc..b5d2d157b 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1596,7 +1596,7 @@ (ert-deftest test-org-table/to-generic ()
(orgtbl-to-generic
(org-table-to-lisp "| a | b |\n| c | d |") '(:skipcols (2)))))
(should
- (equal "a\nc"
+ (equal "<c>\na\nc"
(orgtbl-to-generic
(org-table-to-lisp
"| / | <c> | <c> |\n| # | a | b |\n|---+---+---|\n| | c | d |")
--
2.45.2
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
