Re: [PATCH] Introduce "export features"

[Ihor Radchenko]

Sébastien Miquel  writes:

> Ihor Radchenko writes:
 The traditional user-facing approach for toggling staff in export is
 setting export options.
>>> Indeed. Then I suggest that such use be described in the manual.
>>> Having the user create a whole new option in order to toggle a
>>> `chikenize` package seems a bit unwieldy, however.
>> What do you suggest instead of creating a new option?
>
> Timothy's patch supports having a feature depend on a variable. I was
> thinking here of a user variable (instead of an already defined org
> variable) that could be toggled per document.
>
> It is fairly orthogonal to the original purpose of the patch, but it
> does scratch an annoying itch of mine: there's very little support in
> org to minutely tweak the export on a per document basis, beyond the
> default options.

Defining a new option is easy:

(org-export-define-derived-backend 'my-html 'html
  :options-alist
  '((:my-option "MY_OPTION_KEYWORD" nil "my-option" my-option-variable)))

Then, you can either

#+:BIND: my-option-variable t
or
#+OPTIONS: my-option:t
or
#+MY_OPTION_KEYWORD: t

Of course, you will also need to plug support of :my-option as needed
via filters or altering transcoders.

Timothy, since you are going to provide a macro for altering the
existing backends, it would also make sense to generalize it for
altering other aspects of the backends, like option-alist,
filters-alist, and menu-entry.

> One way to do this (easier than setting up a new option), is to define
> any variable, make the feature depend on it, and set it using the
> `#+bind` keyword. Combined with these export features, this could be
> used to easily tweak the LaTeX preamble, per document.
>
> If we find this use case to be legitimate and useful, I suggested
> earlier making it even easier with a `#+org_export_features` keyword.
> It would only take a list of feature names, and not require any
> variable tied to the feature.

I do not see much point compared to option alist, which is more
flexible. And you can always define a feature with condition to match
against "#\\+org_export_features.+my-feature".

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] Introduce "export features"

[Sébastien Miquel]

Ihor Radchenko writes:

The traditional user-facing approach for toggling staff in export is
setting export options.

Indeed. Then I suggest that such use be described in the manual.
Having the user create a whole new option in order to toggle a
`chikenize` package seems a bit unwieldy, however.

What do you suggest instead of creating a new option?


Timothy's patch supports having a feature depend on a variable. I was
thinking here of a user variable (instead of an already defined org
variable) that could be toggled per document.

It is fairly orthogonal to the original purpose of the patch, but it
does scratch an annoying itch of mine: there's very little support in
org to minutely tweak the export on a per document basis, beyond the
default options.

One way to do this (easier than setting up a new option), is to define
any variable, make the feature depend on it, and set it using the
`#+bind` keyword. Combined with these export features, this could be
used to easily tweak the LaTeX preamble, per document.

If we find this use case to be legitimate and useful, I suggested
earlier making it even easier with a `#+org_export_features` keyword.
It would only take a list of feature names, and not require any
variable tied to the feature.

--
Sébastien Miquel

Re: [PATCH] Introduce "export features"

[Ihor Radchenko]

Sébastien Miquel  writes:

>> The traditional user-facing approach for toggling staff in export is
>> setting export options.
>
> Indeed. Then I suggest that such use be described in the manual.
> Having the user create a whole new option in order to toggle a
> `chikenize` package seems a bit unwieldy, however.

What do you suggest instead of creating a new option?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] Introduce "export features"

[Timothy]

Hi Sebastien,

> I think it would be useful to support an easy way to toggle a feature
> on and off. Either the manual should describe the best way to make a
> feature depend on a user variable (this requires using the #+BIND
> keyword, I guess), or I’d like org to support a new keyword, such as
>  : #+org_export_features
> that would take a list of features, and enable them. Here, I am
> thinking of features such as the `chikenize` example in the manual.

A large part of the idea here is that these “export features” will automatically
 be used when relevant, without having to be manually enabled/disabled.

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .

Re: [PATCH] Introduce "export features"

[Sébastien Miquel]

Ihor Radchenko writes:

I think it would be useful to support an easy way to toggle a feature
on and off. Either the manual should describe the best way to make a
feature depend on a user variable (this requires using the #+BIND
keyword, I guess), or I'd like org to support a new keyword, such as
   : #+org_export_features
that would take a list of features, and enable them. Here, I am
thinking of features such as the `chikenize` example in the manual.

The traditional user-facing approach for toggling staff in export is
setting export options.


Indeed. Then I suggest that such use be described in the manual.
Having the user create a whole new option in order to toggle a
`chikenize` package seems a bit unwieldy, however.

--
Sébastien Miquel

Re: [PATCH] Introduce "export features"

[Ihor Radchenko]

Sébastien Miquel  writes:

> I think it would be useful to support an easy way to toggle a feature
> on and off. Either the manual should describe the best way to make a
> feature depend on a user variable (this requires using the #+BIND
> keyword, I guess), or I'd like org to support a new keyword, such as
>   : #+org_export_features
> that would take a list of features, and enable them. Here, I am
> thinking of features such as the `chikenize` example in the manual.

The traditional user-facing approach for toggling staff in export is
setting export options.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] Introduce "export features"

[Sébastien Miquel]

Hi,

Timothy writes:

Notably, I’ve now got a draft manual entry (see the last patch attached), which
should go a long way to better explaining what this is without asking you to
wade through all the code comments 


Thank you for sharing your work Timothy, I've been using your export
features for a while.

I think it would be useful to support an easy way to toggle a feature
on and off. Either the manual should describe the best way to make a
feature depend on a user variable (this requires using the #+BIND
keyword, I guess), or I'd like org to support a new keyword, such as
 : #+org_export_features
that would take a list of features, and enable them. Here, I am
thinking of features such as the `chikenize` example in the manual.

I have not tested it, but after a quick look at the patch, enabled
features should be available to the export transcoder functions,
through their =info= argument, aren't they ? This seems very useful.

--
Sébastien Miquel

Re: [PATCH] Introduce "export features"

[Timothy]

Hi All,

Following some feedback I’ve received from a few people (including off-list),
here’s a v2 set of patches.

Notably, I’ve now got a draft manual entry (see the last patch attached), which
should go a long way to better explaining what this is without asking you to
wade through all the code comments :)

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .
>From a8ed768515e4cf305ba52566f372e096f8ed449a Mon Sep 17 00:00:00 2001
From: TEC 
Date: Sun, 19 Feb 2023 12:28:31 +0800
Subject: [PATCH 1/6] org-compat: Add ensure-list as org-ensure-list

* lisp/org-compat.el (org-ensure-list): Add `ensure-list' from Emacs 28,
as `org-ensure-list'.
---
 lisp/org-compat.el | 12 
 1 file changed, 12 insertions(+)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index fadb51df6..e11de3639 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -193,6 +193,18 @@   (defun org-format-prompt (prompt default  format-args)
 default)))
  ": ")))
 
+(if (fboundp 'ensure-list)
+(defalias 'org-ensure-list #'ensure-list)
+  (defun org-ensure-list (object)
+"Return OBJECT as a list.
+If OBJECT is already a list, return OBJECT itself.  If it's
+not a list, return a one-element list containing OBJECT.
+
+Compatability substitute for `ensure-list' in Emacs 28."
+(if (listp object)
+object
+  (list object
+
 
 ;;; Emacs < 27.1 compatibility
 
-- 
2.39.1

>From 3a1d9eeb4e77a15f1466bc2377838b38d97d5b22 Mon Sep 17 00:00:00 2001
From: TEC 
Date: Mon, 25 Jul 2022 23:37:13 +0800
Subject: [PATCH 2/6] ox: Introduce conditional/generated preamble

* lisp/ox.el (org-export-detect-features, org-export-expand-features,
org-export-generate-features-preamble): New functions for detecting
features and generating content based on them.
(org-export-conditional-features): Customisation for feature detection.
(org-export-as): Add detected to features to info in the slot :features.
(org-export-update-features): Add a convenience function for users to
edit the feature condition/implementation lists.
(org-export--annotate-info, org-export-detect-features,
org-export-define-derived-backend, org-export-define-backend,
org-export-conditional-features): Refactor backend feature
conditions/implementations into a struct field.  This allows for parent
inheritance to be properly managed, and leads into future work making
features more widely used in the export process.
(org-export-expand-features, org-export-resolve-feature-implementations,
org-export-generate-features-preamble,
org-export-expand-feature-snippets): The main functions for working with
export features.
(org-export-process-features, org-export-update-features): Introduce
`org-export-process-features' to simplify the application of features to
INFO.
---
 lisp/ox.el | 609 -
 1 file changed, 604 insertions(+), 5 deletions(-)

diff --git a/lisp/ox.el b/lisp/ox.el
index 7e4042bb8..7d9c5eb26 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -1030,7 +1030,7 @@ ;;; Defining Back-ends
 
 (cl-defstruct (org-export-backend (:constructor org-export-create-backend)
   (:copier nil))
-  name parent transcoders options filters blocks menu)
+  name parent transcoders options filters blocks menu feature-conditions feature-implementations)
 
 ;;;###autoload
 (defun org-export-get-backend (name)
@@ -1136,6 +1136,62 @@ (defun org-export-get-all-filters (backend)
 	(setq filters (append filters (org-export-backend-filters backend
   filters)))
 
+(defvar org-export-conditional-features)
+
+(defun org-export-get-all-feature-conditions (backend)
+  "Return full feature condition alist for BACKEND.
+
+BACKEND is an export back-end, as return by, e.g,,
+`org-export-create-backend'.  Return value is an alist where keys
+are feature conditions, and values are feature symbols.
+
+Unlike `org-export-backend-feature-conditions', this function
+also returns conditions inherited from parent back-ends, if any."
+  (when (symbolp backend) (setq backend (org-export-get-backend backend)))
+  (and backend
+   (let ((conditions (org-export-backend-feature-conditions backend))
+ parent)
+ (while (setq parent (org-export-backend-parent backend))
+   (setq backend (org-export-get-backend parent))
+   (dolist (condition (org-export-backend-feature-conditions backend))
+ (push condition conditions)))
+ (dolist (condition org-export-conditional-features)
+   (unless (assq (car condition) conditions)
+ (push condition conditions)))
+ conditions)))
+
+(defun org-export-get-all-feature-implementations (backend)
+  "Return full feature implementation alist for BACKEND.
+
+BACKEND is an export back-end, as return by, e.g,,

Re: Re: [PATCH] Introduce "export features"

[Pedro Andres Aranda Gutierrez]

Hi,

Ihor wrote:
>> Timothy  writes:
>>
>> “export features” allow for the specification of qualities of the org
buffer
>> being exported that imply certain “features”, and how those features may
be
>> implemented in a particular export.

Looks like a cool "feature" that would bring some structure into
my chaotic configuration...

>>(defun org-export-update-features (backend  feature-settings)
>>  "Update FEATURE-SETTINGS for BACKEND.

>From what you explain, maybe "...-update-configs" may be a name that tells
me more

>Also, you will need to add ORG-NEWS entry and document the new feature
>system in "13.17 Advanced Export Configuration" and "A.4 Adding Export
>Back-ends" sections of the manual.

But I'd need ^that^ to judge better

Best, /PA

-- 
Fragen sind nicht da um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

Re: [PATCH] Introduce "export features"

[No Wayman]

often we include content in export templates that is only 
relevant in

particular situations.

“export features” allow for the specification of qualities of 
the org buffer
being exported that imply certain “features”, and how those 
features may be

implemented in a particular export.


now `\usepackage{svg}' is automatically added when exporting a 
buffer that

includes SVG images.



┌
│ (org-export-update-features 'beamer
│   (beamer-metropolis
│:condition (string-match-p \"metropolis$\" (plist-get info 
:beamer-theme))

│:snippet my-org-beamer-metropolis-tweaks
│:order 3))
└


As discussed on the Doom Emacs Discord, I strongly suggest 
renaming to avoid overloading the term "feature".
To me, your description indicates sinppets are inserted 
contextually.
Org has a similar notion of contexts with 
`org-capture-templates-contexts'.
A similar name would be better as it does not overload the term 
"feature", which already has a different meaning in elisp.

e.g. org-export-backend-contexts

Re: [PATCH] Introduce "export features"

[Ihor Radchenko]

Timothy  writes:

> “export features” allow for the specification of qualities of the org buffer
> being exported that imply certain “features”, and how those features may be
> implemented in a particular export.
>
> This is done by augmenting the backend struct with two new fields:
> `feature-conditions' and `feature-implementations'.
>
> The feature conditions are resolved during the annotation of `info', in the 
> Org
> buffer after `#+include' expansion and the removal of comments.
>
> The feature implementations are expanded by the backend itself, in the case of
> `ox-latex' this currently means during preamble construction.

I am in favour of this.

Some comments on the code are below.

> +  `(("^[ \t]*#\\+print_bibliography:" . bibliography)

This will also match bibliography statements, which are not keywords.
For example, inside quote blocks.

> +  :group 'org-export-general
> +  :type '(alist :key-type
> +(choice (regexp :tag "Feature test regexp")
> +(variable :tag "Feature variable")
> +(function :tag "Feature test function"))
> +:value-type
> +(choice (symbol :tag "Feature symbol")
> +(repeat symbol :tag "Feature symbols"

:package-version is missing in this defcustom. It should be added.

> +(defun org-export-detect-features (info)
> +  "Detect features from `org-export-conditional-features' in INFO."
> +  (let (case-fold-search)

This unconditionally sets case-sensitive search. Thus, for example,
#+PRINT_BIBLIOGRAPHY (as opposed to #+print_bibliography) will not be
recognized as 'bibliography feature.

> +(delete-dups
> + (mapcan
> +  (lambda (construct-feature)

This lambda could be a private function instead. A function would be
byte-compiled.

>  (defcustom org-latex-default-packages-alist
>'(("AUTO" "inputenc"  t ("pdflatex"))
>  ("T1"   "fontenc"   t ("pdflatex"))
> -("" "graphicx"  t)

You need to update :package-version upon changing the defcustom value.

> + Generated preamble
> +
> +(defcustom org-latex-conditional-features

defcustom :keywords are missing here. Please, add.

> +(,(lambda (info)
> +(org-element-map (plist-get info :parse-tree)
> +'link
> +  (lambda (link)
> +(and (member (org-element-property :type link)
> + '("http" "https" "ftp" "file"))
> + (file-name-extension (org-element-property :path link))
> + (equal (downcase (file-name-extension
> +   (org-element-property :path link)))
> +"svg")))
> +  info t))
> + . svg)

This is a duplicate of an entry in `org-export-conditional-features'. Is
it intentional?

> +(defcustom org-latex-feature-implementations

:package-version is missing.

> +  "Alist describing how export features should be supported in the preamble.
> +
> +Implementation alist has the feature symbol as the car, with the
> +cdr forming a plist with the following keys:
> +- :snippet, which is either,
> +  - A string, which should be included in the preamble verbatim.
> +  - A variable, the value of which should be included in the preamble.
> +  - A function, which is called with two arguments — the export info,
> +and the list of feature flags. The returned value is included in
> +the preamble.
> +- :requires, a feature or list of features which are needed.
> +- :when, a feature or list of features which imply this feature.
> +- :prevents, a feature or list of features that should be masked.
> +- :order, for when inclusion order matters. Feature implementations
> +  with a lower order appear first.  The default is 0."
> +  :group 'org-export-general

What is this group?

> +  :type '(plist :key-type
> +  (choice (const :snippet)
> +  (const :requires)
> +  (const :when)
> +  (const :prevents)
> +  (const :order)
> +  (const :trigger))
> +  :value-type
> +  (choice (string :tag "Verbatim content")
> +  (variable :tag "Content variable")
> +  (function :tag "Generating function"

The docstring and :type are rather generic wrt backend. Can they be
abstracted away?

> -(defun org-latex-guess-inputenc (header)
> +(defun org-latex-guess-inputenc (info)

It is a breaking change. Can we make something to not break existing
third-party code?

> -(defun org-latex-guess-babel-language (header info)
> +(defun org-latex-guess-babel-language (info)

Again, backwards-incompatible.

> -(defun org-latex-guess-polyglossia-language (header info)
> +(defun org-latex-guess-polyglossia-language (info)

backwards-incompatible

> +(concat
> + ;; Time-stamp.
> + (and (plist-get info :time-stamp-file)
> +  (format-time-string "%% Created %Y-%m-%d %a %H:%M\n"))

May it also be a feature?

> + 

[PATCH] Introduce "export features"

[Timothy]

Hello everyone,

I’m thrilled to finally be presenting a feature that I’ve been incubating for a
while now that I call “export features”. This work is based on the observation
that often we include content in export templates that is only relevant in
particular situations.

This leaves one having to choose between a “kitchen sink” approach where
everything that could be used is all included, or a “manual inclusion” approach
where the template is minimal and the relevant setup code must be manually
included each time.

I think it’s fair to say neither situation is ideal, and I’ve become
dissatisfied enough that I’ve sunk some time into working on a better way of
handling this. In this patch set it’s just being applied to LaTeX preambles, but
this is just a start — there are plans to apply this more broadly.

“export features” allow for the specification of qualities of the org buffer
being exported that imply certain “features”, and how those features may be
implemented in a particular export.

This is done by augmenting the backend struct with two new fields:
`feature-conditions' and `feature-implementations'.

The feature conditions are resolved during the annotation of `info', in the Org
buffer after `#+include' expansion and the removal of comments.

The feature implementations are expanded by the backend itself, in the case of
`ox-latex' this currently means during preamble construction.

With this change, `ox-latex' produces more minimal /and/ more capable exports 
out of
the box. The number of default packages has been ~halved, but OOTB capability
has been improved by dynamically adding them to the preamble when needed, and
now `\usepackage{svg}' is automatically added when exporting a buffer that
includes SVG images.

This also opens new frontiers for user customisation. For instance, adding
particular beamer customisations when using the `metropolis' beamer theme with 
the
following snippet:

┌
│ (org-export-update-features 'beamer
│   (beamer-metropolis
│:condition (string-match-p \"metropolis$\" (plist-get info :beamer-theme))
│:snippet my-org-beamer-metropolis-tweaks
│:order 3))
└


Hopefully this gives you an idea of the feature. See the patches attached for
the implementation (and some hopefully informative code comments). Feel free to
toss any you may have question my way :)

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .
>From 5575a0f18277ef34f4003c1bccf650e4237e6048 Mon Sep 17 00:00:00 2001
From: TEC 
Date: Mon, 25 Jul 2022 23:37:13 +0800
Subject: [PATCH 1/6] ox: Introduce conditional/generated preamble

* lisp/ox.el (org-export-detect-features, org-export-expand-features,
org-export-generate-features-preamble): New functions for detecting
features and generating content based on them.
* lisp/ox.el (org-export-conditional-features): Customisation for
feature detection.
* lisp/ox.el (org-export-as): Add detected to features to info in the
slot :features.
---
 lisp/ox.el | 217 +
 1 file changed, 217 insertions(+)

diff --git a/lisp/ox.el b/lisp/ox.el
index 0a48e850a..1a75ed28d 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -2030,6 +2030,221 @@ (defun org-export-expand (blob contents  with-affiliated)
 	(funcall (intern (format "org-element-%s-interpreter" type))
 		 blob contents
 
+
+;;; Conditional/Generated Preamble
+;;
+;; Many formats have some version of a preamble, whether it be HTML's
+;; ... or the content before LaTeX's \begin{document}.
+;; Depending on the particular features in the Org document being
+;; exported, different setup snippets will be needed.  There's the
+;; "everything and the kitchen sink" approach of adding absolutely
+;; everything that might be needed, and the post-translation editing
+;; with filters approach, but neither really solve this problem nicely.
+;;
+;; The conditional/generated preamble defines mechanisms of detecting
+;; which "features" are used in a document, handles interactions
+;; between features, and provides/generates preamble content to
+;; support the features.
+
+(defcustom org-export-conditional-features
+  `(("^[ \t]*#\\+print_bibliography:" . bibliography)
+(,(lambda (info)
+   (org-element-map (plist-get info :parse-tree)
+   'link
+ (lambda (link)
+   (and (member (org-element-property :type link)
+'("http" "https" "ftp" "file"))
+(file-name-extension (org-element-property :path link))
+(string= (downcase (file-name-extension
+(org-element-property :path link)))
+ "svg")))
+ info t))
+ . svg)
+(,(lambda (info)
+   (org-element-map (plist-get info :parse-tree)
+   'link
+ (lambda