Ihor Radchenko <yanta...@posteo.net> writes: > I'd like to add a new section to Org mode manual. > The new section will describe all the steps performed by Org export > process. This should hopefully create a more clear picture on how > various export hooks and filters are used. > > The patch is attached. > I'd appreciate feedback from people not familiar with ox.el.
Thank you all for the feedback! I am attaching revised version of the patch with most of the comments addressed. I will put more detailed responses inline. "Thomas S. Dye" <tsd@tsdye.online> writes: > I'm not too familiar with ox.el. > > I edited mostly to use an active voice. I put author queries in > parentheses. I haven't paid attention to manual formatting > conventions. > > IMO, more links would likely be helpful. > > * Suggested revision > ... I believe that I have addressed everything you commented on. Karthik Chikmagalur <karthikchikmaga...@gmail.com> writes: > - When exporting a sub-tree, at what stage of the export process is the > buffer narrowed to the sub-tree? I added a clarification on subtree export now. > - Are "inner" and "outer" templates described in the manual, and if they > are could you add a link to those sections when mentioning them in > this summary? I only found references to the plist properties > BEAMER_INNER_THEME etc. This is internal terminology. I changed the wording, expanding on what inner and outer template do. Matt <m...@excalamus.com> writes: > Here are all the hooks and functions for org-export (via =C-h v > org-export--hooks TAB= and =C-h v org-export--function TAB=). I see > 59 of them. > ... > * Feedback 1: > How are the functions not present in the patch handled? - I fixed the obsolete variable names. - `org-export-stack-mode-hook' is not directly relevant to the export process - it is for asynchronous export listing buffer - Syntax-specific filters are applied according to the corresponding Org syntax element. I tried to make it more clear. - Special filters, like `org-export-before-parsing-functions' are described separately. I think I have mentioned all of them. > I would write out "src" as "source". Do we have an official way to > refer to source blocks? For example, we standardize on "Org": > https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/doc/Documentation_Standards.org#n47 We use "source block", "code block", and "source code block" across the manual. Not "src block" though. I went with "code block" in the patch. I am not sure if it is necessary to standardize the above three terms. They are all equally understandable I believe. > Remove the trailing period or add periods to all the others. I tend > leave the period of the last sentence of a list. I'm not sure of a > style guide that recommends one or the other. Maybe someone knows > what's "right"? No idea. We do not have a consistency in the manual either. I went with ";" as Thomas suggested. > Maybe use "converted" instead of "transcoded"? I'm a native speaker > but I wonder if "converted" is a simpler word for people who aren't. "transcoded" is closer to what we use in the code. I tried to use "converted" in more general description, but still hint on the internal terminology.
>From efee8fb5e8aca473b1b80aacc2b38951421225cc Mon Sep 17 00:00:00 2001 Message-ID: <efee8fb5e8aca473b1b80aacc2b38951421225cc.1703683799.git.yanta...@posteo.net> From: Ihor Radchenko <yanta...@posteo.net> Date: Wed, 27 Dec 2023 14:23:29 +0100 Subject: [PATCH v2 1/2] doc/org-manual.org: Fix some obsolete variable names * doc/org-manual.org (Export hooks): Use the new `org-export-before-processing-functions' and `org-export-before-parsing-functions' instead of their obsolete aliases. --- doc/org-manual.org | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 23f250fa7..b35a83434 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16397,12 +16397,14 @@ *** Export hooks :END: #+vindex: org-export-before-processing-hook +#+vindex: org-export-before-processing-functions #+vindex: org-export-before-parsing-hook The export process executes two hooks before the actual exporting -begins. The first hook, ~org-export-before-processing-hook~, runs -before any expansions of macros, Babel code, and include keywords in -the buffer. The second hook, ~org-export-before-parsing-hook~, runs -before the buffer is parsed. +begins. The first hook, ~org-export-before-processing-functions~, +runs before any expansions of macros, Babel code, and include keywords +in the buffer. The second hook, +~org-export-before-parsing-functions~, runs before the buffer is +parsed. Functions added to these hooks are called with a single argument: the export backend actually used, as a symbol. You may use them for @@ -16421,7 +16423,7 @@ *** Export hooks ;; the docstring of `org-map-entries' for details. (setq org-map-continue-from (point))))) -(add-hook 'org-export-before-parsing-hook #'my-headline-removal) +(add-hook 'org-export-before-parsing-functions #'my-headline-removal) #+end_src *** Filters -- 2.42.0
>From 704eda63d6abf847efcd79cd97e3560ba4729921 Mon Sep 17 00:00:00 2001 Message-ID: <704eda63d6abf847efcd79cd97e3560ba4729921.1703683799.git.yanta...@posteo.net> In-Reply-To: <efee8fb5e8aca473b1b80aacc2b38951421225cc.1703683799.git.yanta...@posteo.net> References: <efee8fb5e8aca473b1b80aacc2b38951421225cc.1703683799.git.yanta...@posteo.net> From: Ihor Radchenko <yanta...@posteo.net> Date: Tue, 26 Dec 2023 15:15:23 +0100 Subject: [PATCH v2 2/2] doc/org-manual.org: Describe export flow * doc/org-manual.org (Summary of the export process): Explain how the export process is handled in Org mode. --- doc/org-manual.org | 149 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 149 insertions(+) diff --git a/doc/org-manual.org b/doc/org-manual.org index b35a83434..12e7d031a 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16505,6 +16505,155 @@ *** Defining filters for individual files ,#+END_SRC #+end_example +*** Summary of the export process +:PROPERTIES: +:UNNUMBERED: notoc +:END: + +Org mode export is a multi-step process that works on a temporary copy +of the buffer. On high-level, the export process consists of 4 major +steps: + +1. Process the temporary copy, making necessary changes to the buffer + text; + +2. Parse the buffer, converting plain Org markup into abstract syntax + tree (AST); + +3. Convert the AST to text, as prescribed by the selected export + backend; + +4. Post-process the resulting exported text. + + +#+texinfo: @noindent +Process temporary copy of the source Org buffer [fn::Unless +otherwise specified, each step of the export process only operates on +the accessible portion of the buffer. When subtree export is selected +(see [[*The Export Dispatcher]]), the buffer is narrowed to the body of +the selected subtree, so that the rest of the buffer text, except +export keywords, does not contribute to the export output.]: + +1. Execute ~org-export-before-processing-functions~ (see [[*Export hooks]]); + +2. Expand =#+include= keywords in the whole buffer (see + [[*Include Files]]); + +3. Remove commented subtrees in the whole buffer (see [[*Comment + Lines]]); + +4. Replace macros in the whole buffer (see [[*Macro Replacement]]); + +5. When ~org-export-use-babel~ is non-nil (default), process code + blocks: + + - Leave code blocks inside archived subtrees (see [[*Internal + archiving]]) as is; + + - Evaluate all the other code blocks according to code block + headers (see [[*Limit code block evaluation]]); + + - Remove code, results of evaluation, both, or neither according + to =:exports= header argument (see [[*Exporting Code Blocks]]). + + +#+texinfo: @noindent +Parse the temporary buffer, creating AST: + +1. Execute ~org-export-before-parsing-functions~ (see [[*Export hooks]]). + The hook functions may still modify the buffer; + +2. Calculate export option values according to subtree-specific export + settings, in-buffer keywords, =#+BIND= keywords, and buffer-local + and global customization. The whole buffer is considered; + +3. Determine contributing bibliographies and record them into export + options (see [[*Citations]]). The whole buffer is considered; + +4. Execute ~org-export-filter-options-functions~; + +5. Parse the accessible portion of the temporary buffer to generate + AST. The AST is a nested list of lists representing Org syntax + elements (see [[https://orgmode.org/worg/dev/org-element-api.html][Org Element API]] for more details): + + : (org-data ... + : (heading + : (section + : (paragraph (plain-text) (bold (plain-text)))) + : (heading) + : (heading (section ...)))) + + Past this point, modifications in the temporary buffer copy no + longer affect export; Org export works only with the AST; + +6. Remove elements that will not be exported from the AST: + + - Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= export + keywords, and =task=, =inline=, =arch= export options (see + [[*Export Settings]]); + + - Comments; + + - Clocks, drawers, fixed-width environments, footnotes, LaTeX + environments and fragments, node properties, planning lines, + property drawers, statistics cookies, timestamps, timestamps, + etc according to =#+OPTIONS= keyword (see [[*Export Settings]]); + + - Table rows containing width and alignment markers (see [[*Column + Width and Alignment]]); + + - Table columns containing recalc marks (see [[*Advanced features]]). + +7. Expand environment variables in file link AST nodes, according to + the =expand-links= export option (see [[*Export Settings]]); + +8. Execute ~org-export-filter-parse-tree-functions~. These + functions can modify AST by side effect; + +9. Replace citation AST nodes and =#+print_bibliography= keyword AST + nodes as prescribed by the selected citation export processor + (see [[*Citation export processors]]). + + +#+texinfo: @noindent +Convert the AST to text by traversing the AST nodes, depth-first: + +1. Convert the leaf nodes (without children) to text as prescribed + by "transcoders" in the selected export backend + [fn:: See transcoders and ~:translate-alist~ in the docstrings + of ~org-export-define-backend~ and ~org-export-define-derived-backend~.]; + +2. Pass the converted nodes through the corresponding export + filters (see [[*Filters]]); + +3. Concatenate all the converted child nodes to produce parent + node contents; + +4. Convert the nodes with children to text, passing the nodes + themselves and their contents to the corresponding transcoders + and then to export filters (see [[*Filters]]). + + +#+texinfo: @noindent +Post-process the exported text: + + 1. Post-process the converted AST, as prescribed by the export + backend. [fn:: See ~inner-template~ in the docstring of ~org-export-define-backend~.] + This step usually adds generated content (like Table of Contents) + to the exported text; + + 2. Execute ~org-export-filter-body-functions~; + + 3. Unless body-only export is selected (see [[*The Export Dispatcher]]), + add the necessary metadata to the final document, as prescribed + by the export backend. Examples: Document author/title; HTML + headers/footers; LaTeX preamble; + + 4. Add bibliography metadata, as prescribed by the citation export + processor; + + 5. Execute ~org-export-filter-final-output-functions~. + *** Extending an existing backend :PROPERTIES: :UNNUMBERED: notoc -- 2.42.0
-- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>