Re: [PATCH] org-manual: Describe export process flow

2023-12-26 Thread Matt 


  On Tue, 26 Dec 2023 22:56:00 +0100  Matt  wrote --- 
 
 > I tend leave the period of the last sentence of a list.

Typo.  I meant to write "I tend TO leave the period OFF the last sentence of a 
list."  That is, the item ends without a period.

--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode

Re: [PATCH] org-manual: Describe export process flow

2023-12-26 Thread Matt 
This looks useful.

My only understanding of the problem this addresses is what you wrote:

 > create a more clear picture on how various export hooks and filters
 > are used.

With this understanding, my first question was, "What hooks exist?"  I
see that 
[[https://www.gnu.org/software/emacs/manual/html_node/org/Advanced-Export-Configuration.html][Advanced
 Export Configuration]] gives only
~org-export-before-processing-hook~ and
~org-export-before-parsing-hook~.  The section below it gives a table
of filters.

Here are all the hooks and functions for org-export (via =C-h v
org-export--hooks TAB= and =C-h v org-export--function TAB=).  I see
59 of them.

| Hooks   | Manual (60f357e8b) | Patch |
|-++---|
| org-export-before-processing-hook (obsolete)| X  | X |
| org-export-before-parsing-hook  | X  | X |
| org-export-stack-mode-hook  ||   |
| org-export-filter-options-functions | X  | X |
| org-export-filter-parse-tree-functions  | X  | X |
| org-export-filter-body-functions| X  | X |
| org-export-filter-final-output-functions| X  | X |
| org-export-before-parsing-functions ||   |
| org-export-before-processing-functions  ||   |
| org-export-filter-babel-call-functions  | X  |   |
| org-export-filter-bold-functions| X  |   |
| org-export-filter-center-block-functions| X  |   |
| org-export-filter-clock-functions   | X  |   |
| org-export-filter-code-functions| X  |   |
| org-export-filter-diary-sexp-functions  | X  |   |
| org-export-filter-drawer-functions  | X  |   |
| org-export-filter-dynamic-block-functions   | X  |   |
| org-export-filter-entity-functions  | X  |   |
| org-export-filter-example-block-functions   | X  |   |
| org-export-filter-export-block-functions| X  |   |
| org-export-filter-export-snippet-functions  | X  |   |
| org-export-filter-fixed-width-functions | X  |   |
| org-export-filter-footnote-definition-functions | X  |   |
| org-export-filter-footnote-reference-functions  | X  |   |
| org-export-filter-headline-functions| X  |   |
| org-export-filter-horizontal-rule-functions | X  |   |
| org-export-filter-inline-babel-call-functions   | X  |   |
| org-export-filter-inline-src-block-functions| X  |   |
| org-export-filter-inlinetask-functions  | X  |   |
| org-export-filter-italic-functions  | X  |   |
| org-export-filter-item-functions| X  |   |
| org-export-filter-keyword-functions | X  |   |
| org-export-filter-latex-environment-functions   | X  |   |
| org-export-filter-latex-fragment-functions  | X  |   |
| org-export-filter-line-break-functions  | X  |   |
| org-export-filter-link-functions| X  |   |
| org-export-filter-node-property-functions   | X  |   |
| org-export-filter-paragraph-functions   | X  |   |
| org-export-filter-plain-list-functions  | X  |   |
| org-export-filter-plain-text-functions  | X  |   |
| org-export-filter-planning-functions| X  |   |
| org-export-filter-property-drawer-functions | X  |   |
| org-export-filter-quote-block-functions | X  |   |
| org-export-filter-radio-target-functions| X  |   |
| org-export-filter-section-functions | X  |   |
| org-export-filter-special-block-functions   | X  |   |
| org-export-filter-src-block-functions   | X  |   |
| org-export-filter-statistics-cookie-functions   | X  |   |
| org-export-filter-strike-through-functions  | X  |   |
| org-export-filter-subscript-functions   | X  |   |
| org-export-filter-superscript-functions | X  |   |
| org-export-filter-table-cell-functions

Re: [PATCH] org-manual: Describe export process flow

2023-12-26 Thread Karthik Chikmagalur 
> The patch is attached.
> I'd appreciate feedback from people not familiar with ox.el.

- When exporting a sub-tree, at what stage of the export process is the
  buffer narrowed to the sub-tree?
- Are "inner" and "outer" templates described in the manual, and if they
  are could you add a link to those sections when mentioning them in
  this summary?  I only found references to the plist properties
  BEAMER_INNER_THEME etc.

Karthik

Re: [PATCH] org-manual: Describe export process flow

2023-12-26 Thread Thomas S. Dye 

Aloha Ihor,

Ihor Radchenko  writes:


Hi,

I'd like to add a new section to Org mode manual.
The new section will describe all the steps performed by Org 
export
process. This should hopefully create a more clear picture on 
how

various export hooks and filters are used.

The patch is attached.
I'd appreciate feedback from people not familiar with ox.el.


I'm not too familiar with ox.el.

I edited mostly to use an active voice. I put author queries in 
parentheses.  I haven't paid attention to manual formatting 
conventions.


IMO, more links would likely be helpful.

* Suggested revision

Org mode export is a multi-step process that works on a temporary 
copy of the buffer as follows:
   1. Execute ~org-export-before-processing-hook~ (see 
   [[*Hooks]]);

   2. Expand =#+include= keywords;
   3. Remove commented headings; (author: clarify whether the 
   body is also removed?)

   4. Replace macros;
   5. Export source code blocks, conditional on 
   ~org-export-use-babel~ and, if
  necessary, the individual ~exports~ header arguments (see 
  [[*Exporting Code Blocks]]);
   6. Evaluate source code blocks (see [[*Evaluating Code 
   Blocks]]);

   7. Execute ~org-export-before-parsing-hook~ (see [[*Hooks]]);
   8. Calculate export option values according to in-buffer 
   keywords,
  =#+BIND= keywords, and buffer-local and global 
  customizations;
   9. Calculate (author: Determine instead of Calculate?) 
   bibliography file paths;

   10. Execute ~org-export-filter-options-functions~;
   11. Parse the temporary (author: is this correct?) buffer to 
   generate an

   abstract syntax tree (AST);
   12. Remove elements that will not be exported from the AST:
   1. Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= 
   export

  keywords, and =task=, =inline=, =arch= export options;
   2. Comments;
   3. Clocks, drawers, fixed-width environments, footnotes, 
   LaTeX environments and fragments, node properties, 
   planning lines, property drawers, statistics cookies, and 
   timestamps according to their corresponding export 
   options;
   4. Table rows containing [[*Column Width and 
   Alignment][width and alignment markers]];
   5. Table columns containing [[*Advanced features][recalc 
   marks]];
   13. Expand environment variables in file links according to 
   the =expand-links= export option (author: are we still 
   operating on AST here?);
   14. Execute ~org-export-filter-parse-tree-functions~ (author: 
   ditto);
   15. Create bibliography listing(s) from citation(s) (author: 
   ditto);
   16. Replace =+print_bibliography= keyword(s) with bibliography 
   listing(s) (author: ditto);
   17. Transcode the AST according to export backend using 
   recursive, depth-first search, and passing each transcoded 
   node as a string to the export filter (see [[*Filters]]):
   18. Format transcoded AST according to the backend's "inner" 
   template;
   19. Execute ~org-export-filter-body-functions~ on the 
   transcoded and formatted AST;
   20. Format the result according to the backend's "outer" 
   template;
   21. Finalize bibliography listing(s) (author: correct?) and 
   citation(s); and

   22. Execute ~org-export-filter-final-output-functions~.

Let me know if you have questions.

All the best,
Tom
--
Thomas S. Dye
https://tsdye.online/tsdye

[PATCH] org-manual: Describe export process flow

2023-12-26 Thread Ihor Radchenko 
Hi,

I'd like to add a new section to Org mode manual.
The new section will describe all the steps performed by Org export
process. This should hopefully create a more clear picture on how
various export hooks and filters are used.

The patch is attached.
I'd appreciate feedback from people not familiar with ox.el.

>From 57d290b710c85e0411855937e224d2cecdbd52da Mon Sep 17 00:00:00 2001
Message-ID: <57d290b710c85e0411855937e224d2cecdbd52da.1703600243.git.yanta...@posteo.net>
From: Ihor Radchenko 
Date: Tue, 26 Dec 2023 15:15:23 +0100
Subject: [PATCH] doc/org-manual.org: Describe export flow

* doc/org-manual.org (Summary of the export process): Explain how the
export process is handled in Org mode.
---
 doc/org-manual.org | 54 ++
 1 file changed, 54 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 7db69cbb6..1a9432b4d 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -16391,6 +16391,60 @@ ** Advanced Export Configuration
 :DESCRIPTION: Fine-tuning the export output.
 :END:
 
+*** Summary of the export process
+
+During export, Org mode processes the source Org buffer in multiple
+steps:
+
+1. The source Org mode buffer is copied into temporary throwaway
+   buffer that can be edited by export hooks
+2. ~org-export-before-processing-hook~ is executed (see [[*Export hooks]])
+3. All the =#+include= keywords are expanded
+4. All the commented headings are removed
+5. All the macros are replaced in buffer
+6. When ~org-export-use-babel~ is non-nil (default), all the src
+   blocks and babel calls that are not inside archived headings are
+   processed
+7. ~org-export-before-parsing-hook~ is executed (see [[*Export hooks]])
+8. Export option values are calculated, according to in-buffer
+   keywords, =#+BIND= keywords, buffer-local and global
+   customizations.
+9. Files contributing to bibliography are calculated
+10. ~org-export-filter-options-functions~ is executed
+11. The buffer is parsed, generating abstract syntax tree (AST)
+12. The AST is cleaned from buffer elements that should not be
+exported:
+- Heading are removed according to =SELECT_TAGS= and
+  =EXCLUDE_TAGS= export keywords; =task=, =inline=, =arch= export
+  options
+- All the comments are removed
+- Clocks, drawers, fixed-width environments, footnotes, latex
+  environments and fragments, node properties, planning lines,
+  property drawers, statistics cookies, and timestamps are removed
+  or kept according to the corresponding export options
+- Table rows containing [[*Column Width and Alignment][width and alignment markers]] are removed
+- Table columns containing [[*Advanced features][recalc marks]] are removed
+13. Environment variables are expanded in all the file links when
+=expand-links= export option is set
+14. ~org-export-filter-parse-tree-functions~ is executed
+15. All the citations are processed according to the chosen citation
+backend
+16. =#+print_bibliography= keywords are replaced with bibliography
+listings
+17. AST is transcoded according to the chosen export backend
+- The export happens recursively, depth-first
+- Each transcoded AST node, as a string, is passed to the
+  corresponding export filter (see [[*Filters]])
+18. The transcoded AST body is formatted according to backend's
+"inner" template
+19. The resulting body is passed to
+~org-export-filter-body-functions~
+20. The filtered body is formatted according to backend's outer
+template
+21. The resulting output is processed by citation backend finalizer
+22. ~org-export-filter-final-output-functions~ are applied to the
+final output
+
 *** Export hooks
 :PROPERTIES:
 :UNNUMBERED: notoc
-- 
2.42.0



-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at