Ihor Radchenko <[email protected]> writes: > Rens Oliemans <[email protected]> writes: > >> Hmm, this is not exactly right. It seems that if a link is followed by a >> space (or tab) character, (org-element-property :end link) will be after >> that character. Consider this org file: >> [...] >> is this intentional or am I misunderstanding the properties? While not a >> big deal, it does makes the code a bit more complicated since I have to >> keep track of whether the link contains a trailing space or not. > > Yes, it is intentional. Blanks after each object, including links, is > considered a part of that object. The number of blanks is stored in > :post-blank property.
Cool, thanks. > Rather than having a special code for each link type, you may instead > consider altering the link object itself, followed by > org-element-interpret-data. Hmm, I took a slightly different approach: get the information from properties instead of regex; build strings with format rather than with org-element-interpret-data. See the attached patch, what do you think? Also, for each iteration I find myself squashing new changes onto the relevant commits; creating patches using magit; and attaching them. Two questions about this: 1) is this a/the recommended approach? 2) how do you review this? Do you apply the three commits onto master and review them as a whole? Or do you look at the difference between this patch and the previous iteration (and if so, how)? My mental model is still in the pull request style, which (also?) has some troubles with iterated changes - the GitHub UI makes it uncomfortable for the reviewer if the commits keep amending - and I wonder how you review with this approach. Best, Rens
>From aba2305fd3ff83689026e26b6cccdb966f9e6792 Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Tue, 25 Nov 2025 11:10:46 +0100 Subject: [PATCH 1/3] Find alternative links for YouTube, GitHub and Reddit This commit finds alternative links for the hosts mentioned above since they rely on non-free JS. See https://list.orgmode.org/orgmode/87pl9szmy6.fsf@localhost/ for context. --- .gitignore | 1 + org-contrib/babel/languages/ob-doc-vala.org | 2 +- publish.sh | 59 +++-- worg-urls-rewrite.el | 262 ++++++++++++++++++++ 4 files changed, 296 insertions(+), 28 deletions(-) create mode 100644 worg-urls-rewrite.el diff --git a/.gitignore b/.gitignore index c9dc5b62..c572b901 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,4 @@ patches/ /.project *patch +*.eld diff --git a/org-contrib/babel/languages/ob-doc-vala.org b/org-contrib/babel/languages/ob-doc-vala.org index 3c2f6520..9e1f1dfc 100644 --- a/org-contrib/babel/languages/ob-doc-vala.org +++ b/org-contrib/babel/languages/ob-doc-vala.org @@ -106,7 +106,7 @@ values become a table (see [[Commandline parameters][Commandline parameters]] ex ** Other *** Differences from other supported languages -There currently is no support for the [[:https://orgmode.org/manual/Environment-of-a-Code-Block.html#index-var_002c-header-argument][~:var~]] header. If you need to +There currently is no support for the [[https://orgmode.org/manual/Environment-of-a-Code-Block.html#index-var_002c-header-argument][~:var~]] header. If you need to pass parameters to your Vala code, you can only use ~:flags~ or ~:cmdline~ (see [[Vala-specific header arguments][Vala-specific header arguments]]). diff --git a/publish.sh b/publish.sh index b499de3d..bebe50c6 100755 --- a/publish.sh +++ b/publish.sh @@ -14,6 +14,8 @@ This variable can be set when running publish.sh script: (require 'htmlize) (require 'org-inlinetask) +(load-file "./worg-urls-rewrite.el") + (setq make-backup-files nil debug-on-error t) @@ -58,30 +60,33 @@ This variable can be set when running publish.sh script: (R . t) (gnuplot . t))) -(dolist (org-file (cl-remove-if - (lambda (n) (string-match-p "worg/archive/" n)) - (directory-files-recursively default-directory "\\.org$"))) - (let ((html-file (concat (file-name-directory org-file) - (file-name-base org-file) ".html"))) - (if (and (file-exists-p html-file) - (file-newer-than-file-p html-file org-file) - ;; If there are include files or code, we need to - ;; re-generate the HTML just in case if the included - ;; files are changed. - (with-temp-buffer - (insert-file-contents org-file) - (and - (save-excursion - (goto-char (point-min)) - (not (re-search-forward "#\\+include:" nil t))) - (save-excursion - (goto-char (point-min)) - (not (re-search-forward "#\\+begin_src" nil t)))))) - (message " [skipping] unchanged %s" org-file) - (message "[exporting] %s" (file-relative-name org-file default-directory)) - (with-current-buffer (find-file org-file) - (if worg-publish-stop-on-error - (org-html-export-to-html) - (condition-case err - (org-html-export-to-html) - (error (message (error-message-string err))))))))) +(add-hook 'org-export-before-parsing-functions #'worg-urls-add-alternative-links) +(let ((export-all (worg-urls-libredirect-data-changed-p worg-urls-libredirect-data))) + (dolist (org-file (cl-remove-if + (lambda (n) (string-match-p "worg/archive/" n)) + (directory-files-recursively default-directory "\\.org$"))) + (let ((html-file (concat (file-name-directory org-file) + (file-name-base org-file) ".html"))) + (if (and (not export-all) + (file-exists-p html-file) + (file-newer-than-file-p html-file org-file) + ;; If there are include files or code, we need to + ;; re-generate the HTML just in case if the included + ;; files are changed. + (with-temp-buffer + (insert-file-contents org-file) + (and + (save-excursion + (goto-char (point-min)) + (not (re-search-forward "#\\+include:" nil t))) + (save-excursion + (goto-char (point-min)) + (not (re-search-forward "#\\+begin_src" nil t)))))) + (message " [skipping] unchanged %s" org-file) + (message "[exporting] %s" (file-relative-name org-file default-directory)) + (with-current-buffer (find-file org-file) + (if worg-publish-stop-on-error + (org-html-export-to-html) + (condition-case err + (org-html-export-to-html) + (error (message (error-message-string err)))))))))) diff --git a/worg-urls-rewrite.el b/worg-urls-rewrite.el new file mode 100644 index 00000000..3ac09e1d --- /dev/null +++ b/worg-urls-rewrite.el @@ -0,0 +1,262 @@ +;;; worg-urls-rewrite.el --- Rewrite URLs that refer to pages containing non-free JS to free alternatives +;; Copyright (C) 2026 Rens Oliemans + +;; This program is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; This program is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with this program. If not, see <https://www.gnu.org/licenses/>. + +;;; Commentary: +;; +;; Worg links to many external pages, some of which link to websites +;; that rely on non-free JavaScript, such as GitHub or YouTube pages. +;; This script contains the function `worg-urls-add-alternative-links' +;; which converts these links to free alternatives, such as GotHub and +;; Invidious. +;; +;; Since these alternatives occasionally go down, we use the +;; alternative specified by libredirect, which checks whether these +;; alternatives are live or not. This is seen in +;; `worg-urls-libredirect-url'. +;; +;; This was discussed on the mailing list here: +;; https://list.orgmode.org/87pl9szmy6.fsf@localhost. +;; +;;; Code: + +(defvar worg-urls-libredirect-url "https://raw.githubusercontent.com/libredirect/instances/main/data.json"; + "Location where up-to-date libredirect data is found.") +(defvar worg-urls-libredirect-data nil + "Contents of libredirect data, obtained from `worg-urls-libredirect-url'.") + +(defcustom worg-urls-free-alternatives '((youtube . invidious) + (github . gothub)) + "Alist of alternative frontends to websites containing non-free JS. +The CDR corresponds to a symbol that is known by libredirect, see +https://codeberg.org/LibRedirect/instances.";) +(defcustom worg-urls-redirect-save-filename "redirect-local-data.eld" + "Filename used to save the relevant output of libredirect's data.") + + +(defun worg-urls-libredirect-data-changed-p (&optional data) + "Whether or not libredirect DATA changed since our last export." + (setq worg-urls-libredirect-data (or data (worg-urls--get-libredirect-data))) + (if (not (file-exists-p worg-urls-redirect-save-filename)) + ;; We hadn't saved stuff from our previous export, do so now. + (progn + (worg-urls--save-redirect-data worg-urls-libredirect-data) + t) + ;; Compare results from stored file with data, overwrite stored data if newer + (let* ((stored-data (with-temp-buffer + (insert-file-contents worg-urls-redirect-save-filename) + (read (buffer-string)))) + (data-equal (equal (worg-urls--relevant-redirect-data worg-urls-libredirect-data) + stored-data))) + (unless data-equal + (worg-urls--save-redirect-data worg-urls-libredirect-data)) + (not data-equal)))) + + +(defun worg-urls--get-libredirect-data () + "Return libredirect data from `worg-urls-libredirect-url'." + (condition-case _ + (let ((response (with-current-buffer + (url-retrieve-synchronously worg-urls-libredirect-url nil nil 5) + (prog2 + (re-search-forward "\n\n" nil t) ; skip HTTP headers + (buffer-substring-no-properties (point) (point-max)) + (kill-buffer))))) + (json-parse-string response :object-type 'alist :array-type 'list)) + (t + (message "Worg couldn't access libredirect data, using original URLs.") + nil))) + + +(defun worg-urls--save-redirect-data (&optional data) + "Save redirect DATA in a file with filename `worg-urls-redirect-save-filename'." + (let ((tosave (or data worg-urls-libredirect-data))) + (with-temp-file worg-urls-redirect-save-filename + (prin1 + (worg-urls--relevant-redirect-data data) + (current-buffer))))) + + +(defun worg-urls--relevant-redirect-data (data) + "Extract relevant redirection DATA, defined in `worg-urls-free-alternatives'." + (seq-map (lambda (association) + (let ((from (car association))) + `(,from . ,(worg-urls--first-link-of-data data from)))) + worg-urls-free-alternatives)) + +(defun worg-urls--is-short-gist-path-p (link) + "Test whether LINK links to a gist without a user id. +These redirect to a \"full\" gist.github.com/<user>/<id> link and at the +moment have to be fixed manually - gothub does not support these short +links." + (string-match (rx "//gist.github.com/" (1+ (in digit))) link)) + +(defun worg-urls-add-alternative-links (&optional _) + "Add alternative links for websites containing non-free JS. +For each link that has an alternative (currently YouTube and GitHub), we +insert a link to the free alternative, and change the link text of the +original link to to =(original URL)=. We also redirect reddit links to +old.reddit.com. + +See https://list.orgmode.org/orgmode/87pl9szmy6.fsf@localhost/"; + (unless worg-urls-libredirect-data + (setq worg-urls-libredirect-data (worg-urls--get-libredirect-data))) + + (let ((links (org-element-map (org-element-parse-buffer) 'link #'identity))) + (dolist (link (nreverse links)) + (when (string-prefix-p "http" (org-element-property :type link)) + (let ((path (org-element-property :path link))) + (if-let* ((new-url (worg-urls--find-replacement path))) + (let* ((begin (org-element-property :begin link)) + (contents-begin (org-element-property :contents-begin link)) + (contents-end (org-element-property :contents-end link))) + (save-excursion + (if (and contents-begin contents-end) + ;; Link has description. + (let ((description (buffer-substring contents-begin contents-end))) + ;; If this description refers to the full + ;; domain, this description will be confusing, + ;; since we change the domain here. Warn + ;; about this. + (when (string-match + (rx (or "reddit.com" + "github.com" + "youtube.com")) + description) + (error + (concat "Link description \"%s\" in %s:%s refers" + " to the full domain \"%s\", which we change." + " This is confusing, please change the description.") + description (buffer-file-name) + (line-number-at-pos (org-element-property :contents-begin link)) + (match-string 0 description))) + + ;; Change description to (original URL) + (goto-char contents-begin) + (delete-region contents-begin contents-end) + (insert "(original URL)") + ;; Insert new link with free alternative + (goto-char begin) + (insert (format "[[%s][%s]] " new-url description))) + ;; Else, "bare" link without description. We insert + ;; the original link as bare, and add the old one + ;; with (original URL) as description. + (let* ((end (org-element-property :end link)) + (replacement (format "[[%s]] [[%s][(original URL)]]%s" + new-url + (concat (org-element-property :type link) + ":" + (org-element-property :path link)) + (make-string (org-element-property :post-blank link) ? )))) + (delete-region begin end) + (goto-char begin) + (insert replacement))))) + ;; No replacement found. Check if the link is a short + ;; github gist, this can be fixed manually. + (when (worg-urls--is-short-gist-path-p path) + (error + (concat "Please replace URL %s in %s:%s with its redirect target," + " we cannot convert this short Gist URL to a free alternative.") + path + (buffer-file-name) + (line-number-at-pos (org-element-property :begin link)))))))))) + + +(defun worg-urls--find-replacement (path) + "Take PATH and return an alternative link if known and nil otherwise. +We redirect the following links: +- reddit.com -> old.reddit.com +- (youtube.com|youtu.be) -> alternative from libredirect +- github.com -> alternative from libredirect" + (pcase path + ;; See https://docs.invidious.io/redirector/ + ;; This matches both + ;; - youtube.com/watch?v=<video-id> + ;; - youtube.com/playlist?list=<playlist-id> + ;; with optionally ? and = escaped + ((rx "//" (? "www\.") + "youtube.com" + (group + (or (and "/watch" + (or "?" "%3F") + "v" + (or "=" "%3D")) + (and "/playlist" + (or "?" "%3F") + "list" + (or "=" "%3D")))) + (group (+ not-newline))) + (if-let* ((route (match-string 1 path)) + (id (match-string 2 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'youtube))) + (concat host route id))) + ;; youtu.be/<video-id> + ((rx "//" (? "www\.") + "youtu.be" + "/" + (group (+ not-newline))) + (if-let* ((video-id (match-string 1 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'youtube))) + (concat host "/watch?v=" video-id))) + ;; gothub supports all links currently used in Worg, except for + ;; /<user>/<repo>/<issues>, so we exclude those. + ((and + (rx "//" (? "www\.") + "github.com" + "/" + (group (+ not-newline))) + ;; Exclude issues path. This approach currently also would + ;; erroneously exclude a supported link if author or repo + ;; starts with "issues" + (guard (not (string-match-p "/issues" path)))) + (if-let* ((url (match-string 1 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'github))) + (concat host "/" url))) + ;; gist.github.com links can have the format + ;; gist.github.com/<user>/<id>, but also obsolete + ;; gist.github.com/<id>. These last links are automatically + ;; redirected on github.com, but we do not know where to + ;; redirect them to at this moment, so we cannot convert + ;; these. Therefore we currently only convert + ;; gist.github.com/<user>/<id> links. + ((rx "//" "gist.github.com/" + ;; user + (group (1+ (not (any "/$")))) + "/" + ;; id + (group (+ not-newline))) + (if-let* ((user (match-string 1 path)) + (id (match-string 2 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'github))) + (concat host "/gist/" user "/" id))) + ;; redirect reddit.com to old.reddit.com + ((rx "//" (? "www\.") + "reddit.com" + "/" + (group (+ not-newline))) + (let* ((url (match-string 1 path)) + (host "https://old.reddit.com";)) + (concat host "/" url))))) + +(defun worg-urls--first-link-of-data (data host) + "Get the first available link in DATA of the alternative links for HOST." + (let* ((alternative (cdr (assoc host worg-urls-free-alternatives))) + (object (cdr (assoc alternative data))) + (links (cdr (assoc 'clearnet object)))) + (car links))) + +(provide 'worg-urls-rewrite) + +;;; worg-urls-rewrite.el ends here -- 2.43.0
>From c1984254d49ccaeb8945502605a1e7a57b516d96 Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Tue, 20 Jan 2026 13:50:35 +0100 Subject: [PATCH 2/3] Expand short gist.github.com urls --- org-contrib/index.org | 2 +- org-hacks.org | 2 +- org-tools/index.org | 2 +- org-tutorials/non-beamer-presentations.org | 3 ++- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/org-contrib/index.org b/org-contrib/index.org index df6f2845..95027baf 100644 --- a/org-contrib/index.org +++ b/org-contrib/index.org @@ -430,7 +430,7 @@ See [[file:../exporters/index.org][Exporters]]. - [[https://github.com/eschulte/org-S5][org-S5]] from Eric Schulte - [[https://github.com/relevance/org-html-slideshow.git][org-html-slideshow]] from Stuart Sierra - - [[https://gist.github.com/509761][org-html5presentation.el]] from kinjo + - [[https://gist.github.com/takumikinjo/509761][org-html5presentation.el]] from kinjo For use with the new exporter, /ox-s5.el/ ([[contribfile:lisp/ox-s5.el][link to raw file)]] and /ox-deck.el/ ([[contribfile:lisp/ox-deck.el][link to raw file]]), both by Rick Frankel, are diff --git a/org-hacks.org b/org-hacks.org index cdff7e56..43992de8 100644 --- a/org-hacks.org +++ b/org-hacks.org @@ -67,7 +67,7 @@ Here is a hook function to use archive this effect: *** Picking up a random task in the global TODO list -Tony day [[https://list.orgmode.org/[email protected]][shared]] [[https://gist.github.com/4343164][this gist]] to pick up a +Tony day [[https://list.orgmode.org/[email protected]][shared]] [[https://gist.github.com/tonyday567/4343164][this gist]] to pick up a random task. ** Building and Managing Org diff --git a/org-tools/index.org b/org-tools/index.org index b2a35de3..74bb03f7 100644 --- a/org-tools/index.org +++ b/org-tools/index.org @@ -211,7 +211,7 @@ I have written yet another converter from planner to org: ** Export vCard to org-mode entries -[[https://list.orgmode.org/[email protected]][Simon Thum]] shared some [[https://gist.github.com/4145201][Ruby code]] to export vCards to Org-mode entries. +[[https://list.orgmode.org/[email protected]][Simon Thum]] shared some [[https://gist.github.com/simonthum/4145201][Ruby code]] to export vCards to Org-mode entries. ** ews-orgmode - Exchange calendar to orgmode diff --git a/org-tutorials/non-beamer-presentations.org b/org-tutorials/non-beamer-presentations.org index 33ea005e..e57e4a92 100644 --- a/org-tutorials/non-beamer-presentations.org +++ b/org-tutorials/non-beamer-presentations.org @@ -35,7 +35,8 @@ documents. - org-html5presentation :: Is an Exporter of Org-mode documents to HTML5 slide show presentations. - (see [[https://gist.github.com/509761][gist.github.com/509761]] for code and usage information) + (see [[https://gist.github.com/takumikinjo/509761][gist.github.com/takumikinjo/509761]] for code and usage + information) - [[org-tree-slide]] :: is a simple tool to treat a tree of an org buffer as a single slide. Since each slide is displayed by simple narrowing, -- 2.43.0
>From 5bfbe92c5d1d04c6a2fec0ed7a951e0a739110a1 Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Wed, 4 Feb 2026 12:40:48 +0100 Subject: [PATCH 3/3] Change references of full non-free domains in link descriptions --- index.org | 2 +- org-tests/index.org | 15 ++++++++------- org-tutorials/non-beamer-presentations.org | 5 ++--- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/index.org b/index.org index c61746dd..68711dd3 100644 --- a/index.org +++ b/index.org @@ -91,7 +91,7 @@ To download Worg =.org= files on your machine: - [[file:org-irc.org][Org-mode IRC Channel]] : For live Q&A, go to =#org-mode= on Libera - Check [[https://fosstodon.org/tags/OrgMode][#orgmode]] hashtag on Mastodon - Check [[file:org-web-social.org][latest posts]] on reddit.com and stackoverflow.com -- Browse [[https://www.reddit.com/r/orgmode/][Org-mode discussions on reddit.com]] +- Browse [[https://www.reddit.com/r/orgmode/][Org-mode discussions on Reddit]] - Browse [[https://stackoverflow.com/questions/tagged/org-mode][Org-mode questions on stackoverflow.com]] - [[https://emacslife.com/calendar/][Emacs event calendar]] : Join virtual and in-person Emacs events - [[file:orgmeetup.org][=[[bbb:OrgMeetup]]=]] : talk (online) with Org mode developers and users diff --git a/org-tests/index.org b/org-tests/index.org index f1232ba2..ca6f2cf1 100644 --- a/org-tests/index.org +++ b/org-tests/index.org @@ -64,16 +64,17 @@ _ert.el_ and _ert-x.el_ to your testing directory. This may be accomplished with the following commands entered on the command line. : cd /path/to/org-mode/testing -: curl -O https://github.com/mirrors/emacs/raw/master/lisp/emacs-lisp/ert.el -: curl -O https://github.com/mirrors/emacs/raw/master/lisp/emacs-lisp/ert-x.el +: curl -O https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert.el +: curl -O https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert-x.el Alternatively you may download the files within your browser. - - browse to [[https://github.com/mirrors/emacs/tree/master/lisp/emacs-lisp][github.com/mirrors/emacs/lisp/emacs-lisp]] - - right click ert.el link and select _download linked file_ (or - equivalent) and save to org-mode/testing/ert.el - - right click ert-x.el link and select _download linked file_ (or - equivalent) and save to org-mode/testing/ert-x.el + - browse to [[https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert.el][ert.el in Emacs' source code]] + - Download this file with Ctrl+S, or via Right-click on the page and + "Save Page As" (or equivalent) and save to + =org-mode/testing/ert.el=. + - Do the same for the [[https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert-x.el][ert-x.el]] file, save to + =org-mode/testing/ert-x.el=. That's it - you may now run the tests. diff --git a/org-tutorials/non-beamer-presentations.org b/org-tutorials/non-beamer-presentations.org index e57e4a92..ed1e346c 100644 --- a/org-tutorials/non-beamer-presentations.org +++ b/org-tutorials/non-beamer-presentations.org @@ -34,9 +34,8 @@ documents. HTML export serves as the base of the presentation. - org-html5presentation :: Is an Exporter of Org-mode documents to - HTML5 slide show presentations. - (see [[https://gist.github.com/takumikinjo/509761][gist.github.com/takumikinjo/509761]] for code and usage - information) + HTML5 slide show presentations. (see + [[https://gist.github.com/takumikinjo/509761][this gist by takumikinjo]] for code and usage information) - [[org-tree-slide]] :: is a simple tool to treat a tree of an org buffer as a single slide. Since each slide is displayed by simple narrowing, -- 2.43.0
