Ihor Radchenko <[email protected]> writes: > Maybe it can simply be error?
Yes, makes sense. I have changed it. See updated patch below. Aside: if --debug is passed, ./publish.sh fails for two reasons: 1. the link introduced in 6e5ad15177d65290c9ccbf54258e4b507d3dbec1 [[org-faq/][...]] is an invalid link (though it works as expected) according to org-export 2. the first emacs-lisp src block in org-release-notes.org fails with the message Search failed: "^\\* " not too important for now. > Right. > (load-file "./worg-urls-rewrite.el") works. > (This time, I tested myself) Thanks > Looks mostly good. A few comments below. > >> +++ b/rewrite-websites.js > > This file is not used by anything. Removed > Here, and in rewrite-websites.js, we need to put a GPLv3 license notice. > It is not "a part of Emacs" and not copyrighted by FSF, but we need to > add a header for GPLv3. Added > We can do reddit redirects even when libredirect data is not available. Ok. I have removed the (and worg-urls-libredirect-data ...) and use (if-let* ...) for the non-reddit domains. Another aside: do I have to be careful with BARK keywords here? So "Done." or "Fixed." above instead of "Ok." would mess up BARK?
>From 90161dd59b663217db986730a209b1fa644eeb64 Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Tue, 25 Nov 2025 11:10:46 +0100 Subject: [PATCH 1/4] Find alternative links for YouTube, GitHub and Reddit This commit finds alternative links for the hosts mentioned above since they rely on non-free JS. See https://list.orgmode.org/orgmode/87pl9szmy6.fsf@localhost/ for context. --- .gitignore | 1 + publish.sh | 59 +++++----- worg-urls-rewrite.el | 269 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 302 insertions(+), 27 deletions(-) create mode 100644 worg-urls-rewrite.el diff --git a/.gitignore b/.gitignore index c9dc5b62..c572b901 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,4 @@ patches/ /.project *patch +*.eld diff --git a/publish.sh b/publish.sh index b499de3d..bebe50c6 100755 --- a/publish.sh +++ b/publish.sh @@ -14,6 +14,8 @@ This variable can be set when running publish.sh script: (require 'htmlize) (require 'org-inlinetask) +(load-file "./worg-urls-rewrite.el") + (setq make-backup-files nil debug-on-error t) @@ -58,30 +60,33 @@ This variable can be set when running publish.sh script: (R . t) (gnuplot . t))) -(dolist (org-file (cl-remove-if - (lambda (n) (string-match-p "worg/archive/" n)) - (directory-files-recursively default-directory "\\.org$"))) - (let ((html-file (concat (file-name-directory org-file) - (file-name-base org-file) ".html"))) - (if (and (file-exists-p html-file) - (file-newer-than-file-p html-file org-file) - ;; If there are include files or code, we need to - ;; re-generate the HTML just in case if the included - ;; files are changed. - (with-temp-buffer - (insert-file-contents org-file) - (and - (save-excursion - (goto-char (point-min)) - (not (re-search-forward "#\\+include:" nil t))) - (save-excursion - (goto-char (point-min)) - (not (re-search-forward "#\\+begin_src" nil t)))))) - (message " [skipping] unchanged %s" org-file) - (message "[exporting] %s" (file-relative-name org-file default-directory)) - (with-current-buffer (find-file org-file) - (if worg-publish-stop-on-error - (org-html-export-to-html) - (condition-case err - (org-html-export-to-html) - (error (message (error-message-string err))))))))) +(add-hook 'org-export-before-parsing-functions #'worg-urls-add-alternative-links) +(let ((export-all (worg-urls-libredirect-data-changed-p worg-urls-libredirect-data))) + (dolist (org-file (cl-remove-if + (lambda (n) (string-match-p "worg/archive/" n)) + (directory-files-recursively default-directory "\\.org$"))) + (let ((html-file (concat (file-name-directory org-file) + (file-name-base org-file) ".html"))) + (if (and (not export-all) + (file-exists-p html-file) + (file-newer-than-file-p html-file org-file) + ;; If there are include files or code, we need to + ;; re-generate the HTML just in case if the included + ;; files are changed. + (with-temp-buffer + (insert-file-contents org-file) + (and + (save-excursion + (goto-char (point-min)) + (not (re-search-forward "#\\+include:" nil t))) + (save-excursion + (goto-char (point-min)) + (not (re-search-forward "#\\+begin_src" nil t)))))) + (message " [skipping] unchanged %s" org-file) + (message "[exporting] %s" (file-relative-name org-file default-directory)) + (with-current-buffer (find-file org-file) + (if worg-publish-stop-on-error + (org-html-export-to-html) + (condition-case err + (org-html-export-to-html) + (error (message (error-message-string err)))))))))) diff --git a/worg-urls-rewrite.el b/worg-urls-rewrite.el new file mode 100644 index 00000000..51a646aa --- /dev/null +++ b/worg-urls-rewrite.el @@ -0,0 +1,269 @@ +;;; worg-urls-rewrite.el --- Rewrite URLs that refer to pages containing non-free JS to free alternatives + +;; This file is part of GNU Emacs. +;; +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by7 +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. +;; +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. +;; +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see <https://www.gnu.org/licenses/>. + +;;; Commentary: +;; +;; Worg links to many external pages, some of which link to websites +;; that rely on non-free JavaScript, such as GitHub or YouTube pages. +;; This script contains the function `worg-urls-add-alternative-links' +;; which converts these links to free alternatives, such as GotHub and +;; Invidious. +;; +;; Since these alternatives occasionally go down, we use the +;; alternative specified by libredirect, which checks whether these +;; alternatives are live or not. This is seen in +;; `worg-urls-libredirect-url'. +;; +;; This was discussed on the mailing list here: +;; https://list.orgmode.org/87pl9szmy6.fsf@localhost. +;; +;;; Code: + +(defvar worg-urls-libredirect-url "https://raw.githubusercontent.com/libredirect/instances/main/data.json"; + "Location where up-to-date libredirect data is found.") +(defvar worg-urls-libredirect-data nil + "Contents of libredirect data, obtained from `worg-urls-libredirect-url'.") + +(defcustom worg-urls-free-alternatives '((youtube . invidious) + (github . gothub)) + "Alist of alternative frontends to websites containing non-free JS. +The CDR corresponds to a symbol that is known by libredirect, see +https://codeberg.org/LibRedirect/instances.";) +(defcustom worg-urls-redirect-save-filename "redirect-local-data.eld" + "Filename used to save the relevant output of libredirect's data.") + + +(defun worg-urls-libredirect-data-changed-p (&optional data) + "Whether or not libredirect DATA changed since our last export." + (setq worg-urls-libredirect-data (or data (worg-urls--get-libredirect-data))) + (if (not (file-exists-p worg-urls-redirect-save-filename)) + ;; We hadn't saved stuff from our previous export, do so now. + (progn + (worg-urls--save-redirect-data worg-urls-libredirect-data) + t) + ;; Compare results from stored file with data, overwrite stored data if newer + (let* ((stored-data (with-temp-buffer + (insert-file-contents worg-urls-redirect-save-filename) + (read (buffer-string)))) + (data-equal (equal (worg-urls--relevant-redirect-data worg-urls-libredirect-data) + stored-data))) + (unless data-equal + (worg-urls--save-redirect-data worg-urls-libredirect-data)) + (not data-equal)))) + + +(defun worg-urls--get-libredirect-data () + "Return libredirect data from `worg-urls-libredirect-url'." + (condition-case _ + (let ((response (with-current-buffer + (url-retrieve-synchronously worg-urls-libredirect-url nil nil 5) + (prog2 + (re-search-forward "\n\n" nil t) ; skip HTTP headers + (buffer-substring-no-properties (point) (point-max)) + (kill-buffer))))) + (json-parse-string response :object-type 'alist :array-type 'list)) + (t + (message "Worg couldn't access libredirect data, using original URLs.") + nil))) + + +(defun worg-urls--save-redirect-data (&optional data) + "Save redirect DATA in a file with filename `worg-urls-redirect-save-filename'." + (let ((tosave (or data worg-urls-libredirect-data))) + (with-temp-file worg-urls-redirect-save-filename + (prin1 + (worg-urls--relevant-redirect-data data) + (current-buffer))))) + + +(defun worg-urls--relevant-redirect-data (data) + "Extract relevant redirection DATA, defined in `worg-urls-free-alternatives'." + (seq-map (lambda (association) + (let ((from (car association))) + `(,from . ,(worg-urls--first-link-of-data data from)))) + worg-urls-free-alternatives)) + +(defun worg-urls--is-short-gist-path-p (link) + "Test whether LINK links to a gist without a user id. +These redirect to a \"full\" gist.github.com/<user>/<id> link and at the +moment have to be fixed manually - gothub does not support these short +links." + (string-match (rx "//gist.github.com/" (1+ (in digit))) link)) + +(defun worg-urls-add-alternative-links (&optional _) + "Add alternative links for websites containing non-free JS. +For each link that has an alternative (currently YouTube and GitHub), we +insert a link to the free alternative, and change the link text of the +original link to to =(original URL)=. We also redirect reddit links to +old.reddit.com. + +See https://list.orgmode.org/orgmode/87pl9szmy6.fsf@localhost/"; + (unless worg-urls-libredirect-data + (setq worg-urls-libredirect-data (worg-urls--get-libredirect-data))) + + (let ((links (org-element-map (org-element-parse-buffer) 'link #'identity))) + (dolist (link (nreverse links)) + (when (string-prefix-p "http" (org-element-property :type link)) + (let ((path (org-element-property :path link))) + (if-let* ((new-url (worg-urls--find-replacement path))) + (let* ((begin (org-element-property :begin link)) + (contents-begin (org-element-property :contents-begin link)) + (contents-end (org-element-property :contents-end link))) + (save-excursion + (if (and contents-begin contents-end) + ;; Link has description. + (let ((description (buffer-substring contents-begin contents-end))) + ;; If this description refers to the full + ;; domain, this description will be confusing, + ;; since we change the domain here. Warn + ;; about this. + (when (string-match + (rx (or "reddit.com" + "github.com" + "youtube.com")) + description) + (error + (concat "Link description \"%s\" in %s:%s refers" + " to the full domain \"%s\", which we change." + " This is confusing, please change the description.") + description (buffer-file-name) + (line-number-at-pos (org-element-property :contents-begin link)) + (match-string 0 description))) + + ;; Change description to (original URL) + (goto-char contents-begin) + (delete-region contents-begin contents-end) + (insert "(original URL)") + ;; Insert new link with free alternative + (goto-char begin) + (insert (format "[[%s][%s]] " new-url description))) + ;; Else, "bare" link without description. We insert + ;; the original link as bare, and add the old one + ;; with (original URL) as description. + (let* ((end (org-element-property :end link)) + (original (buffer-substring begin end)) + ;; the URL might be in [[https://...]] + ;; format or "just" in https://... format. + (url (if (string-match (rx string-start + "[[" + (group (seq "https://"; (* anything))) + "]]" + string-end) + original) + (match-string 1 original) + original)) + (replacement (format "[[%s]] [[%s][(original URL)]]" new-url url))) + (delete-region begin end) + (goto-char begin) + (insert replacement))))) + ;; No replacement found. Check if the link is a short + ;; github gist, this can be fixed manually. + (when (worg-urls--is-short-gist-path-p path) + (error + (concat "Please replace URL %s in %s:%s with its redirect target," + " we cannot convert this short Gist URL to a free alternative.") + path + (buffer-file-name) + (line-number-at-pos (org-element-property :begin link)))))))))) + + +(defun worg-urls--find-replacement (path) + "Take PATH and return an alternative link if known and nil otherwise. +We redirect the following links: +- reddit.com -> old.reddit.com +- (youtube.com|youtu.be) -> alternative from libredirect +- github.com -> alternative from libredirect" + (pcase path + ;; See https://docs.invidious.io/redirector/ + ;; This matches both + ;; - youtube.com/watch?v=<video-id> + ;; - youtube.com/playlist?list=<playlist-id> + ;; with optionally ? and = escaped + ((rx "//" (? "www\.") + "youtube.com" + (group + (or (and "/watch" + (or "?" "%3F") + "v" + (or "=" "%3D")) + (and "/playlist" + (or "?" "%3F") + "list" + (or "=" "%3D")))) + (group (+ not-newline))) + (if-let* ((route (match-string 1 path)) + (id (match-string 2 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'youtube))) + (concat host route id))) + ;; youtu.be/<video-id> + ((rx "//" (? "www\.") + "youtu.be" + "/" + (group (+ not-newline))) + (if-let* ((video-id (match-string 1 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'youtube))) + (concat host "/watch?v=" video-id))) + ;; gothub supports all links currently used in Worg, except for + ;; /<user>/<repo>/<issues>, so we exclude those. + ((and + (rx "//" (? "www\.") + "github.com" + "/" + (group (+ not-newline))) + ;; Exclude issues path. This approach currently also would + ;; erroneously exclude a supported link if author or repo + ;; starts with "issues" + (guard (not (string-match-p "/issues" path)))) + (if-let* ((url (match-string 1 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'github))) + (concat host "/" url))) + ;; gist.github.com links can have the format + ;; gist.github.com/<user>/<id>, but also obsolete + ;; gist.github.com/<id>. These last links are automatically + ;; redirected on github.com, but we do not know where to + ;; redirect them to at this moment, so we cannot convert + ;; these. Therefore we currently only convert + ;; gist.github.com/<user>/<id> links. + ((rx "//" "gist.github.com/" + ;; user + (group (1+ (not (any "/$")))) + "/" + ;; id + (group (+ not-newline))) + (if-let* ((user (match-string 1 path)) + (id (match-string 2 path)) + (host (worg-urls--first-link-of-data worg-urls-libredirect-data 'github))) + (concat host "/gist/" user "/" id))) + ;; redirect reddit.com to old.reddit.com + ((rx "//" (? "www\.") + "reddit.com" + "/" + (group (+ not-newline))) + (let* ((url (match-string 1 path)) + (host "https://old.reddit.com";)) + (concat host "/" url))))) + +(defun worg-urls--first-link-of-data (data host) + "Get the first available link in DATA of the alternative links for HOST." + (let* ((alternative (cdr (assoc host worg-urls-free-alternatives))) + (object (cdr (assoc alternative data))) + (links (cdr (assoc 'clearnet object)))) + (car links))) + +(provide 'worg-urls-rewrite) + +;;; worg-urls-rewrite.el ends here -- 2.51.2
>From c32443b2f8557c9e832a7e71162e8cdb47232de9 Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Tue, 20 Jan 2026 13:50:35 +0100 Subject: [PATCH 2/4] Expand short gist.github.com urls --- org-contrib/index.org | 2 +- org-hacks.org | 2 +- org-tools/index.org | 2 +- org-tutorials/non-beamer-presentations.org | 3 ++- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/org-contrib/index.org b/org-contrib/index.org index df6f2845..95027baf 100644 --- a/org-contrib/index.org +++ b/org-contrib/index.org @@ -430,7 +430,7 @@ See [[file:../exporters/index.org][Exporters]]. - [[https://github.com/eschulte/org-S5][org-S5]] from Eric Schulte - [[https://github.com/relevance/org-html-slideshow.git][org-html-slideshow]] from Stuart Sierra - - [[https://gist.github.com/509761][org-html5presentation.el]] from kinjo + - [[https://gist.github.com/takumikinjo/509761][org-html5presentation.el]] from kinjo For use with the new exporter, /ox-s5.el/ ([[contribfile:lisp/ox-s5.el][link to raw file)]] and /ox-deck.el/ ([[contribfile:lisp/ox-deck.el][link to raw file]]), both by Rick Frankel, are diff --git a/org-hacks.org b/org-hacks.org index cdff7e56..43992de8 100644 --- a/org-hacks.org +++ b/org-hacks.org @@ -67,7 +67,7 @@ Here is a hook function to use archive this effect: *** Picking up a random task in the global TODO list -Tony day [[https://list.orgmode.org/[email protected]][shared]] [[https://gist.github.com/4343164][this gist]] to pick up a +Tony day [[https://list.orgmode.org/[email protected]][shared]] [[https://gist.github.com/tonyday567/4343164][this gist]] to pick up a random task. ** Building and Managing Org diff --git a/org-tools/index.org b/org-tools/index.org index b2a35de3..74bb03f7 100644 --- a/org-tools/index.org +++ b/org-tools/index.org @@ -211,7 +211,7 @@ I have written yet another converter from planner to org: ** Export vCard to org-mode entries -[[https://list.orgmode.org/[email protected]][Simon Thum]] shared some [[https://gist.github.com/4145201][Ruby code]] to export vCards to Org-mode entries. +[[https://list.orgmode.org/[email protected]][Simon Thum]] shared some [[https://gist.github.com/simonthum/4145201][Ruby code]] to export vCards to Org-mode entries. ** ews-orgmode - Exchange calendar to orgmode diff --git a/org-tutorials/non-beamer-presentations.org b/org-tutorials/non-beamer-presentations.org index 33ea005e..e57e4a92 100644 --- a/org-tutorials/non-beamer-presentations.org +++ b/org-tutorials/non-beamer-presentations.org @@ -35,7 +35,8 @@ documents. - org-html5presentation :: Is an Exporter of Org-mode documents to HTML5 slide show presentations. - (see [[https://gist.github.com/509761][gist.github.com/509761]] for code and usage information) + (see [[https://gist.github.com/takumikinjo/509761][gist.github.com/takumikinjo/509761]] for code and usage + information) - [[org-tree-slide]] :: is a simple tool to treat a tree of an org buffer as a single slide. Since each slide is displayed by simple narrowing, -- 2.51.2
>From 1dfc6015473c3399cad69adacd358c1cd3ee07ff Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Wed, 4 Feb 2026 12:40:48 +0100 Subject: [PATCH 3/4] Change references of full non-free domains in link descriptions --- index.org | 2 +- org-tests/index.org | 15 ++++++++------- org-tutorials/non-beamer-presentations.org | 5 ++--- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/index.org b/index.org index c61746dd..68711dd3 100644 --- a/index.org +++ b/index.org @@ -91,7 +91,7 @@ To download Worg =.org= files on your machine: - [[file:org-irc.org][Org-mode IRC Channel]] : For live Q&A, go to =#org-mode= on Libera - Check [[https://fosstodon.org/tags/OrgMode][#orgmode]] hashtag on Mastodon - Check [[file:org-web-social.org][latest posts]] on reddit.com and stackoverflow.com -- Browse [[https://www.reddit.com/r/orgmode/][Org-mode discussions on reddit.com]] +- Browse [[https://www.reddit.com/r/orgmode/][Org-mode discussions on Reddit]] - Browse [[https://stackoverflow.com/questions/tagged/org-mode][Org-mode questions on stackoverflow.com]] - [[https://emacslife.com/calendar/][Emacs event calendar]] : Join virtual and in-person Emacs events - [[file:orgmeetup.org][=[[bbb:OrgMeetup]]=]] : talk (online) with Org mode developers and users diff --git a/org-tests/index.org b/org-tests/index.org index f1232ba2..ca6f2cf1 100644 --- a/org-tests/index.org +++ b/org-tests/index.org @@ -64,16 +64,17 @@ _ert.el_ and _ert-x.el_ to your testing directory. This may be accomplished with the following commands entered on the command line. : cd /path/to/org-mode/testing -: curl -O https://github.com/mirrors/emacs/raw/master/lisp/emacs-lisp/ert.el -: curl -O https://github.com/mirrors/emacs/raw/master/lisp/emacs-lisp/ert-x.el +: curl -O https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert.el +: curl -O https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert-x.el Alternatively you may download the files within your browser. - - browse to [[https://github.com/mirrors/emacs/tree/master/lisp/emacs-lisp][github.com/mirrors/emacs/lisp/emacs-lisp]] - - right click ert.el link and select _download linked file_ (or - equivalent) and save to org-mode/testing/ert.el - - right click ert-x.el link and select _download linked file_ (or - equivalent) and save to org-mode/testing/ert-x.el + - browse to [[https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert.el][ert.el in Emacs' source code]] + - Download this file with Ctrl+S, or via Right-click on the page and + "Save Page As" (or equivalent) and save to + =org-mode/testing/ert.el=. + - Do the same for the [[https://cgit.git.savannah.gnu.org/cgit/emacs.git/plain/lisp/emacs-lisp/ert-x.el][ert-x.el]] file, save to + =org-mode/testing/ert-x.el=. That's it - you may now run the tests. diff --git a/org-tutorials/non-beamer-presentations.org b/org-tutorials/non-beamer-presentations.org index e57e4a92..ed1e346c 100644 --- a/org-tutorials/non-beamer-presentations.org +++ b/org-tutorials/non-beamer-presentations.org @@ -34,9 +34,8 @@ documents. HTML export serves as the base of the presentation. - org-html5presentation :: Is an Exporter of Org-mode documents to - HTML5 slide show presentations. - (see [[https://gist.github.com/takumikinjo/509761][gist.github.com/takumikinjo/509761]] for code and usage - information) + HTML5 slide show presentations. (see + [[https://gist.github.com/takumikinjo/509761][this gist by takumikinjo]] for code and usage information) - [[org-tree-slide]] :: is a simple tool to treat a tree of an org buffer as a single slide. Since each slide is displayed by simple narrowing, -- 2.51.2
>From cdd415c11e48dcea6a9f9cf91d3c54dc46a1d0b4 Mon Sep 17 00:00:00 2001 From: Rens Oliemans <[email protected]> Date: Wed, 15 Apr 2026 13:49:51 +0200 Subject: [PATCH 4/4] Fix invalid links --- org-contrib/babel/languages/ob-doc-vala.org | 2 +- org-tutorials/org-ruby.org | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/org-contrib/babel/languages/ob-doc-vala.org b/org-contrib/babel/languages/ob-doc-vala.org index 3c2f6520..9e1f1dfc 100644 --- a/org-contrib/babel/languages/ob-doc-vala.org +++ b/org-contrib/babel/languages/ob-doc-vala.org @@ -106,7 +106,7 @@ values become a table (see [[Commandline parameters][Commandline parameters]] ex ** Other *** Differences from other supported languages -There currently is no support for the [[:https://orgmode.org/manual/Environment-of-a-Code-Block.html#index-var_002c-header-argument][~:var~]] header. If you need to +There currently is no support for the [[https://orgmode.org/manual/Environment-of-a-Code-Block.html#index-var_002c-header-argument][~:var~]] header. If you need to pass parameters to your Vala code, you can only use ~:flags~ or ~:cmdline~ (see [[Vala-specific header arguments][Vala-specific header arguments]]). diff --git a/org-tutorials/org-ruby.org b/org-tutorials/org-ruby.org index e5a8d290..56a15fc0 100644 --- a/org-tutorials/org-ruby.org +++ b/org-tutorials/org-ruby.org @@ -179,10 +179,10 @@ tags: * Project history -Brian Dewey developed the gem, the original repository is <https://github.com/bdewey/org-ruby>. +Brian Dewey developed the gem, the original repository is [[https://github.com/bdewey/org-ruby]]. Later Waldemar Quevedo became the maintainer and currently the gem is built from the -<https://github.com/wallyqs/org-ruby> repository. +[[https://github.com/wallyqs/org-ruby]] repository. * See also -- 2.51.2
