Re: [PATCH] org-id: implement arbitrary cross-file references
On Saturday, May 27th, 2023 at 8:51 AM, Ruijie Yu via "General discussions about Org-mode." wrote: > First of all, I believe this is not within the scope of TINYCHANGE, > because as you see below, you added 58 lines and removed 11. In this > case, the very first thing required before Org maintainers can consider > its inclusion is that you sign the FSF copyright assignment. If you > need more details, Ihor or Bastien could probably elaborate and/or send > you the form. Fair enough -- I'm open to sign the FSF copyright assignment. > First, do we need to wrap it with `save-match-data'? In my personal code > I almost always do, but I need a second opinion on that. > Second, instead of the let-unless-null combination, you can just do this > instead: (when-let ((match ...)) (let* (...) ...)) > And, if those values can't ever be nil, you could even combine the entirety > of the` let*' form into the `when-let' form (and turn it into a` when-let*' > form). Thank you, all done, patch & GH PR updated. -- regards, Sergei KosyrevFrom 960d864dd6587190a78353f27c2de10790560f31 Mon Sep 17 00:00:00 2001 From: Kosyrev Serge Date: Thu, 25 May 2023 19:30:01 +0800 Subject: [PATCH] org-id: implement arbitrary cross-file table referencing * Table formulae can now refer to data from tables in other files. TINYCHANGE --- etc/ORG-NEWS | 10 + lisp/org-id.el| 57 ++- lisp/org-table.el | 2 +- 3 files changed, 58 insertions(+), 11 deletions(-) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 6b40198b5..311067628 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -221,6 +221,16 @@ Running shell blocks with the ~:session~ header freezes Emacs until execution completes. The new ~:async~ header allows users to continue editing with Emacs while a ~:session~ block executes. +*** ~remote~ ~org-table~ can now refer to variables in other files + +Table formulae can now refer to data from tables in other files. + +Example: + +#+BEGIN_src org +#+TBLFM: $2='(identity remote(file:./org-wile-with-tables.org,@@#$Value)) +#+END_src + ** Miscellaneous *** =org-crypt.el= now applies initial visibility settings to decrypted entries diff --git a/lisp/org-id.el b/lisp/org-id.el index aa9610f16..413babcd6 100644 --- a/lisp/org-id.el +++ b/lisp/org-id.el @@ -337,6 +337,40 @@ Move the cursor to that entry in that buffer." (move-marker m nil) (org-fold-show-context))) +(defun org-id-parse-remote-table-ref (refstr) + (save-match-data +(when-let* ((match (string-match "^file:\\([^:]+\\)\\(\\|:.+\\)$" refstr)) +(m1 (match-string 1 refstr)) +(m2 (match-string 2 refstr)) +(filename (cl-remove-if (lambda (c) (member c '(40 41))) +(org-table-formula-substitute-names m1))) +(table-name (org-table-formula-substitute-names m2))) + (list filename table-name + +(defun org-id-find-remote (file table-id markerp) + (if (file-exists-p file) + (let ((buffer (let ((query-about-changed-file nil)) + (find-file-noselect file +(unwind-protect + (with-current-buffer buffer + (goto-char (point-min)) + (let ((pos (progn + (unless (string= table-id "") + (let* ((ident (cl-subseq table-id 1)) +(id-match (search-forward (concat "#+NAME: " ident) nil t))) + (unless id-match + (error "org-id-find-remote: file \"%s\" has no table with NAME \"%s\"." file ident)) + (forward-line))) + (re-search-forward "^|-") + (move-beginning-of-line nil +(cond + ((null pos) nil) + (markerp (move-marker (make-marker) pos buffer)) + (t (cons file pos) + ;; Remove opened buffer in the process. + (unless markerp (kill-buffer buffer +(error "org-id-find-remote: reference to missing file %s" file))) + ;;;###autoload (defun org-id-find (id markerp) "Return the location of the entry with the id ID. @@ -346,16 +380,19 @@ With optional argument MARKERP, return the position as a new marker." (cond ((symbolp id) (setq id (symbol-name id))) ((numberp id) (setq id (number-to-string id - (let ((file (org-id-find-id-file id)) - org-agenda-new-buffers where) -(when file - (setq where (org-id-find-id-in-file id file markerp))) -(unless where - (org-id-update-id-locations nil t) - (setq file (org-id-find-id-file id)) - (when file - (setq where (org-id-find-id-in-file id file markerp -where)) + (let ((remote-match (org-id-parse-remote-table-ref id))) +(if remote-match +(org-id-find-remote (car remote-match) (cadr remote-match) markerp) + (let ((file (org-id-find-id-file id)) +
Re: [PATCH] org-id: implement arbitrary cross-file references
Thanks for the patch. Some in-line comments below. Sergei Kosyrev writes: > From f712fa57a90c68d3d9066b10f49822ea0337b923 Mon Sep 17 00:00:00 2001 > From: Kosyrev Serge > Date: Thu, 25 May 2023 19:30:01 +0800 > Subject: [PATCH] org-id: implement arbitrary cross-file references > > * Table formulae can now refer to data from tables in other files. > > TINYCHANGE First of all, I believe this is not within the scope of TINYCHANGE, because as you see below, you added 58 lines and removed 11. In this case, the very first thing required before Org maintainers can consider its inclusion is that you sign the FSF copyright assignment. If you need more details, Ihor or Bastien could probably elaborate and/or send you the form. > --- > etc/ORG-NEWS | 10 + > lisp/org-id.el| 57 ++- > lisp/org-table.el | 2 +- > 3 files changed, 58 insertions(+), 11 deletions(-) > > diff --git a/lisp/org-id.el b/lisp/org-id.el > index aa9610f16..2fcecbb50 100644 > --- a/lisp/org-id.el > +++ b/lisp/org-id.el > @@ -337,6 +337,40 @@ Move the cursor to that entry in that buffer." > (move-marker m nil) > (org-fold-show-context))) > > +(defun org-id-parse-remote-table-ref (refstr) > + (let ((match (string-match "^file:\\([^:]+\\)\\(\\|:.+\\)$" refstr))) > +(unless (null match) > + (let* ((m1 (match-string 1 refstr)) > + (m2 (match-string 2 refstr)) > + (filename (cl-remove-if (lambda (c) (member c '(40 41))) > + (org-table-formula-substitute-names m1))) > + (table-name (org-table-formula-substitute-names m2))) > +(list filename table-name) First, do we need to wrap it with `save-match-data'? In my personal code I almost always do, but I need a second opinion on that. Second, instead of the let-unless-null combination, you can just do this instead: (when-let ((match ...)) (let* (...) ...)) And, if those values can't ever be nil, you could even combine the entirety of the `let*' form into the `when-let' form (and turn it into a `when-let*' form). No comments on the rest, so I didn't quote them from my response to save bandwidth. -- Best, RY
[PATCH] org-id: implement arbitrary cross-file references
Good day list! This extends org-id in a way that allows table formulae to reference table content in other files. Patch is attached, and can be alternatively observed via github: https://github.com/deepfire/org-mode/pull/1/files This is my first contribution, so caveat emptor! Thank you! -- best regards, Serge KosyrevFrom f712fa57a90c68d3d9066b10f49822ea0337b923 Mon Sep 17 00:00:00 2001 From: Kosyrev Serge Date: Thu, 25 May 2023 19:30:01 +0800 Subject: [PATCH] org-id: implement arbitrary cross-file references * Table formulae can now refer to data from tables in other files. TINYCHANGE --- etc/ORG-NEWS | 10 + lisp/org-id.el| 57 ++- lisp/org-table.el | 2 +- 3 files changed, 58 insertions(+), 11 deletions(-) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 6b40198b5..311067628 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -221,6 +221,16 @@ Running shell blocks with the ~:session~ header freezes Emacs until execution completes. The new ~:async~ header allows users to continue editing with Emacs while a ~:session~ block executes. +*** ~remote~ ~org-table~ can now refer to variables in other files + +Table formulae can now refer to data from tables in other files. + +Example: + +#+BEGIN_src org +#+TBLFM: $2='(identity remote(file:./org-wile-with-tables.org,@@#$Value)) +#+END_src + ** Miscellaneous *** =org-crypt.el= now applies initial visibility settings to decrypted entries diff --git a/lisp/org-id.el b/lisp/org-id.el index aa9610f16..2fcecbb50 100644 --- a/lisp/org-id.el +++ b/lisp/org-id.el @@ -337,6 +337,40 @@ Move the cursor to that entry in that buffer." (move-marker m nil) (org-fold-show-context))) +(defun org-id-parse-remote-table-ref (refstr) + (let ((match (string-match "^file:\\([^:]+\\)\\(\\|:.+\\)$" refstr))) +(unless (null match) + (let* ((m1 (match-string 1 refstr)) + (m2 (match-string 2 refstr)) + (filename (cl-remove-if (lambda (c) (member c '(40 41))) + (org-table-formula-substitute-names m1))) + (table-name (org-table-formula-substitute-names m2))) +(list filename table-name) + +(defun org-id-find-remote (file table-id markerp) + (if (file-exists-p file) + (let ((buffer (let ((query-about-changed-file nil)) + (find-file-noselect file +(unwind-protect + (with-current-buffer buffer + (goto-char (point-min)) + (let ((pos (progn + (unless (string= table-id "") + (let* ((ident (cl-subseq table-id 1)) +(id-match (search-forward (concat "#+NAME: " ident) nil t))) + (unless id-match + (error "org-id-find-remote: file \"%s\" has no table with NAME \"%s\"." file ident)) + (forward-line))) + (re-search-forward "^|-") + (move-beginning-of-line nil +(cond + ((null pos) nil) + (markerp (move-marker (make-marker) pos buffer)) + (t (cons file pos) + ;; Remove opened buffer in the process. + (unless markerp (kill-buffer buffer +(error "org-id-find-remote: reference to missing file %s" file))) + ;;;###autoload (defun org-id-find (id markerp) "Return the location of the entry with the id ID. @@ -346,16 +380,19 @@ With optional argument MARKERP, return the position as a new marker." (cond ((symbolp id) (setq id (symbol-name id))) ((numberp id) (setq id (number-to-string id - (let ((file (org-id-find-id-file id)) - org-agenda-new-buffers where) -(when file - (setq where (org-id-find-id-in-file id file markerp))) -(unless where - (org-id-update-id-locations nil t) - (setq file (org-id-find-id-file id)) - (when file - (setq where (org-id-find-id-in-file id file markerp -where)) + (let ((remote-match (org-id-parse-remote-table-ref id))) +(if remote-match +(org-id-find-remote (car remote-match) (cadr remote-match) markerp) + (let ((file (org-id-find-id-file id)) + org-agenda-new-buffers where) +(when file + (setq where (org-id-find-id-in-file id file markerp))) +(unless where + (org-id-update-id-locations nil t) + (setq file (org-id-find-id-file id)) + (when file + (setq where (org-id-find-id-in-file id file markerp +where ;;; Internal functions diff --git a/lisp/org-table.el b/lisp/org-table.el index ac685c41e..e894d563a 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -6281,7 +6281,7 @@ list of the fields in the rectangle." (let ((case-fold-search t) (id-loc nil) ;; Protect a bunch of variables from being overwritten by ;; the context of the remote table. - org-table-column-names
[BUG] wrong-type-argument syntax-table-p "Syntax table including \"@\" and \"_\" as word constituents.
Hi, there was a recent change to how the tags table is initialized. I think that as a side effect, some tag functions like org-tags-expand don't work unless you open an org-mode buffer first. I have a small bash script that exports my agenda; right in the beginning (before opening any org-mode file) it collects some data by calling org-map-entries. org-map-entries needs org-tags-expand, which now fails because the syntax table hasn't been initialized yet. Minimal case to reproduce this: 1. run: emacs -Q 2. load latest org: (add-to-list 'load-path "/…/org-mode/lisp") (require 'org) 3. eval: (org-tags-expand "+sometag") You'll see: Debugger entered--Lisp error: (wrong-type-argument syntax-table-p "Syntax table including \"@\" and \"_\" as word constit...") org-tags-expand("+sometag") (progn (org-tags-expand "+sometag")) elisp--eval-last-sexp(nil) eval-last-sexp(nil) funcall-interactively(eval-last-sexp nil) If I open any .org file before step 3, then it will work. I think the related commit is: commit 6e6354c074a323780f103aabf45be74104ce3ecf Author: Ihor Radchenko Date: Mon May 8 13:23:15 2023 +0200 org-tags-expand: Do no modify buffer's syntax table by side effect * lisp/org.el (org-mode-tags-syntax-table): New variable holding syntax table for tags. (org-mode): Initialize tag syntax table. (org-make-tags-matcher): Match tags using appropriate syntax table. (org-tags-expand): Do no modify syntax table by side effect. Reported-by: Mattias Engdegård Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63225#68 Of course I could make sure to open an org file (any file) before calling org-map-entries but that seems like a workaround to a bug. I would expect (org-map-entries … 'agenda) to also work before opening any .org file.