Ihor Radchenko <[email protected]> writes: > Ihor Radchenko <[email protected]> writes: > >>>> Also, (org-agenda-prepare-buffers (list file)) is sub-optimal. >>>> org-agenda-prepare-buffers updates the menus, which is slow to do when >>>> adding files one by one. So, it is better to call >>>> org-agenda-prepare-buffers are few times as possible, on a list of files. >>> >>> How would you do that? For files without archives calling the function >>> could be done right before getting the table data. >> >> You can first form a full list of files via org-add-archive-files (if >> needed) in org-dblock-write:clocktable.
So basically adding the archive files twice, once in org-dblock-write:clocktable and then in org-clock-get-table-data-with-archives? >> Then, call >> org-agenda-prepare-buffers once. Then, call >> org-clock-get-table-data-with-archives and/or org-clock--get-table-data1. > > It has been a while. > Bjorn, did you have a chance to work on the patch? Not until this week. Regarding the when-let in org-clock-merge-table-data. Does it make a difference performance wise to append and entry which might be nil vs appending only if a new entry would not be nil? Below the updated patch, plus the other change to make the git hook install work if org mode is a git submodule working.
>From 0a8503504097897f081d9e107d599d6b3e67a09d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Kettunen?= <[email protected]> Date: Tue, 3 Mar 2026 22:47:12 +0200 Subject: [PATCH 1/2] org-clock: clock-report refactor list and function scope * lisp/org-clock.el (org-dblock-write:clocktable): Expand files in directories if any of the entries in scope is a directory. Just like in org-agenda-files. Function scope is now evaluated before any other scope and can return a scope by itself. (org-clock-merge-table-data): A new function which can merge clocktable data for a list of table belonging to a file. (org-clock-get-table-data-with-archives): A new function to get table data for a file with it's archives. * lisp/org.el (org-file-list-expand): (org-agenda-files): Refactor file expansion into separate function. * doc/org-manual.org: Document. * etc/ORG-NEWS: Announce --- doc/org-manual.org | 20 ++++----- etc/ORG-NEWS | 23 ++++++++++ lisp/org-clock.el | 104 +++++++++++++++++++++++++++++++-------------- lisp/org.el | 20 ++++++--- 4 files changed, 119 insertions(+), 48 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 13a16b6d8..57c40d55f 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -7109,16 +7109,16 @@ *** The clock table The scope to consider. This can be any of the following: - | =nil= | the current buffer or narrowed region | - | =file= | the full current buffer | - | =subtree= | the subtree where the clocktable is located | - | =treeN= | the surrounding level N tree, for example =tree3= | - | =tree= | the surrounding level 1 tree | - | =agenda= | all agenda files | - | =("file" ...)= | scan these files | - | =FUNCTION= | scan files returned by calling {{{var(FUNCTION)}}} with no argument | - | =file-with-archives= | current file and its archives | - | =agenda-with-archives= | all agenda files, including archives | + | =nil= | the current buffer or narrowed region | + | =file= | the full current buffer | + | =subtree= | the subtree where the clocktable is located | + | =treeN= | the surrounding level N tree, for example =tree3= | + | =tree= | the surrounding level 1 tree | + | =agenda= | all agenda files | + | =file-with-archives= | current file and its archives | + | =agenda-with-archives= | all agenda files, including archives | + | =([scope] "file" "dir" "...)= | scan these files or files in directories, scope can be file or file-with-archives | + | f=FUNCTION= | call {{{var(FUNCTION)}}} with no argument process any scope it returns | - =:block= :: diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 7d8f29930..8ffb95241 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -138,6 +138,20 @@ A current limitation is that export blocks and keywords are only implemented for events and todos, and not yet for calendar-wide properties. +*** Clocktable options =:scope function= and =:scope (file)= have changed. + +- ~function~ + Now called before any scope is processed. + It can now also return any other scope by itself which is then + processed after. + +- ~(file)~ + Can now also include directories which are resolved just like in + ~org-agenda-files~. + The first entry of this list can also be either ~file~ or ~file-archives~. + If this is the case then all following entries are considered + like the ~file~ or ~file-with-archives~ scope. + ** New and changed options # Changes dealing with changing default values of customizations, @@ -232,6 +246,15 @@ This variable has no effect on links that are already prefixed with Given the completed and total number of tasks, format the percent cookie =[N%]=. +*** New function ~org-agenda-directory-files-recursively~ + +Expand list of flies according to ~org-agenda-file-regexp~. + +*** New function ~org-clock-merge-table-data~ + +Takes a list of clocktable data tables, merge them all +under file or the file name of the first table. + ** Removed or renamed functions and variables ** Miscellaneous diff --git a/lisp/org-clock.el b/lisp/org-clock.el index a1de045fe..30f034128 100644 --- a/lisp/org-clock.el +++ b/lisp/org-clock.el @@ -2687,22 +2687,21 @@ (defun org-dblock-write:clocktable (params) (catch 'exit (let* ((scope (plist-get params :scope)) (base-buffer (org-base-buffer (current-buffer))) + (scope (or (and (functionp scope) + (funcall scope)) + scope)) (files (pcase scope - (`agenda + ((or `agenda `agenda-with-archives) (org-agenda-files t)) - (`agenda-with-archives - (org-add-archive-files (org-agenda-files t))) - (`file-with-archives - (let ((base-file (buffer-file-name base-buffer))) - (and base-file - (org-add-archive-files (list base-file))))) - ((or `nil `file `subtree `tree + ((or `file-with-archives) + (list (buffer-file-name base-buffer))) + ((or `nil `subtree `tree `file (and (pred symbolp) (guard (string-match "\\`tree\\([0-9]+\\)\\'" (symbol-name scope))))) base-buffer) - ((pred functionp) (funcall scope)) ((pred consp) scope) + ((pred stringp) scope) ;; To not break previous function calls here (_ (user-error "Unknown scope: %S" scope)))) (block (plist-get params :block)) (ts (plist-get params :tstart)) @@ -2714,7 +2713,23 @@ (defun org-dblock-write:clocktable (params) (formatter (or (plist-get params :formatter) org-clock-clocktable-formatter 'org-clocktable-write-default)) - cc) + (multifile + ;; Even though `file-with-archives' can consist of + ;; multiple files, we consider this is one extended file + ;; instead. + (and (not hide-files) + (consp files) + (not (eq scope 'file-with-archives)))) + cc) + + (when (consp files) + (when-let* ((cons-scope (car files)) + (cons-scope (and (symbolp cons-scope) + cons-scope))) + (setq scope cons-scope) + (setq files (cdr files))) + (setq files (org-agenda-directory-files-recursively files))) + ;; Check if we need to do steps (when block ;; Get the range text for the header @@ -2728,20 +2743,24 @@ (defun org-dblock-write:clocktable (params) (org-clocktable-steps params) (throw 'exit nil)) - (org-agenda-prepare-buffers (if (consp files) files (list files))) + (org-agenda-prepare-buffers (cond ((memq scope '(file-with-archives agenda-with-archives)) + (org-add-archive-files files)) + ((consp files) files) + (t (list files)))) (let ((origin (point)) (tables - (if (consp files) - (mapcar (lambda (file) - (with-current-buffer (find-buffer-visiting file) - (save-excursion - (save-restriction - (org-clock-get-table-data file params))))) - files) + (if (consp files) + (if (memq scope '(file-with-archives agenda-with-archives)) + (mapcar (lambda (file) + (org-clock-get-table-data-with-archives + file params)) files) + (mapcar (lambda (file) + (org-clock--get-table-data1 file params)) + files)) ;; Get the right restriction for the scope. (save-restriction - (cond + (cond ((not scope)) ;use the restriction as it is now ((eq scope 'file) (widen)) ((eq scope 'subtree) (org-narrow-to-subtree)) @@ -2749,25 +2768,18 @@ (defun org-dblock-write:clocktable (params) (while (org-up-heading-safe)) (org-narrow-to-subtree)) ((and (symbolp scope) - (string-match "\\`tree\\([0-9]+\\)\\'" + (string-match "\\`tree\\([0-9]+\\)\\'" (symbol-name scope))) (let ((level (string-to-number - (match-string 1 (symbol-name scope))))) + (match-string 1 (symbol-name scope))))) (catch 'exit (while (org-up-heading-safe) - (looking-at org-outline-regexp) - (when (<= (org-reduced-level (funcall outline-level)) + (looking-at org-outline-regexp) + (when (<= (org-reduced-level (funcall outline-level)) level) (throw 'exit nil)))) (org-narrow-to-subtree)))) - (list (org-clock-get-table-data nil params))))) - (multifile - ;; Even though `file-with-archives' can consist of - ;; multiple files, we consider this is one extended file - ;; instead. - (and (not hide-files) - (consp files) - (not (eq scope 'file-with-archives))))) + (list (org-clock-get-table-data nil params)))))) (funcall formatter origin @@ -3122,6 +3134,36 @@ (defun org-clocktable-steps (params) (setq start next)) (end-of-line 0)))) +(defun org-clock--get-table-data1 (file params) + "Get clocktable-data for FILE with PARAMS." + (with-current-buffer + (find-buffer-visiting file) + (save-excursion + (save-restriction + (org-clock-get-table-data file params))))) + +(defun org-clock-get-table-data-with-archives (file params) + "Get clocktable data with archives for FILE with parameters PARAMS. +If file-only don't add-archives" + (let* ((file-plus-archives (org-add-archive-files (list file))) + (tables (mapcar (lambda (file) + (org-clock--get-table-data1 file params)) + file-plus-archives))) + (org-clock-merge-table-data tables file))) + +(defun org-clock-merge-table-data (tables &optional file) + "Merge table data for TABLES for FILE. +When FILE isn't given assume FILE as the file of the first table. +TABLES is list of table data returned in the format returned by +`org-clock-get-table-data'. +Returns the same tables but with each table merged." + (let* ((file (or file (caaar tables))) + (total-time 0) entries) + (while-let ((table (pop tables))) + (incf total-time (nth 1 table)) + (setq entries (append entries (car (nthcdr 2 table))))) + (list file total-time entries))) + (defun org-clock-get-table-data (file params) "Get the clocktable data for file FILE, with parameters PARAMS. FILE is only for identification - this function assumes that diff --git a/lisp/org.el b/lisp/org.el index b418bd7ec..d00257fc8 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -16062,6 +16062,18 @@ (defun org-switchb (&optional arg) (mapcar #'list (mapcar #'buffer-name blist)) nil t)))) + +(defun org-agenda-directory-files-recursively (files) + "Expand list of FILES according to `org-agenda-file-regexp'." + (apply 'append + (mapcar (lambda (f) + (if (file-directory-p f) + (directory-files + f t org-agenda-file-regexp) + (list (expand-file-name f org-directory)))) + files))) + + (defun org-agenda-files (&optional unrestricted archives) "Get the list of agenda files. Optional UNRESTRICTED means return the full list even if a restriction @@ -16075,13 +16087,7 @@ (defun org-agenda-files (&optional unrestricted archives) ((stringp org-agenda-files) (org-read-agenda-file-list)) ((listp org-agenda-files) org-agenda-files) (t (error "Invalid value of `org-agenda-files'"))))) - (setq files (apply 'append - (mapcar (lambda (f) - (if (file-directory-p f) - (directory-files - f t org-agenda-file-regexp) - (list (expand-file-name f org-directory)))) - files))) + (setq files (org-agenda-directory-files-recursively files)) (when org-agenda-skip-unavailable-files (setq files (delq nil (mapcar (lambda (file) -- 2.54.0
>From 7569431b36d387e02b981bcb1108f73f0be3e8fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Kettunen?= <[email protected]> Date: Tue, 3 Mar 2026 22:33:27 +0200 Subject: [PATCH 2/2] * mk/targets.mk (GIT_HOOKS): Resolve correctly when in submodule Requires at least Git 2.10, released in 2016 Q3. --- mk/targets.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mk/targets.mk b/mk/targets.mk index a8bec6813..74935bbde 100644 --- a/mk/targets.mk +++ b/mk/targets.mk @@ -9,8 +9,8 @@ SUBDIRS = $(OTHERDIRS) $(LISPDIRS) INSTSUB = $(SUBDIRS:%=install-%) ORG_MAKE_DOC ?= info html pdf -GITDIR = .git/hooks -GITHOOKS = $(notdir $(wildcard git-hooks/*)) +GITDIR = $(shell git rev-parse --git-path hooks) +GITHOOKS = $(notdir $(wildcard $(GITDIR)/git-hooks/*)) ifneq ($(wildcard .git),) # Use the org.el header. -- 2.54.0
