Re: org-persist cache for remote files
Lele Gaifax writes: > Hi Ihor, > > I had some trouble testing your patches, and the outcome is still unclear to > me: apparently (again, cannot be sure this is not caused by "improper" > application of the patches) the new version of org-persist, with default value > for o-p-remote-files (100), removes any remote file from the persistent index, > without even prompting for a password. Oops. Try: - (<= org-persist-remote-files remote-files-num)) + (>= org-persist-remote-files remote-files-num)) Best, Ihor
Re: org-persist cache for remote files
Hi Ihor, I had some trouble testing your patches, and the outcome is still unclear to me: apparently (again, cannot be sure this is not caused by "improper" application of the patches) the new version of org-persist, with default value for o-p-remote-files (100), removes any remote file from the persistent index, without even prompting for a password. I will try again (and harder) as time permits. ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929.
Re: org-persist cache for remote files
Lele Gaifax writes:
> I couldn't test yet, but I saw a couple of other glitches, sorry for not
> noticing first.
Thanks for the proof reading.
I updated the patch accordingly.
Best,
Ihor
>From 557e7cec6dd22b09468b82650feed2020f97d781 Mon Sep 17 00:00:00 2001
Message-Id: <557e7cec6dd22b09468b82650feed2020f97d781.1639907677.git.yanta...@gmail.com>
From: Ihor Radchenko
Date: Sat, 18 Dec 2021 22:14:24 +0800
Subject: [PATCH] org-persist.el: Customise persistence of data for remote
files
* lisp/org-persist.el (org-persist-remote-files): New defcustom
defining strategy for persisting data associated with remote files.
Separate strategy is necessary to avoid unexpected password prompts
while garbage collecting the cache: remote files may require server
login.
(org-persist-gc): Decide whether to store remote file data according
to `org-persist-remote-files'.
Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u
---
lisp/org-persist.el | 42 +++---
1 file changed, 35 insertions(+), 7 deletions(-)
diff --git a/lisp/org-persist.el b/lisp/org-persist.el
index 90f9a3bc9..8e8ecc196 100644
--- a/lisp/org-persist.el
+++ b/lisp/org-persist.el
@@ -57,6 +57,23 @@ (defcustom org-persist-directory (expand-file-name
:group 'org-persist
:type 'directory)
+(defcustom org-persist-remote-files 100
+ "Whether to keep persistent data for remote files.
+
+When this variable is nil, never save persistent data associated with
+remote files. When `t', always keep the data. When a number, keep up
+to that number persistent values for remote files. When
+`check-existence', contact remote server containing the file and only
+keep the data when the file exists on the server.
+
+Note that setting value to `check-existence' will require logging in
+to remote servers. Emacs may show multiple password prompts."
+ :group 'org-persist
+ :type '(choice (const :tag "Never" nil)
+ (const :tag "Always" t)
+ (number :tag "Up to X files")
+ (const :tag "Check if exist on remote" 'check-existence)))
+
(defvar org-persist-index-file "index"
"File name used to store the data index.")
@@ -287,15 +304,26 @@ (defun org-persist-read-all-buffer ()
(defun org-persist-gc ()
"Remove stored data for not existing files or unregistered variables."
- (let (new-index)
+ (let (new-index (remote-files-num 0))
(dolist (index org-persist--index)
- (let ((file (plist-get index :path))
-(persist-file (when (plist-get index :persist-file)
-(org-file-name-concat
- org-persist-directory
- (plist-get index :persist-file)
+ (let* ((file (plist-get index :path))
+ (file-remote (when file (file-remote-p file)))
+ (persist-file (when (plist-get index :persist-file)
+ (org-file-name-concat
+ org-persist-directory
+ (plist-get index :persist-file)
(when (and file persist-file)
- (if (file-exists-p file)
+ (when (and file-remote persist-file)
+(cl-incf remote-files-num))
+ (if (if (not file-remote)
+ (file-exists-p file)
+(pcase org-persist-remote-files
+ ('t t)
+ ('check-existence
+ (file-exists-p file))
+ ((pred #'numberp)
+ (<= org-persist-remote-files remote-files-num))
+ (_ nil)))
(push index new-index)
(when (file-exists-p persist-file)
(delete-file persist-file)
--
2.32.0
Re: org-persist cache for remote files
Lele Gaifax writes: >> + :group 'org-persist >> + :type '(choice (const :tag "Never" nil) >> + (const :tag "Always" t) >> + (number :tag "Keep note more than X files") > > Now that you explained the meaning, maybe better to be explicit here, > mentioning "X remote files"? Sigh, re-reading, I see one more glitch there: s/Keep note/Keep not/ I'm sorry, but real work swallowed all my day, and could not try this out yet. Hopefully I will be able to do some test tomorrow morning. bye, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929.
Re: org-persist cache for remote files
Ihor Radchenko writes: > Lele Gaifax writes: > >> In the meanwhile, here below some notes: > > Thanks! See the updated patch. Great, you're so fast! :-) I couldn't test yet, but I saw a couple of other glitches, sorry for not noticing first. > ... > +(defcustom org-persist-remote-files 100 > + "Whether to keep persistent data for remote files. > + > +When this variable is nil, never save persitent data associated with > +remote files. When `t', always keep the data. When > +`check-existence', contact remote server containing the file and only > +keep the data when the file exists on the server. When a number, keep > +up to that number persistent values for remote files. > ... s/persitent/persistent > +Note that the last option `check-existence' may cause Emacs to show > +password prompts to log in." I'd remove "last" above, since `check-existence' is not the last (described) option anymore. > + :group 'org-persist > + :type '(choice (const :tag "Never" nil) > + (const :tag "Always" t) > + (number :tag "Keep note more than X files") Now that you explained the meaning, maybe better to be explicit here, mentioning "X remote files"? Thanks, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929.
Re: org-persist cache for remote files
Lele Gaifax writes:
> In the meanwhile, here below some notes:
Thanks! See the updated patch.
Best,
Ihor
>From 92878fab54c26800562409f3c686f065b7523a61 Mon Sep 17 00:00:00 2001
Message-Id: <92878fab54c26800562409f3c686f065b7523a61.1639838789.git.yanta...@gmail.com>
From: Ihor Radchenko
Date: Sat, 18 Dec 2021 22:14:24 +0800
Subject: [PATCH] org-persist.el: Customise persistence of data for remote
files
* lisp/org-persist.el (org-persist-remote-files): New defcustom
defining strategy for persisting data associated with remote files.
Separate strategy is necessary to avoid unexpected password promtps
while garbage collecting the cache: remote files may require server
login.
(org-persist-gc): Decide whether to store remote file data according
to `org-persist-remote-files'.
Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u
---
lisp/org-persist.el | 42 +++---
1 file changed, 35 insertions(+), 7 deletions(-)
diff --git a/lisp/org-persist.el b/lisp/org-persist.el
index 90f9a3bc9..4c0673c72 100644
--- a/lisp/org-persist.el
+++ b/lisp/org-persist.el
@@ -57,6 +57,23 @@ (defcustom org-persist-directory (expand-file-name
:group 'org-persist
:type 'directory)
+(defcustom org-persist-remote-files 100
+ "Whether to keep persistent data for remote files.
+
+When this variable is nil, never save persitent data associated with
+remote files. When `t', always keep the data. When
+`check-existence', contact remote server containing the file and only
+keep the data when the file exists on the server. When a number, keep
+up to that number persistent values for remote files.
+
+Note that the last option `check-existence' may cause Emacs to show
+password prompts to log in."
+ :group 'org-persist
+ :type '(choice (const :tag "Never" nil)
+ (const :tag "Always" t)
+ (number :tag "Keep note more than X files")
+ (const :tag "Check if exist on remote" 'check-existence)))
+
(defvar org-persist-index-file "index"
"File name used to store the data index.")
@@ -287,15 +304,26 @@ (defun org-persist-read-all-buffer ()
(defun org-persist-gc ()
"Remove stored data for not existing files or unregistered variables."
- (let (new-index)
+ (let (new-index (remote-files-num 0))
(dolist (index org-persist--index)
- (let ((file (plist-get index :path))
-(persist-file (when (plist-get index :persist-file)
-(org-file-name-concat
- org-persist-directory
- (plist-get index :persist-file)
+ (let* ((file (plist-get index :path))
+ (file-remote (when file (file-remote-p file)))
+ (persist-file (when (plist-get index :persist-file)
+ (org-file-name-concat
+ org-persist-directory
+ (plist-get index :persist-file)
(when (and file persist-file)
- (if (file-exists-p file)
+ (when (and file-remote persist-file)
+(cl-incf remote-files-num))
+ (if (if (not file-remote)
+ (file-exists-p file)
+(pcase org-persist-remote-files
+ ('t t)
+ ('check-existence
+ (file-exists-p file))
+ ((pred #'numberp)
+ (<= org-persist-remote-files remote-files-num))
+ (_ nil)))
(push index new-index)
(when (file-exists-p persist-file)
(delete-file persist-file)
--
2.32.0
Re: org-persist cache for remote files
Ihor Radchenko writes: > Lele Gaifax writes: > >>> Maybe we should better make this a user option? >> >> Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes >> is enabled, the cleanup procedure would ignore remote Org documents and a new >> explicit (interactive) org-persist-gc-remotes would take care of them. >> >>> Or we can go a different route and just not garbage collect remote files >>> until cache gets too large. >> >> Yes, a threshold would work too. > > Can you test the attached 2 patches? They do not seem to break local > files, but I do not have remote file caches to test on. Yes, I'll be able to do some test in a couple of hours. Thank you! In the meanwhile, here below some notes: > ... > Subject: [PATCH] org-persist.el: Customise persistence of data for remote > files > > * lisp/org-persist.el (org-persist-remote-files): New defcustom > defining strategy for persisting data associated with remote files. > Separate strategy is necessary to avoid unexpected password promtps > while garbage collecting the cache: remote files may require server > login. > ... s/promtps/prompts > ... > +(defcustom org-persist-remote-files 100 > + "Whether to keep persistent data for remote files. > + > +When this variable is nil, never save persitent data associated with > +remote files. When `t', always keep the data. When > +`check-existance', contact remote server containing the file and only > +keep the data when the file exists on the server. s/check-existance/check-existence Also, shouldn't the doc mention the meaning of the numberp, expecially since that's the default? > +Note that the last option `check-existance' may cause Emacs to show > +password prompts to log in." Thanks again, bye, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929.
Re: org-persist cache for remote files
Lele Gaifax writes: >> Maybe we should better make this a user option? > > Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes > is enabled, the cleanup procedure would ignore remote Org documents and a new > explicit (interactive) org-persist-gc-remotes would take care of them. > >> Or we can go a different route and just not garbage collect remote files >> until cache gets too large. > > Yes, a threshold would work too. Can you test the attached 2 patches? They do not seem to break local files, but I do not have remote file caches to test on. Best, Ihor >From 592608d3c48cb1e63f9b4427576f0698ca190e9a Mon Sep 17 00:00:00 2001 Message-Id: <592608d3c48cb1e63f9b4427576f0698ca190e9a.1639837163.git.yanta...@gmail.com> From: Ihor Radchenko Date: Sat, 18 Dec 2021 22:13:24 +0800 Subject: [PATCH] org-persist.el: Make org-persist-directory a custom variable * lisp/org-persist.el (org-persist): Create org-persist customisation group. (org-persist-directory): Convert into defcustom. --- lisp/org-persist.el | 34 +- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index f06d4fc61..90f9a3bc9 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -35,19 +35,27 @@ (declare-function org-back-to-heading "org" ( invisible-ok)) (declare-function org-next-visible-heading "org" (arg)) (declare-function org-at-heading-p "org" ( invisible-not-ok)) -(defvar org-persist-directory (expand-file-name -(org-file-name-concat - (let ((cache-dir (when (fboundp 'xdg-cache-home) -(xdg-cache-home - (if (or (seq-empty-p cache-dir) - (not (file-exists-p cache-dir)) - (file-exists-p (org-file-name-concat - user-emacs-directory - "org-persist"))) - user-emacs-directory - cache-dir)) - "org-persist/")) - "Directory where the data is stored.") + +(defgroup org-persist nil + "Persistent cache for Org mode." + :tag "Org persist" + :group 'org) + +(defcustom org-persist-directory (expand-file-name + (org-file-name-concat +(let ((cache-dir (when (fboundp 'xdg-cache-home) + (xdg-cache-home + (if (or (seq-empty-p cache-dir) + (not (file-exists-p cache-dir)) + (file-exists-p (org-file-name-concat + user-emacs-directory + "org-persist"))) + user-emacs-directory +cache-dir)) +"org-persist/")) + "Directory where the data is stored." + :group 'org-persist + :type 'directory) (defvar org-persist-index-file "index" "File name used to store the data index.") -- 2.32.0 >From de89eca758ca1972d75c8830aee224376a1e364e Mon Sep 17 00:00:00 2001 Message-Id: From: Ihor Radchenko Date: Sat, 18 Dec 2021 22:14:24 +0800 Subject: [PATCH] org-persist.el: Customise persistence of data for remote files * lisp/org-persist.el (org-persist-remote-files): New defcustom defining strategy for persisting data associated with remote files. Separate strategy is necessary to avoid unexpected password promtps while garbage collecting the cache: remote files may require server login. (org-persist-gc): Decide whether to store remote file data according to `org-persist-remote-files'. Reported in https://list.orgmode.org/87bl1eilt6.fsf@localhost/T/#u --- lisp/org-persist.el | 41 ++--- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/lisp/org-persist.el b/lisp/org-persist.el index 90f9a3bc9..613290afc 100644 --- a/lisp/org-persist.el +++ b/lisp/org-persist.el @@ -57,6 +57,22 @@ (defcustom org-persist-directory (expand-file-name :group 'org-persist :type 'directory) +(defcustom org-persist-remote-files 100 + "Whether to keep persistent data for remote files. + +When this variable is nil, never save persitent data associated with +remote files. When `t', always keep the data. When +`check-existance', contact remote server containing the file and only +keep the data when the file exists on the server. + +Note that the last option `check-existance' may cause Emacs to show +password prompts to log in." + :group 'org-persist + :type '(choice (const :tag "Never" nil) + (const :tag "Always" t) + (number :tag "Keep note more than X files") + (const :tag "Check if exist on remote" 'check-existence))) + (defvar org-persist-index-file "index" "File name used to
Re: org-persist cache for remote files
Ihor Radchenko writes: > Lele Gaifax writes: > >> I wonder if there is some mechanism I could use to either prevent caching >> of non-local documents or to avoid the check on existence in the >> org-persist-gc. > > I can easily make a change that always garbage-collect non-local > documents without trying to connect to remote server. This can be one > solution of your problem. However, I am not sure if it is the best > solution. > > May I know more about your use cases? Do you frequently open large > remote Org files? If you do, trashing their cache may not be optimal. I do not open remote Org files very often, but surely happens now and then, typically from my laptop to see/edit something I keep on my desktop PC. > Maybe we should better make this a user option? Yes, that's what I had in mind: when (say) org-persist-do-not-auto-gc-remotes is enabled, the cleanup procedure would ignore remote Org documents and a new explicit (interactive) org-persist-gc-remotes would take care of them. > Or we can go a different route and just not garbage collect remote files > until cache gets too large. Yes, a threshold would work too. Thanks a lot, ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. l...@metapensiero.it | -- Fortunato Depero, 1929.
