Re: when does :cache not cache?
Please ignore. Restarting Emacs fixed the problem. For some reason, re-loading org was not enough. I can export without error again. -- : Eric S Fraga, with org release_9.5.4-613-gb3b17c in Emacs 29.0.50
Re: when does :cache not cache?
"Fraga, Eric" writes: > I updated org recently (and again just now) and am getting the error > shown in the attached file. This is related to your patch, I think > given that it fails in accessing the "stable-file", but haven't verified > it. Before I investigate further, I thought I would raise the issue. Can you check the value of org-babel-temporary-stable-directory and its docstring? Best, Ihor
Re: when does :cache not cache?
Hi Ihor,
On Monday, 4 Jul 2022 at 21:06, Ihor Radchenko wrote:
> Ihor Radchenko writes:
>
>> The patch introduces a new functionality to ob-core.el allowing more
>> stable temporary file names. I am not sure if my implementation is the
>> best way to solve the problem, so comments are welcome.
>
> Applied onto main via 080462198.
I updated org recently (and again just now) and am getting the error
shown in the attached file. This is related to your patch, I think
given that it fails in accessing the "stable-file", but haven't verified
it. Before I investigate further, I thought I would raise the issue.
Thank you,
eric
--
Eric S Fraga, @ericsfraga:matrix.org, GnuPG: 0xc89193d8fffcf67d
Debugger entered--Lisp error: (wrong-type-argument stringp nil)
file-exists-p(nil)
org-babel-temp-stable-file(((:cache . "yes") (:colname-names (plotdata "PSO"
"" "PPA" "" "GA" "")) (:eval . "no-export") (:exports . "results") (:file .
"images/permdbetaevolutionplot.png") (:hlines . "no") (:noweb . "yes")
(:result-params "replace" "file") (:result-type . value) (:results . "replace
file") (:rowname-names) (:session) (:tangle . "no") (:var plotdata [...] ))
"gnuplot-")
#f(compiled-function (pair) # [...])
org-babel-gnuplot-process-vars([...])
org-babel-expand-body:gnuplot([...])
org-babel-sha1-hash([...])
org-babel-exp-src-block((src-block (:language "gnuplot" :switches nil
:parameters nil :begin 68881 :end 69528 :number-lines nil :preserve-indent nil
:retain-labels t :use-labels t :label-fmt nil :value " reset\n set logscale
xy\n set xlabel \"n_f\"\n set..." :post-blank 1 :post-affiliated 69077 :name
"permdbetaplotevolution" :header (":var plotdata=permdbetaresultstable :file
\"images/..." ":results file" ":exports results" ":eval no-export") :mode
planning :granularity element :org-element--cache-sync-key (55 . 70525) :cached
t :parent (section (:begin 68881 :end 79730 :contents-begin 68881 :contents-end
79679 :robust-begin 68881 :robust-end 79677 :post-blank 51 :post-affiliated
68881 :mode section :granularity element :org-element--cache-sync-key (55 .
70524) :cached t :parent (headline (:raw-value "results" :begin 68810 :end
79730 :pre-blank 0 :contents-begin 68881 :contents-end 79679 :robust-begin
68883 :robust-end 79677 :level 3 :priority nil :tags (...) :todo-keyword nil
:todo-type nil :post-blank 51 :footnote-section-p nil :archivedp nil
:commentedp nil :post-affiliated 68810 :title "results" :mode nil :granularity
element :org-element--cache-sync-key (55 . 68551) :cached t :parent (headline
...
org-babel-exp-process-buffer()
org-export-as(latex nil nil nil (:output-file "book.tex"))
org-export-to-file(latex "book.tex" nil nil nil nil nil org-latex-compile)
org-latex-export-to-pdf(nil nil nil nil)
(org-open-file (org-latex-export-to-pdf nil s v b))
(if a (org-latex-export-to-pdf t s v b) (org-open-file
(org-latex-export-to-pdf nil s v b)))
(lambda (a s v b) (if a (org-latex-export-to-pdf t s v b) (org-open-file
(org-latex-export-to-pdf nil s v b(nil nil nil nil)
org-export-dispatch(nil)
funcall-interactively(org-export-dispatch nil)
command-execute(org-export-dispatch)
Re: when does :cache not cache?
Ihor Radchenko writes: > The patch introduces a new functionality to ob-core.el allowing more > stable temporary file names. I am not sure if my implementation is the > best way to solve the problem, so comments are welcome. Applied onto main via 080462198. Best, Ihor
Re: when does :cache not cache?
On Thursday, 9 Jun 2022 at 21:35, Ihor Radchenko wrote: > Did you have a chance to test the patch? No. Sorry. Got busy with other things and had temporarily (well, permanently) solved the problem (for me) by adding :eval no-export to all of my gnuplot blocks. -- : Eric S Fraga, with org release_9.5.4-521-g1105da in Emacs 29.0.50
Re: when does :cache not cache?
Hi Eric, Eric S Fraga writes: > I will try your patch later this week (busy with end of term teaching > aspects at the moment). Did you have a chance to test the patch? Best, Ihor
Re: when does :cache not cache?
Dear Ihor, > The reason why caching does not work is related to the way :var > assignments work in ob-gnuplot. We dump the table data into temporary > files and refer to those files in generated gnuplot script body. The > temporary files names change on every execution and the gnuplot script > hash is never going to be the same. Ah, this makes perfect sense! Thank you for looking into this. I will try your patch later this week (busy with end of term teaching aspects at the moment). -- : Eric S Fraga, with org release_9.5.1-243-gad53c5 in Emacs 29.0.50 : Latest paper written in org: https://arxiv.org/abs/2106.05096
Re: when does :cache not cache?
Eric S Fraga writes: > in the paper I am writing, I have a number of gnuplot src blocks, some > of which process a significant amount of data so take some time to > generate the actual plots. The data are static so caching the results > make sense. However, even though I have ":cache yes" on each of these > named src blocks, and I have (for good measure), the property > "header-args:gnuplot" set to ":cache yes" as well, the plots are being > regenerated each time I export to PDF via LaTeX. I was able to reproduce. The reason why caching does not work is related to the way :var assignments work in ob-gnuplot. We dump the table data into temporary files and refer to those files in generated gnuplot script body. The temporary files names change on every execution and the gnuplot script hash is never going to be the same. I am attaching tentative patch to fix the issue. The patch introduces a new functionality to ob-core.el allowing more stable temporary file names. I am not sure if my implementation is the best way to solve the problem, so comments are welcome. Best, Ihor >From ac11b4d08edd577b29a398296364b4340096a6ae Mon Sep 17 00:00:00 2001 Message-Id: From: Ihor Radchenko Date: Sun, 12 Dec 2021 15:31:35 +0800 Subject: [PATCH] ob-gnuplot.el: Make :cache argument work with :var assignments * lisp/ob-core.el (org-babel-temporary-stable-directory): New variable holding a temporary directory name that does not change between Emacs sessions. (org-babel-remove-temporary-stable-directory): New function removing `org-babel-temporary-stable-directory' on Emacs shutdown. (org-babel-temp-stable-file): Generate stable temporary file name for object storage. The file name is constant with for equal objects. (org-babel-execute-src-block): Explicitly identify that if the result is cached. * lisp/ob-gnuplot.el (org-babel-gnuplot-process-vars): Make use of `org-babel-stable-file' to make expanded body stable with respect to :var assignments. Fixes https://orgmode.org/list/87mtn1o5mn@ucl.ac.uk --- lisp/ob-core.el| 52 +- lisp/ob-gnuplot.el | 5 - 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/lisp/ob-core.el b/lisp/ob-core.el index 7a9467b0e..d572423b7 100644 --- a/lisp/ob-core.el +++ b/lisp/ob-core.el @@ -735,7 +735,8 @@ (defun org-babel-execute-src-block ( arg info params) (forward-line) (skip-chars-forward " \t") (let ((result (org-babel-read-result))) - (message (replace-regexp-in-string "%" "%%" (format "%S" result))) + (message (format "Cached: %s" + (replace-regexp-in-string "%" "%%" (format "%S" result result))) ((org-babel-confirm-evaluate info) (let* ((lang (nth 0 info)) @@ -3112,6 +3113,22 @@ (defvar org-babel-temporary-directory Used by `org-babel-temp-file'. This directory will be removed on Emacs shutdown.")) +(defvar org-babel-temporary-stable-directory) +(unless (or noninteractive (boundp 'org-babel-temporary-stable-directory)) + (defvar org-babel-temporary-stable-directory +(or (and (boundp 'org-babel-temporary-stable-directory) + (file-exists-p org-babel-temporary-stable-directory) + org-babel-temporary-stable-directory) +(condition-case nil +(make-directory + (expand-file-name + "babel-stable" + (temporary-file-directory))) + (t nil))) +"Directory to hold temporary files created to execute code blocks. +Used by `org-babel-temp-file'. This directory will be removed on +Emacs shutdown.")) + (defcustom org-babel-remote-temporary-directory "/tmp/" "Directory to hold temporary files on remote hosts." :group 'org-babel @@ -3155,6 +3172,30 @@ (defun org-babel-temp-file (prefix suffix) temporary-file-directory))) (make-temp-file prefix nil suffix +(defun org-babel-temp-stable-file (data prefix suffix) + "Create a temporary file in the `org-babel-remove-temporary-stable-directory'. +The file name is stable with respect to DATA. The file name is +constructed like the following: PREFIXDATAhashSUFFIX." + (if (file-remote-p default-directory) + (let* ((prefix + (concat (file-remote-p default-directory) + (expand-file-name + prefix org-babel-temporary-stable-directory))) + (path (concat prefix (format "%s" (sxhash data)) (or suffix "" +(with-temp-file path) +path) +(let* ((temporary-file-directory + (or (and (boundp 'org-babel-temporary-stable-directory) + (file-exists-p org-babel-temporary-stable-directory) + org-babel-temporary-stable-directory) + temporary-file-directory)) + (path (concat + (expand-file-name + prefix org-babel-temporary-stable-directory) + (format "%s" (sxhash data)) (or suffix "" + (with-temp-file path) + path))) + (defun
Re: when does :cache not cache?
On Sunday, 24 Oct 2021 at 13:49, Emmanuel Charpentier wrote: > Workaround : cache the computations,not the plotting itself (which > should be fast,and must be made on every table, anyway...) : actually, the problem in my case is that the plotting *is* the expensive part! In case, problem solved/avoided by using ":eval never-export". thank you, eric -- : Eric S Fraga via Emacs 28.0.60, Org release_9.5-163-g4eab5b : Latest paper written in org: https://arxiv.org/abs/2106.05096
Re: when does :cache not cache?
Hello all, So, following the code: every time I ask to evaluate the gnuplot src block in the minimal example I posted yesterday, the new-hash that is calculated is different so the cache setting is ignored. Digging deeper leads to me getting lost. The hash does depend on the incorporation of the variables set using :var but I do not see why the hash should be different if the table has not changed. Anyway, that's as far as I got. For the record, I have simply put ":eval never-export" on all of my gnuplot src blocks and the export happens much more quickly now. eric -- : Eric S Fraga via Emacs 28.0.60, Org release_9.5-163-g4eab5b : Latest paper written in org: https://arxiv.org/abs/2106.05096
Re: when does :cache not cache?
On Saturday, 23 Oct 2021 at 11:13, Emmanuel Charpentier wrote: > You may find this John Kitchin's blog > post [...] illuminating. Unfortunately not but thank you. It is a very useful blog post but doesn't cover my case, where I use a :var header to refer to a table. The table is static so I still do not understand why :cache on the src block that references it doesn't work. I may have to dive into the code, I guess. I may be some time... ;-) -- : Eric S Fraga via Emacs 28.0.60, Org release_9.5-163-g4eab5b : Latest paper written in org: https://arxiv.org/abs/2106.05096
