[BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
# Issue Starting emacs with emacs -Q, then navigating to a minimal example org file with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with nothing else running. Also, scrolling commands like C-v are often laggy after the initial hang. The minimal org file begins: """ #+bibliography: ~/cloud/library/lib.bib [cite:@tillyPrimaryAnaplasticLargeCell1997] """ with another 80 citations afterwards, one-by-line, but nothing else. As mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a non-existent or tiny bibtex file the problem goes away, and the in-buffer citations are rendered in a red face. Clearly from this and the below profile the issue is something to do with checking etc the citations for fontification purposes. # profile ## cpu 1,542,884,496 99% - redisplay_internal (C function) 1,542,860,504 99% - jit-lock-function 1,542,860,504 99% - jit-lock-fontify-now 1,542,860,504 99%- jit-lock--run-functions 1,542,860,504 99% - run-hook-wrapped 1,542,860,504 99% - # 1,542,860,504 99% - font-lock-fontify-region 1,542,860,504 99%- font-lock-default-fontify-region 1,542,792,728 99% - font-lock-fontify-keywords-region 1,542,661,211 99% - org-cite-activate 1,542,542,267 99% - org-cite-basic-activate 787,037,416 50%- org-cite-basic--get-entry 4,065,194 0% + org-cite-basic--parse-bibliography 754,769,872 48%- org-cite-basic--all-keys 6,151,200 0% + seq-mapcat 3,850,126 0% + org-cite-basic--parse-bibliography 481,332 0%+ org-element-interpret-data 129,376 0%+ org-cite-basic--print-entry 109,615 0%+ org-cite-get-references 87,264 0% org-element-citation-parser 5,988 0%org-activate-links 5,677 0%org-do-emphasis-faces 3,524 0%org-fontify-meta-lines-and-blocks 3,072 0% + org-activate-footnote-links 3,072 0%org-do-latex-and-related 7,392 0% - eval 7,392 0% if 3,072 0%kill-this-buffer-enabled-p 2,112 0% - tool-bar-make-keymap 2,112 0% - tool-bar-make-keymap-1 2,112 0%- mapcar 2,112 0% # 1,176 0%menu-bar-update-buffers 6,219,935 0% + command-execute 1,280 0% + timer-event-handler 24 0% + eldoc-schedule-timer 21 0% + # 0 0% ... ## memory 5778 95% - redisplay_internal (C function) 5778 95% - jit-lock-function 5778 95% - jit-lock-fontify-now 5778 95%- jit-lock--run-functions 5778 95% - run-hook-wrapped 5778 95% - # 5778 95% - font-lock-fontify-region 5778 95%- font-lock-default-fontify-region 5778 95% - font-lock-fontify-keywords-region 5774 95% - org-cite-activate 5762 95% - org-cite-basic-activate 2939 48%- org-cite-basic--get-entry 1631 26% - org-cite-basic--parse-bibliography 400 6% - set-auto-coding 400 6% find-auto-coding 20 0% + org-cite-list-bibliography-files 4 0% + # 4 0%after-insert-file-set-coding 2819 46%- org-cite-basic--all-keys 1515 25% - org-cite-basic--parse-bibliography 364 6% - set-auto-coding 364 6% find-auto-coding 16 0% - org-cite-list-bibliography-files 8 0% + org-collect-keywords 4 0% # 32 0% + seq-mapcat 4 0%+ org-cite-basic--print-entry 4 0% org-element-citation-parser 4 0%org-activate-links 233 3% - command-execute 218 3% - byte-code 149 2% + read-extended-command 69 1% - find-file-read-args 69 1%- read-file-name 69 1% - read-file-name-default 15 0% - completing-read-default 1 0% - command-execute 1 0%- funcall-interactively 1 0% - minibuffer-complete 1 0% - completion-in-region 1 0% - completion--in-region 1 0%+ # 15 0% - funcall-interactively 6 0% + find-file 5 0% + execute-extended-command 4 0% scroll-up-command 26 0% + ... 4 0% + timer-event-handler 4 0% + # #
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Thanks for posting this! For Nicolas et al, my more general question about performance with oc-basic was related to this, so you can ignore that. On Tue, Mar 15, 2022 at 1:01 PM Jamie Matthews wrote: > > # Issue > Starting emacs with emacs -Q, then navigating to a minimal example org file > with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with > nothing else running. Also, scrolling commands like C-v are often laggy after > the initial hang. > > The minimal org file begins: > """ > #+bibliography: ~/cloud/library/lib.bib > > [cite:@tillyPrimaryAnaplasticLargeCell1997] > """ > with another 80 citations afterwards, one-by-line, but nothing else. As > mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a > non-existent or tiny bibtex file the problem goes away, and the in-buffer > citations are rendered in a red face. Clearly from this and the below profile > the issue is something to do with checking etc the citations for > fontification purposes. > > # profile > > ## cpu > 1,542,884,496 99% - redisplay_internal (C function) > 1,542,860,504 99% - jit-lock-function > 1,542,860,504 99% - jit-lock-fontify-now > 1,542,860,504 99%- jit-lock--run-functions > 1,542,860,504 99% - run-hook-wrapped > 1,542,860,504 99% - # > 1,542,860,504 99% - font-lock-fontify-region > 1,542,860,504 99%- font-lock-default-fontify-region > 1,542,792,728 99% - font-lock-fontify-keywords-region > 1,542,661,211 99% - org-cite-activate > 1,542,542,267 99% - org-cite-basic-activate > 787,037,416 50%- org-cite-basic--get-entry > 4,065,194 0% + org-cite-basic--parse-bibliography > 754,769,872 48%- org-cite-basic--all-keys > 6,151,200 0% + seq-mapcat > 3,850,126 0% + org-cite-basic--parse-bibliography > 481,332 0%+ org-element-interpret-data > 129,376 0%+ org-cite-basic--print-entry > 109,615 0%+ org-cite-get-references > 87,264 0% org-element-citation-parser > 5,988 0%org-activate-links > 5,677 0%org-do-emphasis-faces > 3,524 0%org-fontify-meta-lines-and-blocks > 3,072 0% + org-activate-footnote-links > 3,072 0%org-do-latex-and-related > 7,392 0% - eval > 7,392 0% if > 3,072 0%kill-this-buffer-enabled-p > 2,112 0% - tool-bar-make-keymap > 2,112 0% - tool-bar-make-keymap-1 > 2,112 0%- mapcar > 2,112 0% # > 1,176 0%menu-bar-update-buffers > 6,219,935 0% + command-execute > 1,280 0% + timer-event-handler > 24 0% + eldoc-schedule-timer > 21 0% + # > 0 0% ... > > ## memory > > 5778 95% - redisplay_internal (C function) > 5778 95% - jit-lock-function > 5778 95% - jit-lock-fontify-now > 5778 95%- jit-lock--run-functions > 5778 95% - run-hook-wrapped > 5778 95% - # > 5778 95% - font-lock-fontify-region > 5778 95%- font-lock-default-fontify-region > 5778 95% - font-lock-fontify-keywords-region > 5774 95% - org-cite-activate > 5762 95% - org-cite-basic-activate > 2939 48%- org-cite-basic--get-entry > 1631 26% - org-cite-basic--parse-bibliography > 400 6% - set-auto-coding > 400 6% find-auto-coding > 20 0% + org-cite-list-bibliography-files >4 0% + # >4 0%after-insert-file-set-coding > 2819 46%- org-cite-basic--all-keys > 1515 25% - org-cite-basic--parse-bibliography > 364 6% - set-auto-coding > 364 6% find-auto-coding > 16 0% - org-cite-list-bibliography-files >8 0% + org-collect-keywords >4 0% # F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_23> > 32 0% + seq-mapcat >4 0%+ org-cite-basic--print-entry >4 0% org-element-citation-parser >4 0%org-activate-links > 233 3% - command-execute > 218 3% - byte-code > 149 2% + read-extended-command > 69 1% - find-file-read-args > 69 1%- read-file-name > 69 1% - read-file-name-default > 15 0% - completing-read-default >1 0% - command-execute >1 0%- func
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes: > # Issue > Starting emacs with emacs -Q, then navigating to a minimal example org file > with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with > nothing else running. Also, scrolling commands like C-v are often laggy > after the initial hang. > > The minimal org file begins: > """ > #+bibliography: ~/cloud/library/lib.bib > > [cite:@tillyPrimaryAnaplasticLargeCell1997] > """ > with another 80 citations afterwards, one-by-line, but nothing else. As > mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a > non-existent or tiny bibtex file the problem goes away, and the in-buffer > citations are rendered in a red face. Clearly from this and the below profile > the issue is something to do with checking etc the citations for > fontification purposes. > > # profile > > ## cpu > 1,542,542,267 99% - org-cite-basic-activate > 787,037,416 50%- org-cite-basic--get-entry > 4,065,194 0% + org-cite-basic--parse-bibliography > 754,769,872 48%- org-cite-basic--all-keys Could you try the following: 1. Execute the following code: (require 'elp) (elp-restore-all) (elp-instrument-function #'org-cite-basic--get-entry) (elp-instrument-function #'org-cite-basic-activate) (elp-instrument-function #'org-cite-basic--parse-bibliography) (elp-instrument-function #'org-cite-basic--all-keys) 2. Open your org file where Emacs hangs 3. Run M-x elp-results 4. Report the ELP buffer contents here Best, Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Hi Ihor,
I tried to run that code but get errors for the elp-instrument-function calls
(both with my config and emacs -Q):
```
Debugger entered--Lisp error: (error "ELP cannot profile the function:
org-cite-basic--g...")
error("ELP cannot profile the function: %s" org-cite-basic--get-entry)
elp-instrument-function(org-cite-basic--get-entry)
(progn (elp-instrument-function #'org-cite-basic--get-entry))
elisp--eval-last-sexp(nil)
eval-last-sexp(nil)
funcall-interactively(eval-last-sexp nil)
command-execute(eval-last-sexp)
```
Best,
Jamie
From: Ihor Radchenko
Sent: Saturday, March 19, 2022 8:28 AM
To: Jamie Matthews
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB
bibtex library [9.5.2 (9.5.2-g91681f @
/home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> # Issue
> Starting emacs with emacs -Q, then navigating to a minimal example org file
> with C-x C-f yields a ~10 second hang on an 8-core/16GB RAM machine with
> nothing else running. Also, scrolling commands like C-v are often laggy
> after the initial hang.
>
> The minimal org file begins:
> """
> #+bibliography: ~/cloud/library/lib.bib
>
> [cite:@tillyPrimaryAnaplasticLargeCell1997]
> """
> with another 80 citations afterwards, one-by-line, but nothing else. As
> mentioned in the title, the lib.bib file is ~10MB - if I swap this out for a
> non-existent or tiny bibtex file the problem goes away, and the in-buffer
> citations are rendered in a red face. Clearly from this and the below profile
> the issue is something to do with checking etc the citations for
> fontification purposes.
>
> # profile
>
> ## cpu
> 1,542,542,267 99% - org-cite-basic-activate
> 787,037,416 50%- org-cite-basic--get-entry
> 4,065,194 0% + org-cite-basic--parse-bibliography
> 754,769,872 48%- org-cite-basic--all-keys
Could you try the following:
1. Execute the following code:
(require 'elp)
(elp-restore-all)
(elp-instrument-function #'org-cite-basic--get-entry)
(elp-instrument-function #'org-cite-basic-activate)
(elp-instrument-function #'org-cite-basic--parse-bibliography)
(elp-instrument-function #'org-cite-basic--all-keys)
2. Open your org file where Emacs hangs
3. Run M-x elp-results
4. Report the ELP buffer contents here
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes: > Hi Ihor, > > I tried to run that code but get errors for the elp-instrument-function > calls (both with my config and emacs -Q): > > ``` > Debugger entered--Lisp error: (error "ELP cannot profile the function: > org-cite-basic--g...") Then also add (require 'oc-basic) before the code I provided. I seems that org-cite has not been loaded yet when you tried to execute the code. Best, Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Thanks: ``` org-cite-basic-activate 59 10.724349447 0.1817686346 org-cite-basic--parse-bibliography 129 10.559936049 0.0818599693 org-cite-basic--all-keys59 7.830202561 0.1327152976 org-cite-basic--get-entry 70 2.7772344940 0.0396747784 ``` From: Ihor Radchenko Sent: Saturday, March 19, 2022 9:23 AM To: Jamie Matthews Cc: emacs-orgmode@gnu.org Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)] Jamie Matthews writes: > Hi Ihor, > > I tried to run that code but get errors for the elp-instrument-function > calls (both with my config and emacs -Q): > > ``` > Debugger entered--Lisp error: (error "ELP cannot profile the function: > org-cite-basic--g...") Then also add (require 'oc-basic) before the code I provided. I seems that org-cite has not been loaded yet when you tried to execute the code. Best, Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> Thanks:
>
> ```
> org-cite-basic-activate 59 10.724349447 0.1817686346
> org-cite-basic--parse-bibliography 129 10.559936049 0.0818599693
> org-cite-basic--all-keys59 7.830202561 0.1327152976
> org-cite-basic--get-entry 70 2.7772344940 0.0396747784
> ```
org-cite-basic--parse-bibliography appears to be the main bottleneck.
I tried to write a quick fix (untested).
Can you try to redefine org-cite-basic--parse-bibliography to the
version below (note an extra defvar) and let me know how it goes:
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography (&optional info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file))
(unless (gethash file org-cite-basic--file-id-cache)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
It seems a very significant improvement - the lag on scrolling after the buffer
is loaded is not noticeable, and the initial loading hang essentially
disappears!
I'm slightly surprised that elp-results doesn't seem to show as dramatic an
improvement as it feels for me:
```
org-cite-basic-activate 123 6.5652978389 0.0533764051
org-cite-basic--all-keys123 6.2679872230 0.0509592457
org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436
org-cite-basic--get-entry 147 0.1550045990 0.0010544530
```
From: Ihor Radchenko
Sent: Saturday, March 19, 2022 9:57 AM
To: Jamie Matthews
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB
bibtex library [9.5.2 (9.5.2-g91681f @
/home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> Thanks:
>
> ```
> org-cite-basic-activate 59 10.724349447 0.1817686346
> org-cite-basic--parse-bibliography 129 10.559936049 0.0818599693
> org-cite-basic--all-keys59 7.830202561 0.1327152976
> org-cite-basic--get-entry 70 2.7772344940 0.0396747784
> ```
org-cite-basic--parse-bibliography appears to be the main bottleneck.
I tried to write a quick fix (untested).
Can you try to redefine org-cite-basic--parse-bibliography to the
version below (note an extra defvar) and let me know how it goes:
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography (&optional info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file))
(unless (gethash file org-cite-basic--file-id-cache)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> It seems a very significant improvement - the lag on scrolling after the
> buffer is loaded is not noticeable, and the initial loading hang essentially
> disappears!
Great! Do you mean that there is no apparent slowness at all?
> I'm slightly surprised that elp-results doesn't seem to show as dramatic an
> improvement as it feels for me:
>
> ```
> org-cite-basic-activate 123 6.5652978389 0.0533764051
> org-cite-basic--all-keys123 6.2679872230 0.0509592457
> org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436
> org-cite-basic--get-entry 147 0.1550045990 0.0010544530
> ```
It does. You need to look at the last column. The total time in the
second column is large probably because you interacted (scrolled?) with
the buffer a bit more than first time and the functions got called more
(see second column).
Also, can you try the following version of
org-cite-basic--parse-bibliography again and try to change cite key of
one of the cited entries of the bibliography outside emacs? The key
should be marked red if you try to insert the changed entry in Org.
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography (&optional info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library [9.5.2 (9.5.2-g91681f @ /home/jdm204/.config/emacs/straight/build/org/)]
I can confirm that the key turns red on insert when I altered the key outside
of emacs (with that second version of `org-cite-basic--parse-bibliography`).
However, I'm now noticing that the hang improvement earlier (< a second down
from ~10) doesn't always occur.
Specifically, if I
1. emacs -Q
2. eval your code in scratch
3. C-x C-f to the org file
I get the hang. However, if I then
1. kill the org buffer
2. eval the code again
3. re-find the org file
the hang is gone. Without evaling your code in between, killing the org buffer
and finding it again in the same emacs session reproduces the hang everytime,
which was probably what I did the first time I report the improvement, as in I
didn't check it worked from startup.
From: Ihor Radchenko
Sent: Saturday, March 19, 2022 10:28 AM
To: Jamie Matthews
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB
bibtex library [9.5.2 (9.5.2-g91681f @
/home/jdm204/.config/emacs/straight/build/org/)]
Jamie Matthews writes:
> It seems a very significant improvement - the lag on scrolling after the
> buffer is loaded is not noticeable, and the initial loading hang essentially
> disappears!
Great! Do you mean that there is no apparent slowness at all?
> I'm slightly surprised that elp-results doesn't seem to show as dramatic an
> improvement as it feels for me:
>
> ```
> org-cite-basic-activate 123 6.5652978389 0.0533764051
> org-cite-basic--all-keys123 6.2679872230 0.0509592457
> org-cite-basic--parse-bibliography 270 6.2513217860 0.0231530436
> org-cite-basic--get-entry 147 0.1550045990 0.0010544530
> ```
It does. You need to look at the last column. The total time in the
second column is large probably because you interacted (scrolled?) with
the buffer a bit more than first time and the functions got called more
(see second column).
Also, can you try the following version of
org-cite-basic--parse-bibliography again and try to change cite key of
one of the cited entries of the bibliography outside emacs? The key
should be marked red if you try to insert the changed entry in Org.
(defvar org-cite-basic--file-id-cache nil
"Hash table linking files to their hash.")
(defun org-cite-basic--parse-bibliography (&optional info)
"List all entries available in the buffer.
Each association follows the pattern
(FILE . ENTRIES)
where FILE is the absolute file name of the BibTeX file, and ENTRIES is a hash
table where keys are references and values are association lists between fields,
as symbols, and values as strings or nil.
Optional argument INFO is the export state, as a property list."
(unless (hash-table-p org-cite-basic--file-id-cache)
(setq org-cite-basic--file-id-cache (make-hash-table :test #'equal)))
(if (plist-member info :cite-basic/bibliography)
(plist-get info :cite-basic/bibliography)
(let ((results nil))
(dolist (file (org-cite-list-bibliography-files))
(when (file-readable-p file)
(with-temp-buffer
(when (or (file-has-changed-p file)
(not (gethash file org-cite-basic--file-id-cache)))
(insert-file-contents file)
(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
(let* ((file-id (cons file (gethash file
org-cite-basic--file-id-cache)))
(entries
(or (cdr (assoc file-id org-cite-basic--bibliography-cache))
(let ((table
(pcase (file-name-extension file)
("json" (org-cite-basic--parse-json))
("bib" (org-cite-basic--parse-bibtex
'biblatex))
("bibtex" (org-cite-basic--parse-bibtex
'BibTeX))
(ext
(user-error "Unknown bibliography extension:
%S"
ext)
(push (cons file-id table)
org-cite-basic--bibliography-cache)
table
(push (cons file entries) results)
(when info (plist-put info :cite-basic/bibliography results))
results)))
Best,
Ihor
