Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3

[Ihor Radchenko]

Pedro  writes:

> I am experiencing it since end of July 2023 when I upgraded to the 
> version 9.6.15, but my situation now is stable with that workaround and 
> I have no time now for reporting the problem right now.

FYI, we did have a couple of fixes related to cache in Org 9.7.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3

[Pedro]

Thanks Ihor

I am experiencing it since end of July 2023 when I upgraded to the 
version 9.6.15, but my situation now is stable with that workaround and 
I have no time now for reporting the problem right now.


I expect to have some time at the end of July 2024, and I hope to 
upgrade again orgmode and/or emacs to the latest stable version, then I 
will try again to see if I stop getting cache errors without regularly 
running =(org-element-cache-reset)=


Cheers,
pinmacs


OpenPGP_0x9D64597C3A982DCA.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3

[Ihor Radchenko]

Pedro  writes:

> On 6/22/24 10:32, Ihor Radchenko wrote:
>> - Tim continued his struggle with an elusive bug he experiences when
>>setting tags (C-c C-q) in large Org buffers - setting tags takes
>>forever for him. (There is infinite loop likely lurking somewhere)
>
> I have the same bug, but I said nothing because I don't have right now 
> time to report. When I read it, I just wanted to share that this also 
> happens to me.

To be clear, I have no idea what kind of bug Tim experiences. I just
had a _guess_. It may or may not be the same bug.

Same for your case. I cannot tell without seeing the details.

> I work with huge diary files and I discovered that since the cache is 
> activated by default, when I remove or move certain subtrees/parts looks 
> like makes the cache corrupted...

Did it start happening recently?
If you think that the cache is corrupted, please add
(setq org-element--cache-self-verify t) to you config and watch the
warnings.

>  ... not 
>  only affects tag completion, that yeah, it takes forever, it also works 
>  weird on org-captures applied to huge buffer files

Again, please report these things separately.
I can see that you have problems, but I cannot help you without seeing
more details.

>  From user-view perspective, it was uncomfortable to see it enabled by 
> default, specially when you have problems, specially, when you try to 
> deal with a solution, and is not really documented on the workarounds, 
> etc [0].

The cache was enabled by default after I (0) rewrote the old buggy
implementation (1) enabled the new version in dev builds (Org 9.6-pre)
and fixed all the reported bugs; (2) released Org 9.6 with
org-element--cache-self-verify set to t and fixed the inflow of all the
reports about corrupted cache; (3) disabled cache verification to nil
when the reports stopped coming.

So, at this point, cache issues are extremely rare. (and note how your
[0] link is from a 6-years-old post; I rewrote that old cache version)

> ... It took me nearly 2 hours to stabilize again my workflow so I 
> agree that some information should be explained as stated in a recent 
> conversation [1]

That conversation is tangent. It is not about cache being enabled. It is
about various caches (not just org-element) being saved to disk.

> My vicious workflow regretably involves opening a lot of duplicated 
> buffers through emacsclient, and at some point, it becomes slow, so I do 
> this [2]

>     ;; after doing stuff, cache goes slow, but with this, it improves
>     ;; so this is arbitrary program, that my brain has associated with

Please report this as a bug separately. Ideally, with a reproducer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3

[Pedro]

On 6/22/24 10:32, Ihor Radchenko wrote:

- Tim continued his struggle with an elusive bug he experiences when
   setting tags (C-c C-q) in large Org buffers - setting tags takes
   forever for him. (There is infinite loop likely lurking somewhere)


I have the same bug, but I said nothing because I don't have right now 
time to report. When I read it, I just wanted to share that this also 
happens to me.


I work with huge diary files and I discovered that since the cache is 
activated by default, when I remove or move certain subtrees/parts looks 
like makes the cache corrupted (I could not reproduce exactly how), not 
only affects tag completion, that yeah, it takes forever, it also works 
weird on org-captures applied to huge buffer files. So I have a current 
diary, that tries to get a record of last weeks. When I could not 
archive it (like right now), it has 5 weeks from now, that means 24k 
lines in a file, when I can clean it up, goes to an archive file (so 
then, alleviates org-agenda-files processing), the 2023, which is pretty 
archived, is 263k lines.


From user-view perspective, it was uncomfortable to see it enabled by 
default, specially when you have problems, specially, when you try to 
deal with a solution, and is not really documented on the workarounds, 
etc [0]. It took me nearly 2 hours to stabilize again my workflow so I 
agree that some information should be explained as stated in a recent 
conversation [1]


My vicious workflow regretably involves opening a lot of duplicated 
buffers through emacsclient, and at some point, it becomes slow, so I do 
this [2]


Being said all, I am very happy with how emacs and orgmode are getting 
progress done on managing large files, etc. I just wanted to let you 
know about a problem that I am experimenting, and maybe other people too.


I will note in my agenda on the #9 meeting [3] and I will try to join

Cheers,
pinmacs

[0] In my notes it says:

#+name: sourceblock_created_2024-06-27_14-32-44
#+begin_src text
how to disable Element cache? I am having warnings but I don't have time 
to do bug reporting ( https://orgmode.org/Changes.html )


found the solution (setq org-element-use-cache nil)  src 
https://emacs.stackexchange.com/questions/42006/trouble-with-org-mode-cache-find-error


[ I don't remember where I got this, but I think it should appear in the 
manual saying that this is a workaround until it is fixed 
https://orgmode.org/org.html ]

org-element-cache-reset
#+end_src

[1] email Subject: Please document the caching and its user options, 
From: Eli Zaretskii, Date: 2024-06-12 11:38:05 AM CEST


[2]

#+name: sourceblock_created_2024-06-27_14-33-28
#+begin_src emacs-lisp
;; I use this program when it works slow
(defun kill-duplicate-frames ()
  "Kills duplicate frames of buffers."
  (interactive)
  (let ((buffers '()))
    (dolist (frame (frame-list))
  (with-selected-frame frame
    (let ((buffer (current-buffer)))
  (when (member buffer buffers)
    (delete-frame))
  (add-to-list 'buffers buffer)
  ;; after doing stuff, cache goes slow, but with this, it improves
  ;; so this is arbitrary program, that my brain has associated with 
"make emacs faster again"

  (org-element-cache-reset)
  (message "Killed duplicate frames of buffers."))
#+end_src

[3] email Subject: #9 ((bbb:OrgMeetup)) on Wed, July 10, 19:00 UTC+3, 
From: Ihor Radchenko , Date: 2024-06-26 5:48:46 PM CEST


OpenPGP_0x9D64597C3A982DCA.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

[SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3

[Ihor Radchenko]

It was a long discussion this time (almost 3 hours).
We ended up talking in-depth about Org mode development and some very
technical aspects of it.

- As usual, we started from posting Sacha's News, but with a bit of a twist
  - Since recently, https://orgmode.org shows Org mode-related news on the 
front page
- Mailing list feature discussions, polls, and announcements
- Org mode section of Sacha's News

- Suhail asked about the status of 3 bugs reports posted on Org mailing list 
recently

  - https://yhetil.org/orgmode/87h6e134sp@gmail.com/
- This one is about completion of :async header argument in
  "shell" blocks, where "shell" is not literally shell, but
  "bash", "csh", etc. The completion only works for "shell"
  blocks, but not for other blocks.
  - This is simply an omission in ob-shell.el
  - ob-shell is implemented in the following way
1. Generic "shell" backend is defined
2. Other backends like "bash", "csh", "fish" (full list in
   ~org-babel-shell-names~) are defined based on the generic
   backend in ~org-babel-shell-initialize~
3. ~org-babel-shell-initialize~ fails to define
   ~org-babel-header-args:~ - variable used to
   compute completions.
4. The fix is easy; just need time to get there.
   - [2024-06-22 Sat] Fixed.
 - 
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=e60c

  - https://yhetil.org/orgmode/87cyop3445@gmail.com/
- This is a similar issue related to ~org-babel-shell-initialize~
- This time, ~org-babel--initiate-session~ is not
  defined for specific shells
- The problem is more severe than completion though. It renders
  =C-u C-c '= (~org-edit-special~) unusable.
  - ~org-edit-special~ /without/ prefix argument opens code block at point 
for editing
  - ~org-edit-special~ /with/ prefix argument should instead open
associated session for code block with =:session ...= header arg
- Because of the bug, this does not currently work for shell blocks
  - [2024-06-22 Sat] Fixed.
- 
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5b366a73
- 
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5e9ac146

  - https://yhetil.org/orgmode/87cyonhuq3@gmail.com/
- This report is about regression of =#+bind: ...= keywords in the new Org 
9.7
- Fixed. 
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=96113f3b5

- Karl Voit (publicvoit) shared his recent bug report for helm-org-rifle

  - https://github.com/alphapapa/org-rifle/issues/85

  - The problem he experiences is with parts of outline path being
invisible in the candidates buffer as long as they are folded

  - The likely reason with the way folding is done in Org mode 9.6.
Org 9.6 (and Org 9.7 for Emacs <29) use 'invisible text property
to fold headings. If such folded headings are copied verbatim from
the Org buffer, they may remain invisible.
- If my guess is right, setting ~org-fold-core-style~ to ~overlays~
  should make things work again in helm-org-rifle

  - Note that Org 9.7 now moves towards restoring the old approach -
using overlays to fold staff. This is because built-in isearch.el,
query-replace-regexp, and a number of external packages (like
evil) only support searching (and revealing!) invisible text when
it is hidden via overlays. (I was hoping that it can be possible
to work around such limitation, but constant stream of bug reports
and growing pile of fragile workarounds proved that text hidden
via text properties cannot reliably searched in practice; alas...)
- As an unfortunate side effect of this, I had to revert feature
  introduced in Org 9.6 - searching inside hidden parts of the
  links
- See https://orgmode.org/Changes.html (It is no longer possible
  to reveal hidden parts of the links during isearch)


- We went on talking about org-ql vs. helm-org-rifle

  - org-ql is generally more flexible as it allows more than just
plain text + outline search
- In org-ql, one can quickly match headings by specific
  properties, tags, timestamps, etc
- In addition, org-ql, since recently, has integration with
  built-in completion frameworks, and their extensions like vertico.
- Of course, helm interface is also there - helm-org-ql

  - However, as it turns out, org-ql lacks one important feature Karl
particularly likes.

In helm-org-rifle, when you enter search string interactively, if
the match is inside a heading, the line containing match is
displayed in addition to the outline path to the heading. This
provides a valuable additional context - you can briefly look what
exactly inside a heading is matched.

- [2024-06-15 Sat] alphapapa, in
  https://github.com/alphapapa/org-rifle/issues/85, pointed out
  that one can