Re: [9.4] LOGBOOK visibility
> FWIW, I think Emacs has a branch (feature/noverlay) which has been > reported to improve performance with overlays. AFAICT it's just hanging > there waiting to be merged (though a naive "git merge" reveals some > conflicts); some quick Googling suggests it has last been discussed two > years ago: That branch is around for a very long time. The last attempt on it was back in December [1]. However, all the progress was only in tests, not in actual code. As I mentioned in the discussion about using properties instead of overlays, I prefer to modify org now rather than waiting for someone to finalise and merge that branch. Also, text properties appear to use less memory. Best, Ihor [1] https://lists.gnu.org/archive/html/emacs-devel/2019-12/msg00115.html Kévin Le Gouguec writes: > Thank you for taking the time to make this write-up. > > Nicolas Goaziou writes: > >> With overlays, you can't have your cake >> and eat it too. > > FWIW, I think Emacs has a branch (feature/noverlay) which has been > reported to improve performance with overlays. AFAICT it's just hanging > there waiting to be merged (though a naive "git merge" reveals some > conflicts); some quick Googling suggests it has last been discussed two > years ago: > > https://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00565.html > -- Ihor Radchenko, PhD, Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong University, Xi'an, China Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg
Re: [9.4] LOGBOOK visibility
Thank you for taking the time to make this write-up. Nicolas Goaziou writes: > With overlays, you can't have your cake > and eat it too. FWIW, I think Emacs has a branch (feature/noverlay) which has been reported to improve performance with overlays. AFAICT it's just hanging there waiting to be merged (though a naive "git merge" reveals some conflicts); some quick Googling suggests it has last been discussed two years ago: https://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00565.html
Re: [9.4] LOGBOOK visibility
Hello, Ihor Radchenko writes: > There used to be several types of overlays for headline folding, drawer > folding, and block folding. One of the recent commits made all the > overlays use headline type. As a result all the overlays are merged into > a single overlay upon folding. This was done to reduce the total number > of overlays present in an org buffer, which degrades Emacs performance > on huge org files (see the discussion in [1]). > > What you observe is a consequence of that change. Now, all the drawer > overlays are destroyed when you fold a heading. Isearch would > only see a single huge overlay and unfold it alltogether. For completeness, at the beginning drawers and outline shared the same invisibility value. At some point during 9.X development, I gave drawers their own invisibility value. It provided an overall nicer behaviour, but also introduced some serious slowdown in large files. Recently, as pointed out by Ihor, I switched drawers back to their initial state, i.e., they now share the same invisibility value as the outline. At least, Org on very large files is slightly more manageable, but the behaviour is less nice. With overlays, you can't have your cake and eat it too. > I am currently working on a patch to rewrite the whole folding system. > Your issue should disappear once it is applied. Yes, hopefully, switching to text properties will get us out of this sad situation. Regards, -- Nicolas Goaziou
Re: [9.4] LOGBOOK visibility
Ihor Radchenko writes: > (see the discussion in [1]). Ah, that makes sense! No idea how I managed to miss that thread. > I am currently working on a patch to rewrite the whole folding system. > Your issue should disappear once it is applied. Good to know! > Meanwhile, you may, for example, advice > org-fold--isearch-filter-predicate to re-fold drawers during isearch. Duly noted, thanks for the advice. And thank you and Nicolas for all the hard work!
Re: [9.4] LOGBOOK visibility
> Meanwhile, you may, for example, advice
> org-fold--isearch-filter-predicate to re-fold drawers during isearch.
org-fold--isearch-filter-predicate->isearch-filter-visible
Ihor Radchenko writes:
>> I haven't reached the bottom of this rabbit hole yet. Since I think
>> I've spent all the time I had to spend in this issue for the day, here's
>> where I'm at.
>
> There used to be several types of overlays for headline folding, drawer
> folding, and block folding. One of the recent commits made all the
> overlays use headline type. As a result all the overlays are merged into
> a single overlay upon folding. This was done to reduce the total number
> of overlays present in an org buffer, which degrades Emacs performance
> on huge org files (see the discussion in [1]).
>
> What you observe is a consequence of that change. Now, all the drawer
> overlays are destroyed when you fold a heading. Isearch would
> only see a single huge overlay and unfold it alltogether.
>
> I am currently working on a patch to rewrite the whole folding system.
> Your issue should disappear once it is applied.
>
> Meanwhile, you may, for example, advice
> org-fold--isearch-filter-predicate to re-fold drawers during isearch.
>
> Best,
> Ihor
>
> [1] https://www.mail-archive.com/emacs-orgmode@gnu.org/msg127740.html
>
> Kévin Le Gouguec writes:
>
>> I haven't reached the bottom of this rabbit hole yet. Since I think
>> I've spent all the time I had to spend in this issue for the day, here's
>> where I'm at.
>>
>> If I open my example file[1] with Org 9.3 and master (86fada6b5), and
>> compare the overlays covering the hidden part of the first LOGBOOK
>> drawer, like so:
>>
>> #+begin_src sh
>> #!/bin/bash
>>
>> set -eu
>>
>> point=67
>>
>> orgs=(
>> "${path_to_org_9_3}"
>> "${path_to_org_master}"
>> )
>>
>> for o in "${orgs[@]}"
>> do
>> emacs -Q -batch \
>> -L "${o}"/lisp \
>> -eval "(find-file \"logbooks.org\")" \
>> -eval "(message org-version)" \
>> -eval "(dolist (o (overlays-at ${point}))
>>(message \"from %s to %s\" (overlay-start o) (overlay-end
>> o))
>>(message \"%s\" (overlay-properties o)))"
>> echo
>> done
>> #+end_src
>>
>> Here is what I get:
>>
>>> 9.3
>>> from 32 to 2015
>>> (evaporate t invisible outline isearch-open-invisible #[lambda gibberish
>>> involving (org-show-context 'isearch)])
>>> from 67 to 262
>>> (isearch-open-invisible delete-overlay invisible org-hide-drawer evaporate
>>> t)
>>>
>>> 9.3.7
>>> from 32 to 2015
>>> (isearch-open-invisible delete-overlay invisible outline evaporate t)
>>
>> - The LOGBOOK overlay (from 67 to 262) vanished.
>>
>> - The isearch-open-invisible property of the "project 1" overlay (from
>> 32 to 2015) changed from a lambda[2] to just delete-overlay.
>>
>>
>> Again, not sure this shows there's an issue; for all I know Org 9.4 just
>> works differently and nothing's wrong there. I hope someone can chime
>> in and take it up from here; otherwise I'll resume my investigations
>> sometime later.
>>
>>
>> [1] https://orgmode.org/list/87eepuz0bj@gmail.com/2-logbooks.org
>>
>> [2] Which I guess is the value of
>> outline-isearch-open-invisible-function, set locally by org-mode in
>> the major mode function?
>>
>
> --
> Ihor Radchenko,
> PhD,
> Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
> State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong
> University, Xi'an, China
> Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg
--
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg
Re: [9.4] LOGBOOK visibility
> I haven't reached the bottom of this rabbit hole yet. Since I think
> I've spent all the time I had to spend in this issue for the day, here's
> where I'm at.
There used to be several types of overlays for headline folding, drawer
folding, and block folding. One of the recent commits made all the
overlays use headline type. As a result all the overlays are merged into
a single overlay upon folding. This was done to reduce the total number
of overlays present in an org buffer, which degrades Emacs performance
on huge org files (see the discussion in [1]).
What you observe is a consequence of that change. Now, all the drawer
overlays are destroyed when you fold a heading. Isearch would
only see a single huge overlay and unfold it alltogether.
I am currently working on a patch to rewrite the whole folding system.
Your issue should disappear once it is applied.
Meanwhile, you may, for example, advice
org-fold--isearch-filter-predicate to re-fold drawers during isearch.
Best,
Ihor
[1] https://www.mail-archive.com/emacs-orgmode@gnu.org/msg127740.html
Kévin Le Gouguec writes:
> I haven't reached the bottom of this rabbit hole yet. Since I think
> I've spent all the time I had to spend in this issue for the day, here's
> where I'm at.
>
> If I open my example file[1] with Org 9.3 and master (86fada6b5), and
> compare the overlays covering the hidden part of the first LOGBOOK
> drawer, like so:
>
> #+begin_src sh
> #!/bin/bash
>
> set -eu
>
> point=67
>
> orgs=(
> "${path_to_org_9_3}"
> "${path_to_org_master}"
> )
>
> for o in "${orgs[@]}"
> do
> emacs -Q -batch \
> -L "${o}"/lisp \
> -eval "(find-file \"logbooks.org\")" \
> -eval "(message org-version)" \
> -eval "(dolist (o (overlays-at ${point}))
>(message \"from %s to %s\" (overlay-start o) (overlay-end
> o))
>(message \"%s\" (overlay-properties o)))"
> echo
> done
> #+end_src
>
> Here is what I get:
>
>> 9.3
>> from 32 to 2015
>> (evaporate t invisible outline isearch-open-invisible #[lambda gibberish
>> involving (org-show-context 'isearch)])
>> from 67 to 262
>> (isearch-open-invisible delete-overlay invisible org-hide-drawer evaporate t)
>>
>> 9.3.7
>> from 32 to 2015
>> (isearch-open-invisible delete-overlay invisible outline evaporate t)
>
> - The LOGBOOK overlay (from 67 to 262) vanished.
>
> - The isearch-open-invisible property of the "project 1" overlay (from
> 32 to 2015) changed from a lambda[2] to just delete-overlay.
>
>
> Again, not sure this shows there's an issue; for all I know Org 9.4 just
> works differently and nothing's wrong there. I hope someone can chime
> in and take it up from here; otherwise I'll resume my investigations
> sometime later.
>
>
> [1] https://orgmode.org/list/87eepuz0bj@gmail.com/2-logbooks.org
>
> [2] Which I guess is the value of
> outline-isearch-open-invisible-function, set locally by org-mode in
> the major mode function?
>
--
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg
Re: [9.4] LOGBOOK visibility
I haven't reached the bottom of this rabbit hole yet. Since I think
I've spent all the time I had to spend in this issue for the day, here's
where I'm at.
If I open my example file[1] with Org 9.3 and master (86fada6b5), and
compare the overlays covering the hidden part of the first LOGBOOK
drawer, like so:
#+begin_src sh
#!/bin/bash
set -eu
point=67
orgs=(
"${path_to_org_9_3}"
"${path_to_org_master}"
)
for o in "${orgs[@]}"
do
emacs -Q -batch \
-L "${o}"/lisp \
-eval "(find-file \"logbooks.org\")" \
-eval "(message org-version)" \
-eval "(dolist (o (overlays-at ${point}))
(message \"from %s to %s\" (overlay-start o) (overlay-end o))
(message \"%s\" (overlay-properties o)))"
echo
done
#+end_src
Here is what I get:
> 9.3
> from 32 to 2015
> (evaporate t invisible outline isearch-open-invisible #[lambda gibberish
> involving (org-show-context 'isearch)])
> from 67 to 262
> (isearch-open-invisible delete-overlay invisible org-hide-drawer evaporate t)
>
> 9.3.7
> from 32 to 2015
> (isearch-open-invisible delete-overlay invisible outline evaporate t)
- The LOGBOOK overlay (from 67 to 262) vanished.
- The isearch-open-invisible property of the "project 1" overlay (from
32 to 2015) changed from a lambda[2] to just delete-overlay.
Again, not sure this shows there's an issue; for all I know Org 9.4 just
works differently and nothing's wrong there. I hope someone can chime
in and take it up from here; otherwise I'll resume my investigations
sometime later.
[1] https://orgmode.org/list/87eepuz0bj@gmail.com/2-logbooks.org
[2] Which I guess is the value of
outline-isearch-open-invisible-function, set locally by org-mode in
the major mode function?
Re: [9.4] LOGBOOK visibility
Kévin Le Gouguec writes: > For example, in the attached file, if I search for "bug#5": > > - Org 9.3 keeps the drawers closed, > - Org 9.4 opens every drawer for bugs 4, 5 and 6. > > AFAICT Org 9.3 never opened these drawers unless either > org-startup-folded was showeverything, the user explicitly opened one > individually, or their /content/ matched a search. > > Note that when cycling with S-TAB, even when "[SHOWING] ALL", Org 9.4 > keeps these drawers closed. I've tried to bisect this; if I'm not mistaken, the commit that introduced this behaviour is 1027e0256. I don't know why or how this commit caused this change yet, so I'm not sure if it's deliberate or not.
