Re: [PATCH] Fixing refile cache use for org-goto in indirect buffers.

[Yuchen Pei]

Thanks for the reply.
On Mon 2022-09-19 22:48:34 +0700, Max Nikulin wrote:

> On 19/09/2022 12:16, Yuchen Pei wrote:
>> To reprod:
>> - make sure the org-refile-targets generates a big enough list where
>>the refile cache makes a difference
>> - visit an org file in org-refile-targets
>> - M-x clone-indirect-buffer-other-window
>> - C-0 C-c C-w to clear cache
>> - M-: (org-refile-get-targets)
>
> Have you tried to execute this command in the indirect buffer?

Yes, and it would be instant (assuming (org-refile-get-targets) has been
run in the original buffer).  This is because the code path from
org-goto has an overriding local binding of org-refile-targets to `((nil
. (:maxlevel . ,org-goto-max-level)) before calling
org-refile-get-location which calls org-refile-get-targets.

>
>> - org-goto in the original buffer takes no effort
>> - but, org-goto in the indirect buffer takes time, which is unexpected.
>
>> diff --git a/lisp/org-refile.el b/lisp/org-refile.el
>> index 16cff25bd..7189ef595 100644
>> --- a/lisp/org-refile.el
>> +++ b/lisp/org-refile.el
>> @@ -306,7 +306,10 @@ converted to a headline before refiling."
>>  (dolist (f files)
>>(with-current-buffer (if (bufferp f) f (org-get-agenda-file-buffer f))
>>  (or
>> - (setq tgs (org-refile-cache-get (buffer-file-name) descre))
>> + (setq tgs (org-refile-cache-get
>> +(buffer-file-name (when (bufferp f)
>> +(buffer-base-buffer f)))
>> +descre))
>
> Thank you for the attempt to improve handling of indirect buffers.
>
> I am afraid, more serious refactoring is required to reuse result of
> `buffer-base-buffer', for the previous attempt to avoid issues with
> `buffer-file-name' see
> satotake to emacs-orgmode… [PATCH] org-refile.el: Fix the case of
> *scratch* buffer. Sat, 15 May 2021 19:38:39
> +0900. 
> https://list.orgmode.org/20210515103839.8574-2-doublequotat...@gmail.com
>
> There are several corner cases with `org-refile-cache', `org-goto',
> and buffers.
> - Perhaps buffer name, not file name should be used as the cache key
>   if some buffer is not associated with any file. Alternatively cache
>   should not be used at all.

It seems there may be problem with this idea.  If buffer name is the
key, that will mean a buffer and its indirect clone will generate two
caches, which duplicates the work, no?

> - When an indirect buffer is narrowed down to some region
>   (e.g. created using `org-tree-to-indirect-buffer') jump targets
>   should be filtered to the displayed range.
>
> So the change is an improvement (I would prefer `and' instead of
> `when' in such expression, but it does not really matter). Leaving
> aside other issues and more serious refactoring, it seems, storing
> results to the cache requires a similar fix, so perhaps it is possible
> to move "(setq f ...)" code above of "(or ...)" and reuse f as the
> cache key.

Sure, I will update my patch when I get around to it again.

>
> Please, send patches produced by "git format-patch" command.
>

I guess you had some problem applying the patch?  I did use
(ma)git-format-patch, but had some difficulty of getting gnus to send
the formatted patch (I basically visited the patch file and ran
(message-mode), but it was missing a few header fields compared to
composing a new mail from gnus, so I manually copied over these fields).
This was why I sent two emails.  The first one[1] had an odd extra bit
of Message-ID at the bottom of the message.  Is this the one you were
referring to as not being "standard", or is the other one[2] also not
quite right?

[1] https://lists.gnu.org/archive/html/emacs-orgmode/2022-09/msg00322.html
[2] https://lists.gnu.org/archive/html/emacs-orgmode/2022-09/msg00323.html

Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  <https://ypei.org/assets/ypei-pubkey.txt>

[PATCH] Fixing refile cache use for org-goto in indirect buffers.

[Yuchen Pei]

To reprod:
- make sure the org-refile-targets generates a big enough list where
  the refile cache makes a difference
- visit an org file in org-refile-targets
- M-x clone-indirect-buffer-other-window
- C-0 C-c C-w to clear cache
- M-: (org-refile-get-targets)
- org-goto in the original buffer takes no effort
- but, org-goto in the indirect buffer takes time, which is unexpected.

---
 lisp/org-refile.el | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 16cff25bd..7189ef595 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -306,7 +306,10 @@ converted to a headline before refiling."
(dolist (f files)
  (with-current-buffer (if (bufferp f) f (org-get-agenda-file-buffer f))
(or
-(setq tgs (org-refile-cache-get (buffer-file-name) descre))
+(setq tgs (org-refile-cache-get
+(buffer-file-name (when (bufferp f)
+(buffer-base-buffer f)))
+descre))
 (progn
   (when (bufferp f)
 (setq f (buffer-file-name (buffer-base-buffer f
-- 
2.37.3

[PATCH] Fixing refile cache use for org-goto in indirect buffers.

[Yuchen Pei]

To reprod:
- make sure the org-refile-targets generates a big enough list where
  the refile cache makes a difference
- visit an org file in org-refile-targets
- M-x clone-indirect-buffer-other-window
- C-0 C-c C-w to clear cache
- M-: (org-refile-get-targets)
- org-goto in the original buffer takes no effort
- but, org-goto in the indirect buffer takes time, which is unexpected.

---
 lisp/org-refile.el | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 16cff25bd..7189ef595 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -306,7 +306,10 @@ converted to a headline before refiling."
(dolist (f files)
  (with-current-buffer (if (bufferp f) f (org-get-agenda-file-buffer f))
(or
-(setq tgs (org-refile-cache-get (buffer-file-name) descre))
+(setq tgs (org-refile-cache-get
+(buffer-file-name (when (bufferp f)
+(buffer-base-buffer f)))
+descre))
 (progn
   (when (bufferp f)
 (setq f (buffer-file-name (buffer-base-buffer f
-- 
2.37.3

Message-ID: <877d20vscj@ypei.org>

Re: An org backend to Haddock

[Yuchen Pei]

On Tue 2022-07-19 21:58:26 +0800, Ihor Radchenko wrote:

> Yuchen Pei  writes:
>
>> Given this is not the official version of haddock and my changes are in
>> haddock-api, I'm calling it haddorg-api, for lack of a better name /
>> approach.  I'll be happy to contribute my changes upstream if a
>> different license covering my contribution (AGPLv3+) is accepted.
>>
>> Let me know what you think.
>
> I am pretty sure that you are asking the Haskell people for upstreaming
> ;)

Yes - I sent the email to both haskell-cafe and emacs-orgmode, but
upstream here means haddock / haddock-api indeed, thanks for clarifying.

>
> From Org side, it is always welcome when other Free software projects
> are making use of Org.
>
> Also, I looked into
> https://g.ypei.me/haddock.git/tree/haddock-api/src/Haddock/Backends/Org/Types.hs
>
> Beware of possible edge cases with "*/~|" in formatting. I am not
> sure if text containing such symbols is possible as an input for your
> library, but if it is, things may get formatted wrongly.
>
> For example, text containing "|" anywhere inside a table field must have
> "|" escaped using \vert entity. The markup can be escaped using
> zero-width space.

True.  I have not addressed these edge cases in my code, except a quick
hack to prepend a space whenever any line a src block / result block has
a leading star.  Tables on the other hand are extremely rare in Haskell
package documentation strings, and having a | in them is even rarer.

I do wonder what is the best way to handle cross-package links.
Currently I'm using CUSTOM_ID in the format of
package-name-x.y.z.Module.Name.IdentifierName, as well as
Module.Name.IdentifierName.  But this is a "one big org file" approach,
and can make navigation slow when say the org file reaches 15+MB in
size.

Ideally one should be able to navigate the a hackage worth of
documentations easily, with working cross-package links and low latency
org-goto to jump to any identifier or module, but I don't see how to
achieve that.  To be further investigated.

>
> Best,
> Ihor


Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  <https://ypei.org/assets/ypei-pubkey.txt>

An org backend to Haddock

[Yuchen Pei]

Hello,

I decided to write an Org backend to Haddock[1], so that haskell library
documentation can be generated in org mode markup.  Compared to the
existing backends (html, latex and hoogle), the org format allows
utilising features like the infinite levels of heading hierarchy,
flexible folding / unfolding, cross-package linking, jumping to any
declaration with org-goto and the endless potentials of emacs
customisation.

It seems to me most information and haskell language features one can
find on displayed on hackage are supported by the org backend and
included in the output org files, though there are still some rough
edges and unsupported language features (like infix declarations and
linear types) which I aim to fix.  Some example output can be found at
[2] (I will need to rename the "assets" part of the url as it is not
accurate), including base[4], ghc-prim[5] and ghc-lib-parser[6] (I use
it for reference of the GHC API as it is easier to build than GHC).  For
more information, check out the README file[3].

Given this is not the official version of haddock and my changes are in
haddock-api, I'm calling it haddorg-api, for lack of a better name /
approach.  I'll be happy to contribute my changes upstream if a
different license covering my contribution (AGPLv3+) is accepted.

Let me know what you think.

[1] https://g.ypei.me/haddock.git/tree/haddock-api
[2] https://ypei.org/assets/haddorg-output/
[3] https://g.ypei.me/haddock.git/tree/haddock-api/README.org
[4] https://ypei.org/assets/haddorg-output/base-4.16.1.0.org.gz
[5] https://ypei.org/assets/haddorg-output/ghc-prim-0.8.0.org.gz
[6] https://ypei.org/assets/haddorg-output/ghc-lib-parser-9.2.2.20220307.org.gz

Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  

Re: [BUG?] org-agenda not showing entries with diary sexp timestamps

[Yuchen Pei]

Daniel Fleischer  writes:


Eric S Fraga [2021-09-10 Fri 11:57] wrote:

What works for me, for diary sexp entries, is putting the time 
in the

heading:

* 09:00-10:00 Weekly meeting
  <%%(diary-cyclic 7 9 10 2021)>



This is exactly how the feature is described in the manual:

#+begin_example
Diary-style expression entries
 For more complex date specifications, Org mode supports 
 using the
 special expression diary entries implemented in the Emacs 
 Calendar

 package(2).  For example, with optional time:

  * 22:00-23:00 The nerd meeting on every 2nd Thursday 
  of the month

<%%(diary-float t 4 2)>
#+end_example


Thanks, my bad.  I guess the reason for the time to be included in 
the headline is to be consistent with the diary format.


--
Best,
Yuchen

PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  


signature.asc
Description: PGP signature

Re: [BUG?] org-agenda not showing entries with diary sexp timestamps

[Yuchen Pei]

Daniel Fleischer  writes:


Yuchen Pei [2021-09-10 Fri 18:00] wrote:


* Weekly meeting
 <%%(diary-cyclic 7 9 10 2021) 9:00-10:00>

Invoking org-agenda with a (org-agenda-list) does not show the 
item, whereas scheduled diary sexp or plain timestamp

works fine:

* Weekly meeting
 SCHEDULED: <%%(diary-cyclic 7 9 10 2021) 9:00-10:00>



I can reproduce; adding the hour makes the event not appear 
unless it
has a DEADLINE or SCHEDULED. 


Thanks for confirming.  The org-agenda code is not very readable 
for me with some very long functions, but I'll see if I can fix 
it.  By the way is breaking functions into smaller ones desirable 
for the project?


--
Best,
Yuchen

PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  <https://ypei.me/assets/ypei-pubkey.txt>


signature.asc
Description: PGP signature

[BUG?] org-agenda not showing entries with diary sexp timestamps

[Yuchen Pei]

Hello,

Can anyone reproduce this?

Say I write the following in my agenda org file:

* Weekly meeting
 <%%(diary-cyclic 7 9 10 2021) 9:00-10:00>

Invoking org-agenda with a (org-agenda-list) does not show the 
item, whereas scheduled diary sexp or plain timestamp works fine:


* Weekly meeting
 SCHEDULED: <%%(diary-cyclic 7 9 10 2021) 9:00-10:00>

or

* Weekly meeting
 <2021-09-10 9:00-10:00 +1w>

emacs version:

GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 
3.24.27, cairo version 1.17.4) of 2021-03-27


--
Best,
Yuchen

PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  


signature.asc
Description: PGP signature

pygments support

[Yuchen Pei]

Hello,

I was playing with my cgit setup when I noticed that pygments does 
not support org mode syntax highlighting[1].


Just wondering if anyone has worked on it, or if there's any 
"unofficial" org mode component (lexer?), before I go ahead and 
try to write my own.


[1]: https://pygments.org/languages/

--
Best,
Yuchen

PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  


signature.asc
Description: PGP signature

bug#48199: 28.0.50; Org mode surprisingly usurps Calendar key binding

[Yuchen Pei]

Lars Ingebrigtsen  writes:


Yuchen Pei  writes:

This bug seems to be still around, as it just happened to 
me. Steps

to
reproduce:

- Build a recent version e.g. commit efaed29f3d with `make`
- ./src/emacs
- M-x calendar RET
- i

results in "command-execute: Wrong type argument: commandp,
org-agenda-diary-entry".  Can anyone else reproduce it?


I can't reproduce this with "emacs -Q", or after visiting 
etc/ORG-NEWS,

with the current Emacs trunk.


You are right.  I add `-Q` to the emacs invocation and the problem 
is gone.  I'm gonna try to re-archive this bug.


--
Best,
Yuchen

PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  <https://ypei.me/assets/ypei-pubkey.txt>


signature.asc
Description: PGP signature

bug#48199: 28.0.50; Org mode surprisingly usurps Calendar key binding

[Yuchen Pei]

Looks like this email hadn't go through before I unarchived the 
bug.  Apologies for any duplicate messages.

Yuchen Pei  writes:

This bug seems to be still around, as it just happened to 
me. Steps to

reproduce:

- Build a recent version e.g. commit efaed29f3d with `make`
- ./src/emacs
- M-x calendar RET
- i

results in "command-execute: Wrong type argument: commandp,
org-agenda-diary-entry".  Can anyone else reproduce it?



--
Best,
Yuchen

PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
  <https://ypei.me/assets/ypei-pubkey.txt>


signature.asc
Description: PGP signature