Re: [PATCH] Set Python shell in Org edit buffer

2023-12-29 Thread Liu Hui
Jack Kamm  于2023年12月30日周六 06:20写道:
>
> Liu Hui  writes:
>
> > But it is indeed possible that two sessions are inconsistent, if users
> > intend to have different org-babel-python-command and
> > python-shell-interpreter, which are used by
> > `org-babel-python-initiate-session' and `run-python', respectively.
>
> I have just proposed this patch, which will reduce this discrepancy by
> having ob-python sessions use the same interpreter as `run-python' by
> default:
>
> https://list.orgmode.org/87edf41yeb@gmail.com/T/#u

Thanks! It is a nice change IMO.

> Additionally, we could set `python-shell-interpreter(-args)' in
> `org-babel-edit-prep:python' to make `run-python' in Org Src buffer use
> the same interpreter as :python header or customized
> `org-babel-python-command'.

Agree. I think it can address the concern about inconsistent sessions.



Re: [PATCH] Set Python shell in Org edit buffer

2023-12-09 Thread Liu Hui
Ihor Radchenko  于2023年12月9日周六 18:29写道:
>
> Liu Hui  writes:
>
> >> 2. Check the new variable and attempt to run
> >>`org-babel--associate-session' in org-src-mode definition.
> >
> > I think associating the edit buffer with some session doesn't require
> > starting the session, which is at least feasible for ob-python. When
> > editing python src block, users can use C-c C-p to start the session
> > themselves for evaluating code. So it would be nice to allow a value
> > of 'associate in the customization, which means just running
> > `org-babel--associate-session'.
>
> I think we have a misunderstanding here.
>
> Didn't we just discuss that C-c C-p in python is not equivalent to
> `org-babel-python-initiate-session'?
>
> Also, by "start a new session" I meant "start a new session if there is
> none running already; if a session is already running, unconditionally
> associate Org Src buffer with that running session".

I just want to set 'python-shell-buffer-name' in the edit buffer
according to the :session header and don't need to start the session
even if the session doesn't exist. It may be not a real association by
your definition, so my previous comment is not relevant any more.



Re: [PATCH] Set Python shell in Org edit buffer

2023-12-08 Thread Liu Hui
Ihor Radchenko  于2023年12月8日周五 21:06写道:
>
> Liu Hui  writes:
>
> >> What about displaying a yes/no query about starting a new session when
> >> there is none?
> >
> > I think it is OK. I can add an option to allow users to disable the
> > query. WDYT?
>
> I now have second thoughts about obsoleting
> org-babel--associate-session.
> If we need a customization for the new session dialogue, I expect that
> users will be willing to set it for all src blocks that support such a
> feature, not just python. But then we somehow need to make this global
> customization visible to babel backend authors - something we normally
> do using the set of org-babel-...: functions.
>
> What I am thinking now is
> 1. Introduce a global customization for users to choose whether to
>start a new session in org-src buffers with allowed values t, nil,
>'ask, or an alist of (lang . t/nil/ask) for per-language customization.

I think it makes sense. Currently, org-babel-edit-prep:R/julia always
start the session when editing src blocks with a valid session header,
and other backends don't. It would allow users to change the behavior.

> 2. Check the new variable and attempt to run
>`org-babel--associate-session' in org-src-mode definition.

I think associating the edit buffer with some session doesn't require
starting the session, which is at least feasible for ob-python. When
editing python src block, users can use C-c C-p to start the session
themselves for evaluating code. So it would be nice to allow a value
of 'associate in the customization, which means just running
`org-babel--associate-session'.



Re: [PATCH] Set Python shell in Org edit buffer

2023-12-08 Thread Liu Hui
Ihor Radchenko  于2023年12月7日周四 23:16写道:
>
> Liu Hui  writes:
>
> > Ihor Radchenko  于2023年12月7日周四 18:33写道:
> >
> >> > +(defun org-babel-edit-prep:python (info)
> >> > +  "Set Python shell in Org edit buffer according to INFO."
> >> > +  (let ((session (cdr (assq :session (nth 2 info)
> >> > +(when (and session (not (string= session "none")))
> >> > +  (setq-local python-shell-buffer-name
> >> > +  (org-babel-python-without-earmuffs session)
> >>
> >> Will this work if Python session does not exist yet?
> >
> > If the specified session does not exist, users have to start the
> > session manually if they want to evaluate code directly in the edit
> > buffer. In fact, python-shell-send-* commands will clearly suggest
> > users using 'run-python' or C-c C-p to create the session in this
> > case.
>
> I am afraid that manually starting a python session with `run-python'
> will be misleading. Look at how much
> `org-babel-python-initiate-session-by-key' does. If users instead start
> the session with `run-python' they may get inconsistent results

ob-python has good support for using the existing session that created
by 'run-python', which contributes to a large part of
org-babel-python-initiate-session-by-key. It is common for users to
start a ob-python session with `run-python', even in the edit buffer.

But it is indeed possible that two sessions are inconsistent, if users
intend to have different org-babel-python-command and
python-shell-interpreter, which are used by
`org-babel-python-initiate-session' and `run-python', respectively.

> > Another choice is to use '(org-babel-python-initiate-session session)'
> > in org-babel-edit-prep:python, like ob-R, to create the session when
> > the specified session does not exist, but I feel it is invasive as
> > users may just want to edit the code.
>
> What about displaying a yes/no query about starting a new session when
> there is none?

I think it is OK. I can add an option to allow users to disable the
query. WDYT?



Re: [PATCH] Set Python shell in Org edit buffer

2023-12-07 Thread Liu Hui
Ihor Radchenko  于2023年12月7日周四 18:33写道:

> > +(defun org-babel-edit-prep:python (info)
> > +  "Set Python shell in Org edit buffer according to INFO."
> > +  (let ((session (cdr (assq :session (nth 2 info)
> > +(when (and session (not (string= session "none")))
> > +  (setq-local python-shell-buffer-name
> > +  (org-babel-python-without-earmuffs session)
>
> Will this work if Python session does not exist yet?

If the specified session does not exist, users have to start the
session manually if they want to evaluate code directly in the edit
buffer. In fact, python-shell-send-* commands will clearly suggest
users using 'run-python' or C-c C-p to create the session in this
case.

Another choice is to use '(org-babel-python-initiate-session session)'
in org-babel-edit-prep:python, like ob-R, to create the session when
the specified session does not exist, but I feel it is invasive as
users may just want to edit the code.



Re: [PATCH] Set Python shell in Org edit buffer

2023-12-06 Thread Liu Hui
Ihor Radchenko  于2023年12月6日周三 21:20写道:
>
> Liu Hui  writes:
>
> >> Makes sense.
> >> I think we may drop a note about this new feature in ORG-NEWS.
> >
> > Thanks, I have added it in the attached patch.
> > ...
> > +When editing a Python src block, the editing buffer is now associated
> > +with the Python shell specified by the src block's ~:session~ header
> > +argument.
>
> May you also mention what this means in practice? Like that users can
> now send region for evaluation right from the edit src buffer?
> Otherwise, it may not be very clear for ordinary users what this feature
> adds.

Yes, I have updated the text and you're welcome to improve it. Thanks!
From c503b2ed5116e2abae25459b09abc919074ac54a Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 5 Dec 2023 11:40:38 +0800
Subject: [PATCH] Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-edit-prep:python): New function.
* etc/ORG-NEWS (ob-python now sets ~python-shell-buffer-name~ in Org
edit buffers): Announce the change.
---
 etc/ORG-NEWS  | 7 +++
 lisp/ob-python.el | 7 +++
 2 files changed, 14 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index aef7e1184..f30019d70 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -640,6 +640,13 @@ return a matplotlib Figure object to plot. For output results, the
 current figure (as returned by =pyplot.gcf()=) is cleared before
 evaluation, and then plotted afterwards.
 
+*** ob-python now sets ~python-shell-buffer-name~ in Org edit buffers
+
+When editing a Python src block, the editing buffer is now associated
+with the Python shell specified by the src block's ~:session~ header,
+which means users can now send code directly from the edit buffer,
+e.g., using ~C-c C-c~, to the session specified in the Org buffer.
+
 *** =ob-maxima.el=: Support for ~batch~ and ~draw~
 
 =ob-maxima= has two new header arguments: ~:batch~ and
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 6c568a36d..8ff2c7a1d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -67,6 +67,13 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-edit-prep:python (info)
+  "Set Python shell in Org edit buffer according to INFO."
+  (let ((session (cdr (assq :session (nth 2 info)
+(when (and session (not (string= session "none")))
+  (setq-local python-shell-buffer-name
+  (org-babel-python-without-earmuffs session)
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.25.1



Re: [PATCH] Set Python shell in Org edit buffer

2023-12-05 Thread Liu Hui
Ihor Radchenko  于2023年12月5日周二 19:48写道:
>
> Liu Hui  writes:
>
> > When editing python src block using C-c ', the python shell is not set
> > in Org edit buffer according to the :session header argument of the
> > block. Consequently, commands such as python-shell-send-region cannot
> > send python code to the correct python process. To address this, the
> > attached patch defines org-babel-edit-prep:python in ob-python.el.
>
> Makes sense.
> I think we may drop a note about this new feature in ORG-NEWS.

Thanks, I have added it in the attached patch.

> > In addition, I tried to use org-src-associate-babel-session at first,
> > but found it doesn't work because it is called when enabling
> > org-src-mode, where org-src--babel-info is still nil. It seems it has
> > stopped working since 203bf5870, and I think it is safe to remove it
> > and related stuffs to avoid confusion.
>
> I do not see it being used much, except
> https://github.com/frederic-santos/ob-ess-julia and
> https://github.com/alphapapa/outshine
> Although, ob-ess-julia is deprecated and outshine only mentions this in
> "TODO" comment.
>
> To be safe, I'd prefer to mark `org-src-associate-babel-session'
> obsolete and mention the removal in ORG-NEWS, pointing users to
> `org-babel-edit-prep:..' instead.

Fine by me.
From 295b28fece2f2125b40635118a5a705af4c8ce74 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 5 Dec 2023 11:40:38 +0800
Subject: [PATCH] Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-edit-prep:python): New function.
* etc/ORG-NEWS (ob-python now sets ~python-shell-buffer-name~ in Org
edit buffers): Announce the change.
---
 etc/ORG-NEWS  | 6 ++
 lisp/ob-python.el | 7 +++
 2 files changed, 13 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index aef7e1184..826ea3526 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -640,6 +640,12 @@ return a matplotlib Figure object to plot. For output results, the
 current figure (as returned by =pyplot.gcf()=) is cleared before
 evaluation, and then plotted afterwards.
 
+*** ob-python now sets ~python-shell-buffer-name~ in Org edit buffers
+
+When editing a Python src block, the editing buffer is now associated
+with the Python shell specified by the src block's ~:session~ header
+argument.
+
 *** =ob-maxima.el=: Support for ~batch~ and ~draw~
 
 =ob-maxima= has two new header arguments: ~:batch~ and
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 6c568a36d..8ff2c7a1d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -67,6 +67,13 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-edit-prep:python (info)
+  "Set Python shell in Org edit buffer according to INFO."
+  (let ((session (cdr (assq :session (nth 2 info)
+(when (and session (not (string= session "none")))
+  (setq-local python-shell-buffer-name
+  (org-babel-python-without-earmuffs session)
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.25.1



[PATCH] Set Python shell in Org edit buffer

2023-12-05 Thread Liu Hui
Hi,

When editing python src block using C-c ', the python shell is not set
in Org edit buffer according to the :session header argument of the
block. Consequently, commands such as python-shell-send-region cannot
send python code to the correct python process. To address this, the
attached patch defines org-babel-edit-prep:python in ob-python.el.

In addition, I tried to use org-src-associate-babel-session at first,
but found it doesn't work because it is called when enabling
org-src-mode, where org-src--babel-info is still nil. It seems it has
stopped working since 203bf5870, and I think it is safe to remove it
and related stuffs to avoid confusion.


Best,
Liu Hui
From c8b9c174cf643bd625cedc311d2604e6fc3bb83a Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 5 Dec 2023 11:40:38 +0800
Subject: [PATCH] Set Python shell in Org edit buffer

* lisp/ob-python.el (org-babel-edit-prep:python): New function.
---
 lisp/ob-python.el | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 6c568a36d..8ff2c7a1d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -67,6 +67,13 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defun org-babel-edit-prep:python (info)
+  "Set Python shell in Org edit buffer according to INFO."
+  (let ((session (cdr (assq :session (nth 2 info)
+(when (and session (not (string= session "none")))
+  (setq-local python-shell-buffer-name
+  (org-babel-python-without-earmuffs session)
+
 (defun org-babel-execute:python (body params)
   "Execute Python BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
-- 
2.25.1

From f3c0d401de55d7ad8873c18658abd1d9f6b49d77 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 5 Dec 2023 11:36:07 +0800
Subject: [PATCH] Remove 'org-src-babel-configure-edit-buffer' and related
 stuffs

The functionality has stopped working since 203bf5870.
org-babel-edit-prep:lang should be used instead.

* lisp/org-src.el (org-src-associate-babel-session):
(org-src-babel-configure-edit-buffer):
* lisp/ob-R.el (org-babel-R-associate-session): Remove.
---
 lisp/ob-R.el|  8 
 lisp/org-src.el | 23 ---
 2 files changed, 31 deletions(-)

diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index c48b2cdb7..f3cce20b7 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -294,14 +294,6 @@ (defun org-babel-R-initiate-session (session params)
 	   (buffer-name
 	  (current-buffer))
 
-(defun org-babel-R-associate-session (session)
-  "Associate R code buffer with an R session.
-Make SESSION be the inferior ESS process associated with the
-current code buffer."
-  (setq ess-local-process-name
-	(process-name (get-buffer-process session)))
-  (ess-make-buffer-current))
-
 (defvar org-babel-R-graphics-devices
   '((:bmp "bmp" "filename")
 (:jpg "jpeg" "filename")
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 866ff2dbf..6406b8d29 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -914,29 +914,6 @@ (defun org-src-mode-configure-edit-buffer ()
 
 (add-hook 'org-src-mode-hook #'org-src-mode-configure-edit-buffer)
 
-
-
-;;; Babel related functions
-
-(defun org-src-associate-babel-session (info)
-  "Associate edit buffer with comint session.
-INFO should be a list similar in format to the return value of
-`org-babel-get-src-block-info'."
-  (interactive)
-  (let ((session (cdr (assq :session (nth 2 info)
-(and session (not (string= session "none"))
-	 (org-babel-comint-buffer-livep session)
-	 (let ((f (intern (format "org-babel-%s-associate-session"
-  (nth 0 info)
-   (and (fboundp f) (funcall f session))
-
-(defun org-src-babel-configure-edit-buffer ()
-  "Configure src editing buffer."
-  (when org-src--babel-info
-(org-src-associate-babel-session org-src--babel-info)))
-
-(add-hook 'org-src-mode-hook #'org-src-babel-configure-edit-buffer)
-
 
 ;;; Public API
 
-- 
2.25.1



Re: [BUG] [PATCH] Add yank-media and DND handler [9.6.7 (9.6.7-g6eb773 @ /home/viz/lib/emacs/straight/build/org/)]

2023-10-11 Thread Liu Hui
Visuwesh  于2023年10月11日周三 23:36写道:

> >> + (caddr (read-multiple-choice
> >> + "What to do with dropped file?"
> >> + '((?a "attach" attach)
> >> + (?o "open" open)
> >> + (?f "insert file: link" file-link
> >
> > The dialog box is shown in the center of frame and I find it a little
> > inconvenient compared with a menu popped up just at the mouse
> > location. How about using x-popup-menu?
>
> But isn't that for menus?  I don't know how convenient is a menu
> compared to a dialog box.  But if the consensus is that a menu should be
> popped, then I will adjust the query to use rmc or menu as per
> use-dialog-box.

Thanks for consideration. I don't mean menu keymap but a pop-up menu
that simply returns a candidate, like

(x-popup-menu t
  (list "What to do with dropped file?"
(cons ""
  '(("Attach" . attach)
("Open" . open)
("Insert file: link" . file-link)

The menu is popped up at the location where the file is dropped, so
users only need to move the mouse slightly to select the desired
action, rather than always moving the mouse to the center of frame.

An example in dired mode is attached, and I feel it is a little more
convenient and efficient from the perspective of user interaction ;)


Re: [BUG] [PATCH] Add yank-media and DND handler [9.6.7 (9.6.7-g6eb773 @ /home/viz/lib/emacs/straight/build/org/)]

2023-10-11 Thread Liu Hui
Hi,

Thanks for your work. I have two minor suggestions about the patch.

> + (`file-link
> + (let ((filename (dnd-get-local-file-name url)))
> + (insert (org-link-make-string (concat "file:" filename

I think it is better to abbreviate the file name for file-link, so it
would be consistent with org-insert-link.

> + (caddr (read-multiple-choice
> + "What to do with dropped file?"
> + '((?a "attach" attach)
> + (?o "open" open)
> + (?f "insert file: link" file-link

The dialog box is shown in the center of frame and I find it a little
inconvenient compared with a menu popped up just at the mouse
location. How about using x-popup-menu?



Re: [BUG] [PATCH] Add yank-media and DND handler [9.6.7 (9.6.7-g6eb773 @ /home/viz/lib/emacs/straight/build/org/)]

2023-10-01 Thread Liu Hui
Visuwesh  于2023年10月1日周日 22:28写道:

>  Do you think asking during the time of drop
> would be a viable option too?  I'm thinking of adding the defcustom
> org-dnd-default-method with options such as
>
> . attach -- as implemented here
> . open -- open file
> . file-link -- insert file links
> . ask -- ask what to do after drop
>
> I think the `ask' option would be nice to have too.  Perhaps, even as
> the default?

Yes, I totally agree with asking as the default.



Re: [BUG] [PATCH] Add yank-media and DND handler [9.6.7 (9.6.7-g6eb773 @ /home/viz/lib/emacs/straight/build/org/)]

2023-09-29 Thread Liu Hui
Hi,

在 2023/9/27 16:29, Visuwesh 写道:

> +*** Files and images can be attached by dropping onto Emacs
> +
> +Attachment method other than ~org-attach-method~ for dropped files can
> +be specified using ~org-dnd-default-attach-method~.

> +(defcustom org-dnd-default-attach-method nil
> +  "Default attach method to use when DND action is unspecified.
> +This attach method is used when the DND action is `private'.
> +This is also used when `org-yank-image-save-type' is nil.
> +When nil, use `org-attach-method'."

I think the dnd feature should not be restricted to org-attach. I have
often used it for opening file and inserting file link. How about
supporting them and adding a new variable, such as
org-dnd-default-method?

> +   ('ask (caddr (org-mks
> + '(("c" "Copy" cp)
> +   ("m" "Move" mv)
> +   ("l" "Hard link" ln)
> +   ("s" "Symbolic link" lns))
> + "How to attach?"
> + "Attach using method")))

It is better to pop up a menu that allows users to proceed with the
mouse, e.g. that in `dired-dnd-do-ask-action'. Options like
'open'/'file link' could be added too.

Thanks for your work.



[PATCH] Fix docstring of `org-at-timestamp-p'

2023-08-23 Thread Liu Hui
Hi,

I find that the doc string of `org-at-timestamp-p' about match groups
is not consistent with the behavior. The doc string says

When matching, the match groups are the following:
  group 1: year, if any
  group 2: month, if any
  group 3: day number, if any
  group 4: day name, if any
  group 5: hours, if any
  group 6: minutes, if any

but for timestamps like <2023-08-23 Wed>, (when (org-at-timestamp-p)
(match-string 2)) will return the year 2023 instead of the month. This
patch corrects the group numbers.
From a3c31e4f8f40202fcef61fc7c4b16acaf006e24f Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Wed, 23 Aug 2023 14:27:21 +0800
Subject: [PATCH] * lisp/org.el (org-at-timestamp-p): fix doc string

---
 lisp/org.el | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 3d9d61f69..6b89e14c4 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14843,12 +14843,12 @@ (defun org-at-timestamp-p ( extended)
 values are possible).
 
 When matching, the match groups are the following:
-  group 1: year, if any
-  group 2: month, if any
-  group 3: day number, if any
-  group 4: day name, if any
-  group 5: hours, if any
-  group 6: minutes, if any"
+  group 2: year, if any
+  group 3: month, if any
+  group 4: day number, if any
+  group 5: day name, if any
+  group 7: hours, if any
+  group 8: minutes, if any"
   (let* ((regexp
   (if extended
   (if (eq extended 'agenda)
-- 
2.25.1



Re: [PATCH] ob-python results handling for dicts, dataframes, arrays, and plots

2023-08-21 Thread Liu Hui
> Thanks for reporting these misbehaving examples. I think the root of the
> problem is `org-babel-script-escape', which is too aggressive in
> recursively converting strings to lists. We may need to rewrite our own
> implementation for ob-python.
>
> Also, I agree that moving the python code to an external file will be
> helpful in handling these more complex cases.
>
> I may leave these tasks for future patches. In the meantime, we may have
> to recommend ":results verbatim" for these more complex cases that
> ":results table" doesn't fully handle yet.

Understand. Thanks again for your work!



Re: [PATCH] ob-python results handling for dicts, dataframes, arrays, and plots

2023-08-20 Thread Liu Hui
> > Here we can use '{}'.format(df.index.name) to show the name of index
>
> Patch has been updated to print the index name when it is non-None.

Thanks! It would be nice to also support MultiIndex names using
`result.index.names', e.g.

#+begin_src python :results table
import numpy as np
import pandas as pd

df = pd.DataFrame({
"A": ["foo", "bar", "foo", "bar", "foo", "bar", "foo", "foo"],
"B": ["one", "one", "two", "three", "two", "two", "one", "three"],
"C": np.random.randn(8),
"D": np.random.randn(8)})
return df.groupby(["A", "B"]).agg('sum').round(3)
#+end_src

Another problem is the display of objects like datetime, e.g.

#+begin_src python :results table
import pandas as pd
s = pd.Series(range(3), index=pd.date_range("2000", freq="D", periods=3))
return s.to_frame()
#+end_src

#+RESULTS:
|   | 0 |   |
|---+---+---|
| Timestamp | (2000-01-01 00:00:00 freq= D) | 0 |
| Timestamp | (2000-01-02 00:00:00 freq= D) | 1 |
| Timestamp | (2000-01-03 00:00:00 freq= D) | 2 |

#+begin_src python
from pathlib import Path
import numpy as np

return {'a': 1, 'path': Path('/'), 'array': np.zeros(3)}
#+end_src

#+RESULTS:
| a | 1 |   |
| path  | PosixPath | (/)   |
| array | array | ((0 0 0)) |

I think these objects need to be shown in a single column rather than
two. Besides, if the python code becomes too complex finally, I think
maintaining the python code outside the ob-python.el, as suggested by
Ihor, is a good idea.



Re: [PATCH] ob-python results handling for dicts, dataframes, arrays, and plots

2023-08-16 Thread Liu Hui
Hi,

Thank you for the patch!

> Next, for numpy arrays and pandas dataframes/series: these are
> converted to tables, for example:
>
> #+begin_src python
>   import pandas as pd
>   import numpy as np
>
>   return pd.DataFrame(np.array([[1,2,3],[4,5,6]]),
>   columns=['a','b','c'])
> #+end_src
>
> #+RESULTS:
> |   | a | b | c |
> |---+---+---+---|
> | 0 | 1 | 2 | 3 |
> | 1 | 4 | 5 | 6 |
>
> To avoid conversion, you can specify "raw", "verbatim", "scalar", or
> "output" in the ":results" header argument.

Do we need to limit the table/list size by default, or handle them
only with relevant result type (e.g. `table/list')? Dataframe/array
are often large. The following results are truncated by default
previously, which can be tweaked via np.set_printoptions and
pd.set_option.

#+begin_src python
import numpy as np
return np.random.randint(10, size=(30,40))
#+end_src

#+begin_src python
import numpy as np
return np.random.rand(20,3,4,5)
#+end_src

#+begin_src python
import pandas as pd
import numpy as np

d = {'col1': np.random.rand(100), 'col2': np.random.rand(100)}
return pd.DataFrame(d)
#+end_src

> +def __org_babel_python_format_value(result, result_file, result_params):
> +with open(result_file, 'w') as f:
> +if 'graphics' in result_params:
> +result.savefig(result_file)
> +elif 'pp' in result_params:
> +import pprint
> +f.write(pprint.pformat(result))
> +else:
> +if not set(result_params).intersection(\
> +['scalar', 'verbatim', 'raw']):
> +try:
> +import pandas
> +except ImportError:
> +pass
> +else:
> +if isinstance(result, pandas.DataFrame):
> +result = [[''] + list(result.columns), None] + \

Here we can use '{}'.format(df.index.name) to show the name of index

>  (defun org-babel-python-format-session-value
>  (src-file result-file result-params)
>"Return Python code to evaluate SRC-FILE and write result to RESULT-FILE."
> -  (format "\
> +  (concat org-babel-python--def-format-value
> +  (format "

Maybe `org-babel-python--def-format-value' can be evaluated only once
in the session mode? It would shorten the string sent to the python
shell, where temp files are used for long strings.



Re: [PATCH] ob-python: Fix async evaluation

2023-07-13 Thread Liu Hui
Jack Kamm  于2023年7月13日周四 05:58写道:

> While your test works on its own, it seems to break subsequent tests
> (the next test hangs).

Thanks for pointing out the problem! I find the problem disappears
after removing the `run-python` line, and I have updated the patch
accordingly.
From 4c552eb4eec5d84727775645cdf41acebd7fd1da Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Wed, 12 Jul 2023 18:07:06 +0800
Subject: [PATCH] ob-python: Fix async evaluation

* lisp/ob-python.el (org-babel-python-async-evaluate-session): Bind
`python-shell-buffer-name' inside the temp buffer.
* testing/lisp/test-ob-python.el (test-ob-python/async-local-python-shell):
Add test.
---
 lisp/ob-python.el  | 47 ++
 testing/lisp/test-ob-python.el | 14 ++
 2 files changed, 39 insertions(+), 22 deletions(-)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 0e0539d7a..c15d45b96 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -400,28 +400,31 @@ (defun org-babel-python-async-evaluate-session
session (current-buffer)
"ob_comint_async_python_\\(.+\\)_\\(.+\\)"
'org-babel-chomp 'org-babel-python-async-value-callback)
-  (let ((python-shell-buffer-name (org-babel-python-without-earmuffs session)))
-(pcase result-type
-  (`output
-   (let ((uuid (org-id-uuid)))
- (with-temp-buffer
-   (insert (format org-babel-python-async-indicator "start" uuid))
-   (insert "\n")
-   (insert body)
-   (insert "\n")
-   (insert (format org-babel-python-async-indicator "end" uuid))
-   (python-shell-send-buffer))
- uuid))
-  (`value
-   (let ((tmp-results-file (org-babel-temp-file "python-"))
- (tmp-src-file (org-babel-temp-file "python-")))
- (with-temp-file tmp-src-file (insert body))
- (with-temp-buffer
-   (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
-   (insert "\n")
-   (insert (format org-babel-python-async-indicator "file" tmp-results-file))
-   (python-shell-send-buffer))
- tmp-results-file)
+  (pcase result-type
+(`output
+ (let ((uuid (org-id-uuid)))
+   (with-temp-buffer
+ (insert (format org-babel-python-async-indicator "start" uuid))
+ (insert "\n")
+ (insert body)
+ (insert "\n")
+ (insert (format org-babel-python-async-indicator "end" uuid))
+ (let ((python-shell-buffer-name
+(org-babel-python-without-earmuffs session)))
+   (python-shell-send-buffer)))
+   uuid))
+(`value
+ (let ((tmp-results-file (org-babel-temp-file "python-"))
+   (tmp-src-file (org-babel-temp-file "python-")))
+   (with-temp-file tmp-src-file (insert body))
+   (with-temp-buffer
+ (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
+ (insert "\n")
+ (insert (format org-babel-python-async-indicator "file" tmp-results-file))
+ (let ((python-shell-buffer-name
+(org-babel-python-without-earmuffs session)))
+   (python-shell-send-buffer)))
+   tmp-results-file
 
 (provide 'ob-python)
 
diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el
index 7aac87116..82fbca36e 100644
--- a/testing/lisp/test-ob-python.el
+++ b/testing/lisp/test-ob-python.el
@@ -296,6 +296,20 @@ (ert-deftest test-ob-python/async-output-drawer ()
  (string= (concat src-block result)
   (buffer-string)))
 
+(ert-deftest test-ob-python/async-local-python-shell ()
+  ;; Disable the test on older Emacs as built-in python.el sometimes
+  ;; fail to initialize session.
+  (skip-unless (version<= "28" emacs-version))
+  (when-let ((buf (get-buffer "*Python*")))
+(let (kill-buffer-query-functions)
+  (kill-buffer buf)))
+  (org-test-with-temp-text-in-file
+  "# -*- python-shell-buffer-name: \"Python 3\" -*-
+#+begin_src python :session \"*Python 3*\" :async yes
+1
+#+end_src"
+(should (org-babel-execute-src-block
+
 (provide 'test-ob-python)
 
 ;;; test-ob-python.el ends here
-- 
2.25.1



Re: [PATCH] ob-python: Fix async evaluation

2023-07-12 Thread Liu Hui
Jack Kamm  于2023年7月12日周三 12:51写道:

> The patch looks good, but it would be nice to include a unit test as
> well -- could you update the patch to include one, Liu Hui?

OK, I have added a test to the patch.
From 56fd5e05bc7dc82fb825416100e75663a520 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Wed, 12 Jul 2023 18:07:06 +0800
Subject: [PATCH] ob-python: Fix async evaluation

* lisp/ob-python.el (org-babel-python-async-evaluate-session): Bind
`python-shell-buffer-name' inside the temp buffer.
* testing/lisp/test-ob-python.el (test-ob-python/async-local-python-shell):
Add test.
---
 lisp/ob-python.el  | 47 ++
 testing/lisp/test-ob-python.el | 15 +++
 2 files changed, 40 insertions(+), 22 deletions(-)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 0e0539d7a..c15d45b96 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -400,28 +400,31 @@ (defun org-babel-python-async-evaluate-session
session (current-buffer)
"ob_comint_async_python_\\(.+\\)_\\(.+\\)"
'org-babel-chomp 'org-babel-python-async-value-callback)
-  (let ((python-shell-buffer-name (org-babel-python-without-earmuffs session)))
-(pcase result-type
-  (`output
-   (let ((uuid (org-id-uuid)))
- (with-temp-buffer
-   (insert (format org-babel-python-async-indicator "start" uuid))
-   (insert "\n")
-   (insert body)
-   (insert "\n")
-   (insert (format org-babel-python-async-indicator "end" uuid))
-   (python-shell-send-buffer))
- uuid))
-  (`value
-   (let ((tmp-results-file (org-babel-temp-file "python-"))
- (tmp-src-file (org-babel-temp-file "python-")))
- (with-temp-file tmp-src-file (insert body))
- (with-temp-buffer
-   (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
-   (insert "\n")
-   (insert (format org-babel-python-async-indicator "file" tmp-results-file))
-   (python-shell-send-buffer))
- tmp-results-file)
+  (pcase result-type
+(`output
+ (let ((uuid (org-id-uuid)))
+   (with-temp-buffer
+ (insert (format org-babel-python-async-indicator "start" uuid))
+ (insert "\n")
+ (insert body)
+ (insert "\n")
+ (insert (format org-babel-python-async-indicator "end" uuid))
+ (let ((python-shell-buffer-name
+(org-babel-python-without-earmuffs session)))
+   (python-shell-send-buffer)))
+   uuid))
+(`value
+ (let ((tmp-results-file (org-babel-temp-file "python-"))
+   (tmp-src-file (org-babel-temp-file "python-")))
+   (with-temp-file tmp-src-file (insert body))
+   (with-temp-buffer
+ (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
+ (insert "\n")
+ (insert (format org-babel-python-async-indicator "file" tmp-results-file))
+ (let ((python-shell-buffer-name
+(org-babel-python-without-earmuffs session)))
+   (python-shell-send-buffer)))
+   tmp-results-file
 
 (provide 'ob-python)
 
diff --git a/testing/lisp/test-ob-python.el b/testing/lisp/test-ob-python.el
index 7aac87116..14dae0ef5 100644
--- a/testing/lisp/test-ob-python.el
+++ b/testing/lisp/test-ob-python.el
@@ -296,6 +296,21 @@ (ert-deftest test-ob-python/async-output-drawer ()
  (string= (concat src-block result)
   (buffer-string)))
 
+(ert-deftest test-ob-python/async-local-python-shell ()
+  ;; Disable the test on older Emacs as built-in python.el sometimes
+  ;; fail to initialize session.
+  (skip-unless (version<= "28" emacs-version))
+  (when-let ((buf (get-buffer "*Python*")))
+(let (kill-buffer-query-functions)
+  (kill-buffer buf)))
+  (org-test-with-temp-text-in-file
+  "# -*- python-shell-buffer-name: \"Python 3\" -*-
+#+begin_src python :session \"*Python 3*\" :async yes
+1
+#+end_src"
+(run-python nil nil 'hide)
+(should (org-babel-execute-src-block
+
 (provide 'test-ob-python)
 
 ;;; test-ob-python.el ends here
-- 
2.25.1



[PATCH] ob-python: Fix async evaluation

2023-07-10 Thread Liu Hui
Hi,

To reproduce the bug:

1. create test.org:
──✀──
#+begin_src python :session "*Python 3*" :async t
1
#+end_src

# Local Variables:
# python-shell-buffer-name: "Python 3"
# End:
──✀──

2. emacs -Q -L  --eval "(require 'ob-python)"

3. Open test.org, then start a python shell with M-x run-python, which
should create a buffer named "*Python 3*"

4. Press C-c C-c on the src block. Then an error "No inferior Python
process running" is shown.
From 75ca16a21fe409aeb37b9bf0d97895c00f38d466 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 11 Jul 2023 10:49:07 +0800
Subject: [PATCH] ob-python: Fix async evaluation

* lisp/ob-python.el (org-babel-python-async-evaluate-session): Bind
`python-shell-buffer-name' inside the temp buffer.
---
 lisp/ob-python.el | 47 +--
 1 file changed, 25 insertions(+), 22 deletions(-)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 0e0539d7a..c15d45b96 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -400,28 +400,31 @@ (defun org-babel-python-async-evaluate-session
session (current-buffer)
"ob_comint_async_python_\\(.+\\)_\\(.+\\)"
'org-babel-chomp 'org-babel-python-async-value-callback)
-  (let ((python-shell-buffer-name (org-babel-python-without-earmuffs session)))
-(pcase result-type
-  (`output
-   (let ((uuid (org-id-uuid)))
- (with-temp-buffer
-   (insert (format org-babel-python-async-indicator "start" uuid))
-   (insert "\n")
-   (insert body)
-   (insert "\n")
-   (insert (format org-babel-python-async-indicator "end" uuid))
-   (python-shell-send-buffer))
- uuid))
-  (`value
-   (let ((tmp-results-file (org-babel-temp-file "python-"))
- (tmp-src-file (org-babel-temp-file "python-")))
- (with-temp-file tmp-src-file (insert body))
- (with-temp-buffer
-   (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
-   (insert "\n")
-   (insert (format org-babel-python-async-indicator "file" tmp-results-file))
-   (python-shell-send-buffer))
- tmp-results-file)
+  (pcase result-type
+(`output
+ (let ((uuid (org-id-uuid)))
+   (with-temp-buffer
+ (insert (format org-babel-python-async-indicator "start" uuid))
+ (insert "\n")
+ (insert body)
+ (insert "\n")
+ (insert (format org-babel-python-async-indicator "end" uuid))
+ (let ((python-shell-buffer-name
+(org-babel-python-without-earmuffs session)))
+   (python-shell-send-buffer)))
+   uuid))
+(`value
+ (let ((tmp-results-file (org-babel-temp-file "python-"))
+   (tmp-src-file (org-babel-temp-file "python-")))
+   (with-temp-file tmp-src-file (insert body))
+   (with-temp-buffer
+ (insert (org-babel-python-format-session-value tmp-src-file tmp-results-file result-params))
+ (insert "\n")
+ (insert (format org-babel-python-async-indicator "file" tmp-results-file))
+ (let ((python-shell-buffer-name
+(org-babel-python-without-earmuffs session)))
+   (python-shell-send-buffer)))
+   tmp-results-file
 
 (provide 'ob-python)
 
-- 
2.25.1



Re: [PATCH] ob-python: support header argument `:results file graphics'

2023-07-06 Thread Liu Hui
Jack Kamm  于2023年7月6日周四 11:49写道:

> I would propose the following instead: for ":results output graphics",
> ob-python should plot the gcf, and clear it beforehand. But for
> ":results value graphics", the ob-python block should return a
> matplotlib Figure object to plot, which would allow keeping and
> modifying a Figure between blocks.
>
> I actually proposed that behavior before in this patch:
>
> https://list.orgmode.org/87eenpfe77@gmail.com/
>
> But never wound up applying it -- the patch was rather large, with a lot
> of extra features, and I wasn't sure they were all worth the extra
> complexity.  Then life got in the way, and I never got around to
> revisiting ob-python plotting, until now.

I think your proposal about ":results graphics" is more flexible and
complies the documentation. Since the patch has no real problem and
the feature is useful indeed, I hope it can be merged instead of mine
after the problem of documentation is resolved.

As for other features in the patch, maybe it is better to convert
dict/dataframe/array to table/list only when the result type is
explicitly set to table or list?



Re: [PATCH] ob-python: support header argument `:results file graphics'

2023-07-05 Thread Liu Hui
Jack Kamm  于2023年7月5日周三 13:13写道:

> 1. Do you need to add a call to pyplot.gcf().clear(), in case of
> multiple blocks in a session?

I don't think so. Some users may want to keep the figure between
blocks, and they can always clear the figure themselves when
necessary.

> 2. Would it make sense to wrap in pyplot.rc_context, so that we can use
> the :width and :height arguments like ob-R? E.g.,
>
> with pyplot.rc_context({'figure.figsize': (8,5)}):
>  pyplot.plot([1,2,3,4,5])
>  pyplot.gcf().savefig('filename.png')
>
> Will create a png file with width 8 and height 5.

Thanks for your suggestion and I have added some arguments (e.g. :dpi)
in the updated patch. But rc_context doesn't work reliably with
multiple blocks in a session, i.e., the figure size may not change.

BTW, I have updated the patch to turn off the feature by default,
since it may break existing src blocks using `graphics'. WDYT? Thanks.



Re: [PATCH] ob-python: support header argument `:results file graphics'

2023-07-05 Thread Liu Hui
Ihor Radchenko  于2023年7月4日周二 19:29写道:

> That said, your patch should still work fine even with these
> considerations. But we may need to sort out this undocumented behaviour.
> Probably, an error like in `org-babel-graphical-output-file' should be
> thrown by `org-babel-generate-file-param'.

Yes. The main concern is that some users may rely on the undocumented
behavior.


> What about
>
> #+begin_src python :file "test.png" :results graphics file
> import matplotlib.pyplot as plt
> plt.plot([1,2,3,4,5])
> plt.savefig('test.png')
> plt.plot([5,4,3,2,1])
> plt.show()
> #+end_src

OK, I think that it is better to turn off the feature by default and I
have updated the patch. Inspired by ob-R, some extra header arguments
are supported. Thanks.
From 5f3fc8602c0eb1f6d0c072bd0c1a6bb883b8c9f9 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Wed, 5 Jul 2023 12:48:36 +0800
Subject: [PATCH] ob-python: Graphics output enhancement

* lisp/ob-python.el (org-babel-python-graphics-output-function): New
variable.
(org-babel-execute:python): Allow to save graphics output with a
user-defined function when the result type is `file graphics'.
(org-babel-python-matplotlib-graphics-output): New helper function to
generate code to save matplotlib-based graphics.
---
 lisp/ob-python.el | 33 +
 1 file changed, 33 insertions(+)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 0e0539d7a..4c304f83b 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -62,6 +62,14 @@ (defcustom org-babel-python-None-to 'hline
   :package-version '(Org . "8.0")
   :type 'symbol)
 
+(defvar org-babel-python-graphics-output-function nil
+  "A function generating Python code for producing graphics output.
+Specifically, for src blocks with `:results file graphics' header
+argument, the code returned by the function will be appended to
+the end of the src block and should produce the graphics file.
+It can be set to `org-babel-python-matplotlib-graphics-output'
+for matplotlib-based graphics.")
+
 (defun org-babel-execute:python (body params)
   "Execute a block of Python code with Babel.
 This function is called by `org-babel-execute-src-block'."
@@ -72,6 +80,9 @@ (defun org-babel-execute:python (body params)
 		   (cdr (assq :session params
  (result-params (cdr (assq :result-params params)))
  (result-type (cdr (assq :result-type params)))
+ (graphics-file (and (member "graphics" result-params)
+ (ignore-errors
+   (org-babel-graphical-output-file params
 	 (return-val (when (eq result-type 'value)
 		   (cdr (assq :return params
 	 (preamble (cdr (assq :preamble params)))
@@ -81,6 +92,10 @@ (defun org-babel-execute:python (body params)
 	   (org-babel-expand-body:generic
 	body params
 	(org-babel-variable-assignments:python params))
+   (when graphics-file
+ (if (functionp org-babel-python-graphics-output-function)
+ (funcall org-babel-python-graphics-output-function
+  graphics-file params)))
 	   (when return-val
 	 (format (if session "\n%s" "\nreturn %s") return-val
  (result (org-babel-python-evaluate
@@ -149,6 +164,24 @@ (defun org-babel-python-table-or-string (results)
 res)
   res)))
 
+(defun org-babel-python-matplotlib-graphics-output (out-file params)
+  "Return the Python code for saving graphics to `OUT-FILE'."
+  (let* ((allowed-args '(:dpi :format :backend))
+ (args (mapconcat
+	(lambda (pair)
+		  (if (member (car pair) allowed-args)
+		  (format ",%s=%S"
+			  (substring (symbol-name (car pair)) 1)
+			  (cdr pair)) ""))
+	params "")))
+(format "
+import matplotlib.pyplot
+if len(matplotlib.pyplot.get_fignums()) > 0:
+matplotlib.pyplot.gcf().savefig('%s'%s)
+else:
+raise RuntimeWarning('No figure is found')"
+out-file args)))
+
 (defvar org-babel-python-buffers '((:default . "*Python*")))
 
 (defun org-babel-python-session-buffer (session)
-- 
2.25.1



Re: [PATCH] ob-python: support header argument `:results file graphics'

2023-07-03 Thread Liu Hui
Ihor Radchenko  于2023年7月3日周一 19:41写道:

> This already works, even without the patch.

Yes. It means the patch doesn't break current usage.

> > In this case, `graphics' can be removed too.
>
> May you elaborate?
>
> #+begin_src python :results file
> implies that the result of execution is file _contents_.

It is consistent with current behavior, e.g. `:results file' is used
to produce graphics file
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-python.html

> This is not acceptable. If src blocks worked in the past, they should
> continue working without adjustments in user' setup. We can only break
> backwards-compatibility when there is a strong reason to do so. Adding a
> new feature is not such a reason.

Any src blocks worked in the past still work with this patch, without
any adjustments. The patch only makes the following src block, which
cannot produce `test.png' in the past, can produce `test.png'.

#+begin_src python :file "test.png" :results graphics file
import matplotlib.pyplot as plt
plt.plot([1,2,3,4,5])
#+end_src



Re: [PATCH] ob-python: support header argument `:results file graphics'

2023-07-03 Thread Liu Hui
Ihor Radchenko  于2023年7月3日周一 17:28写道:

> This might be a useful feature, but it will break the existing
> conventions, if used as in the patch. Currently, the above combination
> of :file and :results is treated as the following:
>
> ‘graphics’
>  When used along with ‘file’ type, the result is a link to the file
>  specified in ‘:file’ header argument.  However, unlike plain ‘file’
>  type, code block output is not written to the disk.  The block is
>  expected to generate the file by its side-effects only, as in the
>  following example:
>
>   #+begin_src shell :results file link :file "org-mode-unicorn.svg"
> wget -c "https://orgmode.org/resources/img/org-mode-unicorn.svg;
>   #+end_src
>
>   #+RESULTS:
>   [[file:org-mode-unicorn.svg]]
>
>  If ‘:file’ header argument is omitted, interpret source block
>  result as the file path.

I have updated the patch and ‘:file’ header argument can be omitted
now, e.g.

#+begin_src python :results graphics file
import matplotlib.pyplot as plt
plt.plot([1,2,3,4,5])
plt.savefig('test.png')
return 'test.png'
#+end_src

In this case, `graphics' can be removed too.

> What if the user wants to use, for example, pyplot instead of
> matplotlib?

pyplot is a module of matplotlib. If you mean users want to use other
graphics libraries, they may advise `org-babel-python-save-graphics'
if they want to use `graphics' in the header argument. If necessary,
ob-python may support such libraries too.
From 8ea69d66552ee0217b7f5c8089b91424b92c3b5a Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Mon, 3 Jul 2023 11:13:58 +0800
Subject: [PATCH] ob-python: support header argument `:results file graphics'

* lisp/ob-python.el (org-babel-execute:python): Save current figure to
file when the result type is `file graphics'.
(org-babel-python-save-graphics): New helper function to generate code
to save graphics.
---
 lisp/ob-python.el | 20 
 1 file changed, 20 insertions(+)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 0e0539d7a..6ee52313c 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -72,6 +72,9 @@ (defun org-babel-execute:python (body params)
 		   (cdr (assq :session params
  (result-params (cdr (assq :result-params params)))
  (result-type (cdr (assq :result-type params)))
+ (graphics-file (and (member "graphics" result-params)
+ (ignore-errors
+   (org-babel-graphical-output-file params
 	 (return-val (when (eq result-type 'value)
 		   (cdr (assq :return params
 	 (preamble (cdr (assq :preamble params)))
@@ -81,6 +84,8 @@ (defun org-babel-execute:python (body params)
 	   (org-babel-expand-body:generic
 	body params
 	(org-babel-variable-assignments:python params))
+   (when graphics-file
+ (org-babel-python-save-graphics graphics-file params))
 	   (when return-val
 	 (format (if session "\n%s" "\nreturn %s") return-val
  (result (org-babel-python-evaluate
@@ -149,6 +154,21 @@ (defun org-babel-python-table-or-string (results)
 res)
   res)))
 
+(defun org-babel-python-save-graphics (out-file _)
+  "Return the code for saving graphics to `OUT-FILE'."
+  (format "
+try:
+import matplotlib.pyplot
+if len(matplotlib.pyplot.get_fignums()) > 0:
+matplotlib.pyplot.gcf().savefig('%s')
+else:
+raise RuntimeError('No figure is found. You need to use matplotlib\
+ functions, such as plot and imshow, to produce the graphics.')
+except ModuleNotFoundError:
+raise RuntimeError('ob-python depends on matplotlib for saving graphics')
+"
+  out-file))
+
 (defvar org-babel-python-buffers '((:default . "*Python*")))
 
 (defun org-babel-python-session-buffer (session)
-- 
2.25.1



[PATCH] ob-python: support header argument `:results file graphics'

2023-07-02 Thread Liu Hui
Hi,

This patch adds graphics output support for ob-python via matplotlib.
Specifically, it allows to use header argument `:results file
graphics' as follows:

#+begin_src python :file "test.png" :results graphics file
import matplotlib.pyplot as plt
plt.plot([1,2,3,4,5])
#+end_src

The feature is described in the documentation as follows and has been
supported by ob-R, ob-julia, etc.

> ‘graphics’
>  When used along with ‘file’ type, the result is a link to the file
>  specified in ‘:file’ header argument.
From ee8d236310cd4bde4e33610b2b794022861fbd6a Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Mon, 3 Jul 2023 11:13:58 +0800
Subject: [PATCH] ob-python: support header argument `:results file graphics'

* lisp/ob-python.el (org-babel-execute:python): Save current figure to
file when the result type is `file graphics'.
(org-babel-python-save-graphics): New helper function to generate code
to save graphics.
---
 lisp/ob-python.el | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 0e0539d7a..9fae49a4d 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -72,6 +72,8 @@ (defun org-babel-execute:python (body params)
 		   (cdr (assq :session params
  (result-params (cdr (assq :result-params params)))
  (result-type (cdr (assq :result-type params)))
+ (graphics-file (and (member "graphics" result-params)
+ (org-babel-graphical-output-file params)))
 	 (return-val (when (eq result-type 'value)
 		   (cdr (assq :return params
 	 (preamble (cdr (assq :preamble params)))
@@ -81,6 +83,8 @@ (defun org-babel-execute:python (body params)
 	   (org-babel-expand-body:generic
 	body params
 	(org-babel-variable-assignments:python params))
+   (when graphics-file
+ (org-babel-python-save-graphics graphics-file params))
 	   (when return-val
 	 (format (if session "\n%s" "\nreturn %s") return-val
  (result (org-babel-python-evaluate
@@ -149,6 +153,21 @@ (defun org-babel-python-table-or-string (results)
 res)
   res)))
 
+(defun org-babel-python-save-graphics (out-file _)
+  "Return the code for saving graphics to `OUT-FILE'."
+  (format "
+try:
+import matplotlib.pyplot
+if len(matplotlib.pyplot.get_fignums()) > 0:
+matplotlib.pyplot.gcf().savefig('%s')
+else:
+raise RuntimeError('No figure is found. You need to use matplotlib\
+ functions, such as plot and imshow, to produce the graphics.')
+except ModuleNotFoundError:
+raise RuntimeError('ob-python depends on matplotlib for saving graphics')
+"
+  out-file))
+
 (defvar org-babel-python-buffers '((:default . "*Python*")))
 
 (defun org-babel-python-session-buffer (session)
-- 
2.25.1



[PATCH] org-agenda-filter-by-category: fix 'no category' error

2023-02-02 Thread Liu Hui
Hello,

To reproduce the bug:

1. emacs -Q --eval "(require 'org-habit)"
2. Open the following org file:

---begin--
* TODO foo
SCHEDULED: <2023-02-01 Wed .+2d/4d>
:PROPERTIES:
:STYLE:habit
:CATEGORY: foo
:END:

* TODO bar
SCHEDULED: <2023-02-01 Wed>
- end --

3. M-x org-agenda, then press < a to create an agenda buffer
4. Go to the line containing foo, press <

Expected result: the agenda buffer shows only foo

Actual result:
org-agenda-filter-by-category: No category at point
From 2ac07aeb64d0b12708b54ec8ef26265c64af630e Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Wed, 1 Feb 2023 13:35:49 +0800
Subject: [PATCH] org-agenda: Fix `org-agenda-filter-by-category'

* lisp/org-agenda.el (org-agenda-filter-by-category): Use
`org-agenda-get-category' to get the category at current line.
---
 lisp/org-agenda.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 4396efe81..1d1f2271b 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -8210,7 +8210,7 @@ (defun org-agenda-filter-by-category (strip)
   (if (and org-agenda-filtered-by-category
 	   org-agenda-category-filter)
   (org-agenda-filter-show-all-cat)
-(let ((cat (org-no-properties (org-get-at-eol 'org-category 1
+(let ((cat (org-no-properties (org-agenda-get-category
   (cond
((and cat strip)
 (org-agenda-filter-apply
-- 
2.25.1



Re: org-agenda-tag-filter-preset: maybe a recent bug?

2022-10-17 Thread Liu Hui


Hi Garjola,

The preset of filter is not supposed to be used with individual
command. The docstring of 'org-agenda-tag-filter-preset' says:

> The preset filter is a global property of the entire agenda view. In
> a block agenda, it will not work reliably to define a filter for one
> of the individual blocks. You need to set it in the global options
> and expect it to be applied to the entire view.

So you just need to preset the filter in the global options, e.g.

  ;; multi-block view
  ("W" "Work Daily Action List"
   ((agenda ""))
   ((org-agenda-tag-filter-preset
 (quote
  ("+work")

or

  ("W" "Work Daily Action List"
   agenda ""
   ((org-agenda-tag-filter-preset
 (quote
  ("+work")



Re: [PATCH] Re: [BUG] org-agenda-remove-restriction-lock does not remove file lock [9.5.2 (release_9.5.2-17-gea6b74 @ /nix/store/iqqk7iqfwmfc6r78xg2knyq7hww2mhs4-emacs-git-20220225.0/share/emacs/29.0.

2022-10-12 Thread Liu Hui

Hi Ihor,

Ihor Radchenko  writes:

> Could you please add docstrings and possibly code comments for
> `org-agenda-restrict', `org-agenda-restrict-begin',
> `org-agenda-restrict-end', `org-agenda-last-dispatch-buffer', and
> possibly other elated variables?

I have added docstrings for related variables except
`org-agenda-last-dispatch-buffer', which is actually not used by any
other org-mode code. Please see the patch below.

>From 907499f16769e5a5353170c13f09595584530139 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Wed, 12 Oct 2022 14:02:05 +0800
Subject: [PATCH] org-agenda: Make sure file restriction can be removed

* lisp/org-agenda.el: (org-agenda-restrict):
(org-agenda-restrict-begin):
(org-agenda-restrict-end):
(org-agenda-overriding-restriction): add docstrings.
(org-agenda):
(org-agenda-set-restriction-lock): Set `org-agenda-restrict' non-nil
during both temporary and extended file restriction.
(org-agenda-remove-restriction-lock): Revert commit df0e96ba4.
* testing/lisp/test-org-agenda.el (test-org-agenda/file-restriction):
Add a test.
---
 lisp/org-agenda.el  | 43 +
 testing/lisp/test-org-agenda.el | 28 +
 2 files changed, 56 insertions(+), 15 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 86ed6a5f5..82ca2913a 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2202,7 +2202,17 @@ string that it returns."
 (org-remap org-agenda-mode-map 'move-end-of-line 'org-agenda-end-of-line)
 
 (defvar org-agenda-menu) ; defined later in this file.
-(defvar org-agenda-restrict nil)
+(defvar org-agenda-restrict nil
+  "Non-nil means agenda restriction is active.
+This is an internal flag indicating either temporary or extended
+agenda restriction.  Specifically, it is set to t if the agenda
+is restricted to an entire file, and is set to the corresponding
+buffer if the agenda is restricted to a part of a file, e.g. a
+region or a substree.  In the latter case,
+`org-agenda-restrict-begin' and `org-agenda-restrict-end' are set
+to the beginning and the end of the part.
+
+See also `org-agenda-set-restriction-lock'.")
 (defvar org-agenda-follow-mode nil)
 (defvar org-agenda-entry-text-mode nil)
 (defvar org-agenda-clockreport-mode nil)
@@ -2734,10 +2744,16 @@ that have been changed along."
 
 ;;; Agenda dispatch
 
-(defvar org-agenda-restrict-begin (make-marker))
-(defvar org-agenda-restrict-end (make-marker))
+(defvar org-agenda-restrict-begin (make-marker)
+  "Internal variable used to mark the restriction beginning.
+It is only relevant when `org-agenda-restrict' is a buffer.")
+(defvar org-agenda-restrict-end (make-marker)
+  "Internal variable used to mark the restriction end.
+It is only relevant when `org-agenda-restrict' is a buffer.")
 (defvar org-agenda-last-dispatch-buffer nil)
-(defvar org-agenda-overriding-restriction nil)
+(defvar org-agenda-overriding-restriction nil
+  "Non-nil means extended agenda restriction is active.
+This is an internal flag set by `org-agenda-set-restriction-lock'.")
 
 (defcustom org-agenda-custom-commands-contexts nil
   "Alist of custom agenda keys and contextual rules.
@@ -2962,12 +2978,12 @@ Pressing `<' twice means to restrict to the current subtree or region
 	(move-marker org-agenda-restrict-begin (point))
 	(move-marker org-agenda-restrict-end
 			 (progn (org-end-of-subtree t)
-	 ((and (eq restriction 'buffer)
-	   (or (< 1 (point-min))
-		   (< (point-max) (1+ (buffer-size)
-	  (setq org-agenda-restrict (current-buffer))
-	  (move-marker org-agenda-restrict-begin (point-min))
-	  (move-marker org-agenda-restrict-end (point-max)
+	 ((eq restriction 'buffer)
+  (if (not (buffer-narrowed-p))
+  (setq org-agenda-restrict t)
+(setq org-agenda-restrict (current-buffer))
+	(move-marker org-agenda-restrict-begin (point-min))
+	(move-marker org-agenda-restrict-end (point-max))
 
   ;; For example the todo list should not need it (but does...)
   (cond
@@ -7958,7 +7974,7 @@ subtree."
 	  (message "Locking agenda restriction to subtree"))
   (put 'org-agenda-files 'org-restrict
 	   (list (buffer-file-name (buffer-base-buffer
-  (setq org-agenda-restrict nil)
+  (setq org-agenda-restrict t)
   (setq org-agenda-overriding-restriction 'file)
   (move-marker org-agenda-restrict-begin nil)
   (move-marker org-agenda-restrict-end nil)
@@ -7969,14 +7985,11 @@ subtree."
 (defun org-agenda-remove-restriction-lock ( noupdate)
   "Remove agenda restriction lock."
   (interactive "P")
-  (if (not (or org-agenda-restrict org-agenda-overriding-restriction))
+  (if (not org-agenda-restrict)
   (message "No agenda restriction to remove.")
 (delete-overlay org-agenda-restriction-lock-overlay)
 (delete-overlay org-speedbar-restriction-lock-overlay)

Re: [PATCH] Re: [BUG] org-agenda-remove-restriction-lock does not remove file lock [9.5.2 (release_9.5.2-17-gea6b74 @ /nix/store/iqqk7iqfwmfc6r78xg2knyq7hww2mhs4-emacs-git-20220225.0/share/emacs/29.0.

2022-10-09 Thread Liu Hui

Hi Ihor,

> Ihor Radchenko  writes:
>
> > Visuwesh  writes:
> >
> >> C-u C-c C-x < followed by C-c C-x > does not remove the file restriction
> >> lock.
> >>
> >> `org-agenda-remove-restriction-lock' checks for non-nil value of
> >> `org-agenda-restriction' but `org-agenda-set-restriction-lock' explicitly
> >> sets it to nil when TYPE is 'file.  Setting `org-agenda-restriction' to
> >> a dummy value like 'dummy gets the job done.
> >
> > Confirmed.
> >
> > The fix is attached.
> > Setting org-agenda-restriction to non-nil appears to be risky since
> > org-agenda-set-restriction-lock explicitly sets it to nil. So, I use
> > different approach.
>
> Fixed.
> Applied onto main via df0e96ba4.

File restriction can be also temporarily set by pressing '<' in the
agenda dispatcher, e.g. pressing 'C-c a < a' in an org-mode file.
`org-agenda-remove-restriction-lock' still cannot remove the temporary
file restriction with the fix.

Setting `org-agenda-restrict' to a non-nil value is a straightforward
way to fixing both cases. The variable is only tested in several
places and I don't find any problem with the change. Therefore I
suggest the attached patch, where the value of `org-agenda-restrict'
is changed from nil to t during temporary and extended file
restriction.

>From f4e46051fbb13adadbbafeebab343383e1bca35a Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Sun, 9 Oct 2022 15:00:50 +0800
Subject: [PATCH] org-agenda: Make sure file restriction can be removed

* lisp/org-agenda.el: (org-agenda):
(org-agenda-set-restriction-lock): Set `org-agenda-restrict' non-nil
during both temporary and extended file restriction.
(org-agenda-remove-restriction-lock): Revert commit df0e96ba4.
* testing/lisp/test-org-agenda.el (test-org-agenda/file-restriction):
Add a test.
---
 lisp/org-agenda.el  | 19 ---
 testing/lisp/test-org-agenda.el | 28 
 2 files changed, 36 insertions(+), 11 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 86ed6a5f5..a4f766cd9 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2962,12 +2962,12 @@ Pressing `<' twice means to restrict to the current subtree or region
 	(move-marker org-agenda-restrict-begin (point))
 	(move-marker org-agenda-restrict-end
 			 (progn (org-end-of-subtree t)
-	 ((and (eq restriction 'buffer)
-	   (or (< 1 (point-min))
-		   (< (point-max) (1+ (buffer-size)
-	  (setq org-agenda-restrict (current-buffer))
-	  (move-marker org-agenda-restrict-begin (point-min))
-	  (move-marker org-agenda-restrict-end (point-max)
+	 ((eq restriction 'buffer)
+  (if (not (buffer-narrowed-p))
+  (setq org-agenda-restrict t)
+(setq org-agenda-restrict (current-buffer))
+	(move-marker org-agenda-restrict-begin (point-min))
+	(move-marker org-agenda-restrict-end (point-max))

   ;; For example the todo list should not need it (but does...)
   (cond
@@ -7958,7 +7958,7 @@ subtree."
 	  (message "Locking agenda restriction to subtree"))
   (put 'org-agenda-files 'org-restrict
 	   (list (buffer-file-name (buffer-base-buffer
-  (setq org-agenda-restrict nil)
+  (setq org-agenda-restrict t)
   (setq org-agenda-overriding-restriction 'file)
   (move-marker org-agenda-restrict-begin nil)
   (move-marker org-agenda-restrict-end nil)
@@ -7969,14 +7969,11 @@ subtree."
 (defun org-agenda-remove-restriction-lock ( noupdate)
   "Remove agenda restriction lock."
   (interactive "P")
-  (if (not (or org-agenda-restrict org-agenda-overriding-restriction))
+  (if (not org-agenda-restrict)
   (message "No agenda restriction to remove.")
 (delete-overlay org-agenda-restriction-lock-overlay)
 (delete-overlay org-speedbar-restriction-lock-overlay)
 (setq org-agenda-overriding-restriction nil)
-(unless org-agenda-keep-restricted-file-list
-  ;; There is a request to keep the file list in place
-  (put 'org-agenda-files 'org-restrict nil))
 (setq org-agenda-restrict nil)
 (put 'org-agenda-files 'org-restrict nil)
 (move-marker org-agenda-restrict-begin nil)
diff --git a/testing/lisp/test-org-agenda.el b/testing/lisp/test-org-agenda.el
index 256f701df..bd96163e9 100644
--- a/testing/lisp/test-org-agenda.el
+++ b/testing/lisp/test-org-agenda.el
@@ -255,6 +255,34 @@ See https://list.orgmode.org/06d301d83d9e$f8b44340$ea1cc9c0$@tomdavey.com;
   (get-text-property (point) 'day
 (org-test-agenda--kill-all-agendas)))

+(ert-deftest test-org-agenda/file-restriction ()
+  "Test file restriction for org agenda."
+  (org-test-with-temp-text-in-file "* TODO Foo"
+(org-agenda-set-restriction-lock t)
+(org-agenda nil "t")
+(should (search-forw

Re: [PATCH] lisp/org-agenda.el: Fix filter preset problem for sticky agenda

2022-10-08 Thread Liu Hui


Ihor Radchenko  writes:

> Applied onto main with minor amendments (mostly added double space " "
> between sentences in docstrings and the commit message).
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=edf5afc1d833a814cf97e814194b83b82f26a294

Thanks!

> Also, while we are at it, may you take a look at
> https://list.orgmode.org/paxpr06mb7760f3a02d140e7de2baaec6c6...@paxpr06mb7760.eurprd06.prod.outlook.com/?
> AFAIR, it is caused by a similar symbol property approach employed in
> different place in org-agenda.el.

The issue seems related to other problem about the agenda restriction
rather than the use of symbol property. I will reply in other threads.



Re: [PATCH] lisp/org-agenda.el: Fix filter preset problem for sticky agenda

2022-10-05 Thread Liu Hui

Ihor Radchenko  writes:

>> +(defvar org-agenda-filters-preset nil
>> +  "Preset of filters, which becomes buffer-local in org-agenda buffers.")
>
> Can you detail the value format in the docstring?
> It would also be useful to mention `org-agenda-local-vars'. Otherwise,
> the reader might be confused how `defvar' is buffer-local.

Done.

> Also, may you create a test for the reported bug in
> testing/lisp/test-org-agenda.el?

Done.

When writing the test, I find the original patch only addresses the
case where an entry contains multiple commands. Now the udpated patch
can address the case containing one command by changing another global
symbol property to per-buffer text property.

>From 718cb5258a407d8a51eb4a5bac3d0c8025a3f198 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 4 Oct 2022 11:12:41 +0800
Subject: [PATCH] Fix filter preset problem for sticky agenda

* lisp/org-agenda.el (org-agenda-local-vars):
(org-agenda-filters-preset): Add a new variable
`org-agenda-filters-preset' for storing per-buffer filter presets.
(org-agenda):
(org-agenda-filter-any):
(org-agenda-prepare):
(org-agenda-finalize):
(org-agenda-redo):
(org-agenda-filter-by-tag):
(org-agenda-filter-make-matcher):
(org-agenda-set-mode-name):
(org-agenda-reapply-filters): Use `org-agenda-filters-preset' for
getting and setting per-buffer filter presets, rather than modifying
the global symbol property. Change `org-lprops' from symbol property
to per-buffer text property. Delete unused `last-args' symbol
property.
* testing/lisp/test-org-agenda.el (test-org-agenda/sticky-agenda-filter-preset):
(test-org-agenda/redo-setting): add tests.
---
 lisp/org-agenda.el  | 108 +++-
 testing/lisp/test-org-agenda.el |  60 ++
 2 files changed, 109 insertions(+), 59 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index e5df768ff..c303aead1 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2276,6 +2276,7 @@ When nil, `q' will kill the single agenda buffer."
 org-agenda-top-headline-filter
 org-agenda-regexp-filter
 org-agenda-effort-filter
+org-agenda-filters-preset
 org-agenda-markers
 org-agenda-last-search-view-search-was-boolean
 org-agenda-last-indirect-buffer
@@ -2929,10 +2930,6 @@ Pressing `<' twice means to restrict to the current subtree or region
 	(setq org-agenda-restrict nil)
 	(move-marker org-agenda-restrict-begin nil)
 	(move-marker org-agenda-restrict-end nil))
-  ;; Delete old local properties
-  (put 'org-agenda-redo-command 'org-lprops nil)
-  ;; Delete previously set last-arguments
-  (put 'org-agenda-redo-command 'last-args nil)
   ;; Remember where this call originated
   (setq org-agenda-last-dispatch-buffer (current-buffer))
   (unless org-keys
@@ -2981,7 +2978,6 @@ Pressing `<' twice means to restrict to the current subtree or region
 		(setq org-agenda-buffer-name
 		  (or (and (stringp org-match) (format "*Org Agenda(%s:%s)*" org-keys org-match))
 			  (format "*Org Agenda(%s)*" org-keys
-	  (put 'org-agenda-redo-command 'org-lprops lprops)
 	  (cl-progv
 	  (mapcar #'car lprops)
 	  (mapcar (lambda (binding) (eval (cadr binding) t)) lprops)
@@ -3016,7 +3012,10 @@ Pressing `<' twice means to restrict to the current subtree or region
 		   (funcall type org-match))
 		  ;; FIXME: Will signal an error since it's not `functionp'!
 		  ((pred fboundp) (funcall type org-match))
-		  (_ (user-error "Invalid custom agenda command type %s" type)
+		  (_ (user-error "Invalid custom agenda command type %s" type
+  (let ((inhibit-read-only t))
+	(add-text-properties (point-min) (point-max)
+			 `(org-lprops ,lprops
 	  (org-agenda-run-series (nth 1 entry) (cddr entry
((equal org-keys "C")
 	(setq org-agenda-custom-commands org-agenda-custom-commands-orig)
@@ -3808,6 +3807,10 @@ the entire agenda view.  In a block agenda, it will not work reliably to
 define a filter for one of the individual blocks.  You need to set it in
 the global options and expect it to be applied to the entire view.")
 
+(defvar org-agenda-filters-preset nil
+  "Alist of filter types and associated preset of filters.
+This variable is local in org-agenda buffers. See `org-agenda-local-vars'.")
+
 (defconst org-agenda-filter-variables
   '((category . org-agenda-category-filter)
 (tag . org-agenda-tag-filter)
@@ -3818,7 +3821,7 @@ the global options and expect it to be applied to the entire view.")
   "Is any filter active?"
   (cl-some (lambda (x)
 	 (or (symbol-value (cdr x))
-		 (get :preset-filter x)))
+ (assoc-default (car x) org-agenda-filters-preset)))
 	   org-agenda-filter-variables))
 
 (defvar org-agenda-category-filter-preset nil
@@ -3927,10 +3930,6 @@ FILTER-ALIST 

[PATCH] lisp/org-agenda.el: Fix filter preset problem for sticky agenda

2022-10-03 Thread Liu Hui

Hi,

This patch fixes the bug originally reported in
https://list.orgmode.org/59e02fb6.1462370a.fffe8.5...@mx.google.com/
with the following reproducing recipe. The bug still exists in latest
org-mode.

> To reproduce, take the following org file:
>
> ---8<---
> * Foo
>   :PROPERTIES:
>   :CATEGORY: foo
>   :END:
> ** TODO Foo todo one
>DEADLINE: <2017-10-13 Fri>
>- foo 1
>
> * Bar
>   :PROPERTIES:
>   :CATEGORY: bar
>   :END:
> ** TODO Bar todo one
>DEADLINE: <2017-10-13 Fri>
>- bar 1
> --->8---
>
> And the following elisp file to setup a minimal environment:
>
> ---8<---
> (require 'org)
> (setq org-agenda-files (list (expand-file-name "repro.org")))
> (setq org-agenda-sticky t)
> (setq org-agenda-custom-commands
>   '(("f" "foo"
>  ((agenda "" ())
>   (tags-todo "+CATEGORY=\"foo\"" ()))
>  ((org-agenda-category-filter-preset '("+foo"
>
> ("b" "bar"
>  ((agenda "" ())
>   (tags-todo "+CATEGORY=\"bar\"" ()))
>  ((org-agenda-category-filter-preset '("+bar"))
>
> (global-set-key (kbd "C-c a") #'org-agenda)
> --->8---
>
> Then run (something like):
>
> emacs -nw -Q -L ~/git/org-mode/lisp -l repro.el repro.org
>
> Followed by:
>
> C-c a f
> C-c a b
> C-x o
> r
>
> I would expect pressing `r' in the `*Org Agenda(f)*' buffer would
> keep the original `org-agenda-category-filter-preset' and that the
> preset in `*Org Agenda(b)*' should have no bearing on it.  But, as you
> can (probably) see from the mode line, the filter is set to "+bar", and
> refreshing the buffer causes all the TODOs to be lost because it now has
> the wrong filter.

Because the filter presets are stored as symbol properties, they
cannot be used with multiple sticky agenda buffers. The patch fixes
the problem by adding a new variable `org-agenda-filters-preset' for
getting and setting per-buffer filter presets. I have signed the FSF
copyright assignment paper.

Best,
Hui

>From 4fc03d86ab1df761ab26655f043b05b449b68716 Mon Sep 17 00:00:00 2001
From: Liu Hui 
Date: Tue, 4 Oct 2022 11:12:41 +0800
Subject: [PATCH] lisp/org-agenda.el: Fix filter preset problem for sticky
 agenda

* lisp/org-agenda.el (org-agenda-local-vars):
(org-agenda-filters-preset): Add a new variable
`org-agenda-filters-preset' for storing per-buffer filter presets.
(org-agenda-filter-any):
(org-agenda-prepare):
(org-agenda-finalize):
(org-agenda-redo):
(org-agenda-filter-by-tag):
(org-agenda-filter-make-matcher):
(org-agenda-set-mode-name):
(org-agenda-reapply-filters): Use `org-agenda-filters-preset' for
getting and setting per-buffer filter presets, rather than modifying
the global symbol property.
---
 lisp/org-agenda.el | 91 --
 1 file changed, 40 insertions(+), 51 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 2b56dd0fb..7ee0a6ad1 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2276,6 +2276,7 @@ When nil, `q' will kill the single agenda buffer."
 org-agenda-top-headline-filter
 org-agenda-regexp-filter
 org-agenda-effort-filter
+org-agenda-filters-preset
 org-agenda-markers
 org-agenda-last-search-view-search-was-boolean
 org-agenda-last-indirect-buffer
@@ -3808,6 +3809,9 @@ the entire agenda view.  In a block agenda, it will not work reliably to
 define a filter for one of the individual blocks.  You need to set it in
 the global options and expect it to be applied to the entire view.")
 
+(defvar org-agenda-filters-preset nil
+  "Preset of filters, which becomes buffer-local in org-agenda buffers.")
+
 (defconst org-agenda-filter-variables
   '((category . org-agenda-category-filter)
 (tag . org-agenda-tag-filter)
@@ -3818,7 +3822,7 @@ the global options and expect it to be applied to the entire view.")
   "Is any filter active?"
   (cl-some (lambda (x)
 	 (or (symbol-value (cdr x))
-		 (get :preset-filter x)))
+ (assoc-default (car x) org-agenda-filters-preset)))
 	   org-agenda-filter-variables))
 
 (defvar org-agenda-category-filter-preset nil
@@ -3927,10 +3931,6 @@ FILTER-ALIST is an alist of filters we need to apply when
 			(cat . ,org-agenda-category-filter))
 (if (org-agenda-use-sticky-p)
 	(progn
-	  (put 'org-agenda-tag-filter :preset-filter nil)
-	  (put 'org-agenda-category-filter :preset-filter nil)
-	  (put 'org-agenda-regexp-filter :preset-filter nil)
-	  (put 'org-agenda-effort-filter :preset-filter nil)
 	  ;; Popup existing buffer
 	  (org-agenda-prepare-window (get-buffer org-agenda-buffer-name)
  filter-alist)
@@ -3938,14 +3938,6 @@

[BUG] Inline images cannot be displayed [9.5 (release_9.5-108-g93132c @ /tmp/org-mode/lisp/)]

2021-10-15 Thread Liu Hui
Hi,

Org mode cannot display inline images because of an error caused by
org-display-inline-image--width. It seems display-line-numbers-width
is regarded as a number unconditionally.

Recipe:

1. emacs -Q

2. Make the following settings:

   (setq org-image-actual-width nil)
   (setq org-startup-with-inline-images t)

3. Open some org-mode file containing image links, e.g.

   #+attr_org: :width 50%
   [[file:~/test.jpg]]


Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
  -(80 nil)
  (or (and (and (boundp 'visual-fill-column-mode) visual-fill-column-mode) (or 
visual-fill-column-width auto-fill-function)) (if auto-fill-function (progn 
fill-column)) (- (window-text-width) display-line-numbers-width))
  (/ (or (and (and (boundp 'visual-fill-column-mode) visual-fill-column-mode) 
(or visual-fill-column-width auto-fill-function)) (if auto-fill-function (progn 
fill-column)) (- (window-text-width) display-line-numbers-width)) (float 
(window-total-width)))
  (* width (window-pixel-width) (/ (or (and (and (boundp 
'visual-fill-column-mode) visual-fill-column-mode) (or visual-fill-column-width 
auto-fill-function)) (if auto-fill-function (progn fill-column)) (- 
(window-text-width) display-line-numbers-width)) (float (window-total-width
  (round (* width (window-pixel-width) (/ (or (and (and (boundp 
'visual-fill-column-mode) visual-fill-column-mode) (or visual-fill-column-width 
auto-fill-function)) (if auto-fill-function (progn fill-column)) (- 
(window-text-width) display-line-numbers-width)) (float (window-total-width)
  (if (and (floatp width) (<= 0.0 width 2.0)) (round (* width 
(window-pixel-width) (/ (or (and (and (boundp ...) visual-fill-column-mode) (or 
visual-fill-column-width auto-fill-function)) (if auto-fill-function (progn 
fill-column)) (- (window-text-width) display-line-numbers-width)) (float 
(window-total-width) width)
  (let* ((case-fold-search t) (par (org-element-lineage link '(paragraph))) 
(attr-re "^[ \11]*#\\+attr_.*?: +.*?:width +\\(\\S-+\\)") (par-end 
(org-element-property :post-affiliated par)) (attr-width (if (and par (let 
((--mpom ...)) (save-excursion (if ... ...) (save-excursion ... (progn 
(match-string 1 (attr-width-val (cond ((null attr-width) nil) 
((string-match-p "\\`[0-9.]+%" attr-width) (/ (string-to-number attr-width) 
100.0)) (t (string-to-number attr-width (width (or attr-width-val (car 
org-image-actual-width (if (and (floatp width) (<= 0.0 width 2.0)) (round 
(* width (window-pixel-width) (/ (or (and (and ... visual-fill-column-mode) (or 
visual-fill-column-width auto-fill-function)) (if auto-fill-function (progn 
fill-column)) (- (window-text-width) display-line-numbers-width)) (float 
(window-total-width) width))
  (cond ((eq org-image-actual-width t) nil) ((listp org-image-actual-width) 
(let* ((case-fold-search t) (par (org-element-lineage link '(paragraph))) 
(attr-re "^[ \11]*#\\+attr_.*?: +.*?:width +\\(\\S-+\\)") (par-end 
(org-element-property :post-affiliated par)) (attr-width (if (and par (let ... 
...)) (progn (match-string 1 (attr-width-val (cond ((null attr-width) nil) 
((string-match-p "\\`[0-9.]+%" attr-width) (/ ... 100.0)) (t (string-to-number 
attr-width (width (or attr-width-val (car org-image-actual-width (if 
(and (floatp width) (<= 0.0 width 2.0)) (round (* width (window-pixel-width) (/ 
(or ... ... ...) (float ... width))) ((numberp org-image-actual-width) 
org-image-actual-width) (t nil))
  org-display-inline-image--width((link (:type "file" :path "~/test.jpg" 
:format bracket :raw-link "file:~/test.jpg" :application nil :search-option nil 
:begin 25 :end 51 :contents-begin nil :contents-end nil :post-blank 0 :parent 
(paragraph (:begin 2 :end 52 :contents-begin 25 :contents-end 52 :post-blank 0 
:post-affiliated 25 :attr_org (":width 50%") :parent nil)


Emacs  : GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, 
cairo version 1.16.0)
 of 2021-09-22
Package: Org mode version 9.5 (release_9.5-108-g93132c @ /tmp/org-mode/lisp/)



[O] Problem of refile in the org capture buffer

2017-02-17 Thread Liu Hui

Hello,

I find C-c C-w (refile) in the org capture buffer doesn't work correctly 
when there are empty lines (containing the point) in the end of buffer. 
Steps to reproduce:


1. echo -e "* A\n* B" > /tmp/test.org; cat /tmp/test.org
* A
* B

2. Emacs -Q, and evaluate the code:

(setq org-capture-templates
  '(("t" "Todo" entry (file+headline "/tmp/test.org" "A")
 "** test1 %?")))

3. M-x org-capture, and press t to open the org capture buffer

4. In the capture buffer, press C-j to insert a newline, then C-c C-w:

   => the subtree "B" will be refiled rather than the "test1".


The problem is caused by org-capture-finalize, which deletes the empty 
lines and, as a result, makes the point saved by org-capture-refile 
invalid. The following patch should fix the problem:


diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 1a1a500..27cb60b 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -827,6 +827,9 @@ already gone.  Any prefix argument will be passed to 
the refile command."

(base (buffer-base-buffer (current-buffer)))
(org-capture-is-refiling t)
(kill-buffer (org-capture-get :kill-buffer 'local)))
+(and (< (skip-chars-backward " \t\n") 0)
+(not (bobp))
+(setq pos (point)))
 (org-capture-put :kill-buffer nil)
 (org-capture-finalize)
 (save-window-excursion


Emacs  : GNU Emacs 26.0.50.2 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2016-12-17
Package: Org mode version 9.0.4 (release_9.0.4-283-g2064b0 @ 
~/org-mode/lisp/)





[O] Problem of refile in the org capture buffer

2017-02-17 Thread Liu Hui

Hello,

I find C-c C-w (refine) in the org capture buffer doesn't work correctly 
when there are empty lines (containing the point) in the end of buffer. 
Steps to reproduce:


1. echo -e "* A\n* B" > /tmp/test.org; cat /tmp/test.org
* A
* B

2. Emacs -Q, and evaluate the code:

(setq org-capture-templates
  '(("t" "Todo" entry (file+headline "/tmp/test.org" "A")
 "** test1 %?")))

3. M-x org-capture, and press t to open the org capture buffer

4. In the capture buffer, press C-j to insert a newline, then C-c C-w:

   => the subtree "B" will be refiled rather than the "test1".


The problem is caused by org-capture-finalize, which deletes empty lines 
and, as a result, makes the point saved by org-capture-refile invalid. 
The following patch should fix the problem:


diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 1a1a500..27cb60b 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -827,6 +827,9 @@ already gone.  Any prefix argument will be passed to 
the refile command."

(base (buffer-base-buffer (current-buffer)))
(org-capture-is-refiling t)
(kill-buffer (org-capture-get :kill-buffer 'local)))
+(and (< (skip-chars-backward " \t\n") 0)
+(not (bobp))
+(setq pos (point)))
 (org-capture-put :kill-buffer nil)
 (org-capture-finalize)
 (save-window-excursion


Emacs  : GNU Emacs 26.0.50.2 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2016-12-17
Package: Org mode version 9.0.4 (release_9.0.4-283-g2064b0 @ 
~/org-mode/lisp/)