Re: time-warping - retroactively marking DONE?

[Adam Spiers]

On Wed, 8 Jul 2020 at 23:09, No Wayman  wrote:
> I emailed Adam directly with an experimental package I wrote to
> solve the problem of changing the todo-state of an entry at an
> arbitrary time.
> He suggested I posted here as well:
>
> https://github.com/progfolio/epoch/
>
> The package advises current-time to return `epoch-current-time' if
> is set (falling back to the usual current-time if not).
> A macro, `epoch-with-time' is provided which allows a body to be
> executed with current-time set to an arbitrary time.
> Two commands (which I may separate into their own package),
> `epoch-todo' and `epoch-agenda-todo' call their respective
> org-mode commands.
> `org-read-date' is called with the tasks's SCHEDULED or DEADLINE
> time pre-populated so one can easily edit relative to that time.
>
> Still very much a work in progress, but the two commands are
> useful for me so far.
>
> Any ideas, suggestions, criticisms are appreciated.

Many thanks again for this.  It's working great for me!

In case anyone's interested, here's my use-package config (which uses
the awesome straight.el package manager to install it):

https://github.com/aspiers/emacs/blob/aa62bd84b51a02cb0fc980862a63514349d253bf/.emacs.d/init.d/as-org-mode.el#L111-L116

I agree with your observation that it might be nicer to separate out
the org-specific stuff into a separate package, because the epoch
stuff seems useful in its own right outside org-mode.

Re: time-warping - retroactively marking DONE?

[No Wayman]

Adam Spiers  writes:


Many thanks again for this.  It's working great for me!

In case anyone's interested, here's my use-package config (which 
uses

the awesome straight.el package manager to install it):

https://github.com/aspiers/emacs/blob/aa62bd84b51a02cb0fc980862a63514349d253bf/.emacs.d/init.d/as-org-mode.el#L111-L116

I agree with your observation that it might be nicer to separate 
out
the org-specific stuff into a separate package, because the 
epoch

stuff seems useful in its own right outside org-mode.


Thanks for trying it out. There are still some rough edges I have 
to work out, but I plan on working on it more when I get some more 
free time.

Re: Help debugging R source code block output problem with :session

[Dylan Schwilk]

Thanks for the workaround idea of grabbing the results as a 
string.


I'm not sure why this issue cropped up for me right now when I did 
not run into it in previous years. But I have run into other 
:results output issues in the past when ess changed.


Thanks,

Dylan


On Sat 29 Aug 2020 at 15:35, Berry, Charles 
 wrote:


This problem has been bugging people for years and previous 
attempts to solve it have always run up against creating more 
problems in the process of solving this one.


This workaround gives the same results with or without `:session 
"NEW"' and same as OPs first src block:


#+header: :prologue "capture.output({" :epilogue "})"
#+begin_src R :results value verbatim :session "NEW" 
print("  ")

print("one  three")
print("end")
#+end_src

If you do decide to dig into solving this, please be sure that 
remote sessions and graphical outputs are not broken. 
test-ob-R.el does not cover those cases.  In fact, it is pretty 
short, so there are probably other things that could break 
without `make test' complaining.


HTH,

Chuck


On Aug 29, 2020, at 12:24 AM, Jack Kamm  
wrote:


Hi Dylan,

The patch does fix that issue -- but it introduces a different 
bug 
for code blocks with ~:session~: the R block now only produces 
output from the last statement evaluated.


Of course, you're right. Good catch.

Here's another attempt. It fixes the issue by modifying the R 
comint
regular expression, requiring it to match at the beginning of 
the line.



--

[feature request] A new cookie type [!] showing the last note taken

[Ihor Radchenko]

Over the years of using Org I often have a need to add a short note
about how to proceed with some task:

* REVIEW check again, subscribe | sindresorhus/awesome:  Awesome lists 
about all kinds of interesting topics :BOOKMARK:
:PROPERTIES:
:CREATED: [2020-03-15 Sun 18:59]
:Source: https://github.com/sindresorhus/awesome
:END:
:LOGBOOK:
CLOCK: [2020-03-17 Tue 16:18]--[2020-03-17 Tue 17:46] =>  1:28
CLOCK: [2020-03-17 Tue 16:03]--[2020-03-17 Tue 16:18] =>  0:15
- Refiled on [2020-03-16 Mon 23:59]
:END:

In the above example, the short note is "check again, subscribe".
The note is not fixed, but changes as I progress with completing the
task.

This is even more useful for delegated or HOLD tasks where I often need
to add a short note why the task is delegated or put on hold:

** HOLD Finish the text prop org-mode | make babel support org file links in 
header args (:file or :dir) 
[[id:468e0645-68aa-4e14-86de-e5ce153538e3][[2017-09-22 Fri] 
CuNbARBshearstrength]] :HOLD:
:PROPERTIES:
:CREATED: [2020-07-20 Mon 16:53]
:SHOWFROMDATE: 2020-08-15
:END:
:LOGBOOK:
- State "HOLD"   from "NEXT"  [2020-08-10 Mon 15:16] \\
  Finish the text prop org-mode
- Refiled on [2020-07-20 Mon 17:15]
CLOCK: [2020-07-20 Mon 16:53]--[2020-07-20 Mon 16:54] =>  0:01
:END:

Seeing this note directly in the headline without a need to dig into the
task body / LOGBOOK drawer is really handy.

In this last example, I had to duplicate the note taken using built-in
note mechanism into headline, which was inconvenient. It would be handy
if I could simply add a [!] cookie (similar to [/] or [%] cookies) to
the headline to show the last note taken for this task. Then, I could
easily see the reason why the task is blocked or what I am supposed to
do with the task right in agenda view or in the folded headline.
Something like the following

** HOLD [!] make babel support org... :HOLD:
:LOGBOOK:
- State "HOLD"   from "NEXT"  [2020-08-10 Mon 15:16] \\
  Finish the text prop org-mode
- Refiled on [2020-07-20 Mon 17:15]
CLOCK: [2020-07-20 Mon 16:53]--[2020-07-20 Mon 16:54] =>  0:01
:END:

The cookie would be replaced by the last note text, according to
user-defined format (say, "[%s] |"):

** HOLD [Finish the text prop org-mode] | make babel support org... :HOLD:
:LOGBOOK:
- State "HOLD"   from "NEXT"  [2020-08-10 Mon 15:16] \\
  Finish the text prop org-mode
- Refiled on [2020-07-20 Mon 17:15]
CLOCK: [2020-07-20 Mon 16:53]--[2020-07-20 Mon 16:54] =>  0:01
:END:

What do you think?

Best,
Ihor

Re: Help debugging R source code block output problem with :session

[Jack Kamm]

Hi Dylan,

> The patch does fix that issue -- but it introduces a different bug 
> for code blocks with ~:session~: the R block now only produces 
> output from the last statement evaluated.

Of course, you're right. Good catch.

Here's another attempt. It fixes the issue by modifying the R comint
regular expression, requiring it to match at the beginning of the line.

I think this should fix most cases, including the examples you
sent. Still, it's not totally robust -- for example, it will still
mangle multiline strings, if one of the lines starts with a substring
that looks like a prompt.

I'd be interested to hear if the attached patch works for the common
cases you encounter, such as with tibbles.

As an aside, I personally use an alternative implementation of ob-R
sessions that doesn't suffer from this issue, and also provides some
other benefits like async evaluation [1]. I'm planning to submit these
changes to org-mode someday, but am not ready yet. But you may want to
check it out, it was able to solve the issue in the other thread I
linked as well.

[1] https://github.com/jackkamm/ob-session-async

>From 9eaf81d708f88d06f14f9b6b9cf4182dd0fbb997 Mon Sep 17 00:00:00 2001
From: Jack Kamm 
Date: Sat, 29 Aug 2020 00:07:58 -0700
Subject: [PATCH] ob-R: Fix prompt mangling in session output

* lisp/ob-R.el (org-babel-R-evaluate-session): Force comint prompt
regexp to start at beginning of line, to prevent
org-babel-comint-with-output from splitting mid-line.

Fixes https://orgmode.org/list/875zgjh8wn@gmail.com/ and
https://orgmode.org/list/87r1rqled0.fsf@havana/
---
 lisp/ob-R.el | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index 5e9d35f58..10b3b6fe3 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -450,11 +450,13 @@ (defun org-babel-R-evaluate-session
 		  (car (split-string line "\n")))
 		 (substring line (match-end 1))
 		   line))
-	   (org-babel-comint-with-output (session org-babel-R-eoe-output)
-		 (insert (mapconcat 'org-babel-chomp
-(list body org-babel-R-eoe-indicator)
-"\n"))
-		 (inferior-ess-send-input)) "\n"
+	   (with-current-buffer session
+		 (let ((comint-prompt-regexp (concat "^" comint-prompt-regexp)))
+		   (org-babel-comint-with-output (session org-babel-R-eoe-output)
+		 (insert (mapconcat 'org-babel-chomp
+	(list body org-babel-R-eoe-indicator)
+	"\n"))
+		 (inferior-ess-send-input "\n"
 
 (defun org-babel-R-process-value-result (result column-names-p)
   "R-specific processing of return value.
-- 
2.28.0

Re: Problem in 9.3: (next-error) broken

[Kyle Meyer]

Tory S. Anderson writes:

> I discovered teh cause of this error is that new versions of orgmode
> sets next-error-function and so breaks the. I've added a hook to clear
> this in orgmode and I can navigate my grep properly again.

I'm not having any luck triggering the issue, but in any case there
seems to be something missing from that explanation.  Org has set
next-error-function to org-occur-next-match since dd2346134 (Implement
next-error and previous-error functionality for sparse trees,
2011-01-06), and the definition of org-occur-next-match hasn't changed
substantially since.

Re: babel latex headers and image generation commands

[Matt Huszagh]

Bastien  writes:

> If you find the time to share your change as a _patch_ (not the whole
> updated Elisp function), that will allow more readers on this list to
> quickly understand what is at stake.

Ok, I've finally gotten around to taking a crack at this. The patch is
attached. Basically, it allows a lot more control when converting a
latex source block into an svg image file.

Specifically, this new method does not place any restrictions on the
latex contents (the old svg generation required the use of
\documentclass[preview]{standalone}), or on the specific generation
commands (it does require first compiling to pdf then to svg, though I
guess this could be generalized if necessary). Additionally, it allows
you to use a setup independent of the setup used for inline latex
snippets. I think this is important because snippets and latex source
blocks have different use cases. For instance, I use snippets for simple
inline math (e.g. \(x=6\)) and source blocks for complicated full-blown
latex pictures and sets of equations, etc. that are not supported by
latex fragments.

I'll present my use case as an example (see the patch to understand the
customizations).

;; this is particular to my use case, see `latex-preamble-by-backend'
(defun by-backend (blist)
  (let ((ret nil))
(if org-export-current-backend
(let* ((backend-name org-export-current-backend)
   (elem (assoc backend-name blist)))
  (if elem
  (setq ret (cdr elem
  (let ((elem (assoc t blist)))
(setq ret (cdr elem
(eval ret)))

;; custom function for preamble generation
(defun latex-preamble-by-backend (params)
  (concat "\\documentclass{"
  (cdr (assoc :class params))
  "}"
  "\\definecolor{fg}{rgb}{"
  (by-backend '((html . "0,0,0")
(t . (org-latex-color :foreground
  "}\n"
  "\\definecolor{bg}{rgb}{"
  (by-backend '((html . "1,1,1")
(t . (org-latex-color :background
  "}\n"
  "\\def\\pc{"
  (by-backend '((html . "100")
(t . "20")))
  "}\n"))

;; actual set customization
(setq org-babel-latex-preamble
  (lambda (params)
(latex-preamble-by-backend params)))

Now if I define a source block:

#+header: :class math :results file link replace :file tmp/some-file.svg
#+begin_src latex :hidden
{\color{fg}
\begin{equation*}
|z| = \sqrt{x^2 + y^2}
\end{equation*}
}
#+end_src

Using my customization above, this will turn into the following latex

\documentclass{math}\definecolor{fg}{rgb}{0.819608,0.721569,0.592157}
\definecolor{bg}{rgb}{0.0235294,0.137255,0.160784}
\def\pc{20}
\begin{document}{\color{fg}
\begin{equation*}
|z| = \sqrt{x^2 + y^2}
\end{equation*}
}\end{document}

which is then turned into an svg with inkscape, though this could be
dvisvgm, or whatever you want.

Math is a custom document class I've defined. But my function also
allows the setting of other classes (which is useful because some
classes work better for tikz-like images and others work better for
math). In fact, since the preamble generation function can use any of
the source block parameters, I could make it do a bunch of other
conditional stuff. I've also configured it to set my color scheme in a
backend-dependent way. This allows me to get color schemes that match my
emacs theme when generating images for viewing within emacs and another
color scheme (in my case black foreground, white background) when
exporting to html. Note that none of this is enforced on other
people. My patch simply allows you to achieve this sort of flexibility.

I think this patch needs some discussing. I had some trouble deciding on
the best way to implement this functionality because its hard to get
anything to be thematically consistent with the current latex execute
command, which feels like a collection of loosely-related functionality
(for instance, the htlatex backend generates a completely different
latex source than the imagemagick backend, even though the outputs could
be quite similar -- svg and png). I think there's a lot of room for
improvement in the latex execute function that could make it more
general and flexible, but it's hard to do this in a way that doesn't
break existing workflows. My patch tries to be minimally invasive to
these workflows, though it will break workflows in an easily recoverable
way for anyone using htlatex for svg output.

Anyway, I'd be curious to hear thoughts and I'd be interested to discuss
options for further refactoring the latex execute function.

Matt

On Fri, Aug 28, 2020 at 11:10 PM Matt Huszagh  wrote:

> Bastien  writes:
>
> > If you find the time to share your change as a _patch_ (not the whole
> > updated Elisp function), that will allow more readers on this list to
> > quickly understand what is at stake.
>
> Ok, I've finally gotten around to taking a crack at this. The patch is
> attached. Basically, 

Re: new feature for consideration: other-tab for org-agenda-window-setup

[Kyle Meyer]

Kyle Meyer writes:

> I'll let this sit for another day to see if other comments come in.

Applied (e8ebf5d6c).

Re: Bug: org-capture entry sometimes ends without newline, garbles next headline [9.3.6 (9.3.6-elpa @ /home/arne/Disy/.emacs.d/elpa/org-9.3.6/)]

[Dr. Arne Babenhauserheide]

Kyle Meyer  writes:

> Dr. Arne Babenhauserheide writes:
>
>> Sometimes when I use org-capture to create a new headline, that headline
>> ends with a non-empty partial line that isn’t terminated by a newline.
>>
>> This causes the next headline to be corrupted, because the * is appended
>> to the last line of the new entry.
>
> I believe this issue has already been resolved on the master branch.
> There have been a few bug reports and commits in this area; I think
> 6882478ca (capture: Fix org-capture-place-entry narrow bounds,
> 2020-05-29) is the most recent fix.

Thank you for the info!

(how can I close the bug?)

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature

Re: Bug: no math-mode detection for align-environment [9.3.7 (9.3.7-13-ge62ca4-elpaplus @ /home/stefi/.emacs.d/elpa/org-plus-contrib-20200713/)]

[Stefi]

Great news! So this problem will just go away by itself then. Until then
I will use the texmathp-tex-commands solution as a workaround.

Thank you everyone for sharing your thoughts!

On 8/28/20 8:55 AM, Carsten Dominik wrote:
> Hi,
> 
> AUCTeX has taken back this change, realizing that texmathp.el is also
> used in contexts where the mechanisms of AUCTeX to find out about LaTeX
> packages being used does not work.  I do think this is already fixed in
> the latest AUCTeX version.  At least it is fixed in their git.
> 
> - Carsten
> 
> Carsten
> 
> On Fri, Aug 28, 2020 at 7:09 AM Kyle Meyer  > wrote:
> 
> Stefi writes:
> 
> > It might be a change to texmathp.el. It is part of Auctex and
> checks if
> > math mode is on or off. I could not find align environment in the list
> > of default environments. Maybe that has changed.
> >
> > However, I added align and align* to "Texmathp Tex Commands" from the
> > customize browser (open a .tex file to customize auctex).
> Tab-expansion
> > is working now. Can anyone confirm?
> >
> > In my .emacs, it added to custom-set-variables:
> > '(texmathp-tex-commands (quote (("align*" env-on) ("align" env-on
> 
> Yes, that results in expansion on my end as well.
> 
> It looks like this issue was introduced with AUCTeX's 91701704 (Delete
> overhead in extending font lock range of math expression, 2020-06-11).
> If you're getting AUCTeX from ELPA, the regression is included the
> current version there (12.2.4, 2020-06-29).
> 
> At the start of this month, it was fixed in f04a508f (Restore all math
> environments in texmathp.el, 2020-08-01), so the issue should go away
> with the next update that lands on ELPA.
> 

Re: bash babel code gives error

[Kyle Meyer]

Colin Baxter writes:

>> Colin Baxter  writes:
>
> > Hello, I have various .tex files and an org file in the same
> > directory. In the org file is the following simple bash babel-code
>
> > #+begin_src bash sha256sum *.tex > CHECKSUM #+end_src
>
>
> > This code block used to work but no longer.
>
> This is not wholly correct. It does indeed produce a checksum file
> CHECKSUM, but gives the "Error reading results:
> (beginning-of-buffer)". No such error was called in the past (~ a month or
> so ago).

Thanks for reporting.  The _display_ of this warning starts with my
14878f3f9 (ob-core: Display warning on failure to read results,
2020-05-21).  Here's the associated thread on the mailing list:
.

On that commit's parent (eecee2266), the error is triggered and caught
in the same way, but "Error reading results: (beginning-of-buffer)" is
buried in the messages buffer rather than being displayed as a warning.

I'll need to take a closer look at what's going on, though I wouldn't be
surprised if it's been around for a long time.

Re: Bug: org-capture entry sometimes ends without newline, garbles next headline [9.3.6 (9.3.6-elpa @ /home/arne/Disy/.emacs.d/elpa/org-9.3.6/)]

[Kyle Meyer]

Dr. Arne Babenhauserheide writes:

> (how can I close the bug?)

This list isn't hooked up to a tracker, so there's nothing to close.

Re: [RFC] ob-reticulate: R+Python interface from Babel

[Kyle Meyer]

Jack Kamm writes:

> Hi all,
>
> Reticulate is an R package for interfacing between R and Python. It
> allows accessing objects in a Python session from R and vice
> versa. See https://rstudio.github.io/reticulate/ for more info about
> it.
>
> I've written a small patch for using reticulate from org-babel. It
> allows creating a source block of lang "reticulate", which behaves as
> Python for font highlighting and editing, but is executed in an R
> session via reticulate.

Neat, thanks for sharing.

> I'm wondering whether this should go into org-mode, or whether to
> package this separately. I'm also curious whether this would be useful
> to anyone here. Any feedback is appreciated.

It'd be good to hear from others, but in my view this would be fine to
add to Org proper.

I will say that in general new Babel libraries make me nervous.  My
impression is that Babel brings in a good number of bug reports, and a
new library is adding surface to that area [*] while not necessarily
adding eyes and hands.  Of course that worry doesn't apply here, as
you're already taking care of ob-python.

  [*] And that surface can be quite challenging to deal with because it
  is not just Elisp code; it brings in a whole outside language that
  other developers may have no clue about.

> +;;; Code:
> +
> +(require 'ob-R)
> +(require 'ob-python)
> +
> +(defalias 'org-babel-edit-prep:reticulate 'org-babel-edit-prep:R)
> +
> +(defun org-babel-execute:reticulate (body params)
> +  (let* ((tmp-src-file (org-babel-temp-file "reticulate-"))
> +  (result-type (cdr (assq :result-type params
> +(with-temp-file tmp-src-file (insert body))
> +(org-babel-execute:R
> + (format (concat "reticulate::py_run_string(\"%s\")"
> +  (when (equal result-type 'value) "
> +reticulate::py$`__org_babel_python_final`"))
> +  (format org-babel-python--eval-ast
> +  (org-babel-process-file-name
> +   tmp-src-file 'noquote)))
> + params)))
> +
> +(provide 'ob-reticulate)

Well, that's pleasantly few lines of code :)

[PATCH] ob-core: Avoid table conversion warning for empty results

[Kyle Meyer]

Kyle Meyer writes:

> Thanks for reporting.  The _display_ of this warning starts with my
> 14878f3f9 (ob-core: Display warning on failure to read results,
> 2020-05-21).  Here's the associated thread on the mailing list:
> .
>
> On that commit's parent (eecee2266), the error is triggered and caught
> in the same way, but "Error reading results: (beginning-of-buffer)" is
> buried in the messages buffer rather than being displayed as a warning.
>
> I'll need to take a closer look at what's going on, though I wouldn't be
> surprised if it's been around for a long time.

This patch would squelch the inappropriate warning you report while not
hiding other warnings.

-- >8 --
Subject: [PATCH] ob-core: Avoid table conversion warning for empty results

* lisp/ob-core.el (org-babel-import-elisp-from-file): Don't try to
convert empty file to a table.
* testing/lisp/test-ob.el (test-ob/import-elisp-from-file): Add tests.

If org-babel-import-elisp-from-file is called with an empty file
(which many ob- libraries do when there are no results), feeding that
to org-table-import leads to a beginning-of-buffer error.  Before
14878f3f9 (ob-core: Display warning on failure to read results,
2020-05-21), this error was hard to notice because, after catching it,
it was reported as a quickly buried message.  After that commit, it is
displayed as a warning, which is not appropriate for the common and
unproblematic case of empty results.

Avoid the warning by only doing the table conversion if the file has
content.  Another option would be to do the table conversion but add a
beginning-of-buffer arm to the surrounding condition-case.  However,
that risks swallowing other sources of that error.

Reported-by: Colin Baxter 

---
 lisp/ob-core.el | 17 +++--
 testing/lisp/test-ob.el | 28 
 2 files changed, 39 insertions(+), 6 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..5b79919e8 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -3003,13 +3003,18 @@ (defun org-babel-import-elisp-from-file (file-name 
 separator)
 (with-temp-buffer
   (condition-case err
   (progn
-(org-table-import file-name separator)
+(insert-file-contents file-name)
 (delete-file file-name)
-(delq nil
-  (mapcar (lambda (row)
-(and (not (eq row 'hline))
- (mapcar #'org-babel-string-read row)))
-  (org-table-to-lisp
+(let ((pmax (point-max)))
+  ;; If the file was empty, don't bother trying to
+  ;; convert the table.
+  (when (> pmax 1)
+(org-table-convert-region (point-min) pmax separator)
+(delq nil
+  (mapcar (lambda (row)
+(and (not (eq row 'hline))
+ (mapcar #'org-babel-string-read row)))
+  (org-table-to-lisp))
 (error
  (display-warning 'org-babel
   (format "Error reading results: %S" err)
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index afaf13555..580cd7d89 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -2252,6 +2252,34 @@ (ert-deftest test-ob/string-to-number ()
 (should (=  0.1(org-babel--string-to-number "0.1")))
 (should (=  1.0(org-babel--string-to-number "1.0"
 
+(ert-deftest test-ob/import-elisp-from-file ()
+  "Test `org-babel-import-elisp-from-file'."
+  (should
+   (equal
+(org-test-with-temp-text-in-file "line 1\nline 2\n"
+  (cl-letf (((symbol-function 'display-warning)
+(lambda ( _) (error "No warnings should occur"))
+(org-table-convert-region-max-lines 2)))
+   (org-babel-import-elisp-from-file (buffer-file-name
+'(("line" 1)
+  ("line" 2
+  ;; If an error occurs during table conversion, it is shown with
+  ;; `display-warning' rather than as a message to make sure the
+  ;; caller sees it.
+  (should-error
+   (org-test-with-temp-text-in-file "line 1\nline 2\n"
+ (cl-letf (((symbol-function 'display-warning)
+   (lambda ( _) (error "Warning should be displayed")))
+  (org-table-convert-region-max-lines 1))
+   (org-babel-import-elisp-from-file (buffer-file-name)
+  ;; But an empty file (as is the case when there are no execution
+  ;; results) does not trigger a warning.
+  (should-not
+   (org-test-with-temp-text-in-file ""
+ (cl-letf (((symbol-function 'display-warning)
+   (lambda ( _) (error "No warnings should occur"
+   (org-babel-import-elisp-from-file 

Re: Problem in 9.3: (next-error) broken

[Tory S. Anderson]

Perhaps I was on an old version, although I don't think it was that old.

In any case, I keep having org-occur-next-match set away from nil, and I'm 
actually having trouble figuring out which hook will fix it, since 
org-mode-hook doesn't seem to be doing it. Instead I end up manually evaluating 
=(setq next-error-function nil)= every time I need next-error.

Any suggestions?

Kyle Meyer  writes:

> Tory S. Anderson writes:
>
>> I discovered teh cause of this error is that new versions of orgmode
>> sets next-error-function and so breaks the. I've added a hook to clear
>> this in orgmode and I can navigate my grep properly again.
>
> I'm not having any luck triggering the issue, but in any case there
> seems to be something missing from that explanation.  Org has set
> next-error-function to org-occur-next-match since dd2346134 (Implement
> next-error and previous-error functionality for sparse trees,
> 2011-01-06), and the definition of org-occur-next-match hasn't changed
> substantially since.

Re: [PATCH] ob-core: Avoid table conversion warning for empty results

[Colin Baxter]

Dear Kyle,

> Kyle Meyer  writes:

> Kyle Meyer writes:
>> Thanks for reporting.  The _display_ of this warning starts with
>> my 14878f3f9 (ob-core: Display warning on failure to read
>> results, 2020-05-21).  Here's the associated thread on the
>> mailing list:
>> .
>> 
>> On that commit's parent (eecee2266), the error is triggered and
>> caught in the same way, but "Error reading results:
>> (beginning-of-buffer)" is buried in the messages buffer rather
>> than being displayed as a warning.
>> 
>> I'll need to take a closer look at what's going on, though I
>> wouldn't be surprised if it's been around for a long time.

> This patch would squelch the inappropriate warning you report
> while not hiding other warnings.

Thank you. I'll apply the patch to my org-mode and report back.

Best wishes,

Colin Baxter.

Re: Help debugging R source code block output problem with :session

[Berry, Charles]

This problem has been bugging people for years and previous attempts to solve 
it have always run up against creating more problems in the process of solving 
this one.

This workaround gives the same results with or without `:session "NEW"' and 
same as OPs first src block:

#+header: :prologue "capture.output({" :epilogue "})"
#+begin_src R :results value verbatim :session "NEW" 
print("  ")
print("one  three")
print("end")
#+end_src

If you do decide to dig into solving this, please be sure that remote sessions 
and graphical outputs are not broken.  test-ob-R.el does not cover those cases. 
 In fact, it is pretty short, so there are probably other things that could 
break without `make test' complaining.

HTH,

Chuck


> On Aug 29, 2020, at 12:24 AM, Jack Kamm  wrote:
> 
> Hi Dylan,
> 
>> The patch does fix that issue -- but it introduces a different bug 
>> for code blocks with ~:session~: the R block now only produces 
>> output from the last statement evaluated.
> 
> Of course, you're right. Good catch.
> 
> Here's another attempt. It fixes the issue by modifying the R comint
> regular expression, requiring it to match at the beginning of the line.

[PATCH] Expanded ob-python results handling and plotting

[Jack Kamm]

The attached patch adds ob-python value results handling for the
following types of results:

- Dictionaries
- Numpy arrays
- Pandas dataframes
- Matplotlib figures

This is a bigger commit than I'm used to, so I thought I better send it
out before merging, in case someone notices obvious problems I missed.

Overview of changes:

Dictionaries are now transformed into alists before being converted to
lisp. Previously, they had been getting mangled, like so:

#+begin_src python
  return {"a": 1, "b": 2}
#+end_src

#+RESULTS:
| a | : | 1 | b | : | 2 |

But now they appear like so:

#+begin_src python
  return {"a": 1, "b": 2}
#+end_src

#+RESULTS:
| a | 1 |
| b | 2 |

Numpy arrays and pandas dataframes are also converted to tables
automatically now. Tables converted from Pandas dataframes have row and
column names.

To avoid conversion, you can specify "raw", "verbatim", "scalar", or
"output" in the ":results" header argument.

For plotting, you can specify "graphics" in the ":results"
header. You'll also need to provide a ":file" argument. The behavior
depends on whether using output or value results. For output results,
the current figure (pyplot.gcf) is cleared before evaluating, then the
result saved. For value results, the block is expected to return a
matplotlib Figure, which is saved. To set the figure size, do it from
within Python.

Here is an example of how to plot:

#+begin_src python :results output graphics file :file boxplot.svg
  import matplotlib.pyplot as plt
  import seaborn as sns
  plt.figure(figsize=(5, 5))
  tips = sns.load_dataset("tips")
  sns.boxplot(x="day", y="tip", data=tips)
#+end_src

>From 09f9c42bb629a356e1c36f04f69c8baf795b411b Mon Sep 17 00:00:00 2001
From: Jack Kamm 
Date: Tue, 25 Aug 2020 21:57:24 -0700
Subject: [PATCH] ob-python: Add results handling for dicts, dataframes,
 arrays, plots

* lisp/ob-python.el (org-babel-execute:python): Parse graphics-file
from params.
(org-babel-python--def-format-value): Python code for formatting
value results before returning.
(org-babel-python--output-graphics-wrapper): Python code for handling
output graphics results.
(org-babel-python--nonsession-value-wrapper): Replaces
org-babel-python-wrapper-method, org-babel-python-pp-wrapper-method.
(org-babel-python--session-output-wrapper): Renamed from
org-babel-python--exec-tmpfile.
(org-babel-python--session-value-wrapper): Renamed and modified from
org-babel-python--eval-ast.
(org-babel-python-evaluate-external-process): New parameter for
graphics file.
(org-babel-python-evaluate-session): New parameter for graphics file.

Added results handling for dictionaries, Pandas and numpy tables, and
matplotlib plots.
---
 etc/ORG-NEWS  |  16 +-
 lisp/ob-python.el | 122 +++---
 2 files changed, 96 insertions(+), 42 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 10658a970..4f9863a5b 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -66,8 +66,8 @@ to switch to the new signature.
 *** Python session return values must be top-level expression statements
 
 Python blocks with ~:session :results value~ header arguments now only
-return a value if the last line is a top-level expression statement.
-Also, when a None value is returned, "None" will be printed under
+return a value if the last line is a top-level expression statement,
+otherwise the result is None. Also, None will now show up under
 "#+RESULTS:", as it already did with ~:results value~ for non-session
 blocks.
 
@@ -235,6 +235,18 @@ Screen blocks now recognize the =:screenrc= header argument and pass
 its value to the screen command via the "-c" option. The default
 remains =/dev/null= (i.e. a clean screen session)
 
+*** =ob-python.el=: Support for more result types and plotting
+
+=ob-python= now recognizes dictionaries, numpy arrays, and pandas
+dataframes, and will convert them to org-mode tables when appropriate.
+
+When the header argument =:results graphic= is set, =ob-python= will
+use matplotlib to save graphics. The behavior depends on whether value
+or output results are used. For value results, the last line should
+return a matplotlib Figure object to plot. For output results, the
+current figure (as returned by =pyplot.gcf()=) is cleared and then
+plotted.
+
 *** =RET= and =C-j= now obey ~electric-indent-mode~
 
 Since Emacs 24.4, ~electric-indent-mode~ is enabled by default.  In
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 44e1b63e0..92ca82625 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -79,6 +79,8 @@ (defun org-babel-execute:python (body params)
 	  org-babel-python-command))
 	 (session (org-babel-python-initiate-session
 		   (cdr (assq :session params
+	 (graphics-file (and (member "graphics" (assq :result-params params))
+			 (org-babel-graphical-output-file params)))
  (result-params (cdr (assq :result-params params)))
  (result-type (cdr (assq :result-type params)))
 	 (return-val (when (and (eq 

Re: Problem in 9.3: (next-error) broken

[Kyle Meyer]

Tory S. Anderson writes:

> In any case, I keep having org-occur-next-match set away from nil, and
> I'm actually having trouble figuring out which hook will fix it, since
> org-mode-hook doesn't seem to be doing it. Instead I end up manually
> evaluating =(setq next-error-function nil)= every time I need
> next-error.
>
> Any suggestions? 

next-error-function is set in the define-derived-mode body for org-mode,
so something like

(add-hook 'org-mode-hook
  (lambda () (setq next-error-function nil)))

should work, and it appears to on my end.

Re: [PATCH] Expanded ob-python results handling and plotting

[Kyle Meyer]

Jack Kamm writes:

> The attached patch adds ob-python value results handling for the
> following types of results:
>
> - Dictionaries
> - Numpy arrays
> - Pandas dataframes
> - Matplotlib figures

Thanks.  Just a couple of drive-by comments...

> +try:
> +import pandas as pd
> +except ModuleNotFoundError:

ModuleNotFoundError wasn't added until Python 3.6, so I think it'd be
better to use its parent class, ImportError.

> +pass
> +else:
> +if isinstance(result, pd.DataFrame):
> +result = [[''] + list(result.columns), None] + \
> +[[i] + list(row) for i, row in result.iterrows()]

Should handling of Series also be added?

[PATCH] doc: Document C-. for jumping to today when choosing a timestamp

[Adam Spiers]

This useful key binding was previously missing from the manual.

Signed-off-by: Adam Spiers 
---
 doc/org-manual.org | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 3eb745b5d..7c291c3a7 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -6028,6 +6028,7 @@ calendar, or by pressing {{{kbd(RET)}}}, the date 
selected in the
 calendar is combined with the information entered at the prompt.  You
 can control the calendar fully from the minibuffer:
 
+#+kindex: C-.
 #+kindex: <
 #+kindex: >
 #+kindex: M-v
@@ -6041,6 +6042,7 @@ can control the calendar fully from the minibuffer:
 #+kindex: M-S-LEFT
 #+kindex: RET
 #+attr_texinfo: :columns 0.25 0.55
+| {{{kbd(C-.)}}}   | Jump to today. |
 | {{{kbd(RET)}}}   | Choose date at point in calendar.  |
 | {{{kbd(mouse-1)}}}   | Select date by clicking on it. |
 | {{{kbd(S-RIGHT)}}}   | One day forward.   |
-- 
2.27.0

[PATCH] org-refile.el: Add org-refile-reverse which toggles org-reverse-note-order

[Adam Spiers]

This is useful for prepending to the start of the target headline
instead of appending to the end, or vice-versa depending on
org-reverse-note-order.
---
 doc/org-manual.org | 10 ++
 etc/ORG-NEWS   |  9 +
 lisp/org-keys.el   |  1 +
 lisp/org-refile.el | 11 +++
 4 files changed, 31 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 3eb745b5d..e499367b7 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -7190,6 +7190,16 @@ special command:
   Copying works like refiling, except that the original note is not
   deleted.
 
+- {{{kbd(C-c C-M-w)}}} (~org-refile-reverse~) ::
+
+  #+kindex: C-c C-M-w
+  #+findex: org-refile-reverse
+  Works like refiling, except that it temporarily toggles how the
+  value of ~org-reverse-note-order~ applies to the current buffer.  So
+  if ~org-refile~ would append the entry as the last entry under the
+  target header, ~org-refile-reverse~ will prepend it as the first
+  entry, and vice-versa.
+
 ** Archiving
 :PROPERTIES:
 :DESCRIPTION: What to do with finished products.
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 10658a970..a3c8397fc 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -267,6 +267,15 @@ Source code block header argument =:file-mode= can set file
 permissions if =:file= argument is provided.
 
 ** New commands
+*** ~org-refile-reverse~
+
+Use default keybinding == to run command
+~org-refile-reverse~.  It is almost identical to ~org-refile~, except
+that it temporarily toggles how ~org-reverse-note-order~ applies to
+the current buffer.  So if ~org-refile~ would append the entry as the
+last entry under the target heading, ~org-refile-reverse~ will prepend
+it as the first entry, and vice-versa.
+
 *** ~org-table-header-line-mode~
 
 Turn on a minor mode to display the first data row of the table at
diff --git a/lisp/org-keys.el b/lisp/org-keys.el
index 37df29983..902651175 100644
--- a/lisp/org-keys.el
+++ b/lisp/org-keys.el
@@ -582,6 +582,7 @@ (define-key org-mode-map (kbd "") #'org-shifttab)
 (org-defkey org-mode-map (kbd "C-c ;") #'org-toggle-comment)
 (org-defkey org-mode-map (kbd "C-c C-w") #'org-refile)
 (org-defkey org-mode-map (kbd "C-c M-w") #'org-refile-copy)
+(org-defkey org-mode-map (kbd "C-c C-M-w") #'org-refile-reverse)
 (org-defkey org-mode-map (kbd "C-c /") #'org-sparse-tree) ;minor-mode reserved
 (org-defkey org-mode-map (kbd "C-c \\") #'org-match-sparse-tree) ;minor-mode r.
 (org-defkey org-mode-map (kbd "C-c RET") #'org-ctrl-c-ret)
diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 7eb0a9643..c6ff35535 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -384,6 +384,17 @@ (defun org-refile-copy ()
 
 (defvar org-capture-last-stored-marker)
 
+;;;###autoload
+(defun org-refile-reverse ( arg default-buffer rfloc msg)
+  "Invoke `org-refile', but temporarily toggling how
+~org-reverse-note-order~ applies to the current buffer.  So if
+`org-refile' would append the entry as the last entry under the
+target heading, ~org-refile-reverse~ will prepend it as the first
+entry, and vice-versa."
+  (interactive "P")
+  (let ((org-reverse-note-order (not (org-notes-order-reversed-p
+(org-refile arg default-buffer rfloc msg)))
+
 ;;;###autoload
 (defun org-refile ( arg default-buffer rfloc msg)
   "Move the entry or entries at point to another heading.
-- 
2.27.0

Re: Help debugging R source code block output problem with :session

[Dylan Schwilk]

Yes, the modified comint-prompt-regexp works for my use cases and 
I'm back to expected output and what I had last time I taught the 
course. This was my instinct for a simple solution as well, but 
I've not tested against remote sessions and graphical outputs as 
Charles Berry warns.


I'll take a look at your ob-session-async, thanks!

-Dylan


On Sat 29 Aug 2020 at 02:24, Jack Kamm  wrote:


Hi Dylan,

The patch does fix that issue -- but it introduces a different 
bug 
for code blocks with ~:session~: the R block now only produces 
output from the last statement evaluated.


Of course, you're right. Good catch.

Here's another attempt. It fixes the issue by modifying the R 
comint
regular expression, requiring it to match at the beginning of 
the line.


I think this should fix most cases, including the examples you
sent. Still, it's not totally robust -- for example, it will 
still
mangle multiline strings, if one of the lines starts with a 
substring

that looks like a prompt.

I'd be interested to hear if the attached patch works for the 
common

cases you encounter, such as with tibbles.

As an aside, I personally use an alternative implementation of 
ob-R
sessions that doesn't suffer from this issue, and also provides 
some
other benefits like async evaluation [1]. I'm planning to submit 
these
changes to org-mode someday, but am not ready yet. But you may 
want to
check it out, it was able to solve the issue in the other thread 
I

linked as well.

[1] https://github.com/jackkamm/ob-session-async

From 9eaf81d708f88d06f14f9b6b9cf4182dd0fbb997 Mon Sep 17 
00:00:00 2001

From: Jack Kamm 
Date: Sat, 29 Aug 2020 00:07:58 -0700
Subject: [PATCH] ob-R: Fix prompt mangling in session output

* lisp/ob-R.el (org-babel-R-evaluate-session): Force comint 
prompt

regexp to start at beginning of line, to prevent
org-babel-comint-with-output from splitting mid-line.

Fixes https://orgmode.org/list/875zgjh8wn@gmail.com/ and
https://orgmode.org/list/87r1rqled0.fsf@havana/
---
 lisp/ob-R.el | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index 5e9d35f58..10b3b6fe3 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -450,11 +450,13 @@ (defun org-babel-R-evaluate-session
  (car (split-string line "\n")))
 (substring line (match-end 1))
   line))
-	   (org-babel-comint-with-output (session 
org-babel-R-eoe-output)

-(insert (mapconcat 'org-babel-chomp
-(list body 
org-babel-R-eoe-indicator)

-   "\n"))
-(inferior-ess-send-input)) "\n"
+  (with-current-buffer session
+		 (let ((comint-prompt-regexp (concat "^" 
comint-prompt-regexp)))
+		   (org-babel-comint-with-output (session 
org-babel-R-eoe-output)

+(insert (mapconcat 'org-babel-chomp
+	(list body 
org-babel-R-eoe-indicator)

+   "\n"))
+(inferior-ess-send-input "\n"
 
 (defun org-babel-R-process-value-result (result column-names-p)

   "R-specific processing of return value.



--