Re: Cant expand a heading with tab : Subtree (no children)

[Rudolf Adamkovič]

Jonathan Gregory  writes:

> I see this too.

+1 here.

Rudy
-- 
"Thinking is a momentary dismissal of irrelevancies."
--- Richard Buckminster Fuller, 1969

Rudolf Adamkovič  [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union

[FR] org-clock-modify-effort-estimate as autoload

[Jason Hemann]

Howdy,

Presently, org-clock-modify-effort-estimate is not autoloaded. I imagine that 
org currently suggests/favors workflow would be that you’d clock in, or 
interact with org-clock some other way, and only then modify your effort 
estimates.

I found myself wanting to modify a prior effort estimates when not clocked and 
before the org-clock file had been loaded. This then made me think 
org-clock-modify-effort-estimate should be autoloaded, and the function act 
like another entry point.

Best,

Jason Hemann

Re: [PATCH] ob-lua: Support all types and multiple values in results

[Rudolf Adamkovič]

Ihor Radchenko  writes:

> Maybe something like the attached.

Nice:

  src_elisp{"foo"} {{{results(=foo=)}}}
  src_python{return "foo"} {{{results(=foo=)}}}
  src_lua{return "foo"} {{{results(=foo=)}}}

That said, I have just noticed:

  # expected: a, b, c
  src_lua{return "a", "b", "c"} {{{results(="a"  "b"  "c"=)}}}

  # expected: 1, 2
  src_lua{return 1, 2} {{{results(=1\, 2=)}}}

Oops!

We need *better tests*, otherwise all this is kind of useless.

I will hack on this some more...

Rudy
-- 
"Thinking is a momentary dismissal of irrelevancies."
--- Richard Buckminster Fuller, 1969

Rudolf Adamkovič  [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union

Re: Cant expand a heading with tab : Subtree (no children)

[Jonathan Gregory]

On 27 Apr 2024, Alexandros Prekates wrote:

It happens daily , not initially , but after some hour(s) of 
using an org file to try to expand a heading ,that i know that 
it has subheadings , and it wont expand and see in the echo area 
the message : SUBTREE (NO CHILDREN).


I see this too. The only way to access the subtree again is to 
kill the buffer and reopen the file.


Org mode version 9.6.15 (release_9.6.15 @ 
/snap/emacs/current/usr/share/emacs/30.0.50/lisp/org/)


--
Jonathan

Re: [PATCH] Re: [BUG] ob-shell: :shebang changes interpretation of :cmdline

[Max Nikulin]

On 27/04/2024 17:31, Max Nikulin wrote:

On 26/04/2024 18:49, Ihor Radchenko wrote:



+    shell-file-name

...

+    (list shell-command-switch
+  (concat (file-local-name 
script-file)  " " cmdline



Max Nikulin writes:

Using `shell-command-switch' unconditionally may lead to executing
/bin/sh instead of shell specified by `shell-file-name' for script 
files

having no shebang, see

https://superuser.com/questions/502984/writing-shell-scripts-that-will-run-on-any-shell-using-multiple-shebang-lines


I conclude that your concern, while being valid, is a _different_ bug.
Thus, I do not see it as a blocker for my patch - my patch will fix the
*original bug reported on top of this thread*.


My concern is that your patch trying to fix one bug (I am not convinced 
it is an improvement despite it is a step toward consistency) introduces 
another one that is not currently present in the code.


I read current variant of code once more. -c is unconditionally added, 
so your variant just makes the bug apparent. I still do not like behavior of


:cmdline 1 ; touch /tmp/not-an-arg

as I stated in

Re: [DISCUSSION] The meaning of :cmdline header argument across babel 
backends. Sat, 27 Apr 2024 17:53:25 +0700.

https://list.orgmode.org/v0ilf6$34l$1...@ciao.gmane.io

[PATCH] Rewrite `org-clock-sum'

[Morgan Smith]

Hello!

I may have rewritten org-clock-sum yet again.  See attached patch.

* things I want you to tell me
1. Does this look like something that could be eventually merged upstream or am
   I wasting my time?
2. Would you like me to do more performance testing?  I basically only tested
   my use case.  If yes, should I create some test files for benchmarking that
   can be shared?
3. Do you want `org-element-cache-map' fixed before we merge this patch?  If
   yes, please be willing to wait.  I have already spent probably about 8 hours
   looking into it and it still makes my head hurt.

* todo
The patch is like 95% done.  I still gotta

1. Write a decent docstring for `org-clock-ranges'.  Maybe add a news entry for
   it too.

2. Check `org-clock-hd-marker' for open clock.

3. Figure out what to do about open clocks that aren't the current
   one. Historically we ignored them so I guess I should just do that.

4. Maybe test clocking in inlinetasks.  I honestly don't even know what these
   are.

* Benefits of my rewrite

1. New function `org-clock-ranges' which should help third party packages with
   clock range visualization stuff

2. Performance (see table below)
   - We run the filter before doing all the clock range calculations unlike
 before so aggressive filters should run much faster (I didn't test this
 though).

3. Code is easier to understand (subjective)

* Downsides of my rewrite

1. Does it still perform better with the cache disabled?  idk.  Probably not.

2. Radical change.  Likely has bugs

3. Dances around bugs in `org-element-cache-map' but does it actually dance
   around all of them?

* Performance
I didn't see a big difference on the third run so I assume run 1 is with a cold
cache (obtained by running `org-element-cache-reset') and run 2 is with a warm
cache.

I have an almost 3M file of clocking data.  In it I have this source block
which I use to update my 10 clocktables:

#+BEGIN_SRC elisp
(let (;; (gc-cons-threshold (* 50 1000 1000))
  (start-time (current-time)))
  (org-dblock-update t)
  (time-to-seconds (time-since start-time)))
#+END_SRC

The time results are as follows


| patch   | run # | gc-cons-threshold | time (s) |
|-+---+---+--|
| origin/main | 1 |80 | 59.824324488 |
| mine| 1 |80 | 33.397901059 |
| origin/main | 2 |80 | 48.354095581 |
| mine| 2 |80 | 23.581749901 |
| origin/main | 1 |  5000 | 41.856530738 |
| mine| 1 |  5000 | 30.237918254 |
| origin/main | 2 |  5000 | 33.944309156 |
| mine| 2 |  5000 |  19.84887913 |

>From bfc01710186be01aab2186762cf678d360c5476e Mon Sep 17 00:00:00 2001
From: Morgan Smith 
Date: Thu, 11 Apr 2024 12:23:21 -0400
Subject: [PATCH] lisp/org-clock.el (org-clock-sum): Rewrite using element api

---
 lisp/org-clock.el | 191 +++---
 1 file changed, 94 insertions(+), 97 deletions(-)

diff --git a/lisp/org-clock.el b/lisp/org-clock.el
index 65a54579a..8731d6ee5 100644
--- a/lisp/org-clock.el
+++ b/lisp/org-clock.el
@@ -33,15 +33,13 @@
 
 (require 'cl-lib)
 (require 'org)
+(require 'org-element)
 
 (declare-function calendar-iso-to-absolute "cal-iso" (date))
 (declare-function notifications-notify "notifications" ( params))
 (declare-function org-element-property "org-element-ast" (property node))
-(declare-function org-element-contents-end "org-element" (node))
-(declare-function org-element-end "org-element" (node))
 (declare-function org-element-type "org-element-ast" (node  anonymous))
 (declare-function org-element-type-p "org-element-ast" (node types))
-(defvar org-element-use-cache)
 (declare-function org-inlinetask-at-task-p "org-inlinetask" ())
 (declare-function org-inlinetask-goto-beginning "org-inlinetask" ())
 (declare-function org-inlinetask-goto-end "org-inlinetask" ())
@@ -1998,6 +1996,9 @@ With prefix arg SELECT, offer recently clocked tasks for selection."
 (org-clock-sum (car r) (cadr r)
 		   headline-filter (or propname :org-clock-minutes-custom
 
+;;; TODO:
+;; Maybe add more tests?
+;; Are there tests for inlinetasks?
 ;;;###autoload
 (defun org-clock-sum ( tstart tend headline-filter propname)
   "Sum the times for each subtree.
@@ -2008,100 +2009,62 @@ each headline in the time range with point at the headline.  Headlines for
 which HEADLINE-FILTER returns nil are excluded from the clock summation.
 PROPNAME lets you set a custom text property instead of :org-clock-minutes."
   (with-silent-modifications
-(let* ((re (concat "^\\(\\*+\\)[ \t]\\|^[ \t]*"
-		   org-clock-string
-		   "[ \t]*\\(?:\\(\\[.*?\\]\\)-+\\(\\[.*?\\]\\)\\|=>[ \t]+\\([0-9]+\\):\\([0-9]+\\)\\)"))
-	   (lmax 30)
-	   (ltimes (make-vector lmax 0))
-	   (level 0)
-	   (tstart (cond ((stringp tstart) (org-time-string-to-seconds tstart))
-			 ((consp 

Re: [PATCH] ob-lua: Support all types and multiple values in results

[Max Nikulin]

On 26/04/2024 20:40, Ihor Radchenko wrote:

+++ b/lisp/ob-lua.el
@@ -282,6 +282,8 @@ (defvar org-babel-lua-wrapper-method
   end
end
return result
+   elseif type(it) == 'string' then
+  return '\"' .. it .. '\"'


If you are adding quotes around then quote characters in the value 
should be escaped.

Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

[Max Nikulin]

On 26/04/2024 20:09, Ihor Radchenko wrote:

Max Nikulin writes:


However looking wider, I do not like that :cmdline for ob-shell has
different meaning than for other languages, see e.g. ob-sql. Only for
shell this parameter is treated as arguments of a *script*. In other
cases :cmdline is used to specify arguments of *interpreter* and I think
ob-shell should follow this convention.


Alas, we already have the current state of affairs documented in
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-shell.html#orge70bc7b

So, no breaking changes.


It is documented as
" :cmdline  ... [arg_n]

Use the :cmdline header arg to pass arguments to a shell command."

However current implementation allows code injection through args, 
including a trivial one


#+header-arg: :results verbatim
#+begin_src sh :cmdline 1 ; touch /tmp/not-an-arg
  printf '%s\n' "$@"
#+end_src

#+RESULTS:
: 1

"touch ..." *are not arguments of the script*. So users should be 
careful to get documented behavior.



And shell scripts are not like SQL queries - they often do need to check
arguments. So, the current behaviour is justified, IMHO.


stackoverflow is full of suggestion how to pass arguments to a SQL 
script executed by mysql. Unfortunately it is unsafe and allows 
injection of code. psql (PostgreSQL) allows to pass parameters, however 
it is more like :var than script arguments. So it is true that CLI 
clients for SQL databases do not implement positional parameters.


ARGV is treated in a quite specific way by awk. It may contain file 
names, variable assignments, and might be overwritten in BEGIN block. 
However a close ob-awk header argument is :cmd-line, not :cmdline, so 
inconsistency is even greater.



What might be done is introducing _two_ different header arguments - one
for interpreter switches, and another for script/program switches.

Say, :interpreter-cmdline and :script-cmdline.
Then, we can call the current :cmdline behaviour "dwim" and allow users
to be more explicit if necessary.


It is too easy to confuse org-babel, so "dwim" works in simple cases 
only. Independent header arguments make things more clear, I would 
prefer :script-args. The question is whether they should be interpreted 
by shell (flexibility and shooting feet) or more strict syntax `("hello 
world" 1 a) should be used.

Re: [PATCH] Re: [BUG] ob-shell: :shebang changes interpretation of :cmdline

[Max Nikulin]

On 26/04/2024 18:49, Ihor Radchenko wrote:



+shell-file-name

...

+(list shell-command-switch
+  (concat (file-local-name script-file)  " " 
cmdline



Max Nikulin writes:

Using `shell-command-switch' unconditionally may lead to executing
/bin/sh instead of shell specified by `shell-file-name' for script files
having no shebang, see

https://superuser.com/questions/502984/writing-shell-scripts-that-will-run-on-any-shell-using-multiple-shebang-lines


I conclude that your concern, while being valid, is a _different_ bug.
Thus, I do not see it as a blocker for my patch - my patch will fix the
*original bug reported on top of this thread*.


My concern is that your patch trying to fix one bug (I am not convinced 
it is an improvement despite it is a step toward consistency) introduces 
another one that is not currently present in the code.



(with-temp-file script-file
(if shebang (insert shebang "\n")
   (insert "#!" shell-file-name "\n"))
(when padline (insert "\n"))
(insert body))


This code has an issue. Interpretation of relative file names in 
shebangs varies across shells.


If you insist on parsing :cmdline by shell (I do not like it) then you 
may try


  ' 
 '

[BUG] oddity tangling src_blk with :var [9.7-pre (release_9.6.26-1368-g1ae978 @ /home/minshall/.emacs.d/straight/build/org/)]

[Greg Minshall]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

 https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.


hi.  the following org block describes a problem i noticed.  possibly
the fix is simply issuing an appropriate error message (and aborting the
process of tangling.

thanks for looking at this when you have a chance.  and, apologies if
this is a FAQ.

cheers, Greg

#+begin_src org
  ,** introduction

  this is to show an oddity -- a bug, or a security feature? -- in
  tangling a source code block that has a variable (=:var=) header
  argument.

  if the variable =org-babel-load-languages= includes the source type of
  the code block (e.g., "shell", "python", etc.), the variable will be
  set in the tangled file.  if =org-babel-load-languages= does *not*
  include that source type, the variable will be unset in the tangled
  file.  without any error message.

  (note that in a running emacs, *removing* elements from
  =org-babel-load-languages= doesn't seem to cause this problem to
  appear.  maybe this is because the relevant "ob-..." files have
  already been loaded?)

  to see this, first *tangle* this file, to get works.el, fails.el, and
  variable.sh, and then evaluate this code block:

  ,#+name: DemonstrateBehavior
  ,#+begin_src sh :results output
./works.el
echo -n "works.el: "
./variable.sh
./fails.el
echo -n "fails.el: "
./variable.sh
  ,#+end_src

  ,#+RESULTS: DemonstrateBehavior
  : works.el: 2.4
  : fails.el: 

  notice that after running fails.el, the 2.4 disappears.


  ,** this is the test code and blocks

  here is the variable to be included.
  ,#+name: variable
  | 2.4 |

  and, the code block that includes that variable
  ,#+begin_src sh :var variable=variable :shebang #!/bin/sh :tangle variable.sh
echo "${variable}"
  ,#+end_src

  tangling with this works
  ,#+begin_src elisp :tangle works.el :shebang "#!/usr/bin/env -S emacs 
--script"
(org-babel-do-load-languages
 'org-babel-load-languages
 '((emacs-lisp . t)
   (shell . t)))
(org-babel-tangle-file "maybe.org")
  ,#+end_src

  tangling with this fails (notice the absence of =(shell . t)=)
  ,#+begin_src elisp :tangle fails.el :shebang "#!/usr/bin/env -S emacs 
--script"
(org-babel-do-load-languages
 'org-babel-load-languages
 '((emacs-lisp . t)))
(org-babel-tangle-file "maybe.org")
  ,#+end_src
#+end_src


Emacs  : GNU Emacs 30.0.50 (build 14, x86_64-pc-linux-gnu, GTK+ Version 
3.24.38, cairo version 1.18.0)
 of 2023-11-20
Package: Org mode version 9.7-pre (release_9.6.26-1368-g1ae978 @ 
/home/minshall/.emacs.d/straight/build/org/)

Re: [PATCH] Re: [BUG] ob-shell: :shebang changes interpretation of :cmdline

[Ihor Radchenko]

Matt  writes:

> There's a lot to process in this bug.  We've also uncovered, at least, 
> related four bugs.  I think it would help to submit separate bug reports for 
> each of the related issues so that we may discuss them separately, if 
> possible.  Is Woof! the way to do that?  I recall there being some issues 
> with Woof!.  Is there something not in https://tracker.orgmode.org/howto that 
> I should know?

You can just send emails with [BUG] in them, as usual. Or change email
subject to branch out a new thread, as I did for :cmdline argument discussion.

What you need to know about Woof! now is that it is not working. We
need to wait for Bastien to investigate. (Bastien does not have much
time though)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] Re: [BUG] ob-shell: :shebang changes interpretation of :cmdline

[Matt]

Thank you both for your thoughtful replies.  

There's a lot to process in this bug.  We've also uncovered, at least, related 
four bugs.  I think it would help to submit separate bug reports for each of 
the related issues so that we may discuss them separately, if possible.  Is 
Woof! the way to do that?  I recall there being some issues with Woof!.  Is 
there something not in https://tracker.orgmode.org/howto that I should know?

--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode