Re: noweb-start and noweb-end header args

[Amy Grinn]

Ihor Radchenko  writes:

> Amy Grinn  writes:
>
>> Here is a simple way to implement this feature.
>
> Since you are adding a new feature, it should have (1) test coverage;
> (2) be documented in the manual; (3) be announced in etc/ORG-NEWS.

Thank you for the tips, will do!

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

> I kinda disagree with your reasoning but I agree with your conclusion.
> I think it would be strange for third party or user configuration to
> parse the value of a noweb header argument directly.  Org provides a
> lublic api for this which should be backwards compatible:
>
>   (org-babel-noweb-p (nth 2 (org-babel-get-src-block-info t)) :tangle)

Yeah, but this API is not uniform. Most other header arguments must be
retrieved using `assq'. So, it is very common to use `assq' for
everything, even if there is a specific API.
(Ideally, we need to implement a uniform API to retrieve header argument
values)

> The middle way is to create just one new header arg:
>
> :noweb-wrap  [end]

Maybe  should be mandatory. Otherwise, LGTM.


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[Amy Grinn]

Ihor Radchenko  writes:

> :noweb yes <<< >>> is actually backwards-incompatible. Consider
> third-party code that makes assumptions about possible values of
> :noweb
> header argument. If third-party code does a check like
> (equal noweb-value "yes"), the new syntax can break such code.

I kinda disagree with your reasoning but I agree with your conclusion.
I think it would be strange for third party or user configuration to
parse the value of a noweb header argument directly.  Org provides a
lublic api for this which should be backwards compatible:

  (org-babel-noweb-p (nth 2 (org-babel-get-src-block-info t)) :tangle)

However, other parsers besides Emacs exist for Org mode and they may
directly compare the entire noweb argument.

The middle way is to create just one new header arg:

:noweb-wrap  [end]

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

> Here is a simple way to implement this feature.
> ...
> -(defun org-babel-noweb-wrap ( regexp)
> +(defun org-babel-noweb-wrap ( regexp info)
>"Return regexp matching a Noweb reference.
>  
>  Match any reference, or only those matching REGEXP, if non-nil.
>  
>  When matching, reference is stored in match group 1."

INFO argument needs to be documented.

> -  (org-babel-noweb-wrap)
> +  (org-babel-noweb-wrap nil info)

Please, also update all other cases when `org-babel-noweb-wrap' is
called.

Also, `org-babel-goto-named-src-block' uses
org-babel-noweb-wrap-start/end directly. It should be adjusted.

Since you are adding a new feature, it should have (1) test coverage;
(2) be documented in the manual; (3) be announced in etc/ORG-NEWS.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

> To expand on this, some major modes can fundamentally conflict with the
> default noweb syntax.  Here is a valid shell script *and* a valid noweb
> reference to a block named EOF:
>
> cat <> file.txt
> Hello
> EOF
>
> I hope this helps explain why the wrap-start and wrap-end options were
> necessary to include more than a decade ago.  In terms of actually using
> them, it's a bit cumbersome, especially in Org mode buffers that use
> multiple languages.

This makes sense.
I agree that setting noweb reference syntax per-language is useful in
some cases.

> The second diff I sent (under the termux handle, accidentally) is my
> preferred solution (:noweb yes <<< >>>).  This would avoid the need for
> new header arguments to be introduced while maintaining backwards
> compatibility.  It also feels natural to specify the two options
> together: I can't think of a good reason to only need to specify the
> wrap-end option.

:noweb yes <<< >>> is actually backwards-incompatible. Consider
third-party code that makes assumptions about possible values of :noweb
header argument. If third-party code does a check like
(equal noweb-value "yes"), the new syntax can break such code.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

>> Org mode does not _currently_ modify the code. But that's actually wrong
>> - things like escaped ,* or indentation sometimes also stay on the way
>> and produce incorrect fontification. So, rewriting the fontification of
>> src blocks to cleanup the code before fontification is long due.
>> noweb references is just another manifestation of this problem.
>
> I think we're talking past each other a little.  I'm not talking about
> changing the text content of a src block, I'm talking about modifying
> the syntax table of a major mode such as sh-mode to ignore or handle
> <> syntax in an "edit-special" buffer.  That was my
> interpretation of your suggestion of using fontification to solve this
> issue.  And if that's the case, I foresee a lot of edge cases for
> modifying the display of major modes.

That's not what I had in mind. I thought of resolving/replacing noweb
references before fontifying the code. That way, the major mode for src
block will simply not see <> text and will not be confused.

>> I am not in favor of adding features that aim to serve as workarounds to
>> Org mode.
>
> This discussion is not about whether to allow users to modify noweb
> syntax.  That feature is already a part of Org, well documented, and
> utilized.  The feature request I'm making is to allow that modification
> to be done on a per-block level.

Sure, but I wanted to hear why such feature is useful in practice. Your
example with fontification is not something I consider as a good
justification for adding a new feature. You another email provides a
better justification though.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[Amy Grinn]

Amy Grinn  writes:

> Ihor Radchenko  writes:
>
>> Amy Grinn  writes:
>>
>>> I would like to add support for setting 'org-babel-noweb-wrap-start and
>>> 'org-babel-noweb-wrap-end for each src block individually
>>
>> May you please explain the use case when changing the default values
>> is useful?
>
> Of course!  Changing the default values can be useful to prevent syntax
> highlighting errors in a specific language.  In the example I gave, <<<
> and >>> aren't recognized as the beginning of a heredoc in a shell
> script the way <> would be.

To expand on this, some major modes can fundamentally conflict with the
default noweb syntax.  Here is a valid shell script *and* a valid noweb
reference to a block named EOF:

cat <> file.txt
Hello
EOF

I hope this helps explain why the wrap-start and wrap-end options were
necessary to include more than a decade ago.  In terms of actually using
them, it's a bit cumbersome, especially in Org mode buffers that use
multiple languages.

The second diff I sent (under the termux handle, accidentally) is my
preferred solution (:noweb yes <<< >>>).  This would avoid the need for
new header arguments to be introduced while maintaining backwards
compatibility.  It also feels natural to specify the two options
together: I can't think of a good reason to only need to specify the
wrap-end option.

Re: noweb-start and noweb-end header args

[Amy Grinn]

Ihor Radchenko  writes:

> Amy Grinn  writes:
>
>> How much does org mode modify the fontification for an indirect buffer?
>> Without having looked into it, I assume not much or at all.
>> ... I think
>> that approach could be more complex, especially when dealing with a
>> theoretically infinite number of major modes.
>
> Org mode does not _currently_ modify the code. But that's actually wrong
> - things like escaped ,* or indentation sometimes also stay on the way
> and produce incorrect fontification. So, rewriting the fontification of
> src blocks to cleanup the code before fontification is long due.
> noweb references is just another manifestation of this problem.

I think we're talking past each other a little.  I'm not talking about
changing the text content of a src block, I'm talking about modifying
the syntax table of a major mode such as sh-mode to ignore or handle
<> syntax in an "edit-special" buffer.  That was my
interpretation of your suggestion of using fontification to solve this
issue.  And if that's the case, I foresee a lot of edge cases for
modifying the display of major modes.

>> Both solutions could be implemented at the same time.  We could build on
>> the existing functionality of the wrap-end and wrap-start variables
>> while also looking at ways to modify the syntax highlighting without
>> user intervention.
>
> I am not in favor of adding features that aim to serve as workarounds to
> Org mode.

This discussion is not about whether to allow users to modify noweb
syntax.  That feature is already a part of Org, well documented, and
utilized.  The feature request I'm making is to allow that modification
to be done on a per-block level.

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

>> This sounds like XY problem then.
>> If the real problem you want to solve is fontification, we may instead
>> adjust Org mode fontification of source blocks to exclude noweb
>> references.
>
> I see a problem with multiple possible solutions, some more involved
> than others.  The org-babel-noweb-wrap-* variables are already
> customizeable and, in researching a solution to this problem, I have
> found users who set these variables on a file or directory-local level
> already.

Yup. And most of these customizations are aiming to solve the
fontification problem. If we did not have the problem to start with,
re-defining the noweb wrap syntax would be unnecessary in many cases.

> How much does org mode modify the fontification for an indirect buffer?
> Without having looked into it, I assume not much or at all.
> ... I think
> that approach could be more complex, especially when dealing with a
> theoretically infinite number of major modes.

Org mode does not _currently_ modify the code. But that's actually wrong
- things like escaped ,* or indentation sometimes also stay on the way
and produce incorrect fontification. So, rewriting the fontification of
src blocks to cleanup the code before fontification is long due.
noweb references is just another manifestation of this problem.

> Both solutions could be implemented at the same time.  We could build on
> the existing functionality of the wrap-end and wrap-start variables
> while also looking at ways to modify the syntax highlighting without
> user intervention.

I am not in favor of adding features that aim to serve as workarounds to
Org mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[Amy Grinn]

Ihor Radchenko  writes:

> Amy Grinn  writes:
>
 #+name: firewall
 #+begin_src sh  :noweb yes :noweb-start <<< :noweb-end >>>
>>>
>>> May you please explain the use case when changing the default values
>>> is useful?
>>
>> Of course!  Changing the default values can be useful to prevent syntax
>> highlighting errors in a specific language.  In the example I gave, <<<
>> and >>> aren't recognized as the beginning of a heredoc in a shell
>> script the way <> would be.
>
> This sounds like XY problem then.
> If the real problem you want to solve is fontification, we may instead
> adjust Org mode fontification of source blocks to exclude noweb
> references.

I see a problem with multiple possible solutions, some more involved
than others.  The org-babel-noweb-wrap-* variables are already
customizeable and, in researching a solution to this problem, I have
found users who set these variables on a file or directory-local level
already.

How much does org mode modify the fontification for an indirect buffer?
Without having looked into it, I assume not much or at all.  I think
that approach could be more complex, especially when dealing with a
theoretically infinite number of major modes.

Both solutions could be implemented at the same time.  We could build on
the existing functionality of the wrap-end and wrap-start variables
while also looking at ways to modify the syntax highlighting without
user intervention.

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

>>> #+name: firewall
>>> #+begin_src sh  :noweb yes :noweb-start <<< :noweb-end >>>
>>
>> May you please explain the use case when changing the default values
>> is useful?
>
> Of course!  Changing the default values can be useful to prevent syntax
> highlighting errors in a specific language.  In the example I gave, <<<
> and >>> aren't recognized as the beginning of a heredoc in a shell
> script the way <> would be.

This sounds like XY problem then.
If the real problem you want to solve is fontification, we may instead
adjust Org mode fontification of source blocks to exclude noweb
references.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[Amy Grinn]

Ihor Radchenko  writes:

> Amy Grinn  writes:
>
>> I would like to add support for setting 'org-babel-noweb-wrap-start and
>> 'org-babel-noweb-wrap-end for each src block individually using the
>> header args :noweb-start and :noweb-end:
>> ...
>> #+name: firewall
>> #+begin_src sh  :noweb yes :noweb-start <<< :noweb-end >>>
>
> May you please explain the use case when changing the default values
> is useful?

Of course!  Changing the default values can be useful to prevent syntax
highlighting errors in a specific language.  In the example I gave, <<<
and >>> aren't recognized as the beginning of a heredoc in a shell
script the way <> would be.

Re: noweb-start and noweb-end header args

[Ihor Radchenko]

Amy Grinn  writes:

> I would like to add support for setting 'org-babel-noweb-wrap-start and
> 'org-babel-noweb-wrap-end for each src block individually using the
> header args :noweb-start and :noweb-end:
> ...
> #+name: firewall
> #+begin_src sh  :noweb yes :noweb-start <<< :noweb-end >>>

May you please explain the use case when changing the default values is useful?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: noweb-start and noweb-end header args

[termux]

Amy Grinn  writes:

> I would like to add support for setting 'org-babel-noweb-wrap-start and
> 'org-babel-noweb-wrap-end for each src block individually using the
> header args :noweb-start and :noweb-end:

Here's another possible syntax we could use:

:noweb  [wrap-start] [wrap-end]

#+name: message
#+begin_src elisp
  "Firewall is now in safe mode."
#+end_src

#+name: firewall-safe-mode
#+begin_src sh :noweb yes
  echo <>
#+end_src

#+name: firewall
#+begin_src sh :noweb yes <<< >>>
  safe_mode () {
  echo "Error encountered, switching to safe mode."
  <<>>
  exit 1
  }
#+end_src

#+begin_src sh :noweb yes ":-) " " (-:" :tangle "test.sh"
  # Setup firewall
  :-) firewall (-:

  # Do other things
#+end_src

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 4dcfbd3b0..f60b4be12 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -194,15 +194,20 @@ This string must include a \"%s\" which will be replaced by the results."
   :package-version '(Org . "9.1")
   :safe #'booleanp)
 
-(defun org-babel-noweb-wrap ( regexp)
+(defun org-babel-noweb-wrap ( regexp info)
   "Return regexp matching a Noweb reference.
 
 Match any reference, or only those matching REGEXP, if non-nil.
 
 When matching, reference is stored in match group 1."
-  (concat (regexp-quote org-babel-noweb-wrap-start)
-	  (or regexp "\\([^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
-	  (regexp-quote org-babel-noweb-wrap-end)))
+  (let ((noweb (mapcar
+(lambda (token)
+  (if (stringp token) token (symbol-name token)))
+(org-babel-read
+ (format "'(%s)" (cdr (assq :noweb (nth 2 info
+(concat (regexp-quote (or (nth 1 noweb) org-babel-noweb-wrap-start))
+	(or regexp "\\([^ \t\n]\\(?:.*?[^ \t\n]\\)?\\)")
+	(regexp-quote (or (nth 2 noweb) org-babel-noweb-wrap-end)
 
 (defvar org-babel-src-name-regexp
   "^[ \t]*#\\+name:[ \t]*"
@@ -3116,7 +3121,7 @@ block but are passed literally to the \"example-block\"."
   (not (equal (cdr v) "no"))
 	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
-			 (org-babel-noweb-wrap)
+			 (org-babel-noweb-wrap nil info)
 (unless (equal (cons parent-buffer
  (with-current-buffer parent-buffer
(buffer-chars-modified-tick)))

Re: Noweb Function's body without evaluation

[Ihor Radchenko]

suarezmigu...@icloud.com writes:

> #+name: getClientInstanceNameNew
> #+begin_src shell :session something :var connection="admin@10.0.3.149" :var 
> client="example_client" :var apacheDir="/etc/apache/vhosts"
> <>
> client=$client
> apacheDir=$apacheDir
> grep $client $apacheDir/*
> #+end_src
>
> So, the initSSH call works successfully, so that concludes my first question, 
> being that I can now affect the $connection variable. Thank you again!
>
> However, since the SSH call changes the environment, the $client and 
> $apacheDir variables are not defined in the new environment. I did try 
> setting it again like above, but this doesn’t help as the variables do not 
> exist.

Org babel knows not if the environment changes along the way.
Of course, you can break the variable in bash by calling "bash" as one of
the commands in src block. Or you can do more crazy staff and call
"python" (or, say, "ghci") interpreter. There is no sane way to handle
such weird scenarios programmatically.

I suggest you to write some kind of wrapper like

#+name: obl-identity
#+begin_src emacs-lisp :var x=""
x
#+end_src

#+begin_src bash :noweb yes
<>
client=<>
#+end_src

This will not rely on language-specific way to define variables and
instead generate the code you want to execute fully, on Elisp side.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Noweb Function's body without evaluation

[suarezmiguelc]

Wow, works!, thank you very much.

Now, I have a more specific example.

#+name: initSSH
#+begin_src shell :var connection=“admin@10.0.3.200" :noweb yes
ssh -t miguel@172.28.3.249 "sudo -u admin ssh -t $connection 'sudo su'"
#+end_src

#+name: getClientInstanceNameNew
#+begin_src shell :session something :var connection="admin@10.0.3.149" :var 
client="example_client" :var apacheDir="/etc/apache/vhosts"
<>
client=$client
apacheDir=$apacheDir
grep $client $apacheDir/*
#+end_src

So, the initSSH call works successfully, so that concludes my first question, 
being that I can now affect the $connection variable. Thank you again!

However, since the SSH call changes the environment, the $client and $apacheDir 
variables are not defined in the new environment. I did try setting it again 
like above, but this doesn’t help as the variables do not exist.

 99% of the commands I run are remote so, I will keep investigating this. It 
would make more sense to use :dir and tramp, but as the connections are slow, I 
can only connect with this call which works pretty fast.

> 2/4/23 12:06、Ihor Radchenko のメール:
> 
> suarezmigu...@icloud.com writes:
> 
>> I use heavily org-mode for Literate DevOps, so I have a lot of shell 
>> commands that connect through SSH and do some things later, for example:
>> 
>> #+name: initSSH
>> #+begin_src shell :var connection=“admin@somehost"
>> ssh -t miguel@host "sudo -u someuser ssh -t $connection 'sudo su'"
>> #+end_src
>> 
> p ...>
>> #+name: getStorage
>> #+begin_src shell
>> df
>> #+end_src
>> 
>> Which has to be run in a remote server, could be any remote server as I have 
>> to connect to several. So I would like to be able to:
>> 
>> #+begin_src shell
>> <>
>> <>
>> #+end_src
> 
> You can just
> 
> #+begin_src shell :var connection "admin@anotherhost"
> <>
> >
> #+end_src
> 
> -- 
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 

Re: Noweb Function's body without evaluation

[Ihor Radchenko]

suarezmigu...@icloud.com writes:

> I use heavily org-mode for Literate DevOps, so I have a lot of shell commands 
> that connect through SSH and do some things later, for example:
>
> #+name: initSSH
> #+begin_src shell :var connection=“admin@somehost"
> ssh -t miguel@host "sudo -u someuser ssh -t $connection 'sudo su'"
> #+end_src
>
p ...>
> #+name: getStorage
> #+begin_src shell
> df
> #+end_src
>
> Which has to be run in a remote server, could be any remote server as I have 
> to connect to several. So I would like to be able to:
>
> #+begin_src shell
> <>
> <>
> #+end_src

You can just

#+begin_src shell :var connection "admin@anotherhost"
<>
>
#+end_src

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Noweb Function's body without evaluation

[suarezmiguelc]

Hello Ken, thank you for your message,

After reading the very interesting get_property function, I found that even 
though I will probably use it for some cases, it doesn’t apply directly, to my 
case.

For more examples, if I have 1 source code block:

>> #+name: greeting
>> #+begin_src sh :var name="world" :results output :session testing
>> 
>> echo "hello, $name\!"
>> #+end_src


I have three options in noweb to use this:
- Use its body into another begin_src source code block with <>
- Use its result “hello, world!” Into another code block, which results in 
babel trying to execute the hello, command, which doesn’t exist, this with 
<>
- Use its result, the same as above, but with another parameter, results in the 
same but the variable name is different, so <>

I’m trying to do the first, but with another parameter, so Use its body into 
another begin_src source code block with, get the resulting body after changing 
the variable without it being evaluated, so that I get a valid command to get 
to bash, like with <>, but I can specify a different name variable.

I found the [:body] param but, even though it lets me change the variable as I 
want, it then tries to evaluate it, so I get a different value but the same 
hello, command doesn’t exist. Even though what I would want is to get echo 
“hello, $name\!” but, $name is different.

Thank you for your response Ken, I hope I gave a clearer example with the above.


> 16/3/23 5:16、Ken Mankoff のメール:
> 
> Hi,
> 
> I'm not sure that I understand your issue or needs from the provided 
> examples, but I wonder if the example I provide here would be helpful. It 
> bypasses :var an lets you inject a PROPERTY value anywhere. It is also 
> language agnostic. You can use it to execute commands (that are set as 
> PROPERTY values) or set variables to values.
> 
> https://lists.gnu.org/archive/html/emacs-orgmode/2023-03/msg00251.html
> 
>  -k.
> 
> On 2023-03-15 at 18:54 -04, suarezmigu...@icloud.com wrote...
>> Hello Org-mode community. I’m using Emacs Doom Framework, specifically:
>> 
>> Emacs 28.2 (build 1, aarch64-apple-darwin22.3.0, Carbon Version 169
>> AppKit 2299.4) of 2023-02-23.
>> 
>> I use heavily org-mode for Literate DevOps, so I have a lot of shell
>> commands that connect through SSH and do some things later, for
>> example:
>> 
>> #+name: initSSH
>> #+begin_src shell :var connection=“admin@somehost"
>> 
>> ssh -t miguel@host "sudo -u someuser ssh -t $connection 'sudo su'"
>> #+end_src
>> 
>> So then I can call:
>> 
>> #+call: initSSH(connection=“admin@anotherhost”)
>> 
>> With any other header parameters or session, the above works
>> correctly. I cannot use tramp due to network latency issues, so this
>> is the most performance way for me, since I also have to do some
>> multi-hops which are indeed supported in tramp, but it is too slow for
>> me, so I rather only commands.
>> 
>> The thing is that, I then would like to call these not with a #+call
>> function, but add them into a bigger script, let’s say that I define
>> another command:
>> 
>> #+name: getStorage
>> #+begin_src shell
>> 
>> df
>> #+end_src
>> 
>> Which has to be run in a remote server, could be any remote server as
>> I have to connect to several. So I would like to be able to:
>> 
>> #+begin_src shell
>> <>
>> <>
>> #+end_src
>> 
>> 
>> The first doesn’t work as org-mode runs the code and passes the
>> resulting string to bash, which isn’t a command. The latter works
>> normally. So the issue here are the parameters.
>> 
>> So I made another simple example for this:
>> 
>> #+name: greeting
>> #+begin_src sh :var name="world" :results output :session testing
>> 
>> echo "hello, $name\!"
>> #+end_src
>> 
>> #+results: greeting
>> #+begin_src sh
>> 
>> hello, world\!
>> #+end_src
>> 
>> #+begin_src shell
>> <>
>> #+end_src
>> 
>> 
>> This results in sh: hello,: command not found, as it is executing the 
>> function. I see in the documentation that I can:
>> - Call a function’s body with <>
>> - Execute a function and return its results with <>
>> - Execute a function and return its results even with different params with 
>> <>
>> 
>> So right now, the one that’s missing is, call a function’s body with 
>> different parameters. So the
>> function <> is not evaluated.
>> 
>> After searching a lot, I came across:
>> 
>> #+begin_src shell :session testing
>> <>
>> #+end_src
>> 
>> Which results in:
>> 
>> sh-3.2$ PS1="org_babel_sh_prompt> "
>> org_babel_sh_prompt> name='Testin'
>> org_babel_sh_prompt> echo "hello, $name\!"
>> hello, Testin\!
>> org_babel_sh_prompt> echo 'org_babel_sh_eoe'
>> org_babel_sh_eoe
>> org_babel_sh_prompt> hello, Testin\!
>> sh: hello,: command not found
>> org_babel_sh_prompt> echo 'org_babel_sh_eoe'
>> org_babel_sh_eoe
>> org_babel_sh_prompt> 
>> 
>> Which is somewhat what I need since at least the variable is changed,
>> but the result of this execution is also passed to shell so, same
>> error.
>> 
>> I can’t find much documentation about 

Re: Noweb Function's body without evaluation

[Ken Mankoff]

Hi,

I'm not sure that I understand your issue or needs from the provided examples, 
but I wonder if the example I provide here would be helpful. It bypasses :var 
an lets you inject a PROPERTY value anywhere. It is also language agnostic. You 
can use it to execute commands (that are set as PROPERTY values) or set 
variables to values.

https://lists.gnu.org/archive/html/emacs-orgmode/2023-03/msg00251.html

  -k.

On 2023-03-15 at 18:54 -04, suarezmigu...@icloud.com wrote...
> Hello Org-mode community. I’m using Emacs Doom Framework, specifically:
>
> Emacs 28.2 (build 1, aarch64-apple-darwin22.3.0, Carbon Version 169
> AppKit 2299.4) of 2023-02-23.
>
> I use heavily org-mode for Literate DevOps, so I have a lot of shell
> commands that connect through SSH and do some things later, for
> example:
>
> #+name: initSSH
> #+begin_src shell :var connection=“admin@somehost"
>
> ssh -t miguel@host "sudo -u someuser ssh -t $connection 'sudo su'"
> #+end_src
>
> So then I can call:
>
> #+call: initSSH(connection=“admin@anotherhost”)
>
> With any other header parameters or session, the above works
> correctly. I cannot use tramp due to network latency issues, so this
> is the most performance way for me, since I also have to do some
> multi-hops which are indeed supported in tramp, but it is too slow for
> me, so I rather only commands.
>
> The thing is that, I then would like to call these not with a #+call
> function, but add them into a bigger script, let’s say that I define
> another command:
>
> #+name: getStorage
> #+begin_src shell
>
> df
> #+end_src
>
> Which has to be run in a remote server, could be any remote server as
> I have to connect to several. So I would like to be able to:
>
> #+begin_src shell
> <>
> <>
> #+end_src
>
>
> The first doesn’t work as org-mode runs the code and passes the
> resulting string to bash, which isn’t a command. The latter works
> normally. So the issue here are the parameters.
>
> So I made another simple example for this:
>
> #+name: greeting
> #+begin_src sh :var name="world" :results output :session testing
>
> echo "hello, $name\!"
> #+end_src
>
> #+results: greeting
> #+begin_src sh
>
> hello, world\!
> #+end_src
>
> #+begin_src shell
> <>
> #+end_src
>
>
> This results in sh: hello,: command not found, as it is executing the 
> function. I see in the documentation that I can:
> - Call a function’s body with <>
> - Execute a function and return its results with <>
> - Execute a function and return its results even with different params with 
> <>
>
> So right now, the one that’s missing is, call a function’s body with 
> different parameters. So the
> function <> is not evaluated.
>
> After searching a lot, I came across:
>
> #+begin_src shell :session testing
> <>
> #+end_src
>
> Which results in:
>
> sh-3.2$ PS1="org_babel_sh_prompt> "
> org_babel_sh_prompt> name='Testin'
> org_babel_sh_prompt> echo "hello, $name\!"
> hello, Testin\!
> org_babel_sh_prompt> echo 'org_babel_sh_eoe'
> org_babel_sh_eoe
> org_babel_sh_prompt> hello, Testin\!
> sh: hello,: command not found
> org_babel_sh_prompt> echo 'org_babel_sh_eoe'
> org_babel_sh_eoe
> org_babel_sh_prompt> 
>
> Which is somewhat what I need since at least the variable is changed,
> but the result of this execution is also passed to shell so, same
> error.
>
> I can’t find much documentation about this, what is the correct syntax
> here?,
>
> Thank you!

Re: noweb and shell heredocs

[Tom Gillespie]

Hi Łukasz,
One workaround that is fairly reliable is to prefix the names
of the blocks to be nowebbed with an &. So #+name: block-name
becomes #+name:  Then you reference it as
<<>> and the heredoc syntax is broken. Best,
Tom

Re: noweb and shell heredocs

[Dr. Arne Babenhauserheide]

Immanuel Litzroth  writes:

> You can set the delimiters used for noweb code.
> org-babel-noweb-wrap-end and org-babel-noweb-wrap-end.
>
> I think I set them to @@ in shell code.

I almost always use {{{...}}} via a footer in my org-files:

# Local Variables:
# org-babel-noweb-wrap-start: "{{{"
# org-babel-noweb-wrap-end: "}}}"
# End:

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de


signature.asc
Description: PGP signature

Re: noweb and shell heredocs

[Immanuel Litzroth]

You can set the delimiters used for noweb code.
org-babel-noweb-wrap-end and org-babel-noweb-wrap-end.

I think I set them to @@ in shell code.
Immanuel

On Thu, Nov 25, 2021 at 7:41 PM Łukasz Stelmach  wrote:
>
> Hi,
>
> Is there anything I may try to stop shell syntax hihgliting in code
> blocks being fooled by noweb refs?
>
> --
> Kind regards,
> Łukasz Stelmach



-- 
-- A man must either resolve to point out nothing new or to become a
slave to defend it. -- Sir Isaac Newton

Re: noweb syntax clashing with shell here document syntax

[Sebastian Miele]

Hi Immanuel,

Immanuel Litzroth  writes:

> You can choose which delimiters signal noweb.
> see the documentation of org-babel-noweb-wrap-start and
> org-babel-noweb-wrap-end.

Thank you!  That indeed should make my problem solvable in a perfect
way.

Best wishes
Sebastian

Re: noweb syntax clashing with shell here document syntax

[Immanuel Litzroth]

You can choose which delimiters signal noweb.
see the documentation of org-babel-noweb-wrap-start and
org-babel-noweb-wrap-end.
Immanuel

On Mon, Mar 22, 2021 at 5:35 PM Sebastian Miele
 wrote:
>
> Hello!
>
> The noweb syntax seems to clash with the syntax of here documents in
> shell scripts.  In the block like
>
> #+BEGIN_SRC shell :noweb yes
>   echo a
>   <>
>   echo b
> #+END_SRC
>
> everything following the line with the the noweb reference does not
> get fontified properly.  I suspect that the problem already is known
> for a long time.  Is that true?  If yes, what is the status of this
> issue?  Is there a known workaround?
>
> At least in ZSH it may very well be the case that a line ending with
> '>>' never is valid, except in cases where something like an '< before it is not an opening of a here document.  So it possibly is
> something that could be fixed in sh-script.el.  But I do not know
> exactly.
>
> Anybody who already knows more about this?
>
> Best wishes
> Sebastian
>


-- 
-- Researching the dual problem of finding the function that has a
given point as fixpoint.

Re: noweb expansion when eval and not tangle?

[Timothy]

George Mauer  writes:

> I would like noweb to expand during eval but *not* tangle. Is this possible?
>
> I realize I can achieve the same effect with multiple code blocks but it
> seems an odd asymmetry to not be able to do this directly.
>
> Is it possible and just not documented?

It is indeed, and it's documented in
https://orgmode.org/manual/Noweb-Reference-Syntax.html.

‘eval’
Expansion of noweb syntax references in the body of the code block only 
before evaluating. 

--
Timothy.

Re: noweb

[Nick Dokos]

"Fraga, Eric"  writes:

> My approach to this is to create three blocks that are tangled and a
> separate block (or more than one if you have different tests you want to
> perform) for evaluation that references those three blocks (via noweb)
> but is not tangled.

That works, but shouldn't Nuno's approach with :noweb eval work? What use
is it to include unexpanded noweb markers in the tangled output?

-- 
Nick

"There are only two hard problems in computer science: cache
invalidation, naming things, and off-by-one errors." -Martin Fowler

Re: noweb

[Marco Wahl]

Nuno Salgado  writes:

> My question was to figure out what's the point to tangle the code
> without substituition!

Your imagination is the limit.  E.g. this could be useful to document
the ":noweb eval" behavior opposed to ":noweb yes".

Re: noweb

[Fraga, Eric]

My approach to this is to create three blocks that are tangled and a
separate block (or more than one if you have different tests you want to
perform) for evaluation that references those three blocks (via noweb)
but is not tangled.
-- 
Eric S Fraga via Emacs 28.0.50, Org release_9.3.1-94-g0ac6a9

Re: noweb

[Nuno Salgado]

Marco,

Thank you for your help. I read it again and didn't find. But no
problem.

My question was to figure out what's the point to tangle the code without 
substituition!

NS


Marco Wahl  writes:

> Nuno Salgado  writes:
>
>>   Vars definition:
>>   #+NAME:DEFVARS
>>
>>   #+BEGIN_SRC shell :tangle yes
>> v1=1;
>> v2=2;
>>   #+END_SRC
>>
>>   Script1:
>>
>>   #+BEGIN_SRC shell :tangle yes :noweb eval
>> <>
>> echo $v1;
>>   #+END_SRC
>>
>> This works great when I do C-c C-c in each script.
>>
>> But when I do org-babel-tangle, the code gets two <>.
>>
>> Does it makes sense? Since I set noweb = eval why does it exports
>> <>?
>>
>> Could you please help me turning around this problem without removing every 
>> reference <>
>
> You can find the answer in the documentation, I think.  See e.g. (info
> "(org) Noweb Reference Syntax").
>
>
> HTH

Re: noweb

[Nuno Salgado]

Diego,

Thank you for your help.

Yes, I want the scripts to be all in one file in order to execute them
all. But it's nice do execute then individualy (C-c C-c) to test and debug.

I think I have to have 2 blocks of code for each script: one for tangle,
with no <> and another to test, with <> and <>;


N

Diego Zamboni  writes:

> Hi Nuno,
>
> ":noweb eval" means that noweb references are only expanded during
> evaluation of the code, but not during export. This is why you get
> the literal <> references in exported output. Here are the
> possible values of :noweb and what they mean: https://orgmode.org/
> manual/noweb.html
>
> Also note that if all of this is in the same file, both Script1 and
> Script2 will be tangled to the same file, you may want to specify
> different filenames as the value of :tangle (see https://orgmode.org/
> manual/tangle.html#tangle).
>
> Hope this helps,
> --Diego
>
>
> On Wed, Jan 22, 2020 at 8:41 PM Nuno Salgado  wrote:
>
> Hi,
>
> I'm writing an installation script in org-mode.
>
> I'm doing something like this:
>
>   Vars definition:
>   #+NAME:DEFVARS
>   #+BEGIN_SRC shell :tangle yes
>     v1=1;
>     v2=2;
>   #+END_SRC
>
>   Script1:
>   #+BEGIN_SRC shell :tangle yes :noweb eval
>     <>
>     echo $v1;
>   #+END_SRC
>
>   Script2:
>   #+BEGIN_SRC shell :tangle yes :noweb eval
>     <>
>     echo $v2;
>     echo $v1;
>   #+END_SRC
>
> This works great when I do C-c C-c in each script.
>
> But when I do org-babel-tangle, the code gets two <>.
>
> Does it makes sense? Since I set noweb = eval why does it exports
> <>?
>
> Could you please help me turning around this problem without
> removing every reference <>
>
> Thank you very much.
>
>
> Regards,
> NS
>

Re: noweb

[Diego Zamboni]

Hi Nuno,

":noweb eval" means that noweb references are only expanded during
evaluation of the code, but not during export. This is why you get the
literal <> references in exported output. Here are the possible
values of :noweb and what they mean: https://orgmode.org/manual/noweb.html

Also note that if all of this is in the same file, both Script1 and Script2
will be tangled to the same file, you may want to specify different
filenames as the value of :tangle (see
https://orgmode.org/manual/tangle.html#tangle).

Hope this helps,
--Diego


On Wed, Jan 22, 2020 at 8:41 PM Nuno Salgado  wrote:

> Hi,
>
> I'm writing an installation script in org-mode.
>
> I'm doing something like this:
>
>   Vars definition:
>   #+NAME:DEFVARS
>   #+BEGIN_SRC shell :tangle yes
> v1=1;
> v2=2;
>   #+END_SRC
>
>   Script1:
>   #+BEGIN_SRC shell :tangle yes :noweb eval
> <>
> echo $v1;
>   #+END_SRC
>
>   Script2:
>   #+BEGIN_SRC shell :tangle yes :noweb eval
> <>
> echo $v2;
> echo $v1;
>   #+END_SRC
>
> This works great when I do C-c C-c in each script.
>
> But when I do org-babel-tangle, the code gets two <>.
>
> Does it makes sense? Since I set noweb = eval why does it exports
> <>?
>
> Could you please help me turning around this problem without removing
> every reference <>
>
> Thank you very much.
>
>
> Regards,
> NS
>
>

Re: noweb

[Marco Wahl]

Nuno Salgado  writes:

>   Vars definition:
>   #+NAME:DEFVARS
>
>   #+BEGIN_SRC shell :tangle yes
> v1=1;
> v2=2;
>   #+END_SRC
>
>   Script1:
>
>   #+BEGIN_SRC shell :tangle yes :noweb eval
> <>
> echo $v1;
>   #+END_SRC
>
> This works great when I do C-c C-c in each script.
>
> But when I do org-babel-tangle, the code gets two <>.
>
> Does it makes sense? Since I set noweb = eval why does it exports
> <>?
>
> Could you please help me turning around this problem without removing every 
> reference <>

You can find the answer in the documentation, I think.  See e.g. (info
"(org) Noweb Reference Syntax").


HTH
-- 
Marco

Re: noweb multiple block together

[Ken Mankoff]

Hi Sebastian,

On 2019-11-24 at 22:24 -08, Sebastian Miele  wrote...
> there is the possibility to give a common name to multiple blocks via
> the header arg :noweb-ref. That works in the way you looking for.

Yes this does what I want. I once knew but then forgot about this. Thank you 
for reminding me.

  -k.

Re: noweb multiple block together

[Sebastian Miele]

Hi Ken,

Ken Mankoff  writes:

> When tangling blocks, I can tangle multiple blocks by setting a
> (sub)-tree level property, or ":tangle foo" in multiple headers. Is
> there a way to achieve the same thing with noweb?
>
> I've tried giving multiple blocks the same "+name:" and then <>,
> but only one seems to be included. Does this feature exist through
> some mechanism?

A block named by #+name: should always be unique per file. Otherwise the
Org tangling code that I looked at until now (which is most or even all
in the master branch) just uses the first one found, without checking
for duplicates.

On the other hand there is the possibility to give a common name to
multiple blocks via the header arg :noweb-ref. That works in the way you
looking for.

There is a further difference between these two kinds of naming: With
blocks named by :noweb-ref it is not possible to do something like
<>. Only <>, i.e. plain inclusion, is possible.

Best wishes
Sebastian