Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Max Nikulin]

On 17/04/2023 22:21, Ihor Radchenko wrote:

Matt writes:

I was able to resolve it by simply requiring `sh-script' in `org-src'.  
Committed and pushed.


Note that it will break Emacs 26.


org-protocol.el defines polyfill for `flatten-tree'.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Matt  writes:

> I was able to resolve it by simply requiring `sh-script' in `org-src'.  
> Committed and pushed.

Note that it will break Emacs 26.
Which will fix itself once we merge compat.el support.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Fri, 07 Apr 2023 11:29:59 -0400  Matt  wrote --- 
 > 
 >   On Tue, 04 Apr 2023 08:30:34 -0400  Ihor Radchenko  wrote --- 
 > 
 >  > See the attached tentative patch.
 > 
 > After applying the patch, I get the following error when trying to load 
 > Emacs:
 > 
 > Warning (comp): /home/ahab/Projects/org-mode/lisp/org.el: Error: Symbol's 
 > value as variable is void sh-ancestor-alist
 > 
 > I wasn't able to resolve it.  I suspect the issue is on my end, such as a 
 > mixed install or the need to  re-byte-compile  `sh-script.el'.  

I was able to resolve it by simply requiring `sh-script' in `org-src'.  
Committed and pushed.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

In the code I posted, I had to explicitly (require 'sh-script) to ensure
that the sh-ancestor-alist is loaded before the code. I'm not enough of an
elisp guru to know if there's a way to defer that.

Cheers,

Derek

On Fri, Apr 7, 2023 at 9:30 AM Matt  wrote:

>
>   On Tue, 04 Apr 2023 08:30:34 -0400  Ihor Radchenko  wrote ---
>
>  > See the attached tentative patch.
>
> After applying the patch, I get the following error when trying to load
> Emacs:
>
> Warning (comp): /home/ahab/Projects/org-mode/lisp/org.el: Error: Symbol's
> value as variable is void sh-ancestor-alist
>
> I wasn't able to resolve it.  I suspect the issue is on my end, such as a
> mixed install or the need to  re-byte-compile  `sh-script.el'.
>
> To run Org from source I do one of the following:
>
> ;; When using my init
> (use-package org :straight (:local-repo "/home/ahab/Projects/org-mode"))
>
> ;; When running emacs -q
> (add-to-list 'load-path "/home/ahab/Projects/org-mode/lisp")
> (require 'org-loaddefs)
>
> If I need to recompile Emacs byte code, I'm not sure how I'd do that since
> I'm running Guix and those files live in the write protected /gnu/store.
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Tue, 04 Apr 2023 08:30:34 -0400  Ihor Radchenko  wrote --- 

 > See the attached tentative patch.

After applying the patch, I get the following error when trying to load Emacs:

Warning (comp): /home/ahab/Projects/org-mode/lisp/org.el: Error: Symbol's value 
as variable is void sh-ancestor-alist

I wasn't able to resolve it.  I suspect the issue is on my end, such as a mixed 
install or the need to  re-byte-compile  `sh-script.el'.  

To run Org from source I do one of the following:

;; When using my init
(use-package org :straight (:local-repo "/home/ahab/Projects/org-mode"))

;; When running emacs -q
(add-to-list 'load-path "/home/ahab/Projects/org-mode/lisp")
(require 'org-loaddefs)

If I need to recompile Emacs byte code, I'm not sure how I'd do that since I'm 
running Guix and those files live in the write protected /gnu/store.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

The patch looks good to me!

Thanks,

Derek

On Tue, Apr 4, 2023 at 6:30 AM Ihor Radchenko  wrote:

> Derek Chen-Becker  writes:
>
> > I fiddled around a little bit this weekend and confirmed that this
> (sloppy)
> > code makes highlighting work for all shell types that sh-script supports:
> >
> > ;;A quick hack to try and support more shells syntax highlight in org
> babel
> > (require 'sh-script)
> > (require 'ob-shell)
> > (let ((shells (seq-filter (lambda (shell) (not (eq shell 'sh)))
> > (flatten-tree sh-ancestor-alist
>
> See the attached tentative patch.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Derek Chen-Becker  writes:

> I fiddled around a little bit this weekend and confirmed that this (sloppy)
> code makes highlighting work for all shell types that sh-script supports:
>
> ;;A quick hack to try and support more shells syntax highlight in org babel
> (require 'sh-script)
> (require 'ob-shell)
> (let ((shells (seq-filter (lambda (shell) (not (eq shell 'sh)))
> (flatten-tree sh-ancestor-alist

See the attached tentative patch.
>From 85e268de9435d94eb6766e3f92f4f56a9dc1aebe Mon Sep 17 00:00:00 2001
Message-Id: <85e268de9435d94eb6766e3f92f4f56a9dc1aebe.1680611510.git.yanta...@posteo.net>
From: Ihor Radchenko 
Date: Tue, 4 Apr 2023 14:30:34 +0200
Subject: [PATCH] org-src: Use `sh-mode' for all the shells it can handle

* lisp/org-src.el (org-src--get-known-shells): New helper function
extracting known shells from `sh-ancestor-alist'.
(org-src-lang-modes): Update the value.

Link: https://orgmode.org/list/cambmz5ntkohmizg4ebnaks9ob-0ahncicfhq9lqmjoci0+i...@mail.gmail.com
---
 lisp/org-src.el | 17 -
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index ec2716885..2669b1b77 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -194,11 +194,17 @@ (defvar org-src-mode-hook nil
 or similar things which you want to have when editing a source code file,
 but which mess up the display of a snippet in Org exported files.")
 
+(defun org-src--get-known-shells ()
+  "List all the shells in `sh-ancestor-alist' for `org-src-lang-modes'.
+The shells are associated with `sh-mode'."
+  (mapcar
+   (lambda (shell) (cons (symbol-name shell) 'sh))
+   (delete-dups (flatten-tree sh-ancestor-alist
+
 (defcustom org-src-lang-modes
-  '(("C" . c)
+  `(("C" . c)
 ("C++" . c++)
 ("asymptote" . asy)
-("bash" . sh)
 ("beamer" . latex)
 ("calc" . fundamental)
 ("cpp" . c++)
@@ -208,9 +214,10 @@ (defcustom org-src-lang-modes
 ("elisp" . emacs-lisp)
 ("ocaml" . tuareg)
 ("screen" . shell-script)
-("shell" . sh)
 ("sqlite" . sql)
-("toml" . conf-toml))
+("toml" . conf-toml)
+("shell" . sh)
+,@(org-src--get-known-shells))
   "Alist mapping languages to their major mode.
 
 The key is the language name.  The value is the mode name, as
@@ -221,7 +228,7 @@ (defcustom org-src-lang-modes
 the user side.  For example, there is no `ocaml-mode' in Emacs,
 but the mode to use is `tuareg-mode'."
   :group 'org-edit-structure
-  :package-version '(Org . "9.6")
+  :package-version '(Org . "9.7")
   :type '(repeat
 	  (cons
 	   (string "Language name")
-- 
2.40.0


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

I fiddled around a little bit this weekend and confirmed that this (sloppy)
code makes highlighting work for all shell types that sh-script supports:

;;A quick hack to try and support more shells syntax highlight in org babel
(require 'sh-script)
(require 'ob-shell)
(let ((shells (seq-filter (lambda (shell) (not (eq shell 'sh)))
(flatten-tree sh-ancestor-alist
  (let ((toAppend (mapcar (lambda (shell) `(,(symbol-name shell) . sh))
shells)))
(setq org-src-lang-modes (-distinct (append toAppend
org-src-lang-modes)

I'm a relative newcomer to elisp, so comments and suggestions are welcome.
This is basically what I meant by "dynamically amend org-src-lang-modes
based on the contents of sh-ancestor-alist".

Thanks,

Derek

On Sat, Apr 1, 2023 at 5:22 PM Matt  wrote:

>
>   On Thu, 30 Mar 2023 04:55:32 -0400  Ihor Radchenko  wrote ---
>  > Matt m...@excalamus.com> writes:
>  >
>  > > I think this approach will work fine.   I tried examples for each
> shell type and keywords like if/then/else and function names are
> highlighted.
>  >
>  > Even for posh (powershell)?
>
> Yes.  It's not great since sh-mode looks for Korn-based keywords.  It does
> string highlighting and common keywords like 'if', 'exit', and 'param'.
>
>

-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Thu, 30 Mar 2023 04:55:32 -0400  Ihor Radchenko  wrote --- 
 > Matt m...@excalamus.com> writes:
 > 
 > > I think this approach will work fine.   I tried examples for each shell 
 > > type and keywords like if/then/else and function names are highlighted.
 > 
 > Even for posh (powershell)?

Yes.  It's not great since sh-mode looks for Korn-based keywords.  It does 
string highlighting and common keywords like 'if', 'exit', and 'param'.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Derek Chen-Becker  writes:

> When I asked the question I was just thinking of using it as a reference
> for expanding the current org-src-lang-modes values, but now that you've
> asked the question, perhaps we could dynamically amend org-src-lang-modes
> based on the contents of sh-ancestor-alist. Does that make sense?

Nope, it does not. May you elaborate how exactly can we derive major
mode for an src block using sh-ancestor-alist?
>From my reading of sh-script.el, we may simply use bash-ts-mode for all
the shells.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

When I asked the question I was just thinking of using it as a reference
for expanding the current org-src-lang-modes values, but now that you've
asked the question, perhaps we could dynamically amend org-src-lang-modes
based on the contents of sh-ancestor-alist. Does that make sense?

Thanks,

Derek

On Fri, Mar 31, 2023 at 7:06 AM Ihor Radchenko  wrote:

> Derek Chen-Becker  writes:
>
> > Would it be fair to use the sh-ancestor-alist as a basis for all of the
> > supported shell variants?
> >
> >
> https://github.com/emacs-mirror/emacs/blob/master/lisp/progmodes/sh-script.el#L177
>
> How can we use it?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Derek Chen-Becker  writes:

> Would it be fair to use the sh-ancestor-alist as a basis for all of the
> supported shell variants?
>
> https://github.com/emacs-mirror/emacs/blob/master/lisp/progmodes/sh-script.el#L177

How can we use it?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

Would it be fair to use the sh-ancestor-alist as a basis for all of the
supported shell variants?

https://github.com/emacs-mirror/emacs/blob/master/lisp/progmodes/sh-script.el#L177

Cheers,

Derek

On Thu, Mar 30, 2023 at 2:54 AM Ihor Radchenko  wrote:

> Matt  writes:
>
> > I think this approach will work fine.   I tried examples for each shell
> type and keywords like if/then/else and function names are highlighted.
>
> Even for posh (powershell)?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Matt  writes:

> I think this approach will work fine.   I tried examples for each shell type 
> and keywords like if/then/else and function names are highlighted.

Even for posh (powershell)?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Wed, 29 Mar 2023 13:04:31 -0400  Derek Chen-Becker  wrote --- 
 > Cool, I would be happy to submit a patch!

Sure, if that's something you'd enjoy.  I'm happy to assist, if needed.
  
 > On Wed, Mar 29, 2023 at 3:33 AM Ihor Radchenko yanta...@posteo.net> wrote:
 >
 > Org provides some defaults for bash in `org-src-lang-modes' but not for
 > other shell flavours. I guess we can add more defaults to
 > `org-src-lang-modes', if we know for sure that e.g. zsh can be fontified
 > sh-mode.

I think this approach will work fine.   I tried examples for each shell type 
and keywords like if/then/else and function names are highlighted.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

Cool, I would be happy to submit a patch!

Thanks,

Derek

On Wed, Mar 29, 2023 at 3:33 AM Ihor Radchenko  wrote:

> Matt  writes:
>
> >  Does anyone know which function is responsible for re-fontifing source
> blocks?
>
> `org-src-font-lock-fontify-block', which is using major-mode's native
> fontification. The major mode is determined by `org-src-get-lang-mode',
> which can be customized using `org-src-lang-modes'.
>
> Org provides some defaults for bash in `org-src-lang-modes' but not for
> other shell flavours. I guess we can add more defaults to
> `org-src-lang-modes', if we know for sure that e.g. zsh can be fontified
> sh-mode.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Matt  writes:

>  Does anyone know which function is responsible for re-fontifing source 
> blocks?

`org-src-font-lock-fontify-block', which is using major-mode's native
fontification. The major mode is determined by `org-src-get-lang-mode',
which can be customized using `org-src-lang-modes'.

Org provides some defaults for bash in `org-src-lang-modes' but not for
other shell flavours. I guess we can add more defaults to
`org-src-lang-modes', if we know for sure that e.g. zsh can be fontified
sh-mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

[BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

Of the shells given in `org-babel-shell-names' (that is, "sh" "bash" "zsh" 
"fish" "csh" "ash" "dash" "ksh" "mksh" "posh"), only "sh" and "bash" have font 
locking in source blocks.

For example,

#+begin_src sh
  if [ -z $TEST ]; then
  echo Fontified
  fi
#+end_src

#+begin_src bash
  if [ -z $TEST ]; then
  echo Fontified
  fi
#+end_src

#+begin_src zsh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src fish
  if [ -z $TEST ]; then
 echo No fontification
  fi
#+end_src

#+begin_src csh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src ash
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src dash
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src ksh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src mksh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src posh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

 Does anyone know which function is responsible for re-fontifing source blocks?

Re: Font lock for org-babel shell scripts?

[Matt]

  On Tue, 28 Mar 2023 13:24:39 -0400  Derek Chen-Becker  wrote --- 

 > I'm trying to figure out whether there's a bug or just a misconfiguration on 
 > my end with font lock for org-babel shell source blocks. If I run emacs 28.2 
 > (with -q) and open the following org file, I can evaluate both source blocks 
 > but only the "bash" block has syntax highlighting. I've confirmed that 
 > opening a zsh script (e.g. with a zsh shebang) highlights correctly. If this 
 > list isn't the right place to ask about this issue, does someone know where 
 > I could look for help?

This is the right place to ask.  And thanks for asking!

I'm able to reproduce it.

I'm not familiar with how Org handles the font lock.  However, I notice that 
calling C-c ' (`org-edit-special') results in the error:  No such language 
mode: zsh-mode

I suspect that without a language mode, Org has no way to know how to color the 
block.

As a quick work around, you can define zsh-mode as shell-script-mode to get 
coloring:

(defalias 'zsh-mode 'shell-script-mode)

Font lock for org-babel shell scripts?

[Derek Chen-Becker]

Hi,

I'm trying to figure out whether there's a bug or just a misconfiguration
on my end with font lock for org-babel shell source blocks. If I run emacs
28.2 (with -q) and open the following org file, I can evaluate both source
blocks but only the "bash" block has syntax highlighting. I've confirmed
that opening a zsh script (e.g. with a zsh shebang) highlights correctly.
If this list isn't the right place to ask about this issue, does someone
know where I could look for help?

Thanks,

Derek

#+begin_src bash
  if [ -z $TEST ]; then
  echo Good
  fi
#+end_src

#+begin_src zsh
  if [ -z $TEST ]; then
  echo Good
  fi
#+end_src

# Local Variables:
# org-babel-load-languages: ((shell . t))
# End:



-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+