Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Max Nikulin]

On 17/04/2023 22:21, Ihor Radchenko wrote:

Matt writes:

I was able to resolve it by simply requiring `sh-script' in `org-src'.  
Committed and pushed.


Note that it will break Emacs 26.


org-protocol.el defines polyfill for `flatten-tree'.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Matt  writes:

> I was able to resolve it by simply requiring `sh-script' in `org-src'.  
> Committed and pushed.

Note that it will break Emacs 26.
Which will fix itself once we merge compat.el support.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Fri, 07 Apr 2023 11:29:59 -0400  Matt  wrote --- 
 > 
 >   On Tue, 04 Apr 2023 08:30:34 -0400  Ihor Radchenko  wrote --- 
 > 
 >  > See the attached tentative patch.
 > 
 > After applying the patch, I get the following error when trying to load 
 > Emacs:
 > 
 > Warning (comp): /home/ahab/Projects/org-mode/lisp/org.el: Error: Symbol's 
 > value as variable is void sh-ancestor-alist
 > 
 > I wasn't able to resolve it.  I suspect the issue is on my end, such as a 
 > mixed install or the need to  re-byte-compile  `sh-script.el'.  

I was able to resolve it by simply requiring `sh-script' in `org-src'.  
Committed and pushed.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

In the code I posted, I had to explicitly (require 'sh-script) to ensure
that the sh-ancestor-alist is loaded before the code. I'm not enough of an
elisp guru to know if there's a way to defer that.

Cheers,

Derek

On Fri, Apr 7, 2023 at 9:30 AM Matt  wrote:

>
>   On Tue, 04 Apr 2023 08:30:34 -0400  Ihor Radchenko  wrote ---
>
>  > See the attached tentative patch.
>
> After applying the patch, I get the following error when trying to load
> Emacs:
>
> Warning (comp): /home/ahab/Projects/org-mode/lisp/org.el: Error: Symbol's
> value as variable is void sh-ancestor-alist
>
> I wasn't able to resolve it.  I suspect the issue is on my end, such as a
> mixed install or the need to  re-byte-compile  `sh-script.el'.
>
> To run Org from source I do one of the following:
>
> ;; When using my init
> (use-package org :straight (:local-repo "/home/ahab/Projects/org-mode"))
>
> ;; When running emacs -q
> (add-to-list 'load-path "/home/ahab/Projects/org-mode/lisp")
> (require 'org-loaddefs)
>
> If I need to recompile Emacs byte code, I'm not sure how I'd do that since
> I'm running Guix and those files live in the write protected /gnu/store.
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Tue, 04 Apr 2023 08:30:34 -0400  Ihor Radchenko  wrote --- 

 > See the attached tentative patch.

After applying the patch, I get the following error when trying to load Emacs:

Warning (comp): /home/ahab/Projects/org-mode/lisp/org.el: Error: Symbol's value 
as variable is void sh-ancestor-alist

I wasn't able to resolve it.  I suspect the issue is on my end, such as a mixed 
install or the need to  re-byte-compile  `sh-script.el'.  

To run Org from source I do one of the following:

;; When using my init
(use-package org :straight (:local-repo "/home/ahab/Projects/org-mode"))

;; When running emacs -q
(add-to-list 'load-path "/home/ahab/Projects/org-mode/lisp")
(require 'org-loaddefs)

If I need to recompile Emacs byte code, I'm not sure how I'd do that since I'm 
running Guix and those files live in the write protected /gnu/store.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

The patch looks good to me!

Thanks,

Derek

On Tue, Apr 4, 2023 at 6:30 AM Ihor Radchenko  wrote:

> Derek Chen-Becker  writes:
>
> > I fiddled around a little bit this weekend and confirmed that this
> (sloppy)
> > code makes highlighting work for all shell types that sh-script supports:
> >
> > ;;A quick hack to try and support more shells syntax highlight in org
> babel
> > (require 'sh-script)
> > (require 'ob-shell)
> > (let ((shells (seq-filter (lambda (shell) (not (eq shell 'sh)))
> > (flatten-tree sh-ancestor-alist
>
> See the attached tentative patch.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Derek Chen-Becker  writes:

> I fiddled around a little bit this weekend and confirmed that this (sloppy)
> code makes highlighting work for all shell types that sh-script supports:
>
> ;;A quick hack to try and support more shells syntax highlight in org babel
> (require 'sh-script)
> (require 'ob-shell)
> (let ((shells (seq-filter (lambda (shell) (not (eq shell 'sh)))
> (flatten-tree sh-ancestor-alist

See the attached tentative patch.
>From 85e268de9435d94eb6766e3f92f4f56a9dc1aebe Mon Sep 17 00:00:00 2001
Message-Id: <85e268de9435d94eb6766e3f92f4f56a9dc1aebe.1680611510.git.yanta...@posteo.net>
From: Ihor Radchenko 
Date: Tue, 4 Apr 2023 14:30:34 +0200
Subject: [PATCH] org-src: Use `sh-mode' for all the shells it can handle

* lisp/org-src.el (org-src--get-known-shells): New helper function
extracting known shells from `sh-ancestor-alist'.
(org-src-lang-modes): Update the value.

Link: https://orgmode.org/list/cambmz5ntkohmizg4ebnaks9ob-0ahncicfhq9lqmjoci0+i...@mail.gmail.com
---
 lisp/org-src.el | 17 -
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index ec2716885..2669b1b77 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -194,11 +194,17 @@ (defvar org-src-mode-hook nil
 or similar things which you want to have when editing a source code file,
 but which mess up the display of a snippet in Org exported files.")
 
+(defun org-src--get-known-shells ()
+  "List all the shells in `sh-ancestor-alist' for `org-src-lang-modes'.
+The shells are associated with `sh-mode'."
+  (mapcar
+   (lambda (shell) (cons (symbol-name shell) 'sh))
+   (delete-dups (flatten-tree sh-ancestor-alist
+
 (defcustom org-src-lang-modes
-  '(("C" . c)
+  `(("C" . c)
 ("C++" . c++)
 ("asymptote" . asy)
-("bash" . sh)
 ("beamer" . latex)
 ("calc" . fundamental)
 ("cpp" . c++)
@@ -208,9 +214,10 @@ (defcustom org-src-lang-modes
 ("elisp" . emacs-lisp)
 ("ocaml" . tuareg)
 ("screen" . shell-script)
-("shell" . sh)
 ("sqlite" . sql)
-("toml" . conf-toml))
+("toml" . conf-toml)
+("shell" . sh)
+,@(org-src--get-known-shells))
   "Alist mapping languages to their major mode.
 
 The key is the language name.  The value is the mode name, as
@@ -221,7 +228,7 @@ (defcustom org-src-lang-modes
 the user side.  For example, there is no `ocaml-mode' in Emacs,
 but the mode to use is `tuareg-mode'."
   :group 'org-edit-structure
-  :package-version '(Org . "9.6")
+  :package-version '(Org . "9.7")
   :type '(repeat
 	  (cons
 	   (string "Language name")
-- 
2.40.0


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

I fiddled around a little bit this weekend and confirmed that this (sloppy)
code makes highlighting work for all shell types that sh-script supports:

;;A quick hack to try and support more shells syntax highlight in org babel
(require 'sh-script)
(require 'ob-shell)
(let ((shells (seq-filter (lambda (shell) (not (eq shell 'sh)))
(flatten-tree sh-ancestor-alist
  (let ((toAppend (mapcar (lambda (shell) `(,(symbol-name shell) . sh))
shells)))
(setq org-src-lang-modes (-distinct (append toAppend
org-src-lang-modes)

I'm a relative newcomer to elisp, so comments and suggestions are welcome.
This is basically what I meant by "dynamically amend org-src-lang-modes
based on the contents of sh-ancestor-alist".

Thanks,

Derek

On Sat, Apr 1, 2023 at 5:22 PM Matt  wrote:

>
>   On Thu, 30 Mar 2023 04:55:32 -0400  Ihor Radchenko  wrote ---
>  > Matt m...@excalamus.com> writes:
>  >
>  > > I think this approach will work fine.   I tried examples for each
> shell type and keywords like if/then/else and function names are
> highlighted.
>  >
>  > Even for posh (powershell)?
>
> Yes.  It's not great since sh-mode looks for Korn-based keywords.  It does
> string highlighting and common keywords like 'if', 'exit', and 'param'.
>
>

-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Thu, 30 Mar 2023 04:55:32 -0400  Ihor Radchenko  wrote --- 
 > Matt m...@excalamus.com> writes:
 > 
 > > I think this approach will work fine.   I tried examples for each shell 
 > > type and keywords like if/then/else and function names are highlighted.
 > 
 > Even for posh (powershell)?

Yes.  It's not great since sh-mode looks for Korn-based keywords.  It does 
string highlighting and common keywords like 'if', 'exit', and 'param'.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Derek Chen-Becker  writes:

> When I asked the question I was just thinking of using it as a reference
> for expanding the current org-src-lang-modes values, but now that you've
> asked the question, perhaps we could dynamically amend org-src-lang-modes
> based on the contents of sh-ancestor-alist. Does that make sense?

Nope, it does not. May you elaborate how exactly can we derive major
mode for an src block using sh-ancestor-alist?
>From my reading of sh-script.el, we may simply use bash-ts-mode for all
the shells.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

When I asked the question I was just thinking of using it as a reference
for expanding the current org-src-lang-modes values, but now that you've
asked the question, perhaps we could dynamically amend org-src-lang-modes
based on the contents of sh-ancestor-alist. Does that make sense?

Thanks,

Derek

On Fri, Mar 31, 2023 at 7:06 AM Ihor Radchenko  wrote:

> Derek Chen-Becker  writes:
>
> > Would it be fair to use the sh-ancestor-alist as a basis for all of the
> > supported shell variants?
> >
> >
> https://github.com/emacs-mirror/emacs/blob/master/lisp/progmodes/sh-script.el#L177
>
> How can we use it?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Derek Chen-Becker  writes:

> Would it be fair to use the sh-ancestor-alist as a basis for all of the
> supported shell variants?
>
> https://github.com/emacs-mirror/emacs/blob/master/lisp/progmodes/sh-script.el#L177

How can we use it?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

Would it be fair to use the sh-ancestor-alist as a basis for all of the
supported shell variants?

https://github.com/emacs-mirror/emacs/blob/master/lisp/progmodes/sh-script.el#L177

Cheers,

Derek

On Thu, Mar 30, 2023 at 2:54 AM Ihor Radchenko  wrote:

> Matt  writes:
>
> > I think this approach will work fine.   I tried examples for each shell
> type and keywords like if/then/else and function names are highlighted.
>
> Even for posh (powershell)?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Matt  writes:

> I think this approach will work fine.   I tried examples for each shell type 
> and keywords like if/then/else and function names are highlighted.

Even for posh (powershell)?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

  On Wed, 29 Mar 2023 13:04:31 -0400  Derek Chen-Becker  wrote --- 
 > Cool, I would be happy to submit a patch!

Sure, if that's something you'd enjoy.  I'm happy to assist, if needed.
  
 > On Wed, Mar 29, 2023 at 3:33 AM Ihor Radchenko yanta...@posteo.net> wrote:
 >
 > Org provides some defaults for bash in `org-src-lang-modes' but not for
 > other shell flavours. I guess we can add more defaults to
 > `org-src-lang-modes', if we know for sure that e.g. zsh can be fontified
 > sh-mode.

I think this approach will work fine.   I tried examples for each shell type 
and keywords like if/then/else and function names are highlighted.

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Derek Chen-Becker]

Cool, I would be happy to submit a patch!

Thanks,

Derek

On Wed, Mar 29, 2023 at 3:33 AM Ihor Radchenko  wrote:

> Matt  writes:
>
> >  Does anyone know which function is responsible for re-fontifing source
> blocks?
>
> `org-src-font-lock-fontify-block', which is using major-mode's native
> fontification. The major mode is determined by `org-src-get-lang-mode',
> which can be customized using `org-src-lang-modes'.
>
> Org provides some defaults for bash in `org-src-lang-modes' but not for
> other shell flavours. I guess we can add more defaults to
> `org-src-lang-modes', if we know for sure that e.g. zsh can be fontified
> sh-mode.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Ihor Radchenko]

Matt  writes:

>  Does anyone know which function is responsible for re-fontifing source 
> blocks?

`org-src-font-lock-fontify-block', which is using major-mode's native
fontification. The major mode is determined by `org-src-get-lang-mode',
which can be customized using `org-src-lang-modes'.

Org provides some defaults for bash in `org-src-lang-modes' but not for
other shell flavours. I guess we can add more defaults to
`org-src-lang-modes', if we know for sure that e.g. zsh can be fontified
sh-mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

[BUG] No font lock in src blocks for shells in org-babel-shell-names (was Re: Font lock for org-babel shell scripts?)

[Matt]

Of the shells given in `org-babel-shell-names' (that is, "sh" "bash" "zsh" 
"fish" "csh" "ash" "dash" "ksh" "mksh" "posh"), only "sh" and "bash" have font 
locking in source blocks.

For example,

#+begin_src sh
  if [ -z $TEST ]; then
  echo Fontified
  fi
#+end_src

#+begin_src bash
  if [ -z $TEST ]; then
  echo Fontified
  fi
#+end_src

#+begin_src zsh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src fish
  if [ -z $TEST ]; then
 echo No fontification
  fi
#+end_src

#+begin_src csh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src ash
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src dash
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src ksh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src mksh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

#+begin_src posh
  if [ -z $TEST ]; then
  echo No fontification
  fi
#+end_src

 Does anyone know which function is responsible for re-fontifing source blocks?