Re: [PATCH 0/4] Add import builtin

[Phi Debian]

On Mon, May 6, 2024 at 7:51 PM Kerin Millar  wrote:

>
>
> I'll put it in a little more detail, though no less plainly. I find the
> terminology of "libraries" and "modules" to be specious in the context of a
> language that has no support for namespaces, awkward scoping rules, a
> problematic implementation of name references, and so on. These
> foundational defects are hardly going to be addressed by a marginally more
> flexible source builtin. Indeed, it is unclear that they can be - or ever
> will be - addressed. Presently, bash is what it is: a messy, slow, useful
> implementation of the Shell Command Language with an increasing number of
> accoutrements, some of which are fine and others of which are less so (and
> virtually impossible to get rid of). As curmudgeonly as it may be to gripe
> over variable and option names, this is why the import of library, as a
> word, does not rest at all well in these quarters. That aside, I do not
> find the premise of the patch series to be a convincing one but would have
> little else to say about its prospective inclusion, provided that the
> behaviour of the posix mode were to be left unchanged in all respects.
>
> --
> Kerin Millar
>
>
Thanx @Kerin, I got an intuitive reluctance with the patch series, but
could not formalize it that way, that is exactly it (specially the nameref
to me :-))

That brings up some questioning about the bash dev workflow. I personally
only monitor bash-bug (not the others bash-*), specially to be aware of new
incoming patches.

Generally, the few patch that shows up here are patches that fix a specific
standing bug, or on some occasion, the declaration of a bug along with a
patch to fix it, they are generally reply with 'thanx for the report and
patch!'

I rarely see patches about "Hey guys I got this great idea what do you
think", so I wonder for this kind of request does git{lab,hub,whatever} be
more appropriate? or may be I should say convenient, a public git project
(clone) generally have all the infrastructure for discussions, enhancement
and fix.

The bash code owner can then pull the great idea when the demand for it
start roaring?

Just questioning, I really have no idea what is the bash dev team way of
receiving enhancement request.

Possible Typo for "Set" Section of Bash Reference Manual

[John]

Hi!

I believe the Bash Reference Manual is missing a key note for using "set 
-o".


On the man page for "bash" 
(https://tiswww.case.edu/php/chet/bash/bash.html), the following line is 
present


 * If *-o* is supplied with no /option-name/, the values of the current
   options are printed. If *+o* is supplied with no /option-name/, a
   series of *set* commands to recreate the current option settings is
   displayed on the standard output.

On the corresponding section for "Set" in the Bash Reference Manual 
(https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html), 
that line of text is *not present*.


Best regards!

Re: 5.3-alpha: the `jobs' builtin prints foreground dead jobs with function substitutions

[Chet Ramey]

On 5/6/24 10:01 AM, Koichi Murase wrote:


Since no one notifies
about foreground jobs that aren't terminated by signals, we can omit
them from the discussion -- they are removed from the jobs list as soon
as they terminate.


I think this is my issue of interpreting the standard, but isn't the
`jobs' utility required by the standard to print "all jobs whose
status has changed and have not been reported by the shell"?


That's the key phrase, isn't it?


If I
literally read it with the theory "$? isn't counted as `reporting'",
in the case `sleep 20; jobs' (where `sleep 20' is not terminated by
signals), the job `sleep 20' isn't notified at its termination, so the
subsequent `jobs' builtin is supposed to print the job "whose status
has not been reported". Am I missing something?


If you like, you can interpret it to mean that $? is sufficient for
reporting normal terminations, the notification applies to abonormal
terminations of interest (which also sets $?), and both can remove
jobs from the jobs list.

Either way, the notifications have to happen, and they should remove
the jobs from the list so jobs doesn't report on them.


The whole issue here is that in some circumstances bash defers that
notification too long, and doesn't do it before the user can run `jobs'.
You don't need to change `jobs' so it doesn't print foreground jobs
that were terminated by a signal, you need to figure out why those jobs
are still in the list when `jobs' runs.


After figuring out, what should I do? 


It's a colloquialism. I have to figure it out, and I believe I have.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [sr #111058] Problem transmitting script arguments

[Chet Ramey]

On 5/6/24 2:15 PM, anonymous wrote:

Follow-up Comment #5, sr #111058 (group bash):

(Author of comment #2)

Indeed, you have a point about there being a dedicated Support category. One
of its problems - which you couldn't have been aware of - is that it doesn't
mesh well with the help-bash and bug-bash mailing lists. In this case, the
initial ticket was cross-posted to the bug-bash list, which doesn't really
make sense to begin with; the Support category would better map to the
help-bash list.


Historically, there was a single category for bug reports and help
requests: support. I enabled the bugs category a couple of weeks ago.
Maybe someday it will receive enough traffic, or siphon off enough
traffic from the support category, to change the notification
destination.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: macOS 14 objc `+initialize` forced crash due to fork after thread

[Chet Ramey]

On 5/6/24 11:17 AM, Tom wrote:

It looks like it's the call to setlocale() that creates the auxiliary
threads, and the call to gettext() that forces the artificial crash.


Yep, that's my understanding.


Thanks. I'll add a call to gettext() that will happen right after bash sets
its default values for the various locale categories.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: 5.3-alpha: the `jobs' builtin prints foreground dead jobs with function substitutions

[Robert Elz]

Date:Mon, 6 May 2024 23:01:16 +0900
From:Koichi Murase 
Message-ID:  



  | Hmm, maybe you mean that a shell is free to anytime remove the
  | terminated foreground jobs from the list regardless of whether it has
  | been reported or not, and "all jobs whose ... reported" do not count
  | the foreground jobs that the shell has removed from the list
  | regardless of reporting. Is this your interpretation?

My interpretation of this would be that POSIX never expects foreground
jobs to be on the jobs list at all, so (as far as it is concerned) there
is nothing related to removing them relevant at all, nor would the jobs
command ever see one.

What an implementation chooses to do with its internal data structs is
beyond the scope of the standard, but just because someone chooses to
put foreground jobs on the jobs list doesn't mean that provisions of
POSIX which apply to jobs it expects on that list apply to anything else
which happens to be there.

kre

Re: [sr #111058] Problem transmitting script arguments

[Kerin Millar]

On Mon, 6 May 2024, at 7:01 PM, Dale R. Worley wrote:
> anonymous  writes:
>> URL:
>>   
>>
>>  Summary: Problem transmitting script arguments
>
>> Date: Sat 04 May 2024 10:08:41 AM UTC By: Anonymous
>> I have the following problem with transmitting arguments to a bash script
>> onward to an inside program call.
>>
>> Lets name the inside program 'Z'.
>> An open number of arguments have to be transmitted from the script 
>> environment
>> to Z's environment. If an argument aa is given enclosed in double-quotes to
>> the script (because there are blanks within the value) these double-quotes 
>> are
>> removed when bash gets hold of it. When I transmit aa by use of $x, $* or $@,
>> the double-quotes are not resurrected by bash, which I think is a tragic
>> mistake because the call of Z obviously suffers a semantic error.
>>
>> So far I could not solve the problem. As this kind of problem cannot be new,
>> is there any recommended way to solve it?
>
> Providing a detailed example would make your requirements clearer.
>
> But if I understand correctly, you want to provide all of the arguments
> that the Bash script receives as arguments to another program, "Z".  The
> standard way to do this is:
>
> Z "$@"
>
> Indeed, it appears that $@ was created with special behavior precisely
> to handle this situation.  From the manual page:
>
>@  Expands  to  the  positional  parameters, starting from one.  In
>   contexts where word splitting is performed,  this  expands  each
>   positional  parameter  to  a separate word; if not within double
>   quotes, these words are subject to word splitting.  In  contexts
>   where  word splitting is not performed, this expands to a single
>   word with each positional parameter separated by a space.   When
>   the  expansion  occurs  within double quotes, each parameter ex‐
>   pands to a separate word.  That is, "$@" is equivalent  to  "$1"
>   "$2"  ...   If the double-quoted expansion occurs within a word,
>   the expansion of the first parameter is joined with  the  begin‐
>   ning  part  of  the original word, and the expansion of the last
>   parameter is joined with the last part  of  the  original  word.
>   When  there  are no positional parameters, "$@" and $@ expand to
>   nothing (i.e., they are removed).
>
> Dale

It's likely that your reply will never be seen by the anonymous Savannah issue 
filer.

-- 
Kerin Millar

Re: [sr #111058] Problem transmitting script arguments

[Dale R. Worley]

anonymous  writes:
> URL:
>   
>
>  Summary: Problem transmitting script arguments

> Date: Sat 04 May 2024 10:08:41 AM UTC By: Anonymous
> I have the following problem with transmitting arguments to a bash script
> onward to an inside program call.
>
> Lets name the inside program 'Z'.
> An open number of arguments have to be transmitted from the script environment
> to Z's environment. If an argument aa is given enclosed in double-quotes to
> the script (because there are blanks within the value) these double-quotes are
> removed when bash gets hold of it. When I transmit aa by use of $x, $* or $@,
> the double-quotes are not resurrected by bash, which I think is a tragic
> mistake because the call of Z obviously suffers a semantic error.
>
> So far I could not solve the problem. As this kind of problem cannot be new,
> is there any recommended way to solve it?

Providing a detailed example would make your requirements clearer.

But if I understand correctly, you want to provide all of the arguments
that the Bash script receives as arguments to another program, "Z".  The
standard way to do this is:

Z "$@"

Indeed, it appears that $@ was created with special behavior precisely
to handle this situation.  From the manual page:

   @  Expands  to  the  positional  parameters, starting from one.  In
  contexts where word splitting is performed,  this  expands  each
  positional  parameter  to  a separate word; if not within double
  quotes, these words are subject to word splitting.  In  contexts
  where  word splitting is not performed, this expands to a single
  word with each positional parameter separated by a space.   When
  the  expansion  occurs  within double quotes, each parameter ex‐
  pands to a separate word.  That is, "$@" is equivalent  to  "$1"
  "$2"  ...   If the double-quoted expansion occurs within a word,
  the expansion of the first parameter is joined with  the  begin‐
  ning  part  of  the original word, and the expansion of the last
  parameter is joined with the last part  of  the  original  word.
  When  there  are no positional parameters, "$@" and $@ expand to
  nothing (i.e., they are removed).

Dale

Re: [PATCH 0/4] Add import builtin

[Kerin Millar]

On Mon, 6 May 2024, at 6:11 PM, Matheus Afonso Martins Moreira wrote:
>>> A native way to source libraries. Built into bash, available to all users.
>> That's the source builtin.
>
> It looks for scripts in directories meant for executables.
> It prefers files with the executable bit set.
> It's a native way to load bash scripts for sure.
> But it's not a native way to load libraries.
>
>> It's just your expectation of the ergonomics but not ours.
>
> What are your expectations?
>
> Do you really think it's reasonable to suggest that
> users implement their own library importing function?
>
> The fact such a function can be written in bash
> is evidence of how powerful bash is. It doesn't
> really follow though that everything that can be
> written in bash should be done in bash.
>
>> He submitted patch seems unneededly large.
>
> The diffstat shows 9 files changed,
> 203 insertions, 92 deletions.
>
> Most of those are very simple
> refactorings of existing code.
>
>> incomplete addition of the long option of builtins
>
> What do you mean? What is missing?
>
>> an incomplete support for the XDG base directories, which should
>> actually be discussed separately.
>
> XDG does not specify the ~/.local and ~/.local/lib directories.
> Only the XDG Data Directory defaults to ~/.local/share if unset.
> That implies a ~/.local directory tree that mirrors that of /usr.
> Seemed simple enough so I just used that path. However, it is
> not actually specified anywhere and there are no variables for
> those directories. So I don't think there's anything that needs
> to be supported here.
>
> Also, it appears the matter of XDG variables
> has already been decided:
>
> https://savannah.gnu.org/support/?108134
> https://savannah.gnu.org/patch/?10431
>
>>  It does seem your personal hangup not giving a filename extension to
>> the script files, which ended up with mixed script files to be sourced
>> and executable files.
>
> Whether to use file extensions or not is the user's choice.
> Bash should work correctly regardless of what the file name is.
>
> Besides, using file extensions does not actually prevent
> false positives. Scripts often end in .sh and .bash too.
> I have named them that way before.
>
> File extensions do not enforce any hard separation
> between executables and libraries. Directories do that.
>
>> If you think that is the problem, your suggestion doesn't actually
>> solve the problem of module managers.
>
> I didn't think it was a problem at first. I just thought it was unnecessary.
> Bash already knows how to do it. We should make it do it for us.
>
> Now I do see it as a problem to be solved. Bash has a restricted shell mode
> which, among many other things, prohibits slashes in file names passed to
> the source builtin. If things are as you claim and the module managers really
> do resolve the paths themselves in order to pass an absolute path to source,
> that means they are all incompatible with bash's restricted shells.
> The proposed native implementation of the module loader is not.
>
>> Module managers will have to implement it by themselves to satisfy
>> their needs (which is obviously different from your ergonomics)
>> even if your feature is added to Bash.
>
> That's true. When I mentioned ergonomics, I wasn't really talking about
> the module managers though. I was talking from the perspective of a
> user who just launched a terminal emulator, got a bash prompt
> and now wants to source some custom functions.
>
>>  Is this your rebuttal to Lawrence's "The sourced files that can be
>>  written with your feature available are exactly the same as the ones
>>  that can be written right now. Much like the periodic requests for
>>  XDG-organized startup files.".
>
> No. It was not actually a rebuttal at all. It was my response
> to the exact text I quoted in the message. Specifically, this text:
>
>> Much like the periodic requests for XDG-organized startup files
>
> I'd really appreciate it if my contribution wasn't regarded
> as just the latest iteration in the infinite loop of annoying
> XDG Base Directory implementation requests from users.
>
>> If you avoid discussion by showing the relation with the authority,
>> it seems a proof of lacking a straightforward counter argument
>
> I avoided discussion because I started to feel like
> I wasn't being taken seriously. I was discussing
> the matter with you in good faith before that.
>
>> Chet mentioned the additional variable for the locations to search,
>> but not a new builtin or a flag.
>
> That's true. I did that of my own initiative.
> After some discussion in the help-bash list,
> I began to think that a builtin was justified.
> Then after discussing it with you I changed
> my mind and rewrote the patch so that it's an
> option for the source builtin instead of an
> entirely new builtin. You made excellent points
> and you convinced me that a new builtin was
> the wrong approach.
>
> I understand that it's not exactly what was discussed.

Re: Re: Re: [PATCH 0/4] Add import builtin

[Matheus Afonso Martins Moreira]

>> A native way to source libraries. Built into bash, available to all users.
> That's the source builtin.

It looks for scripts in directories meant for executables.
It prefers files with the executable bit set.
It's a native way to load bash scripts for sure.
But it's not a native way to load libraries.

> It's just your expectation of the ergonomics but not ours.

What are your expectations?

Do you really think it's reasonable to suggest that
users implement their own library importing function?

The fact such a function can be written in bash
is evidence of how powerful bash is. It doesn't
really follow though that everything that can be
written in bash should be done in bash.

> He submitted patch seems unneededly large.

The diffstat shows 9 files changed,
203 insertions, 92 deletions.

Most of those are very simple
refactorings of existing code.

> incomplete addition of the long option of builtins

What do you mean? What is missing?

> an incomplete support for the XDG base directories, which should
> actually be discussed separately.

XDG does not specify the ~/.local and ~/.local/lib directories.
Only the XDG Data Directory defaults to ~/.local/share if unset.
That implies a ~/.local directory tree that mirrors that of /usr.
Seemed simple enough so I just used that path. However, it is
not actually specified anywhere and there are no variables for
those directories. So I don't think there's anything that needs
to be supported here.

Also, it appears the matter of XDG variables
has already been decided:

https://savannah.gnu.org/support/?108134
https://savannah.gnu.org/patch/?10431

>  It does seem your personal hangup not giving a filename extension to
> the script files, which ended up with mixed script files to be sourced
> and executable files.

Whether to use file extensions or not is the user's choice.
Bash should work correctly regardless of what the file name is.

Besides, using file extensions does not actually prevent
false positives. Scripts often end in .sh and .bash too.
I have named them that way before.

File extensions do not enforce any hard separation
between executables and libraries. Directories do that.

> If you think that is the problem, your suggestion doesn't actually
> solve the problem of module managers.

I didn't think it was a problem at first. I just thought it was unnecessary.
Bash already knows how to do it. We should make it do it for us.

Now I do see it as a problem to be solved. Bash has a restricted shell mode
which, among many other things, prohibits slashes in file names passed to
the source builtin. If things are as you claim and the module managers really
do resolve the paths themselves in order to pass an absolute path to source,
that means they are all incompatible with bash's restricted shells.
The proposed native implementation of the module loader is not.

> Module managers will have to implement it by themselves to satisfy
> their needs (which is obviously different from your ergonomics)
> even if your feature is added to Bash.

That's true. When I mentioned ergonomics, I wasn't really talking about
the module managers though. I was talking from the perspective of a
user who just launched a terminal emulator, got a bash prompt
and now wants to source some custom functions.

>  Is this your rebuttal to Lawrence's "The sourced files that can be
>  written with your feature available are exactly the same as the ones
>  that can be written right now. Much like the periodic requests for
>  XDG-organized startup files.".

No. It was not actually a rebuttal at all. It was my response
to the exact text I quoted in the message. Specifically, this text:

> Much like the periodic requests for XDG-organized startup files

I'd really appreciate it if my contribution wasn't regarded
as just the latest iteration in the infinite loop of annoying
XDG Base Directory implementation requests from users.

> If you avoid discussion by showing the relation with the authority,
> it seems a proof of lacking a straightforward counter argument

I avoided discussion because I started to feel like
I wasn't being taken seriously. I was discussing
the matter with you in good faith before that.

> Chet mentioned the additional variable for the locations to search,
> but not a new builtin or a flag.

That's true. I did that of my own initiative.
After some discussion in the help-bash list,
I began to think that a builtin was justified.
Then after discussing it with you I changed
my mind and rewrote the patch so that it's an
option for the source builtin instead of an
entirely new builtin. You made excellent points
and you convinced me that a new builtin was
the wrong approach.

I understand that it's not exactly what was discussed.
However, I did have a reason for doing it that way.
I think the logic behind the way source uses PATH
is somewhat confusing and adding a new variable
on top of all that would only confuse it further.
For that reason, I added an option instead.
If you 

Re: 5.3-alpha: the `jobs' builtin prints foreground dead jobs with function substitutions

[Koichi Murase]

2024年5月6日(月) 23:01 Koichi Murase :
> > The whole issue here is that in some circumstances bash defers that
> > notification too long, and doesn't do it before the user can run `jobs'.
> > You don't need to change `jobs' so it doesn't print foreground jobs
> > that were terminated by a signal, you need to figure out why those jobs
> > are still in the list when `jobs' runs.
>
> After figuring out, what should I do? If it is the only thing I can
> do, does that mean the jobs builtin continues to print the foreground
> jobs with funsubs/traps/etc? Is it impossible to differentiate the
> foreground jobs from the background ones in the output of the jobs
> builtin?

I've checked the latest devel push. I confirmed the new behaviors that
the signaled foreground jobs are immediately notified even in the middle of
trap/bindx/PROMPT_COMMAND, and therefore the `jobs' builtin wouldn't
print the foreground jobs anymore. This would solve a long-standing
issue of my framework about the ambiguous foreground and background
jobs. Thank you.

Also, I confirmed that a=${ (true) }; b=${ jobs; } would not report
the foreground job `(true)'.

Thank you for your consideration and discussion.

--
Koichi

Re: macOS 14 objc `+initialize` forced crash due to fork after thread

[Tom]

> It looks like it's the call to setlocale() that creates the auxiliary
> threads, and the call to gettext() that forces the artificial crash.

Yep, that's my understanding.

> So any call to gettext() in the parent process before any fork would be
> sufficient to complete the initialization?

It does appear that way; running the following (a modification of the repro I 
put to the `gettext` list):

```
#include 
#include 
#include 

int main() {
  unsetenv("LANG");

  setlocale(LC_ALL, "");

  gettext("");

  if (fork()== 0) {
gettext("test");
  }
}
```

causes the issue to no longer trigger. Note that `LANG` needs to be cleared 
before the call to `setlocale` to trigger the issue.

Re: macOS 14 objc `+initialize` forced crash due to fork after thread

[Chet Ramey]

On 5/3/24 10:31 PM, Tom wrote:


That report has a work-around invoking a syntax error which presumably calls 
out to `setlocale`/`gettext` pre-`fork`, caching the result somewhere (`bash` 
or `gettext`?) such that the offending call post-`fork` on an unknown command 
doesn't trigger this issue.

That feels brittle; a more general approach might be to move both invocations 
either side of the fork (as alluded to by the [gettext maintainer][2]), which 
for `bash` probably means calling `gettext` before `fork`.

[1]: https://trac.macports.org/ticket/68638
[2]: https://lists.gnu.org/archive/html/bug-gettext/2024-05/msg1.html


This seems like a completely Apple-specific problem, since the API
documentation doesn't mention threading, and bash doesn't use any threads
anywhere else. So let's see if we can devise an Apple-specific solution.

It looks like it's the call to setlocale() that creates the auxiliary
threads, and the call to gettext() that forces the artificial crash. Bash
always calls setlocale() as part of initialization, but may not call
gettext() until it has to print a translatable message. So any call to
gettext() in the parent process before any fork would be sufficient to
complete the initialization? I can certainly add a dummy call to gettext()
somewhere, or even a call to initialize and cache the translation of
"command not found" (for instance).

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: 5.3-alpha: the `jobs' builtin prints foreground dead jobs with function substitutions

[Koichi Murase]

Thank you.

2024年5月6日(月) 22:16 Chet Ramey :
> On 5/4/24 8:47 PM, Koichi Murase wrote:
> > If we take the theory that « the notification of the signaled
> > foreground jobs are required because it is not considered `reported'
> > by `$?' », given that the shells don't notify the normally terminated
> > foreground jobs, the jobs builtins are still required to print the
> > unnotified foreground jobs. This theory is not consistent with the
> > actual behavior in shells.
>
> That doesn't follow. The notification isn't performed by the jobs
> builtin (the shell doesn't run `jobs' internally).

Yes, of course. But in this thread, I have been talking about `sleep
20; jobs' [1] where the user explicitly calls the `jobs' builtin.

[1] Second last paragraph in
https://lists.gnu.org/archive/html/bug-bash/2024-05/msg0.html

"The jobs builtin" in my previous reply corresponds to `jobs' called
after `sleep 20' in `sleep 20; jobs'. It's not something internally
called somewhere.

> Since no one notifies
> about foreground jobs that aren't terminated by signals, we can omit
> them from the discussion -- they are removed from the jobs list as soon
> as they terminate.

I think this is my issue of interpreting the standard, but isn't the
`jobs' utility required by the standard to print "all jobs whose
status has changed and have not been reported by the shell"? If I
literally read it with the theory "$? isn't counted as `reporting'",
in the case `sleep 20; jobs' (where `sleep 20' is not terminated by
signals), the job `sleep 20' isn't notified at its termination, so the
subsequent `jobs' builtin is supposed to print the job "whose status
has not been reported". Am I missing something?

Hmm, maybe you mean that a shell is free to anytime remove the
terminated foreground jobs from the list regardless of whether it has
been reported or not, and "all jobs whose ... reported" do not count
the foreground jobs that the shell has removed from the list
regardless of reporting. Is this your interpretation?

> The whole issue here is that in some circumstances bash defers that
> notification too long, and doesn't do it before the user can run `jobs'.
> You don't need to change `jobs' so it doesn't print foreground jobs
> that were terminated by a signal, you need to figure out why those jobs
> are still in the list when `jobs' runs.

After figuring out, what should I do? If it is the only thing I can
do, does that mean the jobs builtin continues to print the foreground
jobs with funsubs/traps/etc? Is it impossible to differentiate the
foreground jobs from the background ones in the output of the jobs
builtin?

--
Koichi

Re: 5.3-alpha: the `jobs' builtin prints foreground dead jobs with function substitutions

[Chet Ramey]

On 5/4/24 8:47 PM, Koichi Murase wrote:


If we take the theory that « the notification of the signaled
foreground jobs are required because it is not considered `reported'
by `$?' », given that the shells don't notify the normally terminated
foreground jobs, the jobs builtins are still required to print the
unnotified foreground jobs. This theory is not consistent with the
actual behavior in shells.


That doesn't follow. The notification isn't performed by the jobs
builtin (the shell doesn't run `jobs' internally). Since no one notifies
about foreground jobs that aren't terminated by signals, we can omit
them from the discussion -- they are removed from the jobs list as soon
as they terminate. Every shell still notifies the user that a foreground
job is terminated by a signal, whether the shell is interactive or not.
When that happens, the job can be removed from the list. The idea is
that the notification, which is useful to users, happens upon job
termination, not when the jobs builtin gets run.

The whole issue here is that in some circumstances bash defers that
notification too long, and doesn't do it before the user can run `jobs'.
You don't need to change `jobs' so it doesn't print foreground jobs
that were terminated by a signal, you need to figure out why those jobs
are still in the list when `jobs' runs.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Re: [PATCH 0/4] Add import builtin

[Koichi Murase]

2024年5月6日(月) 14:28 Matheus Afonso Martins Moreira :
> > I fail to see how this could possibly save "thousands and thousands
> > of lines of code".
>
> How many lines of code were needed to implement the module managers?
> Probably a lot of lines. I guessed it was in the "thousands of lines" 
> ballpark.

I thought that part "it potentially reduces thousands and thousands of
lines of code" in [1] was a part of describing "what Matheus thought
before submitting the patch", but now we shared that the situation in
module managers is not like that. Isn't it?

[1] https://lists.gnu.org/archive/html/bug-bash/2024-05/msg00053.html

Anyway, it's never true that thousands of lines are reduced by the
suggested feature.

> However many lines that is, this feature would allow me
> to reduce them all to zero by not actually needing a
> module manager.

It's trivial that your code reduces to zero because you designed your
builtin to satisfy your needs. That doesn't show anything about its
generality.

> At the very least, it will save users from having to
> implement an import function themselves.
> They could still do it but they won't have to.

That argument would apply to any other useful shell functions, but I
don't think we should make everything builtin.

> >  You keep talking about this proposal as if it would spark a huge
> >  sea change in script maintenance, but I just don't see it, at all.
>
> A native way to source libraries. Built into bash, available to all users.

That's the source builtin.

> With the ergonomics you expect it to have. With good defaults, no setup.

It's just your expectation of the ergonomics but not ours. The
suggestion would be more useful for sure, but it doesn't mean it's the
best. You haven't made any survey on existing implementations.

> That's an incredibly huge change.

If it were really "incredibly huge", I would try to block this change
to carefully think about the impact in the future, other
possibilities, and a possible better solution (e.g. with
shell-function implementations or with a possibly different interface
of a builtin). However, don't worry, I don't think the actual impact
is that large...

> Especially considering how little code
> was required to make it happen.

The submitted patch seems unneededly large. It also touches something
else including incomplete addition of the long option of builtins, and
an incomplete support for the XDG base directories, which should
actually be discussed separately.

> People see bash as a language that scales down instead of up.
> The shellcheck tool helps a lot with shell script development but
> managing complexity is still quite difficult.

What's wrong with it? Even on this list, I think people's attitude
about the future of Bash varies, but I don't think the community
thinks Bash should replace or compete with Python or other general
purpose scripting languages. The shell language is not a language
intended to scale up in general, though it is free to attempt that in
personal projects. I think Bash might acquire a necessary feature in
the future, but I don't think we should hastily add every unnecessary
feature just because that seems useful. Honestly, if we think about
the necessity, I don't think the suggested feature is indispensable.

> This could really change that.
>
> > AFAICT the only thing this proposal addresses is your personal
> > hangup about using PATH to find sourced files
>
> It's not a personal hangup.

It does seem your personal hangup not giving a filename extension to
the script files, which ended up with mixed script files to be sourced
and executable files.

> Alternatively, you reinvent part of the functionality of source
> by resolving the path yourself and giving it an absolute path.
> None of these solutions seem right to me.

If you think that is the problem, your suggestion doesn't actually
solve the problem of module managers. Module managers will have to
implement it by themselves to satisfy their needs (which is obviously
different from your ergonomics) even if your feature is added to Bash.

> > Much like the periodic requests for XDG-organized startup files
>
> I think I deserve at least a little more credit than that.
> [...] _the bash maintainer_
> replied to my email saying he thought it was a
> reasonable feature.

Is this your rebuttal to Lawrence's "The sourced files that can be
written with your feature available are exactly the same as the ones
that can be written right now. Much like the periodic requests for
XDG-organized startup files.". If you avoid discussion by showing the
relation with the authority, it seems a proof of lacking a
straightforward counter argument. In addition, as far as I understand,
Chet mentioned the additional variable for the locations to search,
but not a new builtin or a flag.

> I interpreted that as a strong
> signal that this would be a welcome contribution.

I don't think a welcome contribution means the merge without any
reviews and discussions. If 

Re: 5.3-alpha: the `jobs' builtin prints foreground dead jobs with function substitutions

[Chet Ramey]

On 5/3/24 4:46 PM, Robert Elz wrote:


What I see (in 5.2, and 5.3 alpha 1) is:

jacaranda$ bash-5.3a1 -c '( kill -9 $BASHPID )'
bash-5.3a1: line 1: 29765 Killed  ( kill -9 $BASHPID )

I have no idea what you think that "9" is useful for, IMO it is a
waste of space, and just confusing (and yes, I know SIGKILL is 9).


It's what macOS returns from strsignal(). Take it up with Apple.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Re: Re: Re: [PATCH 0/4] Add import builtin

[Koichi Murase]

2024年5月5日(日) 13:36 Matheus Afonso Martins Moreira :
> > If your purpose is just to solve a small inconvenience of the `source'
> > builtin mixing the namespaces of local scripts, "libraries", and
> > executables, I think the suggested `source -i' or `source -l' would be
> > fine. I think no additional reasoning is needed.
>
> That is my purpose!

OK.

> My solution to the file extensions problem for example is to
> just not have file extensions at all,

This is not normal. I think that is the source of your problem. Even
though it is not strict, executable files (including shell scripts
intended to be executed in a separate process) in PATH usually do not
have the filename extensions. On the other hand, the scripts files
intended to be sourced would typically have a filename extension of
.sh or .bash (or even double extensions such as `.theme.sh,'
`.plugin.sh', and `.alias.sh' depending on the type of the module. In
this way, the namespace of the script files to be sourced and
executable files are "roughly" separated. Of course, it is not strict
and users can still have an executable script with a filename
extension. In some cases, we would have a set of script files to be
sourced without extensions, but those script files are usually
intended to be managed by a specific module manager/loader that
resolves the path by itself (such as /examples/functions/autoload and
bash-completion _comp_load).

Nevertheless, this makes the proposed change less convincing more or
less..Do you think you submitted the patch if you would have had
script files with filename extensions from the beginning? If you
followed the typical arrangement of script files to be sourced, the
problem you faced shouldn't have been that serious.

> > However, in that case, I don't see much difference from
> > « PATH= source » (as far as the module just defines a set of 
> > functions).
>
> It's true that the code is more or less the same from the perspective
> of module managers. However, building this into bash does provide
> a subtle benefit: separation between executable and module paths.
>
> Chet Ramey outlined it in his email:
>
> No need to ensure that they'll not attempt to run external commands.
> The builtin is therefore less limited than a pure PATH solution.

Yes, but that's implied my phrase ``as far as the module just defines
a set of functions''. In case, I note that the functions defined in
the sourced script files can call external commands after the sourcing
completes because the shell function uses "the value of PATH of the
current dynamic context" but not "the value of PATH when the function
is defined".

Anyway, I agree it is a problem that the sourced script cannot call
the external commands if the script wants to do actual processing
(such as initializations) instead of just defining a set of functions
and variables. Nevertheless, it is technically possible to implement
the proper identification of the script file in a shell function with
a few dozens of lines, though it wouldn't be a single line
implementation.

> This would be most useful for a module manager that doesn't resolve
> the module path by itself.

I still don't think it is useful as I already wrote in the previous
reply. I don't think there is a demand to reduce a few lines or a few
dozens of lines from the module manager.

Also, the benefit in the module manager seems too specific. Not many
people try to implement the module manager. The language wouldn't be
designed solely for module managers. The module managers could be an
important piece of a language, but that's not all the important
aspects of a language.

If you were a developer of a module manager and faced a serious
problem in implementing it, which is impossible to solve with an
existing language feature set, the feature request would have been
understood. However, you just seem to raise this example as a
hypothetical project, which seems hardly convincing.

> I agree that a one line of code reduction is underwhelming
> but that's not the only reason to include it. The users are the
> main beneficiaries. This gives users a simple way to load
> modules without using any module managers at all.
> From the perspective of the user, it potentially reduces
> thousands and thousands of lines of code.

I'll reply to it in a corresponding thread by Lawrence.

> Still, I did and still do hope that it will make life easier
> for the maintainers of the module managers too.

If you still hope that, I believe your patches would be blocked until
we test the design by the actual implementation of module managers
based on it and show it's validity. Furthermore, I think it wouldn't
get into Bash because I don't think it's possible to show it. This
feature is simply unnecessary for module managers. Sure it might be
used to reduce a small amount of code, but it's not an indispensable
feature. It's not sufficient to justify the change.

> > Would you try to exclude the possibility
> > of a module manager in the 

Re: Re: [PATCH 0/4] Add import builtin

[Matheus Afonso Martins Moreira]

> Your proposition could not be 'builtin', you want a new 'model' with 
> 'packages/module'

Yes. However, that was because I did not really know bash module managers 
existed.
I immediately adopted a conservative approach once I was informed of their 
existence.

I do not intend to develop the import builtin patch set anymore, I have 
switched to
the source option patch set.

> Another option is to create a fork of a project (here bash) implement your 
> hack,
> provide it (github kind) then see the stats, if your new bash skyrocket,
> may be your hack will be back ported.

Forking bash is not an option for me.
Its ubiquitousness is a major reason why I use it.

> All that to say, I am not too sure that distro (and installer of all sort)
> will be willing to get a fatter bash for higher risk of bugs, hack, secu, 
> name it,
> and would surely ask this feature to be optional,
> then always requiring an init line to enable it.

Making it optional is definitely possible. I'll work on that if needed.
I have made the defaults compile time configurable. I think including the
user's home directory is very useful, it certainly can't be worse than
including the current directory. However, I made it easy to change
should anyone want or need to do so.

I'd like to note that bash has a restricted shell mode which disallows many 
things,
including the sourcing of files via their absolute paths. I made sure that both 
the
import builtin and source --library supported that. I was told that many bash 
module
managers supply an absolute path to source, making them incompatible with 
restricted
shells. Reading the documentation now, it says that restricted mode also 
disallows
overriding PATH. I can also make it so that setting or unsetting 
BASH_LIBRARIES_PATH
is also prohibited. This should make this solution attractive from a security 
perspective.
Administrators can set the bash libraries path and disallow changes to it.

Re: Re: Re: [PATCH 0/9] Add library mode to source builtin

[Matheus Afonso Martins Moreira]

> Libraries are 'compiled collection of object module', can be static/dynamic, 
> dynamic imply a runtime load.

By this definition, dynamic languages and scripting languages don't have 
libraries.
They clearly do have libraries. Sometimes they're called modules but it's the 
same thing.

> This is valid for interpreters with JIT compiler that can produce librairies.

It's also valid for code that's not compiled at all.

> Bash do have libraries by means of loadable .so files, .so files can 
> legitimatly called libraries.

Those are called shared objects, as implied by the .so extension.
Bash documentation also calls them shared objects, not libraries.

https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html#index-enable

> The -f option means to load the new builtin command name from shared object 
> filename,
> on systems that support dynamic loading. Bash will use the value of the 
> BASH_LOADABLES_PATH
> variable as a colon-separated list of directories in which to search for 
> filename. 

Shared objects can also be executables. The dynamic linker is an example.

/usr/lib/ld-linux-x86-64.so.2 --help

Usage: /usr/lib/ld-linux-x86-64.so.2 [OPTION]... EXECUTABLE-FILE 
[ARGS-FOR-PROGRAM...]
You have invoked 'ld.so', the program interpreter for dynamically-linked
ELF programs.  Usually, the program interpreter is invoked automatically
when a dynamically-linked executable is started.

You may invoke the program interpreter program directly from the command
line to load and run an ELF executable file; this is like executing that
file itself, but always uses the program interpreter you invoked,
instead of the program interpreter specified in the executable file you
run.  Invoking the program interpreter directly provides access to
additional diagnostics, and changing the dynamic linker behavior without
setting environment variables (which would be inherited by subprocesses).

Note how I had to use the full path to the linker in order to invoke it.
It's a library. It's not in the PATH. And yet it's an executable.

You can have perfectly normal executables whose functions you can link against
and use as though they were themselves libraries. With GCC, you make them by 
passing
-fPIC -pie -rdynamic to create a position independent executable which includes 
all
defined symbols in the ELF table, not just the used ones. The result of that is 
an
executable whose symbols can be looked up, linked and relocated. In other words,
a library that can also run. As you can see, the lines can be quite blurry 
indeed.

When it comes down to it, the only difference between an executable and a 
library
is whether it does something useful when you run it directly.
The presence or absence of an ELF entry point for the program.
The existence of a main function.

In scripting languages, the main function is implicit.
The code is usually interpreted line by line by a virtual machine.
This interpretation essentially is the main function.
Functions are defined dynamically as the interpreter moves through the code.
So the only way to distinguish libraries from executables is to look at the
what the code does and interpret its purpose.

Does it just define functions and exits? That's a library.
Or does it actually do something with those functions? That's an executable.

> A 'script' is a bash source file with the +x bit set (executable)

https://en.wikipedia.org/wiki/Scripting_language

> In computing, a script is a relatively short and simple set of instructions
that typically automate an otherwise manual process.

What are scripts called when they automate no tasks
and contain nothing but function and variable definitions?

I call them libraries.

Re: Re: [PATCH 0/4] Add import builtin

[Phi Debian]

On Mon, May 6, 2024 at 7:28 AM Matheus Afonso Martins Moreira <
math...@matheusmoreira.com> wrote:

> Yet the feature has been described as "irritating"!
> I really don't understand the cause for this
> and it's making me feel really unwelcome.
>

I think it is not personnal, you proposed something, and other told you
what you propose is overkill due to the simplicity to do the same with no
change in bash, and a 1liner of code in your top level project file.

Your proposition could not be 'builtin', you want a new 'model' with
'packages/module' (not saying library on purpose, way too much for me).
Starting from there your model need at least an init line to say you want
bash operating in your model 'packages/module'. Form there your new model
can be initialized with a 1liner, either a dynloadable builtin (.so file
you provide), or an init file (.sh no +x) you source. At this point
whatever technic, you have your 'import'. Is that big of a deal to ask your
developers to comply with your model and init your 'packages/module'
paradigm bringing 'import' in a 1 line in their top script (here script
mean +x executable bash source file).

TBH, I have a packages system for both bash and ksh since decades, they all
start by an init line at top level, and they never required any shell hack,
and they do far more that intended here, repos, versioning, dependencis,
multi-arch (run time context) etc...

I think it is very good to be enthusiastic about hacking a free software, a
good start point is to fix bugs, even tiny one, even simple docco.

Another option is to create a fork of a project (here bash) implement your
hack, provide it (github kind) then see the stats, if your new bash
skyrocket, may be your hack will be back ported. You could  adverstise your
new bash in stko, each one asking for a package manager in bash you could
reply with your new invention, boosting the stats.

All that to say, I am not too sure that distro (and installer of all sort)
will be willing to get a fatter bash for higher risk of bugs, hack, secu,
name it, and would surely ask this feature to be optional, then always
requiring an init line to enable it.

So I guess it will take a long time before seeing that in the shell, but I
may be wrong.