Re: Enable compgen even when programmable completions are not available?
On 6/30/23 3:25 PM, Robert Elz wrote: > Date:Fri, 30 Jun 2023 18:35:34 +0100 > From:Kerin Millar > Message-ID: <20230630183534.85da7986a24855126bfea...@plushkava.net> > > | This can be trivially foiled. > > You mean it doesn't give you all the variable names? Why not? > Does bash have a bug in this area that I am unaware of? > > Or do you mean that it will sometimes (due to newlines in the values) > be able to be persuaded to give you more than just the var names? > > If the latter, why do you care? The processing can check for each variable > name produced whether it is set or not, and simply skip anything that looks > like it is a variable name, but actually isn't set in the shell. If it is > important that each var name be included just once, just run the output > through sort -u so any duplicates (which will only be caused by that kind > of newline in the value stuff) get removed. > > You're going to need a more sophisticated sed script (but not all that > much more) to remove any "obviously not a variable name followed by =" > lines, but you should be getting all of the variable names, which is the > part that really matters, I am guessing. That is, making sure none are > missing is the challenging part, weeding out fakes is simple. You have very helpfully described how one can design a rube goldberg machine in bash to achieve some important goal. But it rather misses the point that this thread wasn't asking for an answer. The topic is "should compgen be enabled when readline / progcomp is not". The initial premise of the thread described your solution except better -- declare -p, as mentioned in passing by my thread starter and in more depth at the linked gentoo bug. And declare -p is better because it returns the same results as set, except you get even more detail because you can also grep for a leading `^declare`. It was deemed undesirable, nonetheless. Yet another non-compgen hack is described in the gentoo bug. It's still a hack. Brainstorming additional workarounds doesn't accomplish anything above and beyond the first workaround -- which is another way of saying, brainstorming additional workarounds doesn't accomplish anything at all, at the moment. Especially when they are incomplete and come with the caveat that you'll need to do additional sed postprocessing, "check for this and that", and various forms of English prose description about how to accomplish a task... .. that was already worked around with a hack and has a fully implemented patch to do so on the gentoo ticket. Suggesting ways that future versions of bash 5.3 and above can do this *without* hacks, would accomplish something. It would accomplish the ability to migrate away from hacks and over to proper APIs. Maybe we should focus on that. -- Eli Schwartz
Re: Enable compgen even when programmable completions are not available?
oblems which happen as a result. And that's kind of my whole question here to begin with. It's not clear to me that it's a good idea to allow bash to disable part of the bash scripting language at all. The "bash core" I want to assume is that there is no core, only a complete language none of which is optional at the compiled binary level. If that's not going to happen, then the "bash core" I want to assume is whatever is guaranteed to exist that cannot be turned off. Either way, I believe that people disabling readline support don't intrinsically expect that it will change the scripting language to support fewer features. At least the progcomp help text says it enables the complete builtin (but does not mention compgen). The readline option help text doesn't mention that disabling it overrides progcomp to off, so the connection between that and compgen disappearing is non-obvious. My copy of the bash manpage doesn't suggest that these builtins are optional, either. ... I would appreciate guidance on what "the bash scripting language" entails. If that entails "arrays are optional, compgen is optional, process substitution is optional, do not assume that any of this stuff is guaranteed to exist" then so be it. My suggestion is that all these things are declared mandatory, and users who wish to disable support for arrays are instructed that they should not call the shell "bash" at all, but install it as some other binary name. Maybe "tinysh". It would be nice if the option to disable support for compgen was separate from the option to disable support for readline, and the "tinysh" naming recommendation only applied to the former, not the latter. -- Eli Schwartz
Re: Enable compgen even when programmable completions are not available?
On 6/30/23 1:19 PM, Robert Elz wrote: > Date:Thu, 29 Jun 2023 23:05:38 +0100 > From:Kerin Millar > Message-ID: <20230629230538.cbef14a75694143ccf034...@plushkava.net> > > | The thing is that portage also has a legitimate stake in needing > | to enumerate all variable names. > > Why isn't "set" good enough for that? > > (set -o posix; set) | sed 's/=.*$//' | whatever_processes_them > > You need posix mode, so bash doesn't dump functions as well (which IMO > it shouldn't do with set, ever, but has probably been doing it since > prehistoric times, and somewhere, there's probably 1 user depending upon > it...) So then you get what? The exact same thing, with all the attendant flaws, as declare -p which is already known not to work? You still have to sed/grep the contents, and you still have the original declare -p problem that you need to parse the value of the variable in order to figure out *where* it is so you can discard and ignore it... which is impractical without an actual shell parser. -- Eli Schwartz
Re: Enable compgen even when programmable completions are not available?
On 6/29/23 4:39 PM, Chet Ramey wrote: > On 6/25/23 2:38 PM, Eli Schwartz wrote: >> compgen is a useful builtin for inspecting information about the shell >> context e.g. in scripts -- a good example of this is compgen -A function >> or compgen -A variable. >> >> But it's not always available depending on how bash is built, which >> results in people lacking confidence that it can / should be used in >> scripts. See e.g. https://bugs.gentoo.org/909148 > > It's dependent on programmable completion and readline, which are features > that are enabled by default. Who builds a version of bash with those turned > off? What's the rationale for doing that? I assume that, given the option exists in the configure script, there are people who will want to use the option made available to them. One reason might be because they are configuring it for a system that isn't fussed about using bash for an interactive shell (either it is administrated via non-interactive means, or simply because the preferred interactive site shell is e.g. zsh). In such cases, a rationale that readily comes to mind is "this user wanted a smaller, leaner bash binary by disabling unimportant bits that they do not use". And because this is conditional on readline, which is usually an external library dependency (a global system policy decision), reducing the number of shared libraries the dynamic loader has to deal with might be especially interesting. (This is all theorizing -- I quite like bash as an interactive shell and have no intention of building systems with readline disabled. It is nonetheless true that the topic came up because there are Gentoo users who apparently decided to try to do so.) ... The thing is, does it really matter? I think there's a larger issue here, which I mentioned in the Gentoo bug report but probably makes sense to copy/paste here: > The problem with compgen is that it is only available for use when > bash is configured with --enable-progcomp / --enable-readline, which > feels like a powerful argument that script authors are not allowed to > assume that it will exist, regardless of how useful it may be in > non-programmable-completion contexts. > > Maybe the answer is to ask that it always be available as a builtin, > even when the programmable completion system isn't enabled. So: can I? Are my bash scripts valid scripts if they use compgen, or should I be concerned that when I publish them for wide adoption, people are going to report bugs to me that it is broken on their niche system configuration which they are positive they have a good reason for? Should I document in the project readme that in addition to needing a certain minimum version of bash, "you also need to make sure that programmable completions are enabled when compiling your bash binary"? Should I eschew compgen and rely on eval-using hacks like the one Kerin described? -- Eli Schwartz
Re: Enable compgen even when programmable completions are not available?
On 6/26/23 5:32 AM, Kerin Millar wrote: > While it is understandable that an attempt to assign to certain shell > variables would be treated as an error, the combination of not > printing a diganostic message and inducing a non-interactive shell to > exit is rather confusing. Further, declare is granted special > treatment, even after having been defined as a function (which might > be a bug). Yup, while trying out Martin's exact suggestion before initially posting this thread (apologies, Martin -- I should have mentioned that) I was thrown for a loop for several minutes trying to figure out what on earth was going on there when I couldn't even echo the return value to check what happened. -- Eli Schwartz
Enable compgen even when programmable completions are not available?
compgen is a useful builtin for inspecting information about the shell
context e.g. in scripts -- a good example of this is compgen -A function
or compgen -A variable.
But it's not always available depending on how bash is built, which
results in people lacking confidence that it can / should be used in
scripts. See e.g. https://bugs.gentoo.org/909148
Would it be possible to have a slightly more minimal version of it with
readline functionality and "currently inside programmable completions"
functionality stripped out, which just allows it to be used for general
environment introspection?
An alternative for compgen -A function does exist -- declare -F in
combination with e.g. sed. Variables is harder since AFAICT the only way
to print all of them prints multiline values too (declare -p is similar
to declare -f, there is no declare -P similar to declare -F, using
${!a@} to expand variable names does not work generically enough since
you need to specify a prefix -- ${!@} won't expand all of them).
--
Eli Schwartz
Re: Misleading error when attempting to run foreign executable
On 10/11/21 07:09, Robert Elz wrote: > This is clearly an OS problem, not one in bash. > > POSIX says of ENOENT as it applies to the exec*() set of functions: > > [ENOENT] A component of path or file does not name an existing file > or path or file is an empty string. > > "path" and "file" (which are in italics in the text, that just doesn't > make it to this ascii e-mail) are args to (different instances of) the > exec*() functions (ie: they are not generic words). > > That is, ENOENT is only applicable if the file (or path) named in the > arg to the sys call does not exist. Indeed, but the kernel.org "Linux man pages" project distributes manpages which freely "reinterpret" this: [ENOENT] The file pathname or a script or ELF interpreter does not exist. [ENOEXEC] An executable is not in a recognized format, is for the wrong architecture, or has some other format error that means it cannot be executed. [EINVAL] An ELF executable had more than one PT_INTERP segment (i.e., tried to name more than one interpreter). > This should be fixed at OS level, bash isn't the only shell that exists, > and shells aren't the only programs that attempt to exec binaries which > look as if they should be executable. This would no doubt be a fascinating conversation to have with the Linux kernel developers, but to be honest it looks like an intentional POSIX "deviation", shall we say. In the meantime I think that the most likely response will be "but this is behaving exactly as documented, just add a Linux case to your application to handle Linux error codes". > ps: as an aside, no-one cares about any race conditions here, even if bash > (or some other process) were to attempt to examine the file after an exec > failure to generate a nicer message ... the only thing affected would be > the error message, and if you get weird messages when attempting to run > binaries which are changing as you're doing it, it should be no surprise. Indeed, that was part of why I suggested doing so was harmless. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Misleading error when attempting to run foreign executable
On 10/10/21 20:24, Dmitry Goncharov via Bug reports for the GNU Bourne
Again SHell wrote:
> On Sun, Oct 10, 2021 at 7:52 PM Ángel wrote:
>> Looking at the code, it doesn't even need an extra stat(), it already
>> knows the file exists
>
> Attempts to know ahead of time or check afterwards that the file
> exists are subject to a race condition.
Immediately above the patch diff you are responding to, is:
#if defined (HAVE_HASH_BANG_EXEC)
READ_SAMPLE_BUF (command, sample, sample_len);
if (sample_len > 0)
sample[sample_len - 1] = '\0';
if (sample_len > 2 && sample[0] == '#' && sample[1] == '!')
{
char *interp;
int ilen;
interp = getinterp (sample, sample_len, (int *)NULL);
ilen = strlen (interp);
errno = i;
if (interp[ilen - 1] == '\r')
{
interp = xrealloc (interp, ilen + 2);
interp[ilen - 1] = '^';
interp[ilen] = 'M';
interp[ilen + 1] = '\0';
}
sys_error (_("%s: %s: bad interpreter"), command, interp ? interp
: "");
FREE (interp);
return (EX_NOEXEC);
}
#endif
So I wonder, if bash already in this exact case attempts to open() the
file and read() it to look for a shebang, what's the harm in assuming
(or checking) that it exists in this patch?
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
OpenPGP_signature
Description: OpenPGP digital signature
Re: EPOCHREALTIME
On 8/19/21 12:18 PM, Léa Gris wrote:
> Le 19/08/2021 à 16:41, Eli Schwartz écrivait :
>> On 8/19/21 9:41 AM, Léa Gris wrote:
>
>
>> The error occurs, one would imagine, during the "convert string to
>> float" stage, after parsing argv or forking to bc or whatever, but
>> *before* passing it as an argument to printf(3). Here, bash is just
>> doing good error checking -- if you used
>> strtof("3.14159265358979323844", NULL) under a fr_FR.UTF-8 locale, it
>> would silently drop everything after the ".", and you would
>> "successfully" print 3,, but bash reports an error message.
>
> A programming language shall distinguish between display format and data
> format.
>
> Locale settings are for display format and shall not interfere with
> arguments parsing which is data format, or it create such data
> portability issues.
Whether you are right or wrong is a matter which I'm sublimely
indifferent to.
You seem to have missed the point of my statement, which is that this is
not about bash at all, and if you have an issue here, you should take it
up with the standards body.
The strtof() function of the C programming language is violating your
directive.
> This is exactly how I read the note from the POSIX documentation:
>
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/bc.html#tag_20_09_16
>
>
>> The bc utility always uses the ( '.' ) character to represent
>> a radix point, regardless of any decimal-point character specified as
>> part of the current locale. In languages like C or awk, the
>> character is used in program source, so it can be portable and
>> unambiguous, while the locale-specific character is used in input and
>> output. Because there is no distinction between source and input in
>> bc, this arrangement would not be possible. Using the locale-specific
>> character in bc's input would introduce ambiguities into the language
>
> Especially:
>
>
>> In languages like C or awk, the character is used in program
>> source, so it can be portable and unambiguous
I challenge your reading of the POSIX documentation. The important part
is this:
> Because of such ambiguities, the character is used in input.
> Having input follow different conventions from output would be
> confusing in either pipeline usage or interactive usage, so the
> is also used in output.
POSIX acknowledges that using a comma in the output would be "correct
and follow our own rules, but since it's confusing we won't do it".
POSIX does NOT have similar logic in the documentation for printf, which
*mandates*
> The argument operands shall be treated as strings if the corresponding
> conversion specifier is b, c, or s, and shall be evaluated as if by
> the strtod() function if the corresponding conversion specifier is a,
> A, e, E, f, F, g, or G. Otherwise, they shall be evaluated as
> unsuffixed C integer constants, as described by the ISO C standard,
> with the following extensions:
and, indeed, strtof / strtod("3.14159265358979323844", NULL) in a French
locale is, as I originally explained, going to inherently follow
LC_NUMERIC no matter how silly you might think that is, because that's
how strtod/strtof work.
And you cannot pass a preprocessed float/double to the printf builtin,
because you don't have types, and therefore POSIX specifies how to
interpret it starting from a string.
> printf arguments are program source even if argument comes from a variable.
>
> All things considered, if you are using floating-point numbers in a
> shell script, you are clearly not using the right tool for the job, but
> sometimes options are limited or not under your control.
>
> Having a feature implemented in such a way, *that it cannot be used
> reliably or requires heavy work-around* (especially if you both need to
> process floating-point data in a portable format, and display in locale
> format)… is just calling for frustration and sorry errors:
>
> For the record:
>
> ash -c 'LC_ALL=fr_FR.utf8; printf "Pi: %2.4f\\n" "$(echo "4*a(1)" | bc
> -l)"'
> Pi: 3.1416
>
> bash -c 'LC_ALL=fr_FR.utf8; printf "Pi: %2.4f\\n" "$(echo "4*a(1)" | bc
> -l)"'
> bash: line 1: printf: 3.14159265358979323844: invalid number
> Pi: 3,
>
> dash -c 'LC_ALL=fr_FR.utf8; printf "Pi: %2.4f\\n" "$(echo "4*a(1)" | bc
> -l)"'
> Pi: 3.1416
>
> ksh -c 'LC_ALL=fr_FR.utf8; printf "Pi: %2.4f\\n" "$(echo "4*a(1)" | bc
> -l)"'
> Pi: ksh: printf: 3.14159265358979323844: arithmetic syntax error
> ksh: printf: 3.14159265358979323844: arithmetic synta
Re: EPOCHREALTIME
On 8/19/21 9:41 AM, Léa Gris wrote:
> Le 19/08/2021 à 15:10, hancooper écrivait :
>> Thusly, EPOCHREALTIME should not be made to depend on the locale. I
>> have seen many
>> workarounds, that complicate rather than simplify something that
>> should be straighforward
>> and germaine to direct numeric computation.
>
> A agree 100%
>
> It is as frustrating as printf arguments format being dependent on
> locale settings:
>
> This will fail because of questionable design decision of having a
> mutable argument format:
>
> LC_NUMERIC=fr_FR@UTF-8; printf 'Pi: %2.4f\n` "$(bc -l <<<'4*a(1)')"
>
> Note how the format indicator still use a dot, but the argument format's
> decimal separator is that of the system's locale.
>
> Imagine if C++ or Java had methods with different signature depending on
> system locale. You would scream fool. But for Bash, it was decided it
> was all fine.
Bash printf behaves the same as C here (format indicator and all).
But without typed arguments it needs to parse the string
"3.14159265358979323844" into a float, because everything is a string
and thus doesn't have different signatures ever.
The error occurs, one would imagine, during the "convert string to
float" stage, after parsing argv or forking to bc or whatever, but
*before* passing it as an argument to printf(3). Here, bash is just
doing good error checking -- if you used
strtof("3.14159265358979323844", NULL) under a fr_FR.UTF-8 locale, it
would silently drop everything after the ".", and you would
"successfully" print 3,, but bash reports an error message.
Locales define output formatting for display, not source code formatting
for machine translation. Your entire problem, from beginning to end, has
nothing whatsoever to do with signatures, and everything to do with a
lack of types.
bc doesn't communicate a float-typed value to printf, but it also
doesn't respect LC_NUMERIC (for the reasons specified in the POSIX
description under "Application Usage").
On the other hand, printing the output of bc into a C source file and
compiling it, rather than interpreting it via strtof, would cause it to
be parsed as machine-formatted, not display-formatted, and hence the
LC_NUMERIC separator would not skew the results.
For a C program, because C has types and bash does not.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
OpenPGP_signature
Description: OpenPGP digital signature
Re: gettext feature request
On 8/13/21 2:01 PM, Léa Gris wrote:
> I would have really loved if Bash expanded translations with its
> built-in printf using a `-g` flag for gettext and positional arguments
> support instead of expanding string literal translations.
>
> It would have allowed something like :
>
> ```sh
> #!/usr/bine/env bash
>
> # -g Gettext translation of Hello World into variable my_string
> printf -v my_string -g 'Hello World!'
>
> # Printout using gettext translated format string
> printf -g 'String %1$q has %2$d characters\n' "$my_string" "${#my_string}"
> ```
What would be the point of adding a -g option to printf, which duplicates:
printf -v my_string $"Hello World!"
printf $"String %1\$q has %2\$d characters\n" "$my_string" "${#my_string}"
And then, since you are relying on a separate feature from translation
altogether, that of re-ordered input parameters to printf, ask for that
to be added, which is currently a fatal error:
bash: printf: `$': invalid format character
and doesn't require adding a new option to printf just for the use of
translations.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
OpenPGP_signature
Description: OpenPGP digital signature
Re: gettext feature request
On 7/24/21 2:48 PM, Chet Ramey wrote: > On 7/24/21 10:35 AM, Jean-Jacques Brucker wrote: >> >> Planning to use the /$"string/" feature in my bash code made me think >> too much : https://github.com/foopgp/bash-libs/tree/main/i18n >> >> >> ...what I really *love* to see in bash, is a /$'string'/ feature, >> which doesn't parse any «`» or «$» characters. > > So you want a translation feature without any further interpretation? Or > one just without command substitution? > > What about an option to enable this variant behavior? What do you think > would be a suitable name? In the thread Greg linked, he submitted a patch back in 2009 to implement `shopt -s safelocale`. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: simple prob?
t I'm also an overly restrictive person who grumps at people trying to set: declare -A jobs=(["first run"]=$'0\n' ) Instead you are arguing in bad faith that "That's not a bash-caveat, but a bash feature!" When strictly speaking your code is flawed, it doesn't correctly handle indexed arrays with spaces in the key and doesn't forbid them either. >> This won't protect against all code injections, of course; only the >> ones that contain a whitespace character. >> > > Nothing protect against "all" 'X' or "everything", especially when > talking > about security. Security is best handled as a layered approach with > different > layers protecting against different things. There's no such thing as a > 'Security Magic Bullet'. Just because NOTHING protects against "ALL" > [anything] is > not a reason to take extreme action. In fact realization that NOTHING is > perfect > is one of the better defenses against over-reacting in response to > supposed security > flaws. Removing quotes doesn't protect against anything, it turns a tiny minority of security flaws into inscrutable errors from printf, makes a different tiny minority of valid use cases incorrectly error, then fails to log either problem, abort with errors when it cannot do anything sensible with the input, or anything else of the sort. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: simple prob?
On 6/29/21 4:21 PM, L A Walsh wrote:
> I hope a basic question isn't too offtopic.
Well, if you don't think this is a bug in bash, but something you need
help figuring out, maybe you'd prefer to use the "help-bash" list?
> Say I have some number of jobs running:
>
>> jobs|wc -l
> 3
> ---
> in a function (have tried shopt -s/-u lastpipe; neither way worked)
> njobs() {
> jobs |wc -l
> }
>> njobs
> 3
>
> Would like to pass a varname to njobs to store the answer in, like:
> njobs() {
> jobs|wc -l
> #magic puts val in $v
> printf {$1:+-v $1} "%s\n" "$v"
This is obviously broken, because {$1:+expand} isn't how bash works. Try
swapping the "{" and the "$"...
> }
>
> So I can run:
>
>> njobs n
> echo "$n"
> 3
>
> ---
> How can I put the output into '$v'
> *without* using a temporary file?
>
> This seems so basic, yet its eluding me.
> Could someone throw me a clue-stick?
> Tnx!
What's wrong with the obvious
v=$(jobs | wc -l)
> p.s. - a trivial util func producing jobs:
>
> resleep() { alias my="declare " int="my -i "
> int n=${1:-3} t=${2:-99}; echo "$n jobs @ ${t}s:"
> while ((0 < n--)); do sleep "$t" & done; }
>
>
>
>
>
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
OpenPGP_signature
Description: OpenPGP digital signature
Re: `&>' doesn't behave as expected in POSIX mode
On 6/20/21 6:30 AM, Chris Elvidge wrote: > On 20/06/2021 09:05 am, Oğuz wrote: >> $ set -o posix >> $ uname &>/dev/null >> $ >> >> `uname &' and `>/dev/null' should be parsed as two separate commands; >> that, if I'm not missing anything, is what POSIX says. But bash >> doesn't do that in POSIX mode, and redirects both stderr and stdout to >> `/dev/null'. >> >> >> Oğuz >> >> > > If you want to parse them as two separate commands, separate them. *Neither* behavior is similar to e.g. dash's behavior, or for that matter ksh. Both print the uname output after registering it as a backgrounded job (redirection does not get applied to the command itself). At any rate... are you suggesting the POSIX rules require a space here in order to distinguish the behavior? -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Suggestion/question
On 6/2/21 6:06 PM, jonasxavier wrote: > Hi. > > I would like to know why bash does not follow FHS 3.0 (Filesystem > Hierarchy Standard) suggestions. FHS 3.0 says > > "... 3.8.2. Requirements ... If an application needs to create more > than one dot filethen they should be placed in a subdirectory ..." > > Bash uses ~/.files instead of ~/.bash/files. But in my humble opinion > ~/.config/bash/files would be even better, because $HOME directory > turns a mess full of dot files. As far as I can tell, this was added in FHS 2.3, but not present in FHS 2.2; this means this FHS requirement is 17 years old (January 29, 2004). The earliest available snapshot in bash's git repo is bash-1.14.7.tar.gz, at which point ~/.bashrc was already used. This is in 1994. Someone else might know if it was in use between 1989 and 1994. Changing the location of the rc file that configures e.g. your environment variables, in a backwards-compatible manner, is "complicated". Furthermore, the application doesn't create this file, you do... and XDG basedirs is much more relevant and meaningful than the FHS, especially considering bash is used on far, far more than just Linux (the FHS is a linux standard, it is overseen by the Linux Foundation). Given that bash predates the formation of standards, it is hardly surprising that it has continued to use filenames compatible with the days before standards. So there is your "why". Assuming we don't care about the FHS, but do care about XDG... This was discussed a month ago in the following thread: https://lists.gnu.org/archive/html/bug-bash/2021-05/msg00093.html Personally, I think this would be a nice improvement, though it does lead to the question "how do you set $XDG_CONFIG_HOME before you read bashrc, where environment variables are typically defined". But, as Chet said, "This is not a high priority issue." -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Sort command doesn't sort '@' character correctly
On 5/20/21 12:43 PM, Michael Jensen wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -g -O2 -fdebug-prefix-map=/build/bash-a6qmCk/bash-5.0=. > -fstack-protector-stron> > uname output: Linux ubuntu 5.4.0-73-generic #82-Ubuntu SMP Wed Apr 14 > 17:39:42 UTC 2021 x86_64 x86_> > Machine Type: x86_64-pc-linux-gnu > > Bash Version: 5.0 > Patch Level: 17 > Release Status: release > > Description: > sorting lines where some starts with @ and some doesn't, it will not > sort the lines correct > > Repeat-By: > > echo "xxaxxon" > test.txt > echo "@zorg" >> test.txt > echo "@jupiterlander" >> test.txt > cat test.txt | sort > > Note it prints out: > > @jupiterlander > xxaxxon > @zorg > > Expected: > > @jupiterlander > @zorg > xxaxxon The "sort" utility is not part of bash, so this is the wrong mailing list for the question. However, you will probably find that your vendor's "sort" utility will act as you expect, if you invoke it like this: cat test.txt | LC_ALL=C sort -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: How to use bash loadables in a portable manner?
On 4/25/21 11:44 AM, Eli Schwartz wrote: >> This commit to the development branch adds a default value for >> `BASH_LOADABLES_PATH': >> >> https://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel=fb4ddc2d2b66b776e013ddadfce86270a71c323a >> >> So in the next version you will be able to use them in a portable manner. > > Oh hmm, thanks. Did not notice this. Interestingly, it doesn't seem to be including the default loadablesdir defined in Makefile.inc and bash.pc, but hardcodes several paths used in common configurations to be overridden in a similar manner to -DDEFAULT_PATH_VALUE=... It feels like this could probably stand to be a teensy bit smarter at autodetecting unusual prefixes. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: How to use bash loadables in a portable manner?
On 4/25/21 11:41 AM, Oğuz wrote: > 25 Nisan 2021 Pazar tarihinde Eli Schwartz yazdı: > >> The bash loadables available in the standard make install seem pretty >> useful and people might actually want to use them in scripts. But >> scripts cannot easily know *where* they are located. >> >> >> In the bash manpage is documented: >> >> BASH_LOADABLES_PATH >> A colon-separated list of directories in which the shell looks for >> dynamically loadable builtins specified by the enable command. >> >> I suppose that means bash would like to support discovery of these. >> Unfortunately, it's not very useful to find the builtin ones, since one >> would need to know which paths the current bash binary was configured >> with in order to manually set this variable in the script. >> >> Maybe it would be good to have this variable default to the path >> containing the shipped loadables? Is there any particular reason this is >> not already the case. > > > This commit to the development branch adds a default value for > `BASH_LOADABLES_PATH': > > https://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel=fb4ddc2d2b66b776e013ddadfce86270a71c323a > > So in the next version you will be able to use them in a portable manner. Oh hmm, thanks. Did not notice this. -- Eli Schwartz Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
How to use bash loadables in a portable manner?
The bash loadables available in the standard make install seem pretty useful and people might actually want to use them in scripts. But scripts cannot easily know *where* they are located. In the bash manpage is documented: BASH_LOADABLES_PATH A colon-separated list of directories in which the shell looks for dynamically loadable builtins specified by the enable command. I suppose that means bash would like to support discovery of these. Unfortunately, it's not very useful to find the builtin ones, since one would need to know which paths the current bash binary was configured with in order to manually set this variable in the script. Maybe it would be good to have this variable default to the path containing the shipped loadables? Is there any particular reason this is not already the case. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User
Re: When "set -u", "which" outputs environment: line 1: _declare: unbound
On 4/9/21 4:22 PM, Greg Wooledge wrote: > On Fri, Apr 09, 2021 at 03:08:25PM -0400, Craig Andrews wrote: >> Description: >> With "set -u", invoking "which" results in the output of: >> environment: line 1: _declare: unboundvariable >> That should not happen, and does not happen, with prior versions of >> bash. >> I'm using Fedora 34 (beta). >> >> Repeat-By: >> Run this shell script: >> #!/bin/bash >> set -u >> echo "$(which bash)" >> > > I can think of two possible reasons for this: > > 1) which(1) on your system is a bash script, and the error message is >coming from that script. > > 2) which on your system is a shell function, which is somehow made visible >to the script that you ran (possibly having been exported through the >environment, or declared in a file that's loaded via BASH_ENV). > > Are you able to duplicate the error by running "which bash" in an > interactive shell, or does it only happen in scripts? > > In whatever environment generates the error message, please run "type which" > to find out exactly how which is defined. > > If it's a program (e.g. /usr/bin/which), please run "file" on that program > to see whether it's a (bash) script. > > If it's a function, try to figure out where the function is being defined. Are you ready to have your day truly spoiled? Fedora's package for the external program GNU which, provides this /etc/profile.d/ script (automatically injected into the environment of any interactive shell sessions once you install GNU which -- how charming): https://src.fedoraproject.org/rpms/which/blob/rawhide/f/which2.sh Corresponding to the terrible advice from the GNU which manpage: https://man.archlinux.org/man/which.1#EXAMPLE So, it will on every `which` invocation feed the result of `alias; declare -f` to ` | which --weird-options` and, essentially, reinvent `type -a`. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: select syntax violates the POLA
On 4/1/21 1:18 PM, Greg Wooledge wrote: > On Thu, Apr 01, 2021 at 11:58:13PM +0700, Robert Elz wrote: >> >> | If $1 is not a directory, then you want: >> >> It is a directory, or I'd guess, quite likely a pattern chosen > > It's amazing how many people manage to post their code with NO comments > or explanations of what it's supposed to do, what assumptions are being > made about the inputs, etc. This leaves us to guess. > > I'm guessing that /usr/src/pkg/*/* is a list of files, not a list of > directories. You're guessing that it's a list of directories. > > In the original code, if it's a list of directories, then we have > "ls firstdir seconddir" which expands to a list of mangled filenames > without directory components. And since there's more than one directory, > the array ends up looking like: > > d=(README foo.c foo.h foo README bar.c bar.h foo.c foo.h foobar) > > with repeated filenames, and no knowledge of which filename came out of > which subdirectory. > > I was assuming that this was *not* how it was intended to work, and thus, > I made the guess that /usr/src/pkg/*/$1 expanded to a list of *files*, > and therefore generated a list of full pathnames, e.g. > > d=(/usr/src/pkg/applesauce/Makefile.PL /usr/src/pkg/bananabread/Makefile.PL) > > and then the user can make a meaningful selection from such a list. > > Now, this is obviously just a guess, and I could be extremely wrong > about it. For one thing, they used the variable name "d". That seems > to indicate that it should be a list of directories, rather than a list > of files. > > So, perhaps their code is simply *broken*, and where they *thought* they > would be getting a list of directories, they actually get a list of > mangled useless filenames, because they forgot the -d argument to ls. > >> | d=(/usr/src/pkg/*/"$1") >> >> definitely not that, the quotes are wrong in any case > > They are not! The quotes are wrong, *iff* the intent is that people should be allowed to pass quoted globs as $1 and let those be expanded to produce d=() Which, I can see where Robert is coming from in assuming this. >> (but apart from >> that, if filename expansion happens in array assignments (it doesn't in >> normal ones, and I dislike arrays, so ...) then without the quotes that >> might work. >> >> Alternatively >> d=( $( ls -d /usr/src/pkg/*/$1 ) ) > > This needs quotes around "$1" but yes, it's quite possible that this was > their intent. Of course, this is still *broken*, but it only breaks if > one of the directory names contains whitespace characters. It's likely > that they didn't happen to have any such directories in their test > environment, and therefore, this bug would go unnoticed for a long time. > >> or just >> d=( $( printf %s\\n /usr/src/pkg/*/$1 ) ) > > This is not much better. It still breaks on whitespace. > > If a list of directories (not their contents) is the desired outcome, > then my original code is perfectly fine: > > d=(/usr/src/pkg/*/"$1") > > This will expand to something like > > d=(/usr/src/pkg/vendor1/netcat /usr/src/pkg/vendor2/netcat) > >> Just to be sure.Personally I'd do >> >> set -- /usr/src/pkg/*/$1 >> >> and then simply use the positional parameters. > > This still needs quotes around "$1", and it's basically the same as > the array assignment -- just less flexible, and more suited to sh than > bash. But certainly, this is viable in many cases. > >> Yes. But it turns out not to matter in this case, as none of >> the names will ever contain anything but "normal" characters >> (no spaces, newlines, asterisks, ...) > > Famous Last Words™ It's not too much of a leap if the names are required to be NetBSD ports tree names of packages, which presumably ban such characters and enforce that ban in their tooling. It's not something one would be wise to get into the habit of doing elsewhere, and the quotes don't hurt, so why not add them... unless, again, we assume the intent of the script is in fact to pass in globs. These "Famous Last Words™" are really just a fancy way to repeat your initial, valid objection that the OP gave zero comments or explanation and therefore you're left to guess at the intention without having any contextual knowledge about what /usr/src/pkg is or why one would want to cd into any of them and run make. I suspect Robert does know quite a bit about it though. :) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: missing way to extract data out of data
On 3/23/21 4:46 AM, Andreas Schwab wrote: > On Mär 22 2021, Dale R. Worley wrote: > >> Greg Wooledge writes: >>> Partly true. seq(1) is a Linux thing, and was never part of any >>> tradition, until Linux people started doing it. >> >> Huh. I started with Ultrix, and then SunOS, but don't remember learning >> seq at a later date. > > According to <https://www.freebsd.org/cgi/man.cgi?query=seq>, seq > appeared in Version 8 AT UNIX. > >> I've never tracked down why, but the Perl executable is a lot smaller >> than the Bash executable. > > Is it? > > $ size /usr/bin/perl /bin/bash >textdata bss dec hex filename > 2068661 27364 648 2096673 1ffe21 /usr/bin/perl > 1056850 22188 61040 1140078 11656e /bin/bash > > Of course, a lot of perl is part of loadable modules. How thoroughly do you want to cheat? $ size /usr/bin/perl /bin/bash textdata bss dec hex filename 3840 792 1646481228 /usr/bin/perl 923139 22092 60800 1006031 f59cf /bin/bash $ du -sh /usr/bin/perl /bin/bash 16K /usr/bin/perl 932K/bin/bash Gosh, how does the perl interpreter define an entire language core in 16kb??? Pure magic. Naturally, the loadable modules you mention are very significant either way. $ pacman -Qi bash perl Name: bash Version : 5.1.004-1 [...] Installed Size : 8.19 MiB [...] Name: perl Version : 5.32.1-1 [...] Installed Size : 57.63 MiB And, for extras, it turns out most of the size of perl's no-modules interpreter (that ever so tiny 16kb binary) is in $ du -sh /usr/lib/perl5/5.32/core_perl/CORE/libperl.so 3.7M/usr/lib/perl5/5.32/core_perl/CORE/libperl.so $ size /usr/lib/perl5/5.32/core_perl/CORE/libperl.so textdata bss dec hex filename 3604395 77020 25352 3706767 388f8f /usr/lib/perl5/5.32/core_perl/CORE/libperl.so Which is indeed much larger than bash (one binary, no libbash.so, whether to count the separate libreadline.so in my vendor configuration is debatable). -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Wanted: quoted param expansion that expands to nothing if no params
On 3/24/21 3:49 PM, Ilkka Virta wrote: > On Wed, Mar 24, 2021 at 9:38 PM L A Walsh wrote: > >> Hmmm...Now that I try to show an example, I'm not getting >> the same results. Grrr. Darn Heizenbugs. >> > > Just remember that if you test with printf, it always prints at least once, > which makes it look exactly as if it got an empty string argument, even if > there are none: > > $ set -- > $ printf ":%s:\n" "$@" > :: > $ set -- ; printf ":%s:\n" x "$@" > :x: Testing with set -x is preferred. $ set -x --; printf ":%s:\n" "$@" + printf ':%s:\n' :: $ set -x --; printf ":%s:\n" x "$@" + set -x -- + printf ':%s:\n' x :x: -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Wanted: quoted param expansion that expands to nothing if no params
On 3/23/21 11:24 PM, L A Walsh wrote:
> Too often I end up having to write something like
> if (($#)); then "$@"
> else # = function or executable call
> fi
>
> It would be nice to have a expansion that preserves arg boundaries
> but that expands to nothing when there are 0 parameters
> (because whatever gets called still sees "" as a parameter)
>
> So, example, something like:
>
> $~ == "$@" #for 1 or more params
> $~ == no param when 0 param, # so for the above if/else/endif
> one could just use 1 line:
It's not clear to me, how you expect this to differ from the existing
behavior of "$@" or "${arr[@]}" which already expands to
rather than an actual "" parameter.
> $~
>
> My examples used ~, as I didn't think it was used anywhere.
>
> I can't control how called programs will handle / deal with a
> present, but empty parameter, which is why I thought something that
> expands to nothing in the empty case would seem ideal.
>
> Anyone else have a trivial solution for this problem?
Does "be mindblown when it worked as is to begin with" count?
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
OpenPGP_signature
Description: OpenPGP digital signature
Re: about the local not-on-every-function-separately var issue
On 3/22/21 9:38 PM, Dale R. Worley wrote: > Greg Wooledge writes: >> Now, the big question is WHY you thought something which is not correct. >> >> The most common reasons that people think something which is wrong are: > > In my experience, a common reason is that the documentation does not > concentrate in one place that users are certain to read, a complete, > clear description of the situation. For instance, you give a complete, > clear description: > > Bash uses "dynamic scope" when it expands variables. This means that > it looks first in the current functions local variables; if the variable > isn't found there, it looks in the caller's local variables, and then > in the caller's caller's local variables, and so on, until it reaches > the global scope. > > And that behavior is implied if you read the definition of "local" > closely. But I couldn't find any text in the manual page that states > that directly. It's described pretty plainly in FUNCTIONS. It even mentions dynamic scoping as an explicit term. "Ordinarily, variables and their values are shared between the function and its caller. If a variable is declared local, the variable's visible scope is restricted to that function and its children" It also finishes off its description by pointing out "When the function returns, the global variable is once again visible." > (*Very* closely, as the text refers to "the function and > its children". But function definitions don't nest; what it means is > "the function *invocation* and its child invocations.) Since you yourself felt the need to describe this as "nested" functions, not "child" functions, I'm curious why you thought nested functions as a source code paradigm is a better comparison than, say, parent/child processes as an invocation paradigm? python has nested functions. If I ever heard someone refer to a child function in python, I would not expect it to refer to nested functions -- I'd expect it to refer to the call stack. Going back to the previous definition I pointed out... note the explicit comparison of "caller" and "children" as opposites. > In principle it > should be stated at the point where parameter expansion is introduced, > as it is the *definition* of what parameter expansion does. I don't see what principle makes you think it should be removed from its current location, the description of functions (which is extremely relevant to function scoping) and moved to parameter expansion, which is currently all about the syntax rules non-local and local variables share. Which is still not the location where local is defined and described. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: is it a bug that \e's dont get escaped in declare -p output
On 3/17/21 12:43 PM, Alex fxmbsw7 Ratchev wrote: > it makes the output no more possible if interpreted by tty > the \e's get processed by terminal and no more .. just an example where it > is so: > > var=$'1\e[G\e[K2' ; declare -p var > > if G was H for other line then it completly messes the terminal up > > isnt it better to escape by \e ? .. Why is it the bash script interpreter's job to escape user-supplied strings based on how a terminal emulator chooses to *visually* display the results? -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: doesnt interate the array anymore, what am i doing wrong
On 3/7/21 8:24 PM, Alex fxmbsw7 Ratchev wrote: > and as last note to end this thread: There is no thread. There are, in fact, four threads. I suspect you're using gmail.com's web client (??), which does not generate working emails many times. Specifically, for whatever reason, all your emails are not being sent as responses to other emails. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: syntax error while parsing a case command within `$(...)'
On 2/14/21 3:35 PM, Andreas Schwab wrote: > On Feb 14 2021, Eli Schwartz wrote: > >> Just a running trend that esac does not get recognized without a >> separator. > > $ bash -c 'case x in x) esac' The thread title does mention $(...) as the relevant context up for discussion. Apologies if my terseness made you think I've moved off of the thread subject; I did not do so. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: syntax error while parsing a case command within `$(...)'
On 2/14/21 2:43 PM, Dale R. Worley wrote: > Before we worry about what to change, I want to note that the original > example is syntactically incorrect. The example is > > $ bash -c ': $(case x in x) esac)' > > But the manual page makes it clear that each case must be ended with > ";;". > >case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac > > Now, I haven't investigated what cleverness Bash uses, but all the cases > I've tested that conform to the case syntax are handled correctly inside > this $(...): > > $ bash -c ': $( case x in x) : ;; esac )' > $ bash -c ': $( case x in x) true ;; esac )' > $ bash -c ': $( case x in (x) true ;; esac )' > > It even works with the degenerate case where there are no coices, though > writing it is hard because "esac" is a keyword: > > $ bash -c ': $( case x in > more> esac )' > > This is with an old version, 4.2.53(1). $ bash -c ': $( case x in x) esac ); echo $?' 0 Doesn't seem like a special power of your degenerate case. Just a running trend that esac does not get recognized without a separator. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Feature Request: scanf-like parsing
On 1/25/21 12:48 PM, Alex fxmbsw7 Ratchev wrote: you havent been on #bash freenode irc where it is the reverse to this quote 'no one ..', where also 'bash is no coding language' and x other invalid things are common, to all i disagree, but the masses of ignorant wrong teachen ppl overwealth me and i mostly left I'm fairly positive we've crossed paths on #bash, numerous times. I'm not sure when I joined, exactly, but my current logs go back about 2 years. I know exactly what you're talking about, and you're misrepresenting it. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Feature Request: scanf-like parsing
On 1/25/21 11:28 AM, Alex fxmbsw7 Ratchev wrote: cool, now that u're the main bash coder greycat and co wont fall like cannibals over you for using eval cheers for old code No one ever said eval is evil, if you're using it solely to consume the output of a shell builtin functionality specifically designed to produce eval-safe text. In fact, greycat's official stance on precisely that can be found here (and agrees that Chet's example is reasonable and correct): https://mywiki.wooledge.org/BashFAQ/048#Examples_of_good_use_of_eval Opinions differ on whether it's evil to try writing your own text for eval'ing, since most people doing so seem to get it wrong. But that's neither here nor there. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Bash 5.1 and Ncurses
On 1/15/21 12:12 AM, Jeffrey Walton wrote: Hi Everyone, I configured Bash 5.1 with prefix=/usr/local, --with-curses and --enable-multibyte. The link failed with undefined references, like tputs. I have the wide char Ncurses built and installed in /usr/local. I found I needed to add -ltinfow to LIBS and LDLIBS for the build to succeed. I'm pretty sure we are supposed to use the wide character version of Ncurses nowadays. I believe the wide character version supplies both multibyte and single byte functions, so it should just work with all software nowadays. It seems like configure should be able to find the Ncurses library. Maybe Ncurses needs one of those --with-libncurses-prefix. bash's configure script should report that it is checking "which library has the termcap functions", specifically for the tgetent symbol. It probes libtinfo before libncurses or libncursesw. On your system it might be misdetected. On my linux distro, libtinfo does not exist, except as an alias to libncursesw.so.6 -- but the official Debian package for bash overrides the detection by running: make TERMCAP_LIB="-ltinfo" -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Use of `mktemp' is dangerous, better use `mkstemp'...
On 1/14/21 11:45 PM, Jeffrey Walton wrote: Hi Everyone, I'm building Bash 5.1 from sources. This may be of interest: /usr/bin/ld: ./lib/sh/libsh.a(tmpfile.o): in function `sh_mktmpname': /home/jwalton/Build-Scripts/bash-5.1/lib/sh/tmpfile.c:160: warning: the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp' https://lists.gnu.org/archive/html/bug-bash/2020-07/msg00128.html -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: non-executable files in $PATH cause errors
On 1/10/21 10:49 AM, Ángel wrote: On 2021-01-10 at 08:52 +0100, n952162 wrote: Hello, I consider it a bug that bash (and its hash functionality) includes non-executable files in its execution look-up and then (inevitably) simply reports an error, because its such files aren't executable. Perhaps it's there to support PATH look up for arguments to the bash command. That would also be a bug. Why should it be okay to execute a non-executable script? Supporting users who are too lazy to chmod a file ought to be less important than supporting users who want fine-grain control over what's executable and what's not. Hello I can't reproduce what you report. $ mkdir foo bar $ printf '#!/bin/sh\necho Program "$0"\n' > foo/program $ printf '#!/bin/sh\necho Program "$0"\n' > bar/program $ PATH="$PATH:$PWD/foo:$PWD/bar" $ chmod +x bar/program $ program It is executing bar/program, not foo/program which is earlier in the path, but not executable. Maybe you just made the earlier program not executable, and the old path is still being remembered? You should run hash -r after making executable changes that will make an already-executed command find a different program in the path (in the example above, making foo/program executable, or removing again its +x bit). In the event only $PWD/foo is in PATH, bash fails to find *any* executable "program" command, and it then looks for non-executable forms. $ (PATH="$PATH:$PWD/foo:$PWD/bar"; program) Program /tmp/bar/program $ (PATH="$PATH:$PWD/foo"; program) bash: /tmp/foo/program: Permission denied This is also the behavior of dash, zsh, and ksh, though only bash reported the entire pathname of the file without execute permissions. dash: 2: program: Permission denied zsh: permission denied: program ksh: program: cannot execute [Permission denied] Hardly a bug in bash, merely a difference of opinion between the OP and the people who design shells. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: declare -p name=value thinks 'name=value' is variable
On 1/8/21 4:05 AM, William Park wrote: Another issue I came across. declare -p name=value thinks 'name=value' is the variable. My reading of manpage seems to say 'name' should be used, but not sure. Is this a bug? Workaround is, of course, use separate lines, declare name=value declare -p name There's an obvious inference here, that declare -p only prints, does not create/modify, and only accepts NAMEs, therefore the optional [=VALUE] available to other modes will not be interpreted the way you seem to think it will be, in *this* mode. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Checking executability for asynchronous commands
On 12/29/20 10:28 AM, Chet Ramey wrote: On 12/28/20 5:30 PM, Eli Schwartz wrote: (Though I have to wonder at these amazing AWOL commands that get uninstalled on people all the time right in the middle of their scripts. It's a potential security concern, though that class of vulnerabilities mostly involves executables being changed between testing and execution. Right, the race condition / security concern is specifically based on the idea that one is checking for permission / authority to run a program, possibly as setuid, and it gets replaced by something malicious before being used. If you were going to blindly run the program either way, then having it be *uninstalled* (i.e. does not exist, period) is... probably not going to result in security concerns. It will just fail to run. And it would do so even without the race condition. By all means, let people be concerned about their commands being replaced by attack code. Not about them being rm'ed. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Checking executability for asynchronous commands
On 12/28/20 4:45 PM, Léa Gris wrote: When you handle such logic within Bash you already lost on a race condition when foo is readable and executable when you test it, but when it reaches the actual execution, it is no longer the case. Bash is full of race conditions, because bash was never meant as a General Purpose language. Bash is a good command sequencer. Now if ppl forget to use wait PID after launching a sub-shell background command, then shame on them. The race condition doesn't matter if they anyways need to check the status by waiting on the PID, in order to handle "it executed, but resulted in failure". So it's not very tragic if the race condition resulted in users being unable to see the *enhanced* error message describing the missing dependency executable condition... this entire thread seems to just be about error reporting, right? (Though I have to wonder at these amazing AWOL commands that get uninstalled on people all the time right in the middle of their scripts. Maybe if they use a package manager for both the scripts and the dependency executables, they could fully prevent race conditions *and* not even need to check if their dependencies are installed.) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Checking executability for asynchronous commands
On 12/28/20 8:15 AM, Greg Wooledge wrote: On Sun, Dec 27, 2020 at 08:02:49AM -0500, Eli Schwartz wrote: I'm not sure I understand the question? My interpretation is that for an async *simple* command of the form cmd args & where cmd cannot be executed at all (due to lack of permissions, perhaps, or because the external program is not installed), they want bash to set $? to a nonzero value to indicate that the command couldn't even be started. I've seen similar requests several times over the years. The problem is that the parent bash (the script) doesn't know, and cannot know, that the command was stillborn. Only the child bash process can know this, and by the time this information has become available, the parent bash process has already moved on. The only way the parent can obtain this information is to wait until that information becomes available. Actually, I don't see why one could not circumvent the entire process, and do this instead if cmd=$(type -P foo) && test -x "$foo"; then foo & else echo "error: foo could not be found or is not executable" fi But I do get the initial premise of the thread. I don't get the *defense* being offered though. The logic here seems to be completely bankrupt -- saying bash needs new features (that are not well thought out) so you don't "need" to include code to handle your intentions, is not a winning argument. The OP seems to think that "people will occasionally forget to run `wait`", and wants to know if we "care" that people will forget and if Chet will add new features to bash in order to cater to these forgetful people. This is what I don't understand. Why should we care? The official advice is to run `wait` (or perform executability checks upfront, or whatever). -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: New Feature Request
On 12/27/20 12:38 PM, Saint Michael wrote: Bash is very powerful for its ability to use all kinds of commands and pipe information through them. But there is a single thing that is impossible to achieve except using files on the hard drive or on /tmp. We need a new declare -g (global) where a variable would have its contents changed by subshells and keep it. I.e. all subshells may change the variable and this will not be lost when the subshell exits. Also, a native semaphore technology, so different code being executed in parallel would change the variable in an orderly fashion. I use GNU parallel extensively, basically, my entire business depends on this technology, and now I need to use a database to pass information between functions being executed, back to the main bash script. This is basically ridiculous. At some point we need to turn Bash into more than a shell, a power language. Now I do arbitrary math using embedded Python, but the variable-passing restriction is a big roadblock. Philip Orleans Essentially, you want IPC. But, you do not want to use the filesystem as the communications channel for the IPC. So, what do you propose instead, that isn't the filesystem? How do you think your proposed declare -g would work? (There is already declare -g, maybe you'd prefer declare --superglobal or something?) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Re: Checking executability for asynchronous commands
On 12/27/20 5:01 AM, Markus Elfring wrote: If you have the pid of an asynchronous command -- and the easiest way to get that pid is by referencing $! after it was started -- you can call `wait' with that pid to retrieve the status, even if it's already terminated. Would you care if waiting on such identifications for background processes will occasionally be forgotten? How many efforts would you invest to add potentially missing wait function calls? Would you care if configuring bash to wait on identification of background processes will occasionally be forgotten? Would you care if checking the status of foreground processes and doing different things based on success or failure will occasionally be forgotten? Would you care if will occasionally be forgotten? I'm not sure I understand the question? Writing programs in *any* programming language requires attention to detail and effectively conveying your need to the programming language. bash is no exception, even if people have a terrible habit of treating bash like it should be special or different merely because it uses subprocesses a lot, and is popular. A stronger argument must be made for new features rather than merely "sometimes people are extremely forgetful, we need a new language feature that doesn't fit in well and doesn't behave consistently, so they can be forgetful about that instead". -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User OpenPGP_signature Description: OpenPGP digital signature
Manual still incorrectly mentions PROMPT_COMMANDS
From CWRU/changelog:
3/23
eval.c
- execute_prompt_command: if an array variable PROMPT_COMMANDS exists,
prefer it to PROMPT_COMMAND and execute all of the commands contained as
array elements in index order.
doc/{bash.1,bashref.texi}
- PROMPT_COMMANDS: new array variable
8/25
eval.c
- execute_prompt_command: PROMPT_COMMAND can now be an array, subsuming
PROMPT_COMMANDS, which bash no longer looks for.
But, this never got removed from
https://git.savannah.gnu.org/cgit/bash.git/tree/doc/bashref.texi?h=devel=3f17ceca108d77c5e506f03c8051f0ff13a0e11c#n6846
https://git.savannah.gnu.org/cgit/bash.git/tree/doc/bashref.texi?h=devel=3f17ceca108d77c5e506f03c8051f0ff13a0e11c#n7525
It was also mentioned in the release notes at
https://lists.gnu.org/archive/html/bash-announce/2020-12/msg0.html
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
Re: bash: please provide an easy way to spawn a new process group from script
On 11/1/20 11:48 AM, Oğuz wrote:
> 1 Kasım 2020 Pazar tarihinde clime yazdı:
>> Hello, it doesn't work for me:
>>
>> $ export BASH_LOADABLES_BUILTIN=1
>
>
> by set, I meant to a path where loadable builtin binaries reside
It would be superbly helpful to mention the right variable though...
$ BASH_LOADABLES_PATH=/usr/lib/bash/
$ enable -f setpgid{,}
$ help setpgid
setpgid: setpgid pid pgrpid
invoke the setpgid(2) system call
[...]
Given the proper variable includes "_PATH", it is even self-documenting
that no, you cannot expect "1" to be remotely useful.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: GNU Bash profile code execution vulnerability enquiry
On 10/28/20 1:11 PM, Rachel Alderman wrote: > Hi Bash Maintainers, > > I've been made aware of a GNU Bash profile code execution vulnerability > https://exchange.xforce.ibmcloud.com/vulnerabilities/173116 reported last > December (2019-12-16) > Description: GNU Bash could allow a remote attacker to execute arbitrary > code on the system, caused by improper access control by the Bash profile. > By persuading a victim to open the Bash terminal, an attacker could > exploit this vulnerability to execute arbitrary code on the system. > https://packetstormsecurity.com/files/155687 > CVSS Base Score: 8.8 > CVSS Vector: (CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H) > There is no CVE identifier associated with the vulnerability and I've been > unable to determine whether there is a remediation available. Is anyone > aware of this vulnerability and where it may be tracked in Gnu Bash? I looked at your links. It seems this is a metasploit module of type "payload". Metasploit modules come in different types: - exploit: use a vulnerability to break into a system - payload: once the exploit is successful, inject shellcode into the system to do something malicious This specific payload uses a benevolent feature of GNU bash, subverted to evil purposes: the ability to run initialization commands when opening the terminal. In this case, the initialization command is a malware payload. There is no code execution vulnerability here, bash is a program that exists solely to performs code execution and you are supposed to treat your bash profile as security-sensitive. There is no way for an attacker to exploit this over the network. Bash does not read a profile from the network, and the profile is not accessible over the network. An attacker would need to first log in to your system with full privileges in order to install the malware. The malware would then run locally. Of course, any malware might itself contain a service to communicate over the network and receive updated attack instructions or open a backdoor. But this does not mean Bash itself is vulnerable to network attacks... ... In short: The IBM X-Force Exchange entry is completely incorrect and misunderstood the packetstorm link. The entry should be withdrawn entirely. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: echo builtin doesn't handle end-of-options flag
On 8/16/20 12:21 PM, Eric Blake wrote: > On 8/16/20 10:47 AM, Todd A. Jacobs wrote: > >> Description: >> The echo builtin accepts options, but does not seem to handle >> `--` correctly as the end of options. > > The correct behavior for echo is described here: > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html > "The echo utility shall not recognize the "--" argument in the manner > specified by Guideline 10 of XBD Utility Syntax Guidelines; "--" shall > be recognized as a string operand." > >> The expected behavior >> would be for -- to be removed, and any following flag-like >> substrings printed as-is. > > Your expectations are wrong, they contradict what POSIX says. > >> >> Repeat-By: >> echo "-n" # "" but expecting "-n" > > POSIX says this one is implementation-defined; so whether -n is treated > as an option or as a string to echo has to be documented by the > implementation (bash documents treating it as an option). Err... "Implementations shall not support any options." >> echo -- -n foo # -- -n foo > > POSIX says this has to output '-- -n foo' and a newline. -n wasn't > first, so you no longer have the implementation-defined behavior, but > well-defined. > >> echo -- "-n foo" # -- -n foo > > Also well-defined. > > I see nothing in your report about bash disobeying POSIX, but rather > confusion on your part about what POSIX actually requires. bash does disobey POSIX, but not in the manner which the report specifies. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: echo builtin doesn't handle end-of-options flag
On 8/16/20 11:47 AM, Todd A. Jacobs wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: darwin19.5.0 > Compiler: clang > Compilation CFLAGS: -DSSH_SOURCE_BASHRC -Wno-parentheses -Wno-format-security > uname output: Darwin titan.local 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun > 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64 > Machine Type: x86_64-apple-darwin19.5.0 > > Bash Version: 5.0 > Patch Level: 18 > Release Status: release > > Description: > The echo builtin accepts options, but does not seem to handle > `--` correctly as the end of options. The expected behavior > would be for -- to be removed, and any following flag-like > substrings printed as-is. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html echo does not accept options, and is not permitted to accept options including "--". bash does accept -n, -e, -E in violation of POSIX, unless shopt -s shopt -s xpg_echo is set, but it doesn't implement -- and I don't really see a justification to do so. Either enable xpg_echo or use printf. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Dashes in function names: Undocumented?
On 8/12/20 11:51 AM, Matthew Persico wrote: > Thanks. I will take your word for it, as I cannot seem to find that text at > https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html, unless > that is not the canonical location for the manual. The canonical location is surely the source code: https://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel=091c6bc481bd2b405e39b6ad5967eb4fa2aab597 The link you're using claims to have been last updated on 12 May 2019, and I definitely would not rely on it being updated with every commit. You could get the html doc as a plaintext file from https://git.savannah.gnu.org/cgit/bash.git/plain/doc/bash.html?h=devel and then open it in an HTML viewer (opening the link in a browser does not work on its own because it is Content-Type: text/plain;) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Dashes in function names: Undocumented?
On 8/12/20 10:51 AM, Matthew Persico wrote: > I put a bug report into an emacs group because the bash syntax highlighter > failed to recognize functions whose names have dashes in them. > > The maintainer came back with this: > > I can reproduce this behaviour, but is it really a bug? Aren't the > names with '-' invalid? > The Bash Reference Manual says: > name > A word consisting solely of letters, numbers, and underscores, and > beginning with a letter or underscore. Names are used as shell > variable and function names. Also referred to as an identifier. > https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html > > I looked at the manual and I didn't see positive or negative > acknowledgement that dashes can be used in function names. But it does work. > > Update to manual? The bash-20191127 snapshot updated the manpage documentation for a function definition (to align with reality). It is now defined as: function fname [()] compound-command [redirection] and includes the description: When in posix mode, fname must be a valid shell name and may not be the name of one of the POSIX special builtins. In default mode, a function name can be any unquoted shell word that does not contain $. For context: word - A sequence of characters considered as a single unit by the shell. Also known as a token. name - A word consisting only of alphanumeric characters and underscores, and beginning with an alphabetic character or an underscore. Also referred to as an identifier. > name > A word consisting solely of letters, numbers, and underscores, and > beginning with a letter or underscore. Names are used as shell > variables. Also referred to as an identifier. > > function name > A word consisting solely of letters, numbers, underscores, dashes, and > beginning with a letter or underscore. Function names are used to label > shell > functions. > -- Eli Schwartz Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Syntax error in a Map/Hash initializer -- why isn't this supported?
On 8/10/20 5:56 PM, L A Walsh wrote:
> I wanted to use a map that looked like this:
>
> declare -A switches=([num]=(one two three)), where '(one two three)'
> is an associated list. Ideally, I could access it like other arrays:
> for types in ${switches[num][@]}; do...
> or
> switches[num]=(one two three)#gives:
> -bash: switches[num]: cannot assign list to array member
> or
> echo ${switches[num][0]} (="one").
>
> I defaulted to going around it by making it a string, like:
> switches[num]="one|two|three"
Correct, bash does not implement multidimensional arrays and its syntax
error if you try anyway, implements the minimum heuristics necessary to
convey that fact to you.
> or
> switches[num]="(one two three)" but why? It seems obvious that bash
> knows what I'm trying to do, so why not just do it?
.. I'm sorry, did you just state your disapproval at bash for not
implementing something, but having a useful error message in the
process? Would you prefer "bash: error: an unknown error occurred"?
> Some nested constructs seem to work:
>> b=(1 2 3)
>> a=(4 5 6)
>> echo ${a[${b[1]}]}
> 6
This is not a multidimensional array, obviously. It is two
single-dimensional arrays, with a value lookup in one array that is then
reused as the input key to perform another lookup in the second array.
> but more often than not, they don't. Is there a reason to disallow such?
Today's session of "Linda Walsh is year-by-year terrible at asking
questions".
Instead of asking:
"Can bash please implement multidimensional arrays as I think they're
nifty and would like to use them."
We are getting:
"Bash supports multidimensional arrays because $HACK but it doesn't let
me do them in X Y Z manner, is there a reason to arbitrarily forbid them
or can this pointless restriction be lifted."
This is a variant of http://catb.org/~esr/faqs/smart-questions.html#symptoms
Please read the whole article, but pay particular attention to the
section on "don't describe your problem in terms of the incorrect
diagnostic theory you've already decided is the real problem".
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: process substitution error handling
On 8/6/20 12:36 PM, k...@plushkava.net wrote: > On 06/08/2020 17:21, Eli Schwartz wrote: >> On 8/6/20 11:31 AM, Jason A. Donenfeld wrote: >>> That doesn't always work: >>> >>> set -e >>> while read -r line; do >>> echo "$line" & >>> done < <(echo 1; sleep 1; echo 2; sleep 1; exit 77) >>> sleep 1 >>> wait $! >>> echo done > > I wonder why wait $! doesn't do the job here. Because `echo "$line" &` sets a new value for $! after the <() did. More to the point, you want to wait $! *before* running any commands in the while loop, because if the <() failed, it might not be a good idea to run those commands. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: process substitution error handling
On 8/6/20 11:31 AM, Jason A. Donenfeld wrote:
> That doesn't always work:
>
> set -e
> while read -r line; do
>echo "$line" &
> done < <(echo 1; sleep 1; echo 2; sleep 1; exit 77)
> sleep 1
> wait $!
> echo done
So instead of your contrived case, write it properly. Check the process
substitution first, and make sure as a bonus you don't run anything if
if it failed:
set -e
mapfile -t lines < <(echo 1; sleep 1; echo 2; sleep 1; exit 77)
wait $!
for line in "${lines[@]}"; do
echo "$line" &
sleep 1
wait $!
echo done
(And in bash 5.1 you can remember each &'ed command to wait on later.)
> Either way, tagging on `wait $!` everywhere, and hoping it works like
> I want feels pretty flimsy. Are you sure you're opposed to set -o
> procsuberr that would do the right thing for most common use cases?
You're asking to add broken behavior which does the wrong thing in
bizarre, outrageous ways in many cases.
Your rationale is it works in your specific case, and you have a
mysterious aversion to wait $! which you haven't explained. (Why is it
flimsy? It performs exactly as documented.)
I think the problem here is twofold:
You have an emotional attachment to errexit and you think it solves
problems, so you want to turn everything into it.
Ultimately, you really don't have issues with "flimsy" anything, but you
want bash to be a different language, one with a formal exception model,
strong types, and possibly objects. Such a language would then react to
*programmer errors* by failing to compile, or dumping a traceback of the
call site, and could be used as a beginner-friendly language that
doesn't surprise the newbies.
There are a number of good languages like that. Shell isn't one of them.
Sometimes you need to understand how edge cases work.
Nevertheless, there are ways to code robustly in it. In another
language, you'd run forked commands upfront and save their results to
some sort of object or variable, and only then process it (after
checking the return code or using a function that automatically raises
exceptions on failure).
You can do that in bash too, as I demonstrated with mapfile and wait $!
It isn't really a terrible disadvantage that you need more characters to
type out your intent. Most particularly when your intent is "I want this
command to run async, but I also want it to exit the script at XXX
location if previous aysnc commands have failed". You cannot really get
around explicitly tagging the place where you want to raise errors.
Using an RNG to randomly raise the error automatically at unpredictable
locations isn't a valuable feature to add.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: process substitution error handling
On 8/6/20 9:15 AM, k...@plushkava.net wrote: > You beat me to it. I was just about to suggest wait $! || exit. Indeed, > I mentioned the same in a recent bug report against wireguard-tools. So if I understand correctly, you reported the lack of wait $! || exit in a script, and the script author instead responded by requesting a new feature in bash that does the same thing, except after a random interval during another command's execution? > I concur. The scripts I looked at tended heavily towards error handling > at a distance and were already subject to one or two amusing errexit > pitfalls. lol, I bet we could fix that by adding even more error handling at a distance. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: process substitution error handling
On 8/6/20 6:05 AM, Jason A. Donenfeld wrote: > Hi, > > It may be a surprise to some that this code here winds up printing > "done", always: > > $ cat a.bash > set -e -o pipefail > while read -r line; do >echo "$line" > done < <(echo 1; sleep 1; echo 2; sleep 1; false; exit 1) > sleep 1 > echo done > > $ bash a.bash > 1 > 2 > done > > The reason for this is that process substitution right now does not > propagate errors. Well, yes, it is an async command. But errexit has lots of other amusing traps, like $ echo $(false) > It's sort of possible to almost make this better > with `|| kill $$` or some variant, and trap handlers, but that's very > clunky and fraught with its own problems. > > Therefore, I propose a `set -o substfail` option for the upcoming bash > 5.1, which would cause process substitution to propagate its errors > upwards, even if done asynchronously. Propagate the return value of async processes like this: wait $! || die "async command failed with return status $?" > It'd certainly make a lot of my scripts more reliable. The use of errexit is the focus of a long-running holy war. Detractors would point out a very lengthy list of reasons why it's conceptually broken by design. Some of those reasons are documented here (including process substitution): http://mywiki.wooledge.org/BashFAQ/105 I recommend you do NOT claim this feature is a magic panacea that will make your scripts reliable; instead, just say you would find it useful. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bashbug's default editor
On 7/31/20 11:15 AM, Chet Ramey wrote: > On 7/31/20 11:05 AM, Eli Schwartz wrote: >> On 7/31/20 9:24 AM, Chet Ramey wrote: >>> On 7/31/20 4:14 AM, jazz_...@arcor.de wrote: >>> >>>> Bash Version: 5.0 >>>> Patch Level: 17 >>>> Release Status: release >>>> >>>> Description: bashbug doesn't use vi as default editor >>> >>> This is not a bug. >> >> The documentation is confusing (and IMHO wrong). >> >> "If EDITOR is not set, bashbug attempts to locate a number of >> alternative editors, including emacs, and defaults to vi." >> >> The word "defaults" there implies that vi is the preferred autolocated >> editor, but the intention is to have it the least preferred. > > I don't think it implies that. It's the default choice if there are no > other alternatives. In the sentence in the bashbug manpage, does the word "default" refer to the probing or what happens when probing fails? My belief is that people reading the manpage will understand it to mean the former (more natural reading). Your belief seems to be that people will understand it to mean the latter (I don't feel the sentence conveys this). ... The OP here seems to have interpreted it the way I did. So clearly it's confusing to at least 2 people out of millions. ... Another possible tweak of the documentation: "If EDITOR is not set, bashbug attempts to locate a number of alternative editors, including emacs, before defaulting to vi." and -> before -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bashbug's default editor
On 7/31/20 9:24 AM, Chet Ramey wrote: > On 7/31/20 4:14 AM, jazz_...@arcor.de wrote: > >> Bash Version: 5.0 >> Patch Level: 17 >> Release Status: release >> >> Description: bashbug doesn't use vi as default editor > > This is not a bug. The documentation is confusing (and IMHO wrong). "If EDITOR is not set, bashbug attempts to locate a number of alternative editors, including emacs, and defaults to vi." The word "defaults" there implies that vi is the preferred autolocated editor, but the intention is to have it the least preferred. Perhaps... "If EDITOR is not set, bashbug attempts to locate a number of alternative editors, including emacs, before falling back to vi." -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Bug Report concerning backslash in bash5
On 7/29/20 1:35 AM, Dale R. Worley wrote: > Ralph Beckmann writes: >> I found this misbehaviour in Bash 5 (e.g. GNU bash, version >> 5.0.16(1)-release (x86_64-pc-linux-gnu)): >> >> $ BLA="1\.2"; echo 'x/'$BLA'/y/' >> \x/1\.2/\y/ >> >> I don't see any reasonable reason for the generated backslashes here. > > My guess is that you're running into the fact that there are two types > of quoting character. One quotes *any* character that follows it, and > thus it never appears in "the output" unless it was doubled in the > input. The other type *only* quotes characters that are somewhow > special in that particular context. Reading the manual page: > >Enclosing characters in double quotes preserves the literal value of >all characters within the quotes, with the exception of $, `, \, and, >when history expansion is enabled, !. The characters $ and ` retain >their special meaning within double quotes. The backslash retains its >special meaning only when followed by one of the following characters: >$, `, ", \, or . > > So backslash-inside-double-quotes-in-bash is of the second type, it only > quotes things that would otherwise be special. So the value of $BLA is > 1-\-.-2, whereas if the period was replaced by $, $BLA would only have 3 > characters: > > $ BLA="1\$2"; echo 'x/'$BLA'/y/' > x/1$2/y/ > I'm not sure what point you're trying to make here. Are you arguing that *printing* these backslash escapes is valid behavior here, and the reporter has merely misunderstood the shell rules? -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Error expanding variable containing a directory name
On 7/23/20 12:08 PM, Lutz Adam wrote: > Description: > The content of $ML is "/media/lad". There's a directory > /media/lad/p24. Typing the command > ls $ML/p24 > a backslash is put befor "$" and the line looks like: > ls \$ML/p24/ > Typing the ENTER key shows: > ls: cannot access '$ML/p24/': No such file or directory. > > Repeat-By: > Create a variable containing a directory name. Put a > subdirectory there under. > Type a "$" and the variable followed by a slash, the name of the > subdirectora and the TAB-key. > > > The OS is Linux Mint 20 The built-in bash tab completion actually does this okay. Are you using the bash-completion project addon functionality? [eschwartz@arch ~]$ ls $HOME/ --> rewrites in-place to... [eschwartz@arch ~]$ ls \$HOME/ [eschwartz@arch ~]$ bash --norc --noprofile bash-5.0$ ls $HOME/ Display all 163 possibilities? (y or n) [...] -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Shell does not recognize umlaut in URL
On 7/23/20 11:56 AM, Lutz Adam wrote: > Description: > If a URL on the command line or in a file contains an umlaut as in > https://dict.leo.org/franz*ö*sisch-deutsch > the shell stops recognising the URL after the umlaut "ö". Only > the part > "https://dict.leo.org/franz*ö*; is underlined and used in a call > to a browser.** > > Repeat-By: > Put the above URL on the command line. > 1. Hover over the ULR. Only the left part is underlined. > 2. Press and hold CTRL, then left-click the URL: the URL is > opened in a browser, > but only the left part appears in the address field. > > The OS is Linux Mint 20 bash doesn't underline urls or open them in browsers. This is performed by the terminal emulator program. Please report this to the support medium provided by your current terminal emulator. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Return from function depending on number of parameters
On 7/4/20 7:54 AM, Chris Elvidge wrote: > On 03/07/2020 11:16 pm, Eli Schwartz wrote: >> On 7/3/20 2:00 PM, Chris Elvidge wrote: >>> I've used 'return $((!$#))' and 'return $[!$#]' to return an error if no >>> parameters given to function. >>> >>> Tested in a bash script 'exit $((!$#)) / $[!$#]' - both work. >>> >>> 'echo $((!$#)) / $[!$#]' - both echo 1 when no params, 0 when any >>> number of params. >>> >>> I'm told ( https://wiki.bash-hackers.org/scripting/obsolete ) that >>> $[...] is obsolete and that $((...)) should be used instead. OK so far. >>> >>> However 'N=0; echo $((!$N))' gives an error at the bash prompt. 'echo >>> $[!$N]' echo's 1 as expected. >> >> "gives an error" is a useless bug report. It works for me. >> >> $ N=0; echo $((!$N)) >> 1 >> >> My initial reaction to reading this thread is head scratching! >> >> As the other reply mentioned, there's actually a good explanation for >> why we get different results -- I disabled an annoying feature. >> >> $ set -o histexpand >> >> Now here's a useful bug report. "When I run this, I get the following >> incorrect results or error message": >> >> $ N=0; echo $((!$N)) >> N=0; echo $((histexpandN)) >> 0 >> $ N=0; echo $((!$N)) >> N=0; echo $(()N)) >> -bash: syntax error near unexpected token `)' >> $ N=0 >> $ echo $((!$N)) >> echo $((N=0N)) >> -bash: N=0N: value too great for base (error token is "0N") >> >> ... >> >> From there, people can give useful advice for solving the problem. (My >> preferred advice is "disable histexpand".) >> > > Thanks for the pointers to a better bug report. > And thanks for the info on histexpand. I obviously should read the > manual more carefully - I'd never come across it - as it's set by > default, not in any profile or bashrc file. It's the section "HISTORY EXPANSION", and set -o histexpand is equivalent to set -H, which controls this option. Many people first realize it exists only when they've been bitten by it. :) > Did you mean set +o histexpand to disable the feature? "set +o histexpand" is in my bashrc to disable the feature. "set -o histexpand" is what I used at the console to temporarily re-enable it, in order to reproduce your issue. As you can see from $ N=0; echo $((!$N)) N=0; echo $((histexpandN)) 0 the first history expansion inserted the word "histexpand" into my command, because the previous command used the word. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Return from function depending on number of parameters
On 7/3/20 2:00 PM, Chris Elvidge wrote: > I've used 'return $((!$#))' and 'return $[!$#]' to return an error if no > parameters given to function. > > Tested in a bash script 'exit $((!$#)) / $[!$#]' - both work. > > 'echo $((!$#)) / $[!$#]' - both echo 1 when no params, 0 when any > number of params. > > I'm told ( https://wiki.bash-hackers.org/scripting/obsolete ) that > $[...] is obsolete and that $((...)) should be used instead. OK so far. > > However 'N=0; echo $((!$N))' gives an error at the bash prompt. 'echo > $[!$N]' echo's 1 as expected. "gives an error" is a useless bug report. It works for me. $ N=0; echo $((!$N)) 1 My initial reaction to reading this thread is head scratching! As the other reply mentioned, there's actually a good explanation for why we get different results -- I disabled an annoying feature. $ set -o histexpand Now here's a useful bug report. "When I run this, I get the following incorrect results or error message": $ N=0; echo $((!$N)) N=0; echo $((histexpandN)) 0 $ N=0; echo $((!$N)) N=0; echo $(()N)) -bash: syntax error near unexpected token `)' $ N=0 $ echo $((!$N)) echo $((N=0N)) -bash: N=0N: value too great for base (error token is "0N") ... From there, people can give useful advice for solving the problem. (My preferred advice is "disable histexpand".) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: ‘command … &’ creates subshell
On 7/1/20 2:01 PM, Robert Elz wrote: > The lack of an easy method to force execution of a command from the > filesystem is very likely because there is generally no reason to do > that. That is, no-one ever really wanted it, so no method was invented. > > However, for whatever reason, people keep asking how. Either by directly > asking the question, or by picking one of the popular (but incorrect) > methods, and either claiming it works, or complaining when it doesn't. > > As best I can tell, other than doing a search of PATH, and then using the > full path name (which is what $(type -P cmd) does in bash, more succinctly > than shells that don't have this -P option to type), [...] One reason might be because the busybox applets for programs which are *normally* external disk utilities, can be lacking in functionality, often by a lot. In some cases, *hilariously* so (look at busybox ps, it's basically a long list of FIXMEs for POSIX requirements). I will confess however that I usually solve this by not considering busybox to be a suitable /bin/sh. The more general solution, possibly, would be e.g. invoking gawk, not awk, if you know you need extended functionality. awk might be a builtin (is, with busybox), but gawk probably isn't! -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: ‘command … &’ creates subshell
On 7/1/20 11:29 AM, Dmitry Alexandrov wrote: > Robert Elz wrote: >> | > If you need to ensure a disk executable is used, >> | Of course. Why "command" otherwise? >> >> That doesn't actually work, "command" can run built-ins > > Ah, yes, thanks for correction. > >> there is actually no method ‹…› which guarantees execution of an executable >> from an external file, other than by using the complete path name > > Well, I am not sure whether itʼs guaranteed, but the ‘exec’ as above does > execute an external command in GNU Bash. And in busybox, just like "env" it runs busybox builtin applets. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: ‘command … &’ creates subshell
On 7/1/20 6:24 AM, Robert Elz wrote: > Date:Wed, 01 Jul 2020 00:43:14 +0300 > From:Dmitry Alexandrov > Message-ID: > > | > If you need to ensure a disk executable is used, > | Of course. Why "command" otherwise? > > That doesn't actually work, "command" can run built-ins, there is > actually no method (not even via use of "env") which guarantees > execution of an executable from an external file, other than by > using the complete path name (containing at least one '/') of the > desired file. Indeed -- that's why I specifically used the bashism $(type -P ...) as type -P forces the printing of an external file executable. Regarding use of env, I presume you're referring to the busybox behavior here, where busybox's builtin env applet will still execute other builtin applets. That's the only case I'm aware of that will not guarantee execution of an executable from an external file -- so shouldn't env be safe to use as such, if you know: a) your interpreter is bash rather than /bin/sh, b) the external disk executable "env" is not a symlink to busybox -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: ‘command … &’ creates subshell
On 6/30/20 4:37 PM, Dmitry Alexandrov wrote: > Chet Ramey wrote: >> On 6/29/20 4:33 PM, Dmitry Alexandrov wrote: >>> 1) unexpected, as (a) it does not seem to be documented, (b) itʼs >>> counter-intuitive and (c) other bourne-like shells (busybox ash, dash, >>> ksh93, mksh, zsh) does not do that; >> >> It's an opportunity for optimization, not a bug > > I am not saying that this is a bug, only that it is inconsistent with almost > any other bourne-like shell. And I understood Chet's point to be that you shouldn't have "expected" something which isn't guaranteed. Just because it works out that way, doesn't mean it has to; it's not like other shells document this behavior. Relying on undocumented behavior isn't the greatest. :p >>> 2) inconvenient, as in some cases PID of the parent shell is not an >>> appropriate substitute for the PID of the real process. > >> since the results are correct. > > The results are _different_ across shell and, unfortunately, results provided > by Bash are the least useful. Consider: > > $ command … & > [1] 30737 > $ strace -p $! > strace: Process 30737 attached > wait4(-1, > > :-/ Useful replacements: You could decline to use "command", and it doesn't involve a shell process. $ cat & [1] 2092238 $ ps --pid $! --ppid $! PID TTY TIME CMD 2092238 pts/42 00:00:00 cat [1]+ Stopped cat If you need to ensure a disk executable is used, $ "$(type -P cat)" & [1] 2092352 $ ps --pid $! --ppid $! PID TTY TIME CMD 2092352 pts/42 00:00:00 cat [1]+ Stopped "$(type -P cat)" Anyway, Chet didn't say it won't be changed to do what you expected; after all, it's acknowledged as an opportunity for optimization, and optimization is good.. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: An xtrace variant
On 6/28/20 9:49 PM, Dale R. Worley wrote: > Eli Schwartz writes: >> Why not just run bash -x script-name without the bash -l option and >> without $BASH_ENV set? >> >> The first is implicitly true based on your stated command line. The >> second doesn't seem like a high bar to set, and it's not exactly default >> behavior... if you really do need $BASH_ENV can't you do the set -x at >> the end of that file? > > That's a good point, and I admit I've never studied out all of the logic > of Bash init files; I assumed that it executed ~/.bashrc as a matter of > course. > > So as long as I don't need the facilities in .bashrc for the script, > then "BASH_ENV= bash ..." suffices. You DON'T have the facilities in .bashrc for scripts per default. If your script relies on it, then it shouldn't. If it still does rely on it, then it should manually source it to ensure it is actually available, or you should factor out significant code into a shell library. While you're at it you could make a shell library that looks for $XTRACE and invokes 'set -x; unset XTRACE', which gets rid of the PITA you're worried about too, and lets you choose exactly where in the script you start debugging, so you can also skip preambles/lists of constants. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Undocumented feature: Unnamed fifo '<(:)'
On 6/29/20 6:06 AM, Robert Elz wrote: > Perhaps, but that would be a bizarre way to accomplish that. I got > the impression that the assumption was that there would somehow be just > one fork, no matter how many times the conversion was required. I > think that might be possible using bash - but not using date(1) to do > the conversions, it would need a purpose written command which read date > specs from stdin and wrote time_t's (in decimal) to stdout. If we are going with purpose-written commands I'd advise the OP to write a bash loadable builtin for date. There is already a strftime one, but it only lets you convert an epoch to an arbitrary format. And once we are in bash loadable builtins land, one can also add an option to save the output into a bash variable (like printf -v) instead of printing it to stdout. There are a couple of example loadable builtins that do this sort of thing already. No forks involved, not even one for the lifetime of the script. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Undocumented feature: Unnamed fifo '<(:)'
On 6/28/20 4:21 PM, Robert Elz wrote: > I noticed that explanation, but like Dennis, I fail to see how the > complicated version does any more than pretend there are less forks > happening. Was the speed of this actually measured, and if so, where > are the comparative results? Sure, and that can be pointed out, but that's a long way away from what Dennis actually said, which is "why not just use a function", implying that the use or not of a function is relevant here. > Either way, to make the conversion, the date command needs to be run > (in the complicated version, setbuf as well, which means an extra exec > at least) - running a command means a fork, and all we have to start > with is bash, so bash needs to fork to run date, each time it needs > to run. > > What evidence is there that the complicated way, with all of its extra > file opens, etc, is faster than the simple way, or involves less forks? IIRC bash will (if it can) optimize out $(cmd) to fork+exec cmd, rather than fork bash, then fork+exec cmd. Perhaps the OP is assuming that the fifo dance will result in one fork, rather than two? Pointing out this flawed assumption is a useful data point, saying "why not just use a function" is not. (In fact, I'd assume the $(function) is actively a bad idea as it would prevent bash from optimizing out the $(/usr/bin/date) fork.) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: An xtrace variant
On 6/28/20 3:19 PM, Dale R. Worley wrote: > When debugging a shell script, a useful command is: > > bash -x script-name > > The deficiency of this is that "-x" causes all simple commands in your > startup files to be echoed as they are executed, and often this output > is a lot longer than the output from the commands in the script. > > For a long time, I've wanted a variant of -x that only echoed the simple > commands after bash is done executing the startup files. I finally did > a test implementation, naming a new switch "-X". If it is set at the > end of the execution of the startup files, then bash sets "-x", which > causes subsequent simple commands to be echoed. Why not just run bash -x script-name without the bash -l option and without $BASH_ENV set? The first is implicitly true based on your stated command line. The second doesn't seem like a high bar to set, and it's not exactly default behavior... if you really do need $BASH_ENV can't you do the set -x at the end of that file? -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Undocumented feature: Unnamed fifo '<(:)'
On 6/28/20 11:55 AM, Dennis Williamson wrote:
> How is running your
>
> echo >&9 $_string
> read -t 1 -u 8 _out
>
> many times better than running your
>
>_out=$(date -d "$_string" +%s)
>
> many times?
>
> Why don't you just use a function?
>
> date_to_epoch () {
> date -d "$1" +%s
> }
>
> _out=$(date_to_epoch "$_string")
You seem to be suggesting that a dedicated function will reduce
boilerplate commands, resulting in more readable code.
You COMPLETELY failed to even read the reporter's message, which
specifically stated "In order to reduce forks and make some tasks a lot
quicker [...]"
Very plainly, the reporter does not care how much code is used, but does
care that $() forks a process. It's literally the entire point of the
discussion: to replace forking with redirection to fifos.
Now, you can argue that this should or should not be a goal, but it's
quite silly to conflate the goal with "your goal could be achieved by
using a function".
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: problem with extra space; setting? cygwin only?
On 6/23/20 3:33 PM, L A Walsh wrote:
> On Tue, Jun 23, 2020 at 12:30 PM Greg Wooledge wrote:
>
>> On Tue, Jun 23, 2020 at 12:21:24PM -0700, L A Walsh wrote:
>>> set output
>>> output=""printf ${1:+-v $1} "%s:%s" 23 myproc
>>> -Bash: printf: ` output': not a valid identifier
>>
>> Your command is simply wrong. You've got two double-quotes in a row,
>> which simply cancel each other out, as they contain only the empty
>> string. So, you are running this:
>>
>> output=printf ${1:+-v $1} %s:%s 23 myproc
> It's a gmail feature to mangle your input in weird ways when you
> cut/paste.
> the line with 'output=""' is separate from the "printf" line. Sorry.
Why would you use their incompetent webmail if you know it is broken?
Be that as it may, I can trivially reproduce your error by destroying my
own $IFS
$ output="" IFS=/
$ set output
$ printf ${1:+-v $1} "%s:%s" 23 myproc
bash: printf: ` output': not a valid identifier
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: hash -l with empty hash table prints to stdout
On 6/16/20 11:07 PM, bel...@web.de wrote: > `shopt' doesn't appear to have an option for giving output re-usable as > input (?). `shopt -p`, surely. Or `shopt -o -p`. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: hash -l with empty hash table prints to stdout
On 6/16/20 10:13 PM, Dale R. Worley wrote: > What a weird problem! Yes, "hash -l" should produce output that can be > read by bash. But sending the "bash: hash table empty" message to > stderr put you in the situation where "hash -l >file" doesn't really > leave the output of "hash -l" in "file". Of course it would. The output of hash -l would be "bash-parseable code to recreate the hash table", as in fact is currently documented. And *diagnostic* messages are not output, and should not be treated as output, and don't belong in that file one way or another. If one really, truly wanted to collect both the expected output *and* the diagnostic messages, one could redirect stderr too... but I don't foresee this as being a problem. ;) People would be more likely to redirect it to /dev/null. > Maybe you could make the empty > message be a comment? > > $ hash -l > # hash: hash table empty > > That is acceptable input for bash, has the right effect when read by > bash, is human-readable, and is capturable by redirecting stdout. As I suggested, yes. :) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: hash -l with empty hash table prints to stdout
On 6/16/20 9:30 AM, Chet Ramey wrote: > On 6/15/20 4:47 PM, bel...@web.de wrote: > >> Bash Version: 5.0 >> Patch Level: 3 >> Release Status: release >> >> Description: > >> `hash -l' is supposed to generate output that is reusable as input >> (`help hash' says so, at least). In case the hash table is empty, a >> string not re-usable as input is output to stdout. The exit code >> indicates success in this case. The string being output is locale- >> dependant (and therefore is hard to filter out in a work-around). > > It's an interesting question. An empty hash table is not an error > condition, and doesn't result in a non-zero exit status, so output to > stderr is not appropriate. You might want to temporarily enable posix mode, > which suppresses the message. stderr is also a reasonably appropriate place to emit debug info or warnings. I would consider this a warning condition... -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bash errexit shell option does not work in some cases.
On 6/8/20 10:02 AM, Chet Ramey wrote: > On 6/8/20 7:24 AM, Greg Wooledge wrote: >> This is part of the set -e nightmare, which is what we keep trying to >> get you (and everyone else) to understand. > > Most of the issue with `set -e' is folks not understanding when it's > not in effect. Well, Greg does have a pretty good point (which is a point I for one prioritize in my decision process for "do I personally recommend the use of set -e") in that another part of the issue is its doing different things across point releases of a shell. And that's not counting the times when the issue is people not understanding when it *is* in effect. Or people who know exactly when it is and isn't in effect, and have no issue, but decline to use it because the precise nature of "when it is and isn't in effect" decreases the usefulness of it sufficiently to make them decide they're better off implementing error checking a different way... and *that* is, I think, what Greg is pointing out. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Command substitution
On 6/2/20 9:44 PM, Dale R. Worley wrote: > Naively, I expect that > > FOO="$( command2 )" > command1 $FOO > > has the same effect as > > command1 $( command2 ) > > and > > FOO="$( command2 )" > command1 "$FOO" > > has the same effect as > > command1 "$( command2 )" > > Has anyone pushed the boundaries of this and can tell me whether there > are gotchas? What boundaries were you expecting? If this is related to the thread about the value of $? after command substitution in variable assignment vs. command arguments... that's already a gotcha according to many people. Despite being both documented and logical. Aside for that, obviously in one case you don't have a "FOO" variable in the shell environment, which does seem like a fairly major difference as it's a very common pattern for people to reuse the results of command substitution multiple times. Perhaps you meant to say "executes the same command process" instead of "has the same effect"? -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bash errexit shell option does not work in some cases.
On 6/1/20 6:28 AM, Robert Elz wrote: > Date:Sun, 31 May 2020 22:46:48 -0400 > From: Eli Schwartz > Message-ID: <5a7df0ba-3ad1-1f35-1107-09fdd5950...@archlinux.org> > > While I generally agree with ... > > | Don't use errexit > > but not really with: > > | it does not mean what you think it does. > > (it does mean what I think it means ...) (I think shell developers are exempt from this generalization. :p) > in this case: > > | You're merely the latest person to discover that this applies to you: > | http://mywiki.wooledge.org/BashFAQ/105 > > doesn't really apply - the cases reported are simply bugs in bash, > a shell implementing -e correctly should exit as it was expected to. > > This all results from earlier posix specs where -e and simple commands > were linked - the wording has been improved (still short of perfect) > and simple commands are no longer relevant (other than that exit status > generally originates with the result from a simple command). As I am a poor user, I could not figure out whether the grammar allowed or forbade this this result. So this is actually that unbelievable of unbelievables, a script which is broken because of set -e and it's actually because set -e has a bug? Thanks for clarifying. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bash errexit shell option does not work in some cases.
On 5/31/20 10:32 PM, Hyunho Cho wrote: > > GNU bash, version 5.0.16(1)-release (x86_64-pc-linux-gnu) > Operating System: Ubuntu 20.04 LTS > Kernel: Linux 5.4.0-33-generic > Architecture: x86-64 > > > bash errexit shell option does not work in some cases. > after execution a command, exitcode is set to "1" but shell script does not > exited. You're merely the latest person to discover that this applies to you: http://mywiki.wooledge.org/BashFAQ/105 tl;dr Don't use errexit, it does not mean what you think it does. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: behavior of arithmetic evaluation operator inside extended test operator
On 5/19/20 1:34 PM, Inian Vasanth wrote:
> Thanks Greg for the explanation. Yes your explanation aligns with my
> understanding too.
>
> My recommendation was to check if this behavior needs to be documented as a
> side-note in this section of
> http://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Bash-Conditional-Expressions
> to
> explain that, any other primaries other than the ones mentioned above will
> be evaluated as a literal string result. I also tried finding an
> explanation in your wiki at
> https://mywiki.wooledge.org/BashGuide/TestsAndConditionals, but there
> wasn't an explicit point made.
The constructs:
(list)
{ list; }
[[ expression ]]
(( expression ))
for name [ [ in [ word ... ] ] ; ] do list ; done
for (( expr1 ; expr2 ; expr3 )) ; do list ; done
select name [ in word ] ; do list ; done
case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac
if list; then list; [ elif list; then list; ] ... [ else list; ] fi
while list-1; do list-2; done
until list-1; do list-2; done
are listed as *competing* definitions of "what is a compound command".
I don't see anywhere in the definition of "CONDITIONAL EXPRESSIONS" that
states you may use completely arbitrary compound commands as the
expression within a [[ compound command. It seems to me that
"CONDITIONAL EXPRESSIONS" is pretty unambiguously explicit on what
constitutes one.
Any possible reading which even implies that
(( expression ))
is valid when nested inside of
[[ expression ]]
as
[[ (( expression )) ]]
must also imply that one can do
[[ [[ expression ]] ]]
or [[ for i in one two three; do cmd; done ]]
So I'm completely baffled why this might need further clarification.
As for your claim that
[[ $(( 100 < 3 )) ]]
is doing "undocumented arithmetic evaluation",
> Word splitting and pathname expansion are not performed on the words
> between the [[ and ]]; tilde expansion, parameter and variable
> expansion, arithmetic expansion, command substitution, process
> substitution, and quote removal are performed.
And as duly noted in the documentation, $(( 100 < 3 )) is Arithmetic
Expansion and is being performed on the words inside the [[ and ]].
> explain that, any other primaries other than the ones mentioned above
> will be evaluated as a literal string result
The documentation *already* states that. Nothing is valid in a
conditional expression other than the listed primaries, *but* one of
those documented primaries is `[[ string ]]` which is defined to be a
primary identical to `[[ -n string ]]`. Only strings are valid for this
primary, no exceptions. Other primaries don't even accept strings at
all, they accept filenames; filenames just so happen to be a subcategory
of strings. The same can be said of the -t primary, which accepts a file
descriptor number, which is a subcategory of integers, not strings (you
may then argue that an integer is a subcategory of a string), or various
primaries which operate on variable names, which are subcategories of
strings, etc. etc.
The documentation doesn't need changing. People need to realize when
they read the documentation that a conditional expression happens after
other stages of the shell execution process, and that their
understanding of conditional expressions is incomplete if they don't
understand where they are permitted to use them and how to use tilde
expansion, parameter and variable expansion, arithmetic expansion,
command substitution, process substitution, and quote removal in order
to preprocess words to then use in a conditional expression.
To improve their understanding, they must therefore read the definition
of the [[ ]] syntax. Thus, enlightenment shall be obtained.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
bash-20200417 snapshot breaks examples/loadables/push.c
In the latest devel push:
$ make -C examples/loadables/
make: Entering directory '/home/eschwartz/git/bash/examples/loadables'
gcc -fPIC -DHAVE_CONFIG_H -DSHELL -g -O2 -Wno-parentheses
-Wno-format-security -I. -I.. -I../.. -I../../lib -I../../builtins -I.
-I../../include -I/home/eschwartz/git/bash
-I/home/eschwartz/git/bash/lib -I/home/eschwartz/git/bash/builtins -c
-o push.o push.c
push.c: In function ‘push_builtin’:
push.c:95:17: error: too few arguments to function ‘wait_for’
95 | xstatus = wait_for (pid);
| ^~~~
In file included from push.c:30:
../../jobs.h:283:12: note: declared here
283 | extern int wait_for PARAMS((pid_t, int));
|^~~~
make: *** [Makefile:100: push.o] Error 1
make: Leaving directory '/home/eschwartz/git/bash/examples/loadables'
From the changelog:
+{jobs,nojobs}.c,jobs.h
+ - wait_for: now takes a second argument, a flags word
+
+{execute_cmd,subst}.c
+ - wait_for: change all callers to add second argument to wait_for
Seems like one caller was missed. :)
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: Are there any plans for more readable, modern syntaxes for If statements?
On 3/13/20 7:59 AM, John McKown wrote:
> I do things like:
>
> test && { true_command1;true_command2; : ; } || {
> false_command1; false_command2; false_command3; }
>
> Notice the command : (colon) as the last command in the true. The reason
> that I put it there is to guarantee that the $? from the series of
> "true_commnds" is 0 (aka "true") so that the commands in the || are not
> executed.
OP: "I want readable, modern syntax for If statements, inspired by C."
You: "Don't use If, use &&, which doesn't look like a C If either."
I'm... not following? I mean, yes, the things which you do are valid
bash syntax, and yes, they seem to do what you want them to do, but I
don't *really* understand how they tie in to the premise of the thread.
In contrast, "abuse parser macros" is a valid if terrible answer to the
question of changing the syntax of a bash feature. Something which is
otherwise not really an option, since 40-year-old languages do not
change fundamental syntax on a whim as though they are, I dunno, perl6.
(I am still mindblown that "I want a modern syntax for " can
possibly lead to "imitate C, which is 50 years old and thus astoundingly
even less modern than the Bourne shell by an entire decade".)
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: Are there any plans for more readable, modern syntaxes for If statements?
On 3/12/20 3:00 PM, Vaidas BoQsc wrote:
> Are there any plans to have alternative, a more modern C or D like syntaxes
> for IF statements, While loop and the like in Bash?
>
> Would it be better to just make a new shell and command language similar to
> Bash instead of adding these alternatives?
$ if (true) { echo success; }
bash: syntax error near unexpected token `{'
$ alias {=then }=fi
$ if (true) { echo success; }
success
Is this what you wanted? :p
Personally, I don't really see the problem. I already need to remember
how to do these in multiple languages, and any of them look weird from
the context of another language. Can you describe why you feel the bash
syntax isn't "modern enough"?
What would you propose for backwards compatibility, and what is the
likelihood people will actually use a different syntax to do the same
thing, which only works on very new systems? What about POSIX sh scripts?
This seems like a fairly big proposal for something I'm not even seeing
a definite argument as being actually wrong.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: command_not_found_handle() flaw
On 3/10/20 4:04 PM, Greg Wooledge wrote:
> On Tue, Mar 10, 2020 at 08:57:26PM +0100, Phi Debian wrote:
>> Secondly, if yuio insist on going on the subshell path, then $$ and $PPID
>> ought to be setup appropriately, here ther are plain bogus.
>
> wooledg:~$ command_not_found_handle() { echo "\$\$=$$ BASHPID=$BASHPID"; }
> wooledg:~$ echo "\$\$=$$ BASHPID=$BASHPID"
> $$=859 BASHPID=859
> wooledg:~$ wyeiruwyer
> $$=859 BASHPID=15619
>
> It looks correct to me. $$ is the PID of the main script, and $BASHPID
> is the PID of the subshell.
I think perhaps what the reporter did not realize is that this is the
expected behavior of a subshell.
$ command_not_found_handle; (command_not_found_handle); sddssdfcdsfsf
$$=20819 BASHPID=20819
$$=20819 BASHPID=40810
$$=20819 BASHPID=40811
i.e. $$ doesn't change, whether invoked via command_not_found_handle or
any other form of subshell.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: Different history expansion behaivor: true `# # !xxx` vs. true `# !xxx`
On 3/3/20 2:27 PM, Chet Ramey wrote: > On 3/2/20 9:04 PM, Clark Wang via Bug reports for the GNU Bourne Again > SHell wrote: >> This is from stackoverflow ( https://stackoverflow.com/questions/60166019/ ) >> and it looks like a bug: >> >> $ bash --version >> GNU bash, version 5.0.7(3)-release (x86_64-pc-linux-gnu) >> Copyright (C) 2019 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > >> This is free software; you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. >> $ >> $ set -H >> $ true `# !xxx` >> bash: !xxx`: event not found > > Well, the history comment character (#) is not found at the start of a > word, so the rest of the line is processed for history expansion. > >> $ true `# # !xxx` > > The history comment character is found at the start of a word and history > expansion skips the rest of the line. > > Readline history expansion knows very little about shell syntax; in > particular, it doesn't know backquotes. It never has. And the quick test to show that this is the case... $ set -H $ true `# !xxx` bash: !xxx`: event not found $ true ` # !xxx` $ true `# !!` true `# true ` # !xxx`` $ -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bash 5.0.11 – Output not redirected
On 2/26/20 10:48 AM, Greg Wooledge wrote: > If you're trying to pre-populate the input buffer of a shell in a newly > launched terminal emulator, you're about to go down some very deep > rabbit holes. I suggest you look at tmux or screen. Those may have > some of the features you're looking for, if my guess is on target. A simpler solution if you (rhet.) just want to run a command automatically in gnome-terminal and then wait for more commands, is, instead of starting "bash" (or $SHELL) by default, specify that gnome-terminal should run the command: bash --rcfile <(cat ~/.bashrc; echo "some_commands") That would run "some_commands" as part of the shell initialization, though it wouldn't wait for confirmation or line editing and it wouldn't print what you're running without also doing some explicit echo/printf or set -x. Kind of a confusing way to do things, but it could get the job done. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: bash 5.0.11 – Output not redirected
On 2/26/20 10:39 AM, Ricky Tigg wrote: > Tested on *Gnome*; Component: bash.x86_64 5.0.11 > > Used Shell; > $ echo $SHELL > /bin/bash > > Following command opens the queried graphical terminal emulator; > $ echo q | gnome-terminal > > Result in opened terminal: > [yk@localhost ~]$ > > Redirected output is thus not present as input there. Then maybe you should be reporting a bug to gnome-terminal due to gnome-terminal not acting as you expected and doing something with the input which it has been given. How is this a bash bug? -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: ${!variable@operator} does not work for variables without values; inconsistencies between present and absent [@] for @A and @a
On 2/19/20 7:46 PM, Arfrever Frehtes Taifersar Arahesis wrote:
> Eli Schwartz 2020-02-20 23:49 UTC:
>> Your examples are all (still) broken.
>
> This would affect only 10 examples from 120, so only 8.33 % of
> examples, far from all examples.
[...]
> Majority (3 of 4) of bugs reported by me in this thread are unaffected
> by above discussion about ${!...} and are certainly still valid.
But your examples are founded in a misunderstanding of how bash
variables work, why should I expend the mental bandwidth to unroll your
micro-optimized for loop full of eval to figure out which examples are
valid?
You've immediately jumped from "I don't get the result I expected" to
"there's a bug in bash". I've pointed out that there are gaps in your
understanding of how bash works. Maybe it's time to revisit your initial
assumptions going in, open yourself up to the possibility that you might
be wrong, rather than bash, and finally, please, *please* just elaborate
your test cases in a readable manner without making them conditional on
correct use of eval. Then we could all discuss whether a specific bit of
behavior makes sense.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: ${!variable@operator} does not work for variables without values; inconsistencies between present and absent [@] for @A and @a
On 2/19/20 6:18 PM, Arfrever Frehtes Taifersar Arahesis wrote:
> While described problems exist, there were typos in original reproduction
> steps.
>
> Corrected output also reveals another bug:
> ${!variable[@]@A} for (indexed or associative) arrays does not include
> indexes.
Your examples are all (still) broken. Do not try to reproduce bugs using:
VAR1=(aaa)
Instead, use:
VAR1=(aaa bbb)
Reformulate all test cases where you expect the bbb to be printed, but
it isn't.
$ VAR1=(aaa bbb)
$ ref=VAR1
$ echo "${!ref[@]}"
0
$ echo "${!ref[@]@A}"
declare -a VAR2='aaa'
$ ref2=VAR1[@]
$ args() { printf '<%s> ' "$@"; printf \\n; }
$ args "${!ref2}"
$ args "${!ref2@A}"
<>
You cannot use ${!ref[@]}, you need the array subscript as part of the
set value of "ref" and then indirectly refer to ref.
$ declare -A VAR2=([foo]=bar [baz]="how interesting")
$ args "${!VAR2[@]}"
From man bash:
```
${!name[@]}
${!name[*]}
List of array keys. If name is an array variable, expands to
the list of array indices (keys) assigned in name. If name is not an
array, expands to
0 if name is set and null otherwise. When @ is used and the
expansion appears within double quotes, each key expands to a separate word.
```
And, as predicted, if instead of using ${!ref[@]} you use
${!varname[@]}, you get meaningful information. If you try to look at
the array keys of "ref" itself, all it prints is the number 0, because:
ref=VAR1
and since it is set and non-null...
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: ${variable@A} does not work for associative arrays
On 2/19/20 3:05 PM, Arfrever Frehtes Taifersar Arahesis wrote:
> BASH 5.0.16.
>
> $ VAR1=aaa
> $ declare -a VAR2=(aaa)
> $ declare -A VAR3=([aaa]=aaa)
> $ declare -p VAR{1,2,3}
> declare -- VAR1="aaa"
> declare -a VAR2=([0]="aaa")
> declare -A VAR3=([aaa]="aaa" )
> $ echo "${VAR1@A}"
> VAR1='aaa'
> $ echo "${VAR2@A}"
> declare -a VAR2='aaa'
Well, that looks a bit off.
$ VAR2=(aaa )
$ echo "${VAR2@A}"
declare -a VAR2='aaa'
$ echo "${VAR2[@]@A}"
declare -a VAR2=([0]="aaa" [1]="")
I suggest next time, do your tests with a multi-element indexed array,
since it will make it obvious when your test case is incorrectly
printing only one element.
> $ echo "${VAR3@A}"
>
> $
>
> --
> Arfrever Frehtes Taifersar Arahesis
>
--
Eli Schwartz
Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: possible bug in Bash 5.0
On 12/25/19 5:41 PM, Xin Wu wrote: > Hi, > > I found the single-bracket [ ... ] and double-bracket [[ ... ]] behave > differently for string comparison in the follow simple Bash-script. > > # comma (,) is before two (2) in ASCII > a=,rst > b=2rst > if [ "$a" \> "$b" ]; then > echo "single-bracket" > fi > if [[ "$a" > "$b" ]]; then > echo "double-bracket" > fi > > According to some web pages I was expecting that single-bracket and > double-bracket should give same result. I'm not a Bash-expert, therefore > I'm not sure whether this is a real bug in Bash 5.0. > > Merry Christmas! > > Xin > > PS: the web pages are: > > * http://tldp.org/LDP/abs/html/comparison-ops.html > * http://tldp.org/LDP/abs/html/refcards.html > * http://mywiki.wooledge.org/BashFAQ/031 On the last of these three web pages is the note: "As of bash 4.1, string comparisons using < or > respect the current locale when done in [[, but not in [ or test. In fact, [ and test have never used locale collating order even though past man pages said they did. Bash versions prior to 4.1 do not use locale collating order for [[ either." Try using the C locale, for example: LC_COLLATE=C Then see what the double brackets does. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Unicode range and enumeration support.
On 12/24/19 9:21 AM, Robert Elz wrote: > The -u option appaeared in 7th edition Bell Labs Unix (1979) [I would > assume it was actually implemented somewhat earlier, that's when 7th > edition was released to the rest of the world.] Thanks, the POSIX 2004 manual is just the earliest standard I can easily track down (via pubs.opengroup.org) so I know it's "at least" that old. :D > Also note that sort -u and sort | uniq are not quite the same, the -u > option only considers the key fields when deciding which records (lines) > are unique (of course, with no key options, the whole line is the key, > in which case they are more or less the same). Hmm, is that "more or less" the same, or actually the same? Seems like it would be actually the same... in which case I'd rephrase it to say "sort -u can do things that uniq can't, because it takes an optional key modifier". (uniq does have -s / -f which is sort of kind of partially approaching the same thing, but eh, doesn't really count.) -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Unicode range and enumeration support.
On 12/24/19 12:34 AM, L A Walsh wrote:
> I'm not sure what you want me to say about the range you chose,
> other than it would be about 128,000 characters. It would be about the
> same argument, for or against in using
> {241..128169}. I know you are trying to make some point, but
> I'm missing it.
Yes, quite clearly you are missing lots. Though I cannot help but feel
the problem is when people try to explain it, you ignore the explanations.
>>> It is in unicode code point order. Which is what you would use
>>> for unicode. If you want to sort via unicode, use the -u switch.
>>>
>>
>> That isn't what the sort -u option does, and you know it. I hope.
>>
> Yeah...though, _I don't remember it_, _if_ I'm not entering it in
> a command to use it. I.e just as I'm about to type "|uniq", I
> remember that I'll remember that '-u' is a recently added 'shortcut'
> to get the same output.
What is "recent" about sort -u? I can find it listed as a mandatory
option in the POSIX 2004 manual.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: Unicode range and enumeration support.
On 12/20/19 7:35 PM, L A Walsh wrote:
> On 2019/12/18 11:46, Greg Wooledge wrote:
>> To put it another way: you can write code that determines whether
>> an input character $c matches a glob or regex like [Z-a]. (Maybe.)
>>
>> But, you CANNOT write code to generate all of the characters from Z to a
>>
> This generates characters from decimal 8300 - 8400 (because that range
> includes raised and lowered digits which have the number and value
> properties equivalent to 0-9.
>
>
>
> No? 8300, 8400 arbitrary code points that contain raised and lowered
> numbers
> that have the number property (as does 0..9):
>
> perl -we' use strict; use v5.16;
> my $c;
> for ($c=8300;$c<8400;++$c) {
> my $o=chr $c;
> printf "%s", $o if $o=~/\pN/; #match unicode property "is_num"
> };printf "\n"'
> ⁰⁴⁵⁶⁷⁸⁹₀₁₂₃₄₅₆₇₈₉
>
> Q.E.D.
>
>
> Is that sufficient proof?
It's sufficient proof that you're wrong, yes.
Given the discussion was about collation, not simply enumerating
codepoints in order of their codepoint values, it would be helpful to
actually, you know, collate them.
Given your sample text range:
$ printf %s\\n ⁰ ⁴ ⁵ ⁶ ⁷ ⁸ ⁹ ₀ ₁ ₂ ₃ ₄ ₅ ₆ ₇ ₈ ₉ | sort
⁰
₀
₁
₂
₃
⁴
₄
⁵
₅
⁶
₆
⁷
₇
⁸
₈
⁹
₉
This is plainly not in byte order.
Now you need to ask yourself the question: which locale do you want to
sort according to? I used en_US.UTF-8. Please don't say "C.UTF-8",
because that's not actually a thing. And the plain C locale won't work
for obvious reasons...
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: Unicode range and enumeration support.
On 12/18/19 3:13 PM, Greg Wooledge wrote:
> On Wed, Dec 18, 2019 at 03:08:20PM -0500, Eli Schwartz wrote:
>> So all bash needs to do to print {Z..a} is to take Z == ASCII decimal 90
>> and a == ASCII decimal 97, then enumerate the numbers 90-97 and
>> translate them into ascii. No locale awareness is needed, no heuristics,
>> no invocation of the locale subsystem, you don't even need to hardcode
>> the ASCII range in source code.
>
> Until you want to use bash on an EBCDIC system. ;-)
Oof, that was mean. :p (Also, why does this still exist.)
(But I guess we all realize that this just means bash needs to rely on
the existing support for translating the ASCII locale, and still doesn't
need to enumerate a lookup code of characters for this especial purpose.)
>> And that's why bash can support enumerating a range of ASCII characters
>> in LC_COLLATE=C order, when it cannot (easily) do so using other locales.
>
> Yup.
>
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: Unicode range and enumeration support.
On 12/18/19 2:46 PM, Greg Wooledge wrote:
> Sorting these characters is also possible, once they have been generated.
> This is (I think!) what allows things like [Z-a] to work at all: you
> can check whether $c is >= 'Z' and <= 'a', without knowing what all of
> the characters in between are. But you can't ask "what comes after Z".
>
> wooledg:~$ for ((i=1; i<=200; i++)); do printf -v tmp %04x "$i"; printf -v c
> "\\u$tmp"; if [[ $c = [[:alpha:]] ]]; then printf %s\\n "$c"; fi; done | sort
> | tr -d \\n; echo
> aAªÁÀÂÅÄÃÆbBcCÇdDeEÈfFgGhHiIjJkKlLmMnNoOºpPqQrRsStTuUvVwWxXyYzZµ
>
> Again, this is only PART of the set, and is not intended to be a
> complete enumeration of the :alpha: characters in my system's locale.
There's no need to sort ASCII characters, though, since the collation
order of [A-z] in the C locale is defined by their numeric codepoint
order. That is a guarantee that doesn't follow through in other locales.
So all bash needs to do to print {Z..a} is to take Z == ASCII decimal 90
and a == ASCII decimal 97, then enumerate the numbers 90-97 and
translate them into ascii. No locale awareness is needed, no heuristics,
no invocation of the locale subsystem, you don't even need to hardcode
the ASCII range in source code.
And that's why bash can support enumerating a range of ASCII characters
in LC_COLLATE=C order, when it cannot (easily) do so using other locales.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: unquoted expansion not working (was Re: Not missing, but very hard to see)
On 12/14/19 5:48 AM, L A Walsh wrote: >> If on the other hand Chet decides that the man page's wording is not >> accurate, then the issue about whether to quote punctuation characters >> becomes irrelevant. The generation of punctuation characters is an >> unexpected consequence, and the answer is Don't Do That. >> > - > No -- you don't go backwards -- it could break existing scripts that > use that That's a ridiculous argument, essentially saying bash must retain bug-level compatibility with every previous version of every behavior. https://xkcd.com/1172/ But it's also beside the point, because I don't actually see anyone making the argument you're opposing. The proposal is to update the man page to declare the current behavior as "expected", and possibly include a note "abusing brace expansion to produce characters other than ASCII letters is fragile and results in non-intuitive behavior. Users are advised to not do it". The alternative, changing bash's behavior to make the output of my strings which happen to resemble brace expansion, suddenly produce an array of expanded characters, is "breaking my script what are you doing even stop making bash behave differently from version to version why don't you even comprehend backwards compatibility ARG". See, I can play the "people do unexpected things, don't break backwards compatibility" absurdity game too. > for one, but two -- are you going to limit it only to every > range of letters and numbers in unicode? It seems allowing it to work > generally would be easier than trying to only have it work on the multiple > ranges of unicode -- which I'd point out is consistent with bash growing to > be able to use unicode (when it used to not be so), and most features > working with unicode. So this is actually not an unreasonable suggestion, I guess. Although I can't really imagine a practical use for it, whereas generally with other "works with unicode data" features, I can. > May not happen overnight, but a fix to quote meta-chars and allow all > characters, including, _eventually_ 'null' embedded in strings, since that > does happen in the MS world and in C++ where counts instead of zstrings are > more frequently found, but I won't be holding my breath for that result. Wait what. Are you actually serious? Because this looks like trolling. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: Not missing, but very hard to see (was Re: Backslash missing in brace expansion)
On 12/12/19 9:57 PM, L A Walsh wrote:
>
>
> On 2019/12/12 13:01, Ilkka Virta wrote:
>> On 12.12. 21:43, L A Walsh wrote:
>>
>>> On 2019/12/06 14:14, Chet Ramey wrote:
>>>
>>> Seems very hard to print out that backquote though. Closest I got
>>> was bash converting it to "''":
>>>
>>
>> The backquote is in [6], and the backslash disappears, you just get
>> the pair of quotes in [2] because that's how printf %q outputs an
>> empty string.
>>
> -
>
> I'm sorry, but you are mistaken.
How so?
> The characters from 'Z' (0x5A) through 'z' (0x61) are:
>
> 0x5A 0x5B 0x5C 0x5D 0x5E 0x5F 0x60 0x61
> Z [ \ ] ^ _ ` a
>
> the backslash comes between the two square brackets.
>
> Position [6] is the "Grave Accent" (or backquote).
>
> It is quoted properly.
But... that's exactly what was said.
> As for %q printing an empty string for 0x5C
>
> "%q" causes printf to output the corresponding argument in a
> format that can be reused as shell input.
>
> For that string to be empty would mean there is no character at hex
> value 0x5C (unicode U+005C), which isn't so.
But there isn't. An empty string was passed as an argument to printf,
because the backslash was *converted* via escaping, into an empty
string, *before* it was passed on the command line as an argv element to
the printf builtin.
Do you think that because printf is a builtin, and you didn't use
/bin/printf, that somehow means it is exempt from the usual rule of how
shells work, and gets to see its own argv before the parser reinterprets it?
>>
>>>> read -r -a a< <(printf "%q " {Z..a})
>>>> my -p a
>>>>
>>> declare -a a=([0]="Z" [1]="\\[" [2]="''" [3]="\\]" [4]="\\^" [5]="_"
>>> [6]="\\\`" [7]="a")
>>>
>>
>>
>>
--
Eli Schwartz
Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: make install failed; dump core in mkdir
On 12/2/19 9:56 AM, Greg Wooledge wrote: > On Mon, Dec 02, 2019 at 09:51:12PM +0700, pepa65 wrote: >> On 12/2/19 9:38 PM, Andreas Kusalananda Kähäri wrote:> for dirpath do >>> command mkdir -p "$dirpath" || return >>> done >> It is very sad that the 'do' is not optional (non-backwards compatibility >> breaking feature request!), and it has to be the start of the next >> commandline (after ';' or the start of a new line), so: >> >> for dirpath >> do command mkdir -p "$dirpath" || return >> done > > You're mistaken. > > wooledg:~$ set -- a b c > wooledg:~$ for i do echo "$i"; done > a > b > c Interesting, I didn't expect that to work. It doesn't work if you use 'in', which I normally do. $ for i in "$@" do echo "$i"; done bash: syntax error near unexpected token `done' I guess that makes sense -- the 'in' will try to treat the words "do" "echo" "$i" as iteration elements, but without 'in', there can only be one argument to 'for', so bash can safely parse 'do' without the ; separator. But I think I'll still use them despite being not strictly needed, or I'll just confuse myself once I do need them. :p I've used them for too long by now, anyway. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: man bash does not list 'in' as a builtin command
On 11/25/19 4:53 PM, Greg Wooledge wrote:
> On Mon, Nov 25, 2019 at 01:43:41PM -0800, Peter Benjamin wrote:
>> Description:
>> 'in' is a builtin command and is not listed in the man page as such.
>
> It's actually a keyword. It's part of the "for" and "case" syntax.
>
> wooledg:~$ type in
> in is a shell keyword
>
> for NAME in WORDS; do ...; done
>
> case WORD in PATTERN) ... ;; esac
And, it's even listed in the manpage, if you have patience to go through
every mention of the pattern '\bin\b' and filter out the plain English
ones...
RESERVED WORDS
Reserved words are words that have a special meaning to the
shell. The following words are recognized as reserved when unquoted and
either the first word of
a simple command (see SHELL GRAMMAR below) or the third word of a
case or for command:
! case coproc do done elif else esac fi for function if in
select then until while { } time [[ ]]
So the OP has their wish. It is documented in a section of the manpage
they did not think to check.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: set -e ignored in subshell if part of command list
On 11/14/19 6:02 PM, Ángel wrote:
> I would say that the confusing part is that the behavior of the subshell
> is dependant on *where* it is being executed in the parent.
>
> In general terms, I would expect
> ( )
> to be roughly equivalent to
> bash -c ""
>
> i.e. being executed on its own context and unable to affect the
> parent
But that's totally wrong for *numerous* reasons.
> but the provided case shows that
> ( set -e; false; echo here ) && echo bar
> behaves differently than
> bash -c "set -e; false; echo here" && echo there
What about
var=value; (echo "$var")
which behaves differently from
var=value; bash -c 'echo "$var"'
or,
func() { echo "hi there"; }; (func)
which behaves differently from
func() { echo "hi there"; }; bash -c 'func'
> since the initial subshell has an advanced knowledge that there will be
> a later command joined by an and.
An initial subshell has advanced knowledge about lots of things from the
parent shell, it is deeply, worryingly dangerous to write code that
doesn't take this into account. Things will tend to go pretty badly wrong.
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
signature.asc
Description: OpenPGP digital signature
Re: set -e ignored in subshell if part of command list
On 11/13/19 10:59 AM, Shaun Crampton wrote: > But the commands in the subshell execute inside a different shell > execution context so they shouldn't have > their own set -e context (Section 2.12)? > > I don't see where the spec says that the subshell has to inherit the > and/or list-ness of the > parent context. Section 2.12 doesn't mention that as being one of the > things that a subshell inherits > (and unless I'm missing a good use-case, it seems like a pretty > useless thing to inherit in a subshell > or a function that happens to be called on the LHS of an and/or). You're trying to defend the behavior you *want* set -e to have, even though many people rely on set -e having the behavior it does, indeed, have. Why do you require that yet another behavior change be made, so that it is even harder to know what set -e will do across various shells, and across various *versions* of a shell? set -e already has more than enough exceptions to every single rule it ever had. It's a collection of inchoate nonsensical lexical rules. Don't use it, for all the reasons enumerated here: https://mywiki.wooledge.org/BashFAQ/105 Everyone will be much happier if they just don't use set -e, and, instead, implement actual real error handling that a) works, b) does what they expect in their own special snowflake case. -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
