Re: REQUEST - bash floating point math support
On Fri, Jun 21, 2024, 10:21 PM Chet Ramey wrote: > On 6/21/24 3:59 PM, alex xmb sw ratchev wrote: > > > > If floating point math support is added to bash, I would expect > it to > > > be able to handle floating point literals in these forms as well. > > > > I'm not planning to do this any time soon. > > > > > > sorry my forgetness ... why ? > > Because if floating point math support gets added to bash, someone has to > do the work. It's not a priority for me right now, and no one else is > offering. > ah okk , small fair point all k , thxx keep up the good work .. greets .. -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ > >
Re: REQUEST - bash floating point math support
Anybody else with the knowledge to tackle this? I am not capable of even writing C code correctly. On Fri, Jun 21, 2024 at 4:22 PM Chet Ramey wrote: > > On 6/21/24 3:59 PM, alex xmb sw ratchev wrote: > > > > If floating point math support is added to bash, I would expect it to > > > be able to handle floating point literals in these forms as well. > > > > I'm not planning to do this any time soon. > > > > > > sorry my forgetness ... why ? > > Because if floating point math support gets added to bash, someone has to > do the work. It's not a priority for me right now, and no one else is > offering. > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ >
Re: REQUEST - bash floating point math support
On 6/21/24 3:59 PM, alex xmb sw ratchev wrote: > If floating point math support is added to bash, I would expect it to > be able to handle floating point literals in these forms as well. I'm not planning to do this any time soon. sorry my forgetness ... why ? Because if floating point math support gets added to bash, someone has to do the work. It's not a priority for me right now, and no one else is offering. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: REQUEST - bash floating point math support
On 6/15/24 9:29 AM, Koichi Murase wrote:
I still feel it would be best if POSIX could be updated to allow the
implementations to extend the interpretation when the conversion by
strtod(3) fails. The current restriction is not an explicit one but
something deduced from the four statements of the POSIX:
1) the floating-point conversion (%f, etc) of the printf utility is
performed by strtod(3),
2) strtod(3) uses LC_NUMERIC to determine the radix character,
3) strtod(3) "fails" by setting endptr = nptr when the string does not
have an expected form,
4) the printf utility needs to print a diagnostic message and fail
when the argument is not converted to a value appropriate to the
conversion specification (%f, etc.).
I doubt that the current restriction on the printf utility not
allowing C floating-point literals is intentional.
I'm pretty sure it is. If you assume that all string-to-float conversions
are performed using strtod/strtof/strtold - the standard provides no other
way to do that - then the explicit mention of strtod prevails, for
consistency's sake if no other. The standard would not refer to strtod
explicitly if it did not want to inherit strtod's restriction to a locale-
specific radix character. In other places where the standard relaxes such
restrictions, it does so explicitly ("if the implementation wishes to allow
additional formats, blah, blah").
Now, you might be able to file an interpretation request and get the group
to (grudgingly) allow `.' and the locale's radix character to be used
interchangeably.
Is there a reason
that we should not interpret C floating-point literals (except the
passive reason that a non-trivial combination of POSIX statements does
not allow it)?
I think it's pretty explicit.
I feel it is better to allow the printf implementations
to extend the conversion rather than trying to invent strange and
inconsistent behaviors among the arithmetic expressions, the
arithmetic expansions, and the parameter expansions just to work
around the problem.
I might be able to do that as a settable shopt option that gets turned
off in posix mode, in a later version.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: REQUEST - bash floating point math support
On Fri, Jun 21, 2024, 9:58 PM Chet Ramey wrote:
> On 6/17/24 8:04 AM, Zachary Santer wrote:
> > On Mon, Jun 17, 2024 at 3:48 AM Léa Gris wrote:
> >>
> >> Le 17/06/2024 à 09:17, Koichi Murase écrivait :
> >>> declare -i numvar=${localeFormatted/[!0-9]/.}
> >>
> >> This would break with negative numbers.
> >>
> >> I know no other radix separator than comma or dot. If there are other
> >> radix to replace, it can be listed in a character class.
> >>
> >> Lets say there are locales that uses , ; or :
> >>
> >> declare -i numvar=${localeFormatted/[,;:]/.}
> >
> > Do different locales use different characters in the place of 'e' in
> > "[-]d.ddde±dd" and 'p' in "[-]0xh.p±d"? (These pulled out of 'man
> > 3p fprintf' - the 'e' and 'a' conversion specifiers.)
>
> No. The only locale-specific portion of a floating-point number is the
> radix character.
>
>
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/strtod.html#tag_16_587
>
> It's specified for both input and output (printf).
>
> >
> > If floating point math support is added to bash, I would expect it to
> > be able to handle floating point literals in these forms as well.
>
> I'm not planning to do this any time soon.
>
sorry my forgetness ... why ?
--
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
>
Re: REQUEST - bash floating point math support
On 6/17/24 8:04 AM, Zachary Santer wrote:
On Mon, Jun 17, 2024 at 3:48 AM Léa Gris wrote:
Le 17/06/2024 à 09:17, Koichi Murase écrivait :
declare -i numvar=${localeFormatted/[!0-9]/.}
This would break with negative numbers.
I know no other radix separator than comma or dot. If there are other
radix to replace, it can be listed in a character class.
Lets say there are locales that uses , ; or :
declare -i numvar=${localeFormatted/[,;:]/.}
Do different locales use different characters in the place of 'e' in
"[-]d.ddde±dd" and 'p' in "[-]0xh.p±d"? (These pulled out of 'man
3p fprintf' - the 'e' and 'a' conversion specifiers.)
No. The only locale-specific portion of a floating-point number is the
radix character.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/strtod.html#tag_16_587
It's specified for both input and output (printf).
If floating point math support is added to bash, I would expect it to
be able to handle floating point literals in these forms as well.
I'm not planning to do this any time soon.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: REQUEST - bash floating point math support
On Mon, Jun 17, 2024 at 3:48 AM Léa Gris wrote:
>
> Le 17/06/2024 à 09:17, Koichi Murase écrivait :
> >declare -i numvar=${localeFormatted/[!0-9]/.}
>
> This would break with negative numbers.
>
> I know no other radix separator than comma or dot. If there are other
> radix to replace, it can be listed in a character class.
>
> Lets say there are locales that uses , ; or :
>
> declare -i numvar=${localeFormatted/[,;:]/.}
Do different locales use different characters in the place of 'e' in
"[-]d.ddde±dd" and 'p' in "[-]0xh.p±d"? (These pulled out of 'man
3p fprintf' - the 'e' and 'a' conversion specifiers.)
If floating point math support is added to bash, I would expect it to
be able to handle floating point literals in these forms as well. I'm
assuming that floating point literals can be specified in C source
code in these forms. Another reason why this is more work than it
looks like from the surface.
Re: REQUEST - bash floating point math support
Le 17/06/2024 à 09:17, Koichi Murase écrivait :
declare -i numvar=${localeFormatted/[!0-9]/.}
This would break with negative numbers.
I know no other radix separator than comma or dot. If there are other
radix to replace, it can be listed in a character class.
Lets say there are locales that uses , ; or :
declare -i numvar=${localeFormatted/[,;:]/.}
In my opinion, the locale-formatted strings should only appear in the
strings presented to users, such as in the output of `printf'. It is
easier and maintainable to normalize all the internal representations
to be the C floating-point literals.
I have been in IT in France since 3 decades. And I have never ever found
any piece of code that use locale-formatted literals, not even during my
CS education.
Sometimes one have to deal with data sources with numbers formatted in
funny ways, even grouping with illegal characters. Then it is a know
fact that one must deal with sourced data with zealous sanitizing.
Now on the bash shell context. The usual pitfall with floating-point
number parsing comes when some script author tries to parse output of a
command that is not meant to be parsed, that is locale formatted and
don't bother to set LC_NUMERIC=C before running this command
A common pitfall is trying to parse the output of the time builtin command.
It already breaks on locale ignorant scripts. This won't change.
--
Léa Gris
Re: REQUEST - bash floating point math support
2024年6月17日(月) 15:56 Léa Gris :
> Le 16/06/2024 à 23:44, Zachary Santer écrivait :
> Anyway, this could be handled with existing bash string replacement feature:
>
> localeFormatted=3,1415
> declare -i numvar=${localeFormatted/,/.}
Is the radix character in an arbitrary locale ensured to be either `.'
or `,'? I think we should in general need to do
declare -i numvar=${localeFormatted/[!0-9]/.}
Anyway, this is non-trivial for users. I anticipate that people in
typical locales won't care about it, so many scripts would do
`numvar=$localeFormatted` and are tested under en_US.UTF-8 or locales
with `.' being the radix character Then, those scripts are later
broken under random locales.
In my opinion, the locale-formatted strings should only appear in the
strings presented to users, such as in the output of `printf'. It is
easier and maintainable to normalize all the internal representations
to be the C floating-point literals.
--
Koichi
Re: REQUEST - bash floating point math support
Le 16/06/2024 à 23:44, Zachary Santer écrivait :
How do you propose to take an LC_NUMERIC-formatted floating-point
literal and assign it to a variable with the numeric flag or make use
of it in another type of arithmetic context?
This proposal does not include conversion of locale formatted literals.
Anyway, this could be handled with existing bash string replacement feature:
localeFormatted=3,1415
declare -i numvar=${localeFormatted/,/.}
--
Léa Gris
Re: REQUEST - bash floating point math support
On Sat, Jun 15, 2024 at 10:56 AM Léa Gris wrote: > > On 15/06/2024 à 15:29, Koichi Murase wrote : > > at which point does the conversion happens > > The conversion to LC_NUMERIC format only happens during variable > expansion outside of a numerical context. > The numerical context can be explicit if the assigned variable has a > numeric flag; it is implicit within ((numeric context)) > > Tgere are other numerical context in Bash, like string and array > indexes, but those may retain only integer support. > > > So this means that the arithmetic expansions $(()) would produce a > > format incompatible with the arithmetic expression. This seems > > inconsistent to me, but it might be a valid option if there is no > > other option. > > The numerical context with a $ prefix $(()) indeed implies a string > conversion after resolving the computation within. So yes it would > perform the string conversion using th LC_NUMERIC format. > > An other change is that witin a declare statement or if the assigned > variable has a numeric flag it is considered a numerical context. > > # Numerical context of the assigned pi variable allow use of > # C format of the 3.1415 literal to be used in assignment > declare -i pi=3.1415 > > # give a a numeric flag > typeset -i e > # Because a has a numeric flag it accepts the dot radix delimiter within > # the assigned value > e=2.71828 > > # Numerical context pi / 2 computes to 1.57 > # but as it is expanded into a string, it uses > # the comma radix delimiter from LC_NUMERIC=fr_FR.UTF8 > str=$((pi / 2)) > # Then str=1,57 > > # This is illegal, becahse the right-side is expanded > # to string and may not use the correct dot delimiter > declare -i arc=$(((2 * pi) / 3) > > # This is a correct assignemnt > ((arc = (2 * pi) / 3)) > > # Although arc has not been declared with a numeric flag > # it was assigned within a numerical context > > > > I feel it is better to allow the printf implementations > > to extend the conversion rather than trying to invent strange and > > inconsistent behaviors among the arithmetic expressions > > I agree this printf limitation is questionable. > > Anyway, given the slow pace of standard evolution and to not break the > standard. Having consideration for the context when manipulation values > or expanding these values to a string can greatly help workaround this > POSIX C restriction. > > It is really a simple rule: > > 1. Within a numerical context, floating-point literals use the dot radix > delimiter. Internally it could yse whatever fit best like IEEE 754 > double precision binary64 > 2. Outside a numerical context, when creating a string representation of > a floating-point number, the LC_NUMERIC format applies. How do you propose to take an LC_NUMERIC-formatted floating-point literal and assign it to a variable with the numeric flag or make use of it in another type of arithmetic context?
Re: REQUEST - bash floating point math support
On Sat, Jun 15, 2024, 4:56 PM Léa Gris wrote: > On 15/06/2024 à 15:29, Koichi Murase wrote : > > at which point does the conversion happens > > The conversion to LC_NUMERIC format only happens during variable > expansion outside of a numerical context. > The numerical context can be explicit if the assigned variable has a > numeric flag; it is implicit within ((numeric context)) > > Tgere are other numerical context in Bash, like string and array > indexes, but those may retain only integer support. > > > So this means that the arithmetic expansions $(()) would produce a > > format incompatible with the arithmetic expression. This seems > > inconsistent to me, but it might be a valid option if there is no > > other option. > > The numerical context with a $ prefix $(()) indeed implies a string > conversion after resolving the computation within. So yes it would > perform the string conversion using th LC_NUMERIC format. > > An other change is that witin a declare statement or if the assigned > variable has a numeric flag it is considered a numerical context. > > # Numerical context of the assigned pi variable allow use of > # C format of the 3.1415 literal to be used in assignment > declare -i pi=3.1415 > > # give a a numeric flag > typeset -i e > # Because a has a numeric flag it accepts the dot radix delimiter within > # the assigned value > e=2.71828 > > # Numerical context pi / 2 computes to 1.57 > # but as it is expanded into a string, it uses > # the comma radix delimiter from LC_NUMERIC=fr_FR.UTF8 > str=$((pi / 2)) > # Then str=1,57 > > # This is illegal, becahse the right-side is expanded > # to string and may not use the correct dot delimiter > declare -i arc=$(((2 * pi) / 3) > ^ missing a ) # This is a correct assignemnt > ((arc = (2 * pi) / 3)) > > # Although arc has not been declared with a numeric flag > # it was assigned within a numerical context > > > > I feel it is better to allow the printf implementations > > to extend the conversion rather than trying to invent strange and > > inconsistent behaviors among the arithmetic expressions > > I agree this printf limitation is questionable. > > Anyway, given the slow pace of standard evolution and to not break the > standard. Having consideration for the context when manipulation values > or expanding these values to a string can greatly help workaround this > POSIX C restriction. > > It is really a simple rule: > > 1. Within a numerical context, floating-point literals use the dot radix > delimiter. Internally it could yse whatever fit best like IEEE 754 > double precision binary64 > 2. Outside a numerical context, when creating a string representation of > a floating-point number, the LC_NUMERIC format applies. > > -- > Léa Gris > >
Re: REQUEST - bash floating point math support
On 15/06/2024 à 15:29, Koichi Murase wrote : at which point does the conversion happens The conversion to LC_NUMERIC format only happens during variable expansion outside of a numerical context. The numerical context can be explicit if the assigned variable has a numeric flag; it is implicit within ((numeric context)) Tgere are other numerical context in Bash, like string and array indexes, but those may retain only integer support. So this means that the arithmetic expansions $(()) would produce a format incompatible with the arithmetic expression. This seems inconsistent to me, but it might be a valid option if there is no other option. The numerical context with a $ prefix $(()) indeed implies a string conversion after resolving the computation within. So yes it would perform the string conversion using th LC_NUMERIC format. An other change is that witin a declare statement or if the assigned variable has a numeric flag it is considered a numerical context. # Numerical context of the assigned pi variable allow use of # C format of the 3.1415 literal to be used in assignment declare -i pi=3.1415 # give a a numeric flag typeset -i e # Because a has a numeric flag it accepts the dot radix delimiter within # the assigned value e=2.71828 # Numerical context pi / 2 computes to 1.57 # but as it is expanded into a string, it uses # the comma radix delimiter from LC_NUMERIC=fr_FR.UTF8 str=$((pi / 2)) # Then str=1,57 # This is illegal, becahse the right-side is expanded # to string and may not use the correct dot delimiter declare -i arc=$(((2 * pi) / 3) # This is a correct assignemnt ((arc = (2 * pi) / 3)) # Although arc has not been declared with a numeric flag # it was assigned within a numerical context I feel it is better to allow the printf implementations to extend the conversion rather than trying to invent strange and inconsistent behaviors among the arithmetic expressions I agree this printf limitation is questionable. Anyway, given the slow pace of standard evolution and to not break the standard. Having consideration for the context when manipulation values or expanding these values to a string can greatly help workaround this POSIX C restriction. It is really a simple rule: 1. Within a numerical context, floating-point literals use the dot radix delimiter. Internally it could yse whatever fit best like IEEE 754 double precision binary64 2. Outside a numerical context, when creating a string representation of a floating-point number, the LC_NUMERIC format applies. -- Léa Gris
Re: REQUEST - bash floating point math support
2024年6月15日(土) 17:24 Léa Gris : > Indeed printf only knows the string value. But Bash know the variable > has a numeric flag when doing the value expansion, so it expands it > using the current LC_NUMERIC locale in this specific case. Ah, OK. I misunderstood your suggestion somehow differently (such that the expansion result would still use the period as a radix character, but the `print' builtin would automatically convert the format to the one with the locale-dependent radix character before it starts to interpret the arguments). > # Numeric format use comma as radix separator > LC_NUMERIC=fr_FR.UTF8 > > # Numeric variable uses the C locale format internally > declare -i floatVar=3.1415 > > # Printf does not have to know about the variable type > # as it is expanded in the current LC_NUMERIC format > printf '%.2f\n' "$floatVar" > # Prints 3,14 In this case, what is actually stored in the variable floatVar? `3.14' or `3,14' (probably the former if I do not misunderstand)? In other words, at which point does the conversion happens, in the assignment phase or the parameter expansion phase? If the conversion happens in the assignment phase, `floatVar' would not be able to use in arithmetic expressions, which seems unreasonable. If the conversion happens in the parameter expansion phase, it is different from the traditional behavior of parameter expansions. > # Same when it is expanded to assign another variable > # stringVar=3,1415 > # Since the string expansion of numeric variable uses the > # LC_NUMERIC format > stringVar=$floatVar > > # Same rule applies when expanding a numerical context to a string: > echo "$((2 * 3.14))" > # Prints the string 6,28 because LC_NUMERIC=fr_FR.UTF8 > > # Do not use string expansion to assign numeric value or it will fail > # with different locale. Use a numeric context to assign. > # Do: > (( arc = 2 * floatVar )) > # Don't: > # arc=$((2 * floatVar )) So this means that the arithmetic expansions $(()) would produce a format incompatible with the arithmetic expression. This seems inconsistent to me, but it might be a valid option if there is no other option. I still feel it would be best if POSIX could be updated to allow the implementations to extend the interpretation when the conversion by strtod(3) fails. The current restriction is not an explicit one but something deduced from the four statements of the POSIX: 1) the floating-point conversion (%f, etc) of the printf utility is performed by strtod(3), 2) strtod(3) uses LC_NUMERIC to determine the radix character, 3) strtod(3) "fails" by setting endptr = nptr when the string does not have an expected form, 4) the printf utility needs to print a diagnostic message and fail when the argument is not converted to a value appropriate to the conversion specification (%f, etc.). I doubt that the current restriction on the printf utility not allowing C floating-point literals is intentional. Is there a reason that we should not interpret C floating-point literals (except the passive reason that a non-trivial combination of POSIX statements does not allow it)? I feel it is better to allow the printf implementations to extend the conversion rather than trying to invent strange and inconsistent behaviors among the arithmetic expressions, the arithmetic expansions, and the parameter expansions just to work around the problem. -- Koichi
Re: REQUEST - bash floating point math support
Le 15/06/2024 à 02:49, Koichi Murase écrivait : 2024年6月14日(金) 16:18 Léa Gris : Another elegant option would be to expand the existing variables' i flag to tell the variable is numeric rather than integer. Then have printf handle argument variables with the numeric flag as using the LC_NUMERIC=C floating-point format with dot radix point. In that design, how does the `printf' builtin know that the argument string is generated by a specific variable with the numeric flag? The `printf' builtin is a (builtin) command that receives arguments as strings. The arguments do not carry the contexts of how the strings are generated, (though some of the exceptions are the array assignments of the assignment builtins). Indeed printf only knows the string value. But Bash know the variable has a numeric flag when doing the value expansion, so it expands it using the current LC_NUMERIC locale in this specific case. # Numeric format use comma as radix separator LC_NUMERIC=fr_FR.UTF8 # Numeric variable uses the C locale format internally declare -i floatVar=3.1415 # Printf does not have to know about the variable type # as it is expanded in the current LC_NUMERIC format printf '%.2f\n' "$floatVar" # Prints 3,14 # Same when it is expanded to assign another variable # stringVar=3,1415 # Since the string expansion of numeric variable uses the # LC_NUMERIC format stringVar=$floatVar # Same rule applies when expanding a numerical context to a string: echo "$((2 * 3.14))" # Prints the string 6,28 because LC_NUMERIC=fr_FR.UTF8 # Do not use string expansion to assign numeric value or it will fail # with different locale. Use a numeric context to assign. # Do: (( arc = 2 * floatVar )) # Don't: # arc=$((2 * floatVar )) # Note that the numeric flag is not needed within a numeric context echo "$arc" # Prints 6.283 because arc has no numeric flag # so it is expanded verbatin to string echo "$((arc))" # Prints 6,283 because the implied numerical context # expands the value using LC_NUMERIC format -- Léa Gris
Re: REQUEST - bash floating point math support
2024年6月14日(金) 16:18 Léa Gris : > Another elegant option would be to expand the existing variables' i flag > to tell the variable is numeric rather than integer. > > Then have printf handle argument variables with the numeric flag as > using the LC_NUMERIC=C floating-point format with dot radix point. In that design, how does the `printf' builtin know that the argument string is generated by a specific variable with the numeric flag? The `printf' builtin is a (builtin) command that receives arguments as strings. The arguments do not carry the contexts of how the strings are generated, (though some of the exceptions are the array assignments of the assignment builtins). -- Koichi
Re: REQUEST - bash floating point math support
Great idea.
On Fri, Jun 14, 2024 at 3:18 AM Léa Gris wrote:
>
> Le 14/06/2024 à 03:41, Martin D Kealey écrivait :
> > On Thu, 13 Jun 2024 at 09:05, Zachary Santer wrote:
> >
> >>
> >> Let's say, if var is in the form of a C floating-point literal,
> >> ${var@F} would expand it to the locale-dependent formatted number, for
> >> use as an argument to printf or for output directly. And then ${var@f}
> >> would go the other way, taking var that's in the form of a
> >> locale-dependent formatted number, and expanding it to a C
> >> floating-point literal.
> >>
> >
> > How about incorporating the % printf formatter directly, like ${var@%f} for
> > the locale-independent format and %{var@%#f} for the locale-specific format?
> >
> > However any formatting done as part of the expansion assumes that the
> > variable holds a "number" in some fixed format, rather than a localized
> > string.
> > Personally I think this would actually be a good idea, but it would be
> > quite a lot bigger project than simply added FP support.
> >
> > -Martin
>
> Another elegant option would be to expand the existing variables' i flag
> to tell the variable is numeric rather than integer.
>
> Then have printf handle argument variables with the numeric flag as
> using the LC_NUMERIC=C floating-point format with dot radix point.
>
> Expanding the existing i flag would also ensure numerical expressions
> would handle the same value format.
>
> The David method: Take down multiple issues with one stone.
>
>
>
> --
> Léa Gris
>
Re: REQUEST - bash floating point math support
Le 14/06/2024 à 03:41, Martin D Kealey écrivait :
On Thu, 13 Jun 2024 at 09:05, Zachary Santer wrote:
Let's say, if var is in the form of a C floating-point literal,
${var@F} would expand it to the locale-dependent formatted number, for
use as an argument to printf or for output directly. And then ${var@f}
would go the other way, taking var that's in the form of a
locale-dependent formatted number, and expanding it to a C
floating-point literal.
How about incorporating the % printf formatter directly, like ${var@%f} for
the locale-independent format and %{var@%#f} for the locale-specific format?
However any formatting done as part of the expansion assumes that the
variable holds a "number" in some fixed format, rather than a localized
string.
Personally I think this would actually be a good idea, but it would be
quite a lot bigger project than simply added FP support.
-Martin
Another elegant option would be to expand the existing variables' i flag
to tell the variable is numeric rather than integer.
Then have printf handle argument variables with the numeric flag as
using the LC_NUMERIC=C floating-point format with dot radix point.
Expanding the existing i flag would also ensure numerical expressions
would handle the same value format.
The David method: Take down multiple issues with one stone.
--
Léa Gris
Re: REQUEST - bash floating point math support
On Thu, 13 Jun 2024 at 09:05, Zachary Santer wrote:
>
> Let's say, if var is in the form of a C floating-point literal,
> ${var@F} would expand it to the locale-dependent formatted number, for
> use as an argument to printf or for output directly. And then ${var@f}
> would go the other way, taking var that's in the form of a
> locale-dependent formatted number, and expanding it to a C
> floating-point literal.
>
How about incorporating the % printf formatter directly, like ${var@%f} for
the locale-independent format and %{var@%#f} for the locale-specific format?
However any formatting done as part of the expansion assumes that the
variable holds a "number" in some fixed format, rather than a localized
string.
Personally I think this would actually be a good idea, but it would be
quite a lot bigger project than simply added FP support.
-Martin
Re: REQUEST - bash floating point math support
I think that we should go ahead and do it.
On Wed, Jun 12, 2024, 5:06 PM Zachary Santer wrote:
> On Thu, Jun 6, 2024 at 6:34 AM Léa Gris wrote:
> >
> > Le 06/06/2024 à 11:55, Koichi Murase écrivait :
> >
> > > Though, I see your point. It is inconvenient that we cannot pass the
> > > results of arithmetic evaluations to the `printf' builtin. This
> > > appears to be an issue of the printf builtin. I think the `printf'
> > > builtin should be extended to interpret both forms of the numbers, the
> > > locale-dependent formatted number and the floating-point literals.
> >
> > Another way would be to expand string representation of floating-point
> > numbers using the locale.
>
> Parameter transformations could be implemented to serve this purpose.
> Let's say, if var is in the form of a C floating-point literal,
> ${var@F} would expand it to the locale-dependent formatted number, for
> use as an argument to printf or for output directly. And then ${var@f}
> would go the other way, taking var that's in the form of a
> locale-dependent formatted number, and expanding it to a C
> floating-point literal.
>
> In this scenario, any floating point numbers ingested by the bash
> script would be transformed to C floating-point literals before they
> are referenced or manipulated in an arithmetic context, and then they
> would be transformed back into the locale-dependent formatted number
> before being output from the script.
>
> Arithmetic evaluation is performed on the right-hand side of an
> assignment statement assigning to a variable declared with the -i
> integer attribute. A hypothetical -E floating-point variable attribute
> in bash should work the same way. As such, a variable with the -E
> attribute would have to be assigned a value in a format accepted by
> the arithmetic context, i.e. the C floating-point literal format.
>
> So you'd have to do
> declare -E var="${arg@f}"
> in order to safely handle the locale-dependent formatted number arg,
> an argument to the script.
>
> Any floating-point literals within the script would have to be
> formatted as C floating-point literals, at least if they're going to
> be assigned the -E attribute or used in an arithmetic context.
>
> And then you would have to do
> printf '%f' "${var@F}"
> to safely output the value using printf.
>
> Is that unreasonable?
>
> Zack
>
>
Re: REQUEST - bash floating point math support
On Thu, Jun 6, 2024 at 6:34 AM Léa Gris wrote:
>
> Le 06/06/2024 à 11:55, Koichi Murase écrivait :
>
> > Though, I see your point. It is inconvenient that we cannot pass the
> > results of arithmetic evaluations to the `printf' builtin. This
> > appears to be an issue of the printf builtin. I think the `printf'
> > builtin should be extended to interpret both forms of the numbers, the
> > locale-dependent formatted number and the floating-point literals.
>
> Another way would be to expand string representation of floating-point
> numbers using the locale.
Parameter transformations could be implemented to serve this purpose.
Let's say, if var is in the form of a C floating-point literal,
${var@F} would expand it to the locale-dependent formatted number, for
use as an argument to printf or for output directly. And then ${var@f}
would go the other way, taking var that's in the form of a
locale-dependent formatted number, and expanding it to a C
floating-point literal.
In this scenario, any floating point numbers ingested by the bash
script would be transformed to C floating-point literals before they
are referenced or manipulated in an arithmetic context, and then they
would be transformed back into the locale-dependent formatted number
before being output from the script.
Arithmetic evaluation is performed on the right-hand side of an
assignment statement assigning to a variable declared with the -i
integer attribute. A hypothetical -E floating-point variable attribute
in bash should work the same way. As such, a variable with the -E
attribute would have to be assigned a value in a format accepted by
the arithmetic context, i.e. the C floating-point literal format.
So you'd have to do
declare -E var="${arg@f}"
in order to safely handle the locale-dependent formatted number arg,
an argument to the script.
Any floating-point literals within the script would have to be
formatted as C floating-point literals, at least if they're going to
be assigned the -E attribute or used in an arithmetic context.
And then you would have to do
printf '%f' "${var@F}"
to safely output the value using printf.
Is that unreasonable?
Zack
Re: REQUEST - bash floating point math support
On 6/5/24 2:59 AM, Léa Gris wrote: Le 05/06/2024 à 17:09, Koichi Murase écrivait : 2024年6月5日(水) 21:41 Zachary Santer : Bash could potentially detect floating point literals within arithmetic expansions and adjust the operations to use floating point math in that case. [...] ksh and zsh are already behaving in that way, and if Bash would support the floating-point arithmetic, Bash should follow their behavior. Bash isn't even consistent with the floating point data/input format when using printf '%f\n' "$float_string" as it depends on LC_NUMERIC locale. LC_MESSAGES=C LC_NUMERIC=fr_FR.UTF8 printf '%f\n' 3.1415 bash: printf: 3.1415: invalid number 3,00 Chet explained it is because Bash is following the POSIX norm and C printf rules. This is true. The printf(3) engine, which bash uses to display floating point numbers, uses the radix character from the LC_NUMERIC locale category. The arguments to printf(1) have to be converted to floating point format to pass to printf(3). strtod(3), which bash uses to perform this conversion, uses the radix character from LC_NUMERIC as well. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: REQUEST - bash floating point math support
On Wed, Jun 5, 2024 at 4:08 PM Robert Elz wrote: > > That's a perfect case for scaled integers - no-one ever deals with > fractions of cents in this kind of thing (a bank won't ever tell you > that your balance is $5678.17426 for example, even if the interest > calculations computed accurately might arrive at that number.) Additionally, floating-point arithmetic introduces rounding errors that are *very undesirable* in a financial context. There, you will likely see fixed-point arithmetic, which is effectively the same thing as scaled integers.
Re: REQUEST - bash floating point math support
Le 06/06/2024 à 11:55, Koichi Murase écrivait : Though, I see your point. It is inconvenient that we cannot pass the results of arithmetic evaluations to the `printf' builtin. This appears to be an issue of the printf builtin. I think the `printf' builtin should be extended to interpret both forms of the numbers, the locale-dependent formatted number and the floating-point literals. Another way would be to expand string representation of floating-point numbers using the locale. Anyway, yes this POSIX restriction is pointless but causes troubles. Anyway, writing locale agnostic Bash scripts requires some precautions (like setting LC_ALL=C), that too many developers tend to ignore; before running commands without output to parse. Using LC_NUMERIC=C printf ... is an even less known precaution and I have already stumbled on a couple scripts that break because of this exact overlook. -- Léa Gris
Re: REQUEST - bash floating point math support
2024年6月6日(木) 18:09 Léa Gris : > Le 06/06/2024 à 10:29, Koichi Murase écrivait : > > 2024年6月6日(木) 15:59 Léa Gris : > >> Le 05/06/2024 à 17:09, Koichi Murase écrivait : > >>> 2024年6月5日(水) 21:41 Zachary Santer : > Bash could potentially detect floating point literals within > arithmetic expansions and adjust the operations to use floating point > math in that case. [...] > >>> > >>> ksh and zsh are already behaving in that way, and if Bash would > >>> support the floating-point arithmetic, Bash should follow their > >>> behavior. > >> > >> Bash isn't even consistent with the floating point data/input format > >> when using printf '%f\n' "$float_string" as it depends on LC_NUMERIC > >> locale. > > > > Maybe I miss your point, but literals in arithmetic expressions and > > the printf format are unrelated. > > They are in Bash. Bash use the locale format for floating-point number > arguments. The arguments of `printf' are not in an arithmetic context. This is the case even for the currently supported integers. If it were an arithmetic context, « printf %d 1+1 » should have printed `2', but the actual result is an error because the arguments of the `printf' builtin are unrelated to the arithmetic expression. Though, I see your point. It is inconvenient that we cannot pass the results of arithmetic evaluations to the `printf' builtin. This appears to be an issue of the printf builtin. I think the `printf' builtin should be extended to interpret both forms of the numbers, the locale-dependent formatted number and the floating-point literals. A problem is POSIX. I checked POSIX which states that the arguments of the `printf' utility for %f, etc. shall be interpreted by strtod(), and that strtod() shall use the locale-defined radix character and shall fail if it is in an unexpected format. By combining these statements, POSIX does not allow extension of the number format in the arguments of the printf builtin, which seems an unreasonable restriction. -- Koichi
Re: REQUEST - bash floating point math support
Le 06/06/2024 à 10:29, Koichi Murase écrivait : 2024年6月6日(木) 15:59 Léa Gris : Le 05/06/2024 à 17:09, Koichi Murase écrivait : 2024年6月5日(水) 21:41 Zachary Santer : Bash could potentially detect floating point literals within arithmetic expansions and adjust the operations to use floating point math in that case. [...] ksh and zsh are already behaving in that way, and if Bash would support the floating-point arithmetic, Bash should follow their behavior. Bash isn't even consistent with the floating point data/input format when using printf '%f\n' "$float_string" as it depends on LC_NUMERIC locale. Maybe I miss your point, but literals in arithmetic expressions and the printf format are unrelated. They are in Bash. Bash use the locale format for floating-point number arguments. It means: printf %f will break when the argument contains a floating-point number using a different decimal symbol than that of the current locale in the LC_NUMERIC environment variable. For example, if LC_NUMERIC=fr_FR.UTF8, the decimal separator symbol is a comma (not a dot) and printf %s 0.1 will fail because 0.1 is not recognized as a valid floating-point number argument by printf. The implication of this behaviour on floating-point implementation is that the values will not be compatible with all system locale since Bash does not allow a consistent format for the decimal separator symbol when passing floating-point numbers arguments to printf. -- Léa Gris
Re: REQUEST - bash floating point math support
2024年6月6日(木) 15:59 Léa Gris : > Le 05/06/2024 à 17:09, Koichi Murase écrivait : > > 2024年6月5日(水) 21:41 Zachary Santer : > >> Bash could potentially detect floating point literals within > >> arithmetic expansions and adjust the operations to use floating point > >> math in that case. [...] > > > > ksh and zsh are already behaving in that way, and if Bash would > > support the floating-point arithmetic, Bash should follow their > > behavior. > > Bash isn't even consistent with the floating point data/input format > when using printf '%f\n' "$float_string" as it depends on LC_NUMERIC locale. Maybe I miss your point, but literals in arithmetic expressions and the printf format are unrelated. Even if Bash is not consistent with ksh/zsh for the printf format, the floating-point arithmetic of Bash can be made consistent with ksh/zsh. The floating-point support in arithmetic expressions is not specified by POSIX/C. Bash is not required anything by POSIX/C about its support for floating-point arithmetic. Actually, arithmetic expressions are defined as a subset of C language in POSIX, so it is natural to follow the floating-point literals in C if Bash would support floating-point arithmetic. The floating-point literals in C are unrelated to the result of printf and independent of the locale where the C compiler runs. E.g. the decimal point is always `.' in the literals in the C source code. In this sense, even from the perspective of POSIX/C, there is no reason to adopt the output format of `printf' for the floating-point numbers in arithmetic expressions and the results of their evaluation. -- Koichi
Re: REQUEST - bash floating point math support
Le 05/06/2024 à 17:09, Greg Wooledge écrivait : On Wed, Jun 05, 2024 at 09:57:26PM +0700, Robert Elz wrote: Also note that to actually put floating support in the shell, more is needed than just arithmetic, you also need floating comparisons in test (or in bash, in [[ ) and a whole bunch more odds and ends that aren't obvious until you need them, and they're just not there (like a mechanism to convert floats back into integers again, controlling how rounding happens). Ironically, that last one is the one we already *do* have. hobbit:~$ printf '%.0f\n' 11.5 22.5 33.5 12 22 34 As long as you're OK with "banker's rounding", printf does it. As long as you are ok with your script breaking because of LC_NUMERIC local using a different decimal symbol like a comma in French. -- Léa Gris
Re: REQUEST - bash floating point math support
Le 05/06/2024 à 17:09, Koichi Murase écrivait : 2024年6月5日(水) 21:41 Zachary Santer : Bash could potentially detect floating point literals within arithmetic expansions and adjust the operations to use floating point math in that case. [...] ksh and zsh are already behaving in that way, and if Bash would support the floating-point arithmetic, Bash should follow their behavior. Bash isn't even consistent with the floating point data/input format when using printf '%f\n' "$float_string" as it depends on LC_NUMERIC locale. LC_MESSAGES=C LC_NUMERIC=fr_FR.UTF8 printf '%f\n' 3.1415 bash: printf: 3.1415: invalid number 3,00 Chet explained it is because Bash is following the POSIX norm and C printf rules. Now imagine if Bash introduce floating-point support, you write a Bash script with decimal point floating-point numbers and your script is incompatible with systems whose locale use a decimal comma instead. -- Léa Gris
Re: REQUEST - bash floating point math support
Date:Wed, 5 Jun 2024 13:31:20 -0400
From:Saint Michael
Message-ID:
| the most obvious use of floating variables would be to compare
| balances and to branch based on if a balance is lower than a certain
| value
In addition to what Greg suggested, for the very simple case you
outlined ---
That's a perfect case for scaled integers - no-one ever deals with
fractions of cents in this kind of thing (a bank won't ever tell you
that your balance is $5678.17426 for example, even if the interest
calculations computed accurately might arrive at that number.)
So, just do the calculations/tests using cents instead of dollars and
fractional dollars (aka cents) (or pounds & pence, or Euros and whatever,
or ...)
So just
dbal=${balance%.*}
cbal=${balance#*.}
case ${cbal} in ?) cbal=${cbal}0;; esac # just in case balance is 100.5
cents=${dbal}${cbal}
If you need to deal with European notations, use [.,] in each place
instead of just . (From the way you used inserted $balance into the
python script, it is clear there are no grouping characters in the
value - ie: not 1,234.56 or 1 234,56 or similar.)
If sometimes the balance has no cents, either ends in '.' or no '.' at all,
rather than explicitly having .00 at the end, a few extra lines will handle
that as well. You could also easily add more error/sanity checking on
the input values, if needed.
Then you can just use $cents and do integer arithmetic everywhere (and to
print it, if unchanged just use $balance, otherwise use the technique I
described in the previous message, except the scale factor is just 100,
so you want %.2d to print ${cents}%100 - etc).
Money is one of the most obvious cases where floating point isn't needed.
Not only because it is easy to work using the least valuable currency unit,
as above, but also because the calculations tend to be trivial, add, subtract,
and some simple multiplication (add tax at N% or whatever), and that's
about it. No-one takes square roots, or ln() or whatever of their bank
balance, or required credit card payment. But start doing floating point
and you'll soon get people saying "what do you mean I need to use bc to
calculate the natural log of ...? Why can't I just do that in bash?".
And of course, soon to be followed by "now we have floating point numbers,
we need complex numbers and arithmetic as well".
Further, if you're going to use something like python in your
script to work on some aspect, why not just write (almost) all
of the code in python? Sometimes you can make a program in
another language (like python) but where some operations there
are harder to achieve than in shell, where it is reasonable to
combine the two together - have the shell script do whatever is
needed to make it easy for python to do most of the work, then
when that's done, the script can do any required cleanup, etc.
Sometimes even several small python programs linked together by
a shell script might work.
[I wouldn't do that, I detest python, but I would use other similar
systems for the same effect.]
And last "There must be" is not an argument for absolutely anything
unless you can demonstrate why that is so. Nothing you cannot
prove "must be".
kre
ps: and why would floating point be the thing to consider adding,
why not URLs as file names? Surely bash should be able to do
cat < https://some.host/path/to/file > mailto:bug-bash@gnu.org
Why do we need external programs for network access - something far
more likely for a shell script to want to do that real number arithmetic.
And of course, no, this is not a serious suggestion.
Re: REQUEST - bash floating point math support
I think that we should do this in the shell. I mean. It will get done at some point, in the next decades or centuries. Why not do it now? Let's compile some C library or allow inline C On Wed, Jun 5, 2024, 2:12 PM Greg Wooledge wrote: > On Wed, Jun 05, 2024 at 01:31:20PM -0400, Saint Michael wrote: > > the most obvious use of floating variables would be to compare > > balances and to branch based on if a balance is lower than a certain > > value > > I use: > > t=$(python3 -c "import math;print($balance > 0)") > > and the > > if [ "$t" == "False" ];then > > echo "Result <= 0 [$t] Client $clname $clid Balance $balance" > > fi > > There must be a solution without Awk or Python or BC. Internal to bash > > The example you show is just comparing to 0, which is trivial. If > the $balance variable begins with "-" then it's negative. If it's "0" > then it's zero. Otherwise it's positive. > > For comparing two arbitrary variables which contain strings representing > floating point numbers, you're correct -- awk or bc would be the minimal > solution. > >
Re: REQUEST - bash floating point math support
On Wed, Jun 05, 2024 at 01:31:20PM -0400, Saint Michael wrote: > the most obvious use of floating variables would be to compare > balances and to branch based on if a balance is lower than a certain > value > I use: > t=$(python3 -c "import math;print($balance > 0)") > and the > if [ "$t" == "False" ];then > echo "Result <= 0 [$t] Client $clname $clid Balance $balance" > fi > There must be a solution without Awk or Python or BC. Internal to bash The example you show is just comparing to 0, which is trivial. If the $balance variable begins with "-" then it's negative. If it's "0" then it's zero. Otherwise it's positive. For comparing two arbitrary variables which contain strings representing floating point numbers, you're correct -- awk or bc would be the minimal solution.
Re: REQUEST - bash floating point math support
the most obvious use of floating variables would be to compare
balances and to branch based on if a balance is lower than a certain
value
I use:
t=$(python3 -c "import math;print($balance > 0)")
and the
if [ "$t" == "False" ];then
echo "Result <= 0 [$t] Client $clname $clid Balance $balance"
fi
There must be a solution without Awk or Python or BC. Internal to bash
On Wed, Jun 5, 2024 at 11:49 AM Robert Elz wrote:
>
> Date:Wed, 5 Jun 2024 11:09:45 -0400
> From:Greg Wooledge
> Message-ID:
>
> | > to convert floats back into integers again, controlling how
> | > rounding happens).
> |
> | Ironically, that last one is the one we already *do* have.
>
> Yes, I know about printf (and while POSIX doesn't require that floats
> be supported in printf(1), all the implementations I am aware of, do) but:
>
> | As long as you're OK with "banker's rounding"
>
> that is expressly not "controlling how rounding happens" - applications
> dealing with floats sometimes want round to nearest (which is what is
> happening in printf - the tiebreaker algorithm when up or down are equally
> near might be relevant, but usually isn't), others want round down (towards 0,
> that is, simply discard the fractional part - that's easy in sh with ${v%.*},
> though you might need to deal with "-0" as the result - others round up
> (away from 0) (harder in sh, but achievable), others want round towards
> minint (ie: positives round down, negatives round up), and I suppose round
> towards maxint (the opposite) might occur sometimes too, though I don't
> think I've ever seen a use for that one.
>
> Most of this can be done, with some difficulty sometimes, but they
> really ought to be done with arithmetic functions - in fact, it is
> hard to imagine any real floating point work that can be done without
> the ability to define functions that can be used in an arithmetic
> context.
>
> My whole point is that as simple as it seems to "just add float support
> to shell arithmetic" might seem, it wouldn't end there, it is almost
> certainly better to just not go there. There are plenty of other
> languages that can work with floats - not everything needs to be a shell
> script using only shell primitives. Use the appropriate tool, don't
> just pick one, and make it try to be everything.
>
> kre
>
> I have considered all this as I once thought of adding float arith to
> the shell I maintain, and it took very little of this kind of thought
> process to abandon that without ever writing a line of code for it.
>
>
>
Re: REQUEST - bash floating point math support
Date:Wed, 5 Jun 2024 11:09:45 -0400
From:Greg Wooledge
Message-ID:
| > to convert floats back into integers again, controlling how
| > rounding happens).
|
| Ironically, that last one is the one we already *do* have.
Yes, I know about printf (and while POSIX doesn't require that floats
be supported in printf(1), all the implementations I am aware of, do) but:
| As long as you're OK with "banker's rounding"
that is expressly not "controlling how rounding happens" - applications
dealing with floats sometimes want round to nearest (which is what is
happening in printf - the tiebreaker algorithm when up or down are equally
near might be relevant, but usually isn't), others want round down (towards 0,
that is, simply discard the fractional part - that's easy in sh with ${v%.*},
though you might need to deal with "-0" as the result - others round up
(away from 0) (harder in sh, but achievable), others want round towards
minint (ie: positives round down, negatives round up), and I suppose round
towards maxint (the opposite) might occur sometimes too, though I don't
think I've ever seen a use for that one.
Most of this can be done, with some difficulty sometimes, but they
really ought to be done with arithmetic functions - in fact, it is
hard to imagine any real floating point work that can be done without
the ability to define functions that can be used in an arithmetic
context.
My whole point is that as simple as it seems to "just add float support
to shell arithmetic" might seem, it wouldn't end there, it is almost
certainly better to just not go there. There are plenty of other
languages that can work with floats - not everything needs to be a shell
script using only shell primitives. Use the appropriate tool, don't
just pick one, and make it try to be everything.
kre
I have considered all this as I once thought of adding float arith to
the shell I maintain, and it took very little of this kind of thought
process to abandon that without ever writing a line of code for it.
Re: REQUEST - bash floating point math support
2024年6月5日(水) 21:41 Zachary Santer : > Bash could potentially detect floating point literals within > arithmetic expansions and adjust the operations to use floating point > math in that case. [...] ksh and zsh are already behaving in that way, and if Bash would support the floating-point arithmetic, Bash should follow their behavior. Actually, there's already a stub for the double-precision floating-point number for shell-variable values in variables.h [1], though it seems to be there for more than 20 years since 2.05b. [1] https://git.savannah.gnu.org/cgit/bash.git/tree/variables.h?h=devel&id=dbb48b978671394bcb33c9f34656a9aadf40a318#n75 > For completeness, a floating point variable attribute could > potentially be added. 'declare -i var' will cause var to be treated as > an integer, though var can be referenced within an arithmetic > expansion without this attribute. declare -f and -r (real, anybody?) > are already taken for other things, so I'm not sure what the natural > choice of option would be. ksh uses -E for the double-precision floating-point numbers, and `-E' is not used by Bash. By the way, people are writing shell libraries to perform the floating-point arithmetic in pure Bash, though I've never tried them and am unsure about their quality, performance, and API experience. https://github.com/clarity20/shellmath https://github.com/m1ndflay3r/bash-float We can also do the fixed-point arithmetic more easily and efficiently (except for special functions such as sin, cos, tan, exp, log, etc.). These are examples for specific purposes without needing precisions (and thus not all the operations are implemented): https://github.com/aneeshdurg/bash-raytracer/blob/main/math.sh https://github.com/akinomyoga/blesh-contrib/blob/master/colorglass.bash#L247-L381 -- Koichi
Re: REQUEST - bash floating point math support
On Wed, Jun 05, 2024 at 09:57:26PM +0700, Robert Elz wrote: > Also note that to actually put floating support in the shell, more is > needed than just arithmetic, you also need floating comparisons in test > (or in bash, in [[ ) and a whole bunch more odds and ends that aren't > obvious until you need them, and they're just not there (like a mechanism > to convert floats back into integers again, controlling how rounding happens). Ironically, that last one is the one we already *do* have. hobbit:~$ printf '%.0f\n' 11.5 22.5 33.5 12 22 34 As long as you're OK with "banker's rounding", printf does it.
Re: REQUEST - bash floating point math support
Date:Wed, 5 Jun 2024 08:40:51 -0400
From:Zachary Santer
Message-ID:
| The magnification factor is handled as a
| fractional number - 1.5, 1.75, etc. So, to change the magnification
| factor by increments of 0.25 or 0.5, I had to print an expression into
| bc in a command substitution.
You don't need bc for that (and if you did, for scripts, I'd generally
recommend awk - you can easily write whole programs in that instead of just
expressions) all you need is scaled integers.
Treat everything as in units of .001 (scaled by 1000) and just do normal
arithmetic (given that you are never going to want to multiply or divide
magnification factors by each other - if you do, in some other example,
you need to work just a little harder to keep the scale known and sane).
Then when you're ready to pass the result to some program that is expecting
floating point input just:
printf %d.%.3d $(( val / 1000 )) $(( val % 1000 ))
and you're done. You can use whatever scale factor allows the integer
part to avoid overflows (which isn't a limit for most applications) and
gives sufficient precision to the fractional part.
Applications that really need floating point arithmetic are very rarely
going to be appropriate to write as a shell script, if you have one of
those, you really ought to be using a language (compiled, or interpreted)
which can handle it properly, and so you get access to all the maths
functions that floating point applications tend to want to use (and bc
has only a very small fraction of those in its library - awk even less).
Also note that to actually put floating support in the shell, more is
needed than just arithmetic, you also need floating comparisons in test
(or in bash, in [[ ) and a whole bunch more odds and ends that aren't
obvious until you need them, and they're just not there (like a mechanism
to convert floats back into integers again, controlling how rounding happens).
kre
ps: as a real example, where real floating arith is needed, the following
is a fragment from a script, which (when this part is encountered) has in
variable L an ordered list of floating point numbers (which happen to be
durations in seconds, with unspecified precision and range (ie: not integers),
but that explains some constants in the code). The objective is to
find the mean of the values from that list which are > 0.75 * and < 1.25 *
of the (approx) median value (that is, ignoring extreme values, high and low)
provided that most of the values are within that range. The output is that
average (mean) as an integer number of minutes, rounded to the nearest 5.
That is, to give a very rough approximation of what the typical duration is,
in units humans more easily deal with than thousands of seconds (to N decimal
places for some N I don't know or care about). If the result is big enough
that presenting it in hrs & mins is better, other code can make that happen
(that's simple integer arithmetic).
This isn't written for bash (hence uses no bash extensions) but there's
no reason I can think of why bash couldn't run it. The script is run
quite frequently (several times every few minutes) with entirely
acceptable performance (this particular fragment usually gets executed
30 to 40 times each execution of the script - that is, to deal with that
many different lists, the actual number varies).
unset IFS
set -- $( printf '%s\n' $L | sort -n )
N=$#
if test "$N" -eq 0
then
exit
fi
shift $(( (N - 1) / 2 ))
M=$1
printf '%s\n' $L | awk -v M="$M" '
BEGIN {
L = M * 0.75
U = M * 1.25
N=0
O=0
T=0
}
NR == 1 {
if ($1 < L || $1 > U) {
A = ($1 + 30) / 60
A = int( (A + 2.5) / 5 ) * 5
printf "x %.0f", A
exit
}
}
{
if ($1 < L || $1 > U ) {
O = O + 1
next
}
T = T + $1
N = N + 1
}
EN
Re: Request for new 5.2 alpha?
On 3/21/22 3:14 PM, Sam James wrote: Hi, Is there a plan for a 2nd 5.2 alpha? A number of reported bugs have been addressed since and it may help resolve duplicates. The next release will be 5.2-beta. You can always use the devel branch for the latest code. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Request change to output of declare -f funcname
On 10/2/21 1:02 PM, Léa Gris wrote: Le 02/10/2021 à 18:45, Greg Wooledge écrivait : On Sat, Oct 02, 2021 at 06:06:32PM +0200, Léa Gris wrote: Better illustrated how newlines are discarded: $ sudo bash -c 'echo hello echo world' hello world $ sudo -i bash -c 'echo hello echo world' helloecho world OK, that's news to me. But that looks like a bug in sudo. Asking bash to change its behavior to work around a bug in some other program seems like misdirected effort. It is not wholly misdirected since Bash already add semicolon after statements where it is optional since these are followed by newlines. Adding a semicolon after the last statement of a command group would not hurt in any circumstance. But what would such a change help, other than to avoid the consequences of what appears to be a bug in `sudo -i'? Now while reworking the output of declare -f, there could be an option to produce the most compact output without newlines and without indentation; for the purpose of serializing function declarations, similarly as declare -p already serialize variables declaration in a compact one-line even for arrays. Corollary declare -p could have an indented expanded output that would be useful for arrays with an element per line rather than space delimited elements. If you'd like to see these changes, maybe do a sample implementation that you (and others, if you'd like) can evaluate? Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Request change to output of declare -f funcname
On Sat, Oct 2, 2021 at 7:54 PM Andreas Kusalananda Kähäri wrote: > I don't think their issue is that the output from declare -f is ever > invalid as such, but that their usage of the resulting text involves > replacing newlines with spaces, which turns it into invalid shell code. Not spaces, but backslashes followed by newlines. sudo -i echo 'foo bar' prints `foobar', not `foo bar'. Anyway, I agree that this is a bug in sudo that should be fixed there. > > -- > Andreas (Kusalananda) Kähäri > SciLifeLab, NBIS, ICM > Uppsala University, Sweden > > . >
Re: Request change to output of declare -f funcname
Le 02/10/2021 à 18:45, Greg Wooledge écrivait : On Sat, Oct 02, 2021 at 06:06:32PM +0200, Léa Gris wrote: Better illustrated how newlines are discarded: $ sudo bash -c 'echo hello echo world' hello world $ sudo -i bash -c 'echo hello echo world' helloecho world OK, that's news to me. But that looks like a bug in sudo. Asking bash to change its behavior to work around a bug in some other program seems like misdirected effort. It is not wholly misdirected since Bash already add semicolon after statements where it is optional since these are followed by newlines. Adding a semicolon after the last statement of a command group would not hurt in any circumstance. Now while reworking the output of declare -f, there could be an option to produce the most compact output without newlines and without indentation; for the purpose of serializing function declarations, similarly as declare -p already serialize variables declaration in a compact one-line even for arrays. Corollary declare -p could have an indented expanded output that would be useful for arrays with an element per line rather than space delimited elements. These changes to the serialization of declare -f and -p would not affect backward or forward compatibility between serialization types. It would make the declare -f more resilient to newline removal and would allow both compact and indented verbose formatted outputs to coexist.
Re: Request change to output of declare -f funcname
On Sat, Oct 02, 2021 at 09:09:39AM -0400, Greg Wooledge wrote:
> On Sat, Oct 02, 2021 at 01:41:35PM +0200, Léa Gris wrote:
> > $ declare -f hello
> >
> > hello ()
> > {
> > echo 'hello';
> > echo 'world'
> > }
> >
> > The issue is that in some circumstances, newline characters may be handled
> > as space, making the function declaration invalid.
>
> Can you show an example where the output of declare -f is invalid?
I don't think their issue is that the output from declare -f is ever
invalid as such, but that their usage of the resulting text involves
replacing newlines with spaces, which turns it into invalid shell code.
--
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden
.
Re: Request change to output of declare -f funcname
On Sat, Oct 02, 2021 at 06:06:32PM +0200, Léa Gris wrote: > Better illustrated how newlines are discarded: > > $ sudo bash -c 'echo hello > echo world' > > hello > > world > > $ sudo -i bash -c 'echo hello > echo world' > > helloecho world OK, that's news to me. But that looks like a bug in sudo. Asking bash to change its behavior to work around a bug in some other program seems like misdirected effort. unicorn:~$ sudo ~greg/bin/args $'one\ntwo' three 2 args: unicorn:~$ sudo -i ~greg/bin/args $'one\ntwo' three 2 args: I wonder why it does that. It's not helpful in any situation I can think of. It's certainly not documented in the -i section of the man page.
Re: Request change to output of declare -f funcname
Le 02/10/2021 à 15:09, Greg Wooledge écrivait :
On Sat, Oct 02, 2021 at 01:41:35PM +0200, Léa Gris wrote:
$ declare -f hello
hello ()
{
echo 'hello';
echo 'world'
}
The issue is that in some circumstances, newline characters may be handled
as space, making the function declaration invalid.
Can you show an example where the output of declare -f is invalid?
hello (){ echo 'hello';echo 'world';}
LC_MESSAGES=C sudo bash -c "$(declare -f hello);hello"
hello
world
LC_MESSAGES=C sudo -i bash -c "$(declare -f hello);hello"
bash: -c: line 2: syntax error: unexpected end of file
Better illustrated how newlines are discarded:
$ sudo bash -c 'echo hello
echo world'
hello
world
$ sudo -i bash -c 'echo hello
echo world'
helloecho world
Or:
$ sudo bash -c "printf %q\\\n \"$(declare -f hello)\""
$'hello () \n{ \necho \'hello you the\';\necho \'world\'\n}'
sudo -i bash -c "printf %q\\\n \"$(declare -f hello)\""
hello\ \(\)\ \{\ \ \ \ \ echo\ \'hello\ you\ \ the\'\;\ \ \ \ echo\ \'world\'\}
--
Léa Gris
Re: Request change to output of declare -f funcname
there are many paste boards and preview systems that cripple newlines to
spaces
the dude that posted the issue wrote about that case, \n to ' '
On Sat, Oct 2, 2021, 17:08 Greg Wooledge wrote:
> On Sat, Oct 02, 2021 at 04:52:38PM +0200, Alex fxmbsw7 Ratchev wrote:
> > in pasted mails where newlines get converted to spaces
> >
> > On Sat, Oct 2, 2021, 15:21 Greg Wooledge wrote:
> >
> > > On Sat, Oct 02, 2021 at 01:41:35PM +0200, Léa Gris wrote:
> > > > $ declare -f hello
> > > >
> > > > hello ()
> > > > {
> > > > echo 'hello';
> > > > echo 'world'
> > > > }
> > > >
> > > > The issue is that in some circumstances, newline characters may be
> > > handled
> > > > as space, making the function declaration invalid.
> > >
> > > Can you show an example where the output of declare -f is invalid?
>
> Your personal inability to write an email correctly is irrelevant here.
>
>
Re: Request change to output of declare -f funcname
On Sat, Oct 02, 2021 at 04:52:38PM +0200, Alex fxmbsw7 Ratchev wrote:
> in pasted mails where newlines get converted to spaces
>
> On Sat, Oct 2, 2021, 15:21 Greg Wooledge wrote:
>
> > On Sat, Oct 02, 2021 at 01:41:35PM +0200, Léa Gris wrote:
> > > $ declare -f hello
> > >
> > > hello ()
> > > {
> > > echo 'hello';
> > > echo 'world'
> > > }
> > >
> > > The issue is that in some circumstances, newline characters may be
> > handled
> > > as space, making the function declaration invalid.
> >
> > Can you show an example where the output of declare -f is invalid?
Your personal inability to write an email correctly is irrelevant here.
Re: Request change to output of declare -f funcname
in pasted mails where newlines get converted to spaces
On Sat, Oct 2, 2021, 15:21 Greg Wooledge wrote:
> On Sat, Oct 02, 2021 at 01:41:35PM +0200, Léa Gris wrote:
> > $ declare -f hello
> >
> > hello ()
> > {
> > echo 'hello';
> > echo 'world'
> > }
> >
> > The issue is that in some circumstances, newline characters may be
> handled
> > as space, making the function declaration invalid.
>
> Can you show an example where the output of declare -f is invalid?
>
>
Re: Request change to output of declare -f funcname
On Sat, Oct 02, 2021 at 01:41:35PM +0200, Léa Gris wrote:
> $ declare -f hello
>
> hello ()
> {
> echo 'hello';
> echo 'world'
> }
>
> The issue is that in some circumstances, newline characters may be handled
> as space, making the function declaration invalid.
Can you show an example where the output of declare -f is invalid?
Re: Request: a command to do a return completely in normal, ie. not subshell
Budi writes: > Can we have a command to do a return completely (as if it returns from > main function) when it is being in a third nested function call, in > order to get back to shell prompt at once, in normal, ie. not > subshell? You probably want to use the "exit" command, if all of this is in a shell script. Dale
Re: Request For Enhancement - TID variable
Daniel Colascione : > Are you sure that'd help? Parallel runs bash in a bunch of subprocesses, so > looking at PID would suffice to distinguish jobs. Are you sure you weren't > seeing an invariant PID because you were letting the PID variable expansion > happen too early? The $$ is inside the script. I don't see how it could be later. Rerunning the command, I think I misinterprteted the error messages before. The script is a wrapper around another script that is calling svnadmin to create a Subversion repository. svnadmin doesn't like something about its calling context and is complaining that it can't write a directory atomically. Not bash's problem. Carry on! -- http://www.catb.org/~esr/";>Eric S. Raymond
Re: Request For Enhancement - TID variable
On Dez 26 2019, Eric S. Raymond wrote: > RFE: bash should have a TID varuable that returns the vakue of gettid(2). Bash doesn't use threads, so $TID will always be the same as $BASHPID. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: Request For Enhancement - TID variable
On 12/26/19 7:37 AM, Eric S. Raymond wrote: In attempting to use GNU parallel, with some bash scripts, I discovered I had a problem with tempfile collisions due to all of the thread having the same PID. I was able to come up with a workaround, but... RFE: bash should have a TID varuable that returns the vakue of gettid(2). If the bash devs are too busy for this like the idea, I could write the patch. Are you sure that'd help? Parallel runs bash in a bunch of subprocesses, so looking at PID would suffice to distinguish jobs. Are you sure you weren't seeing an invariant PID because you were letting the PID variable expansion happen too early?
Re: request
[quote=Greg]
> You don't know whether it's actually a missing quote.
[/quote]
[quote=t]
There was missing quote. Usually it is correct info from error
and it can tell also what character.
I'm not a professional programmer, but I think that,
Bash only checks the number of quotes ( for bash )
and ignore commands with regular expressions.
( maybe grep, sed , awk )
This is a very quick method to check, but it can not accurately check.
And that is the reason
why it does not know where to look.
The rules of programming are very determined, so exist ways to debug.
I do not ask you to fix this, because build debugger it is not easy.
This is the reason why I wrote only "add suggestion".
I do not know, You can write on the blog or other website ...
And people will write all the terms.
( for sed , awk , grep and other command when single quotes can be ignored )
In future this terms can be used for debug,
first in $() , later everything outside.
[quote=Greg]
> It could be a missing } or a missing fi or a missing esac or a missing
> done, or ANY OTHER missing or incorrect syntax element.
[/quote]
[quote=t]
Not, because for missing "fi" is other error and also
this is not very accurate, but it can point to a broken function.
Poor comfort, but better than nothing.
[quote]
={
On Fri, 14 Jul 2017 09:38:03 -0400
Greg Wooledge wrote:
> On Fri, Jul 14, 2017 at 11:09:25AM +0200, t wrote:
> > #
> > [me@linuxbox me]$ ./trouble.bash
> > ./trouble.bash: line 8430: unexpected EOF while looking for matching "
> > ./trouble.bash: line 8440 systax error: unexpected end of file
>
> Your problem is that your script is about 8300 lines longer than it
> should be. Writing a bash script that's over ~100 lines is crazy.
>
> Yours is over 8000 lines.
>
> I have no words.
>
> > echo "We suggest check $VAR"
> > echo "because this lines of file, contain an odd number of quotes "
>
> But lines are explicitly *allowed* to contain an odd number of quotes.
> This is how multi-line string constants are created.
>
> body="This is the body of an email. It has
> multiple lines. There are so many words
> that they don't all fit on one line."
>
> When the parser encouters a syntax error, it *cannot* know where the real
> problem is; only that there *is* one, somewhere. You as the programmer
> have to be the one to track it down and fix it.
>
> You know that the syntax error originates somewhere *on* or *before*
> the line number where the parser hit the brick wall and stopped. That's
> all you know. You don't know whether it's actually a missing quote.
> It could be a missing } or a missing fi or a missing esac or a missing
> done, or ANY OTHER missing or incorrect syntax element.
>
> That is *one* of the reasons why bash scripts should be kept small.
>
> > 2. Please hide e-mail for google bots, or remove this e-mail after read.
>
> Screw you. (And yes, this is the sanitized version.)
>
--
}
Re: request
On Fri, Jul 14, 2017 at 11:09:25AM +0200, t wrote: > # > [me@linuxbox me]$ ./trouble.bash > ./trouble.bash: line 8430: unexpected EOF while looking for matching " > ./trouble.bash: line 8440 systax error: unexpected end of file Your problem is that your script is about 8300 lines longer than it should be. Writing a bash script that's over ~100 lines is crazy. Yours is over 8000 lines. I have no words. > echo "We suggest check $VAR" > echo "because this lines of file, contain an odd number of quotes " But lines are explicitly *allowed* to contain an odd number of quotes. This is how multi-line string constants are created. body="This is the body of an email. It has multiple lines. There are so many words that they don't all fit on one line." When the parser encouters a syntax error, it *cannot* know where the real problem is; only that there *is* one, somewhere. You as the programmer have to be the one to track it down and fix it. You know that the syntax error originates somewhere *on* or *before* the line number where the parser hit the brick wall and stopped. That's all you know. You don't know whether it's actually a missing quote. It could be a missing } or a missing fi or a missing esac or a missing done, or ANY OTHER missing or incorrect syntax element. That is *one* of the reasons why bash scripts should be kept small. > 2. Please hide e-mail for google bots, or remove this e-mail after read. Screw you. (And yes, this is the sanitized version.)
Re: Request to a new feature on read
On Thu, Apr 16, 2015 at 9:50 AM, Greg Wooledge wrote: > I don't see why such features should be compiled into bash's read builtin. > I'd have no problem with adding better splitting/joining/parsing features > in a more general context, probably operating on a string variable, but > certainly not operating on a file descriptor. I don't think they should be part of `read` either. Some way to extend the BASH_REMATCH mechanism would be better. > > Doesn't the underlying C library only guarantee you a single character of > lookahead when reading? (Or maybe a single byte. I'm way out of date. > My knowledge of C comes from the days when char = byte.) You can't do > all this fancy perl-RE-style lookahead stuff on a stream with only a > single byte/char of lookahead. Hm, maybe you're referring to ungetc? IIRC one byte is the only guarantee when dealing with pipes. I don't really care about having it pattern match while reading a stream. To make that work well would probably involve mmap (and even then, only on regular files). Probably the most portable way to support "fancier" regex is to call into std::regex. Any system with a modern C++ compiler should support ECMAScript regex, which is close to a superset of ERE.
Re: Request to a new feature on read
On Thu, Apr 16, 2015 at 09:39:08AM -0500, Dan Douglas wrote: > On Thu, Apr 16, 2015 at 9:32 AM, Greg Wooledge wrote: > > On Thu, Apr 16, 2015 at 09:29:56AM -0500, Dan Douglas wrote: > >> I find myself in need of something along the lines of Python's > >> `re.split` and `re.findall` all the time. E.g. splitting an ip into an > >> array of octets. > > > > IFS=. read -ra octets <<< "$ip" > > Sure, but validation is then separate if needed. There are plenty of > applications where you want either a multi-character or non-static > delimiter, possibly with pattern matching on the data at the same > time. I don't see why such features should be compiled into bash's read builtin. I'd have no problem with adding better splitting/joining/parsing features in a more general context, probably operating on a string variable, but certainly not operating on a file descriptor. Doesn't the underlying C library only guarantee you a single character of lookahead when reading? (Or maybe a single byte. I'm way out of date. My knowledge of C comes from the days when char = byte.) You can't do all this fancy perl-RE-style lookahead stuff on a stream with only a single byte/char of lookahead.
Re: Request to a new feature on read
While I was developing a small script, I thought about how to use -N flag to a greater extent. Although -N in its own is very limited. It does serve the purpose but not what I need. I also discussed this in #bash freenode, and got some ideas like: pgas: while read -n1 d;do case $d in '')break;; [0-9])var+=$d;;*) echo error;;esac;done A for loop is probably not such a bad idea either. I'll try and see if I can figure out something. Thanks On Thu, Apr 16, 2015 at 3:55 PM, Eduardo A. Bustamante López < dual...@gmail.com> wrote: > Any reason to justify this instead of using a simple loop? > > -- > Eduardo Bustamante > https://dualbus.me/ > -- Met vriendelijke groet, Valentin Bajrami
Re: Request to a new feature on read
On Thu, Apr 16, 2015 at 09:29:56AM -0500, Dan Douglas wrote: > I find myself in need of something along the lines of Python's > `re.split` and `re.findall` all the time. E.g. splitting an ip into an > array of octets. IFS=. read -ra octets <<< "$ip"
Re: Request to a new feature on read
On Thu, Apr 16, 2015 at 9:32 AM, Greg Wooledge wrote: > On Thu, Apr 16, 2015 at 09:29:56AM -0500, Dan Douglas wrote: >> I find myself in need of something along the lines of Python's >> `re.split` and `re.findall` all the time. E.g. splitting an ip into an >> array of octets. > > IFS=. read -ra octets <<< "$ip" Sure, but validation is then separate if needed. There are plenty of applications where you want either a multi-character or non-static delimiter, possibly with pattern matching on the data at the same time.
Re: Request to a new feature on read
On Thu, Apr 16, 2015 at 8:55 AM, Eduardo A. Bustamante López wrote: > Any reason to justify this instead of using a simple loop? I find myself in need of something along the lines of Python's `re.split` and `re.findall` all the time. E.g. splitting an ip into an array of octets. On Thu, Apr 16, 2015 at 5:49 AM, Valentin Bajrami wrote: > Hi, > > According to ''help read'' we can specify -N[chars] to trigger return > automatically. Is it possible to approach read differently? > > For example: $re is some regular expression FWIW, ksh has two redirect operators that can be used together with `read` to get something like this. They're somewhat difficult to use IMO: <#pattern Seeks forward to the beginning of the next line containing pattern. <##patternThe same as <# except that the portion of the file that is skipped is copied to standard output. -- Dan Douglas
Re: Request to a new feature on read
Any reason to justify this instead of using a simple loop? -- Eduardo Bustamante https://dualbus.me/
