bash variable names do not comply w/POSIX character set rules

[Linda Walsh]

Under section 2.5.3, Shell Variables, it mentions:

LC_CTYPE
   Determine the interpretation of sequences of bytes of text data as 
characters (for example, single-byte as opposed to multi-byte 
characters), which characters are defined as letters (character class 
alpha) and  characters (character class blank), and the behavior 
of character classes within pattern matching.


If I have an LC_CTYPE set to UTF-8, then the rules in unicode as
to how the character is defined (alpha, numeric, alphanumeric, etc...)
seem appropriate to use.

In the bash man page, there is a definition of 'name':
  name   A word consisting only of  alphanumeric  characters  and  under-
 scores,  and beginning with an alphabetic character or an under-
 score.  Also referred to as an identifier.

However, I was looking for a char to visually separate
a "class" and a var in the class (would have liked something
like a.b, but "." isn't alpha numeric), but
"LATIN CAPITAL LETTER O WITH STROKE" (U+00D8), is alphabetic,
but doesn't work:

 aØb=1

-bash: aØb=1: command not found

The POSIX portable character set:
6. Character Set
6.1 Portable Character Set

Conforming implementations shall support one or more coded character 
sets. Each supported locale shall include the portable character set, 
which is the set of symbolic names for characters in Portable Character 
Set. This is used to describe characters within the text of 
POSIX.1-2008. The first eight entries in Portable Character Set are 
defined in the ISO/IEC 6429:1992 standard and the rest of the characters 
are defined in the ISO/IEC 10646-1:2000 standard.


ISO10646 = Unicode -- I.e. Posix appears to base its definition of
alphanumeric characters, for example, on the Unicode character set.

So, theoretically, any alphanumeric class char from Unicode should work
as described in the bash manpages, to compose a "name" (variable or
subroutine name), but this doesn't seem to be the case.

I know this isn't a trivial POSIX requirement to meet, but given
Gnu and bash's changes in the shell and unix command behavior, it
seems support of the character set would be the foundation of POSIX
compatibility.

It it were me, I'd probably try to look at the perl-handling (imperfect
as it may be) for unicode -- which has had alot of work put into it and
may be one of the more complete and up-to-date implementations for unicode
character handling.  I'd try to see if there was any part that might
either give ideas for bringing bash into compliance or any code that
might provide a pattern for implementation.  But investigating it further
might yield other, better options for bash.  Dunno.

Is this something that's even been thought about or is planned for?

Thanks!
-Linda

Re: Random parser error of $() when there is a case inside

[Chet Ramey]

On 12/4/15 1:25 PM, Luiz Angelo Daros de Luca wrote:

> Bash Version: 4.2
> Patch Level: 53
> Release Status: release
> 
> Description:
> While writing a script, I faced a random parser error with code like
> 
> A=$(
> ...
> case b in
> c)
> ...
> esac 
> ...
> )
> 
> Minor changes like renaming a variable skips the problem. It seems that the
> parser gets confused in some cases.

This was fixed back in bash-4.3.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Out of bounds heap read in function rl_tilde_expand

[Chet Ramey]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/5/15 4:12 PM, Hanno Böck wrote:
> On Mon, 9 Nov 2015 09:57:22 -0500
> Chet Ramey  wrote:
> 
>> On 11/6/15 9:46 AM, Hanno Böck wrote:
>>> While testing bash with address sanitizer I discovered a heap out of
>>> bounds read. This affects bash 4.3 with the latest patchlevel 42.  
>>
>> Thanks for the report.  This will be fixed in the next release of
>> bash.
> 
> Hi Chet,
> 
> I've seen there was a new bash 4.3 patchlevel, but this one didn't get
> fixed. Hope it wasn't forgotten.

The fix is in the devel branch.

- -- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org

iEYEARECAAYFAlZjVPYACgkQu1hp8GTqdKvWmACfbWu8KxbO4EQaFJ9pelMHd9I6
v4kAn3DY/w6DsSm4/i/VFR52aIngGys7
=wA8E
-END PGP SIGNATURE-

[POSIX] `a='a b'; export a=$a; echo $a` gives `a b` instead of `a`

[Mingye Wang (Arthur2e5)]

Bash Version: 4.3
Patch Level: 42
Release Status: Release

Description:
  According to [POSIX.1], `export` is special builtin *command*.
Therefore as a command, in `a='a b'; export a=$a; echo $a`, `$a` should
get split into words `a` and `b`.

  However, `bash` takes this as `export 'a=a b'` and gives un-POSIX
results. To make sure this is not a `bash` extension since this is
actually a common headache for new users and `bash` may try to extend
its syntax here, I tried `bash --posix`, but it exhibits the same
non-POSIX behavior.

Repeat-By:
  Just run this in `bash`, `bash --posix` and `dash`:

set -xv
a='a b'
export a=$a
echo $a

  [POSIX.1]:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#export
-- 
Regards,

Arthur2e5



signature.asc
Description: OpenPGP digital signature

Re: Script shell suspends after executing second child process

[konsolebox]

On Sun, Nov 15, 2015 at 2:25 PM, konsolebox  wrote:
> On Sun, Nov 15, 2015 at 10:34 AM, Chet Ramey  wrote:
>> On 11/14/15 6:02 PM, Chet Ramey wrote:
>>
>> Here, try this patch.  It sort of splits the difference along the lines of
>> your suggestion, and it behaves correctly for the case that inspired the
>> change originally.
>>
>
> This seems to have fixed it both in bash-20151106 and bash-20150424.
> The parent shell no longer suspends.

I just want to say that I also tried the new updates in git, and the
shell doesn't suspend with those as well. Thank you very much.

Re: Installation of PDF/PS/DVI and HTML files

[Chet Ramey]

On 12/4/15 4:24 AM, Ludovic Courtès wrote:
> Greg Wooledge  skribis:
> 
>> On Thu, Dec 03, 2015 at 01:08:13PM +0200, Ludovic Courtès wrote:
>>> Given that the GCS suggests installing only the Info version of the
>>> manual by default (info "(standards) Standard Targets")
>>
>>> What do you think?
>>
>> I think that's a stupid suggestion.  The de facto standard for "make"
>> followed "make install" on a Unix-like system is to install man pages.
>> If there's an info page, I have no objection to installing that as well,
>> but to omit the standard man pages by default is ridiculous.
> 
> Agreed; apologies for being unclear.
> 
> As Mathieu wrote, I am of course fine installing man and Info manuals by
> default, like GNU packages generally do.
> 
> The suggestion I make is to not install PDF/PS/DVI and HTML files by
> default.  

Again, only the HTML files are installed by `make install'.  The sticking
point here appears to be installing the HTML files, which you can suppress
by running `make install' with htmldir set to the empty string.  Is that
what you're saying?

> This would comply with the GCS and user expectations, and also
> sidestep the bit-for-bit reproducibility issues that generating those
> PDF/PS/DVI/HTML files entails.

So the problem is once again the build and not the install?  Since the
build version appears in the version string, and that changes each time
the binary is rebuilt, bit-by-bit reproducibility is not going to be
generally possible.

However, if it's the build, if something changes when you run make, it
implies that one of the source files changed or that the target did not
exist.  bash-4.4, unlike bash-4.3, will ship with the generated
documentation (look at the bash-4.4-beta distribution, for example).
Given that, under what circumstances would the generated documentation
need to be rebuilt by this `reproducible builds' effort?

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: [POSIX] `a='a b'; export a=$a; echo $a` gives `a b` instead of `a`

[Chet Ramey]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/5/15 6:57 PM, Mingye Wang (Arthur2e5) wrote:
> Bash Version: 4.3
> Patch Level: 42
> Release Status: Release
> 
> Description:
>   According to [POSIX.1], `export` is special builtin *command*.
> Therefore as a command, in `a='a b'; export a=$a; echo $a`, `$a` should
> get split into words `a` and `b`.
> 
>   However, `bash` takes this as `export 'a=a b'` and gives un-POSIX
> results. To make sure this is not a `bash` extension since this is
> actually a common headache for new users and `bash` may try to extend
> its syntax here, I tried `bash --posix`, but it exhibits the same
> non-POSIX behavior.

This has come up before.  The current (amended) version of the Posix
standard  contains the following (approved) interpretation dating from
2010-2011:

http://austingroupbugs.net/view.php?id=351

which addresses exactly this case.

Bash is conformant, dash needs to be updated, or will be when this
amended version of the standard is published.

- -- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org

iEYEARECAAYFAlZjgKEACgkQu1hp8GTqdKsK/ACgnzR6FE55Gyctqhsg/hyX3AIS
Xo8AmwRZmxKJhNo6KcQp5NqpSljio07a
=6FAj
-END PGP SIGNATURE-

Re: Out of bounds heap read in function rl_tilde_expand

[Hanno Böck]

On Mon, 9 Nov 2015 09:57:22 -0500
Chet Ramey  wrote:

> On 11/6/15 9:46 AM, Hanno Böck wrote:
> > While testing bash with address sanitizer I discovered a heap out of
> > bounds read. This affects bash 4.3 with the latest patchlevel 42.  
> 
> Thanks for the report.  This will be fixed in the next release of
> bash.

Hi Chet,

I've seen there was a new bash 4.3 patchlevel, but this one didn't get
fixed. Hope it wasn't forgotten.

-- 
Hanno Böck
http://hboeck.de/

mail/jabber: ha...@hboeck.de
GPG: BBB51E42


pgpR_aB5GlykX.pgp
Description: OpenPGP digital signature

Re: Installation of PDF/PS/DVI and HTML files

[Chet Ramey]

On 12/3/15 6:05 AM, Ludovic Courtès wrote:
> Hello,
> 
> Akira of Debian noticed that ‘make all’ rebuilds and install
> PDF/PS/DVI/HTML documentation by default, which prevents default Bash
> builds from being bit-reproducible¹.

This isn't completely accurate, since `make all' doesn't install anything.
So is it the building in the build directory or the installation not
being minimal enough that is the problem?

> 
> Given that the GCS suggests installing only the Info version of the
> manual by default (info "(standards) Standard Targets"), what about a
> change along the lines of the patch below?

That's not quite what it says.  It suggests that the `install' target
install the info file, but leaves other files up to the discretion of
the package maintainer.


> --- a/doc/Makefile.in
> +++ b/doc/Makefile.in
> @@ -146,9 +146,9 @@ BASHREF_FILES = $(srcdir)/bashref.texi $(srcdir)/fdl.texi 
> $(srcdir)/version.texi
>   ${RM} $@
>   -${DVIPS} $<
>  
> -all: ps info dvi text html
> +all: info
>  nodvi: ps info text html
> -everything: all pdf
> +everything: all pdf dvi text html
>  
>  PSFILES = bash.ps bashbug.ps article.ps builtins.ps rbash.ps 
>  DVIFILES = bashref.dvi bashref.ps

So the problem is that bash builds too much by default?  And it's the build
output that needs to be bit-for-bit reproducible?  In general, this is
impossible, since the version string changes with each new `build version'.

This patch doesn't have anything to do with the install target, which
installs the info file, man pages, and html files by default.  (And if you
don't want the html files installed, run make install with `htmldir' set
to the empty string.)


> In addition, the ‘install’ rule in doc/Makefile.in would need to be
> split in ‘install-info’, ‘install-pdf’, etc. (as explained in the GCS),
> with ‘install’ depending only on ‘install-info’.

Again, that's not quite what the coding standards say.  The standards say
to make sure to install the info file when you run `make install', using
the `install-info' program to do so.

> What do you think?

It's hard to say, since the proposed patch has little to do with the
subject of the message.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: An array variable created by export/readonly builtins inside a function becomes a locale variable to that function unexpectedly

[Chet Ramey]

On 11/27/15 10:18 PM, ziyunfei wrote:
> $ ksh -c 'foo() { readonly a=(1);echo a=$a; }; foo; echo a=$a'
> a=1
> a=1
> $ bash -c 'foo() { readonly a=(1);echo a=$a; }; foo; echo a=$a' # a becomes a 
> local variable
> a=1
> a=

Thanks for the report.  I agree that this is a bug.  `readonly' and
`export' should not create local variables when invoked within a function;
only declare/local/typeset should create local variables.

This will be fixed in the next release of bash.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/