Re: [PATCH] Support configurable mode strings in prompt

[Dylan Cali]

On Sun, Nov 16, 2014 at 9:48 PM, Chet Ramey  wrote:

> Most of this will be in the next release of bash.  I added some things and
> left out the \m prompt expansion for now.
>

Hi Chet,

I finally had a chance to play with the 4.4 beta.  Unfortunately, it seems
to be getting confused when color escapes are used (getting a lot of
garbage when I switch to command mode).  These are the mode strings I used
(just the defaults with green/yellow color escapes):

set vi-ins-mode-string   \1\e[32;1m\2(ins)\1\e[0m\2
set vi-cmd-mode-string \1\e[33;1m\2(cmd)\1\e[0m\2

It breaks even with a very simple PS1, e.g. PS1='$ '

I tried adding the \[ \] guards but that didn't make a difference (I think
those are only interpreted by bash proper right?)

Here is a gif of the buggy behavior:
http://imgur.com/Ri1byLj

By way of comparison here is what it looks like using the original patch
with 4.3 (I merged your naming conventions and the \1 \2 escape support
into the patch):
http://imgur.com/MV5ZsfG

Is it possible something got lost in translation?W


Separately, would it be possible to reconsider including the '\m' escape
functionality?  I think letting the user specify where in the prompt they
want the mode indicator to occur would be a killer feature... I've
certainly enjoyed having that flexibility locally.

Thanks much,
Dylan

Re: bash variable names do not comply w/POSIX character set rules

[Linda Walsh]

Eduardo A. Bustamante López wrote:

 This definition (


http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap03.html#tag_03_230

 ) states:

  3.230 Name

  In the shell command language, a word consisting solely of underscores,
  digits, and alphabetics from the portable character set. The first 

character

  of a name is not a digit.


  (1) -- It appears you /accidently/ left out part of the text under
section 3.230.  The full text:


 3.230 Name

 In the shell command language, a word consisting solely of
 underscores, digits, and alphabetics from the portable character
 set. The first character of a name is not a digit.

 Note: The Portable Character Set is defined in detail in
 P̲o̲r̲t̲a̲b̲l̲e̲ ̲C̲h̲a̲r̲a̲c̲t̲e̲r̲ ̲S̲e̲t̲⁽¹⁾

[§̲⁽¹⁾ 
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap06.html#tag_06_01 
]

 3.231 ...[next section]



   Thank-you.  This slightly clarifies matters as it only
requires the POSIX source.  At the location pointed to by
the hyper-link for "Portable Character Set" under section 6.1
sentences 2-4, it states:


 Each supported locale shall include the portable character set,
 which is the set of symbolic names for characters in Portable
 Character Set. This is used to describe characters within the text
 of IEEE Std 1003.1-2001. The first eight entries in Portable
 Character Set are defined in the ISO/IEC 6429:1992 standard and
 the rest of the characters are defined in the ISO/IEC 10646-1:2000
 standard. 


   FWIW, in full disclosure, in the last dotted paragraph before the
last sentence of section 6.1, there is a requirement that the alphabetic
character fit within 1 byte -- i.e. only characters in what is commonly
called the "Extended ASCII character set" (ex. ISO-8859-1) seem to be
required.  Note, the character 'Ø' is 1 byte.  So, as the quoted
section above mentions using [basically], the Unicode table for "symbolic
names", it doesn't prescribe a specific encoding. I.e. - While the
reference is to ISO-10646 (Unicode), it does not require a
specific encoding. 


   For Unicode values 0-255, ISO-8859-1 encodes the first 256
bytes of Unicode with 1 byte (satisfying the 1-byte posix constraint,
though it is not able to encode Unicode values >=256, which makes
posix's reference to ISO-10646 somewhat specious as only the 1st
256 values can be encoded in 1 byte (that I am aware of).

   Nevertheless, the symbolic name "LATIN CAPITAL LETTER O WITH STROKE
(o slash)" or 'U+00D8' is classified as an alphabetic, which is a subset
of the "alphanumeric" requirement of POSIX. 


   Note under section 9.3.5 "RE Bracket Expression", subsection 6:


 The following character class expressions shall be supported in
 all locales:

 [:alnum:]   [:cntrl:]   [:lower:]   [:space:]
 [:alpha:]   [:digit:]   [:print:]   [:upper:]
 [:blank:]   [:graph:]   [:punct:]   [:xdigit:]

 In addition, character class expressions of the form:

 [:name:]

 are recognized in those locales where the name keyword has been
 given a charclass definition in the LC_CTYPE category.


Note that "aØb" is classified as fully "alphabetic" by bash's
character-class matching facility -- whether in UTF-8 or ISO-8859-1:


 echo $LC_CTYPE

en_US.ISO-8859-1
LC_CTYPE=en_US.UTF-8
...

 declare -xg a=$(echo -n $'\x61\xd8\x62')
 declare -xg b=${a}c
 [[ $a =~ ^[[:alpha:]]+$ ]] && echo alpha

alpha
 [[ $a =~ ^[[:alnum:]]+$ ]] && echo alnum   

alnum

 [[ $b =~ ^[[:alpha:]]+$ ]] && echo alpha

alpha

 [[ $b =~ ^[[:alnum:]]+$ ]] && echo alnum

alnum

Notice bash classifies the string "aØb" as an alphanumeric AND
as an alphabetic character.  I.e.  bash, itself, claims that
"aØb" is a valid identifier.

Also note, it accepts "aØb" as a var and as an environment var
when used indirectly:


 declare -xg $a='a"slash-O"b'
 declare -xg $b='ab"slash-O"c'
 env|/usr/bin/grep -P '^[ab]...?'|hexdump -C

  61 d8 62 63 3d 61 62 22  73 6c 61 73 68 2d 4f 22  |aab"slash-O"|
0010  63 0a 61 d8 62 3d 61 22  73 6c 61 73 68 2d 4f 22  |c.a"slash-O"|
0020  62 0a 61 3d 61 d8 62 0a  62 3d 61 d8 62 63 0a |b.a=a=a|
002f

===



...
 So no, it does not mandate arbitrary unicode alphabetics. Only the 

ones listed

 there.


   Thank-you.  This better makes the case, as it only refers to
the POSIX reference pages.  But it seems that it boils down to the
allowed definition of envirionment variables:
(http://pubs.opengroup.org/onlinepubs/9699919799/)


 2.5.3 Shell Variables



 Variables shall be initialized from the environment (as defined by
 XBD Environment Variables and the exec function in the System
 Interfaces volume of POSIX.1-2008) and can be given new values
 with variable assignment commands.



The XBD interface is a description of API facilities for programs to use --
not an end-user-interface.  In particular, it says: (under section 8.1)

(http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08)


 Environment variable names used by the utilities in the She

Re: Installation of PDF/PS/DVI and HTML files

[Chet Ramey]

On 12/6/15 4:18 PM, Ludovic Courtès wrote:

>>> As Mathieu wrote, I am of course fine installing man and Info manuals by
>>> default, like GNU packages generally do.
>>>
>>> The suggestion I make is to not install PDF/PS/DVI and HTML files by
>>> default.  
>>
>> Again, only the HTML files are installed by `make install'.  The sticking
>> point here appears to be installing the HTML files,
> 
> Right, HTML and PDF/PS/DVI.

Please read what I wrote.  The PDF/PS/dvi files are not installed by any
target.

>> which you can suppress by running `make install' with htmldir set to
>> the empty string.
> 
> I guess that would work, indeed.
> 
>>> This would comply with the GCS and user expectations, and also
>>> sidestep the bit-for-bit reproducibility issues that generating those
>>> PDF/PS/DVI/HTML files entails.

And now we're back to generating the files.

>> So the problem is once again the build and not the install?  Since the
>> build version appears in the version string, and that changes each time
>> the binary is rebuilt, bit-by-bit reproducibility is not going to be
>> generally possible.
> 
> The broader context is that Debian has a policy of rebootstrapping
> packages; that is, even if there’s a ‘make dist’-produced tarball, they
> will run ‘autoreconf’ et al.
> 
> In the case of Bash, that entails a rebuild of the HTML and PDF/PS/DVI
> files.  

Well, it involves generating them, since up through bash-4.3, the standard
distribution does not include them.  Starting with bash-4.4, they'll be in
the distribution, so that will not be a problem.  (If history is any guide,
though, it will be 2-3 years before Debian has cycled to bash-4.4.)

I'm going to assume that even running autoreconf doesn't change enough
files to cause existing targets to be rebuilt when their source doesn't
change (i.e., it's not run with --force).

There are two problems discussed at
> :
> 
>   1. The man2html copy that Bash provides produces non-deterministic
>  output;

Yes, because of a timestamp.  This will not be a problem if the files don't
have to be generated.

How about this:  I will make sure that shipping the generated
documentation will not cause files to be regenerated, and that this will
work whether the build is done in the source directory or not.

> 
>   2. DVI/PS/PDF generation is not deterministic; this is not a
>  Bash-specific issue, but the fact that Bash tries to install these
>  files by default make the issue visible to Debian.

This is simply false.  You seem to be using generate and install
interchangeably.

See above for the solution to generating these files with `make all'.

> While discussing it with Akira and others, it occurred to me that Bash
> shouldn’t be installing HTML/PDF/PS/DVI by default, at least per my
> understanding of the GCS and its implementation in Automake, hence my
> proposal.

So installing the html files is a separate issue, unrelated to the issues
raised by the `reproducible builds' folks.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Installation of PDF/PS/DVI and HTML files

[Ludovic Courtès]

Chet Ramey  skribis:

> On 12/3/15 6:05 AM, Ludovic Courtès wrote:

[...]

>> In addition, the ‘install’ rule in doc/Makefile.in would need to be
>> split in ‘install-info’, ‘install-pdf’, etc. (as explained in the GCS),
>> with ‘install’ depending only on ‘install-info’.
>
> Again, that's not quite what the coding standards say.  The standards say
> to make sure to install the info file when you run `make install', using
> the `install-info' program to do so.

They also specify targets different from ‘install’ to install
documentation in formats other than Info.  This is consistently done in
many GNU packages, notably GCC/Binutils/GDB/libc, and this is how
Automake implements it.

>> What do you think?
>
> It's hard to say, since the proposed patch has little to do with the
> subject of the message.

Sorry if the patch created confusion.  I hope the intent is clearer now.

Thanks,
Ludo’.

Re: Installation of PDF/PS/DVI and HTML files

[Ludovic Courtès]

Chet Ramey  skribis:

> On 12/4/15 4:24 AM, Ludovic Courtès wrote:
>> Greg Wooledge  skribis:
>> 
>>> On Thu, Dec 03, 2015 at 01:08:13PM +0200, Ludovic Courtès wrote:
 Given that the GCS suggests installing only the Info version of the
 manual by default (info "(standards) Standard Targets")
>>>
 What do you think?
>>>
>>> I think that's a stupid suggestion.  The de facto standard for "make"
>>> followed "make install" on a Unix-like system is to install man pages.
>>> If there's an info page, I have no objection to installing that as well,
>>> but to omit the standard man pages by default is ridiculous.
>> 
>> Agreed; apologies for being unclear.
>> 
>> As Mathieu wrote, I am of course fine installing man and Info manuals by
>> default, like GNU packages generally do.
>> 
>> The suggestion I make is to not install PDF/PS/DVI and HTML files by
>> default.  
>
> Again, only the HTML files are installed by `make install'.  The sticking
> point here appears to be installing the HTML files,

Right, HTML and PDF/PS/DVI.

> which you can suppress by running `make install' with htmldir set to
> the empty string.

I guess that would work, indeed.

>> This would comply with the GCS and user expectations, and also
>> sidestep the bit-for-bit reproducibility issues that generating those
>> PDF/PS/DVI/HTML files entails.
>
> So the problem is once again the build and not the install?  Since the
> build version appears in the version string, and that changes each time
> the binary is rebuilt, bit-by-bit reproducibility is not going to be
> generally possible.

The broader context is that Debian has a policy of rebootstrapping
packages; that is, even if there’s a ‘make dist’-produced tarball, they
will run ‘autoreconf’ et al.

In the case of Bash, that entails a rebuild of the HTML and PDF/PS/DVI
files.  There are two problems discussed at
:

  1. The man2html copy that Bash provides produces non-deterministic
 output;

  2. DVI/PS/PDF generation is not deterministic; this is not a
 Bash-specific issue, but the fact that Bash tries to install these
 files by default make the issue visible to Debian.

While discussing it with Akira and others, it occurred to me that Bash
shouldn’t be installing HTML/PDF/PS/DVI by default, at least per my
understanding of the GCS and its implementation in Automake, hence my
proposal.

I hope this clarifies the context.

Thanks,
Ludo’.

Re: Substring Expansion fails with Debian filemanager gui, run in terminal

[Chet Ramey]

On 12/6/15 11:09 AM, Bruce_Miller wrote:

> Bash Version: 4.3
> Patch Level: 42
> Release Status: release
> 
> Description:
> 
> If clicking on a functioning command line script from a filemanager gui and
> selecting 'run in terminal' , substring expansions will fail in multiple
> versions of Debian. 

The simplest explanation is that the file manager uses /bin/sh instead of
$SHELL to run such commands.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: bug with getting an array index or ...

[Chet Ramey]

On 12/6/15 10:18 AM, konsolebox wrote:
> On Sun, Dec 6, 2015 at 9:38 PM, shawn wilson  wrote:
>> declare -a array=(  ); echo "${!array[@]}";
>> echo "${!array[@]:-}"
>>
>> also, "${!array[@]:foo}" and :+foo and :-foo are all empty as well -
>> I'm pretty sure this is not intended?
>>
> ${!name[@]:-word}, ${!name[@]:+word}, and ${!name[@]:offset:length}
> are not explicitly documented as valid forms of parameter expansion,
> and never has been.

Yes.  The only documented way to get that construct to produce the list of
indices is to use it like ${!array[@]}.

When you use it in a different expansion, you get standard variable
indirection: expand ${array[@]} and try to use it as a variable name.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Substring Expansion fails with Debian filemanager gui, run in terminal

[Bruce_Miller]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDO$
uname output: Linux system-x 4.2.0-19-generic #23-Ubuntu SMP Wed Nov 11
11:39:30 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.3
Patch Level: 42
Release Status: release

Description:

If clicking on a functioning command line script from a filemanager gui and
selecting 'run in terminal' , substring expansions will fail in multiple
versions of Debian.

Repeat-By:

#!/bin/bash

# Set of tests to show how Substring Expansion causes failure when script
is run via the filemanager GUI icon
# but succeeds when run directly from the CLI or if using cut as an
alternative method.


#  Succeeds, no problem with Fedora/Gnome 22 live ...

#  Also works with Knoppix 7, with Xterm from both PCManFM and Dolphin
(Konsole is stable but runs nothing from the gui)


#  Fails with Ubuntu, both Unity & OpenBox, both Nautilus and PCManFM,
15.10 (full install) Both Xterm & Gnome Term
#  Plus Unity/Nautilus on 14.04 live, 12.10 live & 12.04LTS installed

#  Fails with KDE Kubuntu 15.10 live, Konsole crashes the filemanager
(dolphin) Xterm needs install but again fails with the substrings

#  Fails Debian with Kernel 4.0.0-1 on Gnome Gparted rescue CD with fluxbox
& PCManFM

#  Also with Debian 8.2 live stable with Gnome & Nautilus, on Kernel
3.16.7 ('open-with'
xterm not available)

#  Is this a dependancy / build problem ??  Bash is Dash dependant with
Debian ... see https://packages.debian.org/jessie/bash

#  Also refer
https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=bash;dist=unstable


printf "\033c" # clear

StringTest="meaningfulwords"

echo; echo " A brief snooze zzz  could crash if run via Debian/Ubuntu
filemanager... "; sleep 3


# comment out the next two lines to get this script to work when started
from certain file manager GUI

echo; echo "  Test1:  ${StringTest:0:4}" # works CLI only, fails from the
GUI

echo; echo "  Test3: ${StringTest: -5} " # fails from the GUI

#

echo; echo "  Test2: ${#StringTest} " # works for both


echo ; echo $StringTest | cut -c 1,2,3,4   # works

Text1=$( echo $StringTest | cut -c 1,2,3,4 )   # works

Text2=$( echo $StringTest | cut -c 4-6 ) # works & all below

echo; echo "  text1: $Text1 "

echo; echo "  text1b: $( echo $StringTest | cut -c 1,2,3,4 ) "

echo; echo "  text2: $Text2 "


echo; echo " Longer snooze to hold a gui launched terminal open" ; echo;
 sleep 10


Fix:

This doesn't happen with Fedora.

Is this a dependancy / build problem ??  Bash is Dash dependant with Debian
... from reference https://packages.debian.org/jessie/bash

Re: bug with getting an array index or ...

[konsolebox]

On Sun, Dec 6, 2015 at 9:38 PM, shawn wilson  wrote:
> declare -a array=(  ); echo "${!array[@]}";
> echo "${!array[@]:-}"
>
> also, "${!array[@]:foo}" and :+foo and :-foo are all empty as well -
> I'm pretty sure this is not intended?
>
${!name[@]:-word}, ${!name[@]:+word}, and ${!name[@]:offset:length}
are not explicitly documented as valid forms of parameter expansion,
and never has been.

If you do that in bash 4.4, you'll get a bad substitution error message.

bug with getting an array index or ...

[shawn wilson]

declare -a array=(  ); echo "${!array[@]}";
echo "${!array[@]:-}"

also, "${!array[@]:foo}" and :+foo and :-foo are all empty as well -
I'm pretty sure this is not intended?

Re: bash variable names do not comply w/POSIX character set rules

[Eduardo A . Bustamante López]

This definition (
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap03.html#tag_03_230
) states:

  3.230 Name

  In the shell command language, a word consisting solely of underscores,
  digits, and alphabetics from the portable character set. The first character
  of a name is not a digit.

This document has a table of the characters included in the portable character
set:

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap06.html#tag_06_01

Which is a subset of the unicode alphanumerics.

So no, it does not mandate arbitrary unicode alphabetics. Only the ones listed
there.