Re: namref dicussion continued: [PATCH] export -r reference

[Dan Douglas]

Oh I see you talked about some of this already in the "chained
nameref" thread. I haven't read that one yet... I wonder why `export
-r` requires a different solution because it's pretty much the same
underlying problem.

Re: namref dicussion continued: [PATCH] export -r reference

[Dan Douglas]

On Thu, May 5, 2016 at 2:46 AM, Piotr Grzybowski  wrote:
>  ok, so Dan wants this patch.

Yes I think your bash patch is probably an improvement at least in
this particular case.

> Maybe you can comment on wether the patches are valid.

The posix export and readonly don't produce locally scoped variables
at all, so it's expected that e.g. `typeset -x` and `export` will
differ.

This is yet another problem that's better solved by immutable C++-like
references. At least, the problem goes away when variables are passed
explicitly.

The exact mechanism isn't really documented. When ksh determines that
a nameref assignment is to a positional parameter during evaluation,
you get a completely different type of reference. That is, if `$1`
expands to `x`, then `typeset -n ref=$1` and `typeset -n ref=x` aren't
equivalent like they are in bash.  In the former case, any use of
`ref` except for `typeset +n` and `unset -n` applies to the exact
object that `$1` referred to at the time the reference was created,
even if it's another nameref in which case that reference is followed.
The latter gives you a bash-like dynamic reference that points to
somewhere within either the current scope or the global scope (due to
static scoping) unless it happens to point to another reference
parameter of course.

When you changed that `typeset` line to `export -r _x`, things become
less clear. Ksh sees that _x is a reference parameter and because of
the rules it does the only thing that makes sense which is apply the
export attribute to whatever was passed to the function for both
`export` or `typeset -x`, though there may be some other differences
between those.

Bash on the other hand never does what you'd want for either `export`
or `typeset -x`. For typeset, it tears down the reference and
redefines it as a new local with the export attribute. It does that
indiscriminately for any usage of typeset, which means you can't
correctly modify an attribute by reference in bash. If you actually
wanted that result, the correct way would be to first use `typeset +n
_x` followed by `typeset -x _x`. If you choose `export`, the correct
behavior is even less clear thanks to dynamic scope.

Your patch is a partial solution that makes `export` modify the
attribute in the way you would expect in the absence of the reference
parameter feature. Ignoring namerefs, the only other way for bash to
modify the attribute of the currently visible dynamically scoped local
without declaring a new local is to use export, which is why bash
behaves as it did. This solution doesn't solve the same problem for
attributes other than export and readonly. Bash can't use e.g.
`typeset -u` by reference. It also doesn't solve the many other
problems caused by this like name conflicts.

Re: declare [-+]n behavior on existing (chained) namerefs

[Chet Ramey]

On 4/28/16 9:49 PM, Grisha Levit wrote:

> In a slightly different version, with `declare -n r; r=a', the function
> exits with code 1 after the `r=a' statement:
> 
> $ declare -nt r=a; f() { declare a; declare -n r; r=a; declare -p a r; };
> f; echo $?
> 1

In this case, you create a self-referencing local nameref, which ends up
resolving to nothing, which causes an assignment error, which results in
function execution being aborted.  You create the self-referencing local
nameref because bash follows the nameref chain for `r', and resolves it
to `a'.  It's as Piotr surmised: as if you ran `typeset -n a; a=a'.

When asked to create a local variable that shadows a nameref from a
different context, declare needs to throw away the results of that nameref
chain search and just create a local variable.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: History modifier :p ignored prior to another history reference

[Piotr Grzybowski]

On 5 May 2016, at 20:08, Piotr Grzybowski wrote:

> [..]
> Currently the design indicates: :p applies to the whole line.

 As Dean just has pointed out to me, this sentence is obviously wrong. it 
should be:

Currently the design indicates: the last :p applies to the whole line, the 
information about existence of others before it, is overwritten.

 sorry,
pg


> On 3 May 2016, at 18:15, Dean Stanton wrote:
> 
>> Configuration Information [Automatically generated, do not change]:
>> Machine: x86_64
>> OS: linux-gnu
>> Compiler: gcc
>> Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
>> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' 
>> -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' 
>> -DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include 
>> -I./lib  -D_GNU_SOURCE -DRECYCLES_PIDS  -O2 -g -pipe -Wall 
>> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
>> --param=ssp-buffer-size=4 -m64 -mtune=generic
>> uname output: Linux dstanton-vm.tintri.com 2.6.32.26-175.fc12.x86_64 #1 SMP 
>> Wed Dec 1 21:39:34 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux
>> Machine Type: x86_64-redhat-linux-gnu
>> 
>> Bash Version: 4.0
>> Patch Level: 38
>> Release Status: release
>> 
>> Description:
>> History modifier :p is not honored when it appears prior to another history 
>> reference on the same command line.
>> 
>>I tried to find a website with known bash bugs (to search to see if 
>> this issue was already discussed), but didn't find such a site.
>> 
>> Repeat-By:
>> $ echo one two three four
>>one two three four
>>$ echo !:2:p ; echo !$  # :p was ignored!
>>echo two ; echo four
>>two
>>four
>> 
>> $ echo one two three four
>>one two three four
>>$ echo !:p !!   # :p was ignored!
>> echo one two three four echo one two three four
>> one two three four echo one two three four
>> 
>> 
>>I expected the entire command line to be only printed and not 
>> executed. The manpage says
>>   p  Print the new command but do not execute it.
>>Instead, the :p was ignored and the entire command line was (printed 
>> and then) executed.
>> 
>>$ echo one two three four
>>one two three four
>>$ echo !:2 ; echo !$:p  # :p was heeded.
>>echo two ; echo four
>> $ echo one two three four
>> $ echo !:1:p !$
>> # :p was ignored
>> echo one four
>> one four
>> 
>> 
> 

Re: declare [-+]n behavior on existing (chained) namerefs

[Chet Ramey]

On 5/2/16 1:55 PM, Piotr Grzybowski wrote:
> Hi,
> 
>  I hope the attached patch does not break to much; it addresses the masking:
> 
>> declare -nt r=a; f() { declare a; declare -n r=a; declare -p a r; }; f
> 
> as per Grisha's report; it tries to enlighten make_local_variable to the 
> existence of nameref by returning existing local nameref in scope (declare -n 
> r=PATH; declare -n r; returns r with value of PATH) and by (hopefully) 
> correctly making existing nameref take precedence over the dereferenced value.
> 
>  Would you please consider it?

Thanks, this is a good fix.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: declare [-+]n behavior on existing (chained) namerefs

[Chet Ramey]

On 4/29/16 12:50 PM, Grisha Levit wrote:

> The only buggy behavior when inside of functions (ignoring scope issues)
> seems to be that `declare -n ref’ unsets the value of $ref if it previously
> pointed to a variable that is unset.
> 
> |$ f() { unset var; declare -n ref=var; declare -n ref; declare -p ref; }; f
> declare -n ref # why did the value of ref disappear? $ f() { local var;
> declare -n ref=var; declare -n ref; declare -p ref; }; f declare -n ref="var" 
> |

I fixed this as part of making other changes prompted by the discussion.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: param expansion with single-character special vars in the environment

[Chet Ramey]

On 5/4/16 3:37 PM, Piotr Grzybowski wrote:
> 
> On 4 May 2016, at 17:51, Chet Ramey wrote:
> 
>> The issue I'm thinking about currently is whether or not to allow nameref
>> variables to have numeric (integer) values.  bash-4.3 didn't allow those
>> values to be expanded, but allowed them to be assigned.  It doesn't seem
>> harmful to change the error to assignment time.
> 
> I vote for error at assignment, whats the point of allowing pointless 
> assignment?

No reason, except for minimal backwards compatibility.  I'm leaning towards
that position myself.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: set isearch-terminators "\r" in .inputrc causes malloc assert fails/death

[Piotr Grzybowski]

hi,

 I cannot replicate it in anyway on mac os x or linux.
 Could you please define "sometimes", and supply the exact version of bash that 
shows this behavior, and the version you refer to as "my old bash"? I have a 
distinct feeling that it is something different than my old bash.

cheers,
pg


On 4 May 2016, at 19:43, Britton Kerin wrote:

> I tried with export INPUTRC=test_inputrc, where test_inputrc contains
> just this:
> 
> set isearch-terminators "\r"
> 
> this causes bash to sometimes spit out things like this:
> 
> bkerin@debian:~$ ls
> 
> malloc: .././variables.c:2497: assertion botched
> malloc: block on free list clobbered
> last command: ls
> Aborting...Aborted
> 
> also sometimes it just dies and takes the terminal with it.
> 
> I'm not sure if this is specific to using \r as an isearch terminator but I
> suspect it is since hitting return normaly causes bash to fire off the command
> you're searching for.  Not sure though.
> 
> My old bash didn't do this, if anyone cares enough to ask I can bring
> that computer back up and check the bash/readline version to narrow
> down the search where this bug crept in.
> 
> I'm really hoping for a fix as reverse search followed by enter to
> bring a command back onto the command line for subsequent editing is
> deeply ingrained for me, I keep re-executing old commands.
> 
> Here is my debian system info:
> 
> -- System Information:
> Debian Release: 8.3
>  APT prefers stable-updates
>  APT policy: (500, 'stable-updates'), (500, 'stable')
> Architecture: amd64 (x86_64)
> 
> Kernel: Linux 4.4.5.emp3 (SMP w/8 CPU cores)
> Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/bash
> Init: systemd (via /run/systemd/system)
> 
> Versions of packages bash depends on:
> ii  base-files   8+deb8u3
> ii  dash 0.5.7-4+b1
> ii  debianutils  4.4+b1
> ii  libc62.19-18+deb8u3
> ii  libncurses5  5.9+20140913-1+b1
> ii  libtinfo55.9+20140913-1+b1
> 
> Versions of packages bash recommends:
> ii  bash-completion  1:2.1-4
> 
> Versions of packages bash suggests:
> pn  bash-doc  
> 
> -- no debconf information
> 
> 
> Thanks,
> Britton
> 

Re: namref dicussion continued: [PATCH] export -r reference

[Piotr Grzybowski]

 ok, a small correction, the problem lies in exportscope:40, but none of the 
patches mentioned solve it, the same behaviour of the test can be recovered 
with just export instead of declare on this line (at least proves that the 
patches dont break it ;-)). some other make_local_variable issue I guess.

pg


On 5 May 2016, at 09:46, Piotr Grzybowski wrote:

> 
> ok, so Dan wants this patch.
> I took a look at your script, there are two cases it addresses: exporting 
> nameref and doing it from inside the function scope, I applied both patches:
> 
> nameref masking in scope:
> https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00012.html
> 
> exporting namerefs:
> https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00024.html
> 
> and with this modification to your script:
> 
> diff --git a/exportscope b/exportscope
> index 05bae73..0125681 100644
> --- a/exportscope
> +++ b/exportscope
> @@ -37,7 +37,8 @@ function h {
>;;
>2)
>typeset -n _x=$2
> -   typeset -x _x='in h2'
> +   export -r _x;
> +   _x='in h2';
>eval "$showEnv" h2
>esac
> }
> @@ -75,4 +76,4 @@ for sh in bash ksh mksh zsh; do
>printf '%s\n\n' "$("$sh" -c "$code" -- "$sh" 2>&1)"
> done
> 
> You can get (difference in test 3):
> 
> #./exportscope 
> bash test 1:
>subprocess f: declare -x x="in f"
>local f: declare -x x="in f"
>global: declare -x x="global"
> bash test 2:
>subprocess g: declare -x x="in g"
>subprocess f: declare -x x="in f"
>local f: declare -x x="in f"
>local g: declare -x x="in g"
>global: declare -- x="unset"
> bash test 3:
>subprocess h2: declare -x x="in h2"
>subprocess h1: declare -x x="in h2"
>local h1: declare -x x="in h2"
> 
> Maybe you can comment on wether the patches are valid.
> 
> cheers,
> pg
> 
> 
> 
> On 4 May 2016, at 22:40, Dan Douglas wrote:
> 
>> Yeah I was just looking for this old script last night and just found it :)
>> 
>> https://gist.github.com/ormaaj/04923e11e8bdc27688ad
>> 
>> If you scroll down to the output for "test 3" where "h" gets called
>> and passes a local "x" to a function that creates a reference to it
>> and exports the reference you can see that bash calls "x" unset in
>> both the first and second scope. As I recall we were discussing the
>> way exported locals interact between scopes at the time, not namerefs,
>> but I added the case for namerefs since I wasn't sure how this should
>> work. I should probably run it again with the current devel branch.
>> 
>> Even though bash doesn't yet support references for parameters like
>> ksh93, neither does mksh, which also shows differences from bash.
> 

Re: namref dicussion continued: [PATCH] export -r reference

[Piotr Grzybowski]

 ok, so Dan wants this patch.
 I took a look at your script, there are two cases it addresses: exporting 
nameref and doing it from inside the function scope, I applied both patches:

nameref masking in scope:
https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00012.html

exporting namerefs:
https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00024.html

and with this modification to your script:

diff --git a/exportscope b/exportscope
index 05bae73..0125681 100644
--- a/exportscope
+++ b/exportscope
@@ -37,7 +37,8 @@ function h {
;;
2)
typeset -n _x=$2
-   typeset -x _x='in h2'
+   export -r _x;
+   _x='in h2';
eval "$showEnv" h2
esac
 }
@@ -75,4 +76,4 @@ for sh in bash ksh mksh zsh; do
printf '%s\n\n' "$("$sh" -c "$code" -- "$sh" 2>&1)"
 done

You can get (difference in test 3):

#./exportscope 
bash test 1:
subprocess f: declare -x x="in f"
local f: declare -x x="in f"
global: declare -x x="global"
bash test 2:
subprocess g: declare -x x="in g"
subprocess f: declare -x x="in f"
local f: declare -x x="in f"
local g: declare -x x="in g"
global: declare -- x="unset"
bash test 3:
subprocess h2: declare -x x="in h2"
subprocess h1: declare -x x="in h2"
local h1: declare -x x="in h2"

 Maybe you can comment on wether the patches are valid.

cheers,
pg



On 4 May 2016, at 22:40, Dan Douglas wrote:

> Yeah I was just looking for this old script last night and just found it :)
> 
> https://gist.github.com/ormaaj/04923e11e8bdc27688ad
> 
> If you scroll down to the output for "test 3" where "h" gets called
> and passes a local "x" to a function that creates a reference to it
> and exports the reference you can see that bash calls "x" unset in
> both the first and second scope. As I recall we were discussing the
> way exported locals interact between scopes at the time, not namerefs,
> but I added the case for namerefs since I wasn't sure how this should
> work. I should probably run it again with the current devel branch.
> 
> Even though bash doesn't yet support references for parameters like
> ksh93, neither does mksh, which also shows differences from bash.