Re: Global variable modification by nameref chain

[Chet Ramey]

On 6/14/16 12:16 PM, Dan Douglas wrote:
> On Sun, Jun 12, 2016 at 8:33 PM, Chet Ramey  wrote:
>> 3. Honor the assignment and delete the nameref variable, creating a new
>>one, like bash-4.3:
>>
>> $ ../bash-4.3-patched/bash ./x1
>> declare -n a="b"
>> declare -n b="a[1]"
>> declare -a a='([1]="foo")'
>> declare -n b="a[1]"
> 
> I kind of like this option because it's consistent with `typeset -n`
> meaning "operate directly on the ref variable", and declare together
> with assignment meaning "redefine the variable" in most cases. I'm not
> so sure a warning is needed since replacing the variable with a new
> definition would be the usual thing to do with any other attributes,
> and the identifier literals used within one scope should be under
> control.

OK, just to play devil's advocate: why discard the existing value, since
any attribute other than nameref would result in it being preserved as
index 0?

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Global variable modification by nameref chain

[Dan Douglas]

On Sun, Jun 12, 2016 at 8:33 PM, Chet Ramey  wrote:
> 3. Honor the assignment and delete the nameref variable, creating a new
>one, like bash-4.3:
>
> $ ../bash-4.3-patched/bash ./x1
> declare -n a="b"
> declare -n b="a[1]"
> declare -a a='([1]="foo")'
> declare -n b="a[1]"

I kind of like this option because it's consistent with `typeset -n`
meaning "operate directly on the ref variable", and declare together
with assignment meaning "redefine the variable" in most cases. I'm not
so sure a warning is needed since replacing the variable with a new
definition would be the usual thing to do with any other attributes,
and the identifier literals used within one scope should be under
control.

Re: Global variable modification by nameref chain

[Chet Ramey]

On 6/9/16 6:06 AM, Dan Douglas wrote:

>  $ bash -c 'typeset -n a=b b; b=a[1]; a=foo; typeset -p a b' # bash 4.3
> declare -a a='([1]="foo")'
> declare -n b="a[1]"
>  $ ./bash -c 'typeset -n a=b b; b=a[1]; typeset -p a b; a=foo' # 4.4
> declare -n a="b"
> declare -n b="a[1]"
> ./bash: line 0: `a[1]': not a valid identifier
> 
> (That's the confusing error mentioned previously btw)

There are a few ways we can go on this.  Using your script

$ cat x1
typeset -n a=b b
b=a[1]
typeset -p a b
a=foo
typeset -p a b

we can

1. Honor the nameref attribute and reject the assignment, as the current
   devel git branch does:

$ ../bash-20160603/bash ./x1
declare -n a="b"
declare -n b="a[1]"
./x1: line 4: `a[1]': not a valid identifier
declare -n a="b"
declare -n b="a[1]"

2. Honor the assignment and remove the nameref attribute, treating the
   variable as a scalar:

$ ./bash ./x1
declare -n a="b"
declare -n b="a[1]"
./x1: line 4: warning: a: removing nameref attribute
declare -a a=([0]="b" [1]="foo")
declare -n b="a[1]"

3. Honor the assignment and delete the nameref variable, creating a new
   one, like bash-4.3:

$ ../bash-4.3-patched/bash ./x1
declare -n a="b"
declare -n b="a[1]"
declare -a a='([1]="foo")'
declare -n b="a[1]"

4. Honor the assignment and remove the nameref attribute and value, but
   with a warning:

$ ./bash ./x1
declare -n a="b"
declare -n b="a[1]"
./x1: line 4: warning: a: removing nameref attribute
declare -a a='([1]="foo")'
declare -n b="a[1]"

5. The bizarre ksh93 behavior is always an option:

$ ksh93 ./x1
typeset -n a=.deleted
typeset -n -a b=('b[1]')
./x1: line 4: .deleted: is read only



-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Global variable modification by nameref chain

[Dan Douglas]

On Thu, Jun 9, 2016 at 4:34 AM, Dan Douglas  wrote:
> How about just doing it similar to the way mksh resolves arithmetic
> variable loops? As each variable is visited, add it to a list (or hash
> set) and check whether that node was already visited. That should work
> without having to consider scope issues.
>
> $ mksh -c 'a=b b=c c=d d=1; echo $((a))'
> 1
> $ mksh -c 'a=b b=c c=d d=a; echo $((a))'
> mksh: a: expression recurses on parameter 'a'
>
> This nicely says which variable was to blame. I've noticed bash devel
> right now can give messages about "invalid identifiers" which are
> technically valid in most contexts so it's tough to track down
> problems in complicated reference chains.

Sorry one more follow-up. I should note bash has allowed deep
recursion in variable loops for ages using `${!var}` indirection, even
for arrays, and so long as you were careful about unbounded recursion
that never caused a problem before. This would be a
backwards-compatibility breaker if it applied generally. I don't think
namerefs need to be fantastically different from the old indirection
semantics so long as we're ignoring scope.

The only difference between namerefs and old-style indirection is that
an individual array element can't be a nameref and a nameref can't be
an array, so you need to prevent something like this which bash
already does...

 $ bash -c 'typeset -n a=b b; b=a[1]; a=foo; typeset -p a b' # bash 4.3
declare -a a='([1]="foo")'
declare -n b="a[1]"
 $ ./bash -c 'typeset -n a=b b; b=a[1]; typeset -p a b; a=foo' # 4.4
declare -n a="b"
declare -n b="a[1]"
./bash: line 0: `a[1]': not a valid identifier

(That's the confusing error mentioned previously btw)

Re: Global variable modification by nameref chain

[Dan Douglas]

How about just doing it similar to the way mksh resolves arithmetic
variable loops? As each variable is visited, add it to a list (or hash
set) and check whether that node was already visited. That should work
without having to consider scope issues.

$ mksh -c 'a=b b=c c=d d=1; echo $((a))'
1
$ mksh -c 'a=b b=c c=d d=a; echo $((a))'
mksh: a: expression recurses on parameter 'a'

This nicely says which variable was to blame. I've noticed bash devel
right now can give messages about "invalid identifiers" which are
technically valid in most contexts so it's tough to track down
problems in complicated reference chains.

Re: Global variable modification by nameref chain

[Dan Douglas]

On Sat, Jun 4, 2016 at 5:48 PM, Grisha Levit  wrote:
> On May 23, 2016 1:42 PM, "Chet Ramey"  wrote:
>> > Should the assignment work?  I'm considering changing the
>> > assignments to
>> > work more like the references.
>> >
>> > I think it would be useful for the assignment to work, as that allows
>> > functions to take variable names as arguments without worrying about
>> > name
>> > collisions.
>>
>> I don't like the fact that variable binding simply ignores nameref loops
>> and treats them the same as the variable not being found.  That's the
>> difference here.
>
> What if (in a function scope) a nameref self-reference were unambiguously
> treated as referring to a variable in the next higher scope, other that
> declare [+-] on an existing ref?

I've thought about this as a workaround but resolving loops that way
will probably lead to problems. Needless to say I prefer doing it the
way I previously described where ref=ref and ref=$1 are distinct.
That's separate from the subject of nameref loops between locals of
the same scope of course.

You can already see some of the eventual problems in ksh if you try to
define a nameref within a compound that points to a local and pass the
compound by reference or copy it between functions using typeset
-c/-m. It's a huge mess of bugs.

Re: Global variable modification by nameref chain

[Grisha Levit]

On May 23, 2016 1:42 PM, "Chet Ramey"  wrote:
> > Should the assignment work?  I'm considering changing the
assignments to
> > work more like the references.
> >
> > I think it would be useful for the assignment to work, as that allows
> > functions to take variable names as arguments without worrying about
name
> > collisions.
>
> I don't like the fact that variable binding simply ignores nameref loops
> and treats them the same as the variable not being found.  That's the
> difference here.

What if (in a function scope) a nameref self-reference were unambiguously
treated as referring to a variable in the next higher scope, other that
declare [+-] on an existing ref?

Re: Global variable modification by nameref chain

[Chet Ramey]

On 5/23/16 11:32 AM, Grisha Levit wrote:
> 
> On Sun, May 22, 2016 at 10:08 PM, Chet Ramey  > wrote:
> 
> Should the assignment work?  I'm considering changing the assignments to
> work more like the references.
> 
> 
> I think it would be useful for the assignment to work, as that allows
> functions to take variable names as arguments without worrying about name
> collisions.  Would it also be possible to have the reference resolve the
> same way?  If not, then I suspect consistency in assignments and references
> would be preferable.

I don't like the fact that variable binding simply ignores nameref loops
and treats them the same as the variable not being found.  That's the
difference here.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Global variable modification by nameref chain

[Grisha Levit]

On Sun, May 22, 2016 at 10:08 PM, Chet Ramey  wrote:

> Should the assignment work?  I'm considering changing the assignments to
> work more like the references.
>

I think it would be useful for the assignment to work, as that allows
functions to take variable names as arguments without worrying about name
collisions.  Would it also be possible to have the reference resolve the
same way?  If not, then I suspect consistency in assignments and references
would be preferable.

Re: Global variable modification by nameref chain

[Chet Ramey]

On 5/22/16 6:06 PM, Grisha Levit wrote:

> The expansion part really seems like the bug here, and can be demonstrated
> without any crazy cross-scope chains.
> 
> As I understand it, the intention is indeed to allow namerefs to point to
> outer-scope variables, even if they happen to have the same name as the
> reference, since a function like
> 
> |add_X() { local -n ref=$1; ref+=X; } |
> 
> works correctly even if it is passed |ref| as the argument:

That was the original intent, but I am now reconsidering it.  ksh93 doesn't
seem to allow nameref self-references at all, and I am thinking about
whether or not that is the best way to avoid problems like this.

> |$ ref=; add_X ref; echo "$ref" X |
> 
> However, if we try to access |$ref| inside the function, we get an error,
> though the += operation works just fine:
> 
> |$ add_X_echo() { local -n ref=$1; ref+=X; echo "$ref"; } $ ref=;
> add_X_echo; echo "$ref" bash: warning: ref: circular name reference X |

Should the assignment work?  I'm considering changing the assignments to
work more like the references.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Global variable modification by nameref chain

[Grisha Levit]

The nameref resolution search doesn’t re-start back at the original
context. It’s like a symbolic link — it continues at the same context as
the resolved nameref.

If the assignment to the global variable is intentional, then shouldn’t
expansion use the global variable’s value as well?

The expansion part really seems like the bug here, and can be demonstrated
without any crazy cross-scope chains.

As I understand it, the intention is indeed to allow namerefs to point to
outer-scope variables, even if they happen to have the same name as the
reference, since a function like

add_X() { local -n ref=$1; ref+=X; }

works correctly even if it is passed ref as the argument:

$ ref=; add_X ref; echo "$ref"
X

However, if we try to access $ref inside the function, we get an error,
though the += operation works just fine:

$ add_X_echo() { local -n ref=$1; ref+=X; echo "$ref"; }
$ ref=; add_X_echo; echo "$ref"
bash: warning: ref: circular name reference

X

​

Re: Global variable modification by nameref chain

[Dan Douglas]

On Fri, May 6, 2016 at 8:28 AM, Grisha Levit  wrote:
> The issue I was trying to raise is that assignment modifies the global
> variable but expansion uses the local value.
> If the assignment to the global variable is intentional, then shouldn't
> expansion use the global variable's value as well?

Nope. There's no workaround other than to unset every variable with a
conflicting name that lies between you and the global. Even if you
want to try that, there's no way to know how many such variables there
are and which one is currently visible.

declare -g is still sometimes useful though. I use it to set bash
special variables and configurations in bashrc to better ensure
they're being put into the actual global scope and should apply
everywhere. `declare -gn` is a little unusual since normally you
wouldn't use -n with other options, but I can see the use for `-gn`.

Re: Global variable modification by nameref chain

[Chet Ramey]

On 4/27/16 9:05 AM, Grisha Levit wrote:
> Is the behavior below expected? In the presence of a local $var, the
> /global/ $var can be modified by assigning a value to a local nameref that
> points to a global nameref that points to $var. However, the local nameref
> expands to the /local/ value of $var.

Yes, that's how it works.  The nameref resolution search doesn't re-start
back at the original context.  It's like a symbolic link -- it continues
at the same context as the resolved nameref.

I don't have any plans to change this before bash-4.4.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Global variable modification by nameref chain

[Grisha Levit]

Is the behavior below expected? In the presence of a local $var, the
*global* $var can be modified by assigning a value to a local nameref that
points to a global nameref that points to $var. However, the local nameref
expands to the *local* value of $var.

lref --> gref --> var

Example:

var=Gf() {
   local var=L
   declare -gn gref=var
   declare -n lref=gref
   echo "Writing 'F' to lref"
   lref=F
   echo "lref points to \$${!lref} and expands to '$lref'"
}echo "Global \$var expands to '$var'"
fecho "Global \$var expands to '$var'"

Global $var expands to *'G'*
Writing *'F'* to lref
lref points to $var and expands to *'L'*
Global $var expands to *'F'*

​