Re: Tilde expansion during command search
On Friday, August 01, 2014 06:20:28 AM Mike Frysinger wrote: On Wed 23 Jul 2014 08:51:19 Dan Douglas wrote: On Wednesday, July 23, 2014 09:28:02 AM you wrote: On 7/23/14, 8:22 AM, Dan Douglas wrote: Hi, from this discussion: https://github.com/koalaman/shellcheck/issues/195#issuecomment-49678200 I can't find any reference that says substituting a literal tilde in PATH should occur during command search. Bash has always done this, even back to the pre-version 1 days, and I don't see any reason to change it now. The only concerns I can think of are inconsistency with programs that use execvp(), or possibly double-expansion in the event of a user name or any path containing ~. how so ? execvp doesn't do tilde expansion. only the shell does. That's the problem. A program using execvp should be able to expect the same result as that of the shell for a given PATH value. Because of this, it can't, unless exec has the same implementation. and it does it once at assignment. The example from my first post demonstrates that Bash substitutes a literal tilde in PATH during the actual path search before executing a command, in addition to during assignments and word expansions. -- Dan Douglas signature.asc Description: This is a digitally signed message part.
Re: Tilde expansion during command search
On 08/01/2014 07:53 AM, Dan Douglas wrote: how so ? execvp doesn't do tilde expansion. only the shell does. That's the problem. A program using execvp should be able to expect the same result as that of the shell for a given PATH value. Because of this, it can't, unless exec has the same implementation. and it does it once at assignment. The example from my first post demonstrates that Bash substitutes a literal tilde in PATH during the actual path search before executing a command, in addition to during assignments and word expansions. And that bash is the odd one out in comparison to all other shells. I think bash is buggy for performing tilde expansion during PATH lookup for literal tildes contained in PATH, because no other shell does it, and POSIX does not specify that it would happen. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Tilde expansion during command search
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 8/1/14, 11:26 AM, Eric Blake wrote: On 08/01/2014 07:53 AM, Dan Douglas wrote: how so ? execvp doesn't do tilde expansion. only the shell does. That's the problem. A program using execvp should be able to expect the same result as that of the shell for a given PATH value. Because of this, it can't, unless exec has the same implementation. and it does it once at assignment. The example from my first post demonstrates that Bash substitutes a literal tilde in PATH during the actual path search before executing a command, in addition to during assignments and word expansions. And that bash is the odd one out in comparison to all other shells. I think bash is buggy for performing tilde expansion during PATH lookup for literal tildes contained in PATH, because no other shell does it, and POSIX does not specify that it would happen. Again: if it were a real problem, you'd think it would have come up more often in the past 25 years since it was implemented. - -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlPbsi0ACgkQu1hp8GTqdKtKGwCZAa+2Js8aibplmc17DPEBtEYd NQUAni/qbNXIQLiTQRfW052G9asC3lnY =oOKU -END PGP SIGNATURE-
Re: Tilde expansion during command search
On Wed, Jul 23, 2014 at 11:00 PM, lolilolicon loliloli...@gmail.com wrote: On Wed, Jul 23, 2014 at 10:43 PM, Eric Blake ebl...@redhat.com wrote: But we are talking about the case where ~ IS quoted during the assignment, and only bash then re-expands it during path lookup. That's my point. Let me try quoting this again, Note that the tildes are expanded during the assignment to PATH, _not when PATH is accessed during command search._ in which I was trying to imply that, Bash *does* expand the tildes _when PATH is accessed during command search._ I'm sorry if it felt like I was taking that sentence out of context. Well, I was; but I think it's fair to assume that whoever wrote that sentence would agree that tilde expansion should not be performed when PATH is accessed during command search. If the tilde were escaped when the PATH were assigned, should we give the tilde a second chance and expand it when the PATH were accessed? To which POSIX would probably say, Hell no. We're reluctant enough to allow tilde expansion in assignments already! How reluctant? If you look at the spec[1], you will see things like: Tilde expansion generally occurs only at the beginning of words, but an exception based on _historical practice_ has been included: and more directly Consideration was given to prohibiting this behaviour because any of the following are reasonable substitutes: So, has anyone asked the POSIX people about this? [1] http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#tag_001_006_001
Re: Tilde expansion during command search
On Thursday, July 24, 2014 02:34:13 PM lolilolicon wrote: On Wed, Jul 23, 2014 at 11:00 PM, lolilolicon loliloli...@gmail.com wrote: On Wed, Jul 23, 2014 at 10:43 PM, Eric Blake ebl...@redhat.com wrote: But we are talking about the case where ~ IS quoted during the assignment, and only bash then re-expands it during path lookup. That's my point. Let me try quoting this again, Note that the tildes are expanded during the assignment to PATH, _not when PATH is accessed during command search._ in which I was trying to imply that, Bash *does* expand the tildes _when PATH is accessed during command search._ I'm sorry if it felt like I was taking that sentence out of context. Well, I was; but I think it's fair to assume that whoever wrote that sentence would agree that tilde expansion should not be performed when PATH is accessed during command search. The section you quoted isn't present at all in issue 7. The current version of the spec makes tilde expansion within assignments mandatory, including multiple tilde expansions when there are colons. The part that describes how that works moved to XRAT section 2.6.1. However the part that refers to what happens during command search appears to have been removed. I agree that if the quoted sentence were present it would probably imply that this behavior is explicitly disallowed. -- Dan Douglas
Tilde expansion during command search
Hi, from this discussion:
https://github.com/koalaman/shellcheck/issues/195#issuecomment-49678200
I can't find any reference that says substituting a literal tilde in PATH
should occur during command search.
$ ( cd ~
mkdir tmpdir
echo $'#!/bin/sh\necho Hello from ${BASH_SOURCE}!' ~/tmpdir/testscript
chmod u+x ~/tmpdir/testscript
PATH=\~/tmpdir
typeset -p PATH
testscript )
declare -x PATH=~/tmpdir
Hello from /home/devtest/tmpdir/testscript!
$ echo $BASH_VERSION
4.3.18(1)-release
--
Dan Douglas
Re: Tilde expansion during command search
On 7/23/14, 8:22 AM, Dan Douglas wrote: Hi, from this discussion: https://github.com/koalaman/shellcheck/issues/195#issuecomment-49678200 I can't find any reference that says substituting a literal tilde in PATH should occur during command search. Bash has always done this, even back to the pre-version 1 days, and I don't see any reason to change it now. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Tilde expansion during command search
On Wednesday, July 23, 2014 09:28:02 AM you wrote: On 7/23/14, 8:22 AM, Dan Douglas wrote: Hi, from this discussion: https://github.com/koalaman/shellcheck/issues/195#issuecomment-49678200 I can't find any reference that says substituting a literal tilde in PATH should occur during command search. Bash has always done this, even back to the pre-version 1 days, and I don't see any reason to change it now. The only concerns I can think of are inconsistency with programs that use execvp(), or possibly double-expansion in the event of a user name or any path containing ~. You're probably right in that it's not super critical if it hasn't caused problems so far. -- Dan Douglas
Re: Tilde expansion during command search
On 07/23/2014 07:51 AM, Dan Douglas wrote: On Wednesday, July 23, 2014 09:28:02 AM you wrote: On 7/23/14, 8:22 AM, Dan Douglas wrote: Hi, from this discussion: https://github.com/koalaman/shellcheck/issues/195#issuecomment-49678200 I can't find any reference that says substituting a literal tilde in PATH should occur during command search. Bash has always done this, even back to the pre-version 1 days, and I don't see any reason to change it now. The only concerns I can think of are inconsistency with programs that use execvp(), or possibly double-expansion in the event of a user name or any path containing ~. You're probably right in that it's not super critical if it hasn't caused problems so far. Might be worth asking the POSIX folks if it is allowed by POSIX. What do other shells do? -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Tilde expansion during command search
On Wednesday, July 23, 2014 07:58:26 AM Eric Blake wrote:
On 07/23/2014 07:51 AM, Dan Douglas wrote:
On Wednesday, July 23, 2014 09:28:02 AM you wrote:
On 7/23/14, 8:22 AM, Dan Douglas wrote:
Hi, from this discussion:
https://github.com/koalaman/shellcheck/issues/195#issuecomment-49678200
I can't find any reference that says substituting a literal tilde in PATH
should occur during command search.
Bash has always done this, even back to the pre-version 1 days, and I don't
see any reason to change it now.
The only concerns I can think of are inconsistency with programs that use
execvp(), or possibly double-expansion in the event of a user name or any
path
containing ~.
You're probably right in that it's not super critical if it hasn't caused
problems so far.
Might be worth asking the POSIX folks if it is allowed by POSIX. What
do other shells do?
I tested it in the link from my first post. AFAICT Bash is the only one.
#!/bin/bash
printf '%s\n' '#!/bin/sh' 'echo test' ~/tmpdir/testscript
chmod u+x ~/tmpdir/testscript
s=(bash ksh mksh posh zsh dash bb jsh)
# Testing for parameter expansion.
for sh in ${s[@]}; do
printf '%-4s %s\n' ${sh}: $($sh -c 'PATH=\${HOME}/tmpdir; testscript'
21)
done
echo
# Testing for tilde expansion.
for sh in ${s[@]}; do
printf '%-4s %s\n' ${sh}: $($sh -c 'PATH=\~/tmpdir; testscript' 21)
done
bash: bash: testscript: command not found
ksh: ksh: testscript: not found
mksh: mksh: testscript: not found
posh: posh: testscript: not found
zsh: zsh:1: command not found: testscript
dash: dash: 1: testscript: not found
bb: bb: testscript: not found
jsh: jsh: testscript: not found
bash: test
ksh: ksh: testscript: not found
mksh: mksh: testscript: not found
posh: posh: testscript: not found
zsh: zsh:1: command not found: testscript
dash: dash: 1: testscript: not found
bb: bb: testscript: not found
jsh: jsh: testscript: not found
--
Dan Douglas
signature.asc
Description: This is a digitally signed message part.
Re: Tilde expansion during command search
On Wed, Jul 23, 2014 at 9:58 PM, Eric Blake ebl...@redhat.com wrote: Might be worth asking the POSIX folks if it is allowed by POSIX. What do other shells do? From this page: http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html Note that the tildes are expanded during the assignment to PATH, not when PATH is accessed during command search. (resent with proper CC)
Re: Tilde expansion during command search
On 07/23/2014 08:08 AM, lolilolicon wrote: On Wed, Jul 23, 2014 at 9:58 PM, Eric Blake ebl...@redhat.com wrote: Might be worth asking the POSIX folks if it is allowed by POSIX. What do other shells do? From this page: http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html Note that the tildes are expanded during the assignment to PATH, not when PATH is accessed during command search. No, it wasn't. Re-read the original post: PATH=\~/tmpdir typeset -p PATH which produced this output: declare -x PATH=~/tmpdir which proves that ~ was not expanded in the assignment to PATH, but only in the use of PATH. I'm strongly in favor of keeping bash's behavior when in bash mode, but for 'set -o posix', I worry that bash differing from other shells may mean bash is not compliant with POSIX. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Tilde expansion during command search
On 07/23/2014 08:38 AM, Eric Blake wrote: On 07/23/2014 08:08 AM, lolilolicon wrote: From this page: http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html Note that the tildes are expanded during the assignment to PATH, not when PATH is accessed during command search. No, it wasn't. Re-read the original post: PATH=\~/tmpdir The reference you were quoting used this example: PATH=~hlj/bin:~dwc/bin:$PATH where ~ is unquoted in the assignment, and therefore expanded during the assignment. But we are talking about the case where ~ IS quoted during the assignment, and only bash then re-expands it during path lookup. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Tilde expansion during command search
On Wed, Jul 23, 2014 at 10:43 PM, Eric Blake ebl...@redhat.com wrote: But we are talking about the case where ~ IS quoted during the assignment, and only bash then re-expands it during path lookup. That's my point. Let me try quoting this again, Note that the tildes are expanded during the assignment to PATH, _not when PATH is accessed during command search._ in which I was trying to imply that, Bash *does* expand the tildes _when PATH is accessed during command search._
