Re: Issues with exported functions
On Eric Blake wrote:
> On 09/28/2014 10:31 AM, Ángel González wrote:
> > David A. Wheeler wrote:
> >> 2. Import environment variables *ONLY* when they are requested; do *NOT*
> >> import them by default. Christos Zoulas has proposed this. This *IS* a
> >> real backwards-incompatible change. But most users do *NOT* use this
> >> functionality, and increasingly downstream systems are *already* switching
> >> to this mode. E.G., FreeBSD has already switched to this; function
> >> imports require --import-functions or enabling the IMPORTFUNCTIONS option.
> >> E.G., see: https://svnweb.freebsd.org/ports?view=revision&revision=369341
> >> This change eliminates the entire class of problems. It's still good to
> >> do #1, even with #2, because if someone DOES perform an import, it reduces
> >> the probability of accidentally importing the wrong thing. People are
> >> ALREADY making this change, whether upstream does or not.
> >>
> >
> > There's also the middleground of not parsing the environment variables
> > before they are going to be used. That avoids the issues caused by
> > parsing what is not needed *and* doesn't break backwards compatibility.
> > See the patch I sent a couple days ago.
>
> That patch doesn't address the fact that variables and functions are in
> different namespaces, though. That is, if I do 'export f=1; f(){ echo
> 2;}; export -f f', then what would "$f" be in the child shell?
It would be (and is) 1, as required.
> With just your patch, but not the fix for separate namespaces of 4.3.27, you
> are still trying to pass two separate 'f=...' environment variables,
That's right, exactly what bash < 4.3.27 did.
> with unspecified results, and risk the child getting 'echo "$f"' to see
> the unexpected "() { echo 2; }' instead of the expected "1".
The bash children correctly get the two things, there are no unspecified
results for them.
The problems are:
* Non-bash programs reading f from the environment
* bash programs reading a free-text variable from the environment (which
the attacker can prefix with "() {")
> Your approach of lazy parsing may still have benefits, but it is not a middle
> ground, in that we MUST have separate namespaces (patch 27),
The reasons I listed above justify having separate namespaces. The only
concern not to do it was BC. I agree with patch 27
> and once we have separate namespaces, your patch is no longer adding
> security, just
> optimization.
No. If we had a perfect bash with no parser bugs (which we won't be able
to prove to have reached), then my patch would only be an optmization.
Otherwise it does add a security layer over that of patch 27 -although
the BASH_FUNCT prefix already sets the bar so high it might be
impossible to bypass-.
> >> John Haxby recently posted that "A friend of mine said this could be a
> >> vulnerability gift that keeps on giving."
> >> (http://seclists.org/oss-sec/2014/q3/748). Bash will be a continuous rich
> >> source of system vulnerabilities until it STOPS automatically parsing
> >> normal environment variables; all other shells just pass them through!
> >> I've turned off several websites I control because I have *no* confidence
> >> that the current official bash patches actually stop anyone, and I am
> >> deliberately *not* buying products online today for the same reason. I
> >> suspect others have done the same. I think it's important that bash
> >> change its semantics so that it "obviously has absolutely no problems of
> >> this kind".
> >
> > That's exactly what my patch does, although it wouldn't be transparent
> > if used inside bash (eg. echo $FUNC), as opposed of usage by its
> > children (wouldn't be hard to change, though).
>
> I consider it an important design goal to ensure that ALL exported
> variables cannot be corrupted no matter what their contents are. We
> aren't quite there yet (due to the issues of 'function a=b () {:;}
> corrupting the variable named BASH_FUNC_a even after patch 27 is
> applied).
Using the colon there makes it invalid. Pasting the poc from the other
thread:
> env -i bash -c 'function a=b(){ echo oops;};export -f a=b;export
> BASH_FUNC_a=hi; bash -c "echo \$BASH_FUNC_a"'
> But your own admission that $FUNC may be corrupted is an argument against
> your patch in isolation.
Fair enough, I acknowledged from the beginning that it could be
combined with the other flying patches. I was trying to do only one fix.
Actually, I was thinking that the move should be to use a single
namespace. I realized now that it is a POSIX requirement that they are
separate:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_05
Regards
Re: Issues with exported functions
Chet Ramey wrote: > On 9/28/14, 12:31 PM, Ángel González wrote: > > > There's also the middleground of not parsing the environment variables > > before they are going to be used. That avoids the issues caused by > > parsing what is not needed *and* doesn't break backwards compatibility. > > See the patch I sent a couple days ago. > > That's a reasonably elegant extension, but it doesn't really solve the > problem any better than fixing the underlying parser problem. You still have to fix the parser, obviously. It reduces the attack surface for parsed-but-not-executed functions (and I expect a performance improvement, too). > A user who can figure out how to pass a function with the appropriate name > encoding, > which at this point nobody has figured out to do remotely, can more than > likely arrange for that function to be called, at which point it will be > parsed and any underlying parser bug exposed. Or he can simply put the code to be executed inside the function to be called. There's no merit in exploiting a parser bug in the exported variable grep (before patch 27) if the bash script executes grep.
Re: Issues with exported functions
On 09/28/2014 10:31 AM, Ángel González wrote:
> David A. Wheeler wrote:
>> 2. Import environment variables *ONLY* when they are requested; do *NOT*
>> import them by default. Christos Zoulas has proposed this. This *IS* a
>> real backwards-incompatible change. But most users do *NOT* use this
>> functionality, and increasingly downstream systems are *already* switching
>> to this mode. E.G., FreeBSD has already switched to this; function imports
>> require --import-functions or enabling the IMPORTFUNCTIONS option. E.G.,
>> see: https://svnweb.freebsd.org/ports?view=revision&revision=369341
>> This change eliminates the entire class of problems. It's still good to do
>> #1, even with #2, because if someone DOES perform an import, it reduces the
>> probability of accidentally importing the wrong thing. People are ALREADY
>> making this change, whether upstream does or not.
>>
>
> There's also the middleground of not parsing the environment variables
> before they are going to be used. That avoids the issues caused by
> parsing what is not needed *and* doesn't break backwards compatibility.
> See the patch I sent a couple days ago.
That patch doesn't address the fact that variables and functions are in
different namespaces, though. That is, if I do 'export f=1; f(){ echo
2;}; export -f f', then what would "$f" be in the child shell? With
just your patch, but not the fix for separate namespaces of 4.3.27, you
are still trying to pass two separate 'f=...' environment variables,
with unspecified results, and risk the child getting 'echo "$f"' to see
the unexpected "() { echo 2; }' instead of the expected "1". Your
approach of lazy parsing may still have benefits, but it is not a middle
ground, in that we MUST have separate namespaces (patch 27), and once we
have separate namespaces, your patch is no longer adding security, just
optimization.
>
>> John Haxby recently posted that "A friend of mine said this could be a
>> vulnerability gift that keeps on giving."
>> (http://seclists.org/oss-sec/2014/q3/748). Bash will be a continuous rich
>> source of system vulnerabilities until it STOPS automatically parsing normal
>> environment variables; all other shells just pass them through! I've
>> turned off several websites I control because I have *no* confidence that
>> the current official bash patches actually stop anyone, and I am
>> deliberately *not* buying products online today for the same reason. I
>> suspect others have done the same. I think it's important that bash change
>> its semantics so that it "obviously has absolutely no problems of this kind".
>
> That's exactly what my patch does, although it wouldn't be transparent
> if used inside bash (eg. echo $FUNC), as opposed of usage by its
> children (wouldn't be hard to change, though).
I consider it an important design goal to ensure that ALL exported
variables cannot be corrupted no matter what their contents are. We
aren't quite there yet (due to the issues of 'function a=b () {:;}
corrupting the variable named BASH_FUNC_a even after patch 27 is
applied). But your own admission that $FUNC may be corrupted is an
argument against your patch in isolation.
--
Eric Blake eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
Re: Issues with exported functions
On 9/28/14, 12:31 PM, Ángel González wrote: > There's also the middleground of not parsing the environment variables > before they are going to be used. That avoids the issues caused by > parsing what is not needed *and* doesn't break backwards compatibility. > See the patch I sent a couple days ago. That's a reasonably elegant extension, but it doesn't really solve the problem any better than fixing the underlying parser problem. A user who can figure out how to pass a function with the appropriate name encoding, which at this point nobody has figured out to do remotely, can more than likely arrange for that function to be called, at which point it will be parsed and any underlying parser bug exposed. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
David A. Wheeler wrote: > 2. Import environment variables *ONLY* when they are requested; do *NOT* > import them by default. Christos Zoulas has proposed this. This *IS* a real > backwards-incompatible change. But most users do *NOT* use this > functionality, and increasingly downstream systems are *already* switching to > this mode. E.G., FreeBSD has already switched to this; function imports > require --import-functions or enabling the IMPORTFUNCTIONS option. E.G., > see: https://svnweb.freebsd.org/ports?view=revision&revision=369341 > This change eliminates the entire class of problems. It's still good to do > #1, even with #2, because if someone DOES perform an import, it reduces the > probability of accidentally importing the wrong thing. People are ALREADY > making this change, whether upstream does or not. > There's also the middleground of not parsing the environment variables before they are going to be used. That avoids the issues caused by parsing what is not needed *and* doesn't break backwards compatibility. See the patch I sent a couple days ago. > John Haxby recently posted that "A friend of mine said this could be a > vulnerability gift that keeps on giving." > (http://seclists.org/oss-sec/2014/q3/748). Bash will be a continuous rich > source of system vulnerabilities until it STOPS automatically parsing normal > environment variables; all other shells just pass them through! I've turned > off several websites I control because I have *no* confidence that the > current official bash patches actually stop anyone, and I am deliberately > *not* buying products online today for the same reason. I suspect others > have done the same. I think it's important that bash change its semantics so > that it "obviously has absolutely no problems of this kind". That's exactly what my patch does, although it wouldn't be transparent if used inside bash (eg. echo $FUNC), as opposed of usage by its children (wouldn't be hard to change, though).
Re: Issues with exported functions
On Saturday, September 27, 2014 3:51:23 PM UTC+1, Andreas Schwab wrote: > becker...@gmail.com writes: > > > > > $ (env;echo echo 'hello') | bash > > > > You cannot expect that the output of env is parsable by a shell. First > > and foremost, it lacks any kind of quoting. env may not be the issue, but the at command is fairly ancient in unix terms; it is well documented in the 1983 version of 'The UNIX System' by S.R.Bourne. The at(1) command breaks if the latest bash(4.3.026-1) is used in Arch linux brokenness is observable. The current patches break stuff.
Re: Issues with exported functions
On 9/27/14, 8:06 AM, David A. Wheeler wrote: > I agree it would be MUCH better to append () to the "variable" name if it is > a function export. If that suffix can be included in the official bash > release, I would be delighted. > > There had been earlier claims that () might fail on old systems, but I have > not seen evidence of it. I do not have access to really old systens to > test... if anyone does that would be helpful. At this point it might be best > to focus on protecting current systems. It fails on current systems, including Red Hat. It looks like there are still things to clean up. https://bugzilla.redhat.com/show_bug.cgi?id=1147043 -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
becker...@gmail.com writes: > $ (env;echo echo 'hello') | bash You cannot expect that the output of env is parsable by a shell. First and foremost, it lacks any kind of quoting. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: Issues with exported functions
On Sep 27, 2014, at 2:19 AM, Eric Blake wrote: > The prefix is nice for quick identification, but what is ESSENTIAL is > something that puts shell functions in a namespace that is untouchable > by normal shell variables (the "()" suffix in Florian's patch). If all > you do is add a prefix, but still leave the environment containing what > can still collide with a shell variable name, you are still vulnerable. Repeated for truth.
Re: Issues with exported functions
I currently have this problem with the latest bash from Arch linux which is
biting my at(1) commands. Exported functions make the environment
non-parseable.
##
robin@bunyip:~
$ export -nf psg hsg #clean out my exported functions
robin@bunyip:~
$ (env;echo echo 'hello') | bash
hello
robin@bunyip:~
$ bilbo(){
> echo 'precious'
> }
robin@bunyip:~
$ export -f bilbo
robin@bunyip:~
$ (env;echo echo 'hello') | bash
bash: line 29: syntax error near unexpected token `='
bash: line 29: `BASH_FUNC_bilbo()=() { echo 'precious''
robin@bunyip:~
$
###
latest bash changes can be seen here (various CVE stuff).
https://projects.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/bash&id=66b6f8f71f48b425c53b781c531697988413e503
Re: Issues with exported functions
I agree it would be MUCH better to append () to the "variable" name if it is a function export. If that suffix can be included in the official bash release, I would be delighted. There had been earlier claims that () might fail on old systems, but I have not seen evidence of it. I do not have access to really old systens to test... if anyone does that would be helpful. At this point it might be best to focus on protecting current systems. On September 27, 2014 2:19:49 AM EDT, Eric Blake wrote: >On 09/26/2014 03:47 PM, David A. Wheeler wrote: >> I appreciate the effort made in patch bash43-026, but this patch >doesn't even BEGIN to solve the underlying shellshock problem. This >patch just continues the "whack-a-mole" job of fixing parsing errors >that began with the first patch. Bash's parser is certain have many >many many other vulnerabilities; it was never designed to be >security-relevant! >> >> I strongly recommend *TWO* changes which have been discussed here and >on oss-sec: >> 1. Add a prefix "BASH_FUNC..." (and maybe suffix) as proposed by >Florian Weimer, per: >http://www.openwall.com/lists/oss-security/2014/09/25/13 > >The prefix is nice for quick identification, but what is ESSENTIAL is >something that puts shell functions in a namespace that is untouchable >by normal shell variables (the "()" suffix in Florian's patch). If all >you do is add a prefix, but still leave the environment containing what >can still collide with a shell variable name, you are still vulnerable. > >> This is technically backwards-incompatible, but that will rarely >matter. The specific environment variable mechanism was never >documented in the bash man page, after all, and it works just fine if >both sending & receiving bashes are patched. I would suggest NOT >including the suffix "()", since some old systems might have trouble >with such unusual environment variable names. > >Well, what WOULD you suggest? There MUST be something that makes all >exported functions use env-var names that CANNOT collide with shell >variable names. Do you have proof of a system that chokes with () in >the environment variable name? > >> This change completely eliminates vulnerabilities from CGI and >similar processing, where attacker data is being passed through >environment variables to a receiving system. It also eliminates the >punning that comes when functions and regular environment variables >have the same name, which isn't really POSIX-compliant anyway. >> >> 2. Import environment variables *ONLY* when they are requested; do >*NOT* import them by default. Christos Zoulas has proposed this. This >*IS* a real backwards-incompatible change. But most users do *NOT* use >this functionality, and increasingly downstream systems are *already* >switching to this mode. E.G., FreeBSD has already switched to this; >function imports require --import-functions or enabling the >IMPORTFUNCTIONS option. E.G., see: >https://svnweb.freebsd.org/ports?view=revision&revision=369341 >> This change eliminates the entire class of problems. It's still good >to do #1, even with #2, because if someone DOES perform an import, it >reduces the probability of accidentally importing the wrong thing. >People are ALREADY making this change, whether upstream does or not. > >I am also in favor of both approaches - moving shell functions into a >non-colliding namespace (so that arbitrary contents of regular >variables >CANNOT trigger parser bugs) and making shell function exports >configurable and off by default. > >-- >Eric Blake eblake redhat com+1-919-301-3266 >Libvirt virtualization library http://libvirt.org --- David A.Wheeler
Re: Issues with exported functions
On Saturday, September 27, 2014 8:19:49 AM UTC+2, Eric Blake wrote: > > I am also in favor of both approaches - moving shell functions into a > non-colliding namespace (so that arbitrary contents of regular variables > CANNOT trigger parser bugs) and making shell function exports > configurable and off by default. On that second point: an even lower-hanging fruit is "off by default when invoked as sh". Regardless of bash's evolution, let's stop jeopardizing the very life of the overwhelming mass of #!/bin/sh scripts. Please. -Alex PS: Dunno whether CGI will ever recover from this week's blow. The sh-bash confusion is the real murderer of that innocent victim.
Re: Issues with exported functions
On 09/26/2014 03:47 PM, David A. Wheeler wrote: > I appreciate the effort made in patch bash43-026, but this patch doesn't even > BEGIN to solve the underlying shellshock problem. This patch just continues > the "whack-a-mole" job of fixing parsing errors that began with the first > patch. Bash's parser is certain have many many many other vulnerabilities; > it was never designed to be security-relevant! > > I strongly recommend *TWO* changes which have been discussed here and on > oss-sec: > 1. Add a prefix "BASH_FUNC..." (and maybe suffix) as proposed by Florian > Weimer, per: http://www.openwall.com/lists/oss-security/2014/09/25/13 The prefix is nice for quick identification, but what is ESSENTIAL is something that puts shell functions in a namespace that is untouchable by normal shell variables (the "()" suffix in Florian's patch). If all you do is add a prefix, but still leave the environment containing what can still collide with a shell variable name, you are still vulnerable. > This is technically backwards-incompatible, but that will rarely matter. The > specific environment variable mechanism was never documented in the bash man > page, after all, and it works just fine if both sending & receiving bashes > are patched. I would suggest NOT including the suffix "()", since some old > systems might have trouble with such unusual environment variable names. Well, what WOULD you suggest? There MUST be something that makes all exported functions use env-var names that CANNOT collide with shell variable names. Do you have proof of a system that chokes with () in the environment variable name? > This change completely eliminates vulnerabilities from CGI and similar > processing, where attacker data is being passed through environment variables > to a receiving system. It also eliminates the punning that comes when > functions and regular environment variables have the same name, which isn't > really POSIX-compliant anyway. > > 2. Import environment variables *ONLY* when they are requested; do *NOT* > import them by default. Christos Zoulas has proposed this. This *IS* a real > backwards-incompatible change. But most users do *NOT* use this > functionality, and increasingly downstream systems are *already* switching to > this mode. E.G., FreeBSD has already switched to this; function imports > require --import-functions or enabling the IMPORTFUNCTIONS option. E.G., > see: https://svnweb.freebsd.org/ports?view=revision&revision=369341 > This change eliminates the entire class of problems. It's still good to do > #1, even with #2, because if someone DOES perform an import, it reduces the > probability of accidentally importing the wrong thing. People are ALREADY > making this change, whether upstream does or not. I am also in favor of both approaches - moving shell functions into a non-colliding namespace (so that arbitrary contents of regular variables CANNOT trigger parser bugs) and making shell function exports configurable and off by default. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Issues with exported functions
I appreciate the effort made in patch bash43-026, but this patch doesn't even BEGIN to solve the underlying shellshock problem. This patch just continues the "whack-a-mole" job of fixing parsing errors that began with the first patch. Bash's parser is certain have many many many other vulnerabilities; it was never designed to be security-relevant! I strongly recommend *TWO* changes which have been discussed here and on oss-sec: 1. Add a prefix "BASH_FUNC..." (and maybe suffix) as proposed by Florian Weimer, per: http://www.openwall.com/lists/oss-security/2014/09/25/13 This is technically backwards-incompatible, but that will rarely matter. The specific environment variable mechanism was never documented in the bash man page, after all, and it works just fine if both sending & receiving bashes are patched. I would suggest NOT including the suffix "()", since some old systems might have trouble with such unusual environment variable names. This change completely eliminates vulnerabilities from CGI and similar processing, where attacker data is being passed through environment variables to a receiving system. It also eliminates the punning that comes when functions and regular environment variables have the same name, which isn't really POSIX-compliant anyway. 2. Import environment variables *ONLY* when they are requested; do *NOT* import them by default. Christos Zoulas has proposed this. This *IS* a real backwards-incompatible change. But most users do *NOT* use this functionality, and increasingly downstream systems are *already* switching to this mode. E.G., FreeBSD has already switched to this; function imports require --import-functions or enabling the IMPORTFUNCTIONS option. E.G., see: https://svnweb.freebsd.org/ports?view=revision&revision=369341 This change eliminates the entire class of problems. It's still good to do #1, even with #2, because if someone DOES perform an import, it reduces the probability of accidentally importing the wrong thing. People are ALREADY making this change, whether upstream does or not. John Haxby recently posted that "A friend of mine said this could be a vulnerability gift that keeps on giving." (http://seclists.org/oss-sec/2014/q3/748). Bash will be a continuous rich source of system vulnerabilities until it STOPS automatically parsing normal environment variables; all other shells just pass them through! I've turned off several websites I control because I have *no* confidence that the current official bash patches actually stop anyone, and I am deliberately *not* buying products online today for the same reason. I suspect others have done the same. I think it's important that bash change its semantics so that it "obviously has absolutely no problems of this kind". Thanks for listening! --- David A. Wheeler
Re: Issues with exported functions
In article ,
lolilolicon wrote:
> I think almost as severe as CVE-2014-6271 is that it's still possible to
> mask commands in a bash script by changing it's environment.
> For example, true='() { false;}' or grep='() { /bin/id;}' ...
Yes, and BTW, I don't think this is POSIX compliant:
8.1 Environment Variable Definition
[...] The name space of environment variable names containing
lowercase letters is reserved for applications. Applications can
define any environment variables with names from this name space
without modifying the behavior of the standard utilities.
This means that some application like sudo that needs to clean up
the environment could choose to keep these environment variables
with lowercase letters, and this could have really bad effects if
a bash script is executed.
--
Vincent Lefèvre - Web:
Re: Issues with exported functions
On Fri, Sep 26, 2014 at 3:24 PM, Vincent Lefevre wrote:
> On 2014-09-25 03:54:19 +0800, lolilolicon wrote:
>> [...] that it's still possible to
>> mask commands in a bash script by changing it's environment.
>>
>> For example, true='() { false;}' or grep='() { /bin/id;}' ...
>
> Yes, and BTW, I don't think this is POSIX compliant:
[...]
> This means that some application like sudo that needs to clean up
> the environment could choose to keep these environment variables
> with lowercase letters, and this could have really bad effects if
> a bash script is executed.
Yes, my opinion is ENV is a bad channel for doing function export.
ENV is a shared space, isn't well-specified, relies entirely on policy
instead of any intrinsic mechanism... it's just fundamentally
unsuitable for too much special interpretation.
Re: Issues with exported functions
On 2014-09-25 03:54:19 +0800, lolilolicon wrote:
> I think almost as severe as CVE-2014-6271 is that it's still possible to
> mask commands in a bash script by changing it's environment.
>
> For example, true='() { false;}' or grep='() { /bin/id;}' ...
Yes, and BTW, I don't think this is POSIX compliant:
8.1 Environment Variable Definition
[...] The name space of environment variable names containing
lowercase letters is reserved for applications. Applications can
define any environment variables with names from this name space
without modifying the behavior of the standard utilities.
This means that some application like sudo that needs to clean up
the environment could choose to keep these environment variables
with lowercase letters, and this could have really bad effects if
a bash script is executed.
--
Vincent Lefèvre - Web:
Re: Issues with exported functions
Chet Ramey wrote:
> On 9/25/14, 2:47 PM, lolilolicon wrote:
> > On Fri, Sep 26, 2014 at 2:28 AM, Ángel González wrote:
> > [...]
> >> On the other hand, this approach would be much more interesting if bash
> >> delayed parsing of exported functions until they are used (ie. check
> >
> > This is what function autoload is for in zsh. It's indeed a better
> > approach. It was also suggested by Dan Douglas in this thread ("FPATH
> > mechanism").
>
> Not quite. While autoloaded functions are lazily evaluated, you have to
> pay the price of searching $FPATH and loading them in every shell, and
> there still has to be a mechanism to indicate which functions should be
> autoloaded in each shell.
>
> Chet
I wasn't refering to storing the functions in files, but to lazily parse
the command from the environment. It wouldn't affect the way bash
functions are transfered.
See the naive implementation provided below:
diff --git a/variables.h b/variables.h
index 1a783b9..f496b9d 100644
--- a/variables.h
+++ b/variables.h
@@ -125,6 +125,7 @@ typedef struct _vlist {
#define att_imported 0x0008000 /* came from environment */
#define att_special0x001 /* requires special handling */
#define att_nofree 0x002 /* do not free value on unset */
+#define att_unparsed 0x004 /* it was not parsed after loading from
environ */
#defineattmask_int 0x00ff000
@@ -153,6 +154,7 @@ typedef struct _vlist {
#define imported_p(var)var)->attributes) & (att_imported)))
#define specialvar_p(var) var)->attributes) & (att_special)))
#define nofree_p(var) var)->attributes) & (att_nofree)))
+#define unparsed_p(var)var)->attributes) & (att_unparsed)))
#define tempvar_p(var) var)->attributes) & (att_tempvar)))
diff --git a/variables.c b/variables.c
index 92a5a10..a0e42ff 100644
--- a/variables.c
+++ b/variables.c
@@ -351,34 +351,13 @@ initialize_shell_variables (env, privmode)
the environment in privileged mode. */
if (privmode == 0 && read_but_dont_execute == 0 && STREQN ("() {",
string, 4))
{
- string_length = strlen (string);
- temp_string = (char *)xmalloc (3 + string_length + char_index);
- strcpy (temp_string, name);
- temp_string[char_index] = ' ';
- strcpy (temp_string + char_index + 1, string);
-
- /* Don't import function names that are invalid identifiers from the
-environment, though we still allow them to be defined as shell
-variables. */
- if (legal_identifier (name))
- parse_and_execute (temp_string, name,
SEVAL_NONINT|SEVAL_NOHIST|SEVAL_FUNCDEF|SEVAL_ONECMD);
-
- if (temp_var = find_function (name))
- {
- VSETATTR (temp_var, (att_exported|att_imported));
- array_needs_making = 1;
- }
- else
- {
- if (temp_var = bind_variable (name, string, 0))
+ if (temp_var = bind_function (name, 0))
{
- VSETATTR (temp_var, (att_exported | att_imported |
att_invisible));
+ VSETATTR (temp_var, (att_exported | att_imported |
att_unparsed));
+ SET_EXPORTSTR(temp_var, savestring (string));
array_needs_making = 1;
}
- last_command_exit_value = 1;
- report_error (_("error importing function definition for `%s'"),
name);
- }
}
#if defined (ARRAY_VARS)
# if ARRAY_EXPORT
@@ -1049,8 +1028,11 @@ print_var_function (var)
if (function_p (var) && var_isset (var))
{
- x = named_function_string ((char *)NULL, function_cell(var),
FUNC_MULTILINE|FUNC_EXTERNAL);
- printf ("%s", x);
+ if (!maybe_parse_unparsed_func (var))
+ {
+ x = named_function_string ((char *)NULL, function_cell(var),
FUNC_MULTILINE|FUNC_EXTERNAL);
+ printf ("%s", x);
+ }
}
}
@@ -3929,7 +3911,16 @@ make_env_array_from_var_list (vars)
if (var->exportstr)
value = var->exportstr;
else if (function_p (var))
- value = named_function_string ((char *)NULL, function_cell (var), 0);
+ {
+ if (unparsed_p (var))
+ {
+ value = var->exportstr;
+ }
+ else
+ {
+ value = named_function_string ((char *)NULL,
function_cell (var), 0);
+ }
+ }
#if defined (ARRAY_VARS)
else if (array_p (var))
# if ARRAY_EXPORT
diff --git a/execute_cmd.h b/execute_cmd.h
index 67ae93a..4f4e3ee 100644
--- a/execute_cmd.h
+++ b/execute_cmd.h
@@ -27,6 +27,7 @@ extern struct fd_bitmap *new_fd_bitmap __P((int));
extern void dispose_fd_bitmap __P((struct fd_bitmap *));
extern void close_fd_bitmap __P((struct fd_bitmap *));
extern int executing_li
Re: Issues with exported functions
> Not quite. While autoloaded functions are lazily evaluated, you have to
> pay the price of searching $FPATH and loading them in every shell, and
> there still has to be a mechanism to indicate which functions should be
> autoloaded in each shell.
How about lazy loading functions using the following mechanism:
# code
foo arg1 arg2 arg3
1) Search for an alias called 'foo'
2) Search for a function named 'foo'
3) Search for a variable named 'foo' with '(){ ...; }' as contents
4) Search for a builtin ...
...
So far, this is compatible with what we already have, right?
Now, this method (of skipping the function definition until we
attempt to use it) could allow for a special shopt, which disables
the step '3'.
Something like: shopt -s nofunexp, which changes step 3 to
3) Search for a variable named 'foo' with '(){ ...; }' as contents
IF nofunexp is disabled.
So, I can now put: shopt -s nofunexp in my scripts proactively, to
avoid function exports at all.
Also, a special array variable could be introduced, like:
BASH_ALLOWED_FUNEXP=(foo bar baz), which acts as a whitelist to allow
only a certain set of functions to be taken from the environment.
So, this way, we keep backwards compatibility, but we also have the
opportunity to disallow exported functions.
--
Eduardo Bustamante
Re: Issues with exported functions
> It's not backwards compatible, but who cares? The only time it > matters is if you are mixing old and new bash ON THE SAME SYSTEM, > and TRYING TO EXPORT FUNCTIONS BETWEEN THEM. It might happen during the update of bash. A bash process exec()ed before the update would fail to export a function to a bash process exec()ed after the update. -- Gabriel
Re: Issues with exported functions
> I'd much rather prefer the use of an invalid shell name (such as > f()=...) than a valid shell name (BASH_FUNCTION_foo=()...). Using a BASH_ prefix has some advantages: * Anyone setting such a variable, might expect it could change the behaviour of bash. Any script allowing setting untrusted BASH_ variables to untrusted values should probably expect bad things to happen. * Anyone seeing such a variable (in env) might expect it to change the behaviour of bash. If I see foo(), in env I would probably not think "Of course, it will change bash behaviour." * It avoids name clashes: whateversh might want to use foo() for its own (different) purpose. Maybe both could be used, BASH_FUNCTION_foo()? Is there any portability issue with using '()' on some OSes? -- Gabriel
Re: Issues with exported functions
On Fri, Sep 26, 2014 at 2:53 AM, Linda Walsh wrote: > --- > "prevert" _might_ have been more obviously seen as applicable to a bash > programmer who is "pre-bent" or "pre-twisted" from having programmed in > shell for so long, compared to the nick, lilololicon, with its 'H'y (as in > 'Hentai'-y) > connotations it might invoke about someone who might be just a bit too > "pedo-friendly" or pedo-'filic' (unless everyone knows the inside humor). LOL, you're reinforcing my opinion of your style of argument, but I'll keep that to myself. BUT you spelt my nick wrong :( [...] > My only issue "with" function export is it being a partial solution, > missing Aliases (first and foremost) which, for some things, are more > essential than functions, and missing Array and Hash export -- which have > to be annoyingly emulated and reconstructed between commands. What you're proposing here is the polar opposite of most people's reaction to this bug; many are asking for a way to disable function export, an obscure feature they never expected from bash. I'm not sure how you want other kinds of export to work. I suppose you want them to be inherited from the env? Then you want to invent a protocol / syntax / minilanguage for bash to export and import those. This is IMO not what the env is suitable for, since it's meant to be used as a very simple and flat "name=value" format for passing environment variables around. But maybe you can elaborate? > > Redoing those with each command is equally painful and requires their > redefinition with each bash invocation via BASH_ENV. But the number of > redefines to work around those deficits is insignificant to the number of > functions that rely on persistence, so simple scripts like this work: > > ---show_inc.sh--- > #!/bin/bash include stdalias > include Util/url > > #definitions are important > int a=1+1 > string astring=1+1 > a+=1+1 > astring+=1+1 > printf "Integers: 1+1+1+1 = %s. Strings: 1+1+1+1 = %s\n" "$a" "$astring" > > #2nd example > include Util/url > > string url="http://google.com/foo"; > > echo "proto=$(protocol "$url"), host=$(host "$url"), path=$(path "$url")" Wow. That's advanced... It looks like you have built a language on top of bash. Perhaps, just maybe, your needs have outgrown what bash can reasonably provide? (Feel free to dismiss this, I know this is a subjective thing.)
Re: Issues with exported functions
On 9/25/14, 2:47 PM, lolilolicon wrote:
> On Fri, Sep 26, 2014 at 2:28 AM, Ángel González wrote:
> [...]
>> On the other hand, this approach would be much more interesting if bash
>> delayed parsing of exported functions until they are used (ie. check
>
> This is what function autoload is for in zsh. It's indeed a better
> approach. It was also suggested by Dan Douglas in this thread ("FPATH
> mechanism").
Not quite. While autoloaded functions are lazily evaluated, you have to
pay the price of searching $FPATH and loading them in every shell, and
there still has to be a mechanism to indicate which functions should be
autoloaded in each shell.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
On Sep 25, 2014, at 2:47 PM, lolilolicon wrote:
> On Fri, Sep 26, 2014 at 2:28 AM, Ángel González wrote:
> [...]
>> On the other hand, this approach would be much more interesting if bash
>> delayed parsing of exported functions until they are used (ie. check
>
> This is what function autoload is for in zsh. It's indeed a better
> approach. It was also suggested by Dan Douglas in this thread ("FPATH
> mechanism").
Autoload has plusses and minuses. Shell startup is initially faster because
there's a lot less processing, but each not-yet-loaded function requires
traversal of another search path. And if you have multiple levels of shell
(who, me do :sh in vi?) any newly spawned shells in the same session don't get
the previously loaded functions. Unless, of course, autoloaded functions could
be exported. But I'm not gonna think about that right now, got a lot of work to
do.
Re: Issues with exported functions
lolilolicon wrote: On Thu, Sep 25, 2014 at 9:00 PM, Chet Ramey wrote: > Even if you use it as a rhetorical device, it distracts from (and > detracts from) your argument. It doesn't improve the quality of the > discussion, so it's best not to use it. Agreed. People can take light-hearted side remarks too seriously. *adds "pervert" & "like a girl" to List of Forbidden Terms in Public Mail* Thank you. --- "prevert" _might_ have been more obviously seen as applicable to a bash programmer who is "pre-bent" or "pre-twisted" from having programmed in shell for so long, compared to the nick, lilololicon, with its 'H'y (as in 'Hentai'-y) connotations it might invoke about someone who might be just a bit too "pedo-friendly" or pedo-'filic' (unless everyone knows the inside humor). As for the other phrase... are you from australia as a first guess, maye euro as a 2nd? Needless, one needs to make context and humor extra clear in an international audience (not that I always am a master of subtlety in such). I do not think about optimization. My opinion is the need for the programmer to think about optimization (that's not relevant to the problem at hand) is a symptom of bad design (of the language / tool), or perhaps the programmer is using wrong tool / using the tool wrong. Spoilt youngster!... Bash is an interpreter and not noted for being a high-speed scripting language. As to your particular argument for completions: I think zsh optimizes for this with autoloading mechanism, which is a better design. --- Bash has an autoloading mechanism that people don't use because it's usually faster to put a bunch of defs in 1 place than spread them over 1000 files. For interactive use (which is zsh's emphasis), it's fine, but for something that is as often used as a batch-control language as often as interactively (like bash), its not. Zsh isn't a backbone scripting language. Isn't the mere fact that you have to do this round-around thing to "program safely" a proof by demonstration that exported functions are trouble? === Cf.: Isn't the mere fact that you need to use a "safe path" demonstrate that PATH lookup is trouble? OTOH, using abs-paths is just as problem laden. That you need to use safety precautions to generate robust programs is "par for the course". Everything has its up and down side(s). By defining the programs I will need as aliases up front, the program will die earlier if it's requirements are not present. Nevertheless, I'll redefine progs as functions nearly as often so I can make a program easier to call (aliases are limited to the easiest stuff) from my script. Other functions redefine commands for a login session. I don't want them undefined on sub commands. I'd prefer aliases be exportable (as well as arrays and hashes). If my sub commands are built on others, they'd all go belly-up after the first exec -- more so than when sudo kills them off: like "rd" for rmdir or "+x" for chmod +x -- both of which I need as often with a sudo before them as not (and that tend to fail due to sudo's inability to parse aliases or functions). To give you an idea of function and definition that is skipped at login vs. not: time bash 2>/dev/null < xxx 0.00sec 0.00usr 0.00sys (88.59% cpu) time bash -i 2>/dev/null < xxx User .bashrc called 2nd time 0.21sec 0.14usr 0.05sys (93.21% cpu) Setting up things only 1 time saves considerable time depending on how much is setup for an interactive shell vs. not. Also if my script uses 'sudo' it has to pay such penalties unless I patch sudo. I'll have scripts that make tons of calls to sudo because the rest was designed to operate w/o privs and I try to isolate priv calls, so an unpatch sudo that doesn't preserve environment when told to is a problem. My only issue "with" function export is it being a partial solution, missing Aliases (first and foremost) which, for some things, are more essential than functions, and missing Array and Hash export -- which have to be annoyingly emulated and reconstructed between commands. Redoing those with each command is equally painful and requires their redefinition with each bash invocation via BASH_ENV. But the number of redefines to work around those deficits is insignificant to the number of functions that rely on persistence, so simple scripts like this work: ---show_inc.sh--- #!/bin/bash include stdalias include Util/url #definitions are important int a=1+1 string astring=1+1 a+=1+1 astring+=1+1 printf "Integers: 1+1+1+1 = %s. Strings: 1+1+1+1 = %s\n" "$a" "$astring" #2nd example include Util/url string url="http://google.com/foo"; echo "proto=$(protocol "$url"), host=$(host "$url"), path=$(path "$url")" ---run--- /tmp> show_inc.sh Integers: 1+1+1+1 = 4. Strings: 1+1+1+1 = 1+11+1 proto=http, host=google.com, path=fooshow_inc.sh --- (Yes, I included Util/url twice to show the effect of doing so). Not only are exported functions a useful timesaver, but aliases, ar
Re: Issues with exported functions
On Fri, Sep 26, 2014 at 2:28 AM, Ángel González wrote:
[...]
> On the other hand, this approach would be much more interesting if bash
> delayed parsing of exported functions until they are used (ie. check
This is what function autoload is for in zsh. It's indeed a better
approach. It was also suggested by Dan Douglas in this thread ("FPATH
mechanism").
Re: Issues with exported functions
Steve Simmons wrote:
> ..bash_once defines SET_ONCE and loads literally hundreds of environment
> variables and exports many shell functions that would otherwise have to be
> defined in .bashrc and processed on every freaking run. .bash_once is about
> 50 times larger than .bashrc and .bash_login. Fast. Very fast. But without
> exportable functions, it wouldn't work at all.
>
> As an exercise for the student, consider the utility of this simplified
> excerpt:
>
> for SYSTEM in \
> {foo,bar,baz}.dec.school.edu \
> {alligator,snake-skin,lizard}.reptiles.work.com \
> {misery,serenity,frailty}.films.home.org \
> ; do # Strip off domain, use dash-less name as function name
> export FNAME=${SYSTEM%%.*}
> export FNAME=${FNAME//-/}
> eval $(echo "$FNAME() { ssh $SYSTEM" '"$@";};'" export -f $FNAME")
> unset SYSTEM FNAME
> done
>
> Hint - source those lines, then give the command 'builtin type snakeskin'.
>
> It's probably too much overhead for every bash invocation, but if you only do
> it once per session, it's damned useful.
Is it so much overhead? You are replacing the cost of a bash for loop
(and its two assignations, one export and one unset per iteration) with
the same loop performed in C.
It doesn't seem such a big win, I think you would need to do a more
heavy computation outside the eval for that. Note that if we enter in
micro-optimization measuring, you should also measure the slowing to
every program which need to ignore those hundreds of environment
variables.
On the other hand, this approach would be much more interesting if bash
delayed parsing of exported functions until they are used (ie. check
that they begin with "() {" and store the string as an 'unparsed
function', only parsing them the first time it gets executed in the
process).
This should give a nice speed up when there are exported functions
(parsing is **slow**), and would have avoided the CVE as well. What do
you think, Chet?
FWIW, for this specific example, instead of using bash functions I would
have done that in ssh_config(5).
Host foo bar baz
HostName %h.dec.school.edu
Host alligator snake-skin lizard
HostName %h.reptiles.work.com
Host misery serenity frailty
HostName %h.films.home.org
Yes, you would need to type "ssh snake-skin" instead of "snakeskin" but
they will autocomplete, and also allows to use them on other commands,
which your functions don't cover, like scp, sftp, rsync, svn, hg, git…
Best regards
Re: Issues with exported functions
On 09/25/2014 07:03 AM, Chet Ramey wrote:
> On 9/25/14, 4:52 AM, Gabriel Corona wrote:
>> Hello,
>>
>> As the interface is not specified, would it make sense to:
>>
>> * add a prefix (use BASH_FUNCTION_foo instead of foo for exported
>>function foo);
I'd much rather prefer the use of an invalid shell name (such as
f()=...) than a valid shell name (BASH_FUNCTION_foo=()...).
>>
>> * still expand the variable if it matches the 'exported function'
>>pattern.
>
> Yes, that's one of the approaches under consideration. It raises the
> bar for abuse by requiring that an attacker be able to create environment
> variables with arbitrary names as well as values. It is not,
> unfortunately, backwards compatible.
It's not backwards compatible, but who cares? The only time it matters
is if you are mixing old and new bash ON THE SAME SYSTEM, and TRYING TO
EXPORT FUNCTIONS BETWEEN THEM. But the old bash behavior is so bad that
people are unlikely to want to have both shells installed. As long as
you have only new bash installed, then all parent-child relationships
will both understand the SAME interpretation of 'f()=...' in the
environment as a way to export shell functions, and leave 'f=() {...' as
a raw normal variable and avoid intruding on the user's possible string
space.
--
Eric Blake eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
Re: Issues with exported functions
On Thursday, September 25, 2014 10:29:16 AM Chet Ramey wrote: > On 9/25/14, 10:25 AM, Dan Douglas wrote: > > > Have you considered the FPATH mechanism? Exploiting it requires being able to > > create files and set FPATH accordingly. I've had some success with the > > function loader code in examples/functions/autoload.*. I believe it serves > > mostly the same purpose as exported functions. > > I have thought about it, but it is even less backwards compatible than > the other suggestions. True. I guess one key point is that it's "lazy" and only gets checked if a command doesn't exist. A user could implement function import entirely by means of command_not_found_handle - searching only for a function with the name given in "$1", which should be inherently more secure than looking at every variable at startup. -- Dan Douglas
Re: Issues with exported functions
On 9/25/14, 10:25 AM, Dan Douglas wrote: > Have you considered the FPATH mechanism? Exploiting it requires being able to > create files and set FPATH accordingly. I've had some success with the > function loader code in examples/functions/autoload.*. I believe it serves > mostly the same purpose as exported functions. I have thought about it, but it is even less backwards compatible than the other suggestions. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
On Thursday, September 25, 2014 09:03:03 AM Chet Ramey wrote: > On 9/25/14, 4:52 AM, Gabriel Corona wrote: > > Hello, > > > > As the interface is not specified, would it make sense to: > > > > * add a prefix (use BASH_FUNCTION_foo instead of foo for exported > >function foo); > > > > * still expand the variable if it matches the 'exported function' > >pattern. > > Yes, that's one of the approaches under consideration. It raises the > bar for abuse by requiring that an attacker be able to create environment > variables with arbitrary names as well as values. It is not, > unfortunately, backwards compatible. > Have you considered the FPATH mechanism? Exploiting it requires being able to create files and set FPATH accordingly. I've had some success with the function loader code in examples/functions/autoload.*. I believe it serves mostly the same purpose as exported functions. -- Dan Douglas
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 9:00 PM, Chet Ramey wrote: > Even if you use it as a rhetorical device, it distracts from (and detracts > from) your argument. It doesn't improve the quality of the discussion, so > it's best not to use it. Agreed. People can take light-hearted side remarks too seriously. *adds "pervert" & "like a girl" to List of Forbidden Terms in Public Mail* Thank you.
Re: Issues with exported functions
On 9/25/14, 4:52 AM, Gabriel Corona wrote: > Hello, > > As the interface is not specified, would it make sense to: > > * add a prefix (use BASH_FUNCTION_foo instead of foo for exported >function foo); > > * still expand the variable if it matches the 'exported function' >pattern. Yes, that's one of the approaches under consideration. It raises the bar for abuse by requiring that an attacker be able to create environment variables with arbitrary names as well as values. It is not, unfortunately, backwards compatible. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
On 9/25/14, 6:12 AM, lolilolicon wrote: > On Thu, Sep 25, 2014 at 6:09 PM, Pierre Gaston > wrote: >>> In any event, this is but irrelevant to the discussion. Do not seize the >>> red herring. >> >> >> It is fully relevant when you use a sexist stereotype as an argument. > > No, I didn't use a sexist stereotype "as an argument". Remove that > sentence and nothing changes of my argument. Even if you use it as a rhetorical device, it distracts from (and detracts from) your argument. It doesn't improve the quality of the discussion, so it's best not to use it. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
On 9/25/14, 4:43 AM, Davide Brini wrote: >> Function export is documented. The exact mechanism need not be. > > I'm not arguing about anything, I just have a question. I understand that > with the current method used to export functions, it is not possible to > export a variable to a child whose value begins exactly with the characters > in question. This is correct, but it has rarely (that is to say, never) come up in practice. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
Hello,
As the interface is not specified, would it make sense to:
* add a prefix (use BASH_FUNCTION_foo instead of foo for exported
function foo);
* still expand the variable if it matches the 'exported function'
pattern.
The first point would reduce the probability of a clash with
a user variable:
foo() {
echo Executing function "$@"
}
foo="aa"
export -f foo
export foo
bash -c foo
# OK
bash -c 'echo $foo'
# OK as well
bash -c env
# In fact the environment variable is set twice, I didn't know it was event
possible
dash -c 'echo $foo'
# Dash find the second one: () { echo Executing function "$@" }
The second (more important) point would allow a variable to begin with
"() {}" or similar:
# I'm not aware of the function export feature
# or x comes from somewhere else:
$ x="() {}" bash -c 'echo $x' 2>/dev/null
$ x="() {}" dash -c 'echo $x' 2>/dev/null
() {}
--
Gabriel
Re: Issues with exported functions
Hello,
As the interface is not specified, would it make sense to:
* add a prefix (use BASH_FUNCTION_foo instead of foo for exported
function foo);
* still expand the variable if it matches the 'exported function'
pattern.
The first point would reduce the probability of a clash with
a user variable:
foo() {
echo Executing function "$@"
}
foo="aa"
export -f foo
export foo
bash -c foo
# OK
bash -c 'echo $foo'
# OK as well
bash -c env
# In fact the environment variable is set twice, I didn't know it was event
possible
dash -c 'echo $foo'
# Dash find the second one: () { echo Executing function "$@" }
The second (more important) point would allow a variable to begin with
"() {}" or similar:
$ x="() {}" bash -c 'echo $x' 2>/dev/null
$ x="() {}" dash -c 'echo $x' 2>/dev/null
() {}
--
Gabriel
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 6:09 PM, Pierre Gaston wrote: >> In any event, this is but irrelevant to the discussion. Do not seize the >> red herring. > > > It is fully relevant when you use a sexist stereotype as an argument. No, I didn't use a sexist stereotype "as an argument". Remove that sentence and nothing changes of my argument.
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 1:04 PM, lolilolicon wrote: > On Thu, Sep 25, 2014 at 5:51 PM, Pierre Gaston > wrote: > > > > > > On Thu, Sep 25, 2014 at 12:42 PM, lolilolicon > wrote: > >> > >> On Thu, Sep 25, 2014 at 7:19 AM, Linda Walsh wrote: > >> > lolilolicon wrote: > >> >> > >> >> I don't expect more than a dozen who rely on this... but bash > >> >> programmers can be quite the perverts, so... > >> >> > >> > > >> > Personally I find those who don't read the man page, and then claim > that > >> > documented > >> > behavior is a "bug" are the real "perverts". They expect documented > >> > behavior to work > >> > some way other than is documented... How is that not perverted? > >> > >> You're arguing "like a girl". I didn't say the documented behavior was a > > > > > > uh? really? > > Please go away, it's already bad enough you are discussing things you > don't > > fully understand without being sexist on top of that. > > Isn't the whole point of discussing better understanding? If you have to > fully understand a thing to be allowed to discuss it, then there will be > no discussion allowed. > > You're too easily stoked. Please don't be so sensitive. Notice the double > quotes? I'm using the stereotype as a shorthand. Stereotypes exist and > are widely understood, much like idioms. > > In any event, this is but irrelevant to the discussion. Do not seize the > red herring. > It is fully relevant when you use a sexist stereotype as an argument.
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 5:51 PM, Pierre Gaston wrote: > > > On Thu, Sep 25, 2014 at 12:42 PM, lolilolicon wrote: >> >> On Thu, Sep 25, 2014 at 7:19 AM, Linda Walsh wrote: >> > lolilolicon wrote: >> >> >> >> I don't expect more than a dozen who rely on this... but bash >> >> programmers can be quite the perverts, so... >> >> >> > >> > Personally I find those who don't read the man page, and then claim that >> > documented >> > behavior is a "bug" are the real "perverts". They expect documented >> > behavior to work >> > some way other than is documented... How is that not perverted? >> >> You're arguing "like a girl". I didn't say the documented behavior was a > > > uh? really? > Please go away, it's already bad enough you are discussing things you don't > fully understand without being sexist on top of that. Isn't the whole point of discussing better understanding? If you have to fully understand a thing to be allowed to discuss it, then there will be no discussion allowed. You're too easily stoked. Please don't be so sensitive. Notice the double quotes? I'm using the stereotype as a shorthand. Stereotypes exist and are widely understood, much like idioms. In any event, this is but irrelevant to the discussion. Do not seize the red herring.
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 12:42 PM, lolilolicon wrote: > On Thu, Sep 25, 2014 at 7:19 AM, Linda Walsh wrote: > > lolilolicon wrote: > >> > >> I don't expect more than a dozen who rely on this... but bash > >> programmers can be quite the perverts, so... > >> > > > > Personally I find those who don't read the man page, and then claim that > > documented > > behavior is a "bug" are the real "perverts". They expect documented > > behavior to work > > some way other than is documented... How is that not perverted? > > You're arguing "like a girl". I didn't say the documented behavior was a uh? really? Please go away, it's already bad enough you are discussing things you don't fully understand without being sexist on top of that.
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 7:19 AM, Linda Walsh wrote:
> lolilolicon wrote:
>>
>> I don't expect more than a dozen who rely on this... but bash
>> programmers can be quite the perverts, so...
>>
>
> Personally I find those who don't read the man page, and then claim that
> documented
> behavior is a "bug" are the real "perverts". They expect documented
> behavior to work
> some way other than is documented... How is that not perverted?
You're arguing "like a girl". I didn't say the documented behavior was a
bug. By "perverts", I meant "clever, sometimes too clever".
By that "but", I also meant to deliver that my expectation would
probably turn out to be wrong, "because bash programmers".
I don't mean to sound offensive. Please don't take it too heavily.
> I also find redefining functions with every invocation of bash a major waste
> of cpu cycles.
>
> Tons of functions are defined these days with all the support for
> bash completions.
I do not think about optimization. My opinion is the need for the
programmer to think about optimization (that's not relevant to the
problem at hand) is a symptom of bad design (of the language / tool), or
perhaps the programmer is using wrong tool / using the tool wrong.
As to your particular argument for completions: I think zsh optimizes
for this with autoloading mechanism, which is a better design.
> In scripts, I usually define *aliases* (which are processed before
> functions)
>
> If you want to execute commands in a script, you must make sure
> you are executing those commands. So first, those script better set the
> PATH to a
> standard path. Any script that doesn't has no right to complain about this
> "bug".
>
> 2nd, how many scripts define utils into vars:
>
> cp=/usr/bin/cp
>
> $cp
>
> ---
> The safe way to define cmnds which I try to use in most of my scripts is to
> define
> them as ALIASES that get expanded before functions.
>
> Example:
>
> #!bin/bash
>
> function true () { false; }
>
> if true; then echo "true==true"; else echo "true==false" ; fi
>
> # above demonstrates what seems to be the bug (a function overriding a
> command)
> # how it got there, is not entirely relevant
>
> #-- fixed example:
>
> PATH=/sbin:/usr/sbin:/bin:/usr/sbin:$PATH
> shopt -s expand_aliases
> alias true=$(type -P true)
>
> if true; then echo "true2==true2"; else echo "true2==false2" ; fi
>
>
> The 2nd example works because it defines the PATH and defines an an alias
> expansion that translates to the abs path that the alias was defined with.
>
> If people programmed safely to begin with, this wouldn't have come up as a
> bug, but a feature.
Isn't the mere fact that you have to do this round-around thing to
"program safely" a proof by demonstration that exported functions are
trouble?
>From a theoretical stand point, function export is really a hack. Or a
backdoor as termed by Eric Blake in another thread. When you export a
function, bash really creates and exports a variable,
% bash -c 'fun() { :;}; export -f fun; env -0 | grep -z "^fun="'
fun=() { :
}
So the token '() {' is the only "protocol" for bash function export /
import. A bit too magic, IMO. And of course it forbids importing normal
variables that match the token.
Re: Issues with exported functions
2014-09-25 10:43 GMT+02:00 Davide Brini :
> I'm not arguing about anything, I just have a question. I understand that
> with the current method used to export functions, it is not possible to
> export a variable to a child whose value begins exactly with the characters
> in question. A quick test seems to confirm that:
>
> $ x='() { echo a; }' bash -c 'echo "$x"'
>
> $ x='() { echo a; }' bash -c 'echo "$x"'
> () { echo a; }
>
>
> So is there a way to be able to export variables with arbitrary values,
> including '() {' ? Sorry if this has been discussed before.
>
There's the -p flag, but it has other side effects
$ x='() { echo a; }' bash -pc 'echo "$x"'
() { echo a; }
I also dislike that it parses exported functions by default; it violates
the "don't treat data as code" rule. Sure would be nice if there was a
separate flag that only disables parsing of exported functions.
--
Geir Hauge
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 11:06 AM, lolilolicon wrote:
> On Thu, Sep 25, 2014 at 9:35 AM, Chet Ramey wrote:
> > On 9/24/14, 3:44 PM, lolilolicon wrote:
> >
> >> Personally, I have never needed this feature. I would vote for its
> >> removal: It's very surprising, creates bugs, and is not very useful.
> >
> > There are more things in heaven and earth that are dreamt of in your
> > philosophy.
>
> OK guys! Exported functions are widely used by experts, I get it now.
>
> >
> >> Otherwise, if this feature is going to stay (can anyone enlighten me why
> >> it's useful?), please document it explicitly.
> >
> > Function export is documented. The exact mechanism need not be.
>
> Sure, the mechanism need not be documented, if it didn't matter on the
> interface level. But it does. In particular,
>
> % pat='() { $:*;}' bash -c 'tr "$pat" _ <<< "(x){1}"'
> (x){1}
>
> (This is bash 4.3.25)
>
> This is not the best example, but you get the idea.
>
> Perhaps you have plans to change the implementation?
>
>
How many instance have you found since the introduction of this feature
more than 20 years ago?
Re: Issues with exported functions
On Wed, 24 Sep 2014 21:35:19 -0400, Chet Ramey wrote:
> On 9/24/14, 3:44 PM, lolilolicon wrote:
>
> > Personally, I have never needed this feature. I would vote for its
> > removal: It's very surprising, creates bugs, and is not very useful.
>
> There are more things in heaven and earth that are dreamt of in your
> philosophy.
>
> > Otherwise, if this feature is going to stay (can anyone enlighten me why
> > it's useful?), please document it explicitly.
>
> Function export is documented. The exact mechanism need not be.
I'm not arguing about anything, I just have a question. I understand that
with the current method used to export functions, it is not possible to
export a variable to a child whose value begins exactly with the characters
in question. A quick test seems to confirm that:
$ x='() { echo a; }' bash -c 'echo "$x"'
$ x='() { echo a; }' bash -c 'echo "$x"'
() { echo a; }
So is there a way to be able to export variables with arbitrary values,
including '() {' ? Sorry if this has been discussed before.
Thanks
--
D.
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 9:35 AM, Chet Ramey wrote:
> On 9/24/14, 3:44 PM, lolilolicon wrote:
>
>> Personally, I have never needed this feature. I would vote for its
>> removal: It's very surprising, creates bugs, and is not very useful.
>
> There are more things in heaven and earth that are dreamt of in your
> philosophy.
OK guys! Exported functions are widely used by experts, I get it now.
>
>> Otherwise, if this feature is going to stay (can anyone enlighten me why
>> it's useful?), please document it explicitly.
>
> Function export is documented. The exact mechanism need not be.
Sure, the mechanism need not be documented, if it didn't matter on the
interface level. But it does. In particular,
% pat='() { $:*;}' bash -c 'tr "$pat" _ <<< "(x){1}"'
(x){1}
(This is bash 4.3.25)
This is not the best example, but you get the idea.
Perhaps you have plans to change the implementation?
Re: Issues with exported functions
2014-09-25 6:23 GMT+02:00 Linda Walsh :
>
> Maybe exporting fun?
>
> export fun='() { :;}'
>> bash -c 'declare -pf fun'
>>
> bash: line 0: declare: fun: not found
> ...
> I've never seen functions created with an assignment. Is this a
> new syntax in 4.3?
>
> (still in 4.2.43 here)...
Bash has had this feature since "forever"
$ fun='() { echo "$BASH_VERSION";}' bash1 -c fun
1.14.7(1)
Your bash 4.2.43 is no exception, but the way declare's -p and -f
options interact did change in 4.3, so try with just ''declare -f fun''
instead
--
Geir Hauge
Re: Issues with exported functions
lolilolicon wrote:
Obviously, the newly disclosed CVE-2014-6271 is pretty bad.
It's been patched now, but I think it's worthwhile to further discuss
how exported functions are implemented in bash.
I'm no bash expert: before today I didn't even realize bash functions
can be exported. And I certainly wouldn't expect this to work:
% fun='() { :;}' bash -c 'declare -pf fun'
fun ()
{
:
}
declare -fx fun
So Magic!
??? It doesn't seem to work for me... what am I missing?
fun='() { :;}' bash -c 'declare -pf fun'
bash: line 0: declare: fun: not found
Maybe exporting fun?
export fun='() { :;}'
bash -c 'declare -pf fun'
bash: line 0: declare: fun: not found
...
I've never seen functions created with an assignment. Is this a
new syntax in 4.3?
(still in 4.2.43 here)...
Because bash interprets a variable in the environment that starts with
'() {' as a function, bash cannot import a *variable* whose value
matches that pattern, eg
% var='() {' bash -c 'declare -p var'
bash: var: line 1: syntax error: unexpected end of file
bash: error importing function definition for `var'
bash: line 0: declare: var: not found
This is an undocumented "feature".
Since the only way for bash to support exporting / importing function
definitions via environment variables is to have some magic token like
'() {', the question comes down to whether bash should support exported
functions at all.
Personally, I have never needed this feature. I would vote for its
removal: It's very surprising, creates bugs, and is not very useful.
Otherwise, if this feature is going to stay (can anyone enlighten me why
it's useful?), please document it explicitly.
As it is currently, this is a real pitfall.
Re: Issues with exported functions
On 9/24/14, 3:44 PM, lolilolicon wrote: > Personally, I have never needed this feature. I would vote for its > removal: It's very surprising, creates bugs, and is not very useful. There are more things in heaven and earth that are dreamt of in your philosophy. > Otherwise, if this feature is going to stay (can anyone enlighten me why > it's useful?), please document it explicitly. Function export is documented. The exact mechanism need not be. > As it is currently, this is a real pitfall. Well, certainly, because there's a bug in the (unpatched) implementation. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
On Sep 24, 2014, at 4:06 PM, lolilolicon wrote:
> On Thu, Sep 25, 2014 at 3:53 AM, Greg Wooledge wrote:
>>
>> So, if Chet removes the feature, it would probably break something that
>> someone cares about. Maybe there could be a compile-time option to
>> disable it. Maybe there already is -- I didn't look.
Many bash completion libraries also rely on function exports.
> I don't expect more than a dozen who rely on this... but bash
> programmers can be quite the perverts, so...
A significant number of us have actually read the manual and rely on the
ability of bash to export functions. I have literally hundreds of exported
functions in my environment. Some are defined in a setup file '~/.bash_once',
some get built on the fly by that file. As you might guess, ~/.bash_once is
invokes only once per session by my .bashrc &.bash_login with something like:
if [[ 0 == "${SET_ONCE:=0}" ]] ; then
if [[ -f ~/.bash_once ]] ; then
. ~/.bash_once
else
echo "No ~/.bash_once file for '~/.bashrc' to invoke." >&2
fi
fi
.bash_once defines SET_ONCE and loads literally hundreds of environment
variables and exports many shell functions that would otherwise have to be
defined in .bashrc and processed on every freaking run. .bash_once is about 50
times larger than .bashrc and .bash_login. Fast. Very fast. But without
exportable functions, it wouldn't work at all.
As an exercise for the student, consider the utility of this simplified excerpt:
for SYSTEM in \
{foo,bar,baz}.dec.school.edu \
{alligator,snake-skin,lizard}.reptiles.work.com \
{misery,serenity,frailty}.films.home.org \
; do # Strip off domain, use dash-less name as function name
export FNAME=${SYSTEM%%.*}
export FNAME=${FNAME//-/}
eval $(echo "$FNAME() { ssh $SYSTEM" '"$@";};'" export -f $FNAME")
unset SYSTEM FNAME
done
Hint - source those lines, then give the command 'builtin type snakeskin'.
It's probably too much overhead for every bash invocation, but if you only do
it once per session, it's damned useful.
Consider this one vote against removing function exports.
Steve
Re: Issues with exported functions
lolilolicon wrote:
I don't expect more than a dozen who rely on this... but bash
programmers can be quite the perverts, so...
Personally I find those who don't read the man page, and then claim that
documented
behavior is a "bug" are the real "perverts". They expect documented
behavior to work
some way other than is documented... How is that not perverted?
I also find redefining functions with every invocation of bash a major
waste of cpu cycles.
Tons of functions are defined these days with all the support for
bash completions.
Ages ago, back before bash-2 or 3, I did timings and re-reading in all
functions
every invocation of bash is as bad as wiping the environment and
processing it
as a login.
If that's what you want to do, then do it. But I rely on exported
functions to patch
up and reconstruct my environment. I've lobbied heavily in the "sudo"
area to have them
fix their automatic clearing of functions -- and supposedly this will be
fixed in an upcoming
release.
In scripts, I usually define *aliases* (which are processed before
functions)
If you want to execute commands in a script, you must make sure
you are executing those commands. So first, those script better set
the PATH to a
standard path. Any script that doesn't has no right to complain about
this "bug".
2nd, how many scripts define utils into vars:
cp=/usr/bin/cp
$cp
---
The safe way to define cmnds which I try to use in most of my scripts is
to define
them as ALIASES that get expanded before functions.
Example:
#!bin/bash
function true () { false; }
if true; then echo "true==true"; else echo "true==false" ; fi
# above demonstrates what seems to be the bug (a function overriding a
command)
# how it got there, is not entirely relevant
#-- fixed example:
PATH=/sbin:/usr/sbin:/bin:/usr/sbin:$PATH
shopt -s expand_aliases
alias true=$(type -P true)
if true; then echo "true2==true2"; else echo "true2==false2" ; fi
The 2nd example works because it defines the PATH and defines an an alias
expansion that translates to the abs path that the alias was defined with.
If people programmed safely to begin with, this wouldn't have come up as
a bug, but a feature.
Re: Issues with exported functions
On 9/24/14, 4:07 PM, Greg Wooledge wrote:
> On Thu, Sep 25, 2014 at 03:54:19AM +0800, lolilolicon wrote:
>> I think almost as severe as CVE-2014-6271 is that it's still possible to
>> mask commands in a bash script by changing it's environment.
>>
>> For example, true='() { false;}' or grep='() { /bin/id;}' ...
>
> I'm still waiting for someone to successfully exploit this and post
> it to Chet. Or maybe someone already did, and it's being kept quiet.
Not really. This has always been possible: bash searches the shell
function space before $PATH and exported functions still work, so you
don't need to futz around with variables.
You can simply export a function named `grep', the child shell will
find it, and it will use it. This is one of the reasons that setuid
shells and restricted shells don't import shell functions.
You can usually achieve the same effect by putting a script named grep
in a private bin directory and arranging that that directory appear before
/bin in your $PATH. This, however, can be defeated by shell scripts that
set their own value of $PATH or use full pathnames to commands.
You can cause misbehavior using this, but it will only affect you.
This is not a privilege escalation vulnerability. The danger is how
it can be exploited remotely.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 03:54:19AM +0800, lolilolicon wrote:
> I think almost as severe as CVE-2014-6271 is that it's still possible to
> mask commands in a bash script by changing it's environment.
>
> For example, true='() { false;}' or grep='() { /bin/id;}' ...
I'm still waiting for someone to successfully exploit this and post
it to Chet. Or maybe someone already did, and it's being kept quiet.
Of course, this category of exploit would require a reasonable guess
about which commands a script is going to use.
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 3:53 AM, Greg Wooledge wrote:
> On Thu, Sep 25, 2014 at 03:44:23AM +0800, lolilolicon wrote:
>> Otherwise, if this feature is going to stay (can anyone enlighten me why
>> it's useful?), please document it explicitly.
>
> First, it is documented:
>
> Functions may be exported so that subshells automatically have them
> defined with the -f option to the export builtin.
>
> (Good luck finding that if you didn't know to look for it, though.)
Yeah, I did find that part. But it does not specify *how* it's done,
i.e. there is no mention of '() {', and by reading the above, one
wouldn't expect my first example to work.
>
> Second, it's "useful" in niche cases like this:
>
> foo() { ...; }
> export -f foo
> find . -type f -exec bash -c 'for f; do foo "$f"; done' _ {} +
Oh, this command line feels so wrong...
>
> So, if Chet removes the feature, it would probably break something that
> someone cares about. Maybe there could be a compile-time option to
> disable it. Maybe there already is -- I didn't look.
I don't expect more than a dozen who rely on this... but bash
programmers can be quite the perverts, so...
Re: Issues with exported functions
I think almost as severe as CVE-2014-6271 is that it's still possible to
mask commands in a bash script by changing it's environment.
For example, true='() { false;}' or grep='() { /bin/id;}' ...
Re: Issues with exported functions
On Thu, Sep 25, 2014 at 03:44:23AM +0800, lolilolicon wrote:
> Otherwise, if this feature is going to stay (can anyone enlighten me why
> it's useful?), please document it explicitly.
First, it is documented:
Functions may be exported so that subshells automatically have them
defined with the -f option to the export builtin.
(Good luck finding that if you didn't know to look for it, though.)
Second, it's "useful" in niche cases like this:
foo() { ...; }
export -f foo
find . -type f -exec bash -c 'for f; do foo "$f"; done' _ {} +
So, if Chet removes the feature, it would probably break something that
someone cares about. Maybe there could be a compile-time option to
disable it. Maybe there already is -- I didn't look.
Issues with exported functions
Obviously, the newly disclosed CVE-2014-6271 is pretty bad.
It's been patched now, but I think it's worthwhile to further discuss
how exported functions are implemented in bash.
I'm no bash expert: before today I didn't even realize bash functions
can be exported. And I certainly wouldn't expect this to work:
% fun='() { :;}' bash -c 'declare -pf fun'
fun ()
{
:
}
declare -fx fun
So Magic!
Because bash interprets a variable in the environment that starts with
'() {' as a function, bash cannot import a *variable* whose value
matches that pattern, eg
% var='() {' bash -c 'declare -p var'
bash: var: line 1: syntax error: unexpected end of file
bash: error importing function definition for `var'
bash: line 0: declare: var: not found
This is an undocumented "feature".
Since the only way for bash to support exporting / importing function
definitions via environment variables is to have some magic token like
'() {', the question comes down to whether bash should support exported
functions at all.
Personally, I have never needed this feature. I would vote for its
removal: It's very surprising, creates bugs, and is not very useful.
Otherwise, if this feature is going to stay (can anyone enlighten me why
it's useful?), please document it explicitly.
As it is currently, this is a real pitfall.
