Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
2020-02-03 14:32:57 +0100, casper@oracle.com: [...] > Right. I think it may need some fine tuning but I think it is fine to > avoid the shell when it is not needed. Yes, at least (beside what's already done): - that optimisation must be disabled if the first word is a builtin, special builtin, builtin alias or function, or keyword of the corresponding shell. (beware that for /bin/sh (ksh93), the list of builtins depends on $PATH (if /opt/ast/bin is in front of $PATH, a few more builtins are enabled) $ PATH=/opt/ast/bin:$PATH sh -c 'builtin;alias' | wc -l 82 (plus the keywords) - it must be disabled if the code argument starts with - or + - if the value of the $SHELL environment variable starts with r. There may be other env variables (like _AST_FEATURES) that affect the way the shell parses and runs simple commands. > I was not aware that ksh was all that dangerous; especially as it allows > crossing privilege boundaries using environment variable. It's not limited to ksh. In all shells, you mustn't use unsanitized data in arithmetic expressions. Some shells are worse than others. In dash for instance, the exposure is limitted to $(($var)) / $(($1)), and the damage is limited to assigning variables (var=PATH=7734). $((var)) there is OK (anything other than octal, hex or decimal constants with optional -/+ sign and blanks triggers an error). > Not quite as bad as "Shellshock"; not even close. Still another reason to > avoid the shell when it not actually needed to start a new command. The vulnerability in this case is not in the shell, but in the scripts using that feature (if they forget to sanitize data before using in arithmetic context). The feature could be seen as a misfeature though as it makes it difficult to write safe shell code. That can't be fully fixed though as long as $(($1)) is required (by POSIX) to evaluate the arithmetic expression stored in the first positional parameter. > I'm not sure why we ended up in Solaris with 18 commands which are > basically built-in ksh93 commands that make little sense as individual > executables: > > aliascd fc getopts jobs printtest ulimit > unalias > bg command fg hash kill read type umaskwait > > It seems that is being tested in XPG4.os/procenv/confstr/ > > The only ones that makes sense are "kill" & "print". [...] Except for "print", that's a POSIX requirement (which many systems ignore) as non-special builtins have to be available as standalone commands (at least for exec*p(), env, find -exec, and all the commands that can execute commands). -- Stephane
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
>"casper@oracle.com" wrote: >> The only ones that makes sense are "kill" & "print". > >I would say that "print" is not needed since it is not required to be callable >via exec(), since it is a ksh88/ksh93 private builtin. Right "print" is not tested for in the test suite. Casper
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
"casper@oracle.com" wrote: > I'm not sure why we ended up in Solaris with 18 commands which are > basically built-in ksh93 commands that make little sense as individual > executables: > > aliascd fc getopts jobs printtest ulimit > unalias > bg command fg hash kill read type umaskwait > > It seems that is being tested in XPG4.os/procenv/confstr/ > > The only ones that makes sense are "kill" & "print". I would say that "print" is not needed since it is not required to be callable via exec(), since it is a ksh88/ksh93 private builtin. Jörg -- EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
>2020-02-03 12:40:45 +0100, Joerg Schilling: >[...] >> > It looks like it's caused by an "optimisation" in its >> > libc:exec*(), so /usr/xpg4/bin/sh and POSIX are not to blame >> > after all. >> >> To which Solaris version does this apply? > >That was 11.4 Yes. >> > $ ksh -c 'printf %d 1+1' >> > printf: 1+1 not completely converted >> >> This is the correct expected output for /usr/bin/printf > >Yes, that's the point, /usr/bin/printf was called instead of ksh >(ksh93 here) and its builtin. > >> > What? ksh's printf does take arithmetic expressions as arguments >> > for %d. >> > >> > $ ksh -c 'printf %d 1+1;' >> > 2 >> > $ ksh -c 'printf %d 1+1' ksh >> > 2 >> > >> > Adding that ; special shell character or an extra argument >> > disables the optimisation. >> >> But this seems to be an easteregg from ksh93. >[...] > >printf %d 1+1 to output 2 is expected in ksh where in most >places where a number is expected, any arithmetic expression is >accepted as well. That behaviour was also copied by zsh. > >It causes all sorts of security headaches as arithmetic expressions can assign >variables (like for IFS=1234567890, PATH=7734) or run arbitrary code (like >a=[$(evil)0]) > >$ a=2 b='a[$(evil)0]' ksh -c 'printf %d b' # /usr/bin/printf run >printf: b expected numeric value >$ a=2 b='a[$(evil)0]' ksh -c 'printf "%d" b' # ksh printf run >ksh: printf: evil: not found [No such file or directory] > >The easteregg here is more solaris libc:exec*() bypassing the >execution of a shell in some cases. Right. I think it may need some sine tuning but I think it is fine to avoid the shell when it is not needed. I was not aware that ksh was all that dangerous; especially as it allows crossing privilege boundaries using environment variable. Not quite as bad as "Shellshock"; not even close. Still another reason to avoid the shell when it not actually needed to start a new command. I'm not sure why we ended up in Solaris with 18 commands which are basically built-in ksh93 commands that make little sense as individual executables: aliascd fc getopts jobs printtest ulimit unalias bg command fg hash kill read type umaskwait It seems that is being tested in XPG4.os/procenv/confstr/ The only ones that makes sense are "kill" & "print". Casper
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
2020-02-03 12:40:45 +0100, Joerg Schilling: [...] > > It looks like it's caused by an "optimisation" in its > > libc:exec*(), so /usr/xpg4/bin/sh and POSIX are not to blame > > after all. > > To which Solaris version does this apply? That was 11.4 > > $ ksh -c 'printf %d 1+1' > > printf: 1+1 not completely converted > > This is the correct expected output for /usr/bin/printf Yes, that's the point, /usr/bin/printf was called instead of ksh (ksh93 here) and its builtin. > > What? ksh's printf does take arithmetic expressions as arguments > > for %d. > > > > $ ksh -c 'printf %d 1+1;' > > 2 > > $ ksh -c 'printf %d 1+1' ksh > > 2 > > > > Adding that ; special shell character or an extra argument > > disables the optimisation. > > But this seems to be an easteregg from ksh93. [...] printf %d 1+1 to output 2 is expected in ksh where in most places where a number is expected, any arithmetic expression is accepted as well. That behaviour was also copied by zsh. It causes all sorts of security headaches as arithmetic expressions can assign variables (like for IFS=1234567890, PATH=7734) or run arbitrary code (like a=[$(evil)0]) $ a=2 b='a[$(evil)0]' ksh -c 'printf %d b' # /usr/bin/printf run printf: b expected numeric value $ a=2 b='a[$(evil)0]' ksh -c 'printf "%d" b' # ksh printf run ksh: printf: evil: not found [No such file or directory] The easteregg here is more solaris libc:exec*() bypassing the execution of a shell in some cases. -- Stephane
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
Stephane Chazelas wrote:
> 2020-02-01 10:47:46 +, Stephane Chazelas:
> [...]
> > That doesn't explain why it's different with ${0+type} or when
> > there's more than the one invocation of "type" in the script.
> [...]
>
> OK, I see what's going on.
>
> It looks like it's caused by an "optimisation" in its
> libc:exec*(), so /usr/xpg4/bin/sh and POSIX are not to blame
> after all.
To which Solaris version does this apply?
> From what I can gather from my tests, when exec*()'s filename
> argument is /bin/sh or any of its other paths (/usr/bin/sh,
> /bin/ksh, /usr/bin/ksh93, /bin/./sh...) or /usr/xpg4/bin/sh,
> (but not /usr/xpg4/bin/./sh), the first argument (argv[0]) is
> anything that doesn't start with "r", including "-sh", the
> second is "-c" (not "-cc", not "-uc"...) and the third is some
> very simple shell code, that doesn't contain non-ASCII
> characters nor shell special characters other than spc and tab,
> and no further argument then
> exec*() takes the shell's role at parsing the command line,
> splits it on spc and tab and tries to execute the corresponding
> command by itself. It it can't do it (command not found for
> instance), then it falls back to executing the shell normally.
Strange.
> $ ksh -c 'printf %d 1+1'
> printf: 1+1 not completely converted
This is the correct expected output for /usr/bin/printf
> What? ksh's printf does take arithmetic expressions as arguments
> for %d.
>
> $ ksh -c 'printf %d 1+1;'
> 2
> $ ksh -c 'printf %d 1+1' ksh
> 2
>
> Adding that ; special shell character or an extra argument
> disables the optimisation.
But this seems to be an easteregg from ksh93.
Jörg
--
EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
2020-02-01 10:47:46 +, Stephane Chazelas:
[...]
> That doesn't explain why it's different with ${0+type} or when
> there's more than the one invocation of "type" in the script.
[...]
OK, I see what's going on.
It looks like it's caused by an "optimisation" in its
libc:exec*(), so /usr/xpg4/bin/sh and POSIX are not to blame
after all.
>From what I can gather from my tests, when exec*()'s filename
argument is /bin/sh or any of its other paths (/usr/bin/sh,
/bin/ksh, /usr/bin/ksh93, /bin/./sh...) or /usr/xpg4/bin/sh,
(but not /usr/xpg4/bin/./sh), the first argument (argv[0]) is
anything that doesn't start with "r", including "-sh", the
second is "-c" (not "-cc", not "-uc"...) and the third is some
very simple shell code, that doesn't contain non-ASCII
characters nor shell special characters other than spc and tab,
and no further argument then
exec*() takes the shell's role at parsing the command line,
splits it on spc and tab and tries to execute the corresponding
command by itself. It it can't do it (command not found for
instance), then it falls back to executing the shell normally.
I suppose that's to optimise the simplest cases of system("some
command"), but causes several problems:
If the command is builtin, it changes its behaviour, as already
seen for "type" and /usr/xpg4/bin/sh but also:
$ ksh -c 'printf %d 1+1'
printf: 1+1 not completely converted
What? ksh's printf does take arithmetic expressions as arguments
for %d.
$ ksh -c 'printf %d 1+1;'
2
$ ksh -c 'printf %d 1+1' ksh
2
Adding that ; special shell character or an extra argument
disables the optimisation.
It also bypasses the SHELL=rsh restricted shell mode:
$ SHELL=rsh nawk 'BEGIN{system("/bin/echo test;")}'
sh: /bin/echo: restricted
$ SHELL=rsh nawk 'BEGIN{system("/bin/echo test")}'
test
The check for SHELL == rsh that should have been done by "sh",
is not done there as sh has been bypassed.
In
sh -c -e
It tries to execute -e, instead of (or before) running sh -c -e
which should report an error.
sh -c +c
Also tries to run a +c command instead of running sh.
--
Stephane
Re: Solaris /usr/xpg4/bin/sh builtin handling (Was: About printf %2$s)
2020-01-31 21:26:33 +, Stephane Chazelas:
[...]
> You might have been mislead by Solaris' /usr/xpg4/bin/sh
>
> On Solaris 11,
>
> $ /usr/xpg4/bin/sh -c 'type printf'
> printf is a shell builtin
>
> But
>
> $ truss -ft execve /usr/xpg4/bin/sh -c 'printf "x"'
> 1382: execve("/usr/xpg4/bin/sh", 0x7FEF43B27CC8, 0x7FEF43B27CE8) argc = 3
> 1383: execve("/usr/bin/getconf", 0x7FEF43B27A70, 0x7FEF43B27CE8) argc = 2
> 1382: execve("/usr/bin/printf", 0xFA6897198, 0xFA68971E0) argc = 2
[...]
Even weirder:
$ /usr/xpg4/bin/sh -c 'type printf'
printf is a shell builtin
$ /usr/xpg4/bin/sh -c 'type printf; type printf'
printf is /usr/bin/printf
printf is a tracked alias for /usr/bin/printf
$ /usr/xpg4/bin/sh -c '${0+type} printf'
printf is /usr/bin/printf
$ /usr/xpg4/bin/sh -c '"type" printf'
printf is /usr/bin/printf
$ /usr/xpg4/bin/sh -c 'type "printf"'
printf is /usr/bin/printf
$ cd ~/bin
$ PATH=. /usr/xpg4/bin/sh -c 'type printf'
printf is /export/home/chazelas/bin/printf
$ cd /usr/bin
$ PATH=. /usr/xpg4/bin/sh -c 'type printf'
printf is a shell builtin
$ PATH=$(/usr/xpg4/bin/getconf PATH) /usr/xpg4/bin/sh -c 'type printf'
printf is /usr/bin/printf
That seems to be because its "type" (at least when it's the only
command run in the script) behaves like the "type" found by a
$PATH lookup. And on Solaris 11, /usr/bin/type reports "printf"
as builtin. It looks like that /usr/bin/type, linked to 17 other
commands that are typically builtins is a special build of ksh93
dedicated to run builtins (it's actually larger than the ksh93
executable):
$ (exec -a eval /usr/bin/type 'echo "${.sh.version}"')
builtin: eval: restricted name
Version AJM 93u+ 2012-08-01
Which explains why printf is reported as builtin when a PATH
lookup of "type" finds /usr/bin/type instead of
/usr/xpg4/bin/type as printf is indeed a ksh93 builtin.
That doesn't explain why it's different with ${0+type} or when
there's more than the one invocation of "type" in the script.
It's not limited to "printf":
$ /usr/xpg4/bin/sh -c 'print -h'
print[9]: print: -h: unknown option
Usage: print [-enprsvC] [-f format] [-u fd] [string ...]
$ /usr/xpg4/bin/sh -c 'print "-h"'
/usr/xpg4/bin/sh: print: bad option(s)
That is completely messed up, and I think POSIX and its
messed-up builtin lookup in $PATH specification is largely to
blame.
--
Stephane
