Re: simple prob?
Ha ok, I got it, I was focused on our interactive session. Surely a script running on a UID behalf would be ill-advised to interpret (in the sense of shell evaluation) an arbitrary shell expression. On Fri, Jul 2, 2021 at 6:06 PM Greg Wooledge wrote: > On Fri, Jul 02, 2021 at 05:45:23PM +0200, Phi Debian wrote: > > Regarding the code injection I am not sure I got it. > > > > If you are sitting at a prompt, why would you trick > > > > unicorn:~$ njobs_ref 'x[0$(date>&2)]' > > > > when you could simply type > > unicorn:~$ date > > > > I assume protected script/source (the ones you can't write into), are > wise > > enough not to run command based on user input, in short I guess no > > protected script are doing thing like read in; eval $in :) that is the > > simplest code injection :) and then would never let you have a chance to > > enter 'x[0$(date>&2)]' at any time. > > For functions that you've written exclusively for personal use, it's > not an immediate concern. It's more of a thing that you want to be > aware of for the future. > > Where it becomes important is when you're writing scripts for other > people to use, or which run as different user accounts, or with > different privileges. > > The classic example of this is a script that's run by a web server in a > CGI environment, which accepts query parameters from the end user. If > one of those query parameters is used in an unsafe way, it can execute > undesired commands on the web server. > > Of course, there are *many* other places that shell scripts are used, > such as booting an operating system, starting various services, and > so on. In some of these cases, there is no external input being read, > or the external inputs are "trusted" files owned and edited only by > the system admin (root). But in other cases, untrusted input may be > read. > > So, there's merit in adopting a proactive strategy to shell script > security. Maintaining a slightly paranoid mindset can help you spot > potential security holes and possibly avoid disasters. > >
Re: simple prob?
On Fri, Jul 02, 2021 at 05:45:23PM +0200, Phi Debian wrote: > Regarding the code injection I am not sure I got it. > > If you are sitting at a prompt, why would you trick > > unicorn:~$ njobs_ref 'x[0$(date>&2)]' > > when you could simply type > unicorn:~$ date > > I assume protected script/source (the ones you can't write into), are wise > enough not to run command based on user input, in short I guess no > protected script are doing thing like read in; eval $in :) that is the > simplest code injection :) and then would never let you have a chance to > enter 'x[0$(date>&2)]' at any time. For functions that you've written exclusively for personal use, it's not an immediate concern. It's more of a thing that you want to be aware of for the future. Where it becomes important is when you're writing scripts for other people to use, or which run as different user accounts, or with different privileges. The classic example of this is a script that's run by a web server in a CGI environment, which accepts query parameters from the end user. If one of those query parameters is used in an unsafe way, it can execute undesired commands on the web server. Of course, there are *many* other places that shell scripts are used, such as booting an operating system, starting various services, and so on. In some of these cases, there is no external input being read, or the external inputs are "trusted" files owned and edited only by the system admin (root). But in other cases, untrusted input may be read. So, there's merit in adopting a proactive strategy to shell script security. Maintaining a slightly paranoid mindset can help you spot potential security holes and possibly avoid disasters.
Re: simple prob?
Ha yes I lost sight of <(jobs -t), I think it is a good improvement to the
challenge :)
Regarding the local t, you right, I missed to precise that any function
that do output parameters via nameref should namespace all their locals and
check the namespace that is indeed a bit combersome, that's why I tend to
stay away from that :) yet it was asked..
Here I typed to fast (challenged by providing the shortest answer :) ) and
throwed away my basics principles :)
Regarding the code injection I am not sure I got it.
If you are sitting at a prompt, why would you trick
unicorn:~$ njobs_ref 'x[0$(date>&2)]'
when you could simply type
unicorn:~$ date
I assume protected script/source (the ones you can't write into), are wise
enough not to run command based on user input, in short I guess no
protected script are doing thing like read in; eval $in :) that is the
simplest code injection :) and then would never let you have a chance to
enter 'x[0$(date>&2)]' at any time.
In all case since doing output parameter require some kind of name spacing
check it would reject input of the form 'x[0$(date>&2)]'
I guess a typical output parameter function should ressemble something like
this
function foo
{ [[ ! "$1" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]] &&
echo "Invalid parameter name '$1'" >&2 &&
return 1
[ "${1:0:4}" = "foo_" ] && echo "Namespace collision '$1' " >&2 &&
return 1
typeset -n foo_out="$1"
foo_out="value"
}
This one reject the bad 'x[0$(date>&2)]' only accept scalar variable names
as output parameter and reject scalar in the foo() namespace.
On Fri, Jul 2, 2021 at 2:01 PM Greg Wooledge wrote:
> On Fri, Jul 02, 2021 at 09:09:34AM +0200, Phi Debian wrote:
> > PW$ function njobs
> > > { [ "$1" != "n" ] && typeset -n n="$1"
> > >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > > }
>
> <<<$() is a poor imitation of < <() which is what you really want.
>
> readarray -t t < <(jobs); n=${#t[@]}
>
> Combining <<< and $() only gives an approximation of the same result
> (the command substitution strips off all trailing newline characters,
> and then the here-string syntax appends one newline), and it's less
> efficient, because it involves slurping the entire input into memory,
> then writing it out to a temporary file, then opening the temporary
> file for input and unlinking it, then reading it back in a second time.
>
> Using < <() avoids the newline alterations and the temporary file.
>
> (Note: in some sufficiently new versions of bash, there may not be a
> temporary file in some cases. But it's still not the best solution,
> as it still involves storing and reading the whole output multiple times.)
>
>
> Stepping back a moment, you're using the name reference version of the
> "pass an output variable by reference" strategy. This requires bash 4.3,
> which is reasonable, and it requires some additional sanity checks which
> you did not show.
>
> What's really interesting to me is that you did a *partial* sanity check,
> refusing to create a circular name reference if the user passed "n"
> as their output variable name. But you forgot to check for "t", which is
> another local variable you're using. Also, in Linda's original example,
> the output variable was literally named "n", so choosing that as your
> name reference and explicitly disallowing it is a really spiteful choice.
>
> Finally, you didn't do any sanity checking of the output variable name
> (beyond comparing it to one of your two local variable names), so your
> function is susceptible to the same code injection attacks we discussed
> earlier in the thread.
>
> unicorn:~$ njobs_ref() { typeset -n n="$1"; n=42; }
> unicorn:~$ njobs_ref 'x[0$(date>&2)]'
> Fri Jul 2 07:54:49 EDT 2021
>
> As I mentioned a few days ago, all variants of the "pass a variable name
> by reference" method are basically equivalent to each other, and all
> of them need input sanity checking in order to avoid code injections.
> (Some of the variants avoid *some* flavors of code injection, but none of
> them avoid this one.)
>
>
Re: simple prob?
On Fri, Jul 02, 2021 at 09:09:34AM +0200, Phi Debian wrote:
> PW$ function njobs
> > { [ "$1" != "n" ] && typeset -n n="$1"
> >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > }
<<<$() is a poor imitation of < <() which is what you really want.
readarray -t t < <(jobs); n=${#t[@]}
Combining <<< and $() only gives an approximation of the same result
(the command substitution strips off all trailing newline characters,
and then the here-string syntax appends one newline), and it's less
efficient, because it involves slurping the entire input into memory,
then writing it out to a temporary file, then opening the temporary
file for input and unlinking it, then reading it back in a second time.
Using < <() avoids the newline alterations and the temporary file.
(Note: in some sufficiently new versions of bash, there may not be a
temporary file in some cases. But it's still not the best solution,
as it still involves storing and reading the whole output multiple times.)
Stepping back a moment, you're using the name reference version of the
"pass an output variable by reference" strategy. This requires bash 4.3,
which is reasonable, and it requires some additional sanity checks which
you did not show.
What's really interesting to me is that you did a *partial* sanity check,
refusing to create a circular name reference if the user passed "n"
as their output variable name. But you forgot to check for "t", which is
another local variable you're using. Also, in Linda's original example,
the output variable was literally named "n", so choosing that as your
name reference and explicitly disallowing it is a really spiteful choice.
Finally, you didn't do any sanity checking of the output variable name
(beyond comparing it to one of your two local variable names), so your
function is susceptible to the same code injection attacks we discussed
earlier in the thread.
unicorn:~$ njobs_ref() { typeset -n n="$1"; n=42; }
unicorn:~$ njobs_ref 'x[0$(date>&2)]'
Fri Jul 2 07:54:49 EDT 2021
As I mentioned a few days ago, all variants of the "pass a variable name
by reference" method are basically equivalent to each other, and all
of them need input sanity checking in order to avoid code injections.
(Some of the variants avoid *some* flavors of code injection, but none of
them avoid this one.)
Re: simple prob?
On Fri, Jul 2, 2021 at 11:15 AM Alex fxmbsw7 Ratchev wrote: > good debugging, yea i missed the n # count char, i type stupidly from a > cell foun, .. > and yea leftover arr ekements exceots when u just use the first one as n > i just wanted to show shorter code > Yes I like those little one liners when other goes long way with multi fork/exec etc :), just fun challenges :)
Re: simple prob?
good debugging, yea i missed the n # count char, i type stupidly from a
cell foun, ..
and yea leftover arr ekements exceots when u just use the first one as n
i just wanted to show shorter code
i also made somewhen a which_pid_exited resolver etc
peace
On Fri, Jul 2, 2021, 11:10 Phi Debian wrote:
>
>
> On Fri, Jul 2, 2021 at 9:24 AM Alex fxmbsw7 Ratchev
> wrote:
>
>> jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
>>
>
> This give
> PW$ jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
> PW$ echo $jobsn
> 3644 3645 3646 3647
>
> I guess you meant jobsn=${#jobsn[@]}
> ^ You missed the '#'
>
> Yet there are some left over
> PW$ jobsn=( $( jobs -p ) ) jobsn=${#jobsn[@]}
> PW$ echo ${jobsn[@]}
> 4 3645 3646 3647
>
> So it is not clean, and the OP want a function with a named output arg.
>
> Yet to elaborate on your technic may be this one is a little cleaner
>
> PW$ jobsn=$(printf '%c' $(jobs -p)) jobsn=${#jobsn} ; echo $jobsn
> 4
>
Re: simple prob?
On Fri, Jul 2, 2021 at 9:24 AM Alex fxmbsw7 Ratchev
wrote:
> jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
>
This give
PW$ jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
PW$ echo $jobsn
3644 3645 3646 3647
I guess you meant jobsn=${#jobsn[@]}
^ You missed the '#'
Yet there are some left over
PW$ jobsn=( $( jobs -p ) ) jobsn=${#jobsn[@]}
PW$ echo ${jobsn[@]}
4 3645 3646 3647
So it is not clean, and the OP want a function with a named output arg.
Yet to elaborate on your technic may be this one is a little cleaner
PW$ jobsn=$(printf '%c' $(jobs -p)) jobsn=${#jobsn} ; echo $jobsn
4
Re: simple prob?
jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
On Fri, Jul 2, 2021, 09:10 Phi Debian wrote:
> On Tue, Jun 29, 2021 at 10:23 PM L A Walsh wrote:
>
> > I hope a basic question isn't too offtopic.
> > Say I have some number of jobs running:
> >
> > > jobs|wc -l
> > 3
> > ---
> >
> > Would like to pass a varname to njobs to store the answer in, like:
> >
> > So I can run:
> >
> > > njobs n
> > echo "$n"
> > 3
> >
> >
> This a a double question 'how to', and I see no bash bugs here.
>
> The 2 questions are
> - how do I pass a variable name as an output argument to a function ('n' in
> your 'jobs n' example)
> - how to set a variable in a sub command? that is no doable, a sub command
> can't return variable to its parent, so you obviously have to do things
> diffrently.
>
> A simple 2 liners, solve all this, with no code injection blah
>
>
> PW$ jobs
> [1] Running sleep 111 &
> [2] Running sleep 111 &
> [3]- Running sleep 111 &
> [4]+ Running sleep 111 &
>
> PW$
>
> PW$ function njobs
> > { [ "$1" != "n" ] && typeset -n n="$1"
> >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > }
>
> PW$ njobs n ; echo $n
> 4
>
> # explanations (you may skip here)
> #===
> [ "$1" != "n" ] && typeset -n n="$1"
> This make sure the given output variable name is a valid SHELL identifier,
> providing anything not valid in "$1" will break there.
> This also enforce that the given $1 output variable name doesn't match our
> own local nameref name, if it match we don't do our nameref, and re-use the
> upper scope output variable name, that by definition is a valid variable
> name if we got that far.
>
> typeset -a t
> define a local array that we will fill, being local mean the booking is
> done at function return
>
> readarray t <<<"$(jobs)" ;
> Fill the array with your command you want to count lines for.
>
> n=${#t[@]}
> Fill the output variable
>
> All is safe, all is clean, no 'apparent' temp file, no sub command :)
>
> Shell programing is fun :)
>
Re: simple prob?
On Tue, Jun 29, 2021 at 10:23 PM L A Walsh wrote:
> I hope a basic question isn't too offtopic.
> Say I have some number of jobs running:
>
> > jobs|wc -l
> 3
> ---
>
> Would like to pass a varname to njobs to store the answer in, like:
>
> So I can run:
>
> > njobs n
> echo "$n"
> 3
>
>
This a a double question 'how to', and I see no bash bugs here.
The 2 questions are
- how do I pass a variable name as an output argument to a function ('n' in
your 'jobs n' example)
- how to set a variable in a sub command? that is no doable, a sub command
can't return variable to its parent, so you obviously have to do things
diffrently.
A simple 2 liners, solve all this, with no code injection blah
PW$ jobs
[1] Running sleep 111 &
[2] Running sleep 111 &
[3]- Running sleep 111 &
[4]+ Running sleep 111 &
PW$
PW$ function njobs
> { [ "$1" != "n" ] && typeset -n n="$1"
>typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> }
PW$ njobs n ; echo $n
4
# explanations (you may skip here)
#===
[ "$1" != "n" ] && typeset -n n="$1"
This make sure the given output variable name is a valid SHELL identifier,
providing anything not valid in "$1" will break there.
This also enforce that the given $1 output variable name doesn't match our
own local nameref name, if it match we don't do our nameref, and re-use the
upper scope output variable name, that by definition is a valid variable
name if we got that far.
typeset -a t
define a local array that we will fill, being local mean the booking is
done at function return
readarray t <<<"$(jobs)" ;
Fill the array with your command you want to count lines for.
n=${#t[@]}
Fill the output variable
All is safe, all is clean, no 'apparent' temp file, no sub command :)
Shell programing is fun :)
Re: simple prob made into a security tragedy...oh well.... ;^/
On Thu, Jul 01, 2021 at 02:12:10AM -0700, L A Walsh wrote:
> What are you talking about?
>
> njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
>
> I don't see any arrays, let alone indexed.
The arrays were in the calls. We demonstrated a few cases already.
Since you clearly weren't paying attention, here they are again.
First, the classic code injection:
unicorn:~$ njobs 'x[0$(date>&2)]'
Thu Jul 1 07:12:35 EDT 2021
Second, a user who wants to (legitimately) use an existing array, with
a space in the key:
unicorn:~$ declare -A jobs
unicorn:~$ njobs 'jobs[first time]'
bash: printf: `jobs[first': not a valid identifier
Third, a user who wants to (legitimately) use an existing array, where
the indexed reference is accidentally treated as a glob:
unicorn:~$ touch jobsx
unicorn:~$ njobs 'jobs[sixth]'
unicorn:~$ declare -p jobs jobsx
declare -A jobs
declare -- jobsx="0
"
Fourth, a user who accidentally or maliciously passes additional printf
options:
unicorn:~$ njobs 'foo -v bar'
unicorn:~$ declare -p foo bar
bash: declare: foo: not found
declare -- bar="0
"
Quoting "$1" in your code would stop three of these four examples from
doing the wrong thing. It makes no difference in the first case.
Looking at the Subject: header reminds me of:
43. A "quick" or "simple" question will be neither.
What you perceive as a "simple prob" (and for the love of glob, can you at
least spell out "problem" correctly?!) is absolutely NOT simple. It's the
tip of an exploding carcinogenic iceberg full of frozen soul-eating
demons. Everyone knows about this iceberg. There are buoys all around
it warning sailors to stay away. All the navigational maps have marked
this territory as too dangerous.
Despite this, you've decided it would be a great idea to have a little
picnic on top of it.
And people don't understand why we're so cranky.
Re: simple prob made into a security tragedy...oh well.... ;^/
On 2021/06/29 19:11, Eli Schwartz wrote:
This is a ridiculous argument and you know it. You, personally, are
writing code which does not get used in security contexts, which is your
right. This in no way means that refusing to quote variables which
"cannot be word-split" stops *any* security errors. The "illegal input"
was not related to the security bypass (as Greg points out, removing the
space prevents word splitting and executes the same security bypass code).
Your response should have been:
More likely "is", if I needed security I wouldn't likely write in
a script language, but more like with audit, w/Biba integrity and
Bell-LaPadula sensitivity models that we planned to port to linux, I'd have
written it in 'C'.
Trix or Trusted IRIX was certified, for C2+ under the then, orange
book standard. Even had a 128-bit luid, which later implementers
changed to a less parallel 'loginuid', mainly for auditing.
I'd been presenting sgi's security plan at the linux security conference in
France, as well as some presentation in London. It seems I was good at
explaining what had been a confusing security model in the place of
my then manager. I wasn't good at politics, but my manager prided himself
on his bookshelf copy of Machiavelli's, 'The Prince' as having everything
a manager needed to know... among other things, for him to be able to
put a sensitivity+integrity Policy, 'SMACK' in the linux kernel.
Instead you are arguing in bad faith...
---
You are arguing about a 1-liner that took unfiltered output
from locate to search for keywords. You wanna work that up into
bad faith, good luck.
your code is flawed, it doesn't
correctly handle indexed arrays with spaces in the key and doesn't
forbid them either.
What are you talking about?
njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
I don't see any arrays, let alone indexed.
This won't protect against all code injections, of course;
---
It does in the target environment. The key is to look at the
security policy requirements and environment before going off and making
assumptions about "faith" that might bounce back when used for
design issues relating to a 1-line search expression.
Re: simple prob?
On 6/29/21 5:05 PM, L A Walsh wrote: So...hmmm...how is it that jobs picks up the right answer in what would seem to be a subshell? Special cased? Yes. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: simple prob?
On 2021/06/29 16:51, Greg Wooledge wrote:
On Tue, Jun 29, 2021 at 04:29:05PM -0700, L A Walsh wrote:
njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
Which is detected as "illegal input" and disallowed. If you don't enable
some security errors, they can't be as easily introduced.
Are you *still* insisting that your failure to quote is a SECURITY
FEATURE?
Come *on*!
In this case, not quoting was deliberately intended as variable
names wouldn't need it. Any security consideration was purely
secondary. I'm an avid quoter where it is needed, but I no longer
quote for the sake of quoting as I once did. In a similar manner
I try to not overuse parentheses, just for the sake of it.
As I stated before, my scripts are most often for myself. If I needed
security, I'd probably write in a compiled language rather than a
scripting one.
unicorn:~$ njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
unicorn:~$ njobs 'x[0$(date>&2)]'
Tue Jun 29 19:49:16 EDT 2021
All I had to do was remove the space. You're not even trying.
Your failure to quote is simply a failure. If you want to prevent
code injection attacks, you need to sanity-check the input.
There is no other way.
Re: simple prob?
On Tue, Jun 29, 2021 at 10:11:31PM -0400, Eli Schwartz wrote:
> On 6/29/21 7:29 PM, L A Walsh wrote:
> >>> njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
> It also rejects this perfectly reasonable code, which I arbitrarily
> decided not to support because if you want this you're weird:
>
> $ declare -A jobs
> $ njobs 'jobs[first run]'
> error: invalid characters [ ] found in var 'jobs[first run]'
> hacker detected...
It's not the weirdest thing I've seen. Without your validation:
unicorn:~$ declare -A jobs
unicorn:~$ njobs 'jobs[first]'
All cool, right? Uh
unicorn:~$ touch jobss
unicorn:~$ njobs 'jobs[second]'
unicorn:~$ declare -p jobs
declare -A jobs=([first]=$'0\n' )
unicorn:~$ declare -p jobss
declare -- jobss="0
"
Failure to quote strikes again. What if we fix it?
unicorn:~$ nqjobs() { printf ${1:+-v "$1"} "%s\n" "$(jobs |wc -l)"; }
unicorn:~$ nqjobs 'jobs[third]'
unicorn:~$ nqjobs 'jobs[sixth]'
unicorn:~$ declare -p jobs
declare -A jobs=([sixth]=$'0\n' [first]=$'0\n' [third]=$'0\n' )
Now it works even if there's a file which matches the glob pattern.
> Well, the diagnostic jumped to a conclusion that is probably false.
Your validation is too strict in this case. Right idea, wrong
implementation. Of course, one might argue that allowing an array with
an index is a dangerous precedent, and stick with your strict validation
on principle. That's a design decision, and I could see either one
being correct depending on the goals.
Oh, and by the way, there's at least one more category of errors that
failure-to-quote is susceptible to:
unicorn:~$ unset foo bar
unicorn:~$ njobs 'foo -v bar'
unicorn:~$ declare -p foo bar
bash: declare: foo: not found
declare -- bar="0
"
Without quotes, we get printf -v foo -v bar '%s\n' "$(jobs|wc -l)"
with the result shown.
With quotes, we get:
unicorn:~$ nqjobs 'foo -v bar'
bash: printf: `foo -v bar': not a valid identifier
It's not robust against code injection attacks (we need actual validation
for that), but at least it gives a sensible error message in some common
cases.
tl;dr: When in doubt, quote it. There is a REASON we beat this into
people's heads.
Re: simple prob?
On 6/29/21 7:29 PM, L A Walsh wrote:
> On 2021/06/29 15:49, Greg Wooledge wrote:
>> On Tue, Jun 29, 2021 at 02:58:28PM -0700, L A Walsh wrote:
>>
>>> njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
>>>
>>> Using that with your input:
>>>
>>> njobs 'x[0$(date >&2)]'
>>>
>>> bash: printf: `x[0$(date': not a valid identifier
>>>
>>
>> This is because you didn't quote "$1".
>
> $1 should never be quoted -- it is an identifier, and as such
> cannot contain a space. By quoting it, you are allowing inputs that
> would otherwise be filtered out because they are not valid variable
> names.
You do no filtering right now.
Here's how you filter:
njobs() {
if [[ $(id -un) = lindawalsh ]]; then
# Linda agrees to never misuse this function or let
# anyone else run code on her computer which does
:
elif [[ $1 = *[^a-zA-Z0-9_]* ]]; then
echo "error: invalid characters ${1//[a-zA-Z0-9_]} found in var
${1@Q}" >&2
echo "hacker detected..." >&2
# can be exit 13 if used in a script
return 1
fi
printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)";
}
Now, since I am not Linda and I am a big dummy when it comes to code
reuse and I *am* concerned I might stupidly use this code on input
provided by my friend the huge practical joker, I get this:
$ njobs 'x[0$(date >&2)]'
error: invalid characters [$( >&)] found in var 'x[0$(date >&2)]'
hacker detected...
Of course, I don't bother to quote $1 -- I cherish my right to not quote
that variable. But I know it's safe to do so -- if it has word splitting
tokens, they will raise an error.
Greg's fixed example is still caught by my code (unless the code is
running as the Linda login account):
$ njobs 'x[0$(date>&2)]'
error: invalid characters [$(>&)] found in var 'x[0$(date>&2)]'
hacker detected...
It also rejects this perfectly reasonable code, which I arbitrarily
decided not to support because if you want this you're weird:
$ declare -A jobs
$ njobs 'jobs[first run]'
error: invalid characters [ ] found in var 'jobs[first run]'
hacker detected...
Well, the diagnostic jumped to a conclusion that is probably false.
>> Since you only ever tested
>> the cases where $1 was a valid variable name
>
> It is only designed to work with $1 being an optional, valid variable
> name.
> Anything else should fail. There are times when putting quotes around a
> var will enable problems.
>> , you never ran into that particular result... until now.
>>
>
> I never ran into "invalid input" because I didn't program it to
> accept anything other than a variable name there.
You did program it to accept anything other than a variable name there.
You then made a "gentleman's agreement" to not *use* anything other than
a variable name there.
But "use" is a different word than "accept".
>> As you can see, the unquoted $1 underwent word splitting, so you're
>> effectively running printf -v 'x[0$(date' '>&2)]' '%s\n' "...".
>>
>
> Which is detected as "illegal input" and disallowed. If you don't
> enable
> some security errors, they can't be as easily introduced. I assert that
> you
> should never put quotes around something that is supposed to be a
> variable name
> since valid variable names can not be word-split. Many of the items on
> your
> website about bash cautions, are because bash disallows some sketchy
> constructs.
> That's not a bash-caveat, but a bash feature!
This is a ridiculous argument and you know it. You, personally, are
writing code which does not get used in security contexts, which is your
right. This in no way means that refusing to quote variables which
"cannot be word-split" stops *any* security errors. The "illegal input"
was not related to the security bypass (as Greg points out, removing the
space prevents word splitting and executes the same security bypass code).
Your response should have been:
"""
I assert that you should never put quotes around something that is
supposed to be a variable name since valid variable names can not be
word-split and quoting them implies you would like to handle them as
variable names anyway.
Quotes aren't a security feature. I don't care about security features,
because I don't misuse the code and try to hack my own system. But if
you do want security features, you should detect anything that isn't a
valid variable name and raise an error, then go ahead and use the
variables unquoted.
"""
And hey, I'd even agree with you on that. But I'm also an overly
restrictive person who grumps at people trying to set:
declare -A jobs=(["first run"]=$'0\n' )
Instead you are arguing in bad faith that "That's not a bash-caveat, but
a bash feature!" When strictly speaking your code is flawed, it doesn't
correctly handle indexed arrays with spaces in the key and doesn't
forbid them either.
>> This won't protect against all code injections, of course; only the
>> ones that contain a whitespace character.
>>
>
>
Re: simple prob?
On Tue, Jun 29, 2021 at 04:29:05PM -0700, L A Walsh wrote:
> > > njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
>Which is detected as "illegal input" and disallowed. If you don't enable
> some security errors, they can't be as easily introduced.
Are you *still* insisting that your failure to quote is a SECURITY
FEATURE?
Come *on*!
unicorn:~$ njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
unicorn:~$ njobs 'x[0$(date>&2)]'
Tue Jun 29 19:49:16 EDT 2021
All I had to do was remove the space. You're not even trying.
Your failure to quote is simply a failure. If you want to prevent
code injection attacks, you need to sanity-check the input.
There is no other way.
Re: simple prob?
On 2021/06/29 15:49, Greg Wooledge wrote:
On Tue, Jun 29, 2021 at 02:58:28PM -0700, L A Walsh wrote:
njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
Using that with your input:
njobs 'x[0$(date >&2)]'
bash: printf: `x[0$(date': not a valid identifier
This is because you didn't quote "$1".
$1 should never be quoted -- it is an identifier, and as such
cannot contain a space. By quoting it, you are allowing inputs that
would otherwise be filtered out because they are not valid variable
names.
Since you only ever tested
the cases where $1 was a valid variable name
It is only designed to work with $1 being an optional, valid
variable name.
Anything else should fail. There are times when putting quotes around a
var will enable problems.
, you never ran into that particular result... until now.
I never ran into "invalid input" because I didn't program it to
accept anything other than a variable name there.
As you can see, the unquoted $1 underwent word splitting, so you're
effectively running printf -v 'x[0$(date' '>&2)]' '%s\n' "...".
Which is detected as "illegal input" and disallowed. If you don't
enable
some security errors, they can't be as easily introduced. I assert that you
should never put quotes around something that is supposed to be a
variable name
since valid variable names can not be word-split. Many of the items on your
website about bash cautions, are because bash disallows some sketchy
constructs.
That's not a bash-caveat, but a bash feature!
This won't protect against all code injections, of course; only the
ones that contain a whitespace character.
Nothing protect against "all" 'X' or "everything", especially when
talking
about security. Security is best handled as a layered approach with
different
layers protecting against different things. There's no such thing as a
'Security Magic Bullet'. Just because NOTHING protects against "ALL"
[anything] is
not a reason to take extreme action. In fact realization that NOTHING is
perfect
is one of the better defenses against over-reacting in response to
supposed security
flaws.
Re: simple prob?
On Tue, Jun 29, 2021 at 02:58:28PM -0700, L A Walsh wrote:
> njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
>
> Using that with your input:
>
> njobs 'x[0$(date >&2)]'
>
> bash: printf: `x[0$(date': not a valid identifier
This is because you didn't quote "$1". Since you only ever tested
the cases where $1 was a valid variable name, you never ran into that
particular result... until now.
As you can see, the unquoted $1 underwent word splitting, so you're
effectively running printf -v 'x[0$(date' '>&2)]' '%s\n' "...".
This won't protect against all code injections, of course; only the
ones that contain a whitespace character.
Re: simple prob?
On 2021/06/29 14:02, Greg Wooledge wrote:
declare, printf -v, local -n, eval -- they're mostly equivalent. Some
of them may prevent *some* possible code injections, but none of them
prevent *all* possible code injections.
unicorn:~$ njobs2() { printf -v "$1" %s 42; }
unicorn:~$ njobs2 'x[0$(date >&2)]'
Tue Jun 29 17:00:29 EDT 2021
That's not what I see in my version:
njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
Using that with your input:
njobs 'x[0$(date >&2)]'
bash: printf: `x[0$(date': not a valid identifier
Perhaps some solutions provide more resistance to problems than
others.
FWIW, I would be using 'njobs' in a script where I'm giving it
the input.
No matter which one of these you choose, you still have to sanity-check
the input. Or else declare that you do not care if the user shoots their
own foot off
The user has no access to internal functions inside a script. Though I
do take many precautions against my future self(ves).
Re: simple prob?
On Tue, Jun 29, 2021 at 02:05:46PM -0700, L A Walsh wrote:
>That would be 'me', so I'm going to rule out malicious
> code injection! :-), but that's also why I used printf to
> write the output, the worst that could happen is some varname
> is overwritten with the answer, no?
Sadly, this is not correct. You can still get code injections with
printf -v.
unicorn:~$ njobs2() { printf -v "$1" %s 42; }
unicorn:~$ njobs2 'x[0$(date >&2)]'
Tue Jun 29 17:00:29 EDT 2021
> Simpler -- don't use a variable:
>
> njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
You're adding a newline to the variable. That may not be what you want.
> So...hmmm...how is it that jobs picks up the right answer in what
> would seem to be a subshell? Special cased?
The subshell inherits the job list from its parent -- it just can't
*wait* for any of those jobs, since it isn't their parent. But it can
still count them.
Re: simple prob?
On Tue, 29 Jun 2021 17:02:16 -0400
Greg Wooledge wrote:
> On Tue, Jun 29, 2021 at 09:47:30PM +0100, Kerin Millar wrote:
> > On Tue, 29 Jun 2021 16:35:28 -0400
> > Greg Wooledge wrote:
> >
> > > unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
> > > unicorn:~$ njobs walsh
> > > unicorn:~$ echo "$walsh"
> > > 3
> > >
> > > Now you just need to add sanity-checking on the argument of njobs, to
> > > avoid whatever code injection the malicious caller wants to perform.
> >
> > I can't fathom the switch to eval there. Why not printf -v "$1" %s "$_n",
> > for example? It even rejects invalid identifiers.
>
> declare, printf -v, local -n, eval -- they're mostly equivalent. Some
> of them may prevent *some* possible code injections, but none of them
> prevent *all* possible code injections.
>
> unicorn:~$ njobs2() { printf -v "$1" %s 42; }
> unicorn:~$ njobs2 'x[0$(date >&2)]'
> Tue Jun 29 17:00:29 EDT 2021
>
> No matter which one of these you choose, you still have to sanity-check
> the input. Or else declare that you do not care if the user shoots their
> own foot off (which is a valid stance as long as your code is never used
> in a context where the user can elevate their privileges/capabilites).
>
I see. Thanks.
--
Kerin Millar
Re: simple prob?
On 2021/06/29 13:35, Greg Wooledge wrote:
unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
---
ARG...I thought about that and rejected it because I
thought the "jobs|wc-l" would be in a sub-shell and not pickup
the background jobs! Double arg, this works as well:
sjobs() { local j;read j< <(jobs|wc -l);
printf ${1:+-v $1} "%s\n" "$j"; }
About the only thing that doesn't work are variations on
jobs|wc-l|read n -- since if read is in current process jobs doesn't
pick up the children, but if jobs is in current process, then read
isn't, and 'n' is lost.
Of course I was focusing on -s/-u lastpipe to provide some variation
but that was the only variation that would seem to be guaranteed not
to work. Oi!
Now you just need to add sanity-checking on the argument of njobs, to
avoid whatever code injection the malicious caller wants to perform.
That would be 'me', so I'm going to rule out malicious
code injection! :-), but that's also why I used printf to
write the output, the worst that could happen is some varname
is overwritten with the answer, no?
And maybe consider adding "make sure $1 isn't _n" to the list of your
sanity checks, since the caller can't be permitted to use that particular
variable namhe.
Simpler -- don't use a variable:
njobs() { printf ${1:+-v $1} "%s\n" "$(jobs |wc -l)"; }
---
The use of the builtin expansion off of the parameter has the side
benefit of printing the answer to output if no var is given.
So...hmmm...how is it that jobs picks up the right answer in what
would seem to be a subshell? Special cased?
unicorn:~$ njobs _n
unicorn:~$ echo "$_n"
(At least it didn't go into an infinite loop!)
Re: simple prob?
On 2021/06/29 13:35, Eli Schwartz wrote: Well, if you don't think this is a bug in bash, but something you need help figuring out, maybe you'd prefer to use the "help-bash" list? Actually - The original message was received at Tue, 29 Jun 2021 13:06:34 -0700 The following addresses had permanent fatal errors - (reason: 550-Callout verification failed:) --- I had it backwardsoh well. I did try there first!
Re: simple prob?
On Tue, Jun 29, 2021 at 09:47:30PM +0100, Kerin Millar wrote:
> On Tue, 29 Jun 2021 16:35:28 -0400
> Greg Wooledge wrote:
>
> > unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
> > unicorn:~$ njobs walsh
> > unicorn:~$ echo "$walsh"
> > 3
> >
> > Now you just need to add sanity-checking on the argument of njobs, to
> > avoid whatever code injection the malicious caller wants to perform.
>
> I can't fathom the switch to eval there. Why not printf -v "$1" %s "$_n", for
> example? It even rejects invalid identifiers.
declare, printf -v, local -n, eval -- they're mostly equivalent. Some
of them may prevent *some* possible code injections, but none of them
prevent *all* possible code injections.
unicorn:~$ njobs2() { printf -v "$1" %s 42; }
unicorn:~$ njobs2 'x[0$(date >&2)]'
Tue Jun 29 17:00:29 EDT 2021
No matter which one of these you choose, you still have to sanity-check
the input. Or else declare that you do not care if the user shoots their
own foot off (which is a valid stance as long as your code is never used
in a context where the user can elevate their privileges/capabilites).
Re: simple prob?
On Tue, 29 Jun 2021 16:35:28 -0400
Greg Wooledge wrote:
> On Tue, Jun 29, 2021 at 01:21:44PM -0700, L A Walsh wrote:
> > I hope a basic question isn't too offtopic.
>
> More of a help-bash question, in my opinion, but close enough.
>
> > Say I have some number of jobs running:
> >
> > > jobs|wc -l
> > 3
>
> OK.
>
> > Would like to pass a varname to njobs to store the answer in, like:
> > > njobs n
> > echo "$n"
> > 3
>
> "How do I pass a value back from a function to its caller, you know,
> like a function in any other programming language can do" is one of the
> holy grails of shell programming. There is no sane answer. You appear
> to be going down the "pass a variable name by reference" path, so:
>
> unicorn:~$ jobs | wc -l
> 3
> unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
> unicorn:~$ njobs walsh
> unicorn:~$ echo "$walsh"
> 3
>
> Now you just need to add sanity-checking on the argument of njobs, to
> avoid whatever code injection the malicious caller wants to perform.
I can't fathom the switch to eval there. Why not printf -v "$1" %s "$_n", for
example? It even rejects invalid identifiers.
--
Kerin Millar
Re: simple prob?
On 6/29/21 4:21 PM, L A Walsh wrote:
> I hope a basic question isn't too offtopic.
Well, if you don't think this is a bug in bash, but something you need
help figuring out, maybe you'd prefer to use the "help-bash" list?
> Say I have some number of jobs running:
>
>> jobs|wc -l
> 3
> ---
> in a function (have tried shopt -s/-u lastpipe; neither way worked)
> njobs() {
> jobs |wc -l
> }
>> njobs
> 3
>
> Would like to pass a varname to njobs to store the answer in, like:
> njobs() {
> jobs|wc -l
> #magic puts val in $v
> printf {$1:+-v $1} "%s\n" "$v"
This is obviously broken, because {$1:+expand} isn't how bash works. Try
swapping the "{" and the "$"...
> }
>
> So I can run:
>
>> njobs n
> echo "$n"
> 3
>
> ---
> How can I put the output into '$v'
> *without* using a temporary file?
>
> This seems so basic, yet its eluding me.
> Could someone throw me a clue-stick?
> Tnx!
What's wrong with the obvious
v=$(jobs | wc -l)
> p.s. - a trivial util func producing jobs:
>
> resleep() { alias my="declare " int="my -i "
> int n=${1:-3} t=${2:-99}; echo "$n jobs @ ${t}s:"
> while ((0 < n--)); do sleep "$t" & done; }
>
>
>
>
>
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
OpenPGP_signature
Description: OpenPGP digital signature
Re: simple prob?
On Tue, Jun 29, 2021 at 01:21:44PM -0700, L A Walsh wrote:
> I hope a basic question isn't too offtopic.
More of a help-bash question, in my opinion, but close enough.
> Say I have some number of jobs running:
>
> > jobs|wc -l
> 3
OK.
> Would like to pass a varname to njobs to store the answer in, like:
> > njobs n
> echo "$n"
> 3
"How do I pass a value back from a function to its caller, you know,
like a function in any other programming language can do" is one of the
holy grails of shell programming. There is no sane answer. You appear
to be going down the "pass a variable name by reference" path, so:
unicorn:~$ jobs | wc -l
3
unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
unicorn:~$ njobs walsh
unicorn:~$ echo "$walsh"
3
Now you just need to add sanity-checking on the argument of njobs, to
avoid whatever code injection the malicious caller wants to perform.
And maybe consider adding "make sure $1 isn't _n" to the list of your
sanity checks, since the caller can't be permitted to use that particular
variable name.
unicorn:~$ njobs _n
unicorn:~$ echo "$_n"
(At least it didn't go into an infinite loop!)
simple prob?
I hope a basic question isn't too offtopic.
Say I have some number of jobs running:
jobs|wc -l
3
---
in a function (have tried shopt -s/-u lastpipe; neither way worked)
njobs() {
jobs |wc -l
}
njobs
3
Would like to pass a varname to njobs to store the answer in, like:
njobs() {
jobs|wc -l
#magic puts val in $v
printf {$1:+-v $1} "%s\n" "$v"
}
So I can run:
njobs n
echo "$n"
3
---
How can I put the output into '$v'
*without* using a temporary file?
This seems so basic, yet its eluding me.
Could someone throw me a clue-stick?
Tnx!
p.s. - a trivial util func producing jobs:
resleep() { alias my="declare " int="my -i "
int n=${1:-3} t=${2:-99}; echo "$n jobs @ ${t}s:"
while ((0 < n--)); do sleep "$t" & done; }
