Ash eval wierdness
I'm trying to import a set of variables from a file containing assignments of the form "a=b", one on each line. I am finding that the variable assignments are being lost. Here is a simplified version of my script which illustrates the problem: echo "myvar=my value" | while read opt; do if echo "$opt" | grep -q '='; then var=`echo "$opt" | /bin/sed "s/=.*//"` val=`echo "$opt" | /bin/sed "s/[^=]*=//"` eval $var=\'$val\' echo "(1)" $var=\'$val\' echo "(2) myvar=$myvar" fi done echo "(3) myvar=$myvar" Which yields this: (1) myvar='my value' (2) myvar=my value (3) myvar= I am using Busybox v1.18.4 Any idea what is going on? ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
On 05/25/2011 10:24 AM, Nigel Hathaway wrote: > I'm trying to import a set of variables from a file containing assignments of > the form "a=b", one on each line. I am finding that the variable assignments > are being lost. > > Here is a simplified version of my script which illustrates the problem: > > echo "myvar=my value" | while read opt; do > if echo "$opt" | grep -q '='; then > var=`echo "$opt" | /bin/sed "s/=.*//"` > val=`echo "$opt" | /bin/sed "s/[^=]*=//"` > eval $var=\'$val\' > echo "(1)" $var=\'$val\' > echo "(2) myvar=$myvar" > fi > done > echo "(3) myvar=$myvar" > > Which yields this: > > (1) myvar='my value' > (2) myvar=my value > (3) myvar= > > I am using Busybox v1.18.4 > > Any idea what is going on? Hi Nigel, You will get the same results if you run that logic through bash. The problem is that your eval logic only has effect until the loop (everything after the pipe '|') ends. Run it in bash and you will see the same thing. Chuck ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
Nigel, All, On Wednesday 25 May 2011 162459 Nigel Hathaway wrote: > I'm trying to import a set of variables from a file containing assignments > of the form "a=b", one on each line. I am finding that the variable > assignments are being lost. > > Here is a simplified version of my script which illustrates the problem: > > echo "myvar=my value" | while read opt; do > if echo "$opt" | grep -q '='; then > var=`echo "$opt" | /bin/sed "s/=.*//"` > val=`echo "$opt" | /bin/sed "s/[^=]*=//"` > eval $var=\'$val\' > echo "(1)" $var=\'$val\' > echo "(2) myvar=$myvar" > fi > done > echo "(3) myvar=$myvar" > > Which yields this: > > (1) myvar='my value' > (2) myvar=my value > (3) myvar= > > I am using Busybox v1.18.4 > > Any idea what is going on? That's expected. All that is to the right part of the pipe is running in a sub-shell. So the evaluation is done in a sub-shell. So it is lost as soon as this sub-shell ends. Regards, Yann E. MORIN. PS. Nigel, sorry for the duplicate, previous mail sent form the wrong address, so got refused by the ML. YEM. -- .-..--.. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^| | --==< O_o >==-- '.---: X AGAINST | /e\ There is no | | http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL| """ conspiracy. | '--'---'--'' ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
RE: Ash eval wierdness
Do you know what the solution is? I've tried the 'set' command, but this doesn't seem to behave according standard ash set (according to the 'set' manpage) On 05/25/2011 10:24 AM, Nigel Hathaway wrote: > I'm trying to import a set of variables from a file containing assignments of > the form "a=b", one on each line. I am finding that the variable assignments > are being lost. > > Here is a simplified version of my script which illustrates the problem: > > echo "myvar=my value" | while read opt; do > if echo "$opt" | grep -q '='; then > var=`echo "$opt" | /bin/sed "s/=.*//"` > val=`echo "$opt" | /bin/sed "s/[^=]*=//"` > eval $var=\'$val\' > echo "(1)" $var=\'$val\' > echo "(2) myvar=$myvar" > fi > done > echo "(3) myvar=$myvar" > > Which yields this: > > (1) myvar='my value' > (2) myvar=my value > (3) myvar= > > I am using Busybox v1.18.4 > > Any idea what is going on? Hi Nigel, You will get the same results if you run that logic through bash. The problem is that your eval logic only has effect until the loop (everything after the pipe '|') ends. Run it in bash and you will see the same thing. Chuck ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
Nigel, All,
On Wednesday 25 May 2011 165635 Nigel Hathaway wrote:
Please, don't top-post. Write below the text you are replying to, it's
easier to follow and understand. Also, please wrap long lines to below
80 chars length.
> Do you know what the solution is?
Why do you need to add 'eval' in front of lines?
Does your input file contain only affectations, or other stuff?
Are affectations in this file properly escaped?
It all depends. If you have only affectations, and they are properly escaped
and/or quoted, then just source the file (notice the leading dot):
. "${my_file}"
If not, then pre-process the file, stick 'eval' where it is due, escape
and/or quote where needed, and source the resulting file.
Regards,
Yann E. MORIN.
--
.-..--..
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^|
| --==< O_o >==-- '.---: X AGAINST | /e\ There is no |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL| """ conspiracy. |
'--'---'--''
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
Hallo Chuck! >> I'm trying to import a set of variables from a file containing assignments >> of the form "a=b", one on each line. I am finding that the variable >> assignments are being lost. >> >> Here is a simplified version of my script which illustrates the problem: >> >> echo "myvar=my value" | while read opt; do >> if echo "$opt" | grep -q '='; then >> var=`echo "$opt" | /bin/sed "s/=.*//"` >> val=`echo "$opt" | /bin/sed "s/[^=]*=//"` >> eval $var=\'$val\' >> echo "(1)" $var=\'$val\' >> echo "(2) myvar=$myvar" >> fi >> done >> echo "(3) myvar=$myvar" >> >> Which yields this: >> >> (1) myvar='my value' >> (2) myvar=my value >> (3) myvar= >> >> I am using Busybox v1.18.4 >> >> Any idea what is going on? > You will get the same results if you run that logic through bash. > The problem is that your eval logic only has effect until the loop > (everything after the pipe '|') ends. In principle you are right, but it depends on the version of bash. There exist two strategies which lead to different results in those cases. Some shells run the last process in a pipe in the current shell others the first process. If the shell runs the first process of a pipe, all variable changes done by remaining processes of the pipe are lost. If the shell runs the last process of a pipe, variable changes in the last process are kept and others are lost. In my eyes that last case is the better choice ... but ash uses the other approach :-( Hints: If the variables are in usual shell syntax, why don't you source the file direct to grab the variable values? That way you can use all types of shell syntax for variable value definition, including substitutions, arithmetical expressions etc. [ -f YOUR_VAR_FILE ] && source YOUR_VAR_FILE ... or you can do ... while read opt; do # process your opt value here done http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
On Wed, 2011-05-25 at 14:24 +, Nigel Hathaway wrote: > I'm trying to import a set of variables from a file containing assignments of > the form "a=b", one on each line. I am finding that the variable assignments > are being lost. > > Here is a simplified version of my script which illustrates the problem: > > echo "myvar=my value" | while read opt; do > if echo "$opt" | grep -q '='; then > var=`echo "$opt" | /bin/sed "s/=.*//"` > val=`echo "$opt" | /bin/sed "s/[^=]*=//"` > eval $var=\'$val\' > echo "(1)" $var=\'$val\' > echo "(2) myvar=$myvar" > fi > done > echo "(3) myvar=$myvar" > > Which yields this: > > (1) myvar='my value' > (2) myvar=my value > (3) myvar= > > I am using Busybox v1.18.4 > > Any idea what is going on? > ___ > busybox mailing list > busybox@busybox.net > http://lists.busybox.net/mailman/listinfo/busybox would this solve your problem? grep '^[^[:blank:]]=.*$' file_with_vars_and_other_stuff >/tmp/varfile . /tmp/varfile -C ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
RE: Ash eval wierdness
> > Hallo Chuck! > >>> I'm trying to import a set of variables from a file containing >assignments >>> of the form "a=b", one on each line. I am finding that the >variable >>> assignments are being lost. >>> >>> Here is a simplified version of my script which illustrates the problem: >>> >>> echo "myvar=my value" | while read opt; do >>> if echo "$opt" | grep -q '='; then >>> var=`echo "$opt" | /bin/sed "s/=.*//"` >>> val=`echo "$opt" | /bin/sed "s/[^=]*=//"` >>> eval $var=\'$val\' >>> echo "(1)" $var=\'$val\' >>> echo "(2) myvar=$myvar" >>> fi >>> done >>> echo "(3) myvar=$myvar" >>> >>> Which yields this: >>> >>> (1) myvar='my value' >>> (2) myvar=my value >>> (3) myvar= >>> >>> I am using Busybox v1.18.4 >>> >>> Any idea what is going on? >> You will get the same results if you run that logic through bash. >> The problem is that your eval logic only has effect until the loop >> (everything after the pipe '|') ends. > >In principle you are right, but it depends on the version of bash. There >exist two strategies which lead to different results in those cases. >Some shells run the last process in a pipe in the current shell others >the first process. If the shell runs the first process of a pipe, all >variable changes done by remaining processes of the pipe are lost. If >the shell runs the last process of a pipe, variable changes in the last >process are kept and others are lost. > >In my eyes that last case is the better choice ... but ash uses the >other approach :-( > >Hints: > >If the variables are in usual shell syntax, why don't you source the >file direct to grab the variable values? That way you can use all types >of shell syntax for variable value definition, including substitutions, >arithmetical expressions etc. > >[ -f YOUR_VAR_FILE ] && source YOUR_VAR_FILE > >... or you can do ... > >while read opt; do > # process your opt value here >done >... or another solution I'm using instead ... > >eval $( sed -e SCRIPT_TO_MODIFY_VAR_FILE YOUR_VAR_FILE ) > Thanks for this very simple piece of script gymnastics. This last one is just the solution I was looking for. >-- >Harald ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
Most of the responses didn't spot the spaces in your parameters which I expect is causing most of your problems. How's this for a solution if you have the parameters in "file". eval $(sed -n -e 's/=\(.*\)/="\1"/p' < file) The sed does the equivalent of your grep and individual echo/seds echoing the parameters with quotes to be evaluated in the parent shell. Of course " in the file will cause problems and a whole host of other things could lead to nasty security holes, but then you know that. -- Bob Dunlop ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
On Wed, May 25, 2011 at 4:24 PM, Nigel Hathaway
wrote:
> I'm trying to import a set of variables from a file containing assignments of
> the form "a=b", one on each line. I am finding that the variable assignments
> are being lost.
>
> Here is a simplified version of my script which illustrates the problem:
>
> echo "myvar=my value" | while read opt; do
> if echo "$opt" | grep -q '='; then
> var=`echo "$opt" | /bin/sed "s/=.*//"`
> val=`echo "$opt" | /bin/sed "s/[^=]*=//"`
This is inefficient (to run grep and two seds), and unsafe.
Efficiency:
if opt="a=b=c", then
var="${opt%%=*}" #gives you "a" and
val="${opt#*=}" #gives you "b=c"
Correctness:
if test x"$var" = x"$opt"; then < there is no '='>
squote="'"
if test x"${val#*$squote}" != x"$val"; then < val contains single
quote, using '$val' won't be safe>
> echo "myvar=my value" | while read opt; do
...
> eval $var=\'$val\'
> echo "(1)" $var=\'$val\'
> echo "(2) myvar=$myvar"
> fi
> done
"while ... done http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
On Wed, 2011-05-25 at 17:20 +0200, Harald Becker wrote: > In principle you are right, but it depends on the version of bash. > There exist two strategies which lead to different results in those > cases. Some shells run the last process in a pipe in the current shell > others the first process. If the shell runs the first process of a > pipe, all variable changes done by remaining processes of the pipe are > lost. If the shell runs the last process of a pipe, variable changes > in the last process are kept and others are lost. Is it really the case that there are shells that run the first command in a pipeline in the current shell? I've never heard of this. There are some shells that run the last command in a pipeline in the current shell: ksh does this for example. The last time I checked bash does NOT do this, although maybe this was added in the latest versions? I don't remember if zsh does this or not. All other shells I'm aware of fork _all_ the commands in the pipeline. Either method is acceptable from the standpoint of POSIX, so no shell script that uses #!/bin/sh should ever depend on the "ksh behavior". I agree it's a shame because it is so often useful to preserve variables in the final command. But, that's the way it is and it seems unlikely to change. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
On Wed, May 25, 2011 at 04:33:56PM +0200, Yann E. MORIN wrote: > > echo "myvar=my value" | while read opt; do [...] > > That's expected. All that is to the right part of the pipe is running in a > sub-shell. So the evaluation is done in a sub-shell. So it is lost as soon > as this sub-shell ends. The canonical solution is: while read opt; do ... ; done << EOF myvar=my value EOF Rich ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Ash eval wierdness
Hallo Bob! > eval $(sed -n -e 's/=\(.*\)/="\1"/p' < file) Why do you redirect the input file here? eval $( sed -n -e 's/=\(.*\)/="\1"/p' file ) ... shall do the job. In addition it is possible to strip of spaces around = (if required): eval $( sed -n -e 's/ *= *\(.*\)/="\1"/p' file ) With some more tricky sed scripts you can even suppress comments in the file, select only specific variables for usage and more, e.g. eval $( sed -n \ -e '/^$/d' \ -e '/^#/d' \ -e 's/^\([a-z_]+\) *= *\(.*\)/\1="\2"/' \ -e '/^abc=/p' \ -e '/^xyz=/p' \ file ) ... just to give some examples. -- Harald ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
