Re: Silly shell question

[Benny Siegert]

On Tue, 22 Mar 2016 15:07:09 -0600 (MDT)
Swift Griggs  wrote:

> Thanks for the info, but do you happen to know what the actual mechanism 
> that the child processes is able to "import" the exported variable ? Ie.. 
> is it some special OS glue/magic, or is it just straight getenv() calls by 
> the client shell/app ? I don't see anything magical in the man page for 
> getenv() that would distinguish an exported versus non-exported variable.

It's really simple: Exported variables are in the environment (i.e. 
getenv/setenv). Non-exported variables are just a table inside of the shell 
process itself. They have no special meaning to the OS.

-- 
Benny Siegert 

Re: Silly shell question

[Dan LaBell]

On Mar 22, 2016, at 4:39 PM, Swift Griggs wrote:



In ksh, when you use the 'export' keyword, what is actually going  
on? Does it create a copy of the variable in memory? I doubt it  
since I tried a test and I could see the exported version changing  
even if I just change the original variable:


# FOO=abc
# export FOO
# ksh
[...]

Try printenv before and after.  Man printenv and environ, and getenv.
Basically, exported variables become part an array of character  
strings in

the memory space of child process (as a copy). In a way, it's similar to
to command line arguments. It's actually part of C , not just unix.
In C you can optionally pass a 3rd parameter to main
--
#include 
#include 

int main( int argc, char * argv[] , char **environ)
{
  printf("main %p\n", main);
  printf("argv  %p\n", argv);
  printf("environ %p\n", environ);
  printf("malloc'ed byte %p\n",malloc(1));
}
Compile with symbols, and launch gdb, set a break point and
you can poke around. (print environ[4], print strlen(environ[4]), etc ).



Sorry if this seems like a dumb question. I'm just curious about  
the intrinsics of how
it works. I looked at the source code for ast-ksh, but it's pretty  
huge and hard to
Might want to look at the source for something simpler, like  
printenv, and env.

/usr/src/usr.bin/env/env.c
/usr/src/usr.bin/printenv/printenv.c

Re: Silly shell question

[Eric Haszlakiewicz]

On 3/22/2016 5:16 PM, Johnny Billquist wrote:

On 2016-03-22 22:07, Swift Griggs wrote:

On Tue, 22 Mar 2016, Johnny Billquist wrote:

Only environment variables are propagated to child processes.


Thanks for the info, but do you happen to know what the actual mechanism
that the child processes is able to "import" the exported variable ?
Ie.. is it some special OS glue/magic, or is it just straight getenv()
calls by the client shell/app ? I don't see anything magical in the man
page for getenv() that would distinguish an exported versus non-exported
variable.


A child process gets the environment as a part of the image 
activation. It's actually provided as an argument to each program, 
except most programs do not care to pick it up.


In case it wasn't clear, this refers to a function argument to the main 
function, i.e.:

int main(int argc, char **argv, char **environ)
not command line arguments.

The special OS glue/magic that happens is that the kernel takes the list 
of environment variables passed to the execve() system call, and copies 
them to the stack of the newly executing binary (kern_exec.c if you want 
to go look).  Then, the code that runs before main() (which comes from 
crt0.o and automatically gets included in your program when you link) 
figures out what's on the stack (which also includes the command line 
arguments, i.e. argv) and passes the appropriate things to main().


If your program (/bin/sh in this case) calls one of the exec* functions 
that don't take an explicit environ pointer, the implementation in libc 
(e.g. execl.c) actually calls execve() and passes along the current 
environment for you (which is available as the global variable "environ").


Shell local variables aren't stored in environ at all and aren't passed 
along to the execve() call when the shell forks+execs some new process, 
they're just arbitrary data that the shell process keeps in its memory 
and uses when it sees a variable reference on a command line.
Actually, there's no reason that the shell needs to store even exported 
variables in its own environ global variable either, it just needs to 
make sure that it generates an appropriate array of variables when it 
starts a new process and calls execve().  (that might be what /bin/sh 
actually does, I haven't actually looked)


Eric

Re: Silly shell question

[Swift Griggs]

On Tue, 22 Mar 2016, Jeremy C. Reed wrote:
Yes just use getenv. See the manpage. I wouldn't call it a "client" but 
either a child or replacement.


Got it. That makese sense. Johnny described it as "inheriting the 
environment" and that concept makes a lot of sense.


The concept doesn't exist at that level. Have a look at the execve 
manpage.  Also have a look at the src/bin/sh source too: execcmd in 
eval.c environment in var.c tryexec in exec.c


Juicy. This is perfect. I've got all those open right now, I'm learning a 
ton. I see what you guys are saying about "environment" at a low level. 
Thanks a lot!


-Swift

Re: Silly shell question

[Jeremy C. Reed]

On Tue, 22 Mar 2016, Swift Griggs wrote:

> On Tue, 22 Mar 2016, Johnny Billquist wrote:
> > Only environment variables are propagated to child processes.
> 
> Thanks for the info, but do you happen to know what the actual 
> mechanism that the child processes is able to "import" the exported 
> variable ? Ie.. is it some special OS glue/magic, or is it just 
> straight getenv() calls by the client shell/app ?

Yes just use getenv. See the manpage. I wouldn't call it a "client" but 
either a child or replacement.

> I don't see anything magical in the man page for getenv() that would 
> distinguish an exported versus non-exported variable.

The concept doesn't exist at that level. Have a look at the execve 
manpage.  Also have a look at the src/bin/sh source too:

execcmd in eval.c
environment in var.c
tryexec in exec.c

Re: Silly shell question

[Johnny Billquist]

On 2016-03-22 22:07, Swift Griggs wrote:

On Tue, 22 Mar 2016, Johnny Billquist wrote:

Only environment variables are propagated to child processes.


Thanks for the info, but do you happen to know what the actual mechanism
that the child processes is able to "import" the exported variable ?
Ie.. is it some special OS glue/magic, or is it just straight getenv()
calls by the client shell/app ? I don't see anything magical in the man
page for getenv() that would distinguish an exported versus non-exported
variable.


A child process gets the environment as a part of the image activation. 
It's actually provided as an argument to each program, except most 
programs do not care to pick it up.


getenv() also just access these environment variables that the system 
provided at image activation.


Shell local variables are not accessible to anything outside the shell.

Johnny

--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Silly shell question

[Swift Griggs]

On Tue, 22 Mar 2016, Johnny Billquist wrote:

Only environment variables are propagated to child processes.


Thanks for the info, but do you happen to know what the actual mechanism 
that the child processes is able to "import" the exported variable ? Ie.. 
is it some special OS glue/magic, or is it just straight getenv() calls by 
the client shell/app ? I don't see anything magical in the man page for 
getenv() that would distinguish an exported versus non-exported variable.


-Swift

Re: Silly shell question

[Jeremy C. Reed]

Some ideas to add to your research:

echo $FOO
FOO=def
ksh
echo $FOO
exit
> # FOO=abc
> # export FOO
> # ksh
> # echo $FOO
> abc
> # exit
> # FOO=123
> # ksh
> # echo $FOO
> 123
> # exit
ksh
echo $FOO
unset FOO
echo $FOO
exit
echo $FOO
unset FOO
echo $FOO
ksh
echo $FOO
exit

Re: Silly shell question

[Johnny Billquist]

Oh, and in case it wasn't clear. Any environment variable you have is 
always propagated to all your child processes. (Unless you explicitly 
set a different environment in your exec*() syscall)


Johnny

On 2016-03-22 22:00, Johnny Billquist wrote:

It's simpler than that. KSH (along with SH, BASH and similar shells)
have two sets of variables. Local and environment. A variable can only
exist in one of these. By default, when a variable is created, it is
local. If you export it, it becomes an environment variable.

Only environment variables are propagated to child processes.

(In shells from the csh family, you have different commands for dealing
with local and environment variables, so there it is a bit more obvious
what you are doing.)

 Johnny

On 2016-03-22 21:39, Swift Griggs wrote:


In ksh, when you use the 'export' keyword, what is actually going on?
Does it create a copy of the variable in memory? I doubt it since I
tried a test and I could see the exported version changing even if I
just change the original variable:

# FOO=abc
# export FOO
# ksh
# echo $FOO
abc
# exit
# FOO=123
# ksh
# echo $FOO
123
# exit

So, what is that really happens with exported variables? Does the
shell/app that's starting up check for exported variables and somehow
import them? How can it tell? I noticed there are a LOT of checks around
exported variables in ast-ksh. It appears to be a dangerous proposition
in some cases.

Sorry if this seems like a dumb question. I'm just curious about the
intrinsics of how it works. I looked at the source code for ast-ksh, but
it's pretty huge and hard to follow to answer this question. I'm just
wondering what the key mechanism is. Curiosity only.

-Swift






--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Silly shell question

[Johnny Billquist]

It's simpler than that. KSH (along with SH, BASH and similar shells) 
have two sets of variables. Local and environment. A variable can only 
exist in one of these. By default, when a variable is created, it is 
local. If you export it, it becomes an environment variable.


Only environment variables are propagated to child processes.

(In shells from the csh family, you have different commands for dealing 
with local and environment variables, so there it is a bit more obvious 
what you are doing.)


Johnny

On 2016-03-22 21:39, Swift Griggs wrote:


In ksh, when you use the 'export' keyword, what is actually going on?
Does it create a copy of the variable in memory? I doubt it since I
tried a test and I could see the exported version changing even if I
just change the original variable:

# FOO=abc
# export FOO
# ksh
# echo $FOO
abc
# exit
# FOO=123
# ksh
# echo $FOO
123
# exit

So, what is that really happens with exported variables? Does the
shell/app that's starting up check for exported variables and somehow
import them? How can it tell? I noticed there are a LOT of checks around
exported variables in ast-ksh. It appears to be a dangerous proposition
in some cases.

Sorry if this seems like a dumb question. I'm just curious about the
intrinsics of how it works. I looked at the source code for ast-ksh, but
it's pretty huge and hard to follow to answer this question. I'm just
wondering what the key mechanism is. Curiosity only.

-Swift



--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Silly shell question

[Swift Griggs]

In ksh, when you use the 'export' keyword, what is actually going on? Does 
it create a copy of the variable in memory? I doubt it since I tried a 
test and I could see the exported version changing even if I just change 
the original variable:


# FOO=abc
# export FOO
# ksh
# echo $FOO
abc
# exit
# FOO=123
# ksh
# echo $FOO
123
# exit

So, what is that really happens with exported variables? Does the 
shell/app that's starting up check for exported variables and somehow 
import them? How can it tell? I noticed there are a LOT of checks around 
exported variables in ast-ksh. It appears to be a dangerous proposition in 
some cases.


Sorry if this seems like a dumb question. I'm just curious about the 
intrinsics of how it works. I looked at the source code for ast-ksh, but 
it's pretty huge and hard to follow to answer this question. I'm just 
wondering what the key mechanism is. Curiosity only.


-Swift