Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ken Cunningham]

> On Oct 24, 2020, at 19:51, Ken Cunningham wrote:
> 
> > On Oct 24, 2020, at 3:36 PM, Ryan Schmidt wrote:
> > 
> >> On Oct 24, 2020, at 11:46, Ken Cunningham wrote:
> >> 
> >>> So instead of this method, I will look into running lsearch on *.env, and 
> >>> see if I can sort out how to add some entries if something is already 
> >>> there. 
> >> 
> >> MacPorts base makes this easier for specific variables by offering 
> >> MacPorts variables to set them. For example configure.cflags to set 
> >> CFLAGS, configure.pkg_config_path to set PKG_CONFIG_PATH, etc. We could 
> >> consider offering e.g. configure.dyld_library_path to set 
> >> DYLD_LIBRARY_PATH.
> > 
> > That would indeed make things easier — but with some thought I can 
> > understand why this particular env var is tricky to handle correctly for 
> > the general case...
> > 
> > 1. User might have set it to something that should be honoured, and expect 
> > it to be reinstated after the build is done.
> 
> I didn't remember that DYLD_LIBRARY_PATH was among the small set of 
> environment variables that MacPorts does pass on from the user's shell to the 
> build. MacPorts could certainly set the default value of the hypothetical 
> configure.dyld_library_path to $env(DYLD_LIBRARY_PATH), and ports could then 
> append to it as needed. Of course anything MacPorts does to configure.env, 
> build.env, destroot.env, etc. would only persist for the duration of that 
> phase. MacPorts doesn't need to do anything to "reinstate" the user's value 
> of the env var since MacPorts has never changed it to begin with.
> 
> > 2. Different parts of the port tree (PortGroups, Portfiles) might try to 
> > set it to something. The order could be important.
> 
> Then they'd better add the paths in the right order? Not sure what you're 
> getting at.
> 
> > 3. The build tree won’t exist when the configure.XYZ flags are parsed, and 
> > this particular env var often uses the build tree.
> 
> Doesn't matter provided the path can be computed ahead of time. For example 
> ${workpath} can be computed anytime, even if the path they evaluate to 
> doesn't yet exist.
> 
> And if it can't be computed ahead of time, you do what we do in tons of other 
> ports already and compute it in pre-configure, pre-build, pre-destroot, etc.
> 
> > 4. different parts of the build (build, test, destroot) might need it set 
> > to different things.
> 
> Then they should do that?
> 
> > 5. Makefiles and Cmake files etc might later try to set it themselves!
> 
> Indeed they might, probably for a good reason. If your portfile also has a 
> need to add a path to this variable, then you'll maybe have to patch the 
> build system. MacPorts can't do everything for you automatically. Hopefully 
> the number of ports that need to set this env var is extremely small; it has 
> been so far.
> 
> > 6. All of that hidden away in the environment where it’s that much harder 
> > to see what is going on (or what is broken).
> 
> What are you saying here? That you are dissatisfied that Apple chose to 
> implement DYLD_LIBRARY_PATH as an environment variable instead of as 
> something else? Not much we can do about that. Or that MacPorts does not 
> adequately inform you of what values environment variables are set to? I 
> though it already prints them very clearly. How would you like to see it 
> improved?
> 

Thanks for taking the time to respond. I know you're very busy. I will read 
this over in due course and see if there's anything actionable in here.

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ryan Schmidt]

On Oct 24, 2020, at 21:51, Joshua Root wrote:

> On 2020-10-25 03:46 , Ken Cunningham wrote:
>> Do you think this:
>> 
>> configure.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
>> configure.env-append“DYLD_LIBRARY_PATH=${workpath}/build”
>> 
>> should “just work" to set both without added shenanigans?
> 
> This is a very special case where "set both" makes any sense.

It could make sense. On the other hand, how is MacPorts to know that in this 
particular case you want both values, and that they should be separated by a 
colon? What if you instead really wanted to replace the first value with the 
second value?

> Not all
> environment variables are treated as lists and not all use the same
> separator. This aspect of the behaviour matches that of env(1) currently
> BTW.

It seems clearer to keep all environment variables behaving the same way in 
MacPorts and not to special case certain variable names.

When we need special cases, such as using a colon separator when turning 
configure.pkg_config_path into PKG_CONFIG_PATH, we can implement that for that 
specific option.

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ryan Schmidt]

On Oct 24, 2020, at 19:51, Ken Cunningham wrote:

> On Oct 24, 2020, at 3:36 PM, Ryan Schmidt wrote:
> 
>> On Oct 24, 2020, at 11:46, Ken Cunningham wrote:
>> 
>>> So instead of this method, I will look into running lsearch on *.env, and 
>>> see if I can sort out how to add some entries if something is already 
>>> there. 
>> 
>> MacPorts base makes this easier for specific variables by offering MacPorts 
>> variables to set them. For example configure.cflags to set CFLAGS, 
>> configure.pkg_config_path to set PKG_CONFIG_PATH, etc. We could consider 
>> offering e.g. configure.dyld_library_path to set DYLD_LIBRARY_PATH.
> 
> That would indeed make things easier — but with some thought I can understand 
> why this particular env var is tricky to handle correctly for the general 
> case...
> 
> 1. User might have set it to something that should be honoured, and expect it 
> to be reinstated after the build is done.

I didn't remember that DYLD_LIBRARY_PATH was among the small set of environment 
variables that MacPorts does pass on from the user's shell to the build. 
MacPorts could certainly set the default value of the hypothetical 
configure.dyld_library_path to $env(DYLD_LIBRARY_PATH), and ports could then 
append to it as needed. Of course anything MacPorts does to configure.env, 
build.env, destroot.env, etc. would only persist for the duration of that 
phase. MacPorts doesn't need to do anything to "reinstate" the user's value of 
the env var since MacPorts has never changed it to begin with.

> 2. Different parts of the port tree (PortGroups, Portfiles) might try to set 
> it to something. The order could be important.

Then they'd better add the paths in the right order? Not sure what you're 
getting at.

> 3. The build tree won’t exist when the configure.XYZ flags are parsed, and 
> this particular env var often uses the build tree.

Doesn't matter provided the path can be computed ahead of time. For example 
${workpath} can be computed anytime, even if the path they evaluate to doesn't 
yet exist.

And if it can't be computed ahead of time, you do what we do in tons of other 
ports already and compute it in pre-configure, pre-build, pre-destroot, etc.

> 4. different parts of the build (build, test, destroot) might need it set to 
> different things.

Then they should do that?

> 5. Makefiles and Cmake files etc might later try to set it themselves!

Indeed they might, probably for a good reason. If your portfile also has a need 
to add a path to this variable, then you'll maybe have to patch the build 
system. MacPorts can't do everything for you automatically. Hopefully the 
number of ports that need to set this env var is extremely small; it has been 
so far.

> 6. All of that hidden away in the environment where it’s that much harder to 
> see what is going on (or what is broken).

What are you saying here? That you are dissatisfied that Apple chose to 
implement DYLD_LIBRARY_PATH as an environment variable instead of as something 
else? Not much we can do about that. Or that MacPorts does not adequately 
inform you of what values environment variables are set to? I though it already 
prints them very clearly. How would you like to see it improved?

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Joshua Root]

On 2020-10-25 03:46 , Ken Cunningham wrote:
> Do you think this:
> 
> configure.env-append        "DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
> configure.env-append        “DYLD_LIBRARY_PATH=${workpath}/build”
> 
> should “just work" to set both without added shenanigans?

This is a very special case where "set both" makes any sense. Not all
environment variables are treated as lists and not all use the same
separator. This aspect of the behaviour matches that of env(1) currently
BTW.

 Josh

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ken Cunningham]

> On Oct 24, 2020, at 3:36 PM, Ryan Schmidt  wrote:
> 
> 
> 
> On Oct 24, 2020, at 11:46, Ken Cunningham wrote:
> 
>> So instead of this method, I will look into running lsearch on *.env, and 
>> see if I can sort out how to add some entries if something is already there. 
> 
> MacPorts base makes this easier for specific variables by offering MacPorts 
> variables to set them. For example configure.cflags to set CFLAGS, 
> configure.pkg_config_path to set PKG_CONFIG_PATH, etc. We could consider 
> offering e.g. configure.dyld_library_path to set DYLD_LIBRARY_PATH.

That would indeed make things easier — but with some thought I can understand 
why this particular env var is tricky to handle correctly for the general 
case...

1. User might have set it to something that should be honoured, and expect it 
to be reinstated after the build is done.
2. Different parts of the port tree (PortGroups, Portfiles) might try to set it 
to something. The order could be important.
3. The build tree won’t exist when the configure.XYZ flags are parsed, and this 
particular env var often uses the build tree.
4. different parts of the build (build, test, destroot) might need it set to 
different things.
5. Makefiles and Cmake files etc might later try to set it themselves!
6. All of that hidden away in the environment where it’s that much harder to 
see what is going on (or what is broken).

This one is probably always going to be messy…

Ken

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ryan Schmidt]

On Oct 24, 2020, at 11:46, Ken Cunningham wrote:

> So instead of this method, I will look into running lsearch on *.env, and see 
> if I can sort out how to add some entries if something is already there. 

MacPorts base makes this easier for specific variables by offering MacPorts 
variables to set them. For example configure.cflags to set CFLAGS, 
configure.pkg_config_path to set PKG_CONFIG_PATH, etc. We could consider 
offering e.g. configure.dyld_library_path to set DYLD_LIBRARY_PATH.

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ken Cunningham]

> On Oct 24, 2020, at 9:50 AM, Ken Cunningham  
> wrote:
> 
> 
> 
>> On Oct 24, 2020, at 9:46 AM, Ken Cunningham > > wrote:
>> 
>> configure.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
>> configure.env-append“DYLD_LIBRARY_PATH=${workpath}/build”
>> 
>> but I found the second one just obliterates the first one.
>> 
>> DYLD_LIBRARY_PATH='/opt/local/lib/lilbgcc'
>> 
>> 
> 
> well the second one didn’t obliterate the first — other way around. 
> 
> but they both didn’t make into the env, which was what I was going for with 
> that comment.



I am sorry, I cheated a bit using configure, and there is no ${workpath}/build 
during configure, of course.

The second one did obliterate the first, which is what I saw last night, and it 
shows up better using 

destroot.env-append

where everything exists. For example, with “atk”, adding this:

destroot.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
destroot.env-append“DYLD_LIBRARY_PATH=${workpath}/build”

gives:

“DYLD_LIBRARY_PATH='/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_atk/atk/work/build”'

Ken

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ken Cunningham]

> On Oct 24, 2020, at 9:46 AM, Ken Cunningham  
> wrote:
> 
> configure.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
> configure.env-append“DYLD_LIBRARY_PATH=${workpath}/build”
> 
> but I found the second one just obliterates the first one.
> 
> DYLD_LIBRARY_PATH='/opt/local/lib/lilbgcc'
> 
> 

well the second one didn’t obliterate the first — other way around. 

but they both didn’t make into the env, which was what I was going for with 
that comment.

K

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Ken Cunningham]

> On Oct 24, 2020, at 8:52 AM, Joshua Root  wrote:
> 
> On 2020-10-25 02:18 , Ken Cunningham wrote:
>> It's easy to prepend or append to PATH in a Portfile, but I'm having 
>> difficulty seeing how to modify, rather than overwrite, DYLD_LIBRARY_PATH.
>> 
>> Usually you just want to set it, but now that it might or might not be set 
>> in legacysupport 1.1, I was hoping to just modify rather than overwrite it.
>> 
>> I tried a number of different things, but couldn't spot the magic 
>> incantation...am 
>> i missing something simple?
> 
> What did you try? The configure.env, build.env etc. options can be read
> like any other option. You can use lsearch on it to look for entries
> starting with the variable name you're after.
> 
> - Josh


Ah, thanks for the clue.

I had hoped that just something like this would set them both:

configure.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
configure.env-append“DYLD_LIBRARY_PATH=${workpath}/build”

but I found the second one just obliterates the first one.

DYLD_LIBRARY_PATH='/opt/local/lib/lilbgcc'


So I tried variations of this construct, that works for PATH:

configure.env-append"PATH=$env(PATH):${bootstrap_dir}/bin”

eg:

configure.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
configure.env-append
“DYLD_LIBRARY_PATH=$env(DYLD_LIBRARY_PATH):${workpath}/build”


but that just gives 

DEBUG: no such variable
(read trace on "env(DYLD_LIBRARY_PATH)")

because the first one doesn’t get set into the environment until the whole 
Portfile is parsed, obviously in retrospect. 

And likewise testing for it with [info exists …] won’t work either, I can see, 
because it’s not there yet.

So instead of this method, I will look into running lsearch on *.env, and see 
if I can sort out how to add some entries if something is already there. 

It looks like the Makefile-1.0 PortGroup might be doing something along these 
lines — getting a tad complicated, however:


foreach env_var { \
PKG_CONFIG_PATH \
} {
set value [option configure.[string tolower $env_var]]
if {$value ne ""} {
${phase}.env-append "$env_var=[join $value :]"
if {[lsearch -exact ${makefile.override} ${env_var}] != -1} {
${phase}.args-append$env_var="[join $value :]"
}
}
}
=




Do you think this:

configure.env-append"DYLD_LIBRARY_PATH=${prefix}/lib/lilbgcc"
configure.env-append“DYLD_LIBRARY_PATH=${workpath}/build”

should “just work" to set both without added shenanigans?

Ken

Re: how to prepend or append to DYLD_LIBRARY_PATH in Portfile?

[Joshua Root]

On 2020-10-25 02:18 , Ken Cunningham wrote:
> It's easy to prepend or append to PATH in a Portfile, but I'm having 
> difficulty seeing how to modify, rather than overwrite, DYLD_LIBRARY_PATH.
> 
> Usually you just want to set it, but now that it might or might not be set in 
> legacysupport 1.1, I was hoping to just modify rather than overwrite it.
> 
> I tried a number of different things, but couldn't spot the magic 
> incantation...am 
> i missing something simple?

What did you try? The configure.env, build.env etc. options can be read
like any other option. You can use lsearch on it to look for entries
starting with the variable name you're after.

- Josh