Re: [sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

On 27/06/2024 18:28, Ozkan Sezer wrote:

On Thu, Jun 27, 2024 at 5:29 PM Ileana Dumitrescu
 wrote:


Follow-up Comment #16, sr #111069 (group libtool):

I have updated development with a feature test:

https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development=3d1baeeef90a6b9e16c545babe833f28dca155a7

Please tell me if there are any issues with it.


Working for me: -no_fixup_chains support is detected correctly against
several Apple linker versions in my linux-to-mac cross-compile attempts.


I am happy to know this is working for someone else! If no issues
are reported before next week, I will merge this into master and
close this issue.


--
O.S.



--
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354


OpenPGP_0x6570EA01146F7354.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ozkan Sezer]

On Thu, Jun 27, 2024 at 5:29 PM Ileana Dumitrescu
 wrote:
>
> Follow-up Comment #16, sr #111069 (group libtool):
>
> I have updated development with a feature test:
>
> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development=3d1baeeef90a6b9e16c545babe833f28dca155a7
>
> Please tell me if there are any issues with it.

Working for me: -no_fixup_chains support is detected correctly against
several Apple linker versions in my linux-to-mac cross-compile attempts.

--
O.S.

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

Follow-up Comment #16, sr #111069 (group libtool):

I have updated development with a feature test:

https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development=3d1baeeef90a6b9e16c545babe833f28dca155a7

Please tell me if there are any issues with it.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/


signature.asc
Description: PGP signature

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

Update of sr #111069 (group libtool):

  Status:Done => In Progress
 Open/Closed:  Closed => Open   

___

Follow-up Comment #15:

[comment #12 comment #12:]
> This is the commit:
> 
>
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development=001d22d7d587e85a911c71c4d0c798ede8014b77
> 
> It's a good idea to always provide a link to the commit.

I will start adding this into my responses.

> You could remove the comment that "$MACOSX_DEPLOYMENT_TARGET seems to be
deprecated as a darwin environment variable" because I don't know what you're
talking about there and it has nothing to do with this issue.

The comment was added to help developers understand why the
"$MACOSX_DEPLOYMENT_TARGET" environment variable was not used in processing
below it. It was mostly added to avoid confusion, but since it was not
helpful, I have removed it.

> In the check for macOS version -- "11.[[3-7]]*|1[[2-4]]*)" -- you should
change it to "11.[[3-7]]*|1[[2-4]].*)". You want to check whether the version
begins with "12." or "13." or "14.", not whether it begins with "12", "13", or
"14". Fixing this will avoid problems with hypothetical future macOS version
120 and above.
> 
> Similarly, in the check for Xcode CLT version -- "*version:\ 1[[3-5]]*)" --
you should change it to "*version:\ 1[[3-5]].*)" so that you check whether it
begins with "13.", "14.", or "15.", rather than whether it begins with "13",
"14", or "15".
> 
> Sloppy libtool version numbering checks, particularly in this area of the
code, have been the cause of two major fiascos in the macOS world before (the
one where libtool did not anticipate the existence of OS X 10.10 and the one
where it did not anticipate the existence of macOS 11). It would behoove the
libtool project to create a proper version number comparison function and use
it everywhere, rather than faking it with string comparisons, to avoid such
problems in the future.

Thank you for pointing out the missing period! I did a lot of testing locally
with different regular expressions, but I was not thinking far enough into the
future.

> Once you've identified an Xcode CLT version that needs the fix, I'm not sure
why you're repeating most of the definition of the variable with
"_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup
$wl-no_fixup_chains'". Can't you just append the one flag to the existing
variable's value?

Yes, I will edit this.

> Finally, you're only checking the version of the Xcode CLT. The user does
not necessarily have the Xcode CLT installed. They might instead have only the
full Xcode installed. In that case, your code will not add the required flag.
Instead of trying to identify whether the user has the full Xcode or the Xcode
CLT (or, if both, which one is selected for use), you may want to just try to
figure out which linker they have. Unfortunately, I couldn't find an option to
ld to display its version number.

While I would like to verify based on the linker versions that support the
'-no-fixup_chains' flag, I only know of one linker version, 711. After I do
some more testing, I hope to know some more versions to apply the flag to.
However, I _would_ like to change the current version test to a feature test
which checks whether the flag is supported by the linker. Considering all of
the issues with the version testing, the feature test seems like the better
option, but let me know what you think.

For now, I will reopen this issue until a better solution is applied.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/


signature.asc
Description: PGP signature

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

Follow-up Comment #14, sr #111069 (group libtool):

[comment #11 comment #11:]
> [comment #10 comment #10:]
> > I have merged a patch in the development branch to append
'-no_fixup_chains' for macOS versions 11.3*-14* and Xcode versions 13-15* ...
> 
> Thanks for taking care of this.  I did not understand libtool enough to do
it myself.

Thank you for submitting the patch and starting this discussion! Hopefully a
solution can be found that works better though.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/


signature.asc
Description: PGP signature

Re: [sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

On 21/06/2024 21:36, Ozkan Sezer wrote:

I have merged a patch in the development branch to append '-no_fixup_chains'
for macOS versions 11.3*-14* and Xcode versions 13-15*. I have verified this
removes the warning that I was seeing on macOS 12.7.5, Xcode
14.2.0.0.1.1668646533 [1][2]. I have not seen any regressions from this on
macOS 11-14 CI tests, so I will close this issue now.


I believe the patch is this, yes?
http://git.savannah.gnu.org/gitweb/?p=libtool.git;a=commitdiff;h=001d22d7d587e85a911c71c4d0c798ede8014b77

I haven't tested yet, but unless I'm missing something, it relies on the
build environment not being a cross-environment. Can we not check linker
version instead?



I would like to check the linker version, but I am unsure which
versions of the linker to look for after 711 [1], which Carlo
linked previously.

[1] 
https://github.com/Homebrew/brew/blob/f1a7d971f2e5d04097d7a360aa1f9a910ccc20f3/Library/Homebrew/extend/os/mac/extend/ENV/shared.rb#L37-L42


--
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354


OpenPGP_0x6570EA01146F7354.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Dave Allured]

Follow-up Comment #13, sr #111069 (group libtool):

[comment #7 comment #7:]
> ... Newer versions of ld64 already know to disable chained fixups when it is
passed `-undefined dynamic_lookup`. This patch is really only relevant for a
small range of versions of ld64 that doesn't have this behaviour.

Ideally '-no_fixup_chains' should be limited to this small range of linker
versions where the extra flag is actually needed to prevent error, or to
suppress an unwanted warning message.  Can anyone say which versions these
actually are, let alone how to code the proper test for them?


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ryan Carsten Schmidt]

Follow-up Comment #12, sr #111069 (group libtool):

This is the commit:

https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development=001d22d7d587e85a911c71c4d0c798ede8014b77

It's a good idea to always provide a link to the commit.

You could remove the comment that "$MACOSX_DEPLOYMENT_TARGET seems to be
deprecated as a darwin environment variable" because I don't know what you're
talking about there and it has nothing to do with this issue.

In the check for macOS version -- "11.[[3-7]]*|1[[2-4]]*)" -- you should
change it to "11.[[3-7]]*|1[[2-4]].*)". You want to check whether the version
begins with "12." or "13." or "14.", not whether it begins with "12", "13", or
"14". Fixing this will avoid problems with hypothetical future macOS version
120 and above.

Similarly, in the check for Xcode CLT version -- "*version:\ 1[[3-5]]*)" --
you should change it to "*version:\ 1[[3-5]].*)" so that you check whether it
begins with "13.", "14.", or "15.", rather than whether it begins with "13",
"14", or "15".

Sloppy libtool version numbering checks, particularly in this area of the
code, have been the cause of two major fiascos in the macOS world before (the
one where libtool did not anticipate the existence of OS X 10.10 and the one
where it did not anticipate the existence of macOS 11). It would behoove the
libtool project to create a proper version number comparison function and use
it everywhere, rather than faking it with string comparisons, to avoid such
problems in the future.

Once you've identified an Xcode CLT version that needs the fix, I'm not sure
why you're repeating most of the definition of the variable with
"_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup
$wl-no_fixup_chains'". Can't you just append the one flag to the existing
variable's value?

Finally, you're only checking the version of the Xcode CLT. The user does not
necessarily have the Xcode CLT installed. They might instead have only the
full Xcode installed. In that case, your code will not add the required flag.
Instead of trying to identify whether the user has the full Xcode or the Xcode
CLT (or, if both, which one is selected for use), you may want to just try to
figure out which linker they have. Unfortunately, I couldn't find an option to
ld to display its version number.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ozkan Sezer]

> I have merged a patch in the development branch to append '-no_fixup_chains'
> for macOS versions 11.3*-14* and Xcode versions 13-15*. I have verified this
> removes the warning that I was seeing on macOS 12.7.5, Xcode
> 14.2.0.0.1.1668646533 [1][2]. I have not seen any regressions from this on
> macOS 11-14 CI tests, so I will close this issue now.

I believe the patch is this, yes?
http://git.savannah.gnu.org/gitweb/?p=libtool.git;a=commitdiff;h=001d22d7d587e85a911c71c4d0c798ede8014b77

I haven't tested yet, but unless I'm missing something, it relies on the
build environment not being a cross-environment. Can we not check linker
version instead?

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Dave Allured]

Follow-up Comment #11, sr #111069 (group libtool):

[comment #10 comment #10:]
> I have merged a patch in the development branch to append '-no_fixup_chains'
for macOS versions 11.3*-14* and Xcode versions 13-15* ...

Thanks for taking care of this.  I did not understand libtool enough to do it
myself.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

Update of sr #111069 (group libtool):

 Open/Closed:Open => Closed 


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

Update of sr #111069 (group libtool):

  Status:None => Done   

___

Follow-up Comment #10:

I have merged a patch in the development branch to append '-no_fixup_chains'
for macOS versions 11.3*-14* and Xcode versions 13-15*. I have verified this
removes the warning that I was seeing on macOS 12.7.5, Xcode
14.2.0.0.1.1668646533 [1][2]. I have not seen any regressions from this on
macOS 11-14 CI tests, so I will close this issue now.

[1] logs-macos-12-before.zip
[2] logs-macos-12-after.zip

(file #56179, file #56180)

___

Additional Item Attachment:

File name: logs-macos-12-before.zip   Size: 520KiB
   


File name: logs-macos-12-after.zipSize: 533KiB



AGPL NOTICE

These attachments are served by Savane. You can download the corresponding
source code of Savane at
https://git.savannah.nongnu.org/cgit/administration/savane.git/snapshot/savane-6ae798d7549f3da4ca720a69d41cfe87298d1ed1.tar.gz


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Carlo Cabrera]

Follow-up Comment #9, sr #111069 (group libtool):

[comment #8 comment #8:] 
> If you could modify this patch to include a linker version check that should
be sufficient.

You could probably try to parse this output:
```
❯ cc -Wl,-version_details -xc - -o /dev/null <<<'int main(){}' 2>/dev/null
{
"version": "1053.12",
"architectures": [
"armv6",
"armv7",
"armv7s",
"arm64",
"arm64e",
"arm64_32",
"i386",
"x86_64",
"x86_64h",
"armv6m",
"armv7k",
"armv7m",
"armv7em"
],
"lto": {
"runtime_api_version": 29,
"static_api_version": 29,
"version_string": "LLVM version 15.0.0"
},
"tapi": {
"version": "15.0.0",
"version_string": "Apple TAPI version 15.0.0
(tapi-1500.3.2.2)"
}
}
```

You need `version` to be greater than or equal to 711. If it errors you can
probably assume that `-no_fixup_chains` is not supported.

You could probably also parse the output of `ld -v`, but that might be a
little more fiddly.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ileana Dumitrescu]

Follow-up Comment #8, sr #111069 (group libtool):

Thanks for the discussion.

If you could modify this patch to include a linker version check that should
be sufficient. I do not think a feature test is necessary since the behaviour
seems rigidly defined between linker versions, and these fixup chains are only
implemented for the macOS linker.

[comment #7 comment #7:]
> [comment #6 comment #6:]
> > It would be better to understand whether what libtool is doing is
incompatible with chained fixups, and if so, fix that so that it works with
chained fixups.
> libtool passes `-undefined dynamic_lookup` to the linker, which is
incompatible with chained fixups.

It seems like fixup chains are already disabled by default since `-undefined
dynamic_lookup` is always passed. Accepting this patch with a linker test
should result in no new regressions to Libtool and the removal of the warnings
for certain macOS versions.

[comment #7 comment #7:]
> Though, if the idea is to keep chained fixups, then libtool must stop
passing
> `-undefined dynamic_lookup` to the linker. This has the possibility of
> producing undesirable consequences for many downstream projects, however.

Unfortunately I do not know what the downstream consequences would be, so
Libtool will keep `-undefined dynamic_lookup` for now and continue to disable
fixup chains. If someone could characterise the consequences for removing
`-undefined dynamic_lookup, this could be revisited.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Carlo Cabrera]

Follow-up Comment #7, sr #111069 (group libtool):

[comment #6 comment #6:]
> It would be better to understand whether what libtool is doing is
incompatible with chained fixups, and if so, fix that so that it works with
chained fixups.

libtool passes `-undefined dynamic_lookup` to the linker, which is
incompatible with chained fixups.

Newer versions of ld64 already know to disable chained fixups when it is
passed `-undefined dynamic_lookup`. This patch is really only relevant for a
small range of versions of ld64 that doesn't have this behaviour.

Though, if the idea is to keep chained fixups, then libtool must stop passing
`-undefined dynamic_lookup` to the linker. This has the possibility of
producing undesirable consequences for many downstream projects, however.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Ryan Carsten Schmidt]

Follow-up Comment #6, sr #111069 (group libtool):

The libtool that was used to generate gsl 2.8's configure script includes this
patch, which is why gsl 2.8 is failing to build with older macOS toolchains
that don't understand this flag.

It does not seem like libtool should accept this patch. Chained fixups are an
intentional new feature of Apple's linker. It should not be libtool's standard
behavior to attempt to disable this new feature. It would be better to
understand whether what libtool is doing is incompatible with chained fixups,
and if so, fix that so that it works with chained fixups. I found this
third-party document describing what chained fixups are and why they are
desirable:

https://github.com/qyang-nj/llios/blob/main/dynamic_linking/chained_fixups.md


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Carlo Cabrera]

Follow-up Comment #5, sr #111069 (group libtool):

Actually, checking the behaviour of ld64 when it's given unknown flags isn't
hard:

```
❯ clang -xc - -Wl,-nonexistent_flag -o /dev/null <<<'int main(){}'
ld: unknown options: -nonexistent_flag
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
❯ echo $?
1
```

[comment #4 comment #4:]
> Now I wonder if a feature test might be better here in libtool, rather than
a linker version number test.

Yes, I suspect that might be more appropriate. But I don't do any libtool
development, so my opinion probably doesn't count for much.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Dave Allured]

Follow-up Comment #4, sr #111069 (group libtool):

[comment #3 comment #3:]
> I personally tested this a while back

Thanks for letting me know about that testing.  Now I wonder if a feature test
might be better here in libtool, rather than a linker version number test.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Carlo Cabrera]

Follow-up Comment #3, sr #111069 (group libtool):


> My original patch assumes that `-no_fixup_chains` would be silently ignored
by intermediate linker versions older than Xcode 13.  I have not yet tested
this assumption on any such Macs.

This is not a good assumption, I'm afraid. I personally tested this a while
back and found that older versions of ld64 will error when passed the
`-no_fixup_chains` flag.

See https://github.com/pybind/pybind11/pull/4301#issuecomment-1446551317.
(Unfortunately, the actions logs are no longer available from those tests.)


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Dave Allured]

Follow-up Comment #2, sr #111069 (group libtool):

[comment #1 comment #1:]
> Note that the `-no_fixup_chains` flag is supported only on Xcode 13 or newer
(which in turn requires macOS 11.3 or newer).
> 
> See this relevant method used in Homebrew: ...

Thank you for this information.  I have trouble finding definitive
documentation for Apple development tools.

My original patch assumes that `-no_fixup_chains` would be silently ignored by
intermediate linker versions older than Xcode 13.  I have not yet tested this
assumption on any such Macs.

If that is a good assumption, then existing guards in the current 2.4.7
libtool code block should be sufficient, and I recommend merge with no
changes.

If not a good assumption, or it goes against GNU libtool design standards in
any way, such as no unknown flags, then I request help on how to construct the
correct test within libtool, for the Xcode linker version.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Carlo Cabrera]

Follow-up Comment #1, sr #111069 (group libtool):

Note that the `-no_fixup_chains` flag is supported only on Xcode 13 or newer
(which in turn requires macOS 11.3 or newer).

See this relevant method used in Homebrew:

https://github.com/Homebrew/brew/blob/f1a7d971f2e5d04097d7a360aa1f9a910ccc20f3/Library/Homebrew/extend/os/mac/extend/ENV/shared.rb#L37-L42


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[sr #111069] libtool: patch: Fix dynamic_lookup warnings from new Mac linker

[Dave Allured]

URL:
  

 Summary: libtool: patch: Fix dynamic_lookup warnings from new
Mac linker
   Group: GNU Libtool
   Submitter: dallured
   Submitted: Fri 24 May 2024 06:01:06 PM UTC
Category: None
Priority: 5 - Normal
Severity: 3 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
Operating System: Mac OS


___

Follow-up Comments:


---
Date: Fri 24 May 2024 06:01:06 PM UTC By: Dave Allured 
Apple introduced a new linker with macOS 14/Sonoma.  That included new
behavior for dynamic lookups.  Unpatched Mac builds now encounter this
warning, and possible subsequent execution failures:

ld: warning: -undefined dynamic_lookup may not work with chained fixups

The attached one-line patch for m4/libtool.m4 adds "-no_fixup_chains" which
constrains linker to traditional dynamic lookups and safe behavior.  This
patch targets the 2.4.7 release version of m4/libtool.m4.

Caveats.  I am not well experienced in linkers or GNU tool systems.  This
patch is my current best guess as to which libtool file to patch, and what is
the simplest remedy for that warning and consequences.

The current 2.4.7 version m4/libtool.m4 includes guards for platform and OS
version which I think are sufficient for this new flag.  I tested this patch
in a downstream package only, GNU GSL.  I tested this on several OS versions
including down level linkers, using Macports CI (github actions).  This did
not encounter any "unknown option" problems.  So I think that new guards are
not needed.

This patch was inspired by a downstream fix a year ago.  Please follow the
embedded link to Cpython, for a much more extensive analysis:
https://github.com/lcm-proj/lcm/pull/416

I like this:  "Compared to using chained fix-ups, the only potential downside
for us is a slightly larger dylib"






___
File Attachments:


---
Name: patch-libtool.darwin.no_fixup_chains.diff  Size: 473B


AGPL NOTICE

These attachments are served by Savane. You can download the corresponding
source code of Savane at
https://git.savannah.nongnu.org/cgit/administration/savane.git/snapshot/savane-4b48cbb9570c156bf7d681225b664258d7028914.tar.gz

___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/