Re: Apple ARM binary codesign issue
> On Sep 22, 2020, at 18:54, Jeffrey Walton wrote: > > If you modify the binary or assets in the bundle, you should have to > resign the bundle. I don't think there's anything inherently insecure > about (re)signing a bundle after modification. Or nothing comes to > mind (for me). I see two basic cases: what the person compiling chooses to trust for their own use, and what is suitable for binary distribution. Ad-hoc is fine for the former. But in either case, I think it's more trustworthy if the same entity signs a modification as signed the original compile; otherwise, they don't actually know what was compiled (not that they reviewed all the code anyway, but if they did both, they at least had the opportunity to know what's in there). So if not, one could compile in good faith, another could modify in good faith, but something slipped through...who? How will additional signing requirements impact MacPorts binary distribution (which is a huge timesaver for installs and updates, if one doesn't have to build most packages oneself)?
Re: Apple ARM binary codesign issue
On Tue, Sep 22, 2020 at 6:24 PM Ken Cunningham wrote: > ... > I'm trying to imagine how Jeremy might prevent hackers from surreptitiously > modifying signed binaries with strip or install_name_tool (which is good) > while letting people modify signed binaries with strip or install_name_tool > without invalidating the signature -- I don't immediately see how you can > have it both ways. But maybe Jeremy has some trick that works for this I > can't think of. I have not looked at Apple signing in a while, so take this with a grain of salt... Apple should be signing assets and resources in the bundle, and not just the binary. That's called "semantic authentication". See David Wagner and Bruce Schneier's "Analysis of the SSL 3.0 Protocol" (c. 1996). Lack of semantic authentication is what Android bug 8219321 was all about, a.k.a Bluebox's "master key" bug or the "one key to rule them all." Android patched the 8219321 bug in 2013. If you modify the binary or assets in the bundle, you should have to resign the bundle. I don't think there's anything inherently insecure about (re)signing a bundle after modification. Or nothing comes to mind (for me). The standard caveats apply, like the codesign tool should make an insignificant modification before signing, like adding a whitespace to an XML file or text file. And codesign tool should calculate the digest itself, and not sign a precomputed digest. Never trust someone else's hash. Calculate the hash yourself. Jeff
Re: Apple ARM binary codesign issue
On 2020-09-22, at 12:58 PM, Ryan Schmidt wrote: > > To me it seems unrealistic for Apple to suggest that an infinite number of > open source projects, many of whose developers have never seen a Mac, should > now add code to their build systems to codesign things on macOS. Apple made a > point of stating during WWDC that they love open source software; imposing > busy work on the open source community is not a good way to show that love. As I read it, the linker automatically codesigns the binary when you link, which is usually the final step in the process. So nobody has to change anything there. But if you later modify that final linked binary by stripping it (I guess ) or changing the libraries around with install_name_tool (which I believe MESON does to every single install :>) then you invalidate the signature, as you should IMHO. I'm trying to imagine how Jeremy might prevent hackers from surreptitiously modifying signed binaries with strip or install_name_tool (which is good) while letting people modify signed binaries with strip or install_name_tool without invalidating the signature -- I don't immediately see how you can have it both ways. But maybe Jeremy has some trick that works for this I can't think of. I won't be surprised if the solution is that you have to resign them after doing that, though. Ken
Re: Apple ARM binary codesign issue
On 2020-9-23 05:33 , Ryan Schmidt wrote: > > Send feedback through the Feedback Assistant app. Yes, everyone with any issues with Apple preview software should do this early and often. I've read that number of duplicates is one of the ways they determine issue priority internally. - Josh
mysqld 5.7: server quit without updating PID file
Something changed when updating various mysql-related ports recently: I can no longer start mysqld. Error is: ERROR! The server quit without updating PID file (/Users/me/Databases/mysql/data/Blackstone.local.pid). Below is my /opt/local/etc/mysql57/my.cnf file. It includes the macports-default.cnf file, whose only non-commented lines are: [mysqld] skip-networking What could be wrong? I’ve searched on-line, and nothing I’ve found so far seems to be relevant (most referring to installations directly from mysql download into /usr/local or else to installations using homebrew. To the best of my knowledge, I have not intervened to change any permissions on the data files, which are in /Users/me/Databases/mysql/data # my.cnf # # Use default MacPorts settings !include /opt/local/etc/mysql57/macports-default.cnf # For advice on how to change settings please see # http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html [mysqld] # Remove leading # and set to the amount of RAM for the most important data # cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%. # innodb_buffer_pool_size = 128M # Remove leading # to turn on a very important data integrity option: logging # changes to the binary log between backups. # log_bin = mysql-bin # These are commonly set, remove the # and set as required. # basedir = /usr/local/mysql # basedir = /opt/local/lib/mysql57 # datadir = /usr/local/mysql/data # datadir = /Users/me/mysql/data # datadir = /opt/local/var/db/mysql56 datadir = /Users/me/Databases/mysql/data port = 3306 # server_id = . # socket = /tmp/mysql.sock socket = /opt/local/var/run/mysql57/mysqld.sock # pid-file = /opt/local/var/run/mysqld/mysqld.pid # Remove leading # to set options mainly useful for reporting servers. # The server defaults are faster for transactions and fast SELECTs. # Adjust sizes as needed, experiment to find the optimal values. # join_buffer_size = 128M # sort_buffer_size = 2M # read_rnd_buffer_size = 2M sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES [client] # socket = /tmp/mysql.sock socket = /opt/local/var/run/mysql57/mysqld.sock # end my.cnf --- Murray Eisenbergmurrayeisenb...@gmail.com 503 King Farm Blvd #101 Home (240)-246-7240 Rockville, MD 20850-6667Mobile (413)-427-5334
Re: Apple ARM binary codesign issue
On Sep 22, 2020, at 14:52, Ken Cunningham wrote: > On 2020-09-22, at 11:58 AM, Ryan Schmidt wrote: >> >> I hope that Apple fixes their toolchain to work without such intervention. > > I believe this may ultimately come under the category of "intended > behaviour". To me it seems unrealistic for Apple to suggest that an infinite number of open source projects, many of whose developers have never seen a Mac, should now add code to their build systems to codesign things on macOS. Apple made a point of stating during WWDC that they love open source software; imposing busy work on the open source community is not a good way to show that love. Part of the problem is that I don't understand what codesigning is for. I understand that for distribution in the app store I would codesign an app with my identity. Apple can then revoke apps I've signed if it turns out I've released malicious software. Great, but here we're not talking about signing with an identity, we're talking about ad-hoc signing. All that proves is that the binary hasn't been modified. That's good; it'll protect against corrupted files getting used. But how would it stop malicious code? Couldn't the malicious party just codesign it again after changing it?
Re: Apple ARM binary codesign issue
On 2020-09-22, at 11:58 AM, Ryan Schmidt wrote: > > I hope that Apple fixes their toolchain to work without such intervention. > I believe this may ultimately come under the category of "intended behaviour". I can understand why they would want these changes to invalidate the signature. I suspect we'll have to find a way to re-sign them after making such changes. In addition, we may not find that automatically doing it by wrapping our strip or install_name_tool with codesigners is suitably secure. Ken
Re: Apple ARM binary codesign issue
On Sep 22, 2020, at 14:19, Michael Dickens wrote: > I have macOS 11.0beta7 installed : check! > > Compare / contrast ARM Mac versus MacBook Pro 16 : check! > > I have Xcode 12.2 beta installed : check! > > I've removed "/Library/Developer/CommandLineTools" : check! > > I hope that Apple fixes their toolchain to work without such intervention : > check! > > Do you know that best way we can complain to Apple to fix the toolchain? > > Still doesn't work for me ... those specific files are almost the only ones > that just don't respond to codesign. I'll try reinstalling from scratch ... > maybe ports built with Xcode 12.0 don't entirely work with those built using > 12.2beta? H I was getting the "codesign_allocate helper tool cannot be found or used" error earlier too; the only thing I think I changed since then is getting rid of the Xcode 12.0 beta CLT. Though I do have a couple different MacPorts installs and I was working on the other one before. Send feedback through the Feedback Assistant app. They have responded to one of mine recently.
Re: Apple ARM binary codesign issue
On 22/09/2020 15:19, Michael Dickens wrote: > I have macOS 11.0beta7 installed : check! > > Compare / contrast ARM Mac versus MacBook Pro 16 : check! > > I have Xcode 12.2 beta installed : check! > > I've removed "/Library/Developer/CommandLineTools" : check! > > I hope that Apple fixes their toolchain to work without such intervention : > check! > > Do you know that best way we can complain to Apple to fix the toolchain? Report here bugreport.apple.com Then report the same report here: https://openradar.appspot.com/page/1 (without violating your Apple developer ToS). Andrew signature.asc Description: OpenPGP digital signature
Re: Apple ARM binary codesign issue
I have macOS 11.0beta7 installed : check! Compare / contrast ARM Mac versus MacBook Pro 16 : check! I have Xcode 12.2 beta installed : check! I've removed "/Library/Developer/CommandLineTools" : check! I hope that Apple fixes their toolchain to work without such intervention : check! Do you know that best way we can complain to Apple to fix the toolchain? Still doesn't work for me ... those specific files are almost the only ones that just don't respond to codesign. I'll try reinstalling from scratch ... maybe ports built with Xcode 12.0 don't entirely work with those built using 12.2beta? H Thx! - MLD On Tue, Sep 22, 2020, at 2:58 PM, Ryan Schmidt wrote: > > > On Sep 22, 2020, at 13:29, Michael Dickens wrote: > > > % codesign -v - --ignore-resources > > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > > invalid signature (code or signature have been modified) > > % sudo codesign -s - > > --preserve-metadata=identifier,entitlements,flags,runtime -f > > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > > replacing existing signature > > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > > the codesign_allocate helper tool cannot be found or used > > % which codesign > > /usr/bin/codesign > > % which codesign_allocate > > /usr/bin/codesign_allocate > > > > You need Xcode 12.2 beta, which you probably have, but also make sure > that you don't have old command line tools installed. I don't think the > Xcode 12.0 beta command line tools are compatible, and there isn't an > Xcode 12.2 beta command line tools yet. Delete > /Library/Developer/CommandLineTools. > > > % codesign -v - > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > invalid signature (code or signature have been modified) > In architecture: arm64 > % codesign -s - > --preserve-metadata=identifier,entitlements,flags,runtime -f > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > replacing existing signature > ... > % codesign -v - > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > valid on disk > mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > satisfies its Designated Requirement > % > > > > Mentioned as possible fixes were: (1) inserting MP strip and > > install_name_tool wrappers that sign the binary if the signature is broken; > > or (2) a new step in destroot_finish . > > I hope that Apple fixes their toolchain to work without such intervention. > >
Re: Apple ARM binary codesign issue
On Sep 22, 2020, at 13:29, Michael Dickens wrote: > % codesign -v - --ignore-resources > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > invalid signature (code or signature have been modified) > % sudo codesign -s - > --preserve-metadata=identifier,entitlements,flags,runtime -f > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > replacing existing signature > /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: > the codesign_allocate helper tool cannot be found or used > % which codesign > /usr/bin/codesign > % which codesign_allocate > /usr/bin/codesign_allocate > You need Xcode 12.2 beta, which you probably have, but also make sure that you don't have old command line tools installed. I don't think the Xcode 12.0 beta command line tools are compatible, and there isn't an Xcode 12.2 beta command line tools yet. Delete /Library/Developer/CommandLineTools. % codesign -v - mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: invalid signature (code or signature have been modified) In architecture: arm64 % codesign -s - --preserve-metadata=identifier,entitlements,flags,runtime -f mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: replacing existing signature ... % codesign -v - mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: valid on disk mp/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: satisfies its Designated Requirement % > Mentioned as possible fixes were: (1) inserting MP strip and > install_name_tool wrappers that sign the binary if the signature is broken; > or (2) a new step in destroot_finish . I hope that Apple fixes their toolchain to work without such intervention.
Apple ARM binary codesign issue
There has been some discussion about the recent change Apple made for macOS
11.0beta7 for ARM Mac only (-not- Intel Mac at this time); we in MP-land had
some on this PR < https://github.com/macports/macports-ports/pull/8328 >. As
pointed out, a better venue for discussion would be these lists.
The brief summary is that Apple is instituting code signing for all binaries
during linking as noted here <
https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-universal-apps-beta-release-notes#Updates-in-macOS-Big-Sur-11-Universal-Apps-Beta-7
>. The signing is as-hoc and done automatically; it is invalidated if one
modifies the binary in any way ... for example using "strip", or
"install_name_tool" to change the name of a required library, or even the
self-ID name.
Many MacPorts installs use various post-linking means to tweak the resulting
binary, and hence these are not currently usable under macOS 11.0beta7 or more
recent on ARM Mac only; again: this change is NOT for Intel Mac -- at least not
at this time.
One can test whether the signing is valid via the command: "codesign -v FILE" ;
according to Apple one should use "codesign -v - --ignore-resources FILE"
to be more verbose as well as look at just the non-resource binary. Both work
in my testing.
In my initial testing, some of the binaries can be made to work via the command
"[sudo] codesign -s - --preserve-metadata=identifier,entitlements,flags,runtime
-f FILE" ... but, not all! For example:
{{{
% codesign -v - --ignore-resources
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8:
invalid signature (code or signature have been modified)
% sudo codesign -s - --preserve-metadata=identifier,entitlements,flags,runtime
-f /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8:
replacing existing signature
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: the
codesign_allocate helper tool cannot be found or used
% which codesign
/usr/bin/codesign
% which codesign_allocate
/usr/bin/codesign_allocate
}}}
So in this case, "codesign_allocate" cannot be used for some non-obvious reason
(as it clearly is in the PATH and works for some other codesign efforts). The
same holds true for some of the Python libraries and some of the primary
executables -- but, not all!
Mentioned as possible fixes were: (1) inserting MP strip and install_name_tool
wrappers that sign the binary if the signature is broken; or (2) a new step in
destroot_finish .
Let's pick up the discussion here, and try to work out a fix in short order.
For those of us trying to do anything within MacPorts using ARM Mac, this new
feature is causing significant issues & needs to be dealt with so that we can
get back to real work instead of fighting macOS::codesign .
Cheers! - MLD
