Re: rev-upgrade and checking for +universal dependencies (and build dependencies)

[Saagar Jha]

I haven’t seen it for a while but for a long time full software updates would 
reset SIP state, among other things.

Saagar Jha

> On Jun 18, 2024, at 06:01, René J.V. Bertin  wrote:
> 
> On Tuesday June 18 2024 05:49:39 Saagar Jha wrote:
>> If you turn SIP off, and patch MacPorts to build an arm64e slice, then you 
>> can use trace mode. However your patches will build a broken trace mode for 
>> anyone who has SIP enable
> 
> OK, that's also what I understood (but I don't think that's what your 
> original message actually said).
> 
>> and you will also be very sad if your machine ever boots in that state.
> 
> That shouldn't happen by accident, SIP mode is determined via a firmware 
> toggle, right?
> 
> 
> R.
> 
> 

Re: rev-upgrade and checking for +universal dependencies (and build dependencies)

[Saagar Jha]

If you turn SIP off, and patch MacPorts to build an arm64e slice, then you can 
use trace mode. However your patches will build a broken trace mode for anyone 
who has SIP enabled, and you will also be very sad if your machine ever boots 
in that state.

Saagar Jha

> On Jun 18, 2024, at 05:17, René J.V. Bertin  wrote:
> 
> On Tuesday June 18 2024 05:04:50 Saagar Jha wrote:
> 
>>> With SIP disabled, one can add a kernel boot argument to allow user
>>> space to use Apple's ABI with pointer authentication, which would allow
>>> building a copy of tracelib that works on those binaries.
>>> 
>>> I'm not aware that anybody has successfully done that.
>> 
>> I have. I never chose to upstream it because I figured that nobody cared for 
>> that configuration, and it has the unfortunate quality that it cannot be 
>> done in a way that degrades gracefully when SIP is off. On systems where SIP 
>> is disabled, the loader will look for an arm64e slice or complain loudly. If 
>> you include such a slice and SIP is disabled it will be preferentially 
>> selected over an arm64 version by the fat binary matching algorithm, and 
>> immediately terminate the process upon load. So there’s no good answer here 
>> :(
> 
> Something doesn't add up there. Do you mean those problems arise when you 
> turn SIP back ON, or do you mean that you "successfully did that" with SIP 
> turned ON?
> 
> Either way, it was to be foreseen that Apple would start turning their 
> computers into iThings when they came up with their own CPU design; the fact 
> that they're only barely NOT making it impossible to run Linux on them is a 
> telltale sign.
> 
> R.

Re: rev-upgrade and checking for +universal dependencies (and build dependencies)

[Saagar Jha]

> On Jun 9, 2024, at 12:10, Clemens Lang  wrote:
> 
> On Sat, Jun 08, 2024 at 01:10:50PM +0200, René J.V. Bertin wrote:
>> On Saturday June 08 2024 09:46:34 Clemens Lang wrote:
>> 
>>> Using library preloading – which does not currently work against
>>> Apple's binaries on Apple Silicon Macs.
>> 
>> But those aren't installed through MacPorts. Does it matter here
>> whether SIP & family are enabled or not, btw?
> 
> With SIP disabled, one can add a kernel boot argument to allow user
> space to use Apple's ABI with pointer authentication, which would allow
> building a copy of tracelib that works on those binaries.
> 
> I'm not aware that anybody has successfully done that.

I have. I never chose to upstream it because I figured that nobody cared for 
that configuration, and it has the unfortunate quality that it cannot be done 
in a way that degrades gracefully when SIP is off. On systems where SIP is 
disabled, the loader will look for an arm64e slice or complain loudly. If you 
include such a slice and SIP is disabled it will be preferentially selected 
over an arm64 version by the fat binary matching algorithm, and immediately 
terminate the process upon load. So there’s no good answer here :(

> It doesn't matter whether those are installed through MacPorts. Builds
> use many of those binaries (e.g., the Shell, or clang), and trace mode
> doesn't work on those binaries, rendering it essentially useless.
> Additionally, attempting to inject those binaries causes the program to
> be killed by the kernel, i.e., things break.
> 
> 
>>> Tracelib only works on spawned subprocesses, not the MacPorts tclsh
>>> process itself, so in order for that to work, rev-upgrade would have
>>> to spawn a separate process, which it currently does not.
>> 
>> For this particular case that should be a trivial detail. The
>> rev-upgrade procedure does not itself spawn any helper processes at
>> the moment, correct?
> 
> Correct, but it's not really trivial, it would require a significant
> rewrite and a way to communicate the results across the process
> boundary.
> 
> 
>> The elegance and security of a single command that can be written to
>> handle all possible error situations. Plus you're not crippling any
>> functionality at all.
> 
> Sure, but if I weigh that against the effort to implement it, it's not
> worth it in my book.
> 
> 
>> BTW, in terms of evaluation dependents of a shared library: would it
>> suffice to remove read and/or execute permissions from the file rather
>> than making the file disappear completely? If so it might already be
>> enough to do that for just the file owner, provided the rev-upgrade
>> scan is run as that user.  I suppose that files are normally owned by
>> root in a normal MP installation...
> 
> Many Unix operating systems, including macOS, do not actually care about
> file permissions when you are root:
> 
> :) root@cBookMax:/tmp# ls -lash test
> 8 --  1 root  wheel 4B Jun  9 21:09 test
> :) root@cBookMax:/tmp# cat test
> foo
> 
> -- 
> Clemens
> 

Re: Git-devel has broken git-credential-osxkeychain.c for older systems

[Saagar Jha]

Wait, I use osxkeychain. It’s basically a requirement if you’re pushing to an 
authenticated server over HTTPS and don’t want to have to deal with storing 
keys yourself. I suspect it is used a lot for this.

> On Apr 19, 2024, at 23:52, Kirill A. Korinsky  wrote:
> 
> On Sat, 20 Apr 2024 00:25:47 +0200,
> Sergio Had wrote:
>> 
>> What do we do? :)
> 
> To fix the build you have two options:
> 1. Revert that patch for system before 10.7
> 2. Remove folder contrib/credential/osxkeychain
> 
> I suggest to follow (2) as simpler thay and the good news that osxkeychain
> is something that isn't often used.
> 
> --
> wbr, Kirill

Re: Port Reclaim

[Saagar Jha]

Going back in time to fix a bug before I even asked for it is pretty impressive 
:) Thanks!

Saagar Jha

> On Nov 7, 2023, at 01:14, Joshua Root  wrote:
> 
> On 7/11/2023 17:39, Saagar Jha wrote:
>> I have noticed that “port install” will not mark a port as requested if it 
>> is already installed (e.g. as a dependency of something else you installed). 
>> Perhaps this is something that might be worth changing?
>> Saagar Jha
> 
> <https://trac.macports.org/ticket/55085>
> 
> - Josh

Re: Port Reclaim

[Saagar Jha]

I have noticed that “port install” will not mark a port as requested if it is 
already installed (e.g. as a dependency of something else you installed). 
Perhaps this is something that might be worth changing?

Saagar Jha

> On Nov 6, 2023, at 04:03, Mark Anderson  wrote:
> 
> Is that the only way? I've done that in the past with migrations, but for 
> Sonoma, I did a fresh install and installed ports as I wanted them again from 
> a list I kept. I assumed if I did a `port install` on any port it would have 
> marked those as requested.
> 
> —Mark
> ___
> Mark E. Anderson mailto:m...@macports.org>>
> MacPorts Trac WikiPage <https://trac.macports.org/wiki/mark>
> GitHub Profile <https://github.com/markemer>
> 
> 
> 
> On Mon, Nov 6, 2023 at 6:47 AM Kirill A. Korinsky  <mailto:kir...@korins.ky>> wrote:
>> you may specificly mark your port to be requested via `port setrequested 
>> blabla`
>> 
>> -- 
>> wbr, Kirill
>> 
>>> On 6. Nov 2023, at 11:39, Mark Anderson >> <mailto:m...@macports.org>> wrote:
>>> 
>>> I'm noticing that port reclaim routinely is asking to uninstall all my 
>>> ports. And this is not post migration. The rest of the command works as I 
>>> expect.
>>> 
>>> —Mark
>>> ___
>>> Mark E. Anderson mailto:m...@macports.org>>
>>> MacPorts Trac WikiPage <https://trac.macports.org/wiki/mark>
>>> GitHub Profile <https://github.com/markemer>
>>> 
>> 

Re: New ports.macports.org website

[Saagar Jha]

This looks awesome! We had some discussion earlier of what kinds of things we 
should be posting to social media, and I think major new redesigns and efforts 
at modernization are a great thing to highlight. I’d suggest posting this to 
Twitter, or I could figure out how to do that too if you want.

> On Jul 19, 2021, at 13:08, Mojca Miklavec  wrote:
> 
> Dear MacPorts users and developers,
> 
> I'm really thrilled to see the new version of the ports website
> finally being deployed at
> https://ports.macports.org
> 
> I'm extremely grateful for the excellent work that Arjun did during
> the GSOC summers of 2019 and 2020. He kept maintaining the code after
> GSOC and made the migration to his own server (the previous server was
> running out of resources to be able to cope with the additional
> requirements introduced in the last summer).
> 
> Many thanks also to Amar with a much deeper understanding of Django
> and related technologies who co-mentored the project in 2020 and made
> sure that we ended up with a product of much higher quality than any
> one of us could have achieved.
> 
> Some exciting new features involve:
> - Dark mode.
> - Better REST API.
> - Remove dependency on Google Charts (blocked in some parts of the world).
> - More advanced / extensive search & filtering for ports.
> - Find ports that provide a particular file.
> - The site runs livecheck and reports outdated ports.
> - Ability to log in with GitHub OAuth and follow your favourite ports.
> You can check whether they are outdated or broken on some platforms,
> making it easier to see where contributions are needed.
> 
> You can read a bit more about the project on Arjun's blog:
>https://arjunsalyan.com/gsoc20-report/
> 
> The code lives at
>https://github.com/macports/macports-webapp
> 
> We would be most happy to see contributions to this project, both in
> terms of programming as well as for design, ideas etc.
> Arjun did an impressive job already (everything from backend to
> design), but if we have some talented designers around, please speak
> up.
> 
> I would also like to invite everyone who hasn't done that already to
> opt-in for anonymous statistics submissions:
> 
>sudo port install mpstats
> 
> Thank you,
>Mojca
> 
> PS: The old version will still live at
>https://port-old.macports.org
> for a little while, just in case.

Re: Publicizing MacPorts

[Saagar Jha]

I’m not a MacPorts core maintainer or anything, nor am I a PR expert, but (for 
various reasons) I happen to interact with a lot of users who are curious about 
macOS package managers. Here’s a couple of takeaways I have from those 
conversations, as well as my thoughts on what MacPorts could do in light of 
them.

The biggest issue that I think MacPorts faces is that it has an image of being 
“old and dead”, especially compared to vibrant alternatives such as Homebrew. 
Of course, much of the response I get to MacPorts is “I’ve never heard of it” 
but even from the people who know what it is I hear a lot of “I used that 
several years ago, but then I switched package managers…wait, it’s still 
around?” There are a couple of other undesirable impressions: that MacPorts is 
the place that’s missing packages, or has ancient copies of them, or that 
MacPorts compiles everything from source (there was a thread on this list just 
recently on this topic). Obviously these are not quite true, nor are they 
things that one would want associated with MacPorts. I think the solution to 
this is two-pronged.

The first angle here is one of advocacy: MacPorts is obviously *not* dead, but 
nobody knows about it, because there is nothing to tell them about it. I think 
maintaining even a minimal social media presence would do much here. MacPorts 
should capitalize on its strengths and announce them when appropriate. This 
doesn’t have to be a lot of effort (and I would of course be happy to help with 
it, if that would be useful)–it’s things like spending an hour on a blog post 
with details of what’s in a new release, or posting relevant updates on Twitter 
when appropriate. Uncompromising M1 support has been mentioned in this thread 
already as a major plus for MacPorts, and I fully agree; nothing would have 
sold “this package manager is alive and well” better than showing how well 
MacPorts supported Apple silicon from the get-go. As another datapoint, compare 
https://news.ycombinator.com/from?site=brew.sh 
<https://news.ycombinator.com/from?site=brew.sh> to 
https://news.ycombinator.com/from?site=macports.org 
<https://news.ycombinator.com/from?site=macports.org>; and keep in mind that 
Hacker News is one of the few places on the internet where a Homebrew mention 
is often followed by favorable one for MacPorts. There’s probably some new OSes 
and hardware dropping later this year, perhaps it might be wise to highlight 
that MacPorts runs on them when they come out.

The other part is technical: MacPorts actually needs to live up to what we are 
trying to sell it as. If the website has pages that haven’t been restyled since 
2008, people are going to think that nobody has touched them since then, which 
isn’t how you want to present yourself if you’re an actively maintained 
project. (I know there has been work in the past in this area–I guess we just 
need to build on it.) If you try to install a package and the project tells you 
to use Homebrew to get it on macOS, or you install it in MacPorts and it’s a 
version from four years ago, then I think it is natural to end up with the idea 
that MacPorts doesn’t have packages you want. Where possible, I think we should 
track statistics of which of our packages livecheck as up-to-date, or how good 
our coverage is of things from other package managers. I know this is work and 
perhaps there is not enough manpower to handle it, but perhaps they can help 
inform what should be focused on.

One final thing: I think certain strategies might be counterproductive or 
unhelpful at this stage. Contacting the press is one way to get your message 
out, but it’s a fair bit of work, and fairly rare among software projects 
regardless. Other package managers get publicity just fine without it, and 
frankly if you’re already doing your publicity job well people will write about 
your thing of their own accord. Also, treating Homebrew and other package 
managers as “competition” where they have to lose is, IMO, an unhealthy way of 
looking at the situation. Homebrew caters to many developers and does a very 
good job at it, and I honestly believe that many of them would not enjoy using 
MacPorts. The right way to frame this, in my mind, is to recognize that 
Homebrew et al. have their own policies and ways of doing things, and MacPorts 
has its own. I don’t see us as trying to “steal” users away from other package 
managers–instead, the goal should be to identify those who are unhappy with 
them and help them realize that there are alternatives that might better suit 
their needs. For Homebrew at least I honestly believe that they would be glad 
to send their underserved users our way. We just need to work to make them 
aware of this choice.

Regards,
Saagar Jha

> On Apr 20, 2021, at 12:03, Rainer Müller  <mailto:rai...@macports.org>> wrote:
> 
> On 20/04/2021 12.40, Steven Smith wrote:
>> That’s begging the question of an effective communicatio

Re: M1 CPU features

[Saagar Jha]

J313 is the codename for the M1 MacBook Air.

Saagar Jha

> On Apr 26, 2021, at 11:56, Georges Martin  wrote:
> 
>> Aha, hw.optional! That's useful, thanks Georges!
> 
> You're welcome :-) You also have:
> 
>   hw.optional.amx_version: 2
>   hw.optional.arm64: 1
>   hw.targettype: J313
> 
> "amx" is the Neural Engine and I think "J313" is the code name for the M1.
> 
> You may find this article very interesting:
> 
>   
> https://levelup.gitconnected.com/armv9-what-is-the-big-deal-4528f20f78f3 
> <https://levelup.gitconnected.com/armv9-what-is-the-big-deal-4528f20f78f3>
> 
> It describes the new ARMv9 instruction set with SVE2 and how it compares to 
> Intel/AMD MMX/SSE/AVX and NEON/SVE.
> 
> Question is: would Apple adopt ARMv9 with SVE2 in a M2 for a future Mac Pro ? 
> ;-)
> 
> G.
> 
>> Le 26 avr. 2021 à 20:44, Jason Liu > <mailto:jason...@umich.edu>> a écrit :
>> 
>> Aha, hw.optional! That's useful, thanks Georges!
>> 
>> -- 
>> Jason Liu
>> 
>> 
>> On Mon, Apr 26, 2021 at 2:16 PM Georges Martin > <mailto:jrjsm...@gmail.com>> wrote:
>> $ sysctl hw.optional | grep -E 'neon|armv8'
>> hw.optional.neon: 1
>> hw.optional.neon_hpfp: 1
>> hw.optional.neon_fp16: 1
>> hw.optional.armv8_1_atomics: 1
>> hw.optional.armv8_crc32: 1
>> hw.optional.armv8_2_fhm: 1
>> hw.optional.armv8_2_sha512: 1
>> hw.optional.armv8_2_sha3: 1
>> 
>>> Le 26 avr. 2021 à 19:55, Jason Liu >> <mailto:jason...@umich.edu>> a écrit :
>>> 
>>> On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones >> <mailto:jon...@hep.phy.cam.ac.uk>> wrote:
>>> Thats not at all surprising as those instruction sets are very much 
>>> specific to X86_64 systems.
>>> 
>>> RISC processors, Arm, do have their own sets of SIMD instructions (e.g. 
>>> Neon), but they are entirely different to those on X86_64 machines. 
>>> 
>>> Whether or these are supported on Apple’s M1 processors I have no idea.
>>> 
>>> It looks like the M1 supports Neon SIMD instructions, but not SVE SIMD 
>>> (which I guess is supposed to be similar to AVX?):
>>> 
>>> https://discussions.apple.com/thread/252073619 
>>> <https://discussions.apple.com/thread/252073619>
>>> 
>>> -- 
>>> Jason Liu
>>> 
>>> 
>>> On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones >> <mailto:jon...@hep.phy.cam.ac.uk>> wrote:
>>> 
>>> 
>>>> On 26 Apr 2021, at 6:28 pm, Jason Liu >>> <mailto:jason...@umich.edu>> wrote:
>>>> 
>>>> Thanks Arno :)
>>>> 
>>>> I'm kind of surprised that the M1 doesn't seem to support any SSE or 
>>>> AVX
>>> 
>>> 
>>> Thats not at all surprising as those instruction sets are very much 
>>> specific to X86_64 systems.
>>> 
>>> RISC processors, Arm, do have their own sets of SIMD instructions (e.g. 
>>> Neon), but they are entirely different to those on X86_64 machines. 
>>> 
>>> Whether or these are supported on Apple’s M1 processors I have no idea.
>>> 
>>> Chris
>>> 
>>> 
>>> 
>>>> 
>>>> Does "sysctl machdep.cpu.features" return anything?
>>>> 
>>>> -- 
>>>> Jason Liu
>>>> 
>>>> 
>>>> On Mon, Apr 26, 2021 at 1:23 PM Arno Hautala >>> <mailto:a...@alum.wpi.edu>> wrote:
>>>> > On 26 Apr 2021, at 13:20, Jason Liu >>> > <mailto:jason...@umich.edu>> wrote:
>>>> > 
>>>> > sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse”
>>>> 
>>>> $ sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse”
>>>> machdep.cpu.brand_string: Apple M1
>>>> 
>>>> -- 
>>>> arno  s  hautala/-|   a...@alum.wpi.edu <mailto:a...@alum.wpi.edu>
>>>> 
>>>> pgp b2c9d448
>>>> 
>>>> 
>>> 
>> 
> 

Re: Apple notarisation process - LC_VERSION_MIN_MACOSX

[Saagar Jha]

Modern toolchains encode all that information in LC_BUILD_VERSION instead. Are 
you seeing one of those in the binary?

Saagar Jha

> On Mar 3, 2021, at 11:47, Ken Cunningham  
> wrote:
> 
> We really should have MacPorts libs be utilizable by projects that want to 
> put them on the App Store or use them any way they want to use them, if at 
> all possible.
> 
> The LC_VERSION_MIN_MACOSX issue is interesting.
> 
> There is lots of code in ld64 that appears designed to add this, and to fall 
> back on the DEPLOYMENT_TARGET to set it if it's not on the command line, and 
> to handle mismatches between the DEP target and the command line.
> 
> And yet I too can't see that chunk being added to dylibs either using xcode 
> ld64 or ld64-latest, whether building from source or downloaded from the 
> buildbot.
> 
> Something is amiss -- not sure it is a MacPorts issue, exactly, at this 
> moment. I await inspiration or insight...
> 
> K
> 
> 

Re: Rosetta2 and supported_archs

[Saagar Jha]

Rosetta translation is a per-process thing–you cannot use an Apple silicon 
library directly in an Intel process.

> On Jan 13, 2021, at 11:41, Craig Treleaven  wrote:
> 
>> On Jan 11, 2021, at 9:07 PM, Ryan Schmidt  wrote:
>> On Jan 11, 2021, at 19:38, Craig Treleaven wrote:
>> 
>>> A user with a new Apple Silicon-based Mac had a go at building mythtv.28.  
>>> It failed thusly:
>>> 
>>> --->  Computing dependencies for mythtv.28
>>> Error: Cannot install mythtv-core.28 for the arch 'x86_64' because
>>> Error: its dependency logrotate does not build for the required arch by
>>> default
>>> 
>>> https://trac.macports.org/ticket/62027
>>> 
>>> Given Rosetta2 on these systems, isn’t this a bogus error?  On M1 Macs, we 
>>> might warn that a dependency is being built for an arch different from that 
>>> of the main target but why should it be a fatal error?
>>> 
>>> What was done during the PPC to Intel transition? Did we try to force all 
>>> deps of a port to build with the same arch?
>> 
>> MacPorts 2.6.4 is supposed to support all possible arch demotions. On arm64, 
>> you're supposed to be able to run x86_64. On x86_64, you're supposed to be 
>> able to run i386. On i386, you're supposed to be able to run ppc. Some of 
>> this was already in place before, and it was enhanced for 2.6.4 in this 
>> commit:
>> 
>> https://github.com/macports/macports-base/commit/94f428eda6bd58b57c670c20fb2362ad7012310a
>> 
>> See this ticket:
>> 
>> https://trac.macports.org/ticket/61430
>> 
>> Installing a port for a demoted architecture would require its dependencies 
>> to be installed universal. MacPorts is trying to install the dependencies 
>> universal and is failing on logrotate because it does not have a universal 
>> variant. The solution is to add a universal variant to logrotate. 
>> Alternately, if logrotate does not install any libraries (and it looks like 
>> it does not), add "installs_libs no" to the logrotate portfile so that 
>> MacPorts knows it does not need to check its architecture. Or if logrotate 
>> does install libraries but whichever one of mythtv.28's dependencies depends 
>> on logrotate (looks like it's mythtv-core.28) doesn't link with any of its 
>> libraries, add "depends_skip_archcheck logrotate" to mythtv-core.28.
>> 
> 
> Am I incorrect that Rosetta2 does NOT require library dependencies to be the 
> same arch as the main program?  I thought I understood that an X86_64 program 
> could call an arm64 library (and vice versa) and that Rosetta2 would do the 
> right thing at run time.  If so, MacPorts does not have to supply a 
> mult-arch/universal library.
> 
> I understand that Rosetta2 is not going to around forever—maybe 2 to 5 years. 
>  However, if it does automatically translate libraries at run time, that 
> would permit quite a number of ports to work immediately.  No?
> 
> Craig
> 

Re: macOS Big Sur is version 11 not 11.0

[Saagar Jha]

The macOS SDK in Xcode 11.3 is MacOSX11.1.sdk.

> On Nov 18, 2020, at 18:12, Ryan Schmidt  wrote:
> 
> Based on the fact that Apple has released a beta of macOS Big Sur 11.1 
> already, we can now see that Big Sur should be referred to as version 11, not 
> 11.0 (and it would be reasonable to expect that next year's macOS will be 
> version 12).
> 
> If you are fixing any ports that had been coded to assume the macOS version 
> was always 10.x, be sure that you're not fixing it to simply accept versions 
> 10.x or 11.x. Instead, remove any assumption about the version number so that 
> you won't have to revisit the problem again every year.
> 
> When Josh released MacPorts 2.6.4 recently, he used the number 11.0 on the 
> Big Sur installer package. For the next version, we should use the version 
> number 11 to denote Big Sur.
> 
> I did the same when naming the Big Sur buildbot machines and will change them 
> from 11.0 to 11 soon.
> 
> Part of our decision to use "11.0" came from the way that Apple named the 
> SDK: MacOSX11.0.sdk. We will have to see if they change this to 
> MacOSX11.1.sdk in a future version of Xcode and the CLT. If they do, that 
> would represent a change from their previous strategy, and it would be a 
> problem for MacPorts because the SDK path gets baked into some ports. 
> Previously this was ok since the SDK path would stay the same for the life of 
> the OS version, but if it now changes during the life of the OS we may find 
> ourselves needing to rebuild some ports to update their SDK paths.
> 
> We may also need to adjust how MacPorts selects the SDK version and SDK path, 
> depending on whether Apple changes the SDK name.
> 

Re: Apple ARM binary codesign issue

[Saagar Jha]

As far as I understand, ad-hoc codesigning is not actually really meant to 
protect a file on disk because you can just ad-hoc sign again when you modify 
the file; instead it simplifies some of Apple’s own code because it removes the 
special case of a binary that doesn’t have a signature (which until now has had 
a number of quirks and extra checks throughout the operating system). A more 
cynical interpretation would be that Apple would like to flip the switch to 
“paid developer account-signed software only” at some point in the future, but 
every engineer has denied that this is the goal when asked so I guess that if 
this will happen it hopefully won’t be anytime soon.

I am still unsure why ld adds a signature but strip and install_name_tool don’t 
reapply an ad-hoc signature to a signed binary that they modify. This might be 
worth filing a feedback for.

> On Sep 22, 2020, at 15:24, Ken Cunningham  
> wrote:
> 
> 
> On 2020-09-22, at 12:58 PM, Ryan Schmidt wrote:
>> 
>> To me it seems unrealistic for Apple to suggest that an infinite number of 
>> open source projects, many of whose developers have never seen a Mac, should 
>> now add code to their build systems to codesign things on macOS. Apple made 
>> a point of stating during WWDC that they love open source software; imposing 
>> busy work on the open source community is not a good way to show that love.
> 
> As I read it, the linker automatically codesigns the binary when you link, 
> which is usually the final step in the process. So nobody has to change 
> anything there.
> 
> But if you later modify that final linked binary by stripping it (I guess ) 
> or changing the libraries around with install_name_tool (which I believe 
> MESON does to every single install :>)  then you invalidate the signature, as 
> you should IMHO.
> 
> I'm trying to imagine how Jeremy might prevent hackers from surreptitiously 
> modifying signed binaries with strip or install_name_tool (which is good) 
> while letting people modify signed binaries with strip or install_name_tool 
> without invalidating the signature  -- I don't immediately see how you can 
> have it both ways. But maybe Jeremy has some trick that works for this I 
> can't think of. 
> 
> I won't be surprised if the solution is that you have to resign them after 
> doing that, though.
> 
> Ken
> 

Re: Big Sur will be both 10.16 and 11.0

[Saagar Jha]

It’s Darwin 20:

$ uname -r
20.0.0

Presumably enough software is smart enough to understand how that works, 
although MacPorts has already had to add workarounds for this as you would 
expect…

> On Jul 21, 2020, at 17:17, Jason Liu  wrote:
> 
> Any word on whether Big Sur will still be considered to be Darwin 20?
> 
> -- 
> Jason Liu
> 
> 
> On Tue, Jul 21, 2020 at 5:57 PM Herby G  > wrote:
> Not sure if this has been posted previously or elsewhere, but as the title 
> says:
> 
> https://eclecticlight.co/2020/07/21/big-sur-is-both-10-16-and-11-0-its-official/
>  
> 

Re: Big Sur will be both 10.16 and 11.0

[Saagar Jha]

There is a /System/Library/CoreServices/SystemVersionCompat.plist. But you 
shouldn’t really care anyways, since you were using the system APIs to access 
the version number and not reading from the plist directly, right? ;)

> On Jul 21, 2020, at 19:09, Fred Wright  wrote:
> 
> 
> On Tue, 21 Jul 2020, Herby G wrote:
> 
>> Not sure if this has been posted previously or elsewhere, but as the title
>> says:
>> https://eclecticlight.co/2020/07/21/big-sur-is-both-10-16-and-11-0-its-official/
> 
> This is bizarre.  The numeric version appears in the system itself, in 
> /System/Library/CoreServices/SystemVersion.plist.  Will SYSTEM_VERSION_COMPAT 
> somehow mangle the file contents?
> 
> Maybe it should have been code-named "Spinal Tap". :-)
> 
> Fred Wright

macOS Big Sur/Universal App Quick Start Disclosure List

[Saagar Jha]

Hi,

Every year Apple previews a new version of macOS at WWDC–some of you on the 
list are likely disclosed on the new betas and testing MacPorts on them 
already. If you are and would be interesting in advertising yourself as such, 
please consider adding yourself to the FAQ entry 
<https://trac.macports.org/wiki/FAQ#prerelease> so that you can be a resource 
to other contributors working on getting MacPorts ready for the new OS prior to 
its public release. If you are part of the Universal App Quick Start Program 
and will be testing on Apple silicon, you may wish to mention this as well; 
however please do keep in mind the additional restrictions in its NDA.

Regards,
Saagar Jha

Re: Call for designers for our ports website

[Saagar Jha]

I believe the lack of change there is almost certainly a matter of the 
project’s personal stance rather than “nobody having a problem with it”. In 
fact, after the change was merged in there was a fairly long discussion about 
first disclosing that there were analytics collected at all (which did 
eventually get implemented) and then switching off of Google Analytics or 
making it opt-in, which weren’t. Actually, there were multiple discussions but 
they like the original were generally closed as “WONTFIX” and this has been the 
policy to this day.

Personally, I would be fairly disappointed if MacPorts went opt-in as such 
policies suffer from statistical issues in addition to the obvious 
privacy-related ones.

Saagar Jha

> On Jun 12, 2020, at 16:48, Ken Cunningham  
> wrote:
> 
> Just FYI Homebrew has always been opt-out for stats. Nobody seems to have a 
> problem with that sufficient to make them change that policy.
> 
> We'll never know if that is why they seem to have 10 x the users on their 
> stats page.
> 
> K

Re: [MacPorts] #60590: Macports mirrors are down?

[Saagar Jha]

Though it’s not the default, you can actually tell MacPorts to use Git to sync 
your ports, at least; see the third part of section 2.3.3 in the guide 
<https://guide.macports.org/#installing.macports.git>.

Saagar Jha

> On Jun 5, 2020, at 16:05, Christopher Chavez  wrote:
> 
> On 6/5/2020 12:32 PM, Clemens Lang wrote:
>> or switch to a different source, e.g. git
> As a user, I've thought it would be great if MacPorts used git syncing
> by default. I have used git-over-https since 2016 when I was constrained
> to ports 80/443 by a university campus network. I had found it also
> avoided needlessly writing hundreds of MB to the SSD when updating the
> ports tree. If GitHub outages are a concern, then I would think falling
> back to a mirror hosted by GitLab/SourceForge/Bitbucket/etc. should be
> possible.
> 
> Christopher A. Chavez

Re: OpenBLAS fixed in Xcode 11.2

[Saagar Jha]

Does this <https://trac.macports.org/ticket/58776#comment:18> match your stack 
trace?

Saagar Jha

> On Nov 1, 2019, at 02:09, Jack Howarth  wrote:
> 
> The issues in gmp might not be fixed in Xcode 11.2. The exact issue isn't 
> listed in the current gmp Portfile but 'sudo port test gmp' shows...
> 
> PASS: bit
> ../../test-driver: line 107: 63647 Segmentation fault: 11  "$@" > $log_file 
> 2>&1
> FAIL: t-powm
> 
>  Jack
> 
> On Thu, Oct 31, 2019 at 10:51 PM Ryan Schmidt  <mailto:ryandes...@macports.org>> wrote:
> 
> 
> On Oct 31, 2019, at 20:34, Joshua Root wrote:
> 
> > On 2019-11-1 11:49 , Jack Howarth wrote:
> >>  Although it wasn't fixed in Xcode 11.2 beta 2, the -fcheck-stack
> >> issues with OpenBLAS's test suite are fixed in the final Xcode 11.2
> >> released today.
> >>  Jack
> > 
> > Good to hear. Ryan, could you please update Xcode on the Catalina
> > buildslave ASAP?
> 
> Got it. It will be easier to wait until the portbuilder queue is empty. I'll 
> try to pause it and update it before it goes on to the next batch.

Re: gcc/g++ failures after xcode11 update

[Saagar Jha]

Saagar Jha

> On Sep 24, 2019, at 01:56, Christopher Jones  wrote:
> 
> Hi,
> 
> Note the error is not specific to gcc. Macports clang has the same problem.
> 
>  > clang++-mp-8.0 -O3 ./main.cpp 
> In file included from ./main.cpp:1:
> In file included from 
> /opt/local/libexec/llvm-8.0/bin/../include/c++/v1/iostream:38:
> In file included from 
> /opt/local/libexec/llvm-8.0/bin/../include/c++/v1/ios:215:
> In file included from 
> /opt/local/libexec/llvm-8.0/bin/../include/c++/v1/iosfwd:96:
> /opt/local/libexec/llvm-8.0/bin/../include/c++/v1/wchar.h:119:15: fatal 
> error: 'wchar.h' file not found
> #include_next 
>   ^
> 1 error generated.
> 
> The issue is both macports clang and gcc rely (by default) on /usr/include 
> being present, and this is no longer present by default with Xcode 11, and 
> even if you previously had it was likely wiped out with the last update.
> 
> Note, with 10.14 at least (*) you can add back /usr/include by following the 
> instructions in 
> 
> https://apple.stackexchange.com/questions/337940/why-is-usr-include-missing-i-have-xcode-and-command-line-tools-installed-moja
>  
> <https://apple.stackexchange.com/questions/337940/why-is-usr-include-missing-i-have-xcode-and-command-line-tools-installed-moja>
> 
> or, as you say below, explicitly giving the include path in some way, such as 
> the suggestion below.
> 
> cheers Chris
> 
> (*) The package to add back /usr/include currently does not exist in 10.15 
> beta, so unless it reappears come final release this is going to be more of a 
> problem there….

This package is not going to be coming back: 
https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035623
 
<https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035623>

>> On 24 Sep 2019, at 7:09 am, Joshua Root > <mailto:j...@macports.org>> wrote:
>> 
>> On 2019-9-24 15:31 , Mihir Luthra wrote:
>>> Hi,
>>> 
>>> After the xcode update, there have been many question on stackoverflow
>>> regarding gcc and g++ linking fails. Any ideas on what can be done?
>>> 
>>> https://stackoverflow.com/questions/58072318/cannot-link-any-c-program-with-gcc-on-mac-mojave
>>>  
>>> <https://stackoverflow.com/questions/58072318/cannot-link-any-c-program-with-gcc-on-mac-mojave>
>>> 
>>> https://stackoverflow.com/questions/58073301/linker-error-when-trying-to-use-lzma-in-boostiostreams-from-macports
>>> 
>>> https://stackoverflow.com/questions/58071057/macports-g-fails-to-find-headers-after-recent-xcode-update
>>> 
>>> Mihir
>> 
>> Does passing -isysroot`xcrun --show-sdk-path` to the compiler not work?
>> 
>> While that should make it find headers again, I don't know how many bugs
>> it will uncover. I'm not sure how we would make this work out of the box
>> for everyone, since AIUI, gcc needs to apply different fixups to the
>> system headers depending on the SDK version and does so at build time.
>> So Apple taking away not only the SDK corresponding to the current OS
>> version but also /usr/include is quite problematic. We can ship a gcc
>> supporting the 10.14 SDK or the 10.15 SDK, but not both.
>> 
>> Rebuilding gcc from source would also work.
>> 
>> - Josh
> 

Reporting build failures on macOS Catalina

[Saagar Jha]

I’m migrating my ports to macOS 10.15 Catalina, and as expected some ports fail 
to build on it. Is anyone on the MacPorts team disclosed on the new OS, or a 
place where I can report issues of this sort?

Regards,
Saagar Jha

Re: PRs

[Saagar Jha]

If you have commit access to MacPorts and the pull request author allows it 
<https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork>,
 you can rebase their fork's branch (for example, with git commit --amend 
--no-edit) and trigger a new CI build. I’d ask before doing this though, since 
it’ll kill their Git history.

Saagar Jha

> On May 19, 2019, at 14:21, Rainer Müller  wrote:
> 
> On 19.05.19 20:45, Christopher Chavez wrote:
>> On 5/19/2019 1:40 PM, Mark Anderson wrote:
>>> making it possible to re-run CI from GitHub commits.
>> 
>> As a PR submitter, one workaround I'm aware of is rebasing the PR
>> commit(s). I'm not sure that's something members can/should try though,
>> so a better solution is probably still preferred…
> 
> I would not consider this a workaround, but the best choice we have.
> 
> Re-running the build with the exact same revision would not be that
> useful. Without changes to the source, I would expect the same result.
> As the ports tree and therefore the dependencies have usually advanced
> since the PR was submitted, it makes sense to do a rebase first before
> attempting another build.
> 
> In theory it is possible to restart a build on the Travis CI web
> interface with the same revision, but I am not even sure if all members
> have access to that?
> 
> Rainer

Re: notarization vs MacPorts apps

[Saagar Jha]

Comments inline again.

Saagar Jha

> On Apr 13, 2019, at 22:54, Mojca Miklavec  wrote:
> 
> Can you please reply to the list or is there a good reason not to?

Thanks for the reminder, I’ve re-CC’d macports-dev.

> I'm not saying we should notarize the apps because we have to, but it would 
> not hurt to do some additional security checks in case our compiler gets 
> compromised in some way or if we happen to build some suspicios software / 
> malware.

This check is happening on MacPorts’s servers, right (as opposed to locally on 
user’s computers when they build from source)? You’re building ports and 
signing them with a hypothetical “MacPorts Project” developer certificate, then 
submitting them to be notarized and making the app bundle with a notarization 
ticket stapled to it available to download by users?

> Mojca
> 
> V ned., 14. apr. 2019 02:09 je oseba Saagar Jha  <mailto:saa...@saagarjha.com>> napisala:
> 
> Saagar Jha
> 
>> On Apr 13, 2019, at 08:12, Mojca Miklavec > <mailto:mo...@macports.org>> wrote:
>> 
>> Hi,
>> 
>> On Sat, 13 Apr 2019 at 08:47, Joshua Root  wrote:
>>> On 2019-4-13 07:57 , Jack Howarth wrote:
>>>> What will be the situation with 10.14.5 and its enforcement of
>>>> notarization for Applications and kernel extensions for MacPorts? In
>>>> particular, will the new notarization requirement limit users to the
>>>> MacPorts build machine copies of such packages which have applications
>>>> rather than being able to build those packages locally?
>>>>Jack
>>> 
>>> The MacPorts installer pkg will need to be submitted, but I don't think
>>> much else will change. Using MacPorts-built kernel extensions is already
>>> impossible because of signing requirements (we don't have a kext signing
>>> certificate and I don't think we qualify for one.)
>>> 
>>> For general apps, Gatekeeper doesn't prevent running locally built ones
>>> due to them being unsigned, and I gather than notarization is only
>>> required in the same circumstances as signing.
>> 
>> The developer of MacTeX (which is basically a collection of a large
>> number of command-line tools + really small set of GUI apps) started
>> researching this in more detail. In the past it would have been
>> sufficient to only sign the whole package (dmg) once. Now he needs to
>> take care of every single binary inside the package. From what I
>> understood it can be automated, some of the binaries require
>> additional privileges (I assume that luajittex requires some kind of
>> "JIT" privileges etc). There were some issues with ghostscript which
>> needs to be additionally hardened etc.
>> 
>> I assume that if I use rsync to get the binaries as opposed to
>> fetching them via web browser, they might work OK.
> 
> From my understanding of the WWDC Session 702 from last year 
> <https://developer.apple.com/videos/play/wwdc2018/702/>, if your app is 
> signed (which seems the case based on your response) and is notarized, then 
> it will need to opt into the Hardened Runtime and as add one of the 
> entitlements (com.apple.security.cs.allow-jit, probably) to give it the 
> ability to function correctly. However, if MacPorts is fetching or building 
> the binary itself then I don’t see why notarization is required at all, since 
> I think it should be possible to control whether the com.apple.quarantine 
> xattr exists and hence whether GateKeeper actually checks for a valid 
> notarization ticket stapled to the binary/package/whatever (unless the check 
> is actually performed at every launch, regardless of the extended attribute’s 
> existence?).
> 
>> I don't have a payed developer account, so I probably cannot test
>> anything. But I assume there might be a way to individually notarize
>> individual binaries inside MacPorts packages. While this might not be
>> needed at this very moment, it might be that by putting a certificate
>> on the buildslave, we could:
>> - sign the debugger (which currently needs additional steps to work at all)
> 
> I don’t think this actually requires a paid account–signing with any valid 
> certificate in your keychain works IIRC.
> 
>> - get an additional automated safety check for any malware that might
>> have creeped into the source code unnoticed (with tens of thousands of
>> packages that's not impossible), which cannot hurt
>> 
>> I don't know if a certificate can be issues to a project instead of
>> private person and to what extent one can secure it on the servers.
> 
> I’m pretty sure you can get a standard ($99) developer account membership for 
> organizations.
> 
>> These are just some random ideas, it would be nice to get a more
>> realistic response from someone who's more knowledgable in this area.
>> 
>> Mojca
> 

Re: notarization vs MacPorts apps

[Saagar Jha]

MacPorts actually does sign some apps: for example, HexFiend seems to be ad-hoc 
signed as a result of the Xcode build process. I don’t think GateKeeper 
actually comes into play here because the resulting binary never has the 
com.apple.quarantine xattr set. I do run with SIP and GateKeeper disabled 
normally, though, and Apple hasn’t released a stable build of macOS 10.14.5 
yet, so I’d take my testing with a grain of salt ;)

Regards,
Saagar Jha

> On Apr 12, 2019, at 23:47, Joshua Root  wrote:
> 
> On 2019-4-13 07:57 , Jack Howarth wrote:
>>  What will be the situation with 10.14.5 and its enforcement of
>> notarization for Applications and kernel extensions for MacPorts? In
>> particular, will the new notarization requirement limit users to the
>> MacPorts build machine copies of such packages which have applications
>> rather than being able to build those packages locally?
>> Jack
> 
> The MacPorts installer pkg will need to be submitted, but I don't think
> much else will change. Using MacPorts-built kernel extensions is already
> impossible because of signing requirements (we don't have a kext signing
> certificate and I don't think we qualify for one.)
> 
> For general apps, Gatekeeper doesn't prevent running locally built ones
> due to them being unsigned, and I gather than notarization is only
> required in the same circumstances as signing. (It would be incredibly
> inconvenient for developers to test anything if this were not the case.)
> 
> - Josh

Re: about port:binutils (and port:gdb)

[Saagar Jha]

Last I checked, GDB is pretty much broken on macOS because it frequently hangs 
when launching processes. However, it did a decent job of reading in debugging 
symbols in the rare cases where it did work.

Regards,
Saagar Jha

> On Feb 17, 2019, at 00:57, René J.V. Bertin  wrote:
> 
> Hi,
> 
> How appropriate is the note attached to port:binutils nowadays?
> 
> I installed it recently to test GNU ar in combination with software that 
> tries to use MRI scripts to generate and modify static libraries. A priori 
> every command is prefixed and I haven't noticed any problems with other ports 
> ... yet.
> If there are known problems with certain components, why not put installation 
> of those in a variant? /methinks that it can't hurt to have "standard" 
> versions of utilities like ar...
> 
> The port does conflict with port:gdb, something I had already observed in my 
> Linux adaptations of the two, something I solved in port:gdb with a somewhat 
> hard-handed
> 
> {{{
># we install a certain number of files also installed by port:binutils.
># Avoid existential problems: delete our copy and require binutils so
># everything is complete.
>depends_run-append \
>port:binutils
> 
> post-destroot {
># remove items already installed by port:binutils
>file delete -force ${destroot}${prefix}/share/locale
>file delete -force ${destroot}${prefix}/share/info/bfd.info
>file delete -force ${destroot}${prefix}/lib/libopcodes.la
>file delete -force ${destroot}${prefix}/lib/libopcodes.a
>file delete -force ${destroot}${prefix}/lib/libbfd.la
>file delete -force ${destroot}${prefix}/lib/libbfd.a
>file delete -force {*}[glob -directory ${destroot}${prefix}/include/ *.h]
> }
> }}}
> 
> I haven't (yet) checked this on Mac because my experience with gdb (v7) is 
> that it only works with very simple software and won't detect symbols from 
> libraries (if they don't have the corresponding .dSYM resource installed 
> maybe). Has that improved?
> 
> R.