Re: info

[Ryan Schmidt]

On Dec 4, 2020, at 21:09, Richard L. Hamilton wrote:

> If there was a port-level view of how the buildbots are doing (maybe there is 
> and I haven't found it?) that might come close.

There is no such feature at this time.

> But from what little I figured out looking at the buildbot pages, it looks 
> like something is failing in an early dependency for arm64.

That is very likely. There are a zillion ports failing to build on Apple 
Silicon and macOS 11 and Xcode 12 for numerous reasons as previously discussed. 
Help wanted.

Re: info

[Ryan Schmidt]

On Dec 4, 2020, at 21:32, Ken Cunningham wrote:

> On 2020-12-04 6:56 p.m., Ryan Schmidt wrote:
>> The reason for this is that ports.macports.org only shows status for ports 
>> that were built directly. It does not show status for ports that were built 
>> indirectly
> 
> 
> Hah! We should certainly prioritize that deficiency for fixing... I had no 
> idea that it worked like that.

It's not easy to fix with the way it currently works. Currently 
ports.macports.org takes the status information from each build on 
build.macports.org. Each build is for a port. But if that port's dependencies 
weren't already built, they will be built as well, but there is no way in the 
buildbot API to ask which of the dependencies needed to be built (vs. which 
were already built) nor whether that build succeeded or failed.

This is probably a topic we should revisit after we have upgraded to buildbot 
2. Trying to fix it now would just make more work for us as we try to convert 
to buildbot 2.

Re: info

[Ken Cunningham]

On 2020-12-04 6:56 p.m., Ryan Schmidt wrote:
The reason for this is that ports.macports.org only shows status for 
ports that were built directly. It does not show status for ports that 
were built indirectly



Hah! We should certainly prioritize that deficiency for fixing... I had 
no idea that it worked like that.



K

Re: info

[Richard L. Hamilton]

If there was a port-level view of how the buildbots are doing (maybe there is 
and I haven't found it?) that might come close.

But from what little I figured out looking at the buildbot pages, it looks like 
something is failing in an early dependency for arm64.

I do not have any arm64 (Apple Silicon) Macs; but I did just set up a Big Sur 
(x86_64) VM, for familiarity's sake; I'm not ready to move any bare metal over 
yet. So far, I don't like the look much, but what little I installed (gettext, 
vim, openmotif, along with whatever they depend on) was  painless enough. At 
least minimally, the macosforge Xquartz seems to work, so likely the MacPorts 
xorg-server does too.

> On Dec 4, 2020, at 21:57, Ryan Schmidt  wrote:
> 
> On Dec 4, 2020, at 16:00, James Secan wrote:
> 
>> I think a large number of us are very interested in the status of ports 
>> vis-a-vis both Apple Silicon (M1) and Big Sur.  Some sort of simple 
>> red-yellow-green status board for ports that have been checked would be very 
>> useful.
> 
> Unfortunately that information is not reliably available for now.
> 
>> Verified support for the MacPorts codes I use regularly is a major check-box 
>> on my buy-now list for a new M1 machine.  I suspect this is more easily 
>> visualized than actually produced and maintained.
> 
> Feel free to give us a list of the ports that are important to you and we can 
> try to find out and/or schedule builds of those ports on our Apple Silicon 
> build machine and see what happens.
> 
> 

Re: info

[Ryan Schmidt]

On Dec 4, 2020, at 16:00, James Secan wrote:

> I think a large number of us are very interested in the status of ports 
> vis-a-vis both Apple Silicon (M1) and Big Sur.  Some sort of simple 
> red-yellow-green status board for ports that have been checked would be very 
> useful.

Unfortunately that information is not reliably available for now.

> Verified support for the MacPorts codes I use regularly is a major check-box 
> on my buy-now list for a new M1 machine.  I suspect this is more easily 
> visualized than actually produced and maintained.

Feel free to give us a list of the ports that are important to you and we can 
try to find out and/or schedule builds of those ports on our Apple Silicon 
build machine and see what happens.

Re: info

[Ryan Schmidt]

On Dec 4, 2020, at 15:50, Mojca Miklavec wrote:
> 
> You should be able to install MacPorts and many ports. But you should
> not be surprised if you hit some that will refuse to build and you may
> need to wait for upstream to fix the issue (or try to fix it yourself
> and submit a patch or find someone else capable of fixing it ...).
> 
> You brought up an interesting point though, we should probably publish
> some official statement about arm support on our main website.

I don't think our statement would be very different from our stance with any 
other new Apple product release, be it a new OS version or a new Xcode version: 
probably some things are broken as a result and need to be fixed as the 
problems are identified.


> A lot of relatively simple, well-written software with a well-written
> build system should often work out of the box.

Yes and no. Autotools is one of the oldest and still most widely used build 
systems, yet until automake 1.16.3 which was just released a couple weeks ago 
it misidentified Apple Silicon systems, so any port whose build system was 
built with automake 1.16.2 or earlier may need to be autoreconfed to build 
properly on Apple Silicon. libtool, which many autotools projects use, 
misidentifies macOS 11 and later. Developers have not yet acknowledged the 
problem, much less accepted our patch, so any software using autotools and 
libtool and relying on undefined symbols being resolved at runtime will fail to 
build on macOS 11 and will need to be either autoreconfed or patched or have 
the deployment target set to 10.16. And then there's the implicit function 
declaration problem biting lots of projects as of Xcode 12. Apple Silicon 
systems are affected by all three issues, so in my opinion we have never had a 
time in MacPorts history when more ports were broken at one time on a new 
system than we have now. My recommendation continues to be if you want MacPorts 
to work smoothly then do not upgrade to Xcode 12, do not upgrade to macOS 11, 
and do not buy an Apple Silicon Mac. Or, if you do upgrade, be prepared to help 
us resolve the issues you will inevitably encounter. Filing bug reports about 
what's broken is a great start but we have a zillion open tickets already. What 
we really need is for people to read those tickets and submit pull requests to 
fix them properly.


> But a lot of software may either have some hard-coded assumptions in
> either their build system or the source, it may require some
> intel-specific intrinsics, or it may depend on some complex
> third-party library that doesn't compile. Apple also likes to increase
> security standards each year which may break many ports in various
> ways.

Yup, there are limitless ways that individual software packages could have done 
it wrong in a way that doesn't work on Apple Silicon, above and beyond the big 
three above.


> If you have your favourite port, you can quickly check the build
> results on, say,
>https://ports.macports.org/port/wget/stats
> and check for either green port status or some reported installations
> on arm64, check for open tickets etc. Keep in mind that many port
> builds haven't been attempted yet.
> 
> (I also see that some builds like wget were successful, but missing on
> the list, while some like youtube-dl are redirected to the x86_64
> builder and also don't end up on that list.)

On Dec 4, 2020, at 17:34, Ken Cunningham wrote:

> Go here  and enter your favourite port.
> 
> eg:
> 
> 
> 
> by the way, a “grey” box means untested as yet, not failing.

But please understand that the information on ports.macports.org is not 
authoritative or complete.

A grey box does not mean untested. It means no information is available. Grey 
means we might have built the port and it succeeded or failed, or we might not 
have built the port yet.

Similarly, green or red does not guarantee that the most recent build succeeded 
or failed. It just means that the most recent build *for which data was sent to 
ports.macports.org* succeeded or failed. There may have been a more recent 
build with a different status but which is not shown on the web site.

The reason for this is that ports.macports.org only shows status for ports that 
were built directly. It does not show status for ports that were built 
indirectly, for example as a dependency of another port. This situation is most 
likely to arise when we first start building packages for a particular os/arch, 
and happens less often after everything has been built once.

Authoritative information is the directory listings of 
https://packages.macports.org. If there is a file, the build succeeded. If 
there is not a file, then the build either did not succeed or was not attempted 
or did succeed but the file is not legally distributable.

Re: info

[Ryan Schmidt]

On Dec 4, 2020, at 09:18, Alejandro Imass wrote:

> quick question: is Apple using more or less the same stack and toolchain i.e. 
> Mach + FBSD backbone and LLVM, etc. ? or has something very important changed 
> for Apple silicon?

The very important thing that has changed is that Apple Silicon is arm64 
architecture. Many existing ports have build systems that assume Macs don't use 
arm64 architecture and fail to build as a result. These will need to be 
identified and fixed, hopefully in consultation with the developers of the 
software.

The OS kernel is still Mach, the userland software is still largely BSD based, 
LLVM is still used as the compiler, on macOS 11 on any architecture, just as it 
was on macOS 10.x for a long time.

I hear the process of booting the operating system is quite different on Apple 
Silicon: it's more similar to how iOS devices boot. But these are details that 
do not affect us in MacPorts since we deal only in user installable software.

Re: info

[Ken Cunningham]

>  Some sort of simple red-yellow-green status board for ports that have been 
> checked would be very useful.

You are in luck!

Go here  and enter your favourite port.

eg:



by the way, a “grey” box means untested as yet, not failing.

K

Re: info

[James Secan]

I think a large number of us are very interested in the status of ports 
vis-a-vis both Apple Silicon (M1) and Big Sur.  Some sort of simple 
red-yellow-green status board for ports that have been checked would be very 
useful.  Verified support for the MacPorts codes I use regularly is a major 
check-box on my buy-now list for a new M1 machine.  I suspect this is more 
easily visualized than actually produced and maintained.

Jim
3222 NE 89th St
Seattle, WA 98115
(206) 430-0109

> On Dec 4, 2020, at 1:50 PM, Mojca Miklavec  wrote:
> 
> On Fri, 27 Nov 2020 at 16:27, Giovanni Cantele wrote:
>> 
>> Dear All,.
>> 
>> I’m searching the web but I cannot find any response to the following 
>> question:
>> 
>> is there any ongoing project for porting the whole macports staff on the new 
>> Apple silicon architecture?
> 
> There is no "special ongoing project". There are volunteer owners of
> Apple silicon trying to fix the bugs they encounter.
> MacPorts itself should work, lots of ports work, some complex software
> requires non-trivial patches from upstream.
> 
>> What happens to those who extensively make use of macports and have bought 
>> the recent released MacBook Pro running on the new processors?
> 
> You should be able to install MacPorts and many ports. But you should
> not be surprised if you hit some that will refuse to build and you may
> need to wait for upstream to fix the issue (or try to fix it yourself
> and submit a patch or find someone else capable of fixing it ...).
> 
> You brought up an interesting point though, we should probably publish
> some official statement about arm support on our main website.
> 
> On Fri, 4 Dec 2020 at 16:19, Alejandro Imass wrote:
>> 
>> What you are saying suggests that nothing major has changed except the LLVM 
>> target to arm64, is this correct?
> 
> Disclaimer: I don't have any experience with an arm-based mac.
> 
> As far as MacPorts is concerned, I would say that indeed "almost
> nothing major" has changed in principle (other than the processor,
> which is ... well, a really major change).
> 
> A lot of relatively simple, well-written software with a well-written
> build system should often work out of the box.
> 
> But a lot of software may either have some hard-coded assumptions in
> either their build system or the source, it may require some
> intel-specific intrinsics, or it may depend on some complex
> third-party library that doesn't compile. Apple also likes to increase
> security standards each year which may break many ports in various
> ways.
> 
> If you have your favourite port, you can quickly check the build
> results on, say,
>https://ports.macports.org/port/wget/stats
> and check for either green port status or some reported installations
> on arm64, check for open tickets etc. Keep in mind that many port
> builds haven't been attempted yet.
> 
> (I also see that some builds like wget were successful, but missing on
> the list, while some like youtube-dl are redirected to the x86_64
> builder and also don't end up on that list.)
> 
> Mojca

Re: info

[Mojca Miklavec]

On Fri, 27 Nov 2020 at 16:27, Giovanni Cantele wrote:
>
> Dear All,.
>
> I’m searching the web but I cannot find any response to the following 
> question:
>
> is there any ongoing project for porting the whole macports staff on the new 
> Apple silicon architecture?

There is no "special ongoing project". There are volunteer owners of
Apple silicon trying to fix the bugs they encounter.
MacPorts itself should work, lots of ports work, some complex software
requires non-trivial patches from upstream.

> What happens to those who extensively make use of macports and have bought 
> the recent released MacBook Pro running on the new processors?

You should be able to install MacPorts and many ports. But you should
not be surprised if you hit some that will refuse to build and you may
need to wait for upstream to fix the issue (or try to fix it yourself
and submit a patch or find someone else capable of fixing it ...).

You brought up an interesting point though, we should probably publish
some official statement about arm support on our main website.

On Fri, 4 Dec 2020 at 16:19, Alejandro Imass wrote:
>
> What you are saying suggests that nothing major has changed except the LLVM 
> target to arm64, is this correct?

Disclaimer: I don't have any experience with an arm-based mac.

As far as MacPorts is concerned, I would say that indeed "almost
nothing major" has changed in principle (other than the processor,
which is ... well, a really major change).

A lot of relatively simple, well-written software with a well-written
build system should often work out of the box.

But a lot of software may either have some hard-coded assumptions in
either their build system or the source, it may require some
intel-specific intrinsics, or it may depend on some complex
third-party library that doesn't compile. Apple also likes to increase
security standards each year which may break many ports in various
ways.

If you have your favourite port, you can quickly check the build
results on, say,
https://ports.macports.org/port/wget/stats
and check for either green port status or some reported installations
on arm64, check for open tickets etc. Keep in mind that many port
builds haven't been attempted yet.

(I also see that some builds like wget were successful, but missing on
the list, while some like youtube-dl are redirected to the x86_64
builder and also don't end up on that list.)

Mojca

Re: info

[Alejandro Imass]

On Fri, Nov 27, 2020 at 11:01 AM Artem Loenko via macports-users <
macports-users@lists.macports.org> wrote:

> Hello,
>
> I am in the same boat (and have switched from HomeBrew to MacPorts a few
> weeks ago, so, maybe I am wrong).
>
> MacPorts as a tool works just fine on Macs with Apple Silicon, but many
> ports are “broken” and have to be fixed. Most of them just do not compile
> for `arm64`, and it is not MacPorts fault, some - do
>

quick question: is Apple using more or less the same stack and toolchain
i.e. Mach + FBSD backbone and LLVM, etc. ? or has something very important
changed for Apple silicon?

What you are saying suggests that nothing major has changed except the LLVM
target to arm64, is this correct?

TIA, and pardon my ignorance on the matter.

Best,

-- 
Alex

Re: MacPorts on Apple Silicon Macs (was: Re: info)

[Ryan Schmidt]

On Nov 28, 2020, at 08:32, Joshua Root wrote:

> Ryan Schmidt wrote:
>> We have tens of thousands of ports in MacPorts and it's not always clear to 
>> us which ones people use.
> 
> That at least can be easily remedied by installing the mpstats port. You
> should still file a ticket if something is broken (and there isn't an
> existing ticket) of course.
> 
> The stats naturally have their limitations, but mpstats is an easy and
> effective way of letting us know which ports matter to you.

Yes. But when I am looking at bug reports on Trac or build failures on the 
Buildbot, there is no easy way for me to see the associated stats for that 
port, and even if I look up the stats manually, I must admit I'm not really 
sure how to interpret the results. An issue doesn't always affect every user 
who installs a port.

On the other hand, if we get a bug report that has 20 people Cc'd on it, that 
makes it clear that many people are experiencing that specific issue and that 
it would help many people if we fix it.

Re: MacPorts on Apple Silicon Macs (was: Re: info)

[Joshua Root]

Ryan Schmidt wrote:
> We have tens of thousands of ports in MacPorts and it's not always clear to 
> us which ones people use.

That at least can be easily remedied by installing the mpstats port. You
should still file a ticket if something is broken (and there isn't an
existing ticket) of course.

The stats naturally have their limitations, but mpstats is an easy and
effective way of letting us know which ports matter to you.

- Josh

Re: MacPorts on Apple Silicon Macs (was: Re: info)

[Ryan Schmidt]

On Nov 27, 2020, at 09:15, Giovanni Cantele wrote:

> is there any ongoing project for porting the whole macports staff on the new 
> Apple silicon architecture?
> What happens to those who extensively make use of macports and have bought 
> the recent released MacBook Pro running on the new processors?

Right now we have probably thousands ports that are broken due to three 
different issues, and Apple Silicon machines are affected by all three issues:

1. Xcode 12 makes implicit declaration of functions an error instead of a 
warning. Apple made this change in order to prevent problems that would 
otherwise occur on Apple Silicon machines but it will take a long time before 
either we or the developers of the software patch all of our ports to 
accommodate this change. Instructions for developers wishing to help fix these 
problems are here: 
https://lists.macports.org/pipermail/macports-dev/2020-November/042647.html and 
in my follow-up to that message.

2. macOS Big Sur is version 11, and a lot of software did not anticipate that 
such a version number of macOS would ever exist and fail to build as a result. 
All software that uses autotools and libtool and relies on the use of symbols 
that are undefined at build time is affected, for example, but there are 
limitless other ways that developers could have and did write their software so 
that it would only work on macOS 10.x. The libtool issue may have a simple fix 
that we could use: https://trac.macports.org/ticket/61584 but we have not 
employed that yet.

3. Many ports fail to build on Apple Silicon machines for a variety of other 
reasons which we will need to investigate and fix as we encounter them one by 
one.

MacPorts volunteers are working with the developers of the affected software to 
fix it and fix our ports as time permits. You or anyone interested can help by 
reporting bugs, either to us in the case of ancient software or to the 
developers of the software if it is still being developed, or by fixing the 
problem and sending us or the developers the fix. We have tens of thousands of 
ports in MacPorts and it's not always clear to us which ones people use. By 
filing a bug report about a port or Cc'ing yourself on an existing report about 
a problem, you can help us identify which problems we need to focus on fixing.

I previously advised that if you want to avoid issues with MacPorts ports while 
we try to work this all out, don't upgrade to Big Sur yet. See 
https://lists.macports.org/pipermail/macports-users/2020-November/048940.html. 
I would also advise not upgrading to Xcode 12 yet on Catalina. Since Apple 
Silicon Macs require Big Sur and Xcode 12.2, my recommendation extends to not 
buying an Apple Silicon Mac yet, despite how fantastic I'm sure they will be.

Re: info

[Artem Loenko via macports-users]

Hello,

I am in the same boat (and have switched from HomeBrew to MacPorts a few weeks 
ago, so, maybe I am wrong). 

MacPorts as a tool works just fine on Macs with Apple Silicon, but many ports 
are “broken” and have to be fixed. Most of them just do not compile for 
`arm64`, and it is not MacPorts fault, some - do not compile due to MacPorts 
configuration (rare cases). So far, as far as I understand, the best course of 
action is to report all broken ports to the MacPorts tracker and to developers 
of the tool that does not compile for `arm64` arch. For example, this is a 
ticket about broken Neovim (https://trac.macports.org/ticket/61550 
), but it does not compile due to 
issues on the NeoVim side (that’s why I started a discussion with them 
(https://github.com/neovim/neovim/issues/13399 
).

I use the following `make` target to install MacPorts on Apple Silicon or you 
can just download the package from the site 
(https://www.macports.org/install.php ):

```
macports: ## Install/Upgrade MacPorts
ifeq (, $(shell which port))
$(eval TEMP_PKG := $(shell mktemp -t macports).pkg)
curl --silent --output $(TEMP_PKG) --remote-name 
https://distfiles.macports.org/MacPorts/MacPorts-2.6.4_1-11-BigSur.pkg 
sudo installer -pkg $(TEMP_PKG) -target /
rm -Rf $(TEMP_PKG)
endif
sudo port selfupdate
```

Most of the tools I use just work:

```
The following ports are currently installed:
  autoconf @2.69_5 (active)
  automake @1.16.3_0 (active)
  carthage @0.35.0_0 (active)
  fish @3.1.2_0 (active)
  fzf @0.24.3_0 (active)
  gettext @0.19.8.1_2 (active)
  icu @67.1_2 (active)
  ninja @1.10.1_0 (active)
  tig @2.5.1_0+doc (active)
  tmux @3.1c_0 (active)
``

Regards,
Artem

> On 27 Nov 2020, at 15:37, Jeffrey Walton  wrote:
> 
> On Fri, Nov 27, 2020 at 10:27 AM Giovanni Cantele
>  wrote:
>> 
>> Dear All,.
>> 
>> I’m searching the web but I cannot find any response to the following 
>> question:
>> 
>> is there any ongoing project for porting the whole macports staff on the new 
>> Apple silicon architecture?
>> What happens to those who extensively make use of macports and have bought 
>> the recent released MacBook Pro running on the new processors?
> 
> There are several recent threads about support of the latest hardware
> and software at:
> 
>  * 
> https://lists.macports.org/pipermail/macports-users/2020-November/thread.html
> 
> I understand Macports is working through the issues as they encounter
> them during testing before a release.
> 
> I'm guessing it is a bigger task than just supporting a new OS release.
> 
> Jeff

Re: info

[Jeffrey Walton]

On Fri, Nov 27, 2020 at 10:27 AM Giovanni Cantele
 wrote:
>
> Dear All,.
>
> I’m searching the web but I cannot find any response to the following 
> question:
>
> is there any ongoing project for porting the whole macports staff on the new 
> Apple silicon architecture?
> What happens to those who extensively make use of macports and have bought 
> the recent released MacBook Pro running on the new processors?

There are several recent threads about support of the latest hardware
and software at:

  * 
https://lists.macports.org/pipermail/macports-users/2020-November/thread.html

I understand Macports is working through the issues as they encounter
them during testing before a release.

I'm guessing it is a bigger task than just supporting a new OS release.

Jeff