Re: Forcing python39 on M1
Without intending to criticize any individual effort, I nevertheless respectfully suggest that something is less than ideal from a design & system standpoint, and I offer this because I think macports generally tries to get these high-level decisions architectural decisions right. I think at core it is the assumption that to build successfully means, kind of, that it works. This is much less true for complicated libraries, and more-or-less not true at all for libraries in languages like python, which rightly or wrongly push errors out to runtime instead of link time, or better yet, compile time. In other words, that a python library like scipy "builds" is a nice first step, but means little by itself. Python programmers know that that regression tests are not just a nice-to-have feature, but essential in their world. Let us take the current scenario. The Scipy library is (or was) broken on M1, and there is abundant discussion there about what the issue is and what the fixes might be, and people are implementing workarounds and redesigns, and what have you. The problem, I respectfully suggest, is that somehow this gets all filtered out by the macports process and what you end up with is a "green" port that does not work. And not because anybody is not doing, you say, what they are supposed to. Because, apparently, a port maintainer is "not required" to run the regression test. You might come to a different conclusion about what is desirable or practical for this, but I think you will agree that a well-known problem getting translated, by macports process, to a broken build that is advertised as working indicates that something is not quite right, and perhaps a tweak to the process could be considered. And this is for something that is known not to work upstream. If you see what I mean. Thanks. > On Mar 13, 2021, at 10:38, Todd Doucet wrote: > > > At some point I might try to determine whether I can get a useful python3 > > setup using Macports, but it is time-consuming and the status pages do not > > help, as the comments indicate. > > > > When I tried a few weeks ago, I could install either py38 or py39, and that > > is fine if I want to write "hello world". But to be useful, I need > > libraries like matplotlib, pandas, and and scipy. > > > > But these packages were all available too, and marked happy green in the > > summary page. > > > > But they do not actually work. For example, scipy will quickly crash if > > you do most anything with it. > > > > The cause is well understood upstream on the scipy site. The bug is not > > caused by Macports. > > > > But, really, it is hard for me to understand how these packages get a happy > > green label attached when all the build system, or the port maintainer, has > > to do is run the package-supplied regression test to see if it works. And > > it does not. > > > > As a user, it seems time-consuming and wasteful to install packages that do > > not work but claim to. And it seems just too odd to file a bug report that > > essentially asks the port maintainer to run the regression test. > > > > Maybe this all works now--I do not have the time to deal with it. But > > perhaps my recent experience and frustration could be useful to others. > > > > Here is what I presently do: use the stock python from macOS and install > > local version of the libraries, then run all my code using the "arch > > -x86_64", which runs everything in x86 translation. It actually works for > > everything I do and is only a bit slower than it would be otherwise (but > > still faster than my old x86). > > > > It would be better to have a native arm64 version of the python stuff. My > > understanding is that it is possible to cobble one together, and people > > have done that. (Maybe the fratboys at brew have done that, not sure, I > > cannot bring myself to use brew.) > > > > Eventually the upstream problems will be fixed and eventually there will be > > a macports version of it. Until then, it would be useful to not report > > things working when they are not. > > I understand this is frustrating. > > Apple Silicon is a new platform, and macOS 11 is a new operating system. > Problems are bound to arise due to both. Each problem needs to be > investigated and fixed individually. File bug reports with upstream if it is > an upstream problem (i.e. if the software installs but doesn't work right). > File bug reports with us if it is our problem (i.e. if we need to update to a > new upstream version to fix a problem, or if upstream has fixed the problem > but not yet made a new release and we can easily backport their change). > > Green status on the ports web page means no errors were encountered the last > time the build system tried to build that port. (This includes the condition > where the port indicates that it is known to fail on that system. It would be > more helpful if we could have a differe
Re: Forcing python39 on M1
I missing gimp-app more. It asks even for python27! pmm:~ pet$ sudo port install gimp-app Warning: The macOS 11.2 SDK does not appear to be installed. Ports may not build correctly. Warning: You can install it as part of the Xcode Command Line Tools package by running `xcode-select --install'. ---> Computing dependencies for gimp-app The following dependencies will be installed: gimp2 libmypaint libwmf mypaint-brushes1 py27-cython py27-gobject py27-nose py27-numpy py27-pygtk py27-setuptools python37 xdg-utils Continue? [Y/n]: Warning: The macOS 11.2 SDK does not appear to be installed. Ports may not build correctly. Warning: You can install it as part of the Xcode Command Line Tools package by running `xcode-select --install'. ---> Fetching archive for python37 ---> Attempting to fetch python37-3.7.10_0.darwin_20.arm64.tbz2 from https://packages.macports.org/python37 ---> Attempting to fetch python37-3.7.10_0.darwin_20.arm64.tbz2 from https://cph.dk.packages.macports.org/python37 ---> Attempting to fetch python37-3.7.10_0.darwin_20.arm64.tbz2 from https://lil.fr.packages.macports.org/python37 ---> Fetching distfiles for python37 Error: Failed to fetch python37: python37 cannot yet be built for Apple Silicon Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python37/python37/main.log for details. Error: Follow https://guide.macports.org/#project.tickets to report a bug. Error: Processing of port gimp-app failed Petr __ > Od: "Rainer Müller" > Komu: "Peter West" , "MacPorts Users" > > Datum: 13.03.2021 14:33 > Předmět: Re: Forcing python39 on M1 > >On 12/03/2021 12.16, Peter West wrote: >>> I ran intoa brick wall with gexiv2, which was because the g-ir-* python >>> files >>> were specifying python38. I tried building gobject-introspection with 39, >>> but >>> fails. I don’t know whether I can classify this as a bug and make a report. >>> In >>> any case, has anyone else been tinkering with python39 builds on M1? >> :info:build g-ir-scanner: GLib: warning: 627 warnings suppressed (use >> --warn-all >> to see them) >> :info:build Traceback (most recent call last): >> :info:build File >> "/opt/local/var/macports/build/_Users_pbw_Software_ports_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/./g-ir-scanner", >> line 99, in >> :info:build sys.exit(scanner_main(sys.argv)) >> :info:build File "./giscanner/scannermain.py", line 615, in scanner_main >> :info:build write_output(data, options) >> :info:build File "./giscanner/scannermain.py", line 469, in write_output >> :info:build passthrough_gir(main_f_name, temp_f) >> :info:build File "./giscanner/scannermain.py", line 260, in passthrough_gir >> :info:build parser.parse(path) >> :info:build File "./giscanner/girparser.py", line 60, in parse >> :info:build self.parse_tree(tree) >> :info:build File "./giscanner/girparser.py", line 69, in parse_tree >> :info:build self._parse_api(tree.getroot()) >> :info:build File "./giscanner/girparser.py", line 106, in _parse_api >> :info:build for node in root.getchildren(): >> :info:build AttributeError: 'xml.etree.ElementTree.Element' object has no >> attribute 'getchildren' > >This specific method had been deprecated for a while and was eventually removed >in Python 3.9. It is a known problem at upstream that has already been fixed. >The fix would be included in gobject-introspection >= 1.65.0 > >https://gitlab.gnome.org/GNOME/gobject-introspection/-/issues/325 > >Your best option would be to also bump the port version to get a version >compatible with Python 3.9. > >Rainer > >
Re: Forcing python39 on M1
On Mar 13, 2021, at 10:38, Todd Doucet wrote: > At some point I might try to determine whether I can get a useful python3 > setup using Macports, but it is time-consuming and the status pages do not > help, as the comments indicate. > > When I tried a few weeks ago, I could install either py38 or py39, and that > is fine if I want to write "hello world". But to be useful, I need libraries > like matplotlib, pandas, and and scipy. > > But these packages were all available too, and marked happy green in the > summary page. > > But they do not actually work. For example, scipy will quickly crash if you > do most anything with it. > > The cause is well understood upstream on the scipy site. The bug is not > caused by Macports. > > But, really, it is hard for me to understand how these packages get a happy > green label attached when all the build system, or the port maintainer, has > to do is run the package-supplied regression test to see if it works. And it > does not. > > As a user, it seems time-consuming and wasteful to install packages that do > not work but claim to. And it seems just too odd to file a bug report that > essentially asks the port maintainer to run the regression test. > > Maybe this all works now--I do not have the time to deal with it. But > perhaps my recent experience and frustration could be useful to others. > > Here is what I presently do: use the stock python from macOS and install > local version of the libraries, then run all my code using the "arch > -x86_64", which runs everything in x86 translation. It actually works for > everything I do and is only a bit slower than it would be otherwise (but > still faster than my old x86). > > It would be better to have a native arm64 version of the python stuff. My > understanding is that it is possible to cobble one together, and people have > done that. (Maybe the fratboys at brew have done that, not sure, I cannot > bring myself to use brew.) > > Eventually the upstream problems will be fixed and eventually there will be a > macports version of it. Until then, it would be useful to not report things > working when they are not. I understand this is frustrating. Apple Silicon is a new platform, and macOS 11 is a new operating system. Problems are bound to arise due to both. Each problem needs to be investigated and fixed individually. File bug reports with upstream if it is an upstream problem (i.e. if the software installs but doesn't work right). File bug reports with us if it is our problem (i.e. if we need to update to a new upstream version to fix a problem, or if upstream has fixed the problem but not yet made a new release and we can easily backport their change). Green status on the ports web page means no errors were encountered the last time the build system tried to build that port. (This includes the condition where the port indicates that it is known to fail on that system. It would be more helpful if we could have a different color for this condition.) Red status means errors were encountered the last time the build system tried to build that port. The web page does not indicate whether the status information represents the most recently available version of the port. The build system does not run any tests that a port may have. Ports are not required to have tests and port maintainers are not required to run tests prior to committing updates to ports. You can file a bug report to ask the maintainer to run the tests, but at best the maintainer has a similar enough system as you and sees the same problem when running the tests and can then file a bug report with the developers to get the problem fixed. You can save us time by filing the bug report directly with the developers yourself. Remember that we are all volunteers. We work on what we can in our spare time. We also understandably often work on things that are of interest to us or that affect us. Many MacPorts contributors don't have Apple Silicon systems yet so haven't personally experienced whatever issues might exist there. Therefore we need either bug reports or pull requests from those who do have Apple Silicon systems to get things fixed.
Re: Forcing python39 on M1
To be super-clear, the problems I am describing all relate to the M1 version of the python libraries. > At some point I might try to determine whether I can get a useful python3 > setup using Macports, but it is time-consuming and the status pages do not > help, as the comments indicate. > > When I tried a few weeks ago, I could install either py38 or py39, and that > is fine if I want to write "hello world". But to be useful, I need libraries > like matplotlib, pandas, and and scipy. > > But these packages were all available too, and marked happy green in the > summary page. > > But they do not actually work. For example, scipy will quickly crash if you > do most anything with it. > > The cause is well understood upstream on the scipy site. The bug is not > caused by Macports. > > But, really, it is hard for me to understand how these packages get a happy > green label attached when all the build system, or the port maintainer, has > to do is run the package-supplied regression test to see if it works. And it > does not. > > As a user, it seems time-consuming and wasteful to install packages that do > not work but claim to. And it seems just too odd to file a bug report that > essentially asks the port maintainer to run the regression test. > > Maybe this all works now--I do not have the time to deal with it. But > perhaps my recent experience and frustration could be useful to others. > > Here is what I presently do: use the stock python from macOS and install > local version of the libraries, then run all my code using the "arch > -x86_64", which runs everything in x86 translation. It actually works for > everything I do and is only a bit slower than it would be otherwise (but > still faster than my old x86). > > It would be better to have a native arm64 version of the python stuff. My > understanding is that it is possible to cobble one together, and people have > done that. (Maybe the fratboys at brew have done that, not sure, I cannot > bring myself to use brew.) > > Eventually the upstream problems will be fixed and eventually there will be a > macports version of it. Until then, it would be useful to not report things > working when they are not. > > >> *Sigh*. The ports.macports.org site has not been getting updates since Feb >> 20. See recent threads. Yes, this is *usually* reliable. Not this month. >> >> >> On Sat, Mar 13, 2021 at 8:23 AM Ken Cunningham >> wrote: >>> > Having heard that python39 is the only one (so far) to compile natively >>> > on M1, I’m trying to force the python ports I use to python39 >>> >>> Hello Peter, >>> >>> FYI, an arm64 version of python38 appears to be available: >>> >>> http://packages.macports.org/python38/python38-3.8.8_0.darwin_20.arm64.tbz2 >>> >>> and is “green” on the ports review page: >>> >>> https://ports.macports.org/port/python38/summary >>> >>> >>> The ports.macports.org page can be misleading at times, unfortunately, as >>> it will show “green” if the port has been blocked from building even if it >>> can’t be built, which is no doubt confusing to people at times and there is >>> (I believe) a ticket about that somewhere. >>> >>> The packages.macports.org site is pretty reliable, although to be 100% >>> certain, you do need to actually install the port and examine it with >>> “file” or “arch” or similar. >>> >>> And the fact that the arm64 python38 exists doesn’t necessarily mean a >>> universal python38 exists for x86_64/arm64 or can be built. It might or >>> might not. >>> >>> So there are some caveats to the presence of the python38 file there on >>> packages, but it is there. >>> >>> Best, >>> >>> Ken >
Re: Forcing python39 on M1
At some point I might try to determine whether I can get a useful python3 setup using Macports, but it is time-consuming and the status pages do not help, as the comments indicate. When I tried a few weeks ago, I could install either py38 or py39, and that is fine if I want to write "hello world". But to be useful, I need libraries like matplotlib, pandas, and and scipy. But these packages were all available too, and marked happy green in the summary page. But they do not actually work. For example, scipy will quickly crash if you do most anything with it. The cause is well understood upstream on the scipy site. The bug is not caused by Macports. But, really, it is hard for me to understand how these packages get a happy green label attached when all the build system, or the port maintainer, has to do is run the package-supplied regression test to see if it works. And it does not. As a user, it seems time-consuming and wasteful to install packages that do not work but claim to. And it seems just too odd to file a bug report that essentially asks the port maintainer to run the regression test. Maybe this all works now--I do not have the time to deal with it. But perhaps my recent experience and frustration could be useful to others. Here is what I presently do: use the stock python from macOS and install local version of the libraries, then run all my code using the "arch -x86_64", which runs everything in x86 translation. It actually works for everything I do and is only a bit slower than it would be otherwise (but still faster than my old x86). It would be better to have a native arm64 version of the python stuff. My understanding is that it is possible to cobble one together, and people have done that. (Maybe the fratboys at brew have done that, not sure, I cannot bring myself to use brew.) Eventually the upstream problems will be fixed and eventually there will be a macports version of it. Until then, it would be useful to not report things working when they are not. > *Sigh*. The ports.macports.org site has not been getting updates since Feb > 20. See recent threads. Yes, this is *usually* reliable. Not this month. > > > On Sat, Mar 13, 2021 at 8:23 AM Ken Cunningham > wrote: >> > Having heard that python39 is the only one (so far) to compile natively on >> > M1, I’m trying to force the python ports I use to python39 >> >> Hello Peter, >> >> FYI, an arm64 version of python38 appears to be available: >> >> http://packages.macports.org/python38/python38-3.8.8_0.darwin_20.arm64.tbz2 >> >> and is “green” on the ports review page: >> >> https://ports.macports.org/port/python38/summary >> >> >> The ports.macports.org page can be misleading at times, unfortunately, as it >> will show “green” if the port has been blocked from building even if it >> can’t be built, which is no doubt confusing to people at times and there is >> (I believe) a ticket about that somewhere. >> >> The packages.macports.org site is pretty reliable, although to be 100% >> certain, you do need to actually install the port and examine it with “file” >> or “arch” or similar. >> >> And the fact that the arm64 python38 exists doesn’t necessarily mean a >> universal python38 exists for x86_64/arm64 or can be built. It might or >> might not. >> >> So there are some caveats to the presence of the python38 file there on >> packages, but it is there. >> >> Best, >> >> Ken
Re: Forcing python39 on M1
*Sigh*. The ports.macports.org site has not been getting updates since Feb 20. See recent threads. Yes, this is *usually* reliable. Not this month. On Sat, Mar 13, 2021 at 8:23 AM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > > Having heard that python39 is the only one (so far) to compile natively > on M1, I’m trying to force the python ports I use to python39 > > Hello Peter, > > FYI, an arm64 version of python38 appears to be available: > > http://packages.macports.org/python38/python38-3.8.8_0.darwin_20.arm64.tbz2 > > and is “green” on the ports review page: > > https://ports.macports.org/port/python38/summary > > > The ports.macports.org page can be misleading at times, unfortunately, as > it will show “green” if the port has been blocked from building even if it > can’t be built, which is no doubt confusing to people at times and there is > (I believe) a ticket about that somewhere. > > The packages.macports.org site is pretty reliable, although to be 100% > certain, you do need to actually install the port and examine it with > “file” or “arch” or similar. > > And the fact that the arm64 python38 exists doesn’t necessarily mean a > universal python38 exists for x86_64/arm64 or can be built. It might or > might not. > > So there are some caveats to the presence of the python38 file there on > packages, but it is there. > > Best, > > Ken >
Re: Forcing python39 on M1
> Having heard that python39 is the only one (so far) to compile natively on > M1, I’m trying to force the python ports I use to python39 Hello Peter, FYI, an arm64 version of python38 appears to be available: http://packages.macports.org/python38/python38-3.8.8_0.darwin_20.arm64.tbz2 and is “green” on the ports review page: https://ports.macports.org/port/python38/summary The ports.macports.org page can be misleading at times, unfortunately, as it will show “green” if the port has been blocked from building even if it can’t be built, which is no doubt confusing to people at times and there is (I believe) a ticket about that somewhere. The packages.macports.org site is pretty reliable, although to be 100% certain, you do need to actually install the port and examine it with “file” or “arch” or similar. And the fact that the arm64 python38 exists doesn’t necessarily mean a universal python38 exists for x86_64/arm64 or can be built. It might or might not. So there are some caveats to the presence of the python38 file there on packages, but it is there. Best, Ken
Re: Forcing python39 on M1
Thanks Rainer. Pardon my ignorance, but could you elaborate on “bump the port version” please? I assume that will have ramifications for all the dependents as well. Peter — p...@ehealth.id.au “Two men went up into the temple to pray, one a Pharisee and the other a tax collector.” > On 13 Mar 2021, at 11:32 pm, Rainer Müller wrote: > > On 12/03/2021 12.16, Peter West wrote: >>> I ran intoa brick wall with gexiv2, which was because the g-ir-* python >>> files >>> were specifying python38. I tried building gobject-introspection with 39, >>> but >>> fails. I don’t know whether I can classify this as a bug and make a report. >>> In >>> any case, has anyone else been tinkering with python39 builds on M1? >> :info:build g-ir-scanner: GLib: warning: 627 warnings suppressed (use >> --warn-all >> to see them) >> :info:build Traceback (most recent call last): >> :info:build File >> "/opt/local/var/macports/build/_Users_pbw_Software_ports_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/./g-ir-scanner", >> line 99, in >> :info:build sys.exit(scanner_main(sys.argv)) >> :info:build File "./giscanner/scannermain.py", line 615, in scanner_main >> :info:build write_output(data, options) >> :info:build File "./giscanner/scannermain.py", line 469, in write_output >> :info:build passthrough_gir(main_f_name, temp_f) >> :info:build File "./giscanner/scannermain.py", line 260, in passthrough_gir >> :info:build parser.parse(path) >> :info:build File "./giscanner/girparser.py", line 60, in parse >> :info:build self.parse_tree(tree) >> :info:build File "./giscanner/girparser.py", line 69, in parse_tree >> :info:build self._parse_api(tree.getroot()) >> :info:build File "./giscanner/girparser.py", line 106, in _parse_api >> :info:build for node in root.getchildren(): >> :info:build AttributeError: 'xml.etree.ElementTree.Element' object has no >> attribute 'getchildren' > > This specific method had been deprecated for a while and was eventually > removed > in Python 3.9. It is a known problem at upstream that has already been fixed. > The fix would be included in gobject-introspection >= 1.65.0 > > https://gitlab.gnome.org/GNOME/gobject-introspection/-/issues/325 > > Your best option would be to also bump the port version to get a version > compatible with Python 3.9. > > Rainer
Re: Forcing python39 on M1
On 12/03/2021 12.16, Peter West wrote: >> I ran intoa brick wall with gexiv2, which was because the g-ir-* python files >> were specifying python38. I tried building gobject-introspection with 39, but >> fails. I don’t know whether I can classify this as a bug and make a report. >> In >> any case, has anyone else been tinkering with python39 builds on M1? > :info:build g-ir-scanner: GLib: warning: 627 warnings suppressed (use > --warn-all > to see them) > :info:build Traceback (most recent call last): > :info:build File > "/opt/local/var/macports/build/_Users_pbw_Software_ports_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/./g-ir-scanner", > line 99, in > :info:build sys.exit(scanner_main(sys.argv)) > :info:build File "./giscanner/scannermain.py", line 615, in scanner_main > :info:build write_output(data, options) > :info:build File "./giscanner/scannermain.py", line 469, in write_output > :info:build passthrough_gir(main_f_name, temp_f) > :info:build File "./giscanner/scannermain.py", line 260, in passthrough_gir > :info:build parser.parse(path) > :info:build File "./giscanner/girparser.py", line 60, in parse > :info:build self.parse_tree(tree) > :info:build File "./giscanner/girparser.py", line 69, in parse_tree > :info:build self._parse_api(tree.getroot()) > :info:build File "./giscanner/girparser.py", line 106, in _parse_api > :info:build for node in root.getchildren(): > :info:build AttributeError: 'xml.etree.ElementTree.Element' object has no > attribute 'getchildren' This specific method had been deprecated for a while and was eventually removed in Python 3.9. It is a known problem at upstream that has already been fixed. The fix would be included in gobject-introspection >= 1.65.0 https://gitlab.gnome.org/GNOME/gobject-introspection/-/issues/325 Your best option would be to also bump the port version to get a version compatible with Python 3.9. Rainer
