Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Dima Pasechnik 
TL;DR.
patches are most welcome :-)

Details below.

On Wed, 23 Dec 2020, 02:40 Mouse Mousevich,  wrote:

> A lot of people have a lot of problems with Sage-9.2 on MacOS, installing
> pre-compiled or building from the source.
>
> Sage-9.2 has several idiosyncrasies that for me are hard to explain. One -
> it deliberately refuses to work with Macports-installed packages, even
> though their main difference from similar ones installed via Brew is their
> location. How crazy is that?
>

To me, it is macOS that has idiosyncrasies ;-)
But OK, SageMath problems here are
not deliberate, and caused by severe labour shortage, if you like.
We don't have a single MacPort user among a handful of SageMath developers.
Please feel free to provide patches. (Yes, it means supporting yet another
nonstandard location, more or less, I suppose)


> Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file
> linear_algebra.cpp with clang instead of clang++, and then complain that
> CFLAGS that are (surprise!) C-specific, do not make sense for C++. Need I
> mention that it is impossible to locate clang but not clang++?
>

pplpy upstream is here, it has a very busy maintainer who does not use macOS

https://gitlab.com/videlec/pplpy

- the main developer no longer works on SageMath, as far as I know.

patches are most welcome!


> Now, from generic criticisms to specific errors and logs.
>
> *Platform*
> *-* iMac 2020, Intel Core i9 CPU, 32GM RAM
> - macOS Catalina 10.15.7
> - Xcode-12.3 (with command-line tools installed) in
> /Applications/Xcode.app, and separately installed CLT 12.3 in
> /Library/Developer/CommandLineTools.
>

XCode is a moving target, 9.2 was released before XCode 12.3

- Macports 2.6.4, with a ton of packages installed in /opt/local, which
> include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.
>

one ought to use XCode's clang(++), and only need gfortran from the gcc
package.

Fortran binary location can be explicitly provided by setting FC
environment variable.
(same for C and C++, one can use CC and CXX)

If you like to enable MacPorts, think about providing an analog of
.homebrew-build-env script.


> *Problems*
>
>1. Sage-9.2 config refuses to use already-installed packages in
>/opt/local, spends sh*tload of time and bandwidth downloading what's
>already there (except that it downloads Python-3.8.5, and my machine has
>Python-3.8.6), and finally fails to compile that pile:
>2. pplpy-0.8.4 uses clang (*not* clang++? Why?) to compile ppl/
>linear_algebra.cpp files, and fails because on my machine CFLAGS
>differ from CXXFLAGS (surprise! one sets -std=gnu18, and other one
>sets -std=gnu++17). This bug should manifest on all the .cpp files.
>
> *Logs*
> The main log is 5.7 MB, so I don't want to post it unless asked to.
> Excerpts:
>
> Error building Sage.
>
> The following package(s) may have failed to build (not necessarily
>
> during this run of 'make all-build'):
>
> * package: pplpy-0.8.4
>
>   last build time: Dec 20 22:51
>
>   log file:/Applications/SageMath/logs/pkgs/pplpy-0.8.4.log
>   build directory:
> /Applications/SageMath/local/var/tmp/sage/build/pplpy-0.8.4
>
> and more specifically:
>
> [pplpy-0.8.4]   creating build/temp.macosx-10.9-x86_64-3.8/ppl
>
> [pplpy-0.8.4]   *clang* -Wno-unused-result -Wsign-compare
> -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -O3 -std=gnu18
> -march=native -isysroot
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> -O3 *-std=gnu18* -march=native -isysroot
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> -Ippl -I/Applications/SageMath/local/lib/python3.8/site-packages/gmpy2
> -Ippl -I -I/Applications/SageMath/local
> -I/Applications/SageMath/local/lib/python38.zip
> -I/Applications/SageMath/local/lib/python3.8
> -I/Applications/SageMath/local/lib/python3.8/lib-dynload
> -I/Applications/SageMath/local/lib/python3.8/site-packages
> -I/private/var/folders/_l/4q83bg9j5ysb7qd1n9xpnb4hgn/T/pip-req-build-_4gwq_tw
> -I/Applications/SageMath/local/include/python3.8 -c ppl/linear_algebra.cpp
> -o build/temp.macosx-10.9-x86_64-3.8/ppl/linear_algebra.o
>
> [pplpy-0.8.4]   error: *invalid argument '-std=gnu18' not allowed with
> 'C++'*
>
> [pplpy-0.8.4]   error: *command 'clang' failed* with exit status 1
>
> [pplpy-0.8.4]   Building wheel for pplpy (setup.py): finished with status
> 'error'
>
> [pplpy-0.8.4]   ERROR: Failed building wheel for pplpy
>
> BTW, why is it creating build for macosx-10.9 when the current version of
> the OS is 10.15, and SDK is for 11.1?
>
> On Friday, December 4, 2020 at 11:30:57 AM UTC-5 dim...@gmail.com wrote:
>
>> On Fri, Dec 4, 2020 at 3:26 PM Mathieu Dutour 
>> wrote:
>> >
>> > On Thursday, 3 December 2020 at 19:13:28 UTC+1 dim...@gmail.com wrote:
>> >>
>> >> On Thu, Dec 3, 2020 at 5:39 PM Mathieu Dutour 
>> wrote:
>> >>>
>> >>> Thank you for providi

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Mouse Mousevich 
Please see below.

TL;DR.
> patches are most welcome :-)
>

Understood.
 

> Sage-9.2 has several idiosyncrasies that for me are hard to explain. One - 
>> it deliberately refuses to work with Macports-installed packages, even 
>> though their main difference from similar ones installed via Brew is their 
>> location. How crazy is that?
>>
>
> To me, it is macOS that has idiosyncrasies ;-)
>

True ;-)
But still, my Macports point stands.
 

> We don't have a single MacPort user among a handful of SageMath 
> developers. Please feel free to provide patches. (Yes, it means supporting 
> yet another nonstandard location, more or less, I suppose)
>

The fun part is that to support Macports, you need to look in 
/opt/local/include for header files, in /opt/local/lib for libraries, and 
in /opt/local/bin for executables. How hard does that sound?

And I daresay, that adding those directories to the Sage config code where 
it is currently looking for stuff, is probably easier for a Sage developer 
unfamiliar with Macports, than for a person experienced with Macports that 
has no clue about Sage guts (and isn't eager to dive into those! ;).

Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file 
>> linear_algebra.cpp with clang instead of clang++, and then complain that 
>> CFLAGS that are (surprise!) C-specific, do not make sense for C++. Need I 
>> mention that it is impossible to locate clang but not clang++?
>>
>
> pplpy upstream is here, it has a very busy maintainer who does not use 
> macOS
> https://gitlab.com/videlec/pplpy
> - the main developer no longer works on SageMath, as far as I know.
> patches are most welcome!
>

I'll take a look later, but the likelihood of me being able to fix it isn't 
great.
 

> *Platform*
>> *-* iMac 2020, Intel Core i9 CPU, 32GM RAM
>> - macOS Catalina 10.15.7
>> - Xcode-12.3 (with command-line tools installed) in 
>> /Applications/Xcode.app, and separately installed CLT 12.3 in 
>> /Library/Developer/CommandLineTools.
>>
>
> XCode is a moving target, 9.2 was released before XCode 12.3
>

True, but not really relevant in the context. The big difference/jump came 
with Xcode-10. Xcode-11 and -12 exacerbated that a bit, but I daresay, 
imperceptibly so.
 

> - Macports 2.6.4, with a ton of packages installed in /opt/local, which 
>> include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.
>>
>
> one ought to use XCode's clang(++), and only need gfortran from the gcc 
> package.
>

So one does. ;-)
As you can see from the log, the build uses Xcode's clang and clang++. 

For *my* stuff, especially when I need to deal with sanitizers extensively, 
I use Macports clang. And, of course, Xcode's GCC really sucks, so 
installing a "real" one from Macports was a-must.
 

> Fortran binary location can be explicitly provided by setting FC 
> environment variable.
> (same for C and C++, one can use CC and CXX)
>

It's all done in my config - *except* for Fortran. I don't use it any more 
(for the last 20 years, or so ;), but it's no problem installing it. *Assuming 
Sage would be smart enough to use it* (rather than downloading and building 
it's own GCC and GFortran)!
 

> If you like to enable MacPorts, think about providing an analog of 
> .homebrew-build-env script.
>

That's something constructive that I can probably do. Thanks for bringing 
it up. 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/717dd948-f1c2-4e18-9708-1dd2e68e277bn%40googlegroups.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Dima Pasechnik 
On Wed, 23 Dec 2020, 11:01 Mouse Mousevich,  wrote:

> Please see below.
>
> TL;DR.
>> patches are most welcome :-)
>>
>
> Understood.
>
>
>> Sage-9.2 has several idiosyncrasies that for me are hard to explain. One
>>> - it deliberately refuses to work with Macports-installed packages, even
>>> though their main difference from similar ones installed via Brew is their
>>> location. How crazy is that?
>>>
>>
>> To me, it is macOS that has idiosyncrasies ;-)
>>
>
> True ;-)
> But still, my Macports point stands.
>
>
>> We don't have a single MacPort user among a handful of SageMath
>> developers. Please feel free to provide patches. (Yes, it means supporting
>> yet another nonstandard location, more or less, I suppose)
>>
>
> The fun part is that to support Macports, you need to look in
> /opt/local/include for header files, in /opt/local/lib for libraries, and
> in /opt/local/bin for executables. How hard does that sound?
>
> And I daresay, that adding those directories to the Sage config code where
> it is currently looking for stuff, is probably easier for a Sage developer
> unfamiliar with Macports, than for a person experienced with Macports that
> has no clue about Sage guts (and isn't eager to dive into those! ;).
>

the MacPorts-specific part would be to create a list of packages to
install, and put it into build/pkgs/*/distros/macports
and a similar global location.



> Second, pplpy-0.8.4 is "obtuse" enough to try compiling C++ file
>>> linear_algebra.cpp with clang instead of clang++, and then complain
>>> that CFLAGS that are (surprise!) C-specific, do not make sense for C++.
>>> Need I mention that it is impossible to locate clang but not clang++?
>>>
>>
>> pplpy upstream is here, it has a very busy maintainer who does not use
>> macOS
>> https://gitlab.com/videlec/pplpy
>> - the main developer no longer works on SageMath, as far as I know.
>> patches are most welcome!
>>
>
> I'll take a look later, but the likelihood of me being able to fix it
> isn't great.
>
>
>> *Platform*
>>> *-* iMac 2020, Intel Core i9 CPU, 32GM RAM
>>> - macOS Catalina 10.15.7
>>> - Xcode-12.3 (with command-line tools installed) in
>>> /Applications/Xcode.app, and separately installed CLT 12.3 in
>>> /Library/Developer/CommandLineTools.
>>>
>>
>> XCode is a moving target, 9.2 was released before XCode 12.3
>>
>
> True, but not really relevant in the context. The big difference/jump came
> with Xcode-10. Xcode-11 and -12 exacerbated that a bit, but I daresay,
> imperceptibly so.
>
>
>> - Macports 2.6.4, with a ton of packages installed in /opt/local, which
>>> include clang-11, gcc-10, python38 and python39, and OpenSSL-1.1.1i.
>>>
>>
>> one ought to use XCode's clang(++), and only need gfortran from the gcc
>> package.
>>
>
> So one does. ;-)
> As you can see from the log, the build uses Xcode's clang and clang++.
>
> For *my* stuff, especially when I need to deal with sanitizers
> extensively, I use Macports clang. And, of course, Xcode's GCC really
> sucks, so installing a "real" one from Macports was a-must.
>
>
>> Fortran binary location can be explicitly provided by setting FC
>> environment variable.
>> (same for C and C++, one can use CC and CXX)
>>
>
> It's all done in my config - *except* for Fortran. I don't use it any
> more (for the last 20 years, or so ;), but it's no problem installing it. 
> *Assuming
> Sage would be smart enough to use it* (rather than downloading and
> building it's own GCC and GFortran)!
>
>
>> If you like to enable MacPorts, think about providing an analog of
>> .homebrew-build-env script.
>>
>
> That's something constructive that I can probably do. Thanks for bringing
> it up.
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-support+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-support/717dd948-f1c2-4e18-9708-1dd2e68e277bn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/CAAWYfq1%2B%3D1ESX2PRPvNG%3DbTw36FDabzC0YpR-n3bx-Wd4MLkbQ%40mail.gmail.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Mouse Mousevich 

>
>
>> We don't have a single MacPort user among a handful of SageMath 
>>> developers. Please feel free to provide patches. (Yes, it means supporting 
>>> yet another nonstandard location, more or less, I suppose)
>>>
>>
>> The fun part is that to support Macports, you need to look in 
>> /opt/local/include for header files, in /opt/local/lib for libraries, 
>> and in /opt/local/bin for executables. How hard does that sound?
>>
>> And I daresay, that adding those directories to the Sage config code 
>> where it is currently looking for stuff, is probably easier for a Sage 
>> developer unfamiliar with Macports, than for a person experienced with 
>> Macports that has no clue about Sage guts (and isn't eager to dive into 
>> those! ;).
>>
>
> the MacPorts-specific part would be to create a list of packages to 
> install, and put it into build/pkgs/*/distros/macports 
> and a similar global location.
>

I'm not sure I understand what that means. If a package is already 
installed by Macports - it does not make sense installing it again. More 
so, Macports keeps a *system-wide* installation, rather than a copy for 
each user.

It might make sense telling Macports to install packages that aren't 
already installed. But again, I don't see any benefit or even use in 
something like build/pkgs/*/distros/macports.

And, frankly, I see more harm than benefit in Sage getting stuck with 
copies of packages it builds itself in its private location, becoming 
bloated and stale at the same time. While the package manager (Macports in 
this case) keeps track of everything it "owns", shares one copy among all 
the users, and ensures they're updated.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/1f3efa19-cf0a-4db9-9d8e-e0817c8ce5bbn%40googlegroups.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Mouse Mousevich 
I looked into build/pgks/* (1.5 GB) and it appears that Sage is trying to 
be a package manager, in addition to what people probably want it for. 

I guess at version 9.2 it's rather late to ask why it isn't good enough to 
just list in the README a set of packages that Sage requires to run, and 
have a minimal config that would check for their presence at install time? 
Letting user to worry about, e.g., where and how to install OpenSSL-1.1.x 
or python38?

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/85d43727-0336-406f-b877-35b5ed8bb29dn%40googlegroups.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Samuel Lelièvre 
Le mer. 23 déc. 2020 à 14:55, Mouse Mousevich:
>>
>> the MacPorts-specific part would be to create a list of packages to install, 
>> and put it into build/pkgs/*/distros/macports
>> and a similar global location.
>
> I'm not sure I understand what that means. If a package is already installed
> by Macports - it does not make sense installing it again. More so, Macports
> keeps a system-wide installation, rather than a copy for each user.

For each package `pkgname` Sage uses, the `build/pkg/pkgname` folder
contains information about building and about not-building this package.
The not-building information is a folder `distros` with a bunch of text files,
one per distro, containing the name of the package in that distro that can
be used if present instead of building it. For an example, see

https://github.com/sagemath/sage/tree/develop/build/pkgs/openssl/distros

The various text files there tell the Sage build system what package
to check for. Depending on the distro, that might be one of:

libressl-openssl
libssl-dev
libssl-devel
openssl
openssl openssl-devel
openssl openssl-solibs
openssl-dev
security/openssl

To support using the packages from MacPorts, we need

- a file build/pkgs/macports.txt with information on core packages,
  see all the text files in

  https://github.com/sagemath/sage/tree/develop/build/pkgs/

- and for each "spkg", a macports.txt file containing the name of
  the equivalent MacPorts package to check for (and to recommend
  installing if not present)

See recent work on adding support for voidlinux

https://trac.sagemath.org/query?order=id&desc=1&summary=~voidlinux&summary=~void%20linux

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/CAEcArF00JbEwFLMPu9-192TzhOhhtaZR2wY1aFAOZD2SutDVbQ%40mail.gmail.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread John H Palmieri 
The short answer is that Sage is designed for (among others) research 
mathematicians, who may not have the interest or inclination to learn how 
to install lots of system packages. So from the beginning it included as 
many components as possible. Years ago the presence of MacPorts and other 
similar systems would actually break the Sage build process, so the build 
process may attempt to disable them. Whether it tried to disable them, for 
a long time Sage refused to use any system packages and just built 
everything itself (with a few exceptions, like gcc).

This has been changing recently, and now Sage can use many system packages. 
This is restricted to systems (OS + package manager) used by Sage 
developers, and apparently no developer uses MacPorts, or at least not 
enough to spend time getting Sage to work with it. Homebrew is pretty well 
integrated with the Sage build process.

On Wednesday, December 23, 2020 at 6:11:32 AM UTC-8 u...@ll.mit.edu wrote:

> I looked into build/pgks/* (1.5 GB) and it appears that Sage is trying to 
> be a package manager, in addition to what people probably want it for. 
>
> I guess at version 9.2 it's rather late to ask why it isn't good enough to 
> just list in the README a set of packages that Sage requires to run, and 
> have a minimal config that would check for their presence at install time? 
> Letting user to worry about, e.g., where and how to install OpenSSL-1.1.x 
> or python38?
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/06787647-8d63-4e0b-912b-3ddefc516833n%40googlegroups.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Dima Pasechnik 
On Wed, 23 Dec 2020, 14:11 Mouse Mousevich,  wrote:

> I looked into build/pgks/* (1.5 GB) and it appears that Sage is trying to
> be a package manager, in addition to what people probably want it for.
>
> I guess at version 9.2 it's rather late to ask why it isn't good enough to
> just list in the README a set of packages that Sage requires to run, and
> have a minimal config that would check for their presence at install time?
> Letting user to worry about, e.g., where and how to install OpenSSL-1.1.x
> or python38?
>

we are trying to do it right, with automated testing on the many platforms:
a dozen of different Linux distributions,  some with several versions, with
Conda, with Homebrew, with Cygwin and WSL on Windows.
This needs the information on what packages are needed on each of these.

Letting the user worry about won't help for this task.

> --
> You received this message because you are subscribed to the Google Groups
> "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-support+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-support/85d43727-0336-406f-b877-35b5ed8bb29dn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/CAAWYfq0stPCpz74ssWn%2BSmV1xRQTCoLMX0_MzTx%3DKDmUavRqCw%40mail.gmail.com.

Re: [sage-support] Re: Install of sage on macOS

2020-12-23 Thread Matthias Koeppe 
On Wednesday, December 23, 2020 at 5:55:38 AM UTC-8 u...@ll.mit.edu wrote:

>
>>> the MacPorts-specific part would be to create a list of packages to 
>> install, and put it into build/pkgs/*/distros/macports 
>> and a similar global location.
>>
>
> I'm not sure I understand what that means.
>

You can learn about it by 
reading 
https://doc.sagemath.org/html/en/developer/packaging.html#allowing-for-the-use-of-system-packages

If you want to help with supporting MacPorts 
packages, https://trac.sagemath.org/ticket/30504 


-- 
You received this message because you are subscribed to the Google Groups 
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-support/db30bb01-e714-4b5f-a2cd-2fae1cf45946n%40googlegroups.com.