HI all,
William Stein recently bemoaned the fact that SageMath currently only runs
natively on some brands of Linux, and not natively on the latest Windows or
OSX (that is to say nothing of BSD). [1]
Until recently, a port of SageMath to Windows has seemed like a pipe dream.
However, things have changed a lot, and I think it would now be possible to
achieve this (for some definition of the word "native").
I've tried VM's before and it has always ended in disaster. They either
screw up my networking, they have performance issues, or don't support my
mouse properly, or change my keyboard layout, or it's impossible to get
files from my hard drive into the system easily, or they take up too much
disk space or need to be constantly downloaded to update them, or some
features don't work, or people stop supporting them, etc. The disadvantages
of VMs are so numerous I hardly need to enumerate them. And even if it is a
solution for users, it's hardly a solution for serious Windows developers.
As some people know, I've been recently working on a Julia based "computer
algebra system" for a much more limited subset of "computer algebra" than
Sage targets. What people may not know is that that entire technology stack
works natively on Windows.
I can't express how fantastic it was to be sitting on a train recently,
with no web access whatsoever and to be able to do work on my Windows 10
(64 bit) laptop on the train. And everything ran as fast, or in some cases
faster, than my Linux server. That's the first time that has happened since
I started doing computational stuff about 10 years ago!
MSYS2 as a solution
================
The new technology that makes all this work is MSYS2. Here are some of its
features:
* Installing MSYS2 on Windows couldn't be easier. [2]
* It supports proper Windows exception handling protocols.
* It has an amazing command line package manager called Pacman, which Just
Works TM.
* Unlike Cygwin, it's very minimal, and takes little time to install.
* It's fast. Very fast.
* Parallel make works.
* The resulting binaries are fast, sometimes faster on my laptop than on
the Linux server I usually use for development.
* MSYS2 provides a posix layer! (Applications can be distributed with an
MSYS2 dll for this.)
* Multithreading using pthreads Just Works TM (Applications can be
distributed with a winthreads dll for this. I've actually used this, no
patching required, so I am not speaking theoretically here.)
* Like native Windows, sizeof(long) = 4 and sizeof(long long) = 8 on MSYS2.
This means interop with native Windows applications or assembly objects is
a cynch.
* The resulting applications can be run on Windows as essentially native
applications. They don't have to be run from within the MSYS2 shell. They
can also be distributed as binary packages for those that don't care about
the source code. But here's the thing: it's not *necessary* to distribute
them as binary packages. It's now quite feasible for developers to actually
*build* packages on Windows. And let's face it, this is the type of
developer we actually want on the Windows platform. Simply supporting users
and not developers is just going to increase the maintenance burden for
people who work on other platforms. We want actual Windows developers, not
just users!
The only bad thing about MSYS2 is that autotools is still incredibly slow
(Flint roll-your-own configure takes 5s at most on Windows, whereas
autotools can take some minutes to configure a package). Note: make is not
slow (at least not on Windows 64 -- it's not as fast on 32 bit Windows, but
fortunately almost no one is using Windows 32 any more anyway).
Proposal
=======
What I propose is to begin porting Sage dependencies to Windows (note that
by Windows, I mean Windows 64, i.e. what everyone is using). What this
would mean is porting them to MSYS2 (at least initially). Note it's not a
big step from there to MinGW or MSVC if people want to do that later.
Microsoft have made MSVC much, much more friendly to GNU conventions,
recently. (For some users, porting to Windows natively is synonymous with
supporting MSVC using Visual Solution files. But I'm not suggesting we do
that.)
Since we know that many upstream projects simply will not accept thousands
of patches to their repositories for supporting Windows 64, much less do
they have developer resources to support them on an ongoing basis, here is
what I think needs to be done:
1) Automagically patch their source trees at build time so that long
becomes long long and unsigned long becomes unsigned long long on Windows.
2) Automagically patch their source trees at build time so that
printf/scanf("%ld") becomes printf/scanf("%lld") on Windows.
3) Send a very small number of "Windows" patches upstream for the few
instances of nonportable code that probably doesn't work anywhere except
Linux, let alone on Windows.
Is this feasible?
Yes it is. Note that libpari still works today on Windows 64 (actually
natively with MinGW, not just with MSYS). On Windows 64 their build
automagically patches their source tree. As far as I know, this hasn't been
touched in years, and it still works today as well as the day it was
written (modulo a couple of trivial bugs in their actual build system)!
[Optionally, we should investigate the feasibility of providing Makefiles
for Windows, automatically generated by autotools, but stored on a server
so that they do not have to be generated at build time (of course the user
still has the option of doing that if they really want). This may not be
feasible, but it's worth investigating. Obviously paths would have to be
patched on a per user basis. But Windows is a very homogeneous environment,
so most of what autotools actually does on Windows is pretty pointless.]
What we've done so far
==================
So far, the following packages (and probably many more) work fine on
Windows 64, with the provisions noted:
* MPIR-2.7.0
* MPFR-3.1.3
* Flint-2.4.5
* libpari-2.7.4 -- GP is not patched, and there are some bugs in the Pari
build script, but I've been able to hack them to get libpari to build on
Windows 64. The %ld -> %lld patching is not done yet.
* Antic
* OpenBLAS (the Julia people already build this on Windows, since they use
it heavily)
Python must also work, since I use IJulia on my Windows 64 laptop, which is
basically the same thing as the IPython notebook. This I download as part
of the Anaconda distribution currently.
You can find Windows 32 and 64 dlls for most of the packages mentioned
above, here:
http://nemocas.org/binaries/
My intention is to publish a note every time I update this collection of
dlls and to gradually expand them to cover more packages. We'd eventually
want to use dlltray, or the OpenSUSE cross platform build service [3] for
hosting these (the latter is not trivial at the moment, since their package
management system doesn't handle some of the complexities of building
packages that Sage would rely on).
It would be fantastic if others were to join me in extending this list over
time. I'd be happy to host the dlls there (actually the machine is provided
by William Stein), at least until the hosting requirements start to become
nontrivial.
The only requirement would be to send dlls for Window 32 and 64 and a
tarball of the source used to generate them (or alternatively a git repo I
can clone), so that we aren't violating anyone's GPL. We'd also have to
agree on what exception handling protocol to use. It would be very
convenient for me should that match the one used by the Julia project.
We will shortly be porting Arb to Windows 64. I'll post an update when that
is done.
Note that Julia itself works on Windows and they have an interest in
porting lots of stuff to work natively on Windows. So it's possible some
joint effort could be mutually beneficial.
MPIR vs GMP
==========
One of MPIR's dirty little secrets is we abandon the GMP interface on
Windows 64, for good reason. They make all their _ui and _si function take
an unsigned long/long instead of an unsigned long long/long long, even
though limbs are 64 bits on Windows.
In some timings I did recently, this slows some operations down by a factor
of 6, and in Flint we have to supply a GMP compatibility header, so that we
can use GMP in the same way we use MPIR on Windows 64 (which is pretty time
consuming to write and obviously a performance sink).
As I'm not interested in just having something that works on Windows but
doesn't care about performance, I personally believe that such an effort
should be based on MPIR not GMP for the time being.
Final note
========
The Julia project targeted Windows very early on, and they have attracted
serious, extremely knowledgeable Windows developers (who have been very
helpful to me). I think this speaks for itself.
Bill.
[1] https://plus.google.com/115360165819500279592/posts/HiXvz9bjumu
[2] https://msys2.github.io/
[3] https://build.opensuse.org/
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.
