On 15 September 2014 07:38, Tony Kelman <t...@kelman.net> wrote:
> I've compiled many many dozens of libraries using MinGW-w64 on MSYS, and
> cross-compiling from Cygwin and various Linux distributions, and not
> encountered nearly as many problems as you seem to be. Pretending to be
> autotools but hacking it up yourself is the worst possible thing you could
> do - you're completely nonstandard and nonfunctional in cross-compilation
> for example, and you have as many or more dependencies - requiring bash for
> example is often not acceptable on FreeBSD and similar systems.
>
I take it you are referring to flint rather than MPIR here.
I was referring to hacking the MPIR autotools, which has been necessary
long before MPIR even existed. It was inherited from GMP and certainly
necessary for a multitude of reasons: making fat binaries, distinguishing
microprocessor microarchitectures, linking C code with assembly code cross
platform, compiling on Windows and OSX, excluding compilers which compile
everything else just fine but miscompile bignum libraries, tuning, etc,
etc, etc.
Autotools has always been unsuitable for such things at some level.
>
> Part of the problem is the way MSYS2 identifies itself changes depending
> on what environment variables you have set (or which bat file you start it
> with). The MSYS uname is unrecognized by autotools because MSYS2 is new,
> and you almost never want to actually be compiling with the MSYS host,
> since that causes your application to link to the msys-2.0.dll posix
> compatibility layer.
>
I will have to look into that. It doesn't sound like something we want.
Though I don't really understand why changing the host should change the
way gcc compiles code, unless it does so for cygwin itself.
> It's like using the standard gcc in Cygwin, any users would need Cygwin
> and the associated GPL posix runtime library to use the compiled code. The
> MINGW64 uname is also new and unrecognized, and totally nonstandard. This
> is why you should be querying the COMPILER for information about what
> system the generated code is supposed to run on, NOT the build environment.
>
It was my understanding that the Mingw compiler could be used from Cygwin.
Perhaps I am confused about it, but it seems like interrogating the
compiler may not give the information we need.
> The accepted GNU triples for MinGW are:
>
> i686-pc-mingw32: MinGW.org 32 bit compiler, outdated and obsolete, don't
> use this
> i686-w64-mingw32: 32 bit compiler from MinGW-w64 project, use this for 32
> bit host
> x86_64-w64-mingw32: 64 bit compiler from MinGW-w64 project, use this for
> 64 bit host
>
> Yes the names are confusing and make no sense, but this is the GNU
> standard that every other conforming project in the world is using. LLVM
> decided to go out on a limb and rearrange some of these names recently,
> adding to the confusion, but GCC, autotools, and MinGW are unlikely to
> follow their lead.
>
>
We can't use the accepted gnu triples in MPIR and that has always been the
case for GMP too. They don't give nearly enough information for our needs.
As far as MPIR is concerned the first part of the triple has to be the
microarchitecture of the CPU for assembly language purposes. Then we have
information about the platform/OS. If we reserve that for
microarchitecture, clearly we can't use the triples given above.
Canonicalising triples in MPIR is a two step process. Firstly we call the
standard FSF config.guess, plus any hacks necessary for systems that are
known to be incorrectly identified by the standard config.guess. We update
the standard config.guess every now and again.
We then have our own config.guess, which uses a completely different
convention which gives us the information we require. This has also always
been the case in GMP.
We also use a cpuid utility in conjunction with config.guess to give us
information about the microarchitecture.
Personally I'd like to get rid of autotools in MPIR altogether. It is much
more effort to maintain than it is worth, with thousands of lines of hacks.
And our main autotools maintainer died in 2012. I know much less about it.
>
> On Sunday, September 14, 2014 1:21:42 PM UTC-7, Bill Hart wrote:
>>
>> That was the problem. I've added hack number 40001 to the MPIR autotools
>> to fix this issue. Nemo now passes its tests on Window 64 for the first
>> time ever.
>>
>> Bill.
>>
>> On Sunday, 14 September 2014 21:40:45 UTC+2, Bill Hart wrote:
>>>
>>> I traced through the precise calls to libflint and libgmp that make it
>>> segfault and wrote a few test programs from the msys2 command line, taking
>>> Julia right out of the loop.
>>>
>>> At first I thought that it was only segfaulting when gmp was called from
>>> flint. But in fact I can make it segfault calling gmp directly from the
>>> program, though only in slightly strange circumstances.
>>>
>>> If I take the integer a = 123456789, I can compute the square and cube
>>> of this without problems. But if I try to compute the fourth power it
>>> segfaults.
>>>
>>> If I call gmp_printf it segfaults. And I checked that the mpz_t I'm
>>> trying to print contains valid data and that accessing it doesn't cause a
>>> segfault.
>>>
>>> In fact, the MPIR test suite fails most of its tests. In particular I
>>> notice that C only functions pass their tests, but assembly ones don't.
>>>
>>> I've got a feeling that msys have changed how it reports itself in a way
>>> that autotools hasn't kept up with, causing it to compile in linux assembly
>>> functions instead of Windows ones.
>>>
>>> Given that patching the linux assembly file and not the Windows one
>>> causes msys2 to not barf on that file, I'd say that's a certainty.
>>>
>>> Now I just have to hack autotools one more time to recognise msys2 and I
>>> should be ok. This is why I don't use autotools in flint.
>>>
>>> Bill.
>>>
>>> On Sunday, 14 September 2014 19:21:53 UTC+2, Bill Hart wrote:
>>>>
>>>> I checked that the MPIR dll's produced with the updated msys2 also
>>>> don't work with Julia 0.2.
>>>>
>>>> I can't think of many other variables here. It has to be msys2 related.
>>>> I could try not patching the assembly file it barfs on, but have it built
>>>> from C fallback code. But I know for a fact that file is not being used in
>>>> the functions I'm calling that cause it to segfault.
>>>>
>>>> I can try uninstalling msys2 completely and reinstalling it and mingw64
>>>> and see if I get any joy.
>>>>
>>>> On Sunday, 14 September 2014 19:05:55 UTC+2, Bill Hart wrote:
>>>>
>>>>>
>>>>>
>>>>> On 14 September 2014 17:46, Tony Kelman <to...@kelman.net> wrote:
>>>>>
>>>>>>
>>>>>> > Unfortunately it doesn't even work on my machine. It seems ok for
>>>>>> some calls into the dll, but as soon as I try to say print something
>>>>>> using
>>>>>> a function in the MPIR dll it segfaults. I suppose it must be linked
>>>>>> against subtly different Microsoft dll's than Julia and some kind of
>>>>>> conflict results.
>>>>>>
>>>>>> Or maybe different GCC dll's. Exactly which version of MinGW-w64 are
>>>>>> you using?
>>>>>>
>>>>>
>>>>> I've just updated the msys2 packages with pacman. Recently I updated
>>>>> just some of the packages and not others, and I thought this might have
>>>>> introduced some incompatibilities. I noticed that if I build MPIR even
>>>>> with
>>>>> msys2 and not via Julia, the resulting dll doesn't work any more.
>>>>>
>>>>> After updating msys2 fully to msys 2.0.1, it still doesn't work. So it
>>>>> looks to me like some new issues have been introduced by the more recent
>>>>> msys or something.
>>>>>
>>>>> Their gas also barfs on one of the assembly files which we now have to
>>>>> patch. This issue affects GMP too, not just MPIR since we use the same
>>>>> code
>>>>> for that file.
>>>>>
>>>>>
>>>>>> Might not be compatible with what was used to build Julia. Or there
>>>>>> could be some issue due to having gmp and mpir both loaded within the
>>>>>> same
>>>>>> process?
>>>>>>
>>>>>
>>>>> Just to reiterate, it was working fine before. I even went back to the
>>>>> precise configure invocation I used before in my bash history to ensure I
>>>>> was building MPIR the same way as before, and I checked which alpha
>>>>> version
>>>>> of MPIR I used.
>>>>>
>>>>> Admittedly I was using Julia 0.2.1 before, not Julia 0.3, which I have
>>>>> now upgraded to.
>>>>>
>>>>> I can try downloading Julia 0.2 for Windows again and see if that
>>>>> fixes the issue I guess.
>>>>>
>>>>>
>>>>>> We've seen sort-of-similar issues with blas and lapack and various
>>>>>> packages, though not so much on Windows.
>>>>>>
>>>>>> > The problem must be what libtool is doing on mingw64. Make install
>>>>>> doesn't even copy the generated dll across to the install directory, so
>>>>>> you
>>>>>> have to do this manually.
>>>>>>
>>>>>> Hm, yeah, libtool can be very picky, especially when it comes to
>>>>>> dll's. I think I've used the "subtle and quick to anger" quote on this
>>>>>> list
>>>>>> before when talking about libtool... I've found configuring
>>>>>> with lt_cv_deplibs_check_method=pass_all can sometimes help.
>>>>>>
>>>>>> > I also can't build flint against the Julia mpir and mpfr since
>>>>>> Julia doesn't seem to distribute the gmp.h and mpfr.h files, and flint
>>>>>> requires these when building.
>>>>>>
>>>>>> Oh, right, damn. Sorry I forgot about that! That is an issue, maybe
>>>>>> we should open a separate discussion on that general topic. It has come
>>>>>> up
>>>>>> before that various packages would like to link against the same
>>>>>> libraries
>>>>>> as Julia (in the Blas and Lapack cases we're building with an
>>>>>> incompatible
>>>>>> ABI which makes this actually not a good idea in most cases, but for GMP
>>>>>> and MPFR I think we're configuring them in the default way), so
>>>>>> installing
>>>>>> the corresponding headers would actually be necessary.
>>>>>>
>>>>>> Even though I'm not entirely sure what Nemo or Flint do or whether I
>>>>>> would personally have a use for them, I have some strange desire to see
>>>>>> more Julia packages be painless to install cross-platform and want to
>>>>>> help
>>>>>> here.
>>>>>>
>>>>>
>>>>> I can understand that.
>>>>>
>>>>>
>>>>>> Let me know how the runtime configuration of the text file location
>>>>>> goes,
>>>>>>
>>>>>
>>>>> It won't happen for a while. I'm afraid I'm currently 14 days behind
>>>>> schedule at work. I've spent days trying to get this to work on Windows
>>>>> and
>>>>> have just run out of time to keep working on it. I will have to come back
>>>>> to it when I catch up with everything else that has been piling up.
>>>>> Perhaps
>>>>> I can spend some more time on it next weekend.
>>>>>
>>>>>
>>>>>> then I can mock up what BinDeps would look like. It should simplify
>>>>>> your deps/build.jl script by automating the standard download, extract,
>>>>>> configure, make steps. There are also some Julia idioms like
>>>>>> joinpath("a","b") that would be cleaner than what you're doing now with
>>>>>> if
>>>>>> statements to switch between forward slashes and backslashes, and things
>>>>>> like
>>>>>>
>>>>>> cd(newdir) do
>>>>>> ...
>>>>>> end
>>>>>>
>>>>>> that work in a nicer way including returning to the old directory
>>>>>> even on failure.
>>>>>>
>>>>>>
>>>>>> Thanks, I will try to incorporate these suggestions in a later
>>>>> incarnation of the code.
>>>>>
>>>>> Bill.
>>>>>
>>>>
