Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Thanks for checking that.

I'll try to work on those things this weekend. The amount of time I'll have
to work on Nemo for the next two months will be smallish, but not
completely zero. So we'll get it there eventually.

Bill.

On 23 September 2014 21:16, Tony Kelman  wrote:

> Great. Bumping the git version to a51e15e53d636e38c107e48f1d8ec34a932590ab
> looks like it built okay:
> https://build.opensuse.org/package/show/home:kelman/mingw32-flint
>
> We'll need to adjust Nemo.jl's deps/build.jl script to use BinDeps
> Autotools/WinRPM providers and library_dependency's instead of just running
> shell commands. Will also want to use the library location caching method
> instead of push!'ing to DL_LOAD_PATH.
>
>
> On Tuesday, September 23, 2014 9:39:45 AM UTC-7, Bill Hart wrote:
>>
>> Sorry for the delay, but I have now added a function to flint which takes
>> the location of the CPimport.txt file (which is in the flint source tree in
>> the qadic directory) at runtime.
>>
>> The function is void flint_set_cpimport(char * path); where path is a
>> full path including filename of the (CPimport.txt) file.
>>
>> This is now committed to the flint trunk branch in my github. We should
>> be able to use this by calling it from Julia when initialising anything
>> that requires Conway polynomials. I haven't made the necessary changes to
>> Nemo yet, but most of Nemo works without the changes since only the finite
>> field functionality actually use Conway polynomials.
>>
>> This should allow us to move forward with a binary version of Nemo on
>> Windows, as soon as we decide a good place to install the CPimport.txt
>> file. I guess it could just sit in the Nemo directory and just be made part
>> of the Nemo git checkout.
>>
>> Bill.
>>
>> On Monday, 15 September 2014 14:45:01 UTC+2, Tony Kelman wrote:
>>>
>>> > I think this is very common. But it means that people who really want
>>> their code to actually compile on Windows frequently have an inferior
>>> system with which to do it, because all the real hackers are cross
>>> compiling from Linux.
>>>
>>> Or Cygwin. Part of the point is the exact same process should result in
>>> the same built binaries for cross-compilation from Linux or
>>> cross-compilation from Cygwin. Any complaint about Cygwin in terms of size
>>> or performance applies equally to MSYS2, and MSYS2's newness and multitude
>>> of build/run modes causes more trouble than it's worth in many cases.
>>>
>>> > or provide Visual Studio project files
>>>
>>> Can you blame them? Speaking of awful, impenetrable build systems that
>>> don't translate well to other platforms...
>>>
>>> > MPIR and GMP are *incredibly* complex projects and few people
>>> recognise the subtleties which simply don't arise for other types of
>>> software.
>>>
>>> I'm not familiar with writing this type of software, but plenty familiar
>>> with building it, and I really don't see a whole lot that goes into a
>>> bignum library that doesn't also happen in, say, an optimized BLAS/LAPACK
>>> implementation. Of course those are also notorious for having messy and
>>> fragile build systems.
>>>
>>>
>>>

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

Great. Bumping the git version to a51e15e53d636e38c107e48f1d8ec34a932590ab 
looks like it built 
okay: https://build.opensuse.org/package/show/home:kelman/mingw32-flint

We'll need to adjust Nemo.jl's deps/build.jl script to use BinDeps 
Autotools/WinRPM providers and library_dependency's instead of just running 
shell commands. Will also want to use the library location caching method 
instead of push!'ing to DL_LOAD_PATH.


On Tuesday, September 23, 2014 9:39:45 AM UTC-7, Bill Hart wrote:
>
> Sorry for the delay, but I have now added a function to flint which takes 
> the location of the CPimport.txt file (which is in the flint source tree in 
> the qadic directory) at runtime.
>
> The function is void flint_set_cpimport(char * path); where path is a full 
> path including filename of the (CPimport.txt) file.
>
> This is now committed to the flint trunk branch in my github. We should be 
> able to use this by calling it from Julia when initialising anything that 
> requires Conway polynomials. I haven't made the necessary changes to Nemo 
> yet, but most of Nemo works without the changes since only the finite field 
> functionality actually use Conway polynomials.
>
> This should allow us to move forward with a binary version of Nemo on 
> Windows, as soon as we decide a good place to install the CPimport.txt 
> file. I guess it could just sit in the Nemo directory and just be made part 
> of the Nemo git checkout.
>
> Bill.
>
> On Monday, 15 September 2014 14:45:01 UTC+2, Tony Kelman wrote:
>>
>> > I think this is very common. But it means that people who really want 
>> their code to actually compile on Windows frequently have an inferior 
>> system with which to do it, because all the real hackers are cross 
>> compiling from Linux.
>>
>> Or Cygwin. Part of the point is the exact same process should result in 
>> the same built binaries for cross-compilation from Linux or 
>> cross-compilation from Cygwin. Any complaint about Cygwin in terms of size 
>> or performance applies equally to MSYS2, and MSYS2's newness and multitude 
>> of build/run modes causes more trouble than it's worth in many cases.
>>
>> > or provide Visual Studio project files
>>
>> Can you blame them? Speaking of awful, impenetrable build systems that 
>> don't translate well to other platforms...
>>
>> > MPIR and GMP are *incredibly* complex projects and few people recognise 
>> the subtleties which simply don't arise for other types of software.
>>
>> I'm not familiar with writing this type of software, but plenty familiar 
>> with building it, and I really don't see a whole lot that goes into a 
>> bignum library that doesn't also happen in, say, an optimized BLAS/LAPACK 
>> implementation. Of course those are also notorious for having messy and 
>> fragile build systems.
>>
>>
>>

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Sorry for the delay, but I have now added a function to flint which takes 
the location of the CPimport.txt file (which is in the flint source tree in 
the qadic directory) at runtime.

The function is void flint_set_cpimport(char * path); where path is a full 
path including filename of the (CPimport.txt) file.

This is now committed to the flint trunk branch in my github. We should be 
able to use this by calling it from Julia when initialising anything that 
requires Conway polynomials. I haven't made the necessary changes to Nemo 
yet, but most of Nemo works without the changes since only the finite field 
functionality actually use Conway polynomials.

This should allow us to move forward with a binary version of Nemo on 
Windows, as soon as we decide a good place to install the CPimport.txt 
file. I guess it could just sit in the Nemo directory and just be made part 
of the Nemo git checkout.

Bill.

On Monday, 15 September 2014 14:45:01 UTC+2, Tony Kelman wrote:
>
> > I think this is very common. But it means that people who really want 
> their code to actually compile on Windows frequently have an inferior 
> system with which to do it, because all the real hackers are cross 
> compiling from Linux.
>
> Or Cygwin. Part of the point is the exact same process should result in 
> the same built binaries for cross-compilation from Linux or 
> cross-compilation from Cygwin. Any complaint about Cygwin in terms of size 
> or performance applies equally to MSYS2, and MSYS2's newness and multitude 
> of build/run modes causes more trouble than it's worth in many cases.
>
> > or provide Visual Studio project files
>
> Can you blame them? Speaking of awful, impenetrable build systems that 
> don't translate well to other platforms...
>
> > MPIR and GMP are *incredibly* complex projects and few people recognise 
> the subtleties which simply don't arise for other types of software.
>
> I'm not familiar with writing this type of software, but plenty familiar 
> with building it, and I really don't see a whole lot that goes into a 
> bignum library that doesn't also happen in, say, an optimized BLAS/LAPACK 
> implementation. Of course those are also notorious for having messy and 
> fragile build systems.
>
>
>

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

> I think this is very common. But it means that people who really want 
their code to actually compile on Windows frequently have an inferior 
system with which to do it, because all the real hackers are cross 
compiling from Linux.

Or Cygwin. Part of the point is the exact same process should result in the 
same built binaries for cross-compilation from Linux or cross-compilation 
from Cygwin. Any complaint about Cygwin in terms of size or performance 
applies equally to MSYS2, and MSYS2's newness and multitude of build/run 
modes causes more trouble than it's worth in many cases.

> or provide Visual Studio project files

Can you blame them? Speaking of awful, impenetrable build systems that 
don't translate well to other platforms...

> MPIR and GMP are *incredibly* complex projects and few people recognise 
the subtleties which simply don't arise for other types of software.

I'm not familiar with writing this type of software, but plenty familiar 
with building it, and I really don't see a whole lot that goes into a 
bignum library that doesn't also happen in, say, an optimized BLAS/LAPACK 
implementation. Of course those are also notorious for having messy and 
fragile build systems.

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

On 15 September 2014 13:39, Tony Kelman  wrote:

> > I will have to look into that. It doesn't sound like something we want.
> Though I don't really understand why changing the host should change the
> way gcc compiles code, unless it does so for cygwin itself.
>
> Changing the host changes which compiler gets used to create the binaries.
> In Cygwin or Linux, you can call the MinGW compilers by providing
> --host=x86_64-w64-mingw32 or similar, if you're using conventional
> autotools. It's also possible in Cmake but it's about 5 lines worth of
> flags (or a toolchain file with the same content) - it works, eventually,
> but it's really picky.
>

Sorry, I think I misunderstood what you were referring to when you said
"host". So my response doesn't make any sense in the way that you meant it.
I understand what you are getting at now.


>
> > It was my understanding that the Mingw compiler could be used from
> Cygwin. Perhaps I am confused about it, but it seems like interrogating the
> compiler may not give the information we need.
>
> Right, but when you configure with --host=x86_64-w64-mingw32 in Cygwin or
> Linux, you're using a cross-compiler that is prefixed by the host triple:
> x86_64-w64-mingw32-gcc and similar for binutils. So you can't just ask
> "gcc" for information - in a lot of ways MSYS(2) is a hack to avoid
> worrying about MinGW cross-compilation, and it leads to some bad ways of
> doing things. I think it's better to do the work to make a library possible
> to cross-compile, than to give up and say MSYS(2) is the only method of
> compiling it for Windows. I've been cross-compiling Julia itself and all of
> its dependencies from both Linux and Cygwin for months, I very rarely touch
> MSYS(2) any more.
>

I think this is very common. But it means that people who really want their
code to actually compile on Windows frequently have an inferior system with
which to do it, because all the real hackers are cross compiling from Linux.

There's nothing wrong in that, it just leads to additional frustration for
some people.


> > We can't use the accepted gnu triples in MPIR and that has always been
> the case for GMP too. They don't give nearly enough information for our
> needs.
>
> Of course they don't provide a complete story when you're worrying about
> assembly and various SIMD instruction sets. But that's not the GNU triple's
> job. That information should be determined and represented separately,
> you're not going to include this kind of information in the host prefix of
> the name of a cross-compiler executable for example.
>

Sure, but just to emphasise, we use the standard gnu triples in the first
level of config.guess (except when it's broken). The hack I applied was to
the second level, which uses the MPIR triples convention.


> GMP is also a pretty bad actor in terms of its autotools usage - making a
> symlink to libgmp.dll called libgmp.lib in lib/ is completely the wrong
> thing to do for MinGW.
>

As I understand it, few of their core developers even have access to
Windows boxes. They also reportedly don't support stack unwinding on
Windows (I've never actually checked this) or provide Visual Studio project
files. Their assembly code is just modified with lead-in and lead-out code
so they can use the same assembly code for Windows and Linux, despite the
differences in ABI. (It works relatively well, but it's a hack -- one that
we use quite a bit too, I confess).

One enormous frustration for us is that MSVC has no inline assembler on
Windows 64, and gas format assembly has not traditionally been available in
the assemblers available for Windows. There are many, many problems that we
work around that require many hacks, even on the linux side.

We could fix them all and make it less hackish, but you are talking about
something like a year or two of full time work, just to fix all the build
system issues alone!

Not your fault I know, but sounds like you may have inherited some
> autotools abuses from GMP.
>
>
>

Many of them are inherited from there. But remember that GMP has been in
development since at least 1994! They did things back then that many other
projects have not even thought about doing, to this day.

MPIR and GMP are *incredibly* complex projects and few people recognise the
subtleties which simply don't arise for other types of software.

Bill.

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

> I will have to look into that. It doesn't sound like something we want. 
Though I don't really understand why changing the host should change the 
way gcc compiles code, unless it does so for cygwin itself.

Changing the host changes which compiler gets used to create the binaries. 
In Cygwin or Linux, you can call the MinGW compilers by providing 
--host=x86_64-w64-mingw32 or similar, if you're using conventional 
autotools. It's also possible in Cmake but it's about 5 lines worth of 
flags (or a toolchain file with the same content) - it works, eventually, 
but it's really picky. 

> It was my understanding that the Mingw compiler could be used from 
Cygwin. Perhaps I am confused about it, but it seems like interrogating the 
compiler may not give the information we need.

Right, but when you configure with --host=x86_64-w64-mingw32 in Cygwin or 
Linux, you're using a cross-compiler that is prefixed by the host triple: 
x86_64-w64-mingw32-gcc and similar for binutils. So you can't just ask 
"gcc" for information - in a lot of ways MSYS(2) is a hack to avoid 
worrying about MinGW cross-compilation, and it leads to some bad ways of 
doing things. I think it's better to do the work to make a library possible 
to cross-compile, than to give up and say MSYS(2) is the only method of 
compiling it for Windows. I've been cross-compiling Julia itself and all of 
its dependencies from both Linux and Cygwin for months, I very rarely touch 
MSYS(2) any more.

> We can't use the accepted gnu triples in MPIR and that has always been 
the case for GMP too. They don't give nearly enough information for our 
needs.

Of course they don't provide a complete story when you're worrying about 
assembly and various SIMD instruction sets. But that's not the GNU triple's 
job. That information should be determined and represented separately, 
you're not going to include this kind of information in the host prefix of 
the name of a cross-compiler executable for example. GMP is also a pretty 
bad actor in terms of its autotools usage - making a symlink to libgmp.dll 
called libgmp.lib in lib/ is completely the wrong thing to do for MinGW. 
Not your fault I know, but sounds like you may have inherited some 
autotools abuses from GMP.
 
 

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

On 15 September 2014 07:38, Tony Kelman  wrote:

> I've compiled many many dozens of libraries using MinGW-w64 on MSYS, and
> cross-compiling from Cygwin and various Linux distributions, and not
> encountered nearly as many problems as you seem to be. Pretending to be
> autotools but hacking it up yourself is the worst possible thing you could
> do - you're completely nonstandard and nonfunctional in cross-compilation
> for example, and you have as many or more dependencies - requiring bash for
> example is often not acceptable on FreeBSD and similar systems.
>

I take it you are referring to flint rather than MPIR here.

I was referring to hacking the MPIR autotools, which has been necessary
long before MPIR even existed. It was inherited from GMP and certainly
necessary for a multitude of reasons: making fat binaries, distinguishing
microprocessor microarchitectures, linking C code with assembly code cross
platform, compiling on Windows and OSX, excluding compilers which compile
everything else just fine but miscompile bignum libraries, tuning, etc,
etc, etc.

Autotools has always been unsuitable for such things at some level.


>
> Part of the problem is the way MSYS2 identifies itself changes depending
> on what environment variables you have set (or which bat file you start it
> with). The MSYS uname is unrecognized by autotools because MSYS2 is new,
> and you almost never want to actually be compiling with the MSYS host,
> since that causes your application to link to the msys-2.0.dll posix
> compatibility layer.
>

I will have to look into that. It doesn't sound like something we want.
Though I don't really understand why changing the host should change the
way gcc compiles code, unless it does so for cygwin itself.


> It's like using the standard gcc in Cygwin, any users would need Cygwin
> and the associated GPL posix runtime library to use the compiled code. The
> MINGW64 uname is also new and unrecognized, and totally nonstandard. This
> is why you should be querying the COMPILER for information about what
> system the generated code is supposed to run on, NOT the build environment.
>

It was my understanding that the Mingw compiler could be used from Cygwin.
Perhaps I am confused about it, but it seems like interrogating the
compiler may not give the information we need.


> The accepted GNU triples for MinGW are:
>
> i686-pc-mingw32: MinGW.org 32 bit compiler, outdated and obsolete, don't
> use this
> i686-w64-mingw32: 32 bit compiler from MinGW-w64 project, use this for 32
> bit host
> x86_64-w64-mingw32: 64 bit compiler from MinGW-w64 project, use this for
> 64 bit host
>
> Yes the names are confusing and make no sense, but this is the GNU
> standard that every other conforming project in the world is using. LLVM
> decided to go out on a limb and rearrange some of these names recently,
> adding to the confusion, but GCC, autotools, and MinGW are unlikely to
> follow their lead.
>
>
We can't use the accepted gnu triples in MPIR and that has always been the
case for GMP too. They don't give nearly enough information for our needs.

As far as MPIR is concerned the first part of the triple has to be the
microarchitecture of the CPU for assembly language purposes. Then we have
information about the platform/OS. If we reserve that for
microarchitecture, clearly we can't use the triples given above.

Canonicalising triples in MPIR is a two step process. Firstly we call the
standard FSF config.guess, plus any hacks necessary for systems that are
known to be incorrectly identified by the standard config.guess. We update
the standard config.guess every now and again.

We then have our own config.guess, which uses a completely different
convention which gives us the information we require. This has also always
been the case in GMP.

We also use a cpuid utility in conjunction with config.guess to give us
information about the microarchitecture.

Personally I'd like to get rid of autotools in MPIR altogether. It is much
more effort to maintain than it is worth, with thousands of lines of hacks.
And our main autotools maintainer died in 2012. I know much less about it.


>
> On Sunday, September 14, 2014 1:21:42 PM UTC-7, Bill Hart wrote:
>>
>> That was the problem. I've added hack number 40001 to the MPIR autotools
>> to fix this issue. Nemo now passes its tests on Window 64 for the first
>> time ever.
>>
>> Bill.
>>
>> On Sunday, 14 September 2014 21:40:45 UTC+2, Bill Hart wrote:
>>>
>>> I traced through the precise calls to libflint and libgmp that make it
>>> segfault and wrote a few test programs from the msys2 command line, taking
>>> Julia right out of the loop.
>>>
>>> At first I thought that it was only segfaulting when gmp was called from
>>> flint. But in fact I can make it segfault calling gmp directly from the
>>> program, though only in slightly strange circumstances.
>>>
>>> If I take the integer a = 123456789, I can compute the square and cube
>>> of this without problems. But if I try to

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

I've compiled many many dozens of libraries using MinGW-w64 on MSYS, and 
cross-compiling from Cygwin and various Linux distributions, and not 
encountered nearly as many problems as you seem to be. Pretending to be 
autotools but hacking it up yourself is the worst possible thing you could 
do - you're completely nonstandard and nonfunctional in cross-compilation 
for example, and you have as many or more dependencies - requiring bash for 
example is often not acceptable on FreeBSD and similar systems.

Part of the problem is the way MSYS2 identifies itself changes depending on 
what environment variables you have set (or which bat file you start it 
with). The MSYS uname is unrecognized by autotools because MSYS2 is new, 
and you almost never want to actually be compiling with the MSYS host, 
since that causes your application to link to the msys-2.0.dll posix 
compatibility layer. It's like using the standard gcc in Cygwin, any users 
would need Cygwin and the associated GPL posix runtime library to use the 
compiled code. The MINGW64 uname is also new and unrecognized, and totally 
nonstandard. This is why you should be querying the COMPILER for 
information about what system the generated code is supposed to run on, NOT 
the build environment. The accepted GNU triples for MinGW are:

i686-pc-mingw32: MinGW.org 32 bit compiler, outdated and obsolete, don't 
use this
i686-w64-mingw32: 32 bit compiler from MinGW-w64 project, use this for 32 
bit host
x86_64-w64-mingw32: 64 bit compiler from MinGW-w64 project, use this for 64 
bit host

Yes the names are confusing and make no sense, but this is the GNU standard 
that every other conforming project in the world is using. LLVM decided to 
go out on a limb and rearrange some of these names recently, adding to the 
confusion, but GCC, autotools, and MinGW are unlikely to follow their lead.



On Sunday, September 14, 2014 1:21:42 PM UTC-7, Bill Hart wrote:
>
> That was the problem. I've added hack number 40001 to the MPIR autotools 
> to fix this issue. Nemo now passes its tests on Window 64 for the first 
> time ever.
>
> Bill.
>
> On Sunday, 14 September 2014 21:40:45 UTC+2, Bill Hart wrote:
>>
>> I traced through the precise calls to libflint and libgmp that make it 
>> segfault and wrote a few test programs from the msys2 command line, taking 
>> Julia right out of the loop.
>>
>> At first I thought that it was only segfaulting when gmp was called from 
>> flint. But in fact I can make it segfault calling gmp directly from the 
>> program, though only in slightly strange circumstances.
>>
>> If I take the integer a = 123456789, I can compute the square and cube of 
>> this without problems. But if I try to compute the fourth power it 
>> segfaults.
>>
>> If I call gmp_printf it segfaults. And I checked that the mpz_t I'm 
>> trying to print contains valid data and that accessing it doesn't cause a 
>> segfault.
>>
>> In fact, the MPIR test suite fails most of its tests. In particular I 
>> notice that C only functions pass their tests, but assembly ones don't.
>>
>> I've got a feeling that msys have changed how it reports itself in a way 
>> that autotools hasn't kept up with, causing it to compile in linux assembly 
>> functions instead of Windows ones.
>>
>> Given that patching the linux assembly file and not the Windows one 
>> causes msys2 to not barf on that file, I'd say that's a certainty.
>>
>> Now I just have to hack autotools one more time to recognise msys2 and I 
>> should be ok. This is why I don't use autotools in flint.
>>
>> Bill.
>>
>> On Sunday, 14 September 2014 19:21:53 UTC+2, Bill Hart wrote:
>>>
>>> I checked that the MPIR dll's produced with the updated msys2 also don't 
>>> work with Julia 0.2.
>>>
>>> I can't think of many other variables here. It has to be msys2 related. 
>>> I could try not patching the assembly file it barfs on, but have it built 
>>> from C fallback code. But I know for a fact that file is not being used in 
>>> the functions I'm calling that cause it to segfault.
>>>
>>> I can try uninstalling msys2 completely and reinstalling it and mingw64 
>>> and see if I get any joy.
>>>
>>> On Sunday, 14 September 2014 19:05:55 UTC+2, Bill Hart wrote:



 On 14 September 2014 17:46, Tony Kelman 
 > wrote:

>
> > Unfortunately it doesn't even work on my machine. It seems ok for 
> some calls into the dll, but as soon as I try to say print something 
> using 
> a function in the MPIR dll it segfaults. I suppose it must be linked 
> against subtly different Microsoft dll's than Julia and some kind of 
> conflict results.
>
> Or maybe different GCC dll's. Exactly which version of MinGW-w64 are 
> you using? 
>

 I've just updated the msys2 packages with pacman. Recently I updated 
 just some of the packages and not others, and I thought this might have 
 introduced some incompatibilities. I noticed that if I build MPIR even 
 with 
 

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

That was the problem. I've added hack number 40001 to the MPIR autotools to 
fix this issue. Nemo now passes its tests on Window 64 for the first time 
ever.

Bill.

On Sunday, 14 September 2014 21:40:45 UTC+2, Bill Hart wrote:
>
> I traced through the precise calls to libflint and libgmp that make it 
> segfault and wrote a few test programs from the msys2 command line, taking 
> Julia right out of the loop.
>
> At first I thought that it was only segfaulting when gmp was called from 
> flint. But in fact I can make it segfault calling gmp directly from the 
> program, though only in slightly strange circumstances.
>
> If I take the integer a = 123456789, I can compute the square and cube of 
> this without problems. But if I try to compute the fourth power it 
> segfaults.
>
> If I call gmp_printf it segfaults. And I checked that the mpz_t I'm trying 
> to print contains valid data and that accessing it doesn't cause a segfault.
>
> In fact, the MPIR test suite fails most of its tests. In particular I 
> notice that C only functions pass their tests, but assembly ones don't.
>
> I've got a feeling that msys have changed how it reports itself in a way 
> that autotools hasn't kept up with, causing it to compile in linux assembly 
> functions instead of Windows ones.
>
> Given that patching the linux assembly file and not the Windows one causes 
> msys2 to not barf on that file, I'd say that's a certainty.
>
> Now I just have to hack autotools one more time to recognise msys2 and I 
> should be ok. This is why I don't use autotools in flint.
>
> Bill.
>
> On Sunday, 14 September 2014 19:21:53 UTC+2, Bill Hart wrote:
>>
>> I checked that the MPIR dll's produced with the updated msys2 also don't 
>> work with Julia 0.2.
>>
>> I can't think of many other variables here. It has to be msys2 related. I 
>> could try not patching the assembly file it barfs on, but have it built 
>> from C fallback code. But I know for a fact that file is not being used in 
>> the functions I'm calling that cause it to segfault.
>>
>> I can try uninstalling msys2 completely and reinstalling it and mingw64 
>> and see if I get any joy.
>>
>> On Sunday, 14 September 2014 19:05:55 UTC+2, Bill Hart wrote:
>>>
>>>
>>>
>>> On 14 September 2014 17:46, Tony Kelman  wrote:
>>>

 > Unfortunately it doesn't even work on my machine. It seems ok for 
 some calls into the dll, but as soon as I try to say print something using 
 a function in the MPIR dll it segfaults. I suppose it must be linked 
 against subtly different Microsoft dll's than Julia and some kind of 
 conflict results.

 Or maybe different GCC dll's. Exactly which version of MinGW-w64 are 
 you using? 

>>>
>>> I've just updated the msys2 packages with pacman. Recently I updated 
>>> just some of the packages and not others, and I thought this might have 
>>> introduced some incompatibilities. I noticed that if I build MPIR even with 
>>> msys2 and not via Julia, the resulting dll doesn't work any more.
>>>
>>> After updating msys2 fully to msys 2.0.1, it still doesn't work. So it 
>>> looks to me like some new issues have been introduced by the more recent 
>>> msys or something.
>>>
>>> Their gas also barfs on one of the assembly files which we now have to 
>>> patch. This issue affects GMP too, not just MPIR since we use the same code 
>>> for that file.
>>>  
>>>
 Might not be compatible with what was used to build Julia. Or there 
 could be some issue due to having gmp and mpir both loaded within the same 
 process? 

>>>
>>> Just to reiterate, it was working fine before. I even went back to the 
>>> precise configure invocation I used before in my bash history to ensure I 
>>> was building MPIR the same way as before, and I checked which alpha version 
>>> of MPIR I used.
>>>
>>> Admittedly I was using Julia 0.2.1 before, not Julia 0.3, which I have 
>>> now upgraded to.
>>>
>>> I can try downloading Julia 0.2 for Windows again and see if that fixes 
>>> the issue I guess.
>>>  
>>>
 We've seen sort-of-similar issues with blas and lapack and various 
 packages, though not so much on Windows.

 > The problem must be what libtool is doing on mingw64. Make install 
 doesn't even copy the generated dll across to the install directory, so 
 you 
 have to do this manually. 

 Hm, yeah, libtool can be very picky, especially when it comes to dll's. 
 I think I've used the "subtle and quick to anger" quote on this list 
 before 
 when talking about libtool... I've found configuring 
 with lt_cv_deplibs_check_method=pass_all can sometimes help.

 > I also can't build flint against the Julia mpir and mpfr since Julia 
 doesn't seem to distribute the gmp.h and mpfr.h files, and flint requires 
 these when building.

 Oh, right, damn. Sorry I forgot about that! That is an issue, maybe we 
 should open a separate discussion on that general topic. It has

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

I traced through the precise calls to libflint and libgmp that make it 
segfault and wrote a few test programs from the msys2 command line, taking 
Julia right out of the loop.

At first I thought that it was only segfaulting when gmp was called from 
flint. But in fact I can make it segfault calling gmp directly from the 
program, though only in slightly strange circumstances.

If I take the integer a = 123456789, I can compute the square and cube of 
this without problems. But if I try to compute the fourth power it 
segfaults.

If I call gmp_printf it segfaults. And I checked that the mpz_t I'm trying 
to print contains valid data and that accessing it doesn't cause a segfault.

In fact, the MPIR test suite fails most of its tests. In particular I 
notice that C only functions pass their tests, but assembly ones don't.

I've got a feeling that msys have changed how it reports itself in a way 
that autotools hasn't kept up with, causing it to compile in linux assembly 
functions instead of Windows ones.

Given that patching the linux assembly file and not the Windows one causes 
msys2 to not barf on that file, I'd say that's a certainty.

Now I just have to hack autotools one more time to recognise msys2 and I 
should be ok. This is why I don't use autotools in flint.

Bill.

On Sunday, 14 September 2014 19:21:53 UTC+2, Bill Hart wrote:
>
> I checked that the MPIR dll's produced with the updated msys2 also don't 
> work with Julia 0.2.
>
> I can't think of many other variables here. It has to be msys2 related. I 
> could try not patching the assembly file it barfs on, but have it built 
> from C fallback code. But I know for a fact that file is not being used in 
> the functions I'm calling that cause it to segfault.
>
> I can try uninstalling msys2 completely and reinstalling it and mingw64 
> and see if I get any joy.
>
> On Sunday, 14 September 2014 19:05:55 UTC+2, Bill Hart wrote:
>>
>>
>>
>> On 14 September 2014 17:46, Tony Kelman  wrote:
>>
>>>
>>> > Unfortunately it doesn't even work on my machine. It seems ok for some 
>>> calls into the dll, but as soon as I try to say print something using a 
>>> function in the MPIR dll it segfaults. I suppose it must be linked against 
>>> subtly different Microsoft dll's than Julia and some kind of conflict 
>>> results.
>>>
>>> Or maybe different GCC dll's. Exactly which version of MinGW-w64 are you 
>>> using? 
>>>
>>
>> I've just updated the msys2 packages with pacman. Recently I updated just 
>> some of the packages and not others, and I thought this might have 
>> introduced some incompatibilities. I noticed that if I build MPIR even with 
>> msys2 and not via Julia, the resulting dll doesn't work any more.
>>
>> After updating msys2 fully to msys 2.0.1, it still doesn't work. So it 
>> looks to me like some new issues have been introduced by the more recent 
>> msys or something.
>>
>> Their gas also barfs on one of the assembly files which we now have to 
>> patch. This issue affects GMP too, not just MPIR since we use the same code 
>> for that file.
>>  
>>
>>> Might not be compatible with what was used to build Julia. Or there 
>>> could be some issue due to having gmp and mpir both loaded within the same 
>>> process? 
>>>
>>
>> Just to reiterate, it was working fine before. I even went back to the 
>> precise configure invocation I used before in my bash history to ensure I 
>> was building MPIR the same way as before, and I checked which alpha version 
>> of MPIR I used.
>>
>> Admittedly I was using Julia 0.2.1 before, not Julia 0.3, which I have 
>> now upgraded to.
>>
>> I can try downloading Julia 0.2 for Windows again and see if that fixes 
>> the issue I guess.
>>  
>>
>>> We've seen sort-of-similar issues with blas and lapack and various 
>>> packages, though not so much on Windows.
>>>
>>> > The problem must be what libtool is doing on mingw64. Make install 
>>> doesn't even copy the generated dll across to the install directory, so you 
>>> have to do this manually. 
>>>
>>> Hm, yeah, libtool can be very picky, especially when it comes to dll's. 
>>> I think I've used the "subtle and quick to anger" quote on this list before 
>>> when talking about libtool... I've found configuring 
>>> with lt_cv_deplibs_check_method=pass_all can sometimes help.
>>>
>>> > I also can't build flint against the Julia mpir and mpfr since Julia 
>>> doesn't seem to distribute the gmp.h and mpfr.h files, and flint requires 
>>> these when building.
>>>
>>> Oh, right, damn. Sorry I forgot about that! That is an issue, maybe we 
>>> should open a separate discussion on that general topic. It has come up 
>>> before that various packages would like to link against the same libraries 
>>> as Julia (in the Blas and Lapack cases we're building with an incompatible 
>>> ABI which makes this actually not a good idea in most cases, but for GMP 
>>> and MPFR I think we're configuring them in the default way), so installing 
>>> the corresponding headers would actually 

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

I checked that the MPIR dll's produced with the updated msys2 also don't 
work with Julia 0.2.

I can't think of many other variables here. It has to be msys2 related. I 
could try not patching the assembly file it barfs on, but have it built 
from C fallback code. But I know for a fact that file is not being used in 
the functions I'm calling that cause it to segfault.

I can try uninstalling msys2 completely and reinstalling it and mingw64 and 
see if I get any joy.

On Sunday, 14 September 2014 19:05:55 UTC+2, Bill Hart wrote:
>
>
>
> On 14 September 2014 17:46, Tony Kelman  wrote:
>
>>
>> > Unfortunately it doesn't even work on my machine. It seems ok for some 
>> calls into the dll, but as soon as I try to say print something using a 
>> function in the MPIR dll it segfaults. I suppose it must be linked against 
>> subtly different Microsoft dll's than Julia and some kind of conflict 
>> results.
>>
>> Or maybe different GCC dll's. Exactly which version of MinGW-w64 are you 
>> using? 
>>
>
> I've just updated the msys2 packages with pacman. Recently I updated just 
> some of the packages and not others, and I thought this might have 
> introduced some incompatibilities. I noticed that if I build MPIR even with 
> msys2 and not via Julia, the resulting dll doesn't work any more.
>
> After updating msys2 fully to msys 2.0.1, it still doesn't work. So it 
> looks to me like some new issues have been introduced by the more recent 
> msys or something.
>
> Their gas also barfs on one of the assembly files which we now have to 
> patch. This issue affects GMP too, not just MPIR since we use the same code 
> for that file.
>  
>
>> Might not be compatible with what was used to build Julia. Or there could 
>> be some issue due to having gmp and mpir both loaded within the same 
>> process? 
>>
>
> Just to reiterate, it was working fine before. I even went back to the 
> precise configure invocation I used before in my bash history to ensure I 
> was building MPIR the same way as before, and I checked which alpha version 
> of MPIR I used.
>
> Admittedly I was using Julia 0.2.1 before, not Julia 0.3, which I have now 
> upgraded to.
>
> I can try downloading Julia 0.2 for Windows again and see if that fixes 
> the issue I guess.
>  
>
>> We've seen sort-of-similar issues with blas and lapack and various 
>> packages, though not so much on Windows.
>>
>> > The problem must be what libtool is doing on mingw64. Make install 
>> doesn't even copy the generated dll across to the install directory, so you 
>> have to do this manually. 
>>
>> Hm, yeah, libtool can be very picky, especially when it comes to dll's. I 
>> think I've used the "subtle and quick to anger" quote on this list before 
>> when talking about libtool... I've found configuring 
>> with lt_cv_deplibs_check_method=pass_all can sometimes help.
>>
>> > I also can't build flint against the Julia mpir and mpfr since Julia 
>> doesn't seem to distribute the gmp.h and mpfr.h files, and flint requires 
>> these when building.
>>
>> Oh, right, damn. Sorry I forgot about that! That is an issue, maybe we 
>> should open a separate discussion on that general topic. It has come up 
>> before that various packages would like to link against the same libraries 
>> as Julia (in the Blas and Lapack cases we're building with an incompatible 
>> ABI which makes this actually not a good idea in most cases, but for GMP 
>> and MPFR I think we're configuring them in the default way), so installing 
>> the corresponding headers would actually be necessary.
>>
>> Even though I'm not entirely sure what Nemo or Flint do or whether I 
>> would personally have a use for them, I have some strange desire to see 
>> more Julia packages be painless to install cross-platform and want to help 
>> here. 
>>
>
> I can understand that.
>  
>
>> Let me know how the runtime configuration of the text file location goes, 
>>
>
> It won't happen for a while. I'm afraid I'm currently 14 days behind 
> schedule at work. I've spent days trying to get this to work on Windows and 
> have just run out of time to keep working on it. I will have to come back 
> to it when I catch up with everything else that has been piling up. Perhaps 
> I can spend some more time on it next weekend.
>  
>
>> then I can mock up what BinDeps would look like. It should simplify your 
>> deps/build.jl script by automating the standard download, extract, 
>> configure, make steps. There are also some Julia idioms like 
>> joinpath("a","b") that would be cleaner than what you're doing now with if 
>> statements to switch between forward slashes and backslashes, and things 
>> like
>>
>> cd(newdir) do
>> ...
>> end
>>
>> that work in a nicer way including returning to the old directory even on 
>> failure.
>>
>>
>> Thanks, I will try to incorporate these suggestions in a later 
> incarnation of the code.
>
> Bill.
>

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

On 14 September 2014 17:46, Tony Kelman  wrote:

>
> > Unfortunately it doesn't even work on my machine. It seems ok for some
> calls into the dll, but as soon as I try to say print something using a
> function in the MPIR dll it segfaults. I suppose it must be linked against
> subtly different Microsoft dll's than Julia and some kind of conflict
> results.
>
> Or maybe different GCC dll's. Exactly which version of MinGW-w64 are you
> using?
>

I've just updated the msys2 packages with pacman. Recently I updated just
some of the packages and not others, and I thought this might have
introduced some incompatibilities. I noticed that if I build MPIR even with
msys2 and not via Julia, the resulting dll doesn't work any more.

After updating msys2 fully to msys 2.0.1, it still doesn't work. So it
looks to me like some new issues have been introduced by the more recent
msys or something.

Their gas also barfs on one of the assembly files which we now have to
patch. This issue affects GMP too, not just MPIR since we use the same code
for that file.


> Might not be compatible with what was used to build Julia. Or there could
> be some issue due to having gmp and mpir both loaded within the same
> process?
>

Just to reiterate, it was working fine before. I even went back to the
precise configure invocation I used before in my bash history to ensure I
was building MPIR the same way as before, and I checked which alpha version
of MPIR I used.

Admittedly I was using Julia 0.2.1 before, not Julia 0.3, which I have now
upgraded to.

I can try downloading Julia 0.2 for Windows again and see if that fixes the
issue I guess.


> We've seen sort-of-similar issues with blas and lapack and various
> packages, though not so much on Windows.
>
> > The problem must be what libtool is doing on mingw64. Make install
> doesn't even copy the generated dll across to the install directory, so you
> have to do this manually.
>
> Hm, yeah, libtool can be very picky, especially when it comes to dll's. I
> think I've used the "subtle and quick to anger" quote on this list before
> when talking about libtool... I've found configuring
> with lt_cv_deplibs_check_method=pass_all can sometimes help.
>
> > I also can't build flint against the Julia mpir and mpfr since Julia
> doesn't seem to distribute the gmp.h and mpfr.h files, and flint requires
> these when building.
>
> Oh, right, damn. Sorry I forgot about that! That is an issue, maybe we
> should open a separate discussion on that general topic. It has come up
> before that various packages would like to link against the same libraries
> as Julia (in the Blas and Lapack cases we're building with an incompatible
> ABI which makes this actually not a good idea in most cases, but for GMP
> and MPFR I think we're configuring them in the default way), so installing
> the corresponding headers would actually be necessary.
>
> Even though I'm not entirely sure what Nemo or Flint do or whether I would
> personally have a use for them, I have some strange desire to see more
> Julia packages be painless to install cross-platform and want to help here.
>

I can understand that.


> Let me know how the runtime configuration of the text file location goes,
>

It won't happen for a while. I'm afraid I'm currently 14 days behind
schedule at work. I've spent days trying to get this to work on Windows and
have just run out of time to keep working on it. I will have to come back
to it when I catch up with everything else that has been piling up. Perhaps
I can spend some more time on it next weekend.


> then I can mock up what BinDeps would look like. It should simplify your
> deps/build.jl script by automating the standard download, extract,
> configure, make steps. There are also some Julia idioms like
> joinpath("a","b") that would be cleaner than what you're doing now with if
> statements to switch between forward slashes and backslashes, and things
> like
>
> cd(newdir) do
> ...
> end
>
> that work in a nicer way including returning to the old directory even on
> failure.
>
>
> Thanks, I will try to incorporate these suggestions in a later incarnation
of the code.

Bill.

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

> Of course you can pass an explicit directory to flint's configure to tell 
it where to put this text file. Perhaps there is some canonical place.

Julia has settled on some good default choices for where to put packages by 
now, the problem is users are free to override the default locations and 
can have good reasons for wanting to do so.

> The Free Software Foundation explicitly says that "bindings" from a 
programming language interpreter, not just linking, constitute a derived 
work and therefore the combination becomes GPL'd. They are very explicit 
about this on their website.
> 
> I also had someone write to me the other day and refuse to access the 
source code via Github because they "refused to indemnify Github against 
all damages". It's not clear if I am required under the GPL to send them 
the source code some other way...

Oh dear. Well what I've been doing when building binaries is including the 
Readme and the License files as part of the packaging, so those will always 
get installed along with the dll's.

> We also need to either include the full source code of flint with the 
binary, or a written offer to supply it upon request. Here the FSF claims a 
link to a website is not really good enough because the URL might change. 
As a result of this, I am personally very reluctant to not just supply the 
source code for flint. I also want people to have the git repository 
automatically so they can contribute to it if they want.

Okay. It's not the end of the world to checkout the source code even when 
downloading a binary. Though keep in mind that for Julia 0.4, the plan is 
to transition Julia's package manager from relying on shelling out to 
command-line git to using ccall bindings to libgit2, so it won't 
necessarily be a safe assumption to make that you can always shell out to 
git. That's still a ways out though, and you can always put a try-catch 
around the checkout since it's not an absolute requirement.

> We're also not yet fully in compliance with the GPL. Because it is an 
interpreted environment GPL v2 (which flint uses) requires a notice to be 
printed when Nemo starts up saying that the program comes with no warranty. 
I'll have to add that, if I can figure out how.

That should be pretty easy with a println in your Nemo.jl module file, 
should come up any time anyone calls "using Nemo". You can also write a 
function called __init__() that gets called on package startup. Not sure 
if/where in the docs that's listed.

> Unfortunately it doesn't even work on my machine. It seems ok for some 
calls into the dll, but as soon as I try to say print something using a 
function in the MPIR dll it segfaults. I suppose it must be linked against 
subtly different Microsoft dll's than Julia and some kind of conflict 
results.

Or maybe different GCC dll's. Exactly which version of MinGW-w64 are you 
using? Might not be compatible with what was used to build Julia. Or there 
could be some issue due to having gmp and mpir both loaded within the same 
process? We've seen sort-of-similar issues with blas and lapack and various 
packages, though not so much on Windows.

> The problem must be what libtool is doing on mingw64. Make install 
doesn't even copy the generated dll across to the install directory, so you 
have to do this manually. 

Hm, yeah, libtool can be very picky, especially when it comes to dll's. I 
think I've used the "subtle and quick to anger" quote on this list before 
when talking about libtool... I've found configuring 
with lt_cv_deplibs_check_method=pass_all can sometimes help.

> I also can't build flint against the Julia mpir and mpfr since Julia 
doesn't seem to distribute the gmp.h and mpfr.h files, and flint requires 
these when building.

Oh, right, damn. Sorry I forgot about that! That is an issue, maybe we 
should open a separate discussion on that general topic. It has come up 
before that various packages would like to link against the same libraries 
as Julia (in the Blas and Lapack cases we're building with an incompatible 
ABI which makes this actually not a good idea in most cases, but for GMP 
and MPFR I think we're configuring them in the default way), so installing 
the corresponding headers would actually be necessary.

Even though I'm not entirely sure what Nemo or Flint do or whether I would 
personally have a use for them, I have some strange desire to see more 
Julia packages be painless to install cross-platform and want to help here. 
Let me know how the runtime configuration of the text file location goes, 
then I can mock up what BinDeps would look like. It should simplify your 
deps/build.jl script by automating the standard download, extract, 
configure, make steps. There are also some Julia idioms like 
joinpath("a","b") that would be cleaner than what you're doing now with if 
statements to switch between forward slashes and backslashes, and things 
like

cd(newdir) do
...
end

that work in a nicer way including returning to the o

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

On Sunday, 14 September 2014 15:52:58 UTC+2, Bill Hart wrote:
>
>
>
> On 14 September 2014 06:14, Tony Kelman  wrote:
>
>> > We tend not to use tags. But there's no problem introducing a named tag 
>> to pin things.
>>
>> Any reason why not? Your code is on github but your releases aren't, 
>> would be great if it was possible to track exactly which commits correspond 
>> to exactly which numbered release version.
>>
>
> We have been using branches, not tags for that. We like to only push fixes 
> to release branches, whilst work continues in trunk, and make those minor 
> point releases, 2.4.0, 2.4.1. 2.4.2, etc. 
>
> I suppose we could tag all the releases in their branches too.
>  
>
>>
>> > The way it currently works is the user will specify --prefix. This is 
>> where libflint will be installed (in a subdirectory called lib). Obviously 
>> a default is chosen otherwise.
>> > 
>> > Flint is at first built inside its own source tree. Flint knows where 
>> to find the file relative to this source tree.
>> > 
>> > But when make install is issued, it will be moved across to --prefix.
>> > 
>> > As flint is compiled, --prefix is used to generate an absolute path 
>> where the text file will be stored, and this is baked into the library, 
>> which then also looks in this absolute path.
>> > 
>> > Of course the absolute path is computed at compile time to be some path 
>> relative to --prefix. It is passed to the relevant parts of flint via a -D 
>> define.
>>
>> None of this is going to work if you want the library to be usable on a 
>> different machine than it gets compiled on. That's a pretty big restriction 
>> to put on your users. A bit like using Gentoo - some people think it's fun 
>> out of masochism or something, but not many. Most users would rather save 
>> time and use binaries wherever possible (assuming the results are the same 
>> and you don't introduce any compatibility problems - if you use an 
>> automated build service and a package manager these are easy).
>>
>
> We'd like to fix the problem. We just don't know how. At some point, the 
> library has to read from a large text file. There must be a relocatable way 
> of doing that.
>
> Of course you can pass an explicit directory to flint's configure to tell 
> it where to put this text file. Perhaps there is some canonical place.
>  
>
>>
>> > I don't know of a way to encode a path relative to where the library is 
>> installed. That would be very useful, if possible. But I'm not sure if it 
>> can be done.
>>
>> I'm actually not sure how fopen() with relative paths would work inside 
>> library code - it might depend on the current working directory of the 
>> calling process, which is also something you probably shouldn't be imposing 
>> restrictions on. This path is going to need to be runtime configurable, I 
>> don't see any other way around it.
>>
>
> That should be possible. I'll add a function to set it at runtime.
>  
>
>> Unless you want to generate the file in such a way that it becomes valid 
>> C syntax assigning into one string per line or an array of integers per 
>> line, then use it during compilation as a .c or .h file.
>>
>> > Since git is installed with Julia, I'd like to git clone flint as part 
>> of the install process, so that the user always has the source code of 
>> flint (and the license, etc). 
>>
>> This isn't necessary. The flint source code is on github, it's pretty 
>> simple to document that and point people to the right place if they want to 
>> look at it. Keep in mind that not all Julia package users are going to 
>> understand or care about the source code of a C library. The Julia package 
>> existing and providing access to some of the functionality is enough for 
>> many users.
>>
>> > Flint is GPL, not LGPL. There is no linking exemption for flint. On the 
>> other hand, the FSF seems to be pretty isolated in its position that 
>> dynamic linking to a library results in a derived work. Fortunately there 
>> is almost no FSF code in flint and it is all LGPL'd. No one else is going 
>> to care.
>>
>> I don't know if we've gotten a clear legal intepretation of exactly where 
>> Julia's ccall falls with respect to licensing. I don't think it's exactly 
>> the same thing as linking.
>>
>
> The Free Software Foundation explicitly says that "bindings" from a 
> programming language interpreter, not just linking, constitute a derived 
> work and therefore the combination becomes GPL'd. They are very explicit 
> about this on their website.
>
> I also had someone write to me the other day and refuse to access the 
> source code via Github because they "refused to indemnify Github against 
> all damages". It's not clear if I am required under the GPL to send them 
> the source code some other way...
>
> We're also not yet fully in compliance with the GPL. Because it is an 
> interpreted environment GPL v2 (which flint uses) requires a notice to be 
> printed when Nemo starts up saying that the program co

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

On 14 September 2014 06:14, Tony Kelman  wrote:

> > We tend not to use tags. But there's no problem introducing a named tag
> to pin things.
>
> Any reason why not? Your code is on github but your releases aren't, would
> be great if it was possible to track exactly which commits correspond to
> exactly which numbered release version.
>

We have been using branches, not tags for that. We like to only push fixes
to release branches, whilst work continues in trunk, and make those minor
point releases, 2.4.0, 2.4.1. 2.4.2, etc.

I suppose we could tag all the releases in their branches too.


>
> > The way it currently works is the user will specify --prefix. This is
> where libflint will be installed (in a subdirectory called lib). Obviously
> a default is chosen otherwise.
> >
> > Flint is at first built inside its own source tree. Flint knows where to
> find the file relative to this source tree.
> >
> > But when make install is issued, it will be moved across to --prefix.
> >
> > As flint is compiled, --prefix is used to generate an absolute path
> where the text file will be stored, and this is baked into the library,
> which then also looks in this absolute path.
> >
> > Of course the absolute path is computed at compile time to be some path
> relative to --prefix. It is passed to the relevant parts of flint via a -D
> define.
>
> None of this is going to work if you want the library to be usable on a
> different machine than it gets compiled on. That's a pretty big restriction
> to put on your users. A bit like using Gentoo - some people think it's fun
> out of masochism or something, but not many. Most users would rather save
> time and use binaries wherever possible (assuming the results are the same
> and you don't introduce any compatibility problems - if you use an
> automated build service and a package manager these are easy).
>

We'd like to fix the problem. We just don't know how. At some point, the
library has to read from a large text file. There must be a relocatable way
of doing that.

Of course you can pass an explicit directory to flint's configure to tell
it where to put this text file. Perhaps there is some canonical place.


>
> > I don't know of a way to encode a path relative to where the library is
> installed. That would be very useful, if possible. But I'm not sure if it
> can be done.
>
> I'm actually not sure how fopen() with relative paths would work inside
> library code - it might depend on the current working directory of the
> calling process, which is also something you probably shouldn't be imposing
> restrictions on. This path is going to need to be runtime configurable, I
> don't see any other way around it.
>

That should be possible. I'll add a function to set it at runtime.


> Unless you want to generate the file in such a way that it becomes valid C
> syntax assigning into one string per line or an array of integers per line,
> then use it during compilation as a .c or .h file.
>
> > Since git is installed with Julia, I'd like to git clone flint as part
> of the install process, so that the user always has the source code of
> flint (and the license, etc).
>
> This isn't necessary. The flint source code is on github, it's pretty
> simple to document that and point people to the right place if they want to
> look at it. Keep in mind that not all Julia package users are going to
> understand or care about the source code of a C library. The Julia package
> existing and providing access to some of the functionality is enough for
> many users.
>
> > Flint is GPL, not LGPL. There is no linking exemption for flint. On the
> other hand, the FSF seems to be pretty isolated in its position that
> dynamic linking to a library results in a derived work. Fortunately there
> is almost no FSF code in flint and it is all LGPL'd. No one else is going
> to care.
>
> I don't know if we've gotten a clear legal intepretation of exactly where
> Julia's ccall falls with respect to licensing. I don't think it's exactly
> the same thing as linking.
>

The Free Software Foundation explicitly says that "bindings" from a
programming language interpreter, not just linking, constitute a derived
work and therefore the combination becomes GPL'd. They are very explicit
about this on their website.

I also had someone write to me the other day and refuse to access the
source code via Github because they "refused to indemnify Github against
all damages". It's not clear if I am required under the GPL to send them
the source code some other way...

We're also not yet fully in compliance with the GPL. Because it is an
interpreted environment GPL v2 (which flint uses) requires a notice to be
printed when Nemo starts up saying that the program comes with no warranty.
I'll have to add that, if I can figure out how.

We also need to either include the full source code of flint with the
binary, or a written offer to supply it upon request. Here the FSF claims a
link to a website is not really good enough

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

> We tend not to use tags. But there's no problem introducing a named tag 
to pin things.

Any reason why not? Your code is on github but your releases aren't, would 
be great if it was possible to track exactly which commits correspond to 
exactly which numbered release version.

> The way it currently works is the user will specify --prefix. This is 
where libflint will be installed (in a subdirectory called lib). Obviously 
a default is chosen otherwise.
> 
> Flint is at first built inside its own source tree. Flint knows where to 
find the file relative to this source tree.
> 
> But when make install is issued, it will be moved across to --prefix.
> 
> As flint is compiled, --prefix is used to generate an absolute path where 
the text file will be stored, and this is baked into the library, which 
then also looks in this absolute path.
> 
> Of course the absolute path is computed at compile time to be some path 
relative to --prefix. It is passed to the relevant parts of flint via a -D 
define.

None of this is going to work if you want the library to be usable on a 
different machine than it gets compiled on. That's a pretty big restriction 
to put on your users. A bit like using Gentoo - some people think it's fun 
out of masochism or something, but not many. Most users would rather save 
time and use binaries wherever possible (assuming the results are the same 
and you don't introduce any compatibility problems - if you use an 
automated build service and a package manager these are easy).

> I don't know of a way to encode a path relative to where the library is 
installed. That would be very useful, if possible. But I'm not sure if it 
can be done.

I'm actually not sure how fopen() with relative paths would work inside 
library code - it might depend on the current working directory of the 
calling process, which is also something you probably shouldn't be imposing 
restrictions on. This path is going to need to be runtime configurable, I 
don't see any other way around it. Unless you want to generate the file in 
such a way that it becomes valid C syntax assigning into one string per 
line or an array of integers per line, then use it during compilation as a 
.c or .h file.

> Since git is installed with Julia, I'd like to git clone flint as part of 
the install process, so that the user always has the source code of flint 
(and the license, etc). 

This isn't necessary. The flint source code is on github, it's pretty 
simple to document that and point people to the right place if they want to 
look at it. Keep in mind that not all Julia package users are going to 
understand or care about the source code of a C library. The Julia package 
existing and providing access to some of the functionality is enough for 
many users.

> Flint is GPL, not LGPL. There is no linking exemption for flint. On the 
other hand, the FSF seems to be pretty isolated in its position that 
dynamic linking to a library results in a derived work. Fortunately there 
is almost no FSF code in flint and it is all LGPL'd. No one else is going 
to care.

I don't know if we've gotten a clear legal intepretation of exactly where 
Julia's ccall falls with respect to licensing. I don't think it's exactly 
the same thing as linking. If your non-GPL Julia package depends on a GPL 
library and is going to automatically download that GPL library (either 
from source or from a binary) as part of its installation, then you should 
clearly document that fact. There are companies that forbid their employees 
from using or looking at GPL code. 

> I think I will get things going temporarily on Windows by modifying PATH 
first. Then I will promise a binary distribution method to come, which we 
can work on.

Alright. Just to warn you again, plenty of MinGW users refuse to add MinGW 
permanently to PATH, due to the compatibility issues that introduces. More 
often than not you need to work with several different subtly incompatible 
toolchains, and putting any of those on PATH permanently is 
counter-productive. So this installation method is something that you may 
get to work on your machine, but it's unlikely to work many other places.

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Hi Tony,

A quick reply as I am about to be unavailable for some hours.

On Saturday, 13 September 2014 09:19:27 UTC+2, Tony Kelman wrote:
>
> > Firstly, GMP 5 is currently a drop-in replacement for MPIR for us. We 
> prefer to use MPIR since I am the maintainer of the latter, and significant 
> work has gone into making various features faster. 
>
> If the reverse is true, and MPIR would be a drop-in replacement for GMP 
> for Julia, then it could also be a possibility that Julia could try to use 
> MPIR instead of GMP in the future. I'm not aware of the specific 
> differences other than MPIR intends to be more compatible with MSVC, and 
> compiling Julia with MSVC is an ongoing experiment that I've done a fair 
> amount of work on. Having the maintainer of a dependency library happen to 
> be a user of Julia might be convenient for... reasons.
>
> > For flint we definitely require the latest bleeding edge git version. 
> Even the last official release from us, flint-2.4.4, will not work with 
> Nemo.
>
> We can experiment with bleeding edge git instead. We'll have to see how 
> many more changes will be needed. Presumably you can tag a release of flint 
> once we're happy things are working?
>

We tend not to use tags. But there's no problem introducing a named tag to 
pin things.
 

>
> > The CP text file is absolutely enormous (nearly a megabyte). Encoding it 
> as a string is certainly out of the question (we strictly adhere to the 
> ANSI standard, which limits strings to something like 750 bytes). We've 
> discussed many other ways of handling it, and numerous people have tried 
> various things. It's a major pain in the backside, but totally necessary to 
> have it (mathematically speaking).
> > 
> > Currently it is not too bad in that once flint is installed, so is the 
> file, in a location with a fixed offset from the library (which will have 
> been specified by --prefix). If the user chooses to build flint in the 
> source tree, it is located in a fixed location within the source tree which 
> flint can also find it. This is all fine, so long as nothing gets moved 
> after compilation and installation, which is the usual case, it seems.
>
> So is the path a relative path between the library and the text file, or 
> an absolute path to the text file? When installing from a binary we can 
> keep the relative path consistent pretty easily, absolute path definitely 
> not. If it's an absolute path, would it be feasible to check the 
> compile-time defined path first by default (as is done currently), but fall 
> back to using an environment variable or other runtime path initialization 
> method if the default location fails?
>
>
The way it currently works is the user will specify --prefix. This is where 
libflint will be installed (in a subdirectory called lib). Obviously a 
default is chosen otherwise.

Flint is at first built inside its own source tree. Flint knows where to 
find the file relative to this source tree.

But when make install is issued, it will be moved across to --prefix.

As flint is compiled, --prefix is used to generate an absolute path where 
the text file will be stored, and this is baked into the library, which 
then also looks in this absolute path.

Of course the absolute path is computed at compile time to be some path 
relative to --prefix. It is passed to the relevant parts of flint via a -D 
define.

I don't know of a way to encode a path relative to where the library is 
installed. That would be very useful, if possible. But I'm not sure if it 
can be done.
 

> > The problem with linking against Julia's GMP and MPFR was as follows. We 
> can tell Julia where to find flint by pushing to DL_LOAD_PATH. But this 
> only tells Julia where to find flint. The system linker is the one that 
> tries to subsequently link flint with MPIR and MPFR, and it hasn't got a 
> clue where to find them. What we push on DL_LOAD_PATH is irrelevant, as 
> only Julia knows about that. This actually causes Julia to report that it 
> can't find flint. I spent many, many hours figuring that one out.
>
> I'm not proposing to push anything to DL_LOAD_PATH for GMP or MPFR. Rather 
> we tell flint's build system at compile time where to find Julia's GMP and 
> MPFR via -L. Those libraries will always be either at a consistent relative 
> path vs JULIA_HOME, 
>

Ah, I think JULIA_HOME should be useful. I didn't know about this.
 

> or in standard system library paths if the user (or package manager) built 
> Julia against system GMP/MPFR. I think it would be safe to check if 
> joinpath(JULIA_HOME,"..","lib","libgmp.$(sys.dlext)") is openable via 
> dlopen_e, and if not then falling back to find_library(["libgmp"]).
>

Ok.
 

>
> > Now it occurs to me that perhaps if we build flint against Julia's GMP 
> and MPFR (assuming GMP is version 5 and MPFR version 3.1), then because 
> Julia's libs are placed in the path by Julia, the system linker might just 
> find them. That hadn't occurred to me as

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

> Firstly, GMP 5 is currently a drop-in replacement for MPIR for us. We 
prefer to use MPIR since I am the maintainer of the latter, and significant 
work has gone into making various features faster. 

If the reverse is true, and MPIR would be a drop-in replacement for GMP for 
Julia, then it could also be a possibility that Julia could try to use MPIR 
instead of GMP in the future. I'm not aware of the specific differences 
other than MPIR intends to be more compatible with MSVC, and compiling 
Julia with MSVC is an ongoing experiment that I've done a fair amount of 
work on. Having the maintainer of a dependency library happen to be a user 
of Julia might be convenient for... reasons.

> For flint we definitely require the latest bleeding edge git version. 
Even the last official release from us, flint-2.4.4, will not work with 
Nemo.

We can experiment with bleeding edge git instead. We'll have to see how 
many more changes will be needed. Presumably you can tag a release of flint 
once we're happy things are working?

> The CP text file is absolutely enormous (nearly a megabyte). Encoding it 
as a string is certainly out of the question (we strictly adhere to the 
ANSI standard, which limits strings to something like 750 bytes). We've 
discussed many other ways of handling it, and numerous people have tried 
various things. It's a major pain in the backside, but totally necessary to 
have it (mathematically speaking).
> 
> Currently it is not too bad in that once flint is installed, so is the 
file, in a location with a fixed offset from the library (which will have 
been specified by --prefix). If the user chooses to build flint in the 
source tree, it is located in a fixed location within the source tree which 
flint can also find it. This is all fine, so long as nothing gets moved 
after compilation and installation, which is the usual case, it seems.

So is the path a relative path between the library and the text file, or an 
absolute path to the text file? When installing from a binary we can keep 
the relative path consistent pretty easily, absolute path definitely not. 
If it's an absolute path, would it be feasible to check the compile-time 
defined path first by default (as is done currently), but fall back to 
using an environment variable or other runtime path initialization method 
if the default location fails?

> The problem with linking against Julia's GMP and MPFR was as follows. We 
can tell Julia where to find flint by pushing to DL_LOAD_PATH. But this 
only tells Julia where to find flint. The system linker is the one that 
tries to subsequently link flint with MPIR and MPFR, and it hasn't got a 
clue where to find them. What we push on DL_LOAD_PATH is irrelevant, as 
only Julia knows about that. This actually causes Julia to report that it 
can't find flint. I spent many, many hours figuring that one out.

I'm not proposing to push anything to DL_LOAD_PATH for GMP or MPFR. Rather 
we tell flint's build system at compile time where to find Julia's GMP and 
MPFR via -L. Those libraries will always be either at a consistent relative 
path vs JULIA_HOME, or in standard system library paths if the user (or 
package manager) built Julia against system GMP/MPFR. I think it would be 
safe to check if joinpath(JULIA_HOME,"..","lib","libgmp.$(sys.dlext)") is 
openable via dlopen_e, and if not then falling back to 
find_library(["libgmp"]).

> Now it occurs to me that perhaps if we build flint against Julia's GMP 
and MPFR (assuming GMP is version 5 and MPFR version 3.1), then because 
Julia's libs are placed in the path by Julia, the system linker might just 
find them. That hadn't occurred to me as a possibility. The obvious problem 
is GMP is not MPIR and we are limited to using whatever version of GMP 
Julia chooses. Actually, maybe the git version of flint already supports 
GMP 6 nowadays too.

For Julia 0.3.0, GMP is version 5.1.3, MPFR is version 3.1.2. 
Check 
https://github.com/JuliaLang/julia/blob/768187890c2709bf2ff06818f40e1cdc79bd44b0/deps/Versions.make
 
for the default version numbers used. On Julia master, GMP has been bumped 
to 6.0.0.

> Also, flint uses libgcc_s and pthreads (and libm). We've currently set it 
up to statically build libgcc_s into flint, but we can't do that with 
pthreads. Initially at least we could disable pthreads when building flint 
(there is an option for it) but I guess in the long run we won't want to do 
that. I guess we can for now, but it seems like a compromise.

You don't need to do this if you use WinRPM. If libgcc_s and/or pthreads 
are dynamically linked, the RPM packaging scripts on the build service 
detect that and mark the dll's as dependencies, so they get correctly 
downloaded when you install the package. Linking dynamically will make your 
binaries smaller, and many other things also link to these same libraries.

> Regarding licenses, I would guess the GPL v3 and LGPL v3 license texts 
need to be distributed with Julia binaries, al

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Lots of issues.

Firstly, GMP 5 is currently a drop-in replacement for MPIR for us. We
prefer to use MPIR since I am the maintainer of the latter, and significant
work has gone into making various features faster.

For flint we definitely require the latest bleeding edge git version. Even
the last official release from us, flint-2.4.4, will not work with Nemo.

The CP text file is absolutely enormous (nearly a megabyte). Encoding it as
a string is certainly out of the question (we strictly adhere to the ANSI
standard, which limits strings to something like 750 bytes). We've
discussed many other ways of handling it, and numerous people have tried
various things. It's a major pain in the backside, but totally necessary to
have it (mathematically speaking).

Currently it is not too bad in that once flint is installed, so is the
file, in a location with a fixed offset from the library (which will have
been specified by --prefix). If the user chooses to build flint in the
source tree, it is located in a fixed location within the source tree which
flint can also find it. This is all fine, so long as nothing gets moved
after compilation and installation, which is the usual case, it seems.

The problem with linking against Julia's GMP and MPFR was as follows. We
can tell Julia where to find flint by pushing to DL_LOAD_PATH. But this
only tells Julia where to find flint. The system linker is the one that
tries to subsequently link flint with MPIR and MPFR, and it hasn't got a
clue where to find them. What we push on DL_LOAD_PATH is irrelevant, as
only Julia knows about that. This actually causes Julia to report that it
can't find flint. I spent many, many hours figuring that one out.

Now it occurs to me that perhaps if we build flint against Julia's GMP and
MPFR (assuming GMP is version 5 and MPFR version 3.1), then because Julia's
libs are placed in the path by Julia, the system linker might just find
them. That hadn't occurred to me as a possibility. The obvious problem is
GMP is not MPIR and we are limited to using whatever version of GMP Julia
chooses. Actually, maybe the git version of flint already supports GMP 6
nowadays too.

Also, flint uses libgcc_s and pthreads (and libm). We've currently set it
up to statically build libgcc_s into flint, but we can't do that with
pthreads. Initially at least we could disable pthreads when building flint
(there is an option for it) but I guess in the long run we won't want to do
that. I guess we can for now, but it seems like a compromise.

Regarding licenses, I would guess the GPL v3 and LGPL v3 license texts need
to be distributed with Julia binaries, along with a written offer to
provide the source code (or the actual source code). Certainly the LGPL
allows Julia to dynamically link against LGPL libraries without becoming a
derivative work. I don't want to make absolute statements about that issue,
as IANAL, but I'd be interested in more details of how Julia is satisfying
the conditions summarised here
http://en.wikipedia.org/wiki/GNU_General_Public_License#Terms_and_conditions

Aside from those issues, I don't seem to easily be able to find out what
version of GMP or MPFR Julia is currently using in v0.3. That could be
helpful.

Regarding the .jl extension, I did read that, but am not entirely sure I
know how to change it on github. I will take a look and see if I can change
it.

I didn't realise @unix_only was the opposite of @windows_only. I thought
unix meant Free BSD, Net BSD, Unix, etc. And I couldn't see a way to
combine Unix, Linux and OSX using the macros. Given that OSX is probably
going to need something special, I'm not sure what the best combination of
macros will be.

Popcnt is processor dependent. TLS is more OS dependent and I think
available on Mingw. We can just disable hardware popcnt. It's not vitally
important for more than about one noncritical function.

Thanks for all your comments and your assistance. Unfortunately flint is
slightly complicated by various issues.

For now I'm happy to go with GMP on Windows (ironic given that MPIR was
originally a Windows-friendly fork of GMP). But for the simple reason that
GMP is already there in Julia it shouldn't be a problem for now. In the
long term we definitely want to go with MPIR. I will probably add some
additional files to assure there is no question of compliance with licenses
for Nemo at least.

Bill.

On 13 September 2014 06:22, Tony Kelman  wrote:

> The library cross-compiles okay (at least at the last released version, I
> can try git master if you've made changes that are necessary), see
> https://build.opensuse.org/package/show/home:kelman/mingw32-flint and
> https://build.opensuse.org/package/show/home:kelman/mingw64-flint
>
> It was made a little more challenging by the fact that flint uses a
> hand-rolled configure script that errors on unrecognized arguments. That
> configure script uses --build for what is conventionally meant by --host,
> the two differ in a cross compile. --host is 

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

The library cross-compiles okay (at least at the last released version, I 
can try git master if you've made changes that are necessary), 
see https://build.opensuse.org/package/show/home:kelman/mingw32-flint 
and https://build.opensuse.org/package/show/home:kelman/mingw64-flint

It was made a little more challenging by the fact that flint uses a 
hand-rolled configure script that errors on unrecognized arguments. That 
configure script uses --build for what is conventionally meant by --host, 
the two differ in a cross compile. --host is where the compiled code is 
intended to run, --build is (in standard autotools) the machine that is 
doing the compiling.

There was an existing openSUSE Linux package for flint 
here https://build.opensuse.org/package/show/science/flint with a patch to 
use real autotools, though it's not completely usable for a cross compile 
at the moment since it doesn't handle using GMP instead of MPIR. MPIR is 
not available as a cross-compiled package yet, though making one would 
probably be easy. The existing cross-compiled version of MPFR already links 
to GMP so I think it would be redundant.

Flint has configure-time checks for popcnt and tls that run executables, 
neither of which work properly in a cross-compile. Locally on Windows they 
both succeed, but could those potentially be processor-dependent in any 
way? I could patch configure to override the results for the purposes of 
cross compiling, if you can recommend one way or the other.

On the text file, it's really bad for library relocatability to hard-code 
the compile-time location of required data. Is the content of the file ever 
expected to change? If not, I would just hard-code its contents into a 
string or more appropriate data structure. If so, then its location should 
really be made a runtime setting, either through a library initialization 
API or from an environment variable or something.

I could be wrong, but I don't think rpath does anything on Windows. 
Standard practice for MinGW libraries is to put dll files in bin/ rather 
than lib/ for this reason. If you were using libtool, it would create a 
.dll.a import library stub in lib/ to assist in linking to the dll if you 
wanted to compile a C/C++ application or library with -lflint. Luckily 
Julia doesn't need that stub to just ccall into a dll.

-Tony


On Friday, September 12, 2014 6:11:49 PM UTC-7, Tony Kelman wrote:
>
> > I'm worried about the legality of downloading binaries for users, since 
> the packages involved, GMP/MPIR, MPFR and FLINT are (L)GPL.
>
> Quite a few packages rely on (L)GPL code. The GPL is a long confusing 
> document, and limits commercial application of some code, that's for sure, 
> but to the best of my (IANAL) understanding it's not too onerous to comply 
> with for open-source projects. Mostly including the license in the 
> documentation and being careful about bundling/derived works. The spec 
> files on the build service have to clearly state the license, plenty of 
> libraries there are GPL and/or have GPL dependencies.
>
> > I see now that Julia adds things to the system PATH, which is why it is 
> picking up things in the GitBash bundled in Julia. If I can figure out how 
> to get PATH back to its original state temporarily, I think I can get the 
> build to succeed (supposing the user has MSYS2 and Mingw64 correctly 
> installed -- not easy, admittedly).
>
> You can modify ENV["PATH"] as you like, but I recommend against this 
> approach. Compare the relative difficulty of building Julia from source to 
> downloading and using the binary installer. On a slightly smaller scale, 
> having the package rely on MSYS2 and MinGW vs just downloading binaries 
> will lead to a similar usability comparison for your package. On Linux 
> there's much more variety in compiler and library versions so the best way 
> to download binaries is going through the system package manager, but if 
> your library isn't available in the package manager then building from 
> source is easy because compilers are really easy to install. On Mac it's a 
> little more annoying to install compilers, so a decent number of packages 
> use Homebrew.jl to distribute binaries. (Just have to set up a short 
> "formula" file with instructions on how to build, and bug Elliot, 
> @staticfloat on github, to build binaries for you).
>
> > I agree binary distribution is the normal method on Windows. But I've 
> already been forced to go through one FSF "license compliance review" 
> because a GNU project reported us for being in breach of the license. I'm 
> not doing it again.
> > 
> > If there is a way to do things in strict accordance with the LGPL and 
> GPL I can go down that route. In that case, you can see the invocations 
> here:
>
> I can imagine that experience would give you pause here. If you can link 
> directly to the same version of GMP and MPFR as Julia is using (they are 
> currently required dependencies, so this should be fairly safe - jus

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

> I'm worried about the legality of downloading binaries for users, since 
the packages involved, GMP/MPIR, MPFR and FLINT are (L)GPL.

Quite a few packages rely on (L)GPL code. The GPL is a long confusing 
document, and limits commercial application of some code, that's for sure, 
but to the best of my (IANAL) understanding it's not too onerous to comply 
with for open-source projects. Mostly including the license in the 
documentation and being careful about bundling/derived works. The spec 
files on the build service have to clearly state the license, plenty of 
libraries there are GPL and/or have GPL dependencies.

> I see now that Julia adds things to the system PATH, which is why it is 
picking up things in the GitBash bundled in Julia. If I can figure out how 
to get PATH back to its original state temporarily, I think I can get the 
build to succeed (supposing the user has MSYS2 and Mingw64 correctly 
installed -- not easy, admittedly).

You can modify ENV["PATH"] as you like, but I recommend against this 
approach. Compare the relative difficulty of building Julia from source to 
downloading and using the binary installer. On a slightly smaller scale, 
having the package rely on MSYS2 and MinGW vs just downloading binaries 
will lead to a similar usability comparison for your package. On Linux 
there's much more variety in compiler and library versions so the best way 
to download binaries is going through the system package manager, but if 
your library isn't available in the package manager then building from 
source is easy because compilers are really easy to install. On Mac it's a 
little more annoying to install compilers, so a decent number of packages 
use Homebrew.jl to distribute binaries. (Just have to set up a short 
"formula" file with instructions on how to build, and bug Elliot, 
@staticfloat on github, to build binaries for you).

> I agree binary distribution is the normal method on Windows. But I've 
already been forced to go through one FSF "license compliance review" 
because a GNU project reported us for being in breach of the license. I'm 
not doing it again.
> 
> If there is a way to do things in strict accordance with the LGPL and GPL 
I can go down that route. In that case, you can see the invocations here:

I can imagine that experience would give you pause here. If you can link 
directly to the same version of GMP and MPFR as Julia is using (they are 
currently required dependencies, so this should be fairly safe - just have 
to figure out the best way to do it that would also work if someone is 
using their system's version of GMP and/or MPFR), then you don't need to 
worry about distributing anything except Flint itself. Would GMP be a 
drop-in replacement for MPIR for you? If we use WinRPM, I believe it would 
automatically detect GMP and MPFR (both are already available there) as 
linked DLL's and automatically download them, we just have to declare the 
dependency in the spec file and make sure configure picks them up or is 
told explicitly where they are located. They would be in a standard 
predictable location on the cross-compiling build machine.

> https://github.com/wbhart/Nemo/blob/master/deps/build.jl
> 
> Is that sufficient?

I can see from that script how you're configuring, yes. Downloading and 
building your own copy of MPIR and MPFR seems slightly redundant and 
possibly problematic to load them from within the same process as Julia's 
copies of GMP and MPFR.

> I should mention that there are some complications. Flint needs to be 
built in situ because it needs a large .txt file at runtime which contains 
data it uses (Conway polynomials). The only way to tell flint where it will 
be is through the build system, and the only way to know where it will be 
is to actually build flint.

Can that text file be included along with the binary?

> Moreover, the system linker cannot find the dependencies MPIR and MPFR of 
flint unless the -rpath is set by flint during build. And again, the only 
way -rpath can be set is if flint knows where it is being built.

You can't just provide a full -L path?

A few minor notes, standard convention for Julia packages is to name the 
Git repository include the ".jl" extension in its name. This helps visually 
identify Julia package repositories easily, and is nice to keep consistent.

You're also frequently doing "if on_windows", there's a pre-defined macro 
for that: @windows_only, or the opposite @unix_only.

-Tony

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

I should mention that there are some complications. Flint needs to be built
in situ because it needs a large .txt file at runtime which contains data
it uses (Conway polynomials). The only way to tell flint where it will be
is through the build system, and the only way to know where it will be is
to actually build flint.

Moreover, the system linker cannot find the dependencies MPIR and MPFR of
flint unless the -rpath is set by flint during build. And again, the only
way -rpath can be set is if flint knows where it is being built.

So I think source build is probably the only feasible option, actually.

Having said that, I cannot figure out how to revert the PATH to the system
provided one in Julia. It just seems to clobber it.

Bill.

On 13 September 2014 02:47, Bill Hart  wrote:

> Hi Tony,
>
> I'm worried about the legality of downloading binaries for users, since
> the packages involved, GMP/MPIR, MPFR and FLINT are (L)GPL.
>
> I see now that Julia adds things to the system PATH, which is why it is
> picking up things in the GitBash bundled in Julia. If I can figure out how
> to get PATH back to its original state temporarily, I think I can get the
> build to succeed (supposing the user has MSYS2 and Mingw64 correctly
> installed -- not easy, admittedly).
>
> I agree binary distribution is the normal method on Windows. But I've
> already been forced to go through one FSF "license compliance review"
> because a GNU project reported us for being in breach of the license. I'm
> not doing it again.
>
> If there is a way to do things in strict accordance with the LGPL and GPL
> I can go down that route. In that case, you can see the invocations here:
>
> https://github.com/wbhart/Nemo/blob/master/deps/build.jl
>
> Is that sufficient?
>
> Bill.
>
> On 13 September 2014 02:24, Tony Kelman  wrote:
>
>> Bill,
>>
>> Shelling out in Julia on Windows currently calls into the bundled
>> Git-bash, for the purposes of the package manager. That will hopefully
>> change in 0.4 by replacing the bundled Git-bash with libgit2 bindings, but
>> PR 7339 is currently a bit stalled for lack of anyone who understands the
>> LibGit2.jl / Pkg code digging into it.
>>
>> We've generally tried to avoid the "force users to build packages from
>> source" approach for Windows, since getting MSYS2 and the correct version
>> of MinGW installed and working can be a bit challenging. It's been more
>> effective to upload binaries containing the appropriate DLL's, and use
>> BinDeps to download and extract them upon Pkg.add.
>>
>> The question is how you want to build and where you want to upload those
>> binaries. If your package is possible to cross-compile from Linux to
>> Windows (using Linux versions of the MinGW compilers), then we can write a
>> short spec file and submit it to the OpenSUSE build service, which has a
>> large selection of several hundred cross-compiled libraries for 32 and 64
>> bit Windows: https://build.opensuse.org/project/show/windows:mingw:win64
>> . I had recent success doing this for Lapack, HDF5, Metis, and the stack of
>> COIN-OR libraries used by various JuliaOpt packages. There's a Julia
>> package called WinRPM that works like a package-manager provider with
>> BinDeps, and handles all the dependencies properly.
>>
>> Worrying about cross-compiling is a little confusing at first and more
>> effort to set up initially, but using WinRPM and the build service makes
>> future maintenance and interaction with dependencies much easier in the
>> long term. You only need small tweaks to the spec file to update future
>> versions or rebuild with a different set of options, for example.
>>
>> If you can point me to the source of the package and provide
>> representative configure invocations, I can take a look at working on a
>> spec file with you.
>>
>> -Tony
>>
>>
>> On Friday, September 12, 2014 3:52:33 PM UTC-7, Bill Hart wrote:
>>>
>>> One hint I have is that I keep getting the error message "couldnt make
>>> stderr distinct from stdout".
>>>
>>> Even if I build from a naked ,sys2 sh invoked without any decoration
>>> from the Windows command prompt, everything builds fine.
>>>
>>> Although I can't detect any differences in the versions of gcc, make,
>>> ld, etc, I can detect one difference between the shell from Julia and the
>>> shell from msys2. When I do run(`ls /`) from Julia I am clearly in Git
>>> Bash. When I do it from msys2 I am clearly in msys2.
>>>
>>> So it looks to me like some of the files distributed with Julia's
>>> GitBash are causing issues with building from Julia on Windows. Maybe there
>>> are some dll conflicts or something.
>>>
>>> Either way, I don't know how to fix the problem. Any advice would be
>>> welcome.
>>>
>>> Bill.
>>>
>>> On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote:

 Hi Isaiah,

 Thanks for the offer of help on Windows. I think I am at the stage
 where I need to ask for it.

 Here is the repo so far:

 https://github.com/

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Hi Tony,

I'm worried about the legality of downloading binaries for users, since the
packages involved, GMP/MPIR, MPFR and FLINT are (L)GPL.

I see now that Julia adds things to the system PATH, which is why it is
picking up things in the GitBash bundled in Julia. If I can figure out how
to get PATH back to its original state temporarily, I think I can get the
build to succeed (supposing the user has MSYS2 and Mingw64 correctly
installed -- not easy, admittedly).

I agree binary distribution is the normal method on Windows. But I've
already been forced to go through one FSF "license compliance review"
because a GNU project reported us for being in breach of the license. I'm
not doing it again.

If there is a way to do things in strict accordance with the LGPL and GPL I
can go down that route. In that case, you can see the invocations here:

https://github.com/wbhart/Nemo/blob/master/deps/build.jl

Is that sufficient?

Bill.

On 13 September 2014 02:24, Tony Kelman  wrote:

> Bill,
>
> Shelling out in Julia on Windows currently calls into the bundled
> Git-bash, for the purposes of the package manager. That will hopefully
> change in 0.4 by replacing the bundled Git-bash with libgit2 bindings, but
> PR 7339 is currently a bit stalled for lack of anyone who understands the
> LibGit2.jl / Pkg code digging into it.
>
> We've generally tried to avoid the "force users to build packages from
> source" approach for Windows, since getting MSYS2 and the correct version
> of MinGW installed and working can be a bit challenging. It's been more
> effective to upload binaries containing the appropriate DLL's, and use
> BinDeps to download and extract them upon Pkg.add.
>
> The question is how you want to build and where you want to upload those
> binaries. If your package is possible to cross-compile from Linux to
> Windows (using Linux versions of the MinGW compilers), then we can write a
> short spec file and submit it to the OpenSUSE build service, which has a
> large selection of several hundred cross-compiled libraries for 32 and 64
> bit Windows: https://build.opensuse.org/project/show/windows:mingw:win64
> . I had recent success doing this for Lapack, HDF5, Metis, and the stack of
> COIN-OR libraries used by various JuliaOpt packages. There's a Julia
> package called WinRPM that works like a package-manager provider with
> BinDeps, and handles all the dependencies properly.
>
> Worrying about cross-compiling is a little confusing at first and more
> effort to set up initially, but using WinRPM and the build service makes
> future maintenance and interaction with dependencies much easier in the
> long term. You only need small tweaks to the spec file to update future
> versions or rebuild with a different set of options, for example.
>
> If you can point me to the source of the package and provide
> representative configure invocations, I can take a look at working on a
> spec file with you.
>
> -Tony
>
>
> On Friday, September 12, 2014 3:52:33 PM UTC-7, Bill Hart wrote:
>>
>> One hint I have is that I keep getting the error message "couldnt make
>> stderr distinct from stdout".
>>
>> Even if I build from a naked ,sys2 sh invoked without any decoration from
>> the Windows command prompt, everything builds fine.
>>
>> Although I can't detect any differences in the versions of gcc, make, ld,
>> etc, I can detect one difference between the shell from Julia and the shell
>> from msys2. When I do run(`ls /`) from Julia I am clearly in Git Bash. When
>> I do it from msys2 I am clearly in msys2.
>>
>> So it looks to me like some of the files distributed with Julia's GitBash
>> are causing issues with building from Julia on Windows. Maybe there are
>> some dll conflicts or something.
>>
>> Either way, I don't know how to fix the problem. Any advice would be
>> welcome.
>>
>> Bill.
>>
>> On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote:
>>>
>>> Hi Isaiah,
>>>
>>> Thanks for the offer of help on Windows. I think I am at the stage where
>>> I need to ask for it.
>>>
>>> Here is the repo so far:
>>>
>>> https://github.com/wbhart/Nemo
>>>
>>> The problem I have now is as follows.
>>>
>>> I have msys2 installed on my computer, with Ming64. Whilst each of the
>>> packages MPIR, MPFR and flint builds just fine from within the msys shell,
>>> the tools when called individually from Julia always seem to fail, exactly
>>> as the old Mingw 1.0 tools did. I've checked carefully that the versions of
>>> make, gcc, bash, etc that answer at the command prompt are all from msys2
>>> and I've removed all old mingw and msys installations from my machine
>>> totally. I've tried building dozens of times, but it usually crashes early
>>> on in the MPIR install.
>>>
>>> I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell does.
>>> But I cannot figure out what else it does to stop the crashes.
>>>
>>> Do you have any insight into this?
>>>
>>> I wasn't sure what you meant about contacting you on github, as they've
>>>

Re: [julia-users] Help needed with creating Julia package

[Tony Kelman]

Bill,

Shelling out in Julia on Windows currently calls into the bundled Git-bash, 
for the purposes of the package manager. That will hopefully change in 0.4 
by replacing the bundled Git-bash with libgit2 bindings, but PR 7339 is 
currently a bit stalled for lack of anyone who understands the LibGit2.jl / 
Pkg code digging into it.

We've generally tried to avoid the "force users to build packages from 
source" approach for Windows, since getting MSYS2 and the correct version 
of MinGW installed and working can be a bit challenging. It's been more 
effective to upload binaries containing the appropriate DLL's, and use 
BinDeps to download and extract them upon Pkg.add.

The question is how you want to build and where you want to upload those 
binaries. If your package is possible to cross-compile from Linux to 
Windows (using Linux versions of the MinGW compilers), then we can write a 
short spec file and submit it to the OpenSUSE build service, which has a 
large selection of several hundred cross-compiled libraries for 32 and 64 
bit Windows: https://build.opensuse.org/project/show/windows:mingw:win64 . 
I had recent success doing this for Lapack, HDF5, Metis, and the stack of 
COIN-OR libraries used by various JuliaOpt packages. There's a Julia 
package called WinRPM that works like a package-manager provider with 
BinDeps, and handles all the dependencies properly.

Worrying about cross-compiling is a little confusing at first and more 
effort to set up initially, but using WinRPM and the build service makes 
future maintenance and interaction with dependencies much easier in the 
long term. You only need small tweaks to the spec file to update future 
versions or rebuild with a different set of options, for example.

If you can point me to the source of the package and provide representative 
configure invocations, I can take a look at working on a spec file with you.

-Tony

On Friday, September 12, 2014 3:52:33 PM UTC-7, Bill Hart wrote:
>
> One hint I have is that I keep getting the error message "couldnt make 
> stderr distinct from stdout". 
>
> Even if I build from a naked ,sys2 sh invoked without any decoration from 
> the Windows command prompt, everything builds fine.
>
> Although I can't detect any differences in the versions of gcc, make, ld, 
> etc, I can detect one difference between the shell from Julia and the shell 
> from msys2. When I do run(`ls /`) from Julia I am clearly in Git Bash. When 
> I do it from msys2 I am clearly in msys2.
>
> So it looks to me like some of the files distributed with Julia's GitBash 
> are causing issues with building from Julia on Windows. Maybe there are 
> some dll conflicts or something.
>
> Either way, I don't know how to fix the problem. Any advice would be 
> welcome.
>
> Bill.
>
> On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote:
>>
>> Hi Isaiah,
>>
>> Thanks for the offer of help on Windows. I think I am at the stage where 
>> I need to ask for it.
>>
>> Here is the repo so far:
>>
>> https://github.com/wbhart/Nemo
>>
>> The problem I have now is as follows.
>>
>> I have msys2 installed on my computer, with Ming64. Whilst each of the 
>> packages MPIR, MPFR and flint builds just fine from within the msys shell, 
>> the tools when called individually from Julia always seem to fail, exactly 
>> as the old Mingw 1.0 tools did. I've checked carefully that the versions of 
>> make, gcc, bash, etc that answer at the command prompt are all from msys2 
>> and I've removed all old mingw and msys installations from my machine 
>> totally. I've tried building dozens of times, but it usually crashes early 
>> on in the MPIR install.
>>
>> I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell does. 
>> But I cannot figure out what else it does to stop the crashes.
>>
>> Do you have any insight into this?
>>
>> I wasn't sure what you meant about contacting you on github, as they've 
>> removed the PM service. So I figured I'd just post here, also giving other 
>> knowledgeable individuals a chance to comment.
>>
>> Bill.
>>
>> On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote:
>>>
>>>
>>> This was what I thought of trying first. But I couldn't figure out how 
 it worked out what GitHub repository to associate this with, or whether it 
 would try to create one, possibly scrubbing my existing nemo repository on 
 GitHub. Obviously I don't want to lose my commit history.
>>>
>>>
>>> For Pkg manager purposes, the association will be created later (when 
>>> you register the package).
>>>
>>> It also isn't clear where Julia creates the empty git repository. In the 
 current directory? Or in some subdirectory of the Julia source tree?
>>>
>>>
>>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>>>
>>> For the most part I can just run configure, make, make install for now 
 and set some library paths (if I can figure out what kind of system I am 
 on).
>>>
>>>
>>> There are some macros to help

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

One hint I have is that I keep getting the error message "couldnt make 
stderr distinct from stdout". 

Even if I build from a naked ,sys2 sh invoked without any decoration from 
the Windows command prompt, everything builds fine.

Although I can't detect any differences in the versions of gcc, make, ld, 
etc, I can detect one difference between the shell from Julia and the shell 
from msys2. When I do run(`ls /`) from Julia I am clearly in Git Bash. When 
I do it from msys2 I am clearly in msys2.

So it looks to me like some of the files distributed with Julia's GitBash 
are causing issues with building from Julia on Windows. Maybe there are 
some dll conflicts or something.

Either way, I don't know how to fix the problem. Any advice would be 
welcome.

Bill.

On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote:
>
> Hi Isaiah,
>
> Thanks for the offer of help on Windows. I think I am at the stage where I 
> need to ask for it.
>
> Here is the repo so far:
>
> https://github.com/wbhart/Nemo
>
> The problem I have now is as follows.
>
> I have msys2 installed on my computer, with Ming64. Whilst each of the 
> packages MPIR, MPFR and flint builds just fine from within the msys shell, 
> the tools when called individually from Julia always seem to fail, exactly 
> as the old Mingw 1.0 tools did. I've checked carefully that the versions of 
> make, gcc, bash, etc that answer at the command prompt are all from msys2 
> and I've removed all old mingw and msys installations from my machine 
> totally. I've tried building dozens of times, but it usually crashes early 
> on in the MPIR install.
>
> I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell does. 
> But I cannot figure out what else it does to stop the crashes.
>
> Do you have any insight into this?
>
> I wasn't sure what you meant about contacting you on github, as they've 
> removed the PM service. So I figured I'd just post here, also giving other 
> knowledgeable individuals a chance to comment.
>
> Bill.
>
> On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote:
>>
>>
>> This was what I thought of trying first. But I couldn't figure out how it 
>>> worked out what GitHub repository to associate this with, or whether it 
>>> would try to create one, possibly scrubbing my existing nemo repository on 
>>> GitHub. Obviously I don't want to lose my commit history.
>>
>>
>> For Pkg manager purposes, the association will be created later (when you 
>> register the package).
>>
>> It also isn't clear where Julia creates the empty git repository. In the 
>>> current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>>
>> For the most part I can just run configure, make, make install for now 
>>> and set some library paths (if I can figure out what kind of system I am 
>>> on).
>>
>>
>> There are some macros to help with this: @osx, @linux, @unix (both), and 
>> @windows. There is also a variable called OS_NAME with a platform-specific 
>> value (:Windows, :Linux, etc.) See:
>>
>> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>>
>>  
>>
>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart  
>> wrote:
>>
>>>
>>>
>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:

 Is there documentation somewhere explaining how to do the latter? Or 
> can someone help me with doing the latter?


 You could run `Pkg.generate("Nemo")` and then copy and commit (some of) 
 the resulting files in your own Nemo git tree; there aren't very many.

>>>
>>> This was what I thought of trying first. But I couldn't figure out how 
>>> it worked out what GitHub repository to associate this with, or whether it 
>>> would try to create one, possibly scrubbing my existing nemo repository on 
>>> GitHub. Obviously I don't want to lose my commit history.
>>>  
>>> It also isn't clear where Julia creates the empty git repository. In the 
>>> current directory? Or in some subdirectory of the Julia source tree?
>>>
>>>
 I can't find any documentation explaining where to put the commands in 
> a Pkg to actually git clone flint, build it, install it and set up paths 
> for Nemo. Given the complexities of installing flint for the user, I'd 
> like 
> to have the Julia package manager do this automatically if at all 
> possible. 
> And I see it does seem to be possible. I just can't figure out how.
>

 The Pkg manager will look for a file called `MYPKG/deps/build.jl` and 
 run that if it exists. That's just a Julia file, so you can do whatever 
 you 
 want there (shell out, etc.). 

>>>
>>> Perfect. For the most part I can just run configure, make, make install 
>>> for now and set some library paths (if I can figure out what kind of system 
>>> I am on).
>>>
>>> Finding the Julia installation on the system in order to link against 
>>> the

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Hi Isaiah,

Thanks for the offer of help on Windows. I think I am at the stage where I 
need to ask for it.

Here is the repo so far:

https://github.com/wbhart/Nemo

The problem I have now is as follows.

I have msys2 installed on my computer, with Ming64. Whilst each of the 
packages MPIR, MPFR and flint builds just fine from within the msys shell, 
the tools when called individually from Julia always seem to fail, exactly 
as the old Mingw 1.0 tools did. I've checked carefully that the versions of 
make, gcc, bash, etc that answer at the command prompt are all from msys2 
and I've removed all old mingw and msys installations from my machine 
totally. I've tried building dozens of times, but it usually crashes early 
on in the MPIR install.

I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell does. 
But I cannot figure out what else it does to stop the crashes.

Do you have any insight into this?

I wasn't sure what you meant about contacting you on github, as they've 
removed the PM service. So I figured I'd just post here, also giving other 
knowledgeable individuals a chance to comment.

Bill.

On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote:
>
>
> This was what I thought of trying first. But I couldn't figure out how it 
>> worked out what GitHub repository to associate this with, or whether it 
>> would try to create one, possibly scrubbing my existing nemo repository on 
>> GitHub. Obviously I don't want to lose my commit history.
>
>
> For Pkg manager purposes, the association will be created later (when you 
> register the package).
>
> It also isn't clear where Julia creates the empty git repository. In the 
>> current directory? Or in some subdirectory of the Julia source tree?
>
>
> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>
> For the most part I can just run configure, make, make install for now and 
>> set some library paths (if I can figure out what kind of system I am on).
>
>
> There are some macros to help with this: @osx, @linux, @unix (both), and 
> @windows. There is also a variable called OS_NAME with a platform-specific 
> value (:Windows, :Linux, etc.) See:
>
> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>
>  
>
> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart  > wrote:
>
>>
>>
>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>>
>>> Is there documentation somewhere explaining how to do the latter? Or can 
 someone help me with doing the latter?
>>>
>>>
>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some of) 
>>> the resulting files in your own Nemo git tree; there aren't very many.
>>>
>>
>> This was what I thought of trying first. But I couldn't figure out how it 
>> worked out what GitHub repository to associate this with, or whether it 
>> would try to create one, possibly scrubbing my existing nemo repository on 
>> GitHub. Obviously I don't want to lose my commit history.
>>  
>> It also isn't clear where Julia creates the empty git repository. In the 
>> current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>>> I can't find any documentation explaining where to put the commands in a 
 Pkg to actually git clone flint, build it, install it and set up paths for 
 Nemo. Given the complexities of installing flint for the user, I'd like to 
 have the Julia package manager do this automatically if at all possible. 
 And I see it does seem to be possible. I just can't figure out how.

>>>
>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and 
>>> run that if it exists. That's just a Julia file, so you can do whatever you 
>>> want there (shell out, etc.). 
>>>
>>
>> Perfect. For the most part I can just run configure, make, make install 
>> for now and set some library paths (if I can figure out what kind of system 
>> I am on).
>>
>> Finding the Julia installation on the system in order to link against the 
>> gmp/mpfr might be slightly more difficult.
>>  
>>
>>> One option is to use the BinDeps package which provides primitives for 
>>> interacting with various package managers and build systems:
>>>
>>> https://github.com/JuliaLang/BinDeps.jl
>>>
>>> A very advanced and fully-developed usage example can be found in the 
>>> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>>>
>>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>>>
>>> There are a number of other examples to draw from. Hopefully the above 
>>> links will give you a sense of where to start. I can help out on Linux and 
>>> Windows (@ihnorton on github). 
>>>
>>
>> Thanks.
>>
>> Bill.
>>  
>>
>>>
>>>
>>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart  
>>> wrote:
>>>
 Hi,

 I have been writing a new Julia package, which I have called Nemo (it's 
 essentially a limited computer algebra system).

 I have two specific problems:

 1) The git and Git

Re: [julia-users] Help needed with creating Julia package

[Johan Sigfrids]

Have you looked at  Reexport.jl 
?

On Friday, September 12, 2014 1:57:06 AM UTC+3, Bill Hart wrote:
>
> OK, I can build Nemo. But how do I load modules from Nemo now that it is 
> installed and built.
>
> For example "using Nemo", "using Rings", "using Fields" all fail, 
> complaining that it can't find the modules.
>
> I must be missing a step somewhere.
>
> Bill.
>
> On 12 September 2014 00:47, Bill Hart  > wrote:
>
>> It's ok, I got it. Pkg.build()
>>
>> Bill.
>>
>> On 12 September 2014 00:38, Bill Hart > > wrote:
>>
>>> I had a go at making a preliminary package which should vaguely build on 
>>> Linux when "using Nemo" is run from within the src/ directory.
>>>
>>> I checked this works on my machine at least.
>>>
>>> I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";) 
>>> clones the Nemo repository from within Julia. But this seems to only clone 
>>> the repository and doesn't appear to issue "using Nemo" as I had expected.
>>>
>>> Also typing "using Nemo" manually says that it can't find Nemo. What 
>>> command to users have to issue to get "using Nemo" to actually do something?
>>>
>>> Bill.
>>>
>>>
>>>
>>> On 10 September 2014 16:26, Isaiah Norton >> > wrote:
>>>

 This was what I thought of trying first. But I couldn't figure out how 
> it worked out what GitHub repository to associate this with, or whether 
> it 
> would try to create one, possibly scrubbing my existing nemo repository 
> on 
> GitHub. Obviously I don't want to lose my commit history.


 For Pkg manager purposes, the association will be created later (when 
 you register the package).

 It also isn't clear where Julia creates the empty git repository. In 
> the current directory? Or in some subdirectory of the Julia source tree?


 Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)

 For the most part I can just run configure, make, make install for now 
> and set some library paths (if I can figure out what kind of system I am 
> on).


 There are some macros to help with this: @osx, @linux, @unix (both), 
 and @windows. There is also a variable called OS_NAME with a 
 platform-specific value (:Windows, :Linux, etc.) See:

 http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations

  

 On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart >>> > wrote:

>
>
> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>
>> Is there documentation somewhere explaining how to do the latter? Or 
>>> can someone help me with doing the latter?
>>
>>
>> You could run `Pkg.generate("Nemo")` and then copy and commit (some 
>> of) the resulting files in your own Nemo git tree; there aren't very 
>> many.
>>
>
> This was what I thought of trying first. But I couldn't figure out how 
> it worked out what GitHub repository to associate this with, or whether 
> it 
> would try to create one, possibly scrubbing my existing nemo repository 
> on 
> GitHub. Obviously I don't want to lose my commit history.
>  
> It also isn't clear where Julia creates the empty git repository. In 
> the current directory? Or in some subdirectory of the Julia source tree?
>
>
>> I can't find any documentation explaining where to put the commands 
>>> in a Pkg to actually git clone flint, build it, install it and set up 
>>> paths 
>>> for Nemo. Given the complexities of installing flint for the user, I'd 
>>> like 
>>> to have the Julia package manager do this automatically if at all 
>>> possible. 
>>> And I see it does seem to be possible. I just can't figure out how.
>>>
>>
>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and 
>> run that if it exists. That's just a Julia file, so you can do whatever 
>> you 
>> want there (shell out, etc.). 
>>
>
> Perfect. For the most part I can just run configure, make, make 
> install for now and set some library paths (if I can figure out what kind 
> of system I am on).
>
> Finding the Julia installation on the system in order to link against 
> the gmp/mpfr might be slightly more difficult.
>  
>
>> One option is to use the BinDeps package which provides primitives 
>> for interacting with various package managers and build systems:
>>
>> https://github.com/JuliaLang/BinDeps.jl
>>
>> A very advanced and fully-developed usage example can be found in the 
>> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>>
>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>>
>> There are a number of other examples to draw from. Hopefully the 
>> above links will give you a sense of

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

I guess I figured it out.

On 12 September 2014 01:12, Bill Hart  wrote:

> Yes.
>
> I notice that after building Nemo it is in a directory called nemo, not
> Nemo.
>
> At the moment Nemo.jl simply builds Nemo by including deps/build.jl, since
> this is what Cairo does.
>
> The code the user will be interested in is in Rings.jl and Fields.jl. How
> do they load those?
>
> Bill.
>
> On 12 September 2014 01:08, Leah Hanson  wrote:
>
>> Does your Nemo.jl contain
>>
>> ~~~
>> module Nemo
>> 
>> end
>> ~~~
>>
>> ?
>>
>> On Thu, Sep 11, 2014 at 5:56 PM, Bill Hart 
>> wrote:
>>
>>> OK, I can build Nemo. But how do I load modules from Nemo now that it is
>>> installed and built.
>>>
>>> For example "using Nemo", "using Rings", "using Fields" all fail,
>>> complaining that it can't find the modules.
>>>
>>> I must be missing a step somewhere.
>>>
>>> Bill.
>>>
>>> On 12 September 2014 00:47, Bill Hart 
>>> wrote:
>>>
 It's ok, I got it. Pkg.build()

 Bill.

 On 12 September 2014 00:38, Bill Hart 
 wrote:

> I had a go at making a preliminary package which should vaguely build
> on Linux when "using Nemo" is run from within the src/ directory.
>
> I checked this works on my machine at least.
>
> I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";)
> clones the Nemo repository from within Julia. But this seems to only clone
> the repository and doesn't appear to issue "using Nemo" as I had expected.
>
> Also typing "using Nemo" manually says that it can't find Nemo. What
> command to users have to issue to get "using Nemo" to actually do 
> something?
>
> Bill.
>
>
>
> On 10 September 2014 16:26, Isaiah Norton 
> wrote:
>
>>
>> This was what I thought of trying first. But I couldn't figure out
>>> how it worked out what GitHub repository to associate this with, or 
>>> whether
>>> it would try to create one, possibly scrubbing my existing nemo 
>>> repository
>>> on GitHub. Obviously I don't want to lose my commit history.
>>
>>
>> For Pkg manager purposes, the association will be created later (when
>> you register the package).
>>
>> It also isn't clear where Julia creates the empty git repository. In
>>> the current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>>
>> For the most part I can just run configure, make, make install for
>>> now and set some library paths (if I can figure out what kind of system 
>>> I
>>> am on).
>>
>>
>> There are some macros to help with this: @osx, @linux, @unix (both),
>> and @windows. There is also a variable called OS_NAME with a
>> platform-specific value (:Windows, :Linux, etc.) See:
>>
>> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>>
>>
>>
>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart <
>> goodwillh...@googlemail.com> wrote:
>>
>>>
>>>
>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:

 Is there documentation somewhere explaining how to do the latter?
> Or can someone help me with doing the latter?


 You could run `Pkg.generate("Nemo")` and then copy and commit (some
 of) the resulting files in your own Nemo git tree; there aren't very 
 many.

>>>
>>> This was what I thought of trying first. But I couldn't figure out
>>> how it worked out what GitHub repository to associate this with, or 
>>> whether
>>> it would try to create one, possibly scrubbing my existing nemo 
>>> repository
>>> on GitHub. Obviously I don't want to lose my commit history.
>>>
>>> It also isn't clear where Julia creates the empty git repository. In
>>> the current directory? Or in some subdirectory of the Julia source tree?
>>>
>>>
 I can't find any documentation explaining where to put the commands
> in a Pkg to actually git clone flint, build it, install it and set up 
> paths
> for Nemo. Given the complexities of installing flint for the user, 
> I'd like
> to have the Julia package manager do this automatically if at all 
> possible.
> And I see it does seem to be possible. I just can't figure out how.
>

 The Pkg manager will look for a file called `MYPKG/deps/build.jl`
 and run that if it exists. That's just a Julia file, so you can do 
 whatever
 you want there (shell out, etc.).

>>>
>>> Perfect. For the most part I can just run configure, make, make
>>> install for now and set some library paths (if I can figure out what 
>>> kind
>>> of system I am on).
>>>
>>> Finding the Julia installation on t

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Yes.

I notice that after building Nemo it is in a directory called nemo, not
Nemo.

At the moment Nemo.jl simply builds Nemo by including deps/build.jl, since
this is what Cairo does.

The code the user will be interested in is in Rings.jl and Fields.jl. How
do they load those?

Bill.

On 12 September 2014 01:08, Leah Hanson  wrote:

> Does your Nemo.jl contain
>
> ~~~
> module Nemo
> 
> end
> ~~~
>
> ?
>
> On Thu, Sep 11, 2014 at 5:56 PM, Bill Hart 
> wrote:
>
>> OK, I can build Nemo. But how do I load modules from Nemo now that it is
>> installed and built.
>>
>> For example "using Nemo", "using Rings", "using Fields" all fail,
>> complaining that it can't find the modules.
>>
>> I must be missing a step somewhere.
>>
>> Bill.
>>
>> On 12 September 2014 00:47, Bill Hart 
>> wrote:
>>
>>> It's ok, I got it. Pkg.build()
>>>
>>> Bill.
>>>
>>> On 12 September 2014 00:38, Bill Hart 
>>> wrote:
>>>
 I had a go at making a preliminary package which should vaguely build
 on Linux when "using Nemo" is run from within the src/ directory.

 I checked this works on my machine at least.

 I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";)
 clones the Nemo repository from within Julia. But this seems to only clone
 the repository and doesn't appear to issue "using Nemo" as I had expected.

 Also typing "using Nemo" manually says that it can't find Nemo. What
 command to users have to issue to get "using Nemo" to actually do 
 something?

 Bill.



 On 10 September 2014 16:26, Isaiah Norton 
 wrote:

>
> This was what I thought of trying first. But I couldn't figure out how
>> it worked out what GitHub repository to associate this with, or whether 
>> it
>> would try to create one, possibly scrubbing my existing nemo repository 
>> on
>> GitHub. Obviously I don't want to lose my commit history.
>
>
> For Pkg manager purposes, the association will be created later (when
> you register the package).
>
> It also isn't clear where Julia creates the empty git repository. In
>> the current directory? Or in some subdirectory of the Julia source tree?
>
>
> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>
> For the most part I can just run configure, make, make install for now
>> and set some library paths (if I can figure out what kind of system I am
>> on).
>
>
> There are some macros to help with this: @osx, @linux, @unix (both),
> and @windows. There is also a variable called OS_NAME with a
> platform-specific value (:Windows, :Linux, etc.) See:
>
> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>
>
>
> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart <
> goodwillh...@googlemail.com> wrote:
>
>>
>>
>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>>
>>> Is there documentation somewhere explaining how to do the latter? Or
 can someone help me with doing the latter?
>>>
>>>
>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some
>>> of) the resulting files in your own Nemo git tree; there aren't very 
>>> many.
>>>
>>
>> This was what I thought of trying first. But I couldn't figure out
>> how it worked out what GitHub repository to associate this with, or 
>> whether
>> it would try to create one, possibly scrubbing my existing nemo 
>> repository
>> on GitHub. Obviously I don't want to lose my commit history.
>>
>> It also isn't clear where Julia creates the empty git repository. In
>> the current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>>> I can't find any documentation explaining where to put the commands
 in a Pkg to actually git clone flint, build it, install it and set up 
 paths
 for Nemo. Given the complexities of installing flint for the user, I'd 
 like
 to have the Julia package manager do this automatically if at all 
 possible.
 And I see it does seem to be possible. I just can't figure out how.

>>>
>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl`
>>> and run that if it exists. That's just a Julia file, so you can do 
>>> whatever
>>> you want there (shell out, etc.).
>>>
>>
>> Perfect. For the most part I can just run configure, make, make
>> install for now and set some library paths (if I can figure out what kind
>> of system I am on).
>>
>> Finding the Julia installation on the system in order to link against
>> the gmp/mpfr might be slightly more difficult.
>>
>>
>>> One option is to use the BinDeps package which provides primitives
>>> for interacting with various package managers and build system

Re: [julia-users] Help needed with creating Julia package

[Leah Hanson]

Does your Nemo.jl contain

~~~
module Nemo

end
~~~

?

On Thu, Sep 11, 2014 at 5:56 PM, Bill Hart 
wrote:

> OK, I can build Nemo. But how do I load modules from Nemo now that it is
> installed and built.
>
> For example "using Nemo", "using Rings", "using Fields" all fail,
> complaining that it can't find the modules.
>
> I must be missing a step somewhere.
>
> Bill.
>
> On 12 September 2014 00:47, Bill Hart  wrote:
>
>> It's ok, I got it. Pkg.build()
>>
>> Bill.
>>
>> On 12 September 2014 00:38, Bill Hart 
>> wrote:
>>
>>> I had a go at making a preliminary package which should vaguely build on
>>> Linux when "using Nemo" is run from within the src/ directory.
>>>
>>> I checked this works on my machine at least.
>>>
>>> I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";)
>>> clones the Nemo repository from within Julia. But this seems to only clone
>>> the repository and doesn't appear to issue "using Nemo" as I had expected.
>>>
>>> Also typing "using Nemo" manually says that it can't find Nemo. What
>>> command to users have to issue to get "using Nemo" to actually do something?
>>>
>>> Bill.
>>>
>>>
>>>
>>> On 10 September 2014 16:26, Isaiah Norton 
>>> wrote:
>>>

 This was what I thought of trying first. But I couldn't figure out how
> it worked out what GitHub repository to associate this with, or whether it
> would try to create one, possibly scrubbing my existing nemo repository on
> GitHub. Obviously I don't want to lose my commit history.


 For Pkg manager purposes, the association will be created later (when
 you register the package).

 It also isn't clear where Julia creates the empty git repository. In
> the current directory? Or in some subdirectory of the Julia source tree?


 Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)

 For the most part I can just run configure, make, make install for now
> and set some library paths (if I can figure out what kind of system I am
> on).


 There are some macros to help with this: @osx, @linux, @unix (both),
 and @windows. There is also a variable called OS_NAME with a
 platform-specific value (:Windows, :Linux, etc.) See:

 http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations



 On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart <
 goodwillh...@googlemail.com> wrote:

>
>
> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>
>> Is there documentation somewhere explaining how to do the latter? Or
>>> can someone help me with doing the latter?
>>
>>
>> You could run `Pkg.generate("Nemo")` and then copy and commit (some
>> of) the resulting files in your own Nemo git tree; there aren't very 
>> many.
>>
>
> This was what I thought of trying first. But I couldn't figure out how
> it worked out what GitHub repository to associate this with, or whether it
> would try to create one, possibly scrubbing my existing nemo repository on
> GitHub. Obviously I don't want to lose my commit history.
>
> It also isn't clear where Julia creates the empty git repository. In
> the current directory? Or in some subdirectory of the Julia source tree?
>
>
>> I can't find any documentation explaining where to put the commands
>>> in a Pkg to actually git clone flint, build it, install it and set up 
>>> paths
>>> for Nemo. Given the complexities of installing flint for the user, I'd 
>>> like
>>> to have the Julia package manager do this automatically if at all 
>>> possible.
>>> And I see it does seem to be possible. I just can't figure out how.
>>>
>>
>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and
>> run that if it exists. That's just a Julia file, so you can do whatever 
>> you
>> want there (shell out, etc.).
>>
>
> Perfect. For the most part I can just run configure, make, make
> install for now and set some library paths (if I can figure out what kind
> of system I am on).
>
> Finding the Julia installation on the system in order to link against
> the gmp/mpfr might be slightly more difficult.
>
>
>> One option is to use the BinDeps package which provides primitives
>> for interacting with various package managers and build systems:
>>
>> https://github.com/JuliaLang/BinDeps.jl
>>
>> A very advanced and fully-developed usage example can be found in the
>> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>>
>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>>
>> There are a number of other examples to draw from. Hopefully the
>> above links will give you a sense of where to start. I can help out on
>> Linux and Windows (@ihnorton on github).
>>
>
>

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

OK, I can build Nemo. But how do I load modules from Nemo now that it is
installed and built.

For example "using Nemo", "using Rings", "using Fields" all fail,
complaining that it can't find the modules.

I must be missing a step somewhere.

Bill.

On 12 September 2014 00:47, Bill Hart  wrote:

> It's ok, I got it. Pkg.build()
>
> Bill.
>
> On 12 September 2014 00:38, Bill Hart  wrote:
>
>> I had a go at making a preliminary package which should vaguely build on
>> Linux when "using Nemo" is run from within the src/ directory.
>>
>> I checked this works on my machine at least.
>>
>> I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";)
>> clones the Nemo repository from within Julia. But this seems to only clone
>> the repository and doesn't appear to issue "using Nemo" as I had expected.
>>
>> Also typing "using Nemo" manually says that it can't find Nemo. What
>> command to users have to issue to get "using Nemo" to actually do something?
>>
>> Bill.
>>
>>
>>
>> On 10 September 2014 16:26, Isaiah Norton 
>> wrote:
>>
>>>
>>> This was what I thought of trying first. But I couldn't figure out how
 it worked out what GitHub repository to associate this with, or whether it
 would try to create one, possibly scrubbing my existing nemo repository on
 GitHub. Obviously I don't want to lose my commit history.
>>>
>>>
>>> For Pkg manager purposes, the association will be created later (when
>>> you register the package).
>>>
>>> It also isn't clear where Julia creates the empty git repository. In the
 current directory? Or in some subdirectory of the Julia source tree?
>>>
>>>
>>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>>>
>>> For the most part I can just run configure, make, make install for now
 and set some library paths (if I can figure out what kind of system I am
 on).
>>>
>>>
>>> There are some macros to help with this: @osx, @linux, @unix (both), and
>>> @windows. There is also a variable called OS_NAME with a platform-specific
>>> value (:Windows, :Linux, etc.) See:
>>>
>>> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>>>
>>>
>>>
>>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart >> > wrote:
>>>


 On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>
> Is there documentation somewhere explaining how to do the latter? Or
>> can someone help me with doing the latter?
>
>
> You could run `Pkg.generate("Nemo")` and then copy and commit (some
> of) the resulting files in your own Nemo git tree; there aren't very many.
>

 This was what I thought of trying first. But I couldn't figure out how
 it worked out what GitHub repository to associate this with, or whether it
 would try to create one, possibly scrubbing my existing nemo repository on
 GitHub. Obviously I don't want to lose my commit history.

 It also isn't clear where Julia creates the empty git repository. In
 the current directory? Or in some subdirectory of the Julia source tree?


> I can't find any documentation explaining where to put the commands in
>> a Pkg to actually git clone flint, build it, install it and set up paths
>> for Nemo. Given the complexities of installing flint for the user, I'd 
>> like
>> to have the Julia package manager do this automatically if at all 
>> possible.
>> And I see it does seem to be possible. I just can't figure out how.
>>
>
> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and
> run that if it exists. That's just a Julia file, so you can do whatever 
> you
> want there (shell out, etc.).
>

 Perfect. For the most part I can just run configure, make, make install
 for now and set some library paths (if I can figure out what kind of system
 I am on).

 Finding the Julia installation on the system in order to link against
 the gmp/mpfr might be slightly more difficult.


> One option is to use the BinDeps package which provides primitives for
> interacting with various package managers and build systems:
>
> https://github.com/JuliaLang/BinDeps.jl
>
> A very advanced and fully-developed usage example can be found in the
> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>
> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>
> There are a number of other examples to draw from. Hopefully the above
> links will give you a sense of where to start. I can help out on Linux and
> Windows (@ihnorton on github).
>

 Thanks.

 Bill.


>
>
> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart 
> wrote:
>
>> Hi,
>>
>> I have been writing a new Julia package, which I have called Nemo
>> (it's essentially a limited computer algebra system).
>>
>> I have t

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

It's ok, I got it. Pkg.build()

Bill.

On 12 September 2014 00:38, Bill Hart  wrote:

> I had a go at making a preliminary package which should vaguely build on
> Linux when "using Nemo" is run from within the src/ directory.
>
> I checked this works on my machine at least.
>
> I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";)
> clones the Nemo repository from within Julia. But this seems to only clone
> the repository and doesn't appear to issue "using Nemo" as I had expected.
>
> Also typing "using Nemo" manually says that it can't find Nemo. What
> command to users have to issue to get "using Nemo" to actually do something?
>
> Bill.
>
>
>
> On 10 September 2014 16:26, Isaiah Norton  wrote:
>
>>
>> This was what I thought of trying first. But I couldn't figure out how it
>>> worked out what GitHub repository to associate this with, or whether it
>>> would try to create one, possibly scrubbing my existing nemo repository on
>>> GitHub. Obviously I don't want to lose my commit history.
>>
>>
>> For Pkg manager purposes, the association will be created later (when you
>> register the package).
>>
>> It also isn't clear where Julia creates the empty git repository. In the
>>> current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>>
>> For the most part I can just run configure, make, make install for now
>>> and set some library paths (if I can figure out what kind of system I am
>>> on).
>>
>>
>> There are some macros to help with this: @osx, @linux, @unix (both), and
>> @windows. There is also a variable called OS_NAME with a platform-specific
>> value (:Windows, :Linux, etc.) See:
>>
>> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>>
>>
>>
>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart 
>> wrote:
>>
>>>
>>>
>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:

 Is there documentation somewhere explaining how to do the latter? Or
> can someone help me with doing the latter?


 You could run `Pkg.generate("Nemo")` and then copy and commit (some of)
 the resulting files in your own Nemo git tree; there aren't very many.

>>>
>>> This was what I thought of trying first. But I couldn't figure out how
>>> it worked out what GitHub repository to associate this with, or whether it
>>> would try to create one, possibly scrubbing my existing nemo repository on
>>> GitHub. Obviously I don't want to lose my commit history.
>>>
>>> It also isn't clear where Julia creates the empty git repository. In the
>>> current directory? Or in some subdirectory of the Julia source tree?
>>>
>>>
 I can't find any documentation explaining where to put the commands in
> a Pkg to actually git clone flint, build it, install it and set up paths
> for Nemo. Given the complexities of installing flint for the user, I'd 
> like
> to have the Julia package manager do this automatically if at all 
> possible.
> And I see it does seem to be possible. I just can't figure out how.
>

 The Pkg manager will look for a file called `MYPKG/deps/build.jl` and
 run that if it exists. That's just a Julia file, so you can do whatever you
 want there (shell out, etc.).

>>>
>>> Perfect. For the most part I can just run configure, make, make install
>>> for now and set some library paths (if I can figure out what kind of system
>>> I am on).
>>>
>>> Finding the Julia installation on the system in order to link against
>>> the gmp/mpfr might be slightly more difficult.
>>>
>>>
 One option is to use the BinDeps package which provides primitives for
 interacting with various package managers and build systems:

 https://github.com/JuliaLang/BinDeps.jl

 A very advanced and fully-developed usage example can be found in the
 Cairo package, which has Autotools, Apt, Yum, and several other targets:

 https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl

 There are a number of other examples to draw from. Hopefully the above
 links will give you a sense of where to start. I can help out on Linux and
 Windows (@ihnorton on github).

>>>
>>> Thanks.
>>>
>>> Bill.
>>>
>>>


 On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart 
 wrote:

> Hi,
>
> I have been writing a new Julia package, which I have called Nemo
> (it's essentially a limited computer algebra system).
>
> I have two specific problems:
>
> 1) The git and GitHub repository for Nemo already exists, but I
> haven't created a Julia Pkg yet.
>
> https://github.com/wbhart/nemo
>
> The documentation on creating a Julia Pkg seems to assume you are
> going to start with the Pkg then commit code to the git repository that it
> creates, not create a git/github project and then add the necessary stuff
> to turn it into

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

I had a go at making a preliminary package which should vaguely build on
Linux when "using Nemo" is run from within the src/ directory.

I checked this works on my machine at least.

I also checked that Pkg.clone("https://github.com/wbhart/nemo.git";) clones
the Nemo repository from within Julia. But this seems to only clone the
repository and doesn't appear to issue "using Nemo" as I had expected.

Also typing "using Nemo" manually says that it can't find Nemo. What
command to users have to issue to get "using Nemo" to actually do something?

Bill.



On 10 September 2014 16:26, Isaiah Norton  wrote:

>
> This was what I thought of trying first. But I couldn't figure out how it
>> worked out what GitHub repository to associate this with, or whether it
>> would try to create one, possibly scrubbing my existing nemo repository on
>> GitHub. Obviously I don't want to lose my commit history.
>
>
> For Pkg manager purposes, the association will be created later (when you
> register the package).
>
> It also isn't clear where Julia creates the empty git repository. In the
>> current directory? Or in some subdirectory of the Julia source tree?
>
>
> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>
> For the most part I can just run configure, make, make install for now and
>> set some library paths (if I can figure out what kind of system I am on).
>
>
> There are some macros to help with this: @osx, @linux, @unix (both), and
> @windows. There is also a variable called OS_NAME with a platform-specific
> value (:Windows, :Linux, etc.) See:
>
> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>
>
>
> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart 
> wrote:
>
>>
>>
>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>>
>>> Is there documentation somewhere explaining how to do the latter? Or can
 someone help me with doing the latter?
>>>
>>>
>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some of)
>>> the resulting files in your own Nemo git tree; there aren't very many.
>>>
>>
>> This was what I thought of trying first. But I couldn't figure out how it
>> worked out what GitHub repository to associate this with, or whether it
>> would try to create one, possibly scrubbing my existing nemo repository on
>> GitHub. Obviously I don't want to lose my commit history.
>>
>> It also isn't clear where Julia creates the empty git repository. In the
>> current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>>> I can't find any documentation explaining where to put the commands in a
 Pkg to actually git clone flint, build it, install it and set up paths for
 Nemo. Given the complexities of installing flint for the user, I'd like to
 have the Julia package manager do this automatically if at all possible.
 And I see it does seem to be possible. I just can't figure out how.

>>>
>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and
>>> run that if it exists. That's just a Julia file, so you can do whatever you
>>> want there (shell out, etc.).
>>>
>>
>> Perfect. For the most part I can just run configure, make, make install
>> for now and set some library paths (if I can figure out what kind of system
>> I am on).
>>
>> Finding the Julia installation on the system in order to link against the
>> gmp/mpfr might be slightly more difficult.
>>
>>
>>> One option is to use the BinDeps package which provides primitives for
>>> interacting with various package managers and build systems:
>>>
>>> https://github.com/JuliaLang/BinDeps.jl
>>>
>>> A very advanced and fully-developed usage example can be found in the
>>> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>>>
>>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>>>
>>> There are a number of other examples to draw from. Hopefully the above
>>> links will give you a sense of where to start. I can help out on Linux and
>>> Windows (@ihnorton on github).
>>>
>>
>> Thanks.
>>
>> Bill.
>>
>>
>>>
>>>
>>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart 
>>> wrote:
>>>
 Hi,

 I have been writing a new Julia package, which I have called Nemo (it's
 essentially a limited computer algebra system).

 I have two specific problems:

 1) The git and GitHub repository for Nemo already exists, but I haven't
 created a Julia Pkg yet.

 https://github.com/wbhart/nemo

 The documentation on creating a Julia Pkg seems to assume you are going
 to start with the Pkg then commit code to the git repository that it
 creates, not create a git/github project and then add the necessary stuff
 to turn it into a Julia package.

 Is there documentation somewhere explaining how to do the latter? Or
 can someone help me with doing the latter?

 (I have a couple of small build issues to fix in order for flint to
 work on Windows 6

Re: [julia-users] Help needed with creating Julia package

[Kevin Squire]

You also might want to check out
http://julia.readthedocs.org/en/latest/manual/packages/#package-development

Cheers,
   Kevin

On Wednesday, September 10, 2014, Bill Hart 
wrote:

> Thanks very much for the information. I might have additional specific
> questions along the way, but this definitely helps massively for now.
>
> On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote:
>>
>>
>> This was what I thought of trying first. But I couldn't figure out how it
>>> worked out what GitHub repository to associate this with, or whether it
>>> would try to create one, possibly scrubbing my existing nemo repository on
>>> GitHub. Obviously I don't want to lose my commit history.
>>
>>
>> For Pkg manager purposes, the association will be created later (when you
>> register the package).
>>
>> It also isn't clear where Julia creates the empty git repository. In the
>>> current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>>
>> For the most part I can just run configure, make, make install for now
>>> and set some library paths (if I can figure out what kind of system I am
>>> on).
>>
>>
>> There are some macros to help with this: @osx, @linux, @unix (both), and
>> @windows. There is also a variable called OS_NAME with a platform-specific
>> value (:Windows, :Linux, etc.) See:
>> http://docs.julialang.org/en/release-0.3/manual/calling-c-
>> and-fortran-code/#handling-platform-variations
>>
>>
>>
>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart 
>> wrote:
>>
>>>
>>>
>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:

 Is there documentation somewhere explaining how to do the latter? Or
> can someone help me with doing the latter?


 You could run `Pkg.generate("Nemo")` and then copy and commit (some of)
 the resulting files in your own Nemo git tree; there aren't very many.

>>>
>>> This was what I thought of trying first. But I couldn't figure out how
>>> it worked out what GitHub repository to associate this with, or whether it
>>> would try to create one, possibly scrubbing my existing nemo repository on
>>> GitHub. Obviously I don't want to lose my commit history.
>>>
>>> It also isn't clear where Julia creates the empty git repository. In the
>>> current directory? Or in some subdirectory of the Julia source tree?
>>>
>>>
 I can't find any documentation explaining where to put the commands in
> a Pkg to actually git clone flint, build it, install it and set up paths
> for Nemo. Given the complexities of installing flint for the user, I'd 
> like
> to have the Julia package manager do this automatically if at all 
> possible.
> And I see it does seem to be possible. I just can't figure out how.
>

 The Pkg manager will look for a file called `MYPKG/deps/build.jl` and
 run that if it exists. That's just a Julia file, so you can do whatever you
 want there (shell out, etc.).

>>>
>>> Perfect. For the most part I can just run configure, make, make install
>>> for now and set some library paths (if I can figure out what kind of system
>>> I am on).
>>>
>>> Finding the Julia installation on the system in order to link against
>>> the gmp/mpfr might be slightly more difficult.
>>>
>>>
 One option is to use the BinDeps package which provides primitives for
 interacting with various package managers and build systems:

 https://github.com/JuliaLang/BinDeps.jl

 A very advanced and fully-developed usage example can be found in the
 Cairo package, which has Autotools, Apt, Yum, and several other targets:

 https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl

 There are a number of other examples to draw from. Hopefully the above
 links will give you a sense of where to start. I can help out on Linux and
 Windows (@ihnorton on github).

>>>
>>> Thanks.
>>>
>>> Bill.
>>>
>>>


 On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart 
 wrote:

> Hi,
>
> I have been writing a new Julia package, which I have called Nemo
> (it's essentially a limited computer algebra system).
>
> I have two specific problems:
>
> 1) The git and GitHub repository for Nemo already exists, but I
> haven't created a Julia Pkg yet.
>
> https://github.com/wbhart/nemo
>
> The documentation on creating a Julia Pkg seems to assume you are
> going to start with the Pkg then commit code to the git repository that it
> creates, not create a git/github project and then add the necessary stuff
> to turn it into a Julia package.
>
> Is there documentation somewhere explaining how to do the latter? Or
> can someone help me with doing the latter?
>
> (I have a couple of small build issues to fix in order for flint to
> work on Windows 64 before it will work there. But I will be working on
> those right away.

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Thanks very much for the information. I might have additional specific 
questions along the way, but this definitely helps massively for now.

On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote:
>
>
> This was what I thought of trying first. But I couldn't figure out how it 
>> worked out what GitHub repository to associate this with, or whether it 
>> would try to create one, possibly scrubbing my existing nemo repository on 
>> GitHub. Obviously I don't want to lose my commit history.
>
>
> For Pkg manager purposes, the association will be created later (when you 
> register the package).
>
> It also isn't clear where Julia creates the empty git repository. In the 
>> current directory? Or in some subdirectory of the Julia source tree?
>
>
> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)
>
> For the most part I can just run configure, make, make install for now and 
>> set some library paths (if I can figure out what kind of system I am on).
>
>
> There are some macros to help with this: @osx, @linux, @unix (both), and 
> @windows. There is also a variable called OS_NAME with a platform-specific 
> value (:Windows, :Linux, etc.) See:
>
> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations
>
>  
>
> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart  > wrote:
>
>>
>>
>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>>
>>> Is there documentation somewhere explaining how to do the latter? Or can 
 someone help me with doing the latter?
>>>
>>>
>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some of) 
>>> the resulting files in your own Nemo git tree; there aren't very many.
>>>
>>
>> This was what I thought of trying first. But I couldn't figure out how it 
>> worked out what GitHub repository to associate this with, or whether it 
>> would try to create one, possibly scrubbing my existing nemo repository on 
>> GitHub. Obviously I don't want to lose my commit history.
>>  
>> It also isn't clear where Julia creates the empty git repository. In the 
>> current directory? Or in some subdirectory of the Julia source tree?
>>
>>
>>> I can't find any documentation explaining where to put the commands in a 
 Pkg to actually git clone flint, build it, install it and set up paths for 
 Nemo. Given the complexities of installing flint for the user, I'd like to 
 have the Julia package manager do this automatically if at all possible. 
 And I see it does seem to be possible. I just can't figure out how.

>>>
>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and 
>>> run that if it exists. That's just a Julia file, so you can do whatever you 
>>> want there (shell out, etc.). 
>>>
>>
>> Perfect. For the most part I can just run configure, make, make install 
>> for now and set some library paths (if I can figure out what kind of system 
>> I am on).
>>
>> Finding the Julia installation on the system in order to link against the 
>> gmp/mpfr might be slightly more difficult.
>>  
>>
>>> One option is to use the BinDeps package which provides primitives for 
>>> interacting with various package managers and build systems:
>>>
>>> https://github.com/JuliaLang/BinDeps.jl
>>>
>>> A very advanced and fully-developed usage example can be found in the 
>>> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>>>
>>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>>>
>>> There are a number of other examples to draw from. Hopefully the above 
>>> links will give you a sense of where to start. I can help out on Linux and 
>>> Windows (@ihnorton on github). 
>>>
>>
>> Thanks.
>>
>> Bill.
>>  
>>
>>>
>>>
>>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart  
>>> wrote:
>>>
 Hi,

 I have been writing a new Julia package, which I have called Nemo (it's 
 essentially a limited computer algebra system).

 I have two specific problems:

 1) The git and GitHub repository for Nemo already exists, but I haven't 
 created a Julia Pkg yet.

 https://github.com/wbhart/nemo

 The documentation on creating a Julia Pkg seems to assume you are going 
 to start with the Pkg then commit code to the git repository that it 
 creates, not create a git/github project and then add the necessary stuff 
 to turn it into a Julia package.

 Is there documentation somewhere explaining how to do the latter? Or 
 can someone help me with doing the latter?

 (I have a couple of small build issues to fix in order for flint to 
 work on Windows 64 before it will work there. But I will be working on 
 those right away. I have managed to get it to work with Julia there, just 
 not hacked the fixes into the flint build system yet. Other than this 
 minor 
 thing, I am quite ready to publish Nemo as a package right away (well, 
 apart from a horrible 3x slowdown and excessive memor

Re: [julia-users] Help needed with creating Julia package

[Isaiah Norton]

> This was what I thought of trying first. But I couldn't figure out how it
> worked out what GitHub repository to associate this with, or whether it
> would try to create one, possibly scrubbing my existing nemo repository on
> GitHub. Obviously I don't want to lose my commit history.


For Pkg manager purposes, the association will be created later (when you
register the package).

It also isn't clear where Julia creates the empty git repository. In the
> current directory? Or in some subdirectory of the Julia source tree?


Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master)

For the most part I can just run configure, make, make install for now and
> set some library paths (if I can figure out what kind of system I am on).


There are some macros to help with this: @osx, @linux, @unix (both), and
@windows. There is also a variable called OS_NAME with a platform-specific
value (:Windows, :Linux, etc.) See:
http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations



On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart 
wrote:

>
>
> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>>
>> Is there documentation somewhere explaining how to do the latter? Or can
>>> someone help me with doing the latter?
>>
>>
>> You could run `Pkg.generate("Nemo")` and then copy and commit (some of)
>> the resulting files in your own Nemo git tree; there aren't very many.
>>
>
> This was what I thought of trying first. But I couldn't figure out how it
> worked out what GitHub repository to associate this with, or whether it
> would try to create one, possibly scrubbing my existing nemo repository on
> GitHub. Obviously I don't want to lose my commit history.
>
> It also isn't clear where Julia creates the empty git repository. In the
> current directory? Or in some subdirectory of the Julia source tree?
>
>
>> I can't find any documentation explaining where to put the commands in a
>>> Pkg to actually git clone flint, build it, install it and set up paths for
>>> Nemo. Given the complexities of installing flint for the user, I'd like to
>>> have the Julia package manager do this automatically if at all possible.
>>> And I see it does seem to be possible. I just can't figure out how.
>>>
>>
>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and run
>> that if it exists. That's just a Julia file, so you can do whatever you
>> want there (shell out, etc.).
>>
>
> Perfect. For the most part I can just run configure, make, make install
> for now and set some library paths (if I can figure out what kind of system
> I am on).
>
> Finding the Julia installation on the system in order to link against the
> gmp/mpfr might be slightly more difficult.
>
>
>> One option is to use the BinDeps package which provides primitives for
>> interacting with various package managers and build systems:
>>
>> https://github.com/JuliaLang/BinDeps.jl
>>
>> A very advanced and fully-developed usage example can be found in the
>> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>>
>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>>
>> There are a number of other examples to draw from. Hopefully the above
>> links will give you a sense of where to start. I can help out on Linux and
>> Windows (@ihnorton on github).
>>
>
> Thanks.
>
> Bill.
>
>
>>
>>
>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart 
>> wrote:
>>
>>> Hi,
>>>
>>> I have been writing a new Julia package, which I have called Nemo (it's
>>> essentially a limited computer algebra system).
>>>
>>> I have two specific problems:
>>>
>>> 1) The git and GitHub repository for Nemo already exists, but I haven't
>>> created a Julia Pkg yet.
>>>
>>> https://github.com/wbhart/nemo
>>>
>>> The documentation on creating a Julia Pkg seems to assume you are going
>>> to start with the Pkg then commit code to the git repository that it
>>> creates, not create a git/github project and then add the necessary stuff
>>> to turn it into a Julia package.
>>>
>>> Is there documentation somewhere explaining how to do the latter? Or can
>>> someone help me with doing the latter?
>>>
>>> (I have a couple of small build issues to fix in order for flint to work
>>> on Windows 64 before it will work there. But I will be working on those
>>> right away. I have managed to get it to work with Julia there, just not
>>> hacked the fixes into the flint build system yet. Other than this minor
>>> thing, I am quite ready to publish Nemo as a package right away (well,
>>> apart from a horrible 3x slowdown and excessive memory usage caused by gc,
>>> but I think I've given up on solving that problem for now).)
>>>
>>> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large
>>> external C/assembly libraries which need to get built or be available to
>>> run Nemo. I understand Julia has its own GMP and MPFR which I can probably
>>> link to if they are recent enough.
>>>
>>> Flint needs to be built when the p

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote:
>
> Is there documentation somewhere explaining how to do the latter? Or can 
>> someone help me with doing the latter?
>
>
> You could run `Pkg.generate("Nemo")` and then copy and commit (some of) 
> the resulting files in your own Nemo git tree; there aren't very many.
>

This was what I thought of trying first. But I couldn't figure out how it 
worked out what GitHub repository to associate this with, or whether it 
would try to create one, possibly scrubbing my existing nemo repository on 
GitHub. Obviously I don't want to lose my commit history.
 
It also isn't clear where Julia creates the empty git repository. In the 
current directory? Or in some subdirectory of the Julia source tree?


> I can't find any documentation explaining where to put the commands in a 
>> Pkg to actually git clone flint, build it, install it and set up paths for 
>> Nemo. Given the complexities of installing flint for the user, I'd like to 
>> have the Julia package manager do this automatically if at all possible. 
>> And I see it does seem to be possible. I just can't figure out how.
>>
>
> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and run 
> that if it exists. That's just a Julia file, so you can do whatever you 
> want there (shell out, etc.). 
>

Perfect. For the most part I can just run configure, make, make install for 
now and set some library paths (if I can figure out what kind of system I 
am on).

Finding the Julia installation on the system in order to link against the 
gmp/mpfr might be slightly more difficult.
 

> One option is to use the BinDeps package which provides primitives for 
> interacting with various package managers and build systems:
>
> https://github.com/JuliaLang/BinDeps.jl
>
> A very advanced and fully-developed usage example can be found in the 
> Cairo package, which has Autotools, Apt, Yum, and several other targets:
>
> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl
>
> There are a number of other examples to draw from. Hopefully the above 
> links will give you a sense of where to start. I can help out on Linux and 
> Windows (@ihnorton on github). 
>

Thanks.

Bill.
 

>
>
> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart  > wrote:
>
>> Hi,
>>
>> I have been writing a new Julia package, which I have called Nemo (it's 
>> essentially a limited computer algebra system).
>>
>> I have two specific problems:
>>
>> 1) The git and GitHub repository for Nemo already exists, but I haven't 
>> created a Julia Pkg yet.
>>
>> https://github.com/wbhart/nemo
>>
>> The documentation on creating a Julia Pkg seems to assume you are going 
>> to start with the Pkg then commit code to the git repository that it 
>> creates, not create a git/github project and then add the necessary stuff 
>> to turn it into a Julia package.
>>
>> Is there documentation somewhere explaining how to do the latter? Or can 
>> someone help me with doing the latter?
>>
>> (I have a couple of small build issues to fix in order for flint to work 
>> on Windows 64 before it will work there. But I will be working on those 
>> right away. I have managed to get it to work with Julia there, just not 
>> hacked the fixes into the flint build system yet. Other than this minor 
>> thing, I am quite ready to publish Nemo as a package right away (well, 
>> apart from a horrible 3x slowdown and excessive memory usage caused by gc, 
>> but I think I've given up on solving that problem for now).)
>>
>> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large external 
>> C/assembly libraries which need to get built or be available to run Nemo. I 
>> understand Julia has its own GMP and MPFR which I can probably link to if 
>> they are recent enough. 
>>
>> Flint needs to be built when the package is installed. It takes a long 
>> time to build, e.g. 40 minutes or so on Windows, maybe a third of that on 
>> Linux.
>>
>> I can't find any documentation explaining where to put the commands in a 
>> Pkg to actually git clone flint, build it, install it and set up paths for 
>> Nemo. Given the complexities of installing flint for the user, I'd like to 
>> have the Julia package manager do this automatically if at all possible. 
>> And I see it does seem to be possible. I just can't figure out how.
>>
>> Flint is here:
>>
>> https://github.com/wbhart/flint2
>>
>> Can anyone help, or point me in the right direction?
>>
>> Bill.
>>
>
>

Re: [julia-users] Help needed with creating Julia package

[Bill Hart]

Thanks for the reply. Some follow-up questions inline below.

On Wednesday, 10 September 2014 15:57:21 UTC+2, Leah Hanson wrote:
>
> 1) You can try looking at other packages to see the structure.
>

Thanks, I see you mention Cairo below. I'll try to find that on GitHub for 
some clues.
 

>
> Code goes is `src/` 
>

 Do you mean src/ in the Julia directory, or do you mean the Nemo code 
should go in a subdirectory called /src within the git repository?

And test should be a subdirectory of src/ or a separate directory. (I guess 
I can look at an example for this.)
 

> (and you'll need a `Nemo.jl` in there that will be what Julia runs when 
> you say `using Nemo`). 
>

I do have Nemo.jl, since I read I'd need this. But it is currently empty.

The problem, leading to another question, is that Nemo currently consists 
of two large modules called Rings and Fields (Fields uses Rings, but not 
vice versa). There will be more such modules in the future.

But I can't find a way for Nemo.jl to import everything from Rings and 
Fields and then export all those symbols out to the user if they type say 
`using Nemo`. Is there a way to do this without exporting all the symbols 
again individually from the Nemo module?

Tests go in the `tests` folder; you should have a file `runtests.jl` in 
> there that runs the tests for your package (if you want Julia's 
> auto-testing stuff to work).
>

Great. I wondered how to make that work. This should be easy to add. It 
will just call the two large sets of test functions I currently have.
 

>
> A file named REQUIRE is where any dependencies on other Julia go; it is 
> also where you specify compatible versions of Julia. Every package has one 
> of these, so you can look at them for examples to see the syntax. You 
> probably want a line like `julia 0.3-` if you believe you package works for 
> 0.3 and will work for 0.4, but not in 0.2. 
>

Ok.
 

>
> Like most OSS projects: license in LICENSE.md, some documentation in 
> README.md. (*.md means a plain text Markdown file; there's flexibility on 
> plain text format/filename here; these are just what's made by default by 
> `Pkg.generate`).
>

Ok.
 

>
> This is enough for people to get your package installed via the Julia 
> package manager by running `Pkg.clone("https://github.com/wbhart/nemo";)`.
>

Yes, that is all I want for the time being.
 

>
> Since Pkg.clone will put you package in the "right" place (i.e. where all 
> your other Julia packages are installed), you'll be able to follow the 
> normal package publishing (register with METADATA) instructions.
>

Ok.
 

>
> 2) https://github.com/JuliaLang/BinDeps.jl is the package for installing 
> binary dependencies.
>
> You should also search this mailing list for past BinDeps questions, as 
> there have been a number of them. Looking at other packages that use 
> BinDeps can also be very helpful; Cairo.jl is an example, but anything that 
> wraps C libraries would probably be helpful.
>
>
Will do. That is the keyword I needed, thanks.

Bill.
 

> Best,
> Leah
>
>
>
> On Wed, Sep 10, 2014 at 8:31 AM, Bill Hart  > wrote:
>
>> Hi,
>>
>> I have been writing a new Julia package, which I have called Nemo (it's 
>> essentially a limited computer algebra system).
>>
>> I have two specific problems:
>>
>> 1) The git and GitHub repository for Nemo already exists, but I haven't 
>> created a Julia Pkg yet.
>>
>> https://github.com/wbhart/nemo
>>
>> The documentation on creating a Julia Pkg seems to assume you are going 
>> to start with the Pkg then commit code to the git repository that it 
>> creates, not create a git/github project and then add the necessary stuff 
>> to turn it into a Julia package.
>>
>> Is there documentation somewhere explaining how to do the latter? Or can 
>> someone help me with doing the latter?
>>
>> (I have a couple of small build issues to fix in order for flint to work 
>> on Windows 64 before it will work there. But I will be working on those 
>> right away. I have managed to get it to work with Julia there, just not 
>> hacked the fixes into the flint build system yet. Other than this minor 
>> thing, I am quite ready to publish Nemo as a package right away (well, 
>> apart from a horrible 3x slowdown and excessive memory usage caused by gc, 
>> but I think I've given up on solving that problem for now).)
>>
>> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large external 
>> C/assembly libraries which need to get built or be available to run Nemo. I 
>> understand Julia has its own GMP and MPFR which I can probably link to if 
>> they are recent enough. 
>>
>> Flint needs to be built when the package is installed. It takes a long 
>> time to build, e.g. 40 minutes or so on Windows, maybe a third of that on 
>> Linux.
>>
>> I can't find any documentation explaining where to put the commands in a 
>> Pkg to actually git clone flint, build it, install it and set up paths for 
>> Nemo. Given the complexities of installing flint for the us

Re: [julia-users] Help needed with creating Julia package

[Isaiah Norton]

>
> Is there documentation somewhere explaining how to do the latter? Or can
> someone help me with doing the latter?


You could run `Pkg.generate("Nemo")` and then copy and commit (some of) the
resulting files in your own Nemo git tree; there aren't very many.

I can't find any documentation explaining where to put the commands in a
> Pkg to actually git clone flint, build it, install it and set up paths for
> Nemo. Given the complexities of installing flint for the user, I'd like to
> have the Julia package manager do this automatically if at all possible.
> And I see it does seem to be possible. I just can't figure out how.
>

The Pkg manager will look for a file called `MYPKG/deps/build.jl` and run
that if it exists. That's just a Julia file, so you can do whatever you
want there (shell out, etc.). One option is to use the BinDeps package
which provides primitives for interacting with various package managers and
build systems:

https://github.com/JuliaLang/BinDeps.jl

A very advanced and fully-developed usage example can be found in the Cairo
package, which has Autotools, Apt, Yum, and several other targets:

https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl

There are a number of other examples to draw from. Hopefully the above
links will give you a sense of where to start. I can help out on Linux and
Windows (@ihnorton on github).


On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart 
wrote:

> Hi,
>
> I have been writing a new Julia package, which I have called Nemo (it's
> essentially a limited computer algebra system).
>
> I have two specific problems:
>
> 1) The git and GitHub repository for Nemo already exists, but I haven't
> created a Julia Pkg yet.
>
> https://github.com/wbhart/nemo
>
> The documentation on creating a Julia Pkg seems to assume you are going to
> start with the Pkg then commit code to the git repository that it creates,
> not create a git/github project and then add the necessary stuff to turn it
> into a Julia package.
>
> Is there documentation somewhere explaining how to do the latter? Or can
> someone help me with doing the latter?
>
> (I have a couple of small build issues to fix in order for flint to work
> on Windows 64 before it will work there. But I will be working on those
> right away. I have managed to get it to work with Julia there, just not
> hacked the fixes into the flint build system yet. Other than this minor
> thing, I am quite ready to publish Nemo as a package right away (well,
> apart from a horrible 3x slowdown and excessive memory usage caused by gc,
> but I think I've given up on solving that problem for now).)
>
> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large external
> C/assembly libraries which need to get built or be available to run Nemo. I
> understand Julia has its own GMP and MPFR which I can probably link to if
> they are recent enough.
>
> Flint needs to be built when the package is installed. It takes a long
> time to build, e.g. 40 minutes or so on Windows, maybe a third of that on
> Linux.
>
> I can't find any documentation explaining where to put the commands in a
> Pkg to actually git clone flint, build it, install it and set up paths for
> Nemo. Given the complexities of installing flint for the user, I'd like to
> have the Julia package manager do this automatically if at all possible.
> And I see it does seem to be possible. I just can't figure out how.
>
> Flint is here:
>
> https://github.com/wbhart/flint2
>
> Can anyone help, or point me in the right direction?
>
> Bill.
>

Re: [julia-users] Help needed with creating Julia package

[Leah Hanson]

1) You can try looking at other packages to see the structure.

Code goes is `src/` (and you'll need a `Nemo.jl` in there that will be what
Julia runs when you say `using Nemo`). Tests go in the `tests` folder; you
should have a file `runtests.jl` in there that runs the tests for your
package (if you want Julia's auto-testing stuff to work).

A file named REQUIRE is where any dependencies on other Julia go; it is
also where you specify compatible versions of Julia. Every package has one
of these, so you can look at them for examples to see the syntax. You
probably want a line like `julia 0.3-` if you believe you package works for
0.3 and will work for 0.4, but not in 0.2.

Like most OSS projects: license in LICENSE.md, some documentation in
README.md. (*.md means a plain text Markdown file; there's flexibility on
plain text format/filename here; these are just what's made by default by
`Pkg.generate`).

This is enough for people to get your package installed via the Julia
package manager by running `Pkg.clone("https://github.com/wbhart/nemo";)`.

Since Pkg.clone will put you package in the "right" place (i.e. where all
your other Julia packages are installed), you'll be able to follow the
normal package publishing (register with METADATA) instructions.

2) https://github.com/JuliaLang/BinDeps.jl is the package for installing
binary dependencies.

You should also search this mailing list for past BinDeps questions, as
there have been a number of them. Looking at other packages that use
BinDeps can also be very helpful; Cairo.jl is an example, but anything that
wraps C libraries would probably be helpful.

Best,
Leah



On Wed, Sep 10, 2014 at 8:31 AM, Bill Hart 
wrote:

> Hi,
>
> I have been writing a new Julia package, which I have called Nemo (it's
> essentially a limited computer algebra system).
>
> I have two specific problems:
>
> 1) The git and GitHub repository for Nemo already exists, but I haven't
> created a Julia Pkg yet.
>
> https://github.com/wbhart/nemo
>
> The documentation on creating a Julia Pkg seems to assume you are going to
> start with the Pkg then commit code to the git repository that it creates,
> not create a git/github project and then add the necessary stuff to turn it
> into a Julia package.
>
> Is there documentation somewhere explaining how to do the latter? Or can
> someone help me with doing the latter?
>
> (I have a couple of small build issues to fix in order for flint to work
> on Windows 64 before it will work there. But I will be working on those
> right away. I have managed to get it to work with Julia there, just not
> hacked the fixes into the flint build system yet. Other than this minor
> thing, I am quite ready to publish Nemo as a package right away (well,
> apart from a horrible 3x slowdown and excessive memory usage caused by gc,
> but I think I've given up on solving that problem for now).)
>
> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large external
> C/assembly libraries which need to get built or be available to run Nemo. I
> understand Julia has its own GMP and MPFR which I can probably link to if
> they are recent enough.
>
> Flint needs to be built when the package is installed. It takes a long
> time to build, e.g. 40 minutes or so on Windows, maybe a third of that on
> Linux.
>
> I can't find any documentation explaining where to put the commands in a
> Pkg to actually git clone flint, build it, install it and set up paths for
> Nemo. Given the complexities of installing flint for the user, I'd like to
> have the Julia package manager do this automatically if at all possible.
> And I see it does seem to be possible. I just can't figure out how.
>
> Flint is here:
>
> https://github.com/wbhart/flint2
>
> Can anyone help, or point me in the right direction?
>
> Bill.
>

[julia-users] Help needed with creating Julia package

[Bill Hart]

Hi,

I have been writing a new Julia package, which I have called Nemo (it's 
essentially a limited computer algebra system).

I have two specific problems:

1) The git and GitHub repository for Nemo already exists, but I haven't 
created a Julia Pkg yet.

https://github.com/wbhart/nemo

The documentation on creating a Julia Pkg seems to assume you are going to 
start with the Pkg then commit code to the git repository that it creates, 
not create a git/github project and then add the necessary stuff to turn it 
into a Julia package.

Is there documentation somewhere explaining how to do the latter? Or can 
someone help me with doing the latter?

(I have a couple of small build issues to fix in order for flint to work on 
Windows 64 before it will work there. But I will be working on those right 
away. I have managed to get it to work with Julia there, just not hacked 
the fixes into the flint build system yet. Other than this minor thing, I 
am quite ready to publish Nemo as a package right away (well, apart from a 
horrible 3x slowdown and excessive memory usage caused by gc, but I think 
I've given up on solving that problem for now).)

2) Nemo relies on mpir (or GMP), mpfr and flint, which are large external 
C/assembly libraries which need to get built or be available to run Nemo. I 
understand Julia has its own GMP and MPFR which I can probably link to if 
they are recent enough. 

Flint needs to be built when the package is installed. It takes a long time 
to build, e.g. 40 minutes or so on Windows, maybe a third of that on Linux.

I can't find any documentation explaining where to put the commands in a 
Pkg to actually git clone flint, build it, install it and set up paths for 
Nemo. Given the complexities of installing flint for the user, I'd like to 
have the Julia package manager do this automatically if at all possible. 
And I see it does seem to be possible. I just can't figure out how.

Flint is here:

https://github.com/wbhart/flint2

Can anyone help, or point me in the right direction?

Bill.