Re: [Numpy-discussion] 64bit infrastructure
On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.iowrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. But, I would rather have official builds of NumPy for all platforms with a compiler paid for by a NumPy-sponsor than not have them. I would only want to have those if I can have the whole stack. If that's possible, then (if Christoph is okay with it) why not upload Christoph's binaries also to SF and call them official then? I don't see the point in duplicating his efforts. Ralf -Travis On Aug 20, 2012, at 5:28 PM, Chris Barker wrote: On Sun, Aug 19, 2012 at 11:39 AM, Ralf Gommers The problem is that, unlike 32-bit builds, they can't be made with open source compilers on Windows. So unless we're okay with that, Why does it have to be built with open-source compilers? we're building against the python.org python, yes? Which is built with the MS compiler -- so why the insistance on open-source? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Tue, Aug 21, 2012 at 12:15 AM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Aug 20, 2012 at 3:51 PM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. The MS Express editions, while not open source, are free-to-use, and work fine. Not sure what what do about Fortran, though, but that's a scipy, not a numpy issue, yes? fortran is the issue. Having one or two licenses of say Intel Fortran compiler is not enough because it makes it difficult for people to build on top of scipy. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 11:40 AM, David Cournapeau courn...@gmail.com wrote: On Tue, Aug 21, 2012 at 12:15 AM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Aug 20, 2012 at 3:51 PM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. The MS Express editions, while not open source, are free-to-use, and work fine. Not sure what what do about Fortran, though, but that's a scipy, not a numpy issue, yes? fortran is the issue. Having one or two licenses of say Intel Fortran compiler is not enough because it makes it difficult for people to build on top of scipy. David For those users/developers/packages using NumPy but not SciPy, does this matter? Having just official NumPy 64bit Windows packages would still be very welcome. Is the problem that whatever route NumPy goes down will have potential implications/restrictions for how SciPy could proceed? Peter ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 12:17 PM, Peter numpy-discuss...@maubp.freeserve.co.uk wrote: On Wed, Aug 22, 2012 at 11:40 AM, David Cournapeau courn...@gmail.com wrote: On Tue, Aug 21, 2012 at 12:15 AM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Aug 20, 2012 at 3:51 PM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. The MS Express editions, while not open source, are free-to-use, and work fine. Not sure what what do about Fortran, though, but that's a scipy, not a numpy issue, yes? fortran is the issue. Having one or two licenses of say Intel Fortran compiler is not enough because it makes it difficult for people to build on top of scipy. David For those users/developers/packages using NumPy but not SciPy, does this matter? Having just official NumPy 64bit Windows packages would still be very welcome. Is the problem that whatever route NumPy goes down will have potential implications/restrictions for how SciPy could proceed? Yes. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 11:40 AM, David Cournapeau courn...@gmail.com wrote: On Tue, Aug 21, 2012 at 12:15 AM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Aug 20, 2012 at 3:51 PM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. The MS Express editions, while not open source, are free-to-use, and work fine. Not sure what what do about Fortran, though, but that's a scipy, not a numpy issue, yes? fortran is the issue. Having one or two licenses of say Intel Fortran compiler is not enough because it makes it difficult for people to build on top of scipy. There seems to be a 64-bit version of mingw -- I assume this runs into the same issue that numpy has with new versions of 32-bit mingw? Would it work if that issue were fixed? -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 3:14 PM, Nathaniel Smith n...@pobox.com wrote: On Wed, Aug 22, 2012 at 11:40 AM, David Cournapeau courn...@gmail.com wrote: On Tue, Aug 21, 2012 at 12:15 AM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Aug 20, 2012 at 3:51 PM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. The MS Express editions, while not open source, are free-to-use, and work fine. Not sure what what do about Fortran, though, but that's a scipy, not a numpy issue, yes? fortran is the issue. Having one or two licenses of say Intel Fortran compiler is not enough because it makes it difficult for people to build on top of scipy. There seems to be a 64-bit version of mingw -- I assume this runs into the same issue that numpy has with new versions of 32-bit mingw? Would it work if that issue were fixed? It does have the same issue IIRC, but there was also the issue that scipy built with mingw-w64 did not work at all at that time. Could be different today, I have not looked into it for quite a while now. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. But, I would rather have official builds of NumPy for all platforms with a compiler paid for by a NumPy-sponsor than not have them. I would only want to have those if I can have the whole stack. If that's possible, then (if Christoph is okay with it) why not upload Christoph's binaries also to SF and call them official then? I don't see the point in duplicating his efforts. Yes, I agree. I would really like there to be several build machines, maintained by the community (NumFOCUS could sponsor this) which can be used to build windows binaries from a build specification. I think bento is a good tool. We could accelerate its use by offering machines that automatically build binaries for packages with bento scripts. If someone is interested in doing a project like this for the community, let me know as there are funds available to sponsor it --- just the lack of someone to see it through. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Aug 22, 2012, at 9:28 AM, David Cournapeau wrote: On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. Can you be more specific? Does the calling convention for C-routines created with Intel Fortran differ so much? -Travis David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 3:35 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 9:28 AM, David Cournapeau wrote: On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. Can you be more specific? Does the calling convention for C-routines created with Intel Fortran differ so much? If we were to use intel, it would be with MS compilers, and I have never been able to link a gfortran program with visual studio. I will try to take a look at it again during euroscipy, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 7:35 AM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 9:28 AM, David Cournapeau wrote: On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. Can you be more specific? Does the calling convention for C-routines created with Intel Fortran differ so much? I have the same question as Travis. If you are interested about ABI for Fortran, I have created this FAQ: http://fortran90.org/src/faq.html#are-fortran-compilers-abi-compatible Since NumPy only calls the Fortran routines, but does not expose them, then the only issue is how to build NumPy with (let's say) Intel Fortran. That's a separate issue. Once NumPy is built, then nobody cares, because they only need to interface the C routines, if anything at all. As far as Fortran runtime library goes (which of course is different for Intel and gfortran), I am currently not sure whether it is possible to mix them, but I think you probably can, if numpy .so is using Intel, and my own .so is using gfortran. Finally, if NumPy is build using MSVC, does this force everybody to use the same C compiler? I thought that C compilers are ABI compatible, at least Intel C and gfortran C are ABI compatible. Is MSVC different? Btw, to correctly call Fortran from C, one should always be using the iso_c_binding module, as explained here: http://fortran90.org/src/best-practices.html#interfacing-with-c Then the Fortran code becomes just like any other C library. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 4:12 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Aug 22, 2012 at 7:35 AM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 9:28 AM, David Cournapeau wrote: On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. Can you be more specific? Does the calling convention for C-routines created with Intel Fortran differ so much? I have the same question as Travis. If you are interested about ABI for Fortran, I have created this FAQ: http://fortran90.org/src/faq.html#are-fortran-compilers-abi-compatible Since NumPy only calls the Fortran routines, but does not expose them, then the only issue is how to build NumPy with (let's say) Intel Fortran. That's a separate issue. Once NumPy is built, then nobody cares, because they only need to interface the C routines, if anything at all. As far as Fortran runtime library goes (which of course is different for Intel and gfortran), I am currently not sure whether it is possible to mix them, but I think you probably can, if numpy .so is using Intel, and my own .so is using gfortran. Finally, if NumPy is build using MSVC, does this force everybody to use the same C compiler? I thought that C compilers are ABI compatible, at least Intel C and gfortran C are ABI compatible. Is MSVC different? Btw, to correctly call Fortran from C, one should always be using the iso_c_binding module, as explained here: http://fortran90.org/src/best-practices.html#interfacing-with-c Then the Fortran code becomes just like any other C library. It is unfortunately more complicated than that. 1 regarding fortran runtimes: I have never been able to link a gfortran object file with Visual Studio linker (link.exe). 2 mixing runtimes is never a good idea, because it becomes difficult to avoid passing a pointer from one runtime to the other. Intel fortran compiler obviously knows how to deal with the C runtime of Visual Studio, but gfortran doesn't. 3 gcc and visual studio are ABI compatible in a (very) restricted sense: they share the same calling convention at least in C, but that's pretty much it. Because having multiple copies of a runtime is so common on windows, you cannot easily pass objects between dlls. Travis mentioned FILE*, but that's also true for pointers returned by malloc, file descriptors, etc... See this for example: http://stackoverflow.com/questions/1052491/c-runtime-objects-dll-boundaries Because of 1, if we have a binary with intel + visual studio, we are effectively forcing everyone on windows to use intel fortran compilers. I would rather have the official binaries using open source toolchains. cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 8:50 AM, David Cournapeau courn...@gmail.com wrote: On Wed, Aug 22, 2012 at 4:12 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Aug 22, 2012 at 7:35 AM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 9:28 AM, David Cournapeau wrote: On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. Can you be more specific? Does the calling convention for C-routines created with Intel Fortran differ so much? I have the same question as Travis. If you are interested about ABI for Fortran, I have created this FAQ: http://fortran90.org/src/faq.html#are-fortran-compilers-abi-compatible Since NumPy only calls the Fortran routines, but does not expose them, then the only issue is how to build NumPy with (let's say) Intel Fortran. That's a separate issue. Once NumPy is built, then nobody cares, because they only need to interface the C routines, if anything at all. As far as Fortran runtime library goes (which of course is different for Intel and gfortran), I am currently not sure whether it is possible to mix them, but I think you probably can, if numpy .so is using Intel, and my own .so is using gfortran. Finally, if NumPy is build using MSVC, does this force everybody to use the same C compiler? I thought that C compilers are ABI compatible, at least Intel C and gfortran C are ABI compatible. Is MSVC different? Btw, to correctly call Fortran from C, one should always be using the iso_c_binding module, as explained here: http://fortran90.org/src/best-practices.html#interfacing-with-c Then the Fortran code becomes just like any other C library. It is unfortunately more complicated than that. 1 regarding fortran runtimes: I have never been able to link a gfortran object file with Visual Studio linker (link.exe). You cannot mix the Fortran object .o files between compilers. That will never work, because Fortran compilers are not ABI compatible, see the FAQ. The only way this could work is if you can mix gcc .o file with MSVC linker (I don't know if this is possible or not). If that works, then you should be able to use iso_c_binding in Fortran to produce gcc's compatible .o file and then link it. Which errors were you getting when linking it? Was it a problem with libgfortran runtime (that you of course need to link as well)? This runtime can be a problem. 2 mixing runtimes is never a good idea, because it becomes difficult to avoid passing a pointer from one runtime to the other. Intel fortran compiler obviously knows how to deal with the C runtime of Visual Studio, but gfortran doesn't. By Fortran runtime I mean this library in my Ubuntu: libgfortran.so.3 = /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x7ffaa9ce9000) Here are all the libraries in my typical gfortran program: linux-vdso.so.1 = (0x7e9ff000) libgfortran.so.3 = /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x7ffaa9ce9000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7ffaa99ef000) libgcc_s.so.1 = /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x7ffaa97d8000) libquadmath.so.0 = /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x7ffaa95a2000) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7ffaa91e5000) /lib64/ld-linux-x86-64.so.2 (0x7ffaaa022000) What exactly is the C runtime? libc? The only way to make Fortran talk to C, in a robust and supported way, is to use the
Re: [Numpy-discussion] 64bit infrastructure
On Wed, Aug 22, 2012 at 5:46 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Aug 22, 2012 at 8:50 AM, David Cournapeau courn...@gmail.com wrote: On Wed, Aug 22, 2012 at 4:12 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Wed, Aug 22, 2012 at 7:35 AM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 9:28 AM, David Cournapeau wrote: On Wed, Aug 22, 2012 at 3:25 PM, Travis Oliphant tra...@continuum.io wrote: On Aug 22, 2012, at 3:59 AM, Ralf Gommers wrote: On Tue, Aug 21, 2012 at 12:51 AM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. That would help, yes. MS Express doesn't work under Wine last time I checked by the way. However, the issue is more than just one license. There's a large number of packages that depend on numpy and provide binaries. If they can't make those compatible with numpy ones, that's a problem. Users will first install numpy 64-bit, and then later find out that part of the scientific Python stack isn't available to them anymore. As far as I understand, you don't *have* to build all downstream dependencies with the same compiler that NumPy was built with unless your extension relies on the way C-functions pass structures on the stack (not pointers to them, but structures as a whole) or if it relies on the representation of FILE*. At one time all structures were passed as pointers specifically for this reason. The FILE* situation is a problem, but most extensions don't use NumPy C-API calls that have a FILE* argument. It is much more pervasive than that, unfortunately. And for fortran, it is much worse, because if we build scipy or numpy with Intel Fortran, I think we pretty much force everyone to use intel fortran for *any* binary on top of them. Can you be more specific? Does the calling convention for C-routines created with Intel Fortran differ so much? I have the same question as Travis. If you are interested about ABI for Fortran, I have created this FAQ: http://fortran90.org/src/faq.html#are-fortran-compilers-abi-compatible Since NumPy only calls the Fortran routines, but does not expose them, then the only issue is how to build NumPy with (let's say) Intel Fortran. That's a separate issue. Once NumPy is built, then nobody cares, because they only need to interface the C routines, if anything at all. As far as Fortran runtime library goes (which of course is different for Intel and gfortran), I am currently not sure whether it is possible to mix them, but I think you probably can, if numpy .so is using Intel, and my own .so is using gfortran. Finally, if NumPy is build using MSVC, does this force everybody to use the same C compiler? I thought that C compilers are ABI compatible, at least Intel C and gfortran C are ABI compatible. Is MSVC different? Btw, to correctly call Fortran from C, one should always be using the iso_c_binding module, as explained here: http://fortran90.org/src/best-practices.html#interfacing-with-c Then the Fortran code becomes just like any other C library. It is unfortunately more complicated than that. 1 regarding fortran runtimes: I have never been able to link a gfortran object file with Visual Studio linker (link.exe). You cannot mix the Fortran object .o files between compilers. That will never work, because Fortran compilers are not ABI compatible, see the FAQ. I mean linking .o from the *same* compiler into a library through the MS linker (link.exe instead of gcc one). That's how it works in numpy/scipy so far, where we can link g77-produced .o files with the MS compiler. Most platforms have one linker, e.g. I think intel fortran compiler on linux uses ld underneath. The only way this could work is if you can mix gcc .o file with MSVC linker (I don't know if this is possible or not). If that works, then you should be able to use iso_c_binding in Fortran to produce gcc's compatible .o file and then link it. The problem is NOT communicating between C and fortran. I could produce a simple fortran dll used inside a MSVC program, as long as this fortran dll did not use anything from the fortran runtime. See http://cournape.wordpress.com/2009/03/09/gfortran-visual-studio/ for an actual example. The problem is when you need the fortran runtime (which you do for scipy). Which errors were you getting when linking it? Was it a problem with libgfortran runtime (that you of course need to link as well)? This runtime can be a problem. yes, the problem is gfortran runtime. libgfortran expects some mingw32 stuff, that cannot be linked into something through MS linker. IIRC, you could use enough violence to make it link, but you would get nasty runtime segfaults before anything gets run. 2 mixing runtimes is never a good idea, because it becomes
Re: [Numpy-discussion] 64bit infrastructure
On Sun, Aug 19, 2012 at 11:39 AM, Ralf Gommers The problem is that, unlike 32-bit builds, they can't be made with open source compilers on Windows. So unless we're okay with that, Why does it have to be built with open-source compilers? we're building against the python.org python, yes? Which is built with the MS compiler -- so why the insistance on open-source? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. But, I would rather have official builds of NumPy for all platforms with a compiler paid for by a NumPy-sponsor than not have them. -Travis On Aug 20, 2012, at 5:28 PM, Chris Barker wrote: On Sun, Aug 19, 2012 at 11:39 AM, Ralf Gommers The problem is that, unlike 32-bit builds, they can't be made with open source compilers on Windows. So unless we're okay with that, Why does it have to be built with open-source compilers? we're building against the python.org python, yes? Which is built with the MS compiler -- so why the insistance on open-source? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Mon, Aug 20, 2012 at 3:51 PM, Travis Oliphant tra...@continuum.io wrote: I'm actually not sure, why. I think the issue is making sure that the release manager can actually build NumPy without having to buy a particular compiler. The MS Express editions, while not open source, are free-to-use, and work fine. Not sure what what do about Fortran, though, but that's a scipy, not a numpy issue, yes? -Chris But, I would rather have official builds of NumPy for all platforms with a compiler paid for by a NumPy-sponsor than not have them. -Travis On Aug 20, 2012, at 5:28 PM, Chris Barker wrote: On Sun, Aug 19, 2012 at 11:39 AM, Ralf Gommers The problem is that, unlike 32-bit builds, they can't be made with open source compilers on Windows. So unless we're okay with that, Why does it have to be built with open-source compilers? we're building against the python.org python, yes? Which is built with the MS compiler -- so why the insistance on open-source? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
Message: 4 Date: Fri, 17 Aug 2012 15:25:21 -0700 From: Matthew Brett matthew.br...@gmail.com Subject: Re: [Numpy-discussion] 64bit infrastructure To: Discussion of Numerical Python numpy-discussion@scipy.org Message-ID: CAH6Pt5qa27Zb+UkOZB5EzsLK5UuoB-f8UcwFjdL2YcEBRZ=w...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Hi, On Fri, Aug 17, 2012 at 2:25 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi Sayth, On Thu, Aug 16, 2012 at 8:51 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi I was just wondering if the current absence of 64bit builds was as a result of an infrastructure or funding concern. I like many now have only 64 bit systems and am using the unofficial builds. If it was something we could raise funds to resolve, how much? Or is it more complex than that? Which platform, windows? Ondrej Yes I have a Windows 7 64Bit install and a ubuntu 64 Bit install. Isn't the numpy ubuntu install 64 bit by default? You may have seen that there are numpy and scipy 64-bit windows installers here: http://www.lfd.uci.edu/~gohlke/pythonlibs/ Do they work for you? Best, Matthew -- Thanks Matthew yeah that's the one I am currently using and it is working ok but just thought if there was a way to help 64bit builds become official then I would ask. Sayth ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
On Sun, Aug 19, 2012 at 9:14 AM, Sayth Renshaw flebber.c...@gmail.comwrote: Message: 4 Date: Fri, 17 Aug 2012 15:25:21 -0700 From: Matthew Brett matthew.br...@gmail.com Subject: Re: [Numpy-discussion] 64bit infrastructure To: Discussion of Numerical Python numpy-discussion@scipy.org Message-ID: CAH6Pt5qa27Zb+UkOZB5EzsLK5UuoB-f8UcwFjdL2YcEBRZ= w...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Hi, On Fri, Aug 17, 2012 at 2:25 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi Sayth, On Thu, Aug 16, 2012 at 8:51 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi I was just wondering if the current absence of 64bit builds was as a result of an infrastructure or funding concern. I like many now have only 64 bit systems and am using the unofficial builds. If it was something we could raise funds to resolve, how much? Or is it more complex than that? Which platform, windows? Ondrej Yes I have a Windows 7 64Bit install and a ubuntu 64 Bit install. Isn't the numpy ubuntu install 64 bit by default? You may have seen that there are numpy and scipy 64-bit windows installers here: http://www.lfd.uci.edu/~gohlke/pythonlibs/ Do they work for you? Best, Matthew -- Thanks Matthew yeah that's the one I am currently using and it is working ok but just thought if there was a way to help 64bit builds become official then I would ask. The problem is that, unlike 32-bit builds, they can't be made with open source compilers on Windows. So unless we're okay with that, we stay with the current situation. Fortunately Christoph Golhke's unofficial binaries are excellent. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
Hi Sayth, On Thu, Aug 16, 2012 at 8:51 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi I was just wondering if the current absence of 64bit builds was as a result of an infrastructure or funding concern. I like many now have only 64 bit systems and am using the unofficial builds. If it was something we could raise funds to resolve, how much? Or is it more complex than that? Which platform, windows? Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
Hi Sayth, On Thu, Aug 16, 2012 at 8:51 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi I was just wondering if the current absence of 64bit builds was as a result of an infrastructure or funding concern. I like many now have only 64 bit systems and am using the unofficial builds. If it was something we could raise funds to resolve, how much? Or is it more complex than that? Which platform, windows? Ondrej Yes I have a Windows 7 64Bit install and a ubuntu 64 Bit install. Sayth ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
Donations to NumFOCUS would be helpful in raising money to fund the creation of 64-bit installers.It always comes down to funding, but mostly people's time is the scarce resource here. It is more difficult to build 64-bit binaries on Windows which is the real issue (not the availability of 64-bit Windows machines -- although the right person having access to those machines can be a problem). NumFOCUS would gladly donate long-running EC2 instances running Windows 64-bit windows if there are people who just need access to a machine. -Travis On Aug 16, 2012, at 10:51 PM, Sayth Renshaw wrote: Hi I was just wondering if the current absence of 64bit builds was as a result of an infrastructure or funding concern. I like many now have only 64 bit systems and am using the unofficial builds. If it was something we could raise funds to resolve, how much? Or is it more complex than that? Sayth ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 64bit infrastructure
Hi, On Fri, Aug 17, 2012 at 2:25 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi Sayth, On Thu, Aug 16, 2012 at 8:51 PM, Sayth Renshaw flebber.c...@gmail.com wrote: Hi I was just wondering if the current absence of 64bit builds was as a result of an infrastructure or funding concern. I like many now have only 64 bit systems and am using the unofficial builds. If it was something we could raise funds to resolve, how much? Or is it more complex than that? Which platform, windows? Ondrej Yes I have a Windows 7 64Bit install and a ubuntu 64 Bit install. Isn't the numpy ubuntu install 64 bit by default? You may have seen that there are numpy and scipy 64-bit windows installers here: http://www.lfd.uci.edu/~gohlke/pythonlibs/ Do they work for you? Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
