Re: Compiling and Linking pre-built Windows Python libraries with C++ files on Linux for Windows

[Christian Gollwitzer]

Am 19.03.22 um 01:08 schrieb Ankit Agarwal:

This is a very specific question. I am trying to figure out whether or not
I can use pre-built python libraries and headers on Windows in a MinGW
build on Linux.


With the mingw cross-compiler on Linux that should be possible, however 
I guess it might be difficult to set it up. Setuptools is not good for 
crosscompiling. The easiest way to build Python extensions across 
multiple OSes is cibuildwheel: https://github.com/pypa/cibuildwheel


Especially if your code lives on Github, then you can simply add 
cibuildwheel to your workflow and it will build for multiple OSes and 
Python versions on every commit. For OpenSource projects, Githubs build 
servers are  even for free! Otherwise, the prices are moderate as well.


If you happen to have your own build server farm (as in a commercial 
setting) then you can also use other CI tools with cibuildwheel.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling and Linking pre-built Windows Python libraries with C++ files on Linux for Windows

[Eryk Sun]

On 3/18/22, Ankit Agarwal  wrote:
> Hi,
>
> This is a very specific question. I am trying to figure out whether or not
> I can use pre-built python libraries and headers on Windows in a MinGW
> build on Linux. Essentially I have some python and C++ code which interface
> via cython and pybind. I want to build a self contained C++ binary for
> Windows with the MinGW compiler that runs on Linux. For both Cython and
> PyBind, they need to compile with the python headers and link against the
> python DLL in order to run on Windows.
>
> I know that the python DLL specifically are compiled with the MSVC
> compiler, however since it is in C, the ABI between the DLL should be
> compatible with MinGW, and I should be able to import and link against it.
> My question is will this work, or will there be some other problem that I
> might run into.

MinGW used to link with msvcrt (the private CRT for system components)
instead of ucrt (the universal CRT). If it still does that, then you
won't be able to share some of the POSIX compatibility features
between the two CRTs, such as file descriptors and the locale. Their
FILE stream records are also incompatible. Also, you'll have to be
certain to never free() memory that was allocated by a different CRT.
msvcrt uses a private heap, and ucrt uses the main process heap. For
example (with a debugger attached):

import ctypes
ucrt = ctypes.CDLL('ucrtbase', use_errno=True)
msvcrt = ctypes.CDLL('msvcrt', use_errno=True)
ucrt.malloc.restype = ctypes.c_void_p
msvcrt.free.argtypes = (ctypes.c_void_p,)

>>> b = ucrt.malloc(4096)
>>> msvcrt.free(b)

HEAP[python.exe]: Invalid address specified to
RtlFreeHeap( 024389C1, 024389A58FB0 )
(1b44.1ca0): Break instruction exception - code 8003 (first chance)
ntdll!RtlpBreakPointHeap+0x16:
7ff8`8baa511e cc  int 3
0:000>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling and Linking pre-built Windows Python libraries with C++ files on Linux for Windows

[Dan Stromberg]

On Fri, Mar 18, 2022 at 8:03 PM Ankit Agarwal  wrote:

> Hi,
>
> This is a very specific question. I am trying to figure out whether or not
> I can use pre-built python libraries and headers on Windows in a MinGW
> build on Linux. Essentially I have some python and C++ code which interface
> via cython and pybind. I want to build a self contained C++ binary for
> Windows with the MinGW compiler that runs on Linux. For both Cython and
> PyBind, they need to compile with the python headers and link against the
> python DLL in order to run on Windows.
>
> I know that the python DLL specifically are compiled with the MSVC
> compiler, however since it is in C, the ABI between the DLL should be
> compatible with MinGW, and I should be able to import and link against it.
> My question is will this work, or will there be some other problem that I
> might run into.
>
>
I haven't tried this.

However, I used to cross-compile the Linux kernel from Solaris on Sparc to
Intel.  I just had to:
1) Get the relevant headers and libraries on Solaris
2) Deal with the byte sex issues - Sparc is Big Endian, Intel is Little
Endian
3) Natively compile a little bit of code that was needed by the build
process

You appear to be aware of #1.

You probably won't need to worry about #2, since you're going Intel ->
Intel.

#3 could be an issue for you, but it's just a matter of using two different
compilers for some different parts of the build process - one native, one
cross.

I'd try a little hello world first, then worry about your larger project.

You could also put some feelers out about Cython and Pybind too, to see if
they've been used for cross-compilation before.  If yes, you're probably in
like Flynn, otherwise it could potentially turn out to be a big project.

If cross-compilation doesn't work out, you could probably set up a Windows
virtual machine with an sshd, and build on that.

Either way, you may find Wine useful for testing.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Compiling and Linking pre-built Windows Python libraries with C++ files on Linux for Windows

[Ankit Agarwal]

Hi,

This is a very specific question. I am trying to figure out whether or not
I can use pre-built python libraries and headers on Windows in a MinGW
build on Linux. Essentially I have some python and C++ code which interface
via cython and pybind. I want to build a self contained C++ binary for
Windows with the MinGW compiler that runs on Linux. For both Cython and
PyBind, they need to compile with the python headers and link against the
python DLL in order to run on Windows.

I know that the python DLL specifically are compiled with the MSVC
compiler, however since it is in C, the ABI between the DLL should be
compatible with MinGW, and I should be able to import and link against it.
My question is will this work, or will there be some other problem that I
might run into.

I am using python 3.7 by the way for this.

Let me know what you think,

Thanks :)

Ankit

*Ankit Agarwal*

Software Engineer

an...@applied.co

+1 630.506.4914

Applied Intuition, Inc. 
blog.applied.co
-- 
https://mail.python.org/mailman/listinfo/python-list