[issue3871] cross and native build of python for mingw* hosts
Jason Huntley added the comment: Hey Guys, I've furthered the progress of the patch I've been working on. THis patch builds Python3.3 in mingw64 & msys using gcc 4.7.2 compiler. This patch can also enable and build the following modules: ctypes bz2 multiprocessing sqlite3 ssl pyexpat zlib Of course, there are many others, but these were some of the problem modules I encounted previously. Partial success running the python tests. Many of the basic tests pass, I still have many to resolve. I have one thread test failing I need to resolve as it's preventing from all the tests running. Regardless, seems like some progress. >>I think it would be great if everyone could collaborate better with this >>stuff; I agree; I would really like for us to make this a joint effort. However, I'm noticing a ton of descrepancies with these patches. Some are removing pthreads, others are adding them back. Some patches are ignoring the PC/pyconfig while others are trying to generate a custom mingw version. It seems some are also compiling and using the mingw version outside of the msys shell, which I'm not certain of the advantages there when u can just use the mscv version. I think we need to take a minute to agree on some specific goals and criteria this mingw port should be used for. Just to kick things off, here are some of the requirements which I think we should aim for. * The build must be reproducible in the mingw msys shell, in the default environment described on the mingw64 site, also included some other good doc references: * https://sourceforge.net/apps/trac/mingw-w64/wiki/MSYS * https://sourceforge.net/apps/trac/mingw-w64/wiki/Cross%20Win32%20and%20Win64%20compiler * http://www.gaia-gis.it/gaia-sins/mingw64_how_to.html#env * These documented steps have also been automated by the sripts here https://github.com/onepremise/MinGW-AD64S * Currently, I have the mingw build using the PC platform specified config headers and source. * too many issues reside around enabling posix threads as the pthread_t identifier in mingw doesn't match existing py thread framework returning longs. * As a result, the patch I provided builds with the ntthread implementation. * In addition, PC/pyconfig.h and PC/getpath.c are being used with slight modifications for __MINGW32__ * Initial python mingw64 deployment layout, version identifiers should follow existing scheme, "3.3" not 33: * should deploy libraries to /mingw/lib, libpython3.3.a, Format: libpython3.3.dll, libpython3.3.dll.a. * executable binaries should exist in /mingw/bin. Format: python.exe, python3.3.exe. * The python script library deploys to /mingw/lib/Python3.3. * /mingw/lib/Python3.3/lib-dynload * /mingw/lib/Python3.3/site-packates * Headers located /mingw/include/Python3.3 * Having Python build isn't satisfactory enough to promote a patch, patches should not cause new test failures or regressions. Preferably they should resolve core compilation issues and module compilation issues, while not creating new test failures and preferrably address/fix failing tests. (python -m test) * covered on the dev-guilde, http://docs.python.org/devguide/ Please feel free to contriube, update, remove, discuss etc... >>https://github.com/mingwandroid/crucifixion-freedom/tree/master/patches/python/3.3.0 >> [Python cross and MinGW compilation] >>https://github.com/niXman/mingw-builds/tree/master/patches/Python-3.3.0 >>[MinGW toolchains including Python pretty-printing GDB] I tried some of these patches, but they aren't very organinzed. I really need some docemntaiton to better understand what each patch does and which pairs go together and which don't. Wiki over there is blank. >>in fact, just after Roumen's huge patch we apply some patches to remove large >>parts >>of it so we can re-introduce those independent fixes via separate patches). I had to do the same as well. There were regressions in some of these patches causing the python tests to fail and prevent the python setup.py script from running. >>Our patching strategies are more sane (many patches as opposed to a >>monolithic patch - >>Given how intrusive these patches are, getting them merged into CPython is a >>bit of a >>pipe dream (I'd love to be wrong about that), so making it as easy as >>possible >>for the interested parties to maintain (forward port), build, test and >>package >>Python releases is probably the best we can hope for. If we can make the >>patchessmall and independently reviewable then maybe some parts could be >>merged. I prefer this as well. It will make code reviewing easier. Otherwise, some kind of staging location would be useful. I'm fine using rietveld, the tracker. We just need to name the pa
[issue3871] cross and native build of python for mingw* hosts
Changes by Jason Huntley : Removed file: http://bugs.python.org/file28699/py3k-20121227-MINGW.patch ___ Python tracker <http://bugs.python.org/issue3871> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3871] cross and native build of python for mingw* hosts
Jason Huntley added the comment: Hey Guys, I've furthered the progress of the patch I've been working on. THis patch builds Python3.3 in mingw64 & msys using gcc 4.7.2 compiler. This patch can also enable and build the following modules: ctypes bz2 multiprocessing sqlite3 ssl pyexpat zlib Of course, there are many others, but these were some of the problem modules I encounted previously. Partial success running the python tests. Many of the basic tests pass, I still have many to resolve. I have one thread test failing I need to resolve as it's preventing from all the tests running. Regardless, seems like some progress. >>I think it would be great if everyone could collaborate better with this >>stuff; I agree; I would really like for us to make this a joint effort. However, I'm noticing a ton of descrepancies with these patches. Some are removing pthreads, others are adding them back. Some patches are ignoring the PC/pyconfig while others are trying to generate a custom mingw version. It seems some are also compiling and using the mingw version outside of the msys shell, which I'm not certain of the advantages there when u can just use the mscv version. I think we need to take a minute to agree on some specific goals and criteria this mingw port should be used for. Just to kick things off, here are some of the requirements which I think we should aim for. * The build must be reproducible in the mingw msys shell, in the default environment described on the mingw64 site, also included some other good doc references: * https://sourceforge.net/apps/trac/mingw-w64/wiki/MSYS * https://sourceforge.net/apps/trac/mingw-w64/wiki/Cross%20Win32%20and%20Win64%20compiler * http://www.gaia-gis.it/gaia-sins/mingw64_how_to.html#env * These documented steps have also been automated by the sripts here https://github.com/onepremise/MinGW-AD64S * Currently, I have the mingw build using the PC platform specified config headers and source. * too many issues reside around enabling posix threads as the pthread_t identifier in mingw doesn't match existing py thread framework returning longs. * As a result, the patch I provided builds with the ntthread implementation. * In addition, PC/pyconfig.h and PC/getpath.c are being used with slight modifications for __MINGW32__ * Initial python mingw64 deployment layout, version identifiers should follow existing scheme, "3.3" not 33: * should deploy libraries to /mingw/lib, libpython3.3.a, Format: libpython3.3.dll, libpython3.3.dll.a. * executable binaries should exist in /mingw/bin. Format: python.exe, python3.3.exe. * The python script library deploys to /mingw/lib/Python3.3. * /mingw/lib/Python3.3/lib-dynload * /mingw/lib/Python3.3/site-packates * Headers located /mingw/include/Python3.3 * Having Python build isn't satisfactory enough to promote a patch, patches should not cause new test failures or regressions. Preferably they should resolve core compilation issues and module compilation issues, while not creating new test failures and preferrably address/fix failing tests. (python -m test) * covered on the dev-guilde, http://docs.python.org/devguide/ Please feel free to contriube, update, remove, discuss etc... >>https://github.com/mingwandroid/crucifixion-freedom/tree/master/patches/python/3.3.0 >> [Python cross and MinGW compilation] >>https://github.com/niXman/mingw-builds/tree/master/patches/Python-3.3.0 >>[MinGW toolchains including Python pretty-printing GDB] I tried some of these patches, but they aren't very organinzed. I really need some docemntaiton to better understand what each patch does and which pairs go together and which don't. Wiki over there is blank. >>in fact, just after Roumen's huge patch we apply some patches to remove large >>parts >>of it so we can re-introduce those independent fixes via separate patches). I had to do the same as well. There were regressions in some of these patches causing the python tests to fail and prevent the python setup.py script from running. >>Our patching strategies are more sane (many patches as opposed to a >>monolithic patch - >>Given how intrusive these patches are, getting them merged into CPython is a >>bit of a >>pipe dream (I'd love to be wrong about that), so making it as easy as >>possible >>for the interested parties to maintain (forward port), build, test and >>package >>Python releases is probably the best we can hope for. If we can make the >>patchessmall and independently reviewable then maybe some parts could be >>merged. I prefer this as well. It will make code reviewing easier. Otherwise, some kind of staging location would be useful. I'm fine using rietveld, the tracker. We just need to name the pa
[issue3871] cross and native build of python for mingw* hosts
Jason Huntley added the comment: Finally! I was able to get all building on mingw64, gcc 4.7.2, and python 3.3. I had to take everybody's work here a little further, and I have produced a new patch which will build python33 using the latest minw64 compiler in an msys environment. I cloned the 3.3 branch and applied the latest patch available here, py3k-20121004-MINGW.patch. I had to make quite a few more updates to get all the modules compiling with 4.7.2 and python find all the modules in the correct paths in an msys/mingw64 environment. I've compiled another patch, which addresses all the latest issues. For all interested, run my setup script from the following location: https://github.com/onepremise/MinGW-AD64S/ Just open dos in admin mode and run setup.bat. This will build all the required dependencies and finally python 3.3. -- Added file: http://bugs.python.org/file28699/py3k-20121227-MINGW.patch ___ Python tracker <http://bugs.python.org/issue3871> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3871] cross and native build of python for mingw* hosts
Jason Huntley added the comment: Finally! I was able to get all building on mingw64, gcc 4.7.2, and python 3.3. I had to take everybody's work here a little further, and I have produced a new patch which will build python33 using the latest minw64 compiler in an msys environment. I cloned the 3.3 branch and applied the latest patch available here, py3k-20121004-MINGW.patch. I had to make quite a few more updates to get all the modules compiling with 4.7.2 and python find all the modules in the correct paths in an msys/mingw64 environment. I've compiled another patch, which addresses all the latest issues. For all interested, run my setup script from the following location: https://github.com/onepremise/MinGW-AD64S/ Just open dos in admin mode and run setup.bat. This will build all the required dependencies and finally python 3.3. -- Added file: http://bugs.python.org/file28471/py3k-20121227-MINGW.patch ___ Python tracker <http://bugs.python.org/issue3871> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3871] cross and native build of python for mingw* hosts
Jason Huntley added the comment: I'm attempting to build Python-3.3.0 with mingw64. I get a minute or two into the build and fail with this error: ./Include/pythonrun.h:178:1: warning: function declaration isn't a prototype [-W strict-prototypes] ./Modules/getpath.c: In function 'isfile': ./Modules/getpath.c:153:5: warning: implicit declaration of function '_Py_wstat' [-Wimplicit-function-declaration] ./Modules/getpath.c: In function 'find_env_config_value': ./Modules/getpath.c:298:17: error: too many arguments to function 'wcstok' In file included from c:\projects\tools\osm\dev\mapnik\custom\environment\instal l\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32 /include/io.h:10:0, from ./PC/pyconfig.h:68, from ./Include/Python.h:8, from ./Modules/getpath.c:3: It seems some people here were using this patch against 3.3.0 at one point. Did you guys ever succeed? Once I've applied the patch, i can get a little further, but the build eventually fails again. Also, conftest continually fails to run during configure. I'm using the following settings to get the build going: export "CFLAGS=-I/mingw/include" export "LDFLAGS=-L/mingw/lib" export "CPPFLAGS=-I/mingw/include" export "CFLAGS=$CFLAGS -I./PC -DMS_WIN64 -D__MINGW32__" #http://bugs.python.org/issue3871 #wget http://bugs.python.org/file27474/py3k-20121004-MINGW.patch wget http://bugs.python.org/file26572/python-py3k-20120729-MINGW.patch #patch -t -p1 < "py3k-20121004-MINGW.patch" patch -t -p1 < python-py3k-20120729-MINGW.patch ./configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --prefix=/mingw make If I patch before configure, I get the following exception when running ./configure: bad word @INITSYS@ in @INITSYS@ posixmodule.c mv: cannot stat `config.c': No such file or directory After configure, is where it gets further, until it finally fails on: c:\projects\tools\osm\dev\mapnik\custom\environment\install\mingw64\bin\../lib/g cc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/io.h:311:15: note: previous declaration of 'chmod' was here ./Modules/posixmodule.c:262:32: error: unknown type name 'uid_t' ./Modules/posixmodule.c:262:39: error: unknown type name 'gid_t' You can replicate my environment exactly by running the following deployment scripts from my project here: https://github.com/onepremise/MinGW-AD64S/ Just open dos in admin mode and run setup.bat. Any help is greatly appreciated. Thanks! -- nosy: +jhuntley ___ Python tracker <http://bugs.python.org/issue3871> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
