Re: python-numpy (1.22.0-1) can't be imported
On Sat, Jan 22 2022, Masamichi Hosoda wrote: >> It is something like that, but "-Wl,--export-all-symbols" >> is not used on 1.21.4 and is not needed for most of the other >> modules on 1.22.x >> >> so I am looking for a less extreme action. >> Also to understand how it can impact other python subpackages > > If I understand correctly, I've found the root cause of the issue. > I've sent a pull request to numpy. > https://github.com/numpy/numpy/pull/20874 Based on that pull request, it looks like it's related to the patches used to avoid python C-extension modules with only one exported function mentioned a year ago: https://cygwin.com/pipermail/cygwin/2021-January/247211.html https://cygwin.com/pipermail/cygwin/2021-January/247216.html >From a glance at the Python C API docs, it looks like Python C extension modules are supposed to have only one exported function: https://docs.python.org/3/extending/extending.html#the-module-s-method-table-and-initialization-function https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module The NumPy CFFI tests seem to want to use a few C extension modules as normal shared libraries, which requires that the functions the tests want to use to be exported from the DLL, but this seems to be a somewhat unusual case. I tend to compile my own versions of Cython (without patches), which may have been picked up when I compiled the new version of NumPy. -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
> If I understand correctly, I've found the root cause of the issue. > I've sent a pull request to numpy. > https://github.com/numpy/numpy/pull/20874 A question is asked by NumPy if there is documentation on the Cygwin package's Cython behavior. https://github.com/numpy/numpy/pull/20874#issuecomment-1019442058 Does such a document exist? One of the causes of this issue is a patch in Cygwin's python-cython package like the one below. It removes the `__declspec(dllexport)` attribute from the symbols to be exported. On the other hand, if you install Cython by pip without using the Cygwin package, the exported symbols have the `__declspec(dllexport)` attribute because the patch has not been applied. What is the purpose of removing the `__declspec(dllexport)` attribute in this patch? Is this to make the modules using Cython without considering the Cygwin environment, the module does not have the `__declspec(dllexport)` attribute like Linux environment, works correctly? ``` --- origsrc/Cython-0.29.21/Cython/Utility/ModuleSetupCode.c 2020-07-08 23:44:39.0 +0200 +++ Cython/Utility/ModuleSetupCode.c2021-01-30 08:34:37.402649500 +0100 @@ -709,7 +709,11 @@ static CYTHON_INLINE void * PyThread_tss /// PyModInitFuncType.proto /// #ifndef CYTHON_NO_PYINIT_EXPORT -#define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC +#ifdef __cplusplus +#define __Pyx_PyMODINIT_FUNC extern "C" PyObject * +#else +#define __Pyx_PyMODINIT_FUNC PyObject * +#endif #elif PY_MAJOR_VERSION < 3 // Py2: define this to void manually because PyMODINIT_FUNC adds __declspec(dllexport) to it's definition. ``` -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
> It is something like that, but "-Wl,--export-all-symbols" > is not used on 1.21.4 and is not needed for most of the other > modules on 1.22.x > > so I am looking for a less extreme action. > Also to understand how it can impact other python subpackages If I understand correctly, I've found the root cause of the issue. I've sent a pull request to numpy. https://github.com/numpy/numpy/pull/20874 -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
On 22.01.2022 02:04, Masamichi Hosoda wrote: I have not found the root cause yet. As the 1.21.4-1 imports correctly I removed the 1.22.0-1 until I solve the issue. I do not see anything obvious in upstream source between 1.21.4 and 1.22.0 that gives me any hint on root cause. Also 1.22.1 shows the same problem. I excluded the build chain as rebuilding 1.19.4 worked fine for all 3.6 to 3.9 If I understand correctly, the patch below is just a quick hack, but it solves the problem. ``` --- a/numpy/random/setup.py +++ b/numpy/random/setup.py @@ -147,7 +147,8 @@ include_dirs=['.', 'src', 'src/legacy'], libraries=mtrand_libs, extra_compile_args=EXTRA_COMPILE_ARGS, - extra_link_args=EXTRA_LINK_ARGS, + extra_link_args=(EXTRA_LINK_ARGS + + ['-Wl,--export-all-symbols']), depends=depends + ['mtrand.pyx'], define_macros=defs + LEGACY_DEFS, ) ``` Thanks It is something like that, but "-Wl,--export-all-symbols" is not used on 1.21.4 and is not needed for most of the other modules on 1.22.x so I am looking for a less extreme action. Also to understand how it can impact other python subpackages Regards Marco -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
> I have not found the root cause yet. > As the 1.21.4-1 imports correctly I removed the 1.22.0-1 until I solve > the issue. > > I do not see anything obvious in upstream source between 1.21.4 and > 1.22.0 that gives me any hint on root cause. > > Also 1.22.1 shows the same problem. > I excluded the build chain as rebuilding 1.19.4 worked fine > for all 3.6 to 3.9 If I understand correctly, the patch below is just a quick hack, but it solves the problem. ``` --- a/numpy/random/setup.py +++ b/numpy/random/setup.py @@ -147,7 +147,8 @@ include_dirs=['.', 'src', 'src/legacy'], libraries=mtrand_libs, extra_compile_args=EXTRA_COMPILE_ARGS, - extra_link_args=EXTRA_LINK_ARGS, + extra_link_args=(EXTRA_LINK_ARGS + + ['-Wl,--export-all-symbols']), depends=depends + ['mtrand.pyx'], define_macros=defs + LEGACY_DEFS, ) ``` -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
On 14.01.2022 03:04, airplanemath via Cygwin wrote: On Wed, Jan 12 2022, Marco Atzeri wrote: On 12.01.2022 12:47, ggl329 wrote: Hi Marco, I upgraded python39-numpy to 1.22.0-1, and failed to import numpy. It seems that mtrand.cpython-39-x86_64-cygwin.dll does not have PyInit_mtrand. Could you check if numpy can be imported in your environment? working on it. In theory I have not changed the build between the two versions but there was a patch in 1.19.4-1 for similar issue. Regards Marco I have the same problems with the distribution NumPy package, but I can use a locally-compiled NumPy with no patches. It doesn't look like your build has any patches either: https://cygwin.com/cgi-bin2/package-cat.cgi?file=x86_64%2Fpython-numpy-src%2Fpython-numpy-1.22.0-1-src=numpy so that's fun. Out of curiousity, what options are you passing for cpu-baseline and cpu-dispatch? https://numpy.org/devdocs/reference/simd/build-options.html I don't see the newest cygport in the Cygwin package repository. I have not found the root cause yet. As the 1.21.4-1 imports correctly I removed the 1.22.0-1 until I solve the issue. I do not see anything obvious in upstream source between 1.21.4 and 1.22.0 that gives me any hint on root cause. Also 1.22.1 shows the same problem. I excluded the build chain as rebuilding 1.19.4 worked fine for all 3.6 to 3.9 Regards Marco -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
On Wed, Jan 12 2022, Marco Atzeri wrote: > On 12.01.2022 12:47, ggl329 wrote: >> Hi Marco, >> I upgraded python39-numpy to 1.22.0-1, and failed to import numpy. >> It seems that mtrand.cpython-39-x86_64-cygwin.dll does not have >> PyInit_mtrand. >> Could you check if numpy can be imported in your environment? >> > > working on it. > In theory I have not changed the build between the two versions > but there was a patch in 1.19.4-1 for similar issue. > > Regards > Marco I have the same problems with the distribution NumPy package, but I can use a locally-compiled NumPy with no patches. It doesn't look like your build has any patches either: https://cygwin.com/cgi-bin2/package-cat.cgi?file=x86_64%2Fpython-numpy-src%2Fpython-numpy-1.22.0-1-src=numpy so that's fun. Out of curiousity, what options are you passing for cpu-baseline and cpu-dispatch? https://numpy.org/devdocs/reference/simd/build-options.html I don't see the newest cygport in the Cygwin package repository. -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: python-numpy (1.22.0-1) can't be imported
On 12.01.2022 12:47, ggl329 wrote: Hi Marco, I upgraded python39-numpy to 1.22.0-1, and failed to import numpy. It seems that mtrand.cpython-39-x86_64-cygwin.dll does not have PyInit_mtrand. Could you check if numpy can be imported in your environment? working on it. In theory I have not changed the build between the two versions but there was a patch in 1.19.4-1 for similar issue. Regards Marco -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
