Hi all,
I've been trying to compile NumPy from source on Windows 10, with MSVC compiler
and Intel MKL. Whenever I link to MKL it fails at loading DLLs.
I am running Windows 10.0.18363 with Microsoft Visual Studio 2019 (16.8.5) and
Intel MKL 2017.8.275.
I managed to reproduce the issue with a minimal setup, using latest Python and
NumPy.
1. Download latest Python (3.9.1) and latest NumPy (1.20.1) source.
2. Open a VS command prompt, unpack Python source, build with
PCbuild\build.bat
3. Run mklvars.bat intel64 to get the right environment variables set.
4. Add the Intel compilers (needed for ifort) to PATH:
set PATH=C:\Program Files
(x86)\IntelSWTools\compilers_and_libraries_2017\windows\bin\intel64;%PATH%
1. Create a virtual env, copy a few files from the Python build and activate
the virtual env:
copy Python\PCbuild\amd64\python39.dll venv\Scripts
copy Python\PC\pyconfig.h venv\Include
1. Build NumPy from source and install: pip install . -v
2. Try to import NumPy: python -c "import numpy"
The error message appears as follows:
Traceback (most recent call last):
File "C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py",
line 22, in <module>
from . import multiarray
File
"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py", line
12, in <module>
from . import overrides
File
"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py", line
7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified
module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\path\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py", line
145, in <module>
from . import core
File "C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py",
line 48, in <module>
raise ImportError(msg)
ImportError:
[... useful suggestions that however did not lead to a solution...]
Original error was: DLL load failed while importing _multiarray_umath: The
specified module could not be found.
The MKL libraries are picked up during compilation since it returns:
FOUND:
libraries = ['mkl_rt']
library_dirs = ['C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib\\intel64']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl', 'C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\include',
'C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib']
I tried to analyze the DLL resolution on _multiarray_umath.pyd with
Dependencies (the newer version of Dependency Walker) but it seems that the MKL
DLL loads fine. There are some DLLs that appear as not correctly loaded, but as
far as I understand it is caused by the inspection software limit with Windows
API sets (api-ms-win-core-*, ext-ms-onecore-*, ext-ms-win-*, and similar), not
by actual problems with this DLLs, so I think the system is correctly setup.
If I skip the initialization of MKL environment variables, then the MKL
libraries are not picked and NumPy is compiled to a functional state.
In the past this setup used to work with Python 3.6, VS2015 and a similar
version of Intel MKL.
I was able to reproduce the issue with NumPy 1.16.2, 1.17 and 1.20.1; with
Python 3.8.6 and Python 3.9.1; with Intel MKL 2017 and oneAPI 2020.
Am I missing any obvious step to succeed in this adventure?
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
