Hello,
On 2/15/2021 7:38 AM, Pietro Fontana wrote:
Hi all,
I've been trying to compile NumPy from source on Windows 10, with MSVC
compiler and Intel MKL. Whenever I link to MKL it fails at loading DLLs.
I am running Windows 10.0.18363 with Microsoft Visual Studio 2019
(16.8.5) and Intel MKL 2017.8.275.
I managed to reproduce the issue with a minimal setup, using latest
Python and NumPy.
1. Download latest Python (3.9.1) and latest NumPy (1.20.1) source.
2. Open a VS command prompt, unpack Python source, build with
PCbuild\build.bat
3. Run mklvars.bat intel64 to get the right environment variables set.
4. Add the Intel compilers (needed for ifort) to PATH:
set PATH=C:\Program Files
(x86)\IntelSWTools\compilers_and_libraries_2017\windows\bin\intel64;%PATH%
5. Create a virtual env, copy a few files from the Python build and
activate the virtual env:
copy Python\PCbuild\amd64\python39.dll venv\Scripts
copy Python\PC\pyconfig.h venv\Include
6. Build NumPy from source and install: pip install . -v
7. Try to import NumPy: python -c "import numpy"
The error message appears as follows:
|Traceback (most recent |call||last|):|
||File||"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py"|, line
|22|, |in|<|module|>|
||from|. |import|multiarray|
||File||"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py"|,
line |12|, |in|<|module|>|
||from|. |import|overrides|
||File||"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py"|,
line |7|, |in|<|module|>|
||from|numpy.core._multiarray_umath |import|(|
|ImportError: DLL |load||failed||while|importing _multiarray_umath: The
specified |module|could |not|be found.|
||
|During handling |of|the above |exception|, another |exception|occurred:|
||
|Traceback (most recent |call||last|):|
||File||"<string>"|, line |1|, |in|<|module|>|
||File||"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py"|, line
|145|, |in|<|module|>|
||from|. |import|core|
||File||"C:\path\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py"|, line
|48|, |in|<|module|>|
||raise|ImportError(msg)|
|ImportError:|
|[… useful suggestions that however did not lead to a solution…]|
|Original |error|was: DLL |load||failed||while|importing _multiarray_umath: The
specified |module|could |not|be found.|
The MKL libraries are picked up during compilation since it returns:
|FOUND:|
| libraries = [|'mkl_rt'|]|
| library_dirs = [|'C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib\\intel64'|]|
| define_macros = [(|'SCIPY_MKL_H'|, |None|), (|'HAVE_CBLAS'|, |None|)]|
| include_dirs = [|'C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl'|, |'C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\include'|, |'C:\\Program Files
(x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib'|]|
I tried to analyze the DLL resolution on |_multiarray_umath.pyd| with
Dependencies (the newer version of Dependency Walker) but it seems that
the MKL DLL loads fine. There are some DLLs that appear as not correctly
loaded, but as far as I understand it is caused by the inspection
software limit with Windows API sets (api-ms-win-core-*,
ext-ms-onecore-*, ext-ms-win-*, and similar), not by actual problems
with this DLLs, so I think the system is correctly setup.
If I skip the initialization of MKL environment variables, then the MKL
libraries are not picked and NumPy is compiled to a functional state.
In the past this setup used to work with Python 3.6, VS2015 and a
similar version of Intel MKL.
I was able to reproduce the issue with NumPy 1.16.2, 1.17 and 1.20.1;
with Python 3.8.6 and Python 3.9.1; with Intel MKL 2017 and oneAPI 2020.
Am I missing any obvious step to succeed in this adventure?
Python >= 3.8 will no longer use PATH for resolving dependencies of
extension modules. Use os.add_dll_directory(mkl_bin_path)
<https://docs.python.org/3/library/os.html#os.add_dll_directory> in all
your scripts before importing numpy or add the call to a
_distributor_init.py file in the numpy package directory.
Christoph
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
