matteosal opened a new issue #20675:
URL: https://github.com/apache/incubator-mxnet/issues/20675


   I am building master branch (v2.0) from commit 
fabcd145cd496628791f9f2ea813048360ac33ca and 1.x branch (v1.6) from commit 
6eec9da55c5096079355d1f1a5fa58dcf35d6752 and our internal benchmarks show a 
severe performance loss of v2.0 on Windows.
   
   Build settings for v2.0 are:
   ```
   cmake -G"Visual Studio 15 2017 Win64" -T host=x64 ^
    %= GENERAL FLAGS =% ^
    -DCMAKE_INSTALL_PREFIX=%output_dir% ^
    -DCMAKE_BUILD_TYPE=Release ^
    -DCMAKE_SKIP_BUILD_RPATH=On ^
    -DUSE_OPENCV=OFF ^
    -DUSE_JEMALLOC=OFF ^
    -DUSE_F16C=Off %= float16 support =%^
    -DUSE_INT64_TENSOR_SIZE=ON ^
    -DCMAKE_C_FLAGS="-D_WIN32" ^
    -DCMAKE_CXX_FLAGS="-D_WIN32" ^
    -DCMAKE_C_FLAGS_RELEASE="/MT -DNDEBUG" ^
    -DCMAKE_CXX_FLAGS_RELEASE="/MT -DNDEBUG" ^
    -DMXNET_FORCE_SHARED_CRT=OFF %= link statically to C runtime =%^
    -DUSE_MXNET_LIB_NAMING=OFF ^
    %= MATH BACKENDS =% ^
    -DBLAS=MKL ^
    -DUSE_LAPACK=OFF ^
    -DUSE_ONEDNN=OFF ^
    -DBLA_VENDOR="Intel10_64ilp" ^
    -DBLA_STATIC=OFF ^
    -DMKL_USE_SINGLE_DYNAMIC_LIBRARY=OFF ^
    -DMKL_INCLUDE_DIR=%mkl_dir% ^
    
-DBLAS_LIBRARIES="%mkl_dir%/libiomp5md.lib;%mkl_dir%/mkl_core_dll.lib;%mkl_dir%/mkl_intel_ilp64_dll.lib;%mkl_dir%/mkl_intel_thread_dll.lib"
 ^
    %= OPENMP =% ^
    -DUSE_OPENMP=ON ^
    -DOpenMP_C_FLAGS="-I%mkl_dir% /openmp:experimental" ^
    -DOpenMP_CXX_FLAGS="-I%mkl_dir% /openmp:experimental" ^
    -DOpenMP_C_LIB_NAMES="libiomp5" ^
    -DOpenMP_CXX_LIB_NAMES="libiomp5" ^
    -DOpenMP_libiomp5_LIBRARY="%mkl_dir%/libiomp5md.lib" ^
    %= CUDA =% ^
    -DUSE_CUDA=OFF ^
    %mxnet_dir%
   ```
   
   Build settings for v1.6 are:
   ```
   cmake -G"Visual Studio 15 2017 Win64" -T host=x64 ^
    %= GENERAL FLAGS =% ^
    -DCMAKE_INSTALL_PREFIX=%output_dir% ^
    -DCMAKE_BUILD_TYPE=Release ^
    -DCMAKE_SKIP_BUILD_RPATH=On ^
    -DUSE_OPENCV=OFF ^
    -DUSE_JEMALLOC=OFF ^
    -DUSE_F16C=Off %= float16 support =%^
    -DCMAKE_C_FLAGS="-D_WIN32" ^
    -DCMAKE_CXX_FLAGS="-D_WIN32" ^
    -DCMAKE_C_FLAGS_RELEASE="/MT -DNDEBUG" ^
    -DCMAKE_CXX_FLAGS_RELEASE="/MT -DNDEBUG" ^
    -DUSE_MXNET_LIB_NAMING=OFF ^
    %= MATH BACKENDS =% ^
    -DBLAS=MKL ^
    -DUSE_LAPACK=OFF ^
    -DUSE_MKLDNN=OFF ^
    -DUSE_MKL_IF_AVAILABLE=ON ^
    -DMKL_USE_ILP64=ON ^
    -DMKL_USE_SINGLE_DYNAMIC_LIBRARY=ON ^
    -DMKL_ROOT=%mkl_dir% ^
    -DMKL_INCLUDE_DIR=%mkl_dir% ^
    -DMKL_RT_LIBRARY="%mkl_dir%/libiomp5md.lib;%mkl_dir%/mkl_rt.lib" ^
    %= OPENMP =% ^
    -DUSE_OPENMP=ON ^
    -DOpenMP_C_FLAGS="-I%mkl_dir% /openmp" ^
    -DOpenMP_C_LIB_NAMES="libiomp5" ^
    -DOpenMP_CXX_FLAGS="-I%mkl_dir% /openmp" ^
    -DOpenMP_CXX_LIB_NAMES="libiomp5" ^
    -DOpenMP_libiomp5_LIBRARY="%mkl_dir%/libiomp5md.lib" ^
    %= CUDA =% ^
    -DUSE_CUDA=OFF ^
    %mxnet_dir%
   ```
   Some comments/observations:
   * Linux and Mac (built with analogous settings) do not suffer from this 
problem
   * MKL version is 2021.3 for all builds
   * Performance loss varies from about 1.4x slower to 10x slower on a set of 
models spanning from convnets to rnn to transformers, except for a small stack 
of 3x3 convolutions which is about 1.4x faster (but large convnets are 1.5x - 
2x slower)
   * Removing the flag `/openmp:experimental` passed to `OpenMP_C_FLAGS` and 
`OpenMP_CXX_FLAGS` in the build settings of mxnet 2.0 doesn't have any effect, 
but removing the analogous `/openmp` flag from v1.6 DOES worsen its performance 
(although it still doesn't get as bad as v2.0). I would consider this to be a 
bug in the CMake code because one shouldn't need extra stuff besides 
`USE_OPENMP=ON` to fully leverage openMP. Also, Linux and Mac need the 
analogous of those flags (`-fopenmp` on Linux, `-Xpreprocessor -fopenmp` on 
Mac) on both v1.6 and v2.0 to get optimal performance. This is not evidence 
that the v2.0 slowdown problem on Windows is due to openMP but I think it's 
worth mentioning.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org
For additional commands, e-mail: issues-h...@mxnet.apache.org

Reply via email to