hello,
this email is related to the following PR and JIRA ticket:
- [MXNET-1229] use OpenBLAS, lapack & OpenCV from conan
<https://github.com/apache/incubator-mxnet/pull/13400>
- use conan to manage project dependencies
<https://issues.apache.org/jira/browse/MXNET-1229>
conan <https://conan.io> is an open-source package manager for C++
projects. it allows to manage project dependencies in transparent and
declarative manner.
currently, apache incubator-mxnet project uses the following different ways
to manage its dependencies:
- download GitHub archives during the build
- OpenBLAS <https://github.com/xianyi/OpenBLAS>
- OpenCV <https://github.com/opencv/opencv>
- conda <https://conda.io/> (alternative way to GitHub archives)
- download from CMake
- Intel Math Kernel Library <https://software.intel.com/en-us/mkl> (MKL)
- Git submodules
- cub <https://github.com/dmlc/cub>
- dlpack <https://github.com/dmlc/dlpack>
- dmlc-core <https://github.com/dmlc/dmlc-core>
- googletest <https://github.com/abseil/googletest>
- mkldnn <https://github.com/intel/mkl-dnn>
- mshadow <https://github.com/dmlc/mshadow>
- onnx-tensorrt <https://github.com/onnx/onnx-tensorrt>
- openmp <https://github.com/llvm-mirror/openmp>
- ps-lite <https://github.com/dmlc/ps-lite>
- tvm <https://github.com/dmlc/tvm>
this appears to be very heterogeneous and hard to manage/maintain, as
multiple various commands are in use to achieve dependencies installation,
as well as multiple places are to look for dependency versions and their
updates.
with conan, it may became much more straightforward, as dependencies will
be declared in single place (conanfile) and installed via single command
(conan install).
as project is very complex, and has lots of dependencies, for the first
prototype I've used only very few of dependencies from conan: OpenCV
<https://bintray.com/conan-community/conan/opencv%3Aconan>, OpenBLAS
<https://bintray.com/conan-community/conan/openblas%3Aconan> and lapack
<https://bintray.com/conan-community/conan/lapack%3Aconan>.
others may be easily added then one by one, but they first has to be
packaged (not all of them are packaged yet, e.g. GoogleTest
<https://bintray.com/bincrafters/public-conan/gtest%3Abincrafters> is
available, while MKL is not).
I attach patch which adds an initial conan support as proof of concept.
also, I attach two simple build scripts, which I've used to test (for
Windows and Linux / Mac OS X). Google Mail blocks .sh and .cmd extensions,
so you'll need to rename files.
lemme know if you have any further questions.
yours sincerely, Konstantin
From 0449ef0bc521a3fba09f9042c1b60fd171a73d60 Mon Sep 17 00:00:00 2001
From: SSE4 <tomsks...@gmail.com>
Date: Sun, 25 Nov 2018 03:04:36 +0700
Subject: [PATCH 1/2] - use OpenBLAS, lapack & OpenCV from conan
Signed-off-by: SSE4 <tomsks...@gmail.com>
---
CMakeLists.txt | 6 ++++++
cmake/Modules/FindOpenBLAS.cmake | 7 +++++++
conanfile.py | 11 +++++++++++
3 files changed, 24 insertions(+)
create mode 100644 conanfile.py
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3b8bbd2e027..15ab2ceb2e9 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -2,6 +2,12 @@ cmake_minimum_required(VERSION 3.0.2)
project(mxnet C CXX)
+if(EXISTS ${CMAKE_CURRENT_BINARY_DIR}/conanbuildinfo.cmake)
+ include(${CMAKE_CURRENT_BINARY_DIR}/conanbuildinfo.cmake)
+ conan_basic_setup(TARGETS)
+ message(STATUS "using conan")
+endif()
+
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/build/private/local_config.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/build/private/local_config.cmake)
endif()
diff --git a/cmake/Modules/FindOpenBLAS.cmake b/cmake/Modules/FindOpenBLAS.cmake
index a3a79caae46..cdbb4d38e32 100644
--- a/cmake/Modules/FindOpenBLAS.cmake
+++ b/cmake/Modules/FindOpenBLAS.cmake
@@ -15,6 +15,13 @@
# specific language governing permissions and limitations
# under the License.
+if(TARGET CONAN_PKG::openblas)
+ set(OpenBLAS_FOUND ON)
+ set(OpenBLAS_LIB CONAN_PKG::openblas CONAN_PKG::lapack)
+ set(OpenBLAS_INCLUDE_DIR ${CONAN_INCLUDE_DIRS_OPENBLAS})
+ return()
+endif()
+
file(TO_CMAKE_PATH "$ENV{OpenBLAS_HOME}" OpenBLAS_HOME)
file(TO_CMAKE_PATH "$ENV{OpenBLAS}" OpenBLAS_DIR)
diff --git a/conanfile.py b/conanfile.py
new file mode 100644
index 00000000000..4c7b4a04b94
--- /dev/null
+++ b/conanfile.py
@@ -0,0 +1,11 @@
+from conans import ConanFile
+
+class IncubatorMXNetConan(ConanFile):
+ settings = "os", "compiler", "build_type", "arch"
+ requires = "openblas/0.2.20@conan/stable", "opencv/3.4.3@conan/stable",
"lapack/3.7.1@conan/stable"
+ generators = ["cmake"]
+
+ def configure(self):
+ if self.settings.compiler == "Visual Studio":
+ self.options["lapack"].visual_studio = True
+ self.options["lapack"].shared = True
From ca1f60695bcb5ef94bbbaadce475abe43d9a853f Mon Sep 17 00:00:00 2001
From: SSE4 <tomsks...@gmail.com>
Date: Sun, 25 Nov 2018 17:32:11 +0700
Subject: [PATCH 2/2] - add license to the conanfile.py
Signed-off-by: SSE4 <tomsks...@gmail.com>
---
conanfile.py | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/conanfile.py b/conanfile.py
index 4c7b4a04b94..87e64cf4d15 100644
--- a/conanfile.py
+++ b/conanfile.py
@@ -1,3 +1,20 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
from conans import ConanFile
class IncubatorMXNetConan(ConanFile):
