> So my ideal instructions look more like > > 1. compile MXNet normally from a clean checkout > 2. cd into your own project, configure with `-DMXNET=/path/to/mxnet` and > compile. Sample project provided and part of integration tests. > 3. `my_op.so` built by my build system > 4. dynamically load shared library into MXNet via `mx.library.load()`
@kpuatamazon Heres another idea. In order to build files that used to be in **src/operator** for example, we need to have all the defines, includes, etc that is normally set in the MXNet build: ``` [ 98%] Building CXX object CMakeFiles/external_lib.dir/example/extensions/lib_external_ops/min_ex.cc.o /usr/bin/ccache /usr/bin/c++ -DDMLC_CORE_USE_CMAKE -DDMLC_LOG_FATAL_THROW=1 -DDMLC_LOG_STACK_TRACE_SIZE=0 -DDMLC_MODERN_THREAD_LOCAL=0 -DDMLC_STRICT_CXX11 -DDMLC_USE_CXX11 -DDMLC_USE_CXX11=1 -DDMLC_USE_CXX14 -DMSHADOW_INT64_TENSOR_SIZE=0 -DMSHADOW_IN_CXX11 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_SSE -DMXNET_USE_BLAS_OPEN=1 -DMXNET_USE_LAPACK=1 -DMXNET_USE_LIBJPEG_TURBO=0 -DMXNET_USE_MKLDNN=1 -DMXNET_USE_OPENCV=1 -DMXNET_USE_OPENMP=1 -DMXNET_USE_OPERATOR_TUNING=1 -DMXNET_USE_SIGNAL_HANDLER=1 -DNDEBUG=1 -D__USE_XOPEN2K8 -Dexternal_lib_EXPORTS -I/home/ubuntu/external_ops/3rdparty/mkldnn/include -I/home/ubuntu/external_ops/build/3rdparty/mkldnn/include -I/home/ubuntu/external_ops/include -I/home/ubuntu/external_ops/src -I/home/ubuntu/external_ops/3rdparty/tvm/nnvm/include -I/home/ubuntu/external_ops/3rdparty/tvm/include -I/home/ubuntu/external_ops/3rdparty/dmlc-core/include -I/home/ubuntu/external_ops/3rdparty/dlpack/include -I/home/ubuntu/external_ops/include/mxnet -I/home/ubuntu/external_ops/3rdparty/mshadow -I/home/ubuntu/external_ops/3rdparty/mkldnn/src/../include -I/home/ubuntu/external_ops/build/3rdparty/dmlc-core/include -isystem /usr/local/include -isystem /usr/local/include/opencv4 -Wall -Wno-sign-compare -O3 -fopenmp -fPIC -Wno-unused-parameter -Wno-unknown-pragmas -Wno-unused-local-typedefs -msse3 -mf16c -std=gnu++1z -o CMakeFiles/external_lib.dir/example/extensions/lib_external_ops/min_ex.cc.o -c /home/ubuntu/external_ops/example/extensions/lib_external_ops/min_ex.cc [ 98%] Linking CXX shared library libexternal_lib.so /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/external_lib.dir/link.txt --verbose=1 /usr/bin/c++ -fPIC -Wall -Wno-sign-compare -O3 -fopenmp -shared -Wl,-soname,libexternal_lib.so -o libexternal_lib.so CMakeFiles/external_lib.dir/example/extensions/lib_external_ops/init_lib.cc.o CMakeFiles/external_lib.dir/example/extensions/lib_external_ops/min_ex.cc.o CMakeFiles/external_lib.dir/src/lib_api.cc.o -Wl,-rpath,/home/ubuntu/external_ops/build:/usr/local/lib:/home/ubuntu/external_ops/build/3rdparty/openmp/runtime/src libmxnet.so 3rdparty/mkldnn/src/libdnnl.a /usr/local/lib/libopenblas.so /usr/lib/x86_64-linux-gnu/librt.so /usr/local/lib/libopencv_highgui.so.4.2.0 /usr/local/lib/libopencv_videoio.so.4.2.0 /usr/local/lib/libopencv_imgcodecs.so.4.2.0 /usr/local/lib/libopencv_imgproc.so.4.2.0 /usr/local/lib/libopencv_core.so.4.2.0 3rdparty/openmp/runtime/src/libomp.so -ldl -lpthread -lpthread -llapack 3rdparty/dmlc-core/libdmlc.a /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.so -lpthread -lrt ``` After all, the use case for this feature is to be able to use custom components in a publicly released build of MXNet (without those components compiled in statically). So initially, those components were build right as part of an MXNet build at some point. So they will need to continue being built the same way. With all of MXNet's complicated layout (includes all over the place, not just in **include/mxnet** directory) and 3rd submodules its not currently possible to just build MXNet and link it against a custom library. The hardest part is figuring out all the defines/includes/etc to compile a library with. In this new idea, theres a build target setup called "external_lib" in MXNet's CMakeLists.txt (just like we already have for other extensions examples like "pass_lib") that compiles all *.cc and *.cu files in the **example/extensions/lib_external_ops** directory. After copying your custom files into the **lib_external_ops** directory you can just go and run `cmake` and `make` to build your custom library. Ive updated the README for the example as: https://github.com/apache/incubator-mxnet/pull/18904/files#diff-70cbaa0e978356ecb01db8beb907ab48R31-R38 -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/pull/18904#issuecomment-679528549
