## Problem statement Currently unix- cpu/gpu CI run a host of builds and tests on each submitted PR. After going through the list of variaous flavours of builds I found that we are still having more than required test types and build flavours. I have following suggestions to simply both these CI tests. This will in turn also stabilize the CI further.
# SUGGESTIONS for unix-CPU (sample reference: https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/v1.x/269/pipeline/307) Builds: - We have make builds that can be converted to cmake+ninja to have consistency (e.g. `CPU:MKL, CPU:MKLDNN, CPU:MKLDNN_MKL, CPU:openblas`) - We have native builds w/o MKLDNN (which is ON by default) that can be removed/replaced with ones having MKLDNN enabled. Should we continue to support `native` binaries for both `darwin` and `linux`? (e.g. CPU:openblas, CPU: Openblas cmake TVM_OP OFF, CPU: Openblas cmake debug) Tests: - Julia 0.7 and 1.0 (we can keep only latest one) - All language binding tests that run using w/ and w/o MKLDNN build variants (like R, Python and Scala) can only run on w/ MKLDNN - Removing the following from test suite: - Static build CPU 14.04 Python (uses Make build, should be removed) - Static build CPU 14.04 Python with CMake (tested in CD nightly pipeline) - Static build CPU 14.04 Scala (not sure why this is need, more data appreciated as to why should be keep this) # SUGGESTIONS for unix-GPU (sample reference: https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/v1.x/267/pipeline/51) Builds: - We have make builds that can be converted to cmake+ninja to have consistency (`GPU:CUDA10.1+cuDNN7, GPU:CUDA10.1+cuDNN7+MKLDNN+CPPTEST, GPU:MKLDNN, GPU:MKLDNN_CUDNNOFF`) - We have native builds w/o MKLDNN (which is ON by default) that can be removed/replaced with ones having MKLDNN enabled. - There are debug builds as well(these generate bulky binaries with symbol tables, so shouldn't be removed) Tests: - Update test suits: `capi-cpp-package GPU Makefile` and `cpp-package GPU Makefile` to be built properly when the builds `GPU: CUDA10.1+cuDNN7` and `GPU: CUDA10.1+cuDNN7+MKLDNN+CPPTEST` are migrated from make to cmake+ninja builds. - Tests that run w/o MKLDNN can be transitioned to use MKLDNN builds unless there isn't already one present doing just that. We can simply remove it in that case. - Removing the following from test suite: - Static build GPU 14.04 Python (uses Make build, should be removed) NOTE: All the Above suggestions are meant for `v1.x` branch and reason behind those suggestions is either provided in the RFC or to simply cut down on testing that need not be required. Kindly let me know if its insufficient or I missed something. If you disagree please let me know in comments below why you do. Thank you! @access2rohit -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/20111
