chrishkchris edited a comment on issue #468: Distributted module URL: https://github.com/apache/incubator-singa/pull/468#issuecomment-521635425 Updated on 15 Aug 2019 Latest successful build log just now after the commit 3076113 to add the license header, which also successfully build and run the jupyter notebook example. ``` ubuntu@ip-172-31-17-155:~/incubator-singa$ mkdir build ubuntu@ip-172-31-17-155:~/incubator-singa$ cd build ubuntu@ip-172-31-17-155:~/incubator-singa/build$ cmake -D CMAKE_PREFIX_PATH="/usr/local/cuda/lib64;/usr/local/cuda/" -DENABLE_TEST=OFF -DUSE_CUDA=ON -DUSE_PYTHON3=ON -DUSE_MKLDNN=ON -DUSE_MODULES=OFF -DUSE_DIST=ON .. -- The C compiler identification is GNU 5.4.0 -- The CXX compiler identification is GNU 5.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found suitable version "3.0.0", minimum required is "3.0") -- Found CBLAS: /usr/local/include -- Found GLOG: /usr/include -- Found cuda_v10.0 -- Found CUDNN: /usr/local/cuda/include -- Found Cudnn_7401 at /usr/local/cuda/include /usr/local/cuda/lib64/libcudnn.so -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3") -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so (found suitable version "3.5.2", minimum required is "3") -- Found SWIG: /usr/local/bin/swig (found suitable version "3.0.12", minimum required is "3.0.10") -- Found MKLDNN at /usr/local/include -- Found MPI at /home/ubuntu/mpich-3.3/build/include -- Found MPI lib at /home/ubuntu/mpich-3.3/build/lib/libmpi.so -- Found all lib at /usr/local/lib/libprotobuf.so;/usr/local/lib/libopenblas.so;/usr/lib/x86_64-linux-gnu/libglog.so;/usr/local/cuda/lib64/libcudnn.so;/usr/local/cuda/lib64/libcudart.so;/usr/local/cuda/lib64/libcurand.so;/usr/local/cuda/lib64/libcublas.so;/home/ubuntu/incubator-singa/build/lib/libcnmem.a;/usr/local/lib/libmkldnn.so;/home/ubuntu/mpich-3.3/build/lib/libmpi.so;/home/ubuntu/mpich-3.3/build/lib/libmpicxx.so -- Found NCCL at /usr/local/cuda/include -- Found NCCL lib at /usr/local/cuda/lib/libnccl.so -- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/incubator-singa/build ubuntu@ip-172-31-17-155:~/incubator-singa/build$ make -j2 Scanning dependencies of target cnmem Scanning dependencies of target copy_protobuf [ 1%] Creating directories for 'cnmem' [ 2%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/model.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: model.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) [ 3%] Performing download step (git clone) for 'cnmem' Cloning into 'cnmem'... [ 4%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/caffe.proto [ 5%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/core.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: core.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) [ 6%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/io.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: io.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) [ 7%] Copying Protobuf headers [ 7%] Built target copy_protobuf [ 8%] Building NVCC (Device) object src/CMakeFiles/cuda_compile_1.dir/core/tensor/cuda_compile_1_generated_math_kernel.cu.o Already on 'master' Your branch is up-to-date with 'origin/master'. [ 9%] No patch step for 'cnmem' [ 10%] Performing update step for 'cnmem' Current branch master is up to date. [ 11%] Performing configure step for 'cnmem' -- The C compiler identification is GNU 5.4.0 -- The CXX compiler identification is GNU 5.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/incubator-singa/build/cnmem-prefix/src/cnmem-build [ 12%] Performing build step for 'cnmem' Scanning dependencies of target cnmem [ 50%] Building CXX object CMakeFiles/cnmem.dir/src/cnmem.cpp.o [100%] Linking CXX static library libcnmem.a [100%] Built target cnmem [ 13%] Performing install step for 'cnmem' [100%] Built target cnmem Install the project... -- Install configuration: "" -- Installing: /home/ubuntu/incubator-singa/build/lib/libcnmem.a -- Installing: /home/ubuntu/incubator-singa/build/include/cnmem.h [ 14%] Completed 'cnmem' [ 14%] Built target cnmem Scanning dependencies of target singa_objects [ 15%] Building CXX object src/CMakeFiles/singa_objects.dir/caffe.pb.cc.o [ 16%] Building CXX object src/CMakeFiles/singa_objects.dir/core.pb.cc.o [ 17%] Building CXX object src/CMakeFiles/singa_objects.dir/io.pb.cc.o [ 18%] Building CXX object src/CMakeFiles/singa_objects.dir/model.pb.cc.o [ 19%] Building CXX object src/CMakeFiles/singa_objects.dir/utils/channel.cc.o [ 20%] Building CXX object src/CMakeFiles/singa_objects.dir/utils/logging.cc.o [ 21%] Building CXX object src/CMakeFiles/singa_objects.dir/io/binfile_reader.cc.o [ 22%] Building CXX object src/CMakeFiles/singa_objects.dir/io/binfile_writer.cc.o [ 23%] Building CXX object src/CMakeFiles/singa_objects.dir/io/csv_decoder.cc.o [ 24%] Building CXX object src/CMakeFiles/singa_objects.dir/io/csv_encoder.cc.o [ 25%] Building CXX object src/CMakeFiles/singa_objects.dir/io/image_transformer.cc.o [ 26%] Building CXX object src/CMakeFiles/singa_objects.dir/io/jpg_decoder.cc.o [ 27%] Building CXX object src/CMakeFiles/singa_objects.dir/io/jpg_encoder.cc.o [ 28%] Building CXX object src/CMakeFiles/singa_objects.dir/io/lmdb_reader.cc.o [ 29%] Building CXX object src/CMakeFiles/singa_objects.dir/io/lmdb_writer.cc.o [ 30%] Building CXX object src/CMakeFiles/singa_objects.dir/io/snapshot.cc.o [ 31%] Building CXX object src/CMakeFiles/singa_objects.dir/io/textfile_reader.cc.o [ 32%] Building CXX object src/CMakeFiles/singa_objects.dir/io/textfile_writer.cc.o [ 34%] Building CXX object src/CMakeFiles/singa_objects.dir/io/network/endpoint.cc.o [ 35%] Building CXX object src/CMakeFiles/singa_objects.dir/io/network/message.cc.o [ 36%] Building CXX object src/CMakeFiles/singa_objects.dir/core/device/cpp_cpu.cc.o [ 37%] Building CXX object src/CMakeFiles/singa_objects.dir/core/device/cuda_gpu.cc.o [ 38%] Building CXX object src/CMakeFiles/singa_objects.dir/core/device/device.cc.o [ 39%] Building CXX object src/CMakeFiles/singa_objects.dir/core/device/opencl_device.cc.o [ 40%] Building CXX object src/CMakeFiles/singa_objects.dir/core/device/platform.cc.o [ 41%] Building CXX object src/CMakeFiles/singa_objects.dir/core/memory/memory.cc.o [ 42%] Building CXX object src/CMakeFiles/singa_objects.dir/core/scheduler/scheduler.cc.o [ 43%] Building CXX object src/CMakeFiles/singa_objects.dir/core/tensor/sparse_tensor.cc.o [ 44%] Building CXX object src/CMakeFiles/singa_objects.dir/core/tensor/tensor.cc.o [ 45%] Building CXX object src/CMakeFiles/singa_objects.dir/dist/communicator.cc.o [ 46%] Building CXX object src/CMakeFiles/singa_objects.dir/model/feed_forward_net.cc.o [ 47%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/activation.cc.o [ 48%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/batchnorm.cc.o [ 49%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/concat.cc.o [ 50%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/convolution.cc.o [ 51%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_activation.cc.o [ 52%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_batchnorm.cc.o [ 53%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_convolution.cc.o [ 54%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_dropout.cc.o [ 55%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_lrn.cc.o [ 56%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_pooling.cc.o [ 57%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_rnn.cc.o [ 58%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/cudnn_softmax.cc.o [ 59%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/dense.cc.o [ 60%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/dropout.cc.o [ 61%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/flatten.cc.o [ 62%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/lrn.cc.o [ 63%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/merge.cc.o [ 64%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/opencl_convolution.cc.o [ 65%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/opencl_pooling.cc.o [ 67%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/pooling.cc.o [ 68%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/prelu.cc.o [ 69%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/rnn.cc.o [ 70%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/slice.cc.o [ 71%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/softmax.cc.o [ 72%] Building CXX object src/CMakeFiles/singa_objects.dir/model/layer/split.cc.o [ 73%] Building CXX object src/CMakeFiles/singa_objects.dir/model/optimizer/adagrad.cc.o [ 74%] Building CXX object src/CMakeFiles/singa_objects.dir/model/optimizer/local_all_reduce.cc.o [ 75%] Building CXX object src/CMakeFiles/singa_objects.dir/model/optimizer/nesterov.cc.o [ 76%] Building CXX object src/CMakeFiles/singa_objects.dir/model/optimizer/optimizer.cc.o [ 77%] Building CXX object src/CMakeFiles/singa_objects.dir/model/optimizer/rmsprop.cc.o [ 78%] Building CXX object src/CMakeFiles/singa_objects.dir/model/optimizer/sgd.cc.o [ 79%] Building CXX object src/CMakeFiles/singa_objects.dir/model/loss/mse.cc.o [ 80%] Building CXX object src/CMakeFiles/singa_objects.dir/model/loss/softmax_cross_entropy.cc.o [ 81%] Building CXX object src/CMakeFiles/singa_objects.dir/model/metric/accuracy.cc.o [ 82%] Building CXX object src/CMakeFiles/singa_objects.dir/model/updater/local_updater.cc.o [ 83%] Building CXX object src/CMakeFiles/singa_objects.dir/model/updater/updater.cc.o [ 84%] Building CXX object src/CMakeFiles/singa_objects.dir/model/operation/batchnorm.cc.o [ 85%] Building CXX object src/CMakeFiles/singa_objects.dir/model/operation/convolution.cc.o [ 86%] Building CXX object src/CMakeFiles/singa_objects.dir/model/operation/pooling.cc.o /home/ubuntu/incubator-singa/src/model/operation/pooling.cc: In function ‘singa::Tensor singa::GpuPoolingForward(const singa::CudnnPoolingHandle&, const singa::Tensor&)’: /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:220:51: warning: narrowing conversion of ‘(int)(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::batchsize’ from ‘int’ to ‘long unsigned int’ inside { } [-Wnarrowing] x.device(), x.data_type()); ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:219:31: warning: narrowing conversion of ‘(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::batchsize’ from ‘const int’ to ‘long unsigned int’ inside { } [-Wnarrowing] Tensor output = Tensor({cph.batchsize, cph.channels, cph.pooled_height, cph.pooled_width}, ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:220:51: warning: narrowing conversion of ‘(int)(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::channels’ from ‘int’ to ‘long unsigned int’ inside { } [-Wnarrowing] x.device(), x.data_type()); ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:219:46: warning: narrowing conversion of ‘(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::channels’ from ‘const int’ to ‘long unsigned int’ inside { } [-Wnarrowing] Tensor output = Tensor({cph.batchsize, cph.channels, cph.pooled_height, cph.pooled_width}, ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:220:51: warning: narrowing conversion of ‘(int)(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::pooled_height’ from ‘int’ to ‘long unsigned int’ inside { } [-Wnarrowing] x.device(), x.data_type()); ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:219:60: warning: narrowing conversion of ‘(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::pooled_height’ from ‘const int’ to ‘long unsigned int’ inside { } [-Wnarrowing] Tensor output = Tensor({cph.batchsize, cph.channels, cph.pooled_height, cph.pooled_width}, ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:220:51: warning: narrowing conversion of ‘(int)(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::pooled_width’ from ‘int’ to ‘long unsigned int’ inside { } [-Wnarrowing] x.device(), x.data_type()); ^ /home/ubuntu/incubator-singa/src/model/operation/pooling.cc:219:79: warning: narrowing conversion of ‘(& cph)->singa::CudnnPoolingHandle::<anonymous>.singa::PoolingHandle::pooled_width’ from ‘const int’ to ‘long unsigned int’ inside { } [-Wnarrowing] Tensor output = Tensor({cph.batchsize, cph.channels, cph.pooled_height, cph.pooled_width}, ^ [ 90%] Built target singa_objects Scanning dependencies of target singa [ 91%] Running Python protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/caffe.proto [ 92%] Linking CXX shared library ../lib/libsinga.so [ 93%] Running Python protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/core.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: core.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) [ 94%] Running Python protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/io.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: io.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) [ 95%] Running Python protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/model.proto [libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: model.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) Scanning dependencies of target _singa_wrap [ 96%] Building CXX object python/CMakeFiles/_singa_wrap.dir/__/src/api/singa_wrap.cxx.o [ 97%] Built target singa [ 98%] Linking CXX shared library singa/_singa_wrap.so [100%] Built target _singa_wrap ubuntu@ip-172-31-17-155:~/incubator-singa/build$ cd python ubuntu@ip-172-31-17-155:~/incubator-singa/build/python$ sudo pip3 uninstall singa The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Uninstalling singa-2.1.0: Would remove: /usr/local/bin/singa /usr/local/lib/python3.5/dist-packages/rafiki/* /usr/local/lib/python3.5/dist-packages/singa-2.1.0-py3.5.egg-info /usr/local/lib/python3.5/dist-packages/singa/* Proceed (y/n)? y Successfully uninstalled singa-2.1.0 ubuntu@ip-172-31-17-155:~/incubator-singa/build/python$ sudo pip3 install . The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Processing /home/ubuntu/incubator-singa/build/python Installing collected packages: singa Running setup.py install for singa ... done Successfully installed singa-2.1.0 You are using pip version 19.0.3, however version 19.2.2 is available. You should consider upgrading via the 'pip install --upgrade pip' command. ubuntu@ip-172-31-17-155:~/incubator-singa/build/python$ cd .. ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
