[incubator-mxnet] 01/01: Fix formatting and typos in CD README.md
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch KellenSunderland-patch-1-1
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
commit f9e7243cb4e33a6bedd7e6399a3a02e44c60dd4a
Author: Kellen Sunderland
AuthorDate: Fri Nov 1 12:50:49 2019 -0700
Fix formatting and typos in CD README.md
---
cd/README.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/cd/README.md b/cd/README.md
index a8e3fb7..a3dc038 100644
--- a/cd/README.md
+++ b/cd/README.md
@@ -46,12 +46,12 @@ Currently, 10 variants are supported:
### CD Pipeline Job
-The [CD pipeline job](Jenkinsfile_cd_pipeline) take three parameters:
+The [CD pipeline job](Jenkinsfile_cd_pipeline) take two parameters:
* **RELEASE_BUILD**: Flags the run as a *release build*. The underlying jobs
can then use this environment variable to disambiguate between nightly and
release builds. Defaults to *false*.
* **MXNET_VARIANTS**: A comma separated list of variants to build. Defaults
to *all* variants.
-This job defines and executes the CD pipeline. For example, first publish the
MXNet library, then, in parallel, execute the python and maven releases. Every
step of the pipeline executes a trigger for a release
job](Jenkinsfile_release_job).
+This job defines and executes the CD pipeline. For example, first publish the
MXNet library, then, in parallel, execute the python and maven releases. Every
step of the pipeline executes a trigger for a [release
job](Jenkinsfile_release_job).
### Release Job
@@ -195,4 +195,4 @@ def test(mxnet_variant) {
Examples:
-Both the [statically linked
libmxnet](mxnet_lib/static/Jenkins_pipeline.groovy) and [dynamically linked
libmxnet](mxnet_lib/dynamic/Jenkins_pipeline.groovy) pipelines have long
running compilation and testing stages that **do not** require
specialized/expensive hardware (e.g. GPUs). Therefore, as mush as possible, it
is important to run each stage in on its own node, and design the pipeline to
spend the least amount of time possible on expensive hardware. E.g. for GPU
builds, only run GP [...]
\ No newline at end of file
+Both the [statically linked
libmxnet](mxnet_lib/static/Jenkins_pipeline.groovy) and [dynamically linked
libmxnet](mxnet_lib/dynamic/Jenkins_pipeline.groovy) pipelines have long
running compilation and testing stages that **do not** require
specialized/expensive hardware (e.g. GPUs). Therefore, as much as possible, it
is important to run each stage in on its own node, and design the pipeline to
spend the least amount of time possible on expensive hardware. E.g. for GPU
builds, only run GP [...]
[incubator-mxnet] branch KellenSunderland-patch-1-1 created (now f9e7243)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch KellenSunderland-patch-1-1 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at f9e7243 Fix formatting and typos in CD README.md This branch includes the following new commits: new f9e7243 Fix formatting and typos in CD README.md The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[incubator-mxnet] 01/01: Fixed some formatting and typos in CD Readme.
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch KellenSunderland-patch-1
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
commit 6feb3b6d5b203e5a4370c31074f0499186397887
Author: Kellen Sunderland
AuthorDate: Fri Nov 1 12:49:10 2019 -0700
Fixed some formatting and typos in CD Readme.
---
cd/README.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/cd/README.md b/cd/README.md
index a8e3fb7..711af5b 100644
--- a/cd/README.md
+++ b/cd/README.md
@@ -46,12 +46,12 @@ Currently, 10 variants are supported:
### CD Pipeline Job
-The [CD pipeline job](Jenkinsfile_cd_pipeline) take three parameters:
+The [CD pipeline job](Jenkinsfile_cd_pipeline) take tw parameters:
* **RELEASE_BUILD**: Flags the run as a *release build*. The underlying jobs
can then use this environment variable to disambiguate between nightly and
release builds. Defaults to *false*.
* **MXNET_VARIANTS**: A comma separated list of variants to build. Defaults
to *all* variants.
-This job defines and executes the CD pipeline. For example, first publish the
MXNet library, then, in parallel, execute the python and maven releases. Every
step of the pipeline executes a trigger for a release
job](Jenkinsfile_release_job).
+This job defines and executes the CD pipeline. For example, first publish the
MXNet library, then, in parallel, execute the python and maven releases. Every
step of the pipeline executes a trigger for a [release
job](Jenkinsfile_release_job).
### Release Job
@@ -195,4 +195,4 @@ def test(mxnet_variant) {
Examples:
-Both the [statically linked
libmxnet](mxnet_lib/static/Jenkins_pipeline.groovy) and [dynamically linked
libmxnet](mxnet_lib/dynamic/Jenkins_pipeline.groovy) pipelines have long
running compilation and testing stages that **do not** require
specialized/expensive hardware (e.g. GPUs). Therefore, as mush as possible, it
is important to run each stage in on its own node, and design the pipeline to
spend the least amount of time possible on expensive hardware. E.g. for GPU
builds, only run GP [...]
\ No newline at end of file
+Both the [statically linked
libmxnet](mxnet_lib/static/Jenkins_pipeline.groovy) and [dynamically linked
libmxnet](mxnet_lib/dynamic/Jenkins_pipeline.groovy) pipelines have long
running compilation and testing stages that **do not** require
specialized/expensive hardware (e.g. GPUs). Therefore, as much as possible, it
is important to run each stage in on its own node, and design the pipeline to
spend the least amount of time possible on expensive hardware. E.g. for GPU
builds, only run GP [...]
[incubator-mxnet] branch KellenSunderland-patch-1 created (now 6feb3b6)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch KellenSunderland-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at 6feb3b6 Fixed some formatting and typos in CD Readme. This branch includes the following new commits: new 6feb3b6 Fixed some formatting and typos in CD Readme. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[incubator-mxnet] branch master updated (217ae02 -> 746cbc5)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 217ae02 Fix numpy bugs (#16537) add 746cbc5 Add unit tests for TensorRT integration and fix some bugs (#15399) No new revisions were added by this update. Summary of changes: 3rdparty/onnx-tensorrt | 2 +- ci/docker/install/tensorrt.sh | 7 +- python/mxnet/contrib/tensorrt.py | 2 + src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h | 15 +- src/operator/subgraph/tensorrt/nnvm_to_onnx.cc | 43 +- src/operator/subgraph/tensorrt/onnx_to_tensorrt.cc | 2 +- src/operator/subgraph/tensorrt/tensorrt-inl.h | 119 - src/operator/subgraph/tensorrt/tensorrt.cc | 2 +- tests/python/tensorrt/common.py| 30 -- tests/python/tensorrt/lenet5_common.py | 31 -- tests/python/tensorrt/lenet5_train.py | 12 +- tests/python/tensorrt/test_cvnets.py | 7 +- tests/python/tensorrt/test_ops.py | 527 +++-- tests/python/tensorrt/test_resnet18.py | 2 +- tests/python/tensorrt/test_tensorrt_batchnorm.py | 65 --- tests/python/tensorrt/test_tensorrt_deconv.py | 63 --- tests/python/tensorrt/test_tensorrt_lenet5.py | 28 +- 17 files changed, 687 insertions(+), 270 deletions(-) delete mode 100644 tests/python/tensorrt/common.py delete mode 100644 tests/python/tensorrt/lenet5_common.py delete mode 100644 tests/python/tensorrt/test_tensorrt_batchnorm.py delete mode 100644 tests/python/tensorrt/test_tensorrt_deconv.py
[incubator-mxnet] branch master updated (217ae02 -> 746cbc5)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 217ae02 Fix numpy bugs (#16537) add 746cbc5 Add unit tests for TensorRT integration and fix some bugs (#15399) No new revisions were added by this update. Summary of changes: 3rdparty/onnx-tensorrt | 2 +- ci/docker/install/tensorrt.sh | 7 +- python/mxnet/contrib/tensorrt.py | 2 + src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h | 15 +- src/operator/subgraph/tensorrt/nnvm_to_onnx.cc | 43 +- src/operator/subgraph/tensorrt/onnx_to_tensorrt.cc | 2 +- src/operator/subgraph/tensorrt/tensorrt-inl.h | 119 - src/operator/subgraph/tensorrt/tensorrt.cc | 2 +- tests/python/tensorrt/common.py| 30 -- tests/python/tensorrt/lenet5_common.py | 31 -- tests/python/tensorrt/lenet5_train.py | 12 +- tests/python/tensorrt/test_cvnets.py | 7 +- tests/python/tensorrt/test_ops.py | 527 +++-- tests/python/tensorrt/test_resnet18.py | 2 +- tests/python/tensorrt/test_tensorrt_batchnorm.py | 65 --- tests/python/tensorrt/test_tensorrt_deconv.py | 63 --- tests/python/tensorrt/test_tensorrt_lenet5.py | 28 +- 17 files changed, 687 insertions(+), 270 deletions(-) delete mode 100644 tests/python/tensorrt/common.py delete mode 100644 tests/python/tensorrt/lenet5_common.py delete mode 100644 tests/python/tensorrt/test_tensorrt_batchnorm.py delete mode 100644 tests/python/tensorrt/test_tensorrt_deconv.py
[incubator-mxnet] branch v1.5.x updated: add deconv in TRT subgraph (#15666) (#16043)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch v1.5.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.5.x by this push:
new afe4a71 add deconv in TRT subgraph (#15666) (#16043)
afe4a71 is described below
commit afe4a713210749f8fe300f023ecc2678fbee502d
Author: Kellen Sunderland
AuthorDate: Sat Aug 31 09:24:05 2019 -0700
add deconv in TRT subgraph (#15666) (#16043)
---
src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h | 19 ++-
src/operator/subgraph/tensorrt/nnvm_to_onnx.cc| 46 -
src/operator/subgraph/tensorrt/tensorrt-inl.h | 2 +
tests/python/tensorrt/test_tensorrt_deconv.py | 63 +++
4 files changed, 116 insertions(+), 14 deletions(-)
diff --git a/src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h
b/src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h
index 55b3d93..5a433f1 100644
--- a/src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h
+++ b/src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h
@@ -41,6 +41,8 @@ namespace mxnet {
namespace op {
namespace nnvm_to_onnx {
+enum ConvDeconvType {Convolution, Deconvolution};
+
using namespace nnvm;
using namespace ::onnx;
using int64 = ::google::protobuf::int64;
@@ -48,8 +50,7 @@ using int64 = ::google::protobuf::int64;
std::unordered_map GetPlaceholderShapes(const
ShapeVector& shape_inputs,
const nnvm::IndexedGraph& ig);
-std::unordered_map GetPlaceholderDTypes(const DTypeVector&
-dtype_inputs,
+std::unordered_map GetPlaceholderDTypes(const DTypeVector&
dtype_inputs,
const nnvm::IndexedGraph& ig);
std::unordered_map GetOutputLookup(const
nnvm::IndexedGraph& ig);
@@ -74,12 +75,25 @@ typedef void (*ConverterFunction)(NodeProto *node_proto,
const nnvm::IndexedGraph ,
const array_view
);
+template
+void ConvDeconvConvertHelper(NodeProto *node_proto,
+ const NodeAttrs ,
+ const nnvm::IndexedGraph ,
+ const array_view ,
+ const ConvDeconvParam& param,
+ ConvDeconvType type);
+
// Forward declarations
void ConvertConvolution(NodeProto *node_proto,
const NodeAttrs ,
const nnvm::IndexedGraph ,
const array_view );
+void ConvertDeconvolution(NodeProto *node_proto,
+const NodeAttrs ,
+const nnvm::IndexedGraph ,
+const array_view );
+
void ConvertPooling(NodeProto *node_proto,
const NodeAttrs ,
const nnvm::IndexedGraph ,
@@ -158,6 +172,7 @@ static const std::unordered_map converter_map =
{"BatchNorm", ConvertBatchNorm},
{"clip", ConvertClip},
{"Convolution", ConvertConvolution},
+ {"Deconvolution", ConvertDeconvolution},
{"Concat", ConvertConcatenate},
{"Dropout", ConvertDropout},
{"elemwise_add", ConvertElementwiseAdd},
diff --git a/src/operator/subgraph/tensorrt/nnvm_to_onnx.cc
b/src/operator/subgraph/tensorrt/nnvm_to_onnx.cc
index 6116f29..84580d0 100644
--- a/src/operator/subgraph/tensorrt/nnvm_to_onnx.cc
+++ b/src/operator/subgraph/tensorrt/nnvm_to_onnx.cc
@@ -31,6 +31,7 @@
#include
#include
#include
+#include
#include "../../../common/utils.h"
#include "../../../ndarray/ndarray_function.h"
@@ -170,20 +171,25 @@ std::string ConvertNnvmGraphToOnnx(
return serialized_onnx_graph;
}
-void ConvertConvolution(NodeProto* node_proto, const NodeAttrs& attrs,
-const nnvm::IndexedGraph& /*ig*/,
-const array_view& /*inputs*/)
{
- const auto& conv_param = nnvm::get(attrs.parsed);
-
- node_proto->set_op_type("Conv");
+template
+void ConvDeconvConvertHelper(NodeProto* node_proto, const NodeAttrs& attrs,
+ const nnvm::IndexedGraph& /*ig*/,
+ const array_view&
/*input*/,
+ const ConvDeconvParam& param,
+ ConvDeconvType type) {
+ if (type == ConvDeconvType::Convolution) {
+node_proto->set_op_type("Conv");
+ } else {
+node_proto->set_op_type("ConvTranspose");
+ }
- const mxnet::TShape kernel = conv_param.kernel;
- const mxnet::TShape stride = conv_param.stride;
- const mxnet::TShape dilate = conv_param.dilate;
- const mxnet::TShape pad = conv_param.pad;
- const uint32_t num_group = conv_param.num_group;
+ const mxnet::TShape kernel = param.kernel;
+ const mxnet::TShape stride = param.stride;
+ const mxnet::TShape dilate = param.dilate;
+ const mxnet::TShape pad
[incubator-mxnet] branch master updated: cuDNN support cleanup (#15812)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new c81535c cuDNN support cleanup (#15812)
c81535c is described below
commit c81535ce33a9bb167662af53181f9a42ed5b4bc4
Author: Dick Carter
AuthorDate: Tue Aug 13 09:14:11 2019 -0700
cuDNN support cleanup (#15812)
* Remove old cudnn support (
#include
#include
+#include
/*! \brief Macros/inlines to assist CLion to parse Cuda files (*.cu, *.cuh) */
#ifdef __JETBRAINS_IDE__
@@ -482,13 +483,10 @@ static_assert(CUDNN_PATCHLEVEL < 100 && CUDNN_MINOR < 10,
* want to populate.
*/
inline int MaxForwardAlgos(cudnnHandle_t cudnn_handle) {
-#if CUDNN_MAJOR >= 7
+ STATIC_ASSERT_CUDNN_VERSION_GE(7000);
int max_algos = 0;
CUDNN_CALL(cudnnGetConvolutionForwardAlgorithmMaxCount(cudnn_handle,
_algos));
return max_algos;
-#else
- return 10;
-#endif
}
/*!
@@ -499,13 +497,10 @@ inline int MaxForwardAlgos(cudnnHandle_t cudnn_handle) {
* want to populate.
*/
inline int MaxBackwardFilterAlgos(cudnnHandle_t cudnn_handle) {
-#if CUDNN_MAJOR >= 7
+ STATIC_ASSERT_CUDNN_VERSION_GE(7000);
int max_algos = 0;
CUDNN_CALL(cudnnGetConvolutionBackwardFilterAlgorithmMaxCount(cudnn_handle,
_algos));
return max_algos;
-#else
- return 10;
-#endif
}
/*!
@@ -516,13 +511,10 @@ inline int MaxBackwardFilterAlgos(cudnnHandle_t
cudnn_handle) {
* want to populate.
*/
inline int MaxBackwardDataAlgos(cudnnHandle_t cudnn_handle) {
-#if CUDNN_MAJOR >= 7
+ STATIC_ASSERT_CUDNN_VERSION_GE(7000);
int max_algos = 0;
CUDNN_CALL(cudnnGetConvolutionBackwardDataAlgorithmMaxCount(cudnn_handle,
_algos));
return max_algos;
-#else
- return 10;
-#endif
}
#endif // MXNET_USE_CUDNN
diff --git a/src/executor/attach_op_resource_pass.cc
b/src/executor/attach_op_resource_pass.cc
index aa03a11..160ba8f 100644
--- a/src/executor/attach_op_resource_pass.cc
+++ b/src/executor/attach_op_resource_pass.cc
@@ -82,12 +82,12 @@ void AttachOpResources(
requested.push_back(ResourceManager::Get()->Request(ctx, req));
break;
}
-#if MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 7
+#if MXNET_USE_CUDNN == 1
case ResourceRequest::kCuDNNDropoutDesc: {
requested.push_back(ResourceManager::Get()->Request(ctx, req));
break;
}
-#endif // MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 7
+#endif // MXNET_USE_CUDNN == 1
default:
LOG(FATAL) << "resource type " << req.type << " is not yet
supported";
}
diff --git a/src/imperative/imperative_utils.h
b/src/imperative/imperative_utils.h
index b0476fc..21caafa 100644
--- a/src/imperative/imperative_utils.h
+++ b/src/imperative/imperative_utils.h
@@ -263,12 +263,12 @@ inline void SetDependency(const nnvm::NodeAttrs& attrs,
requested.push_back(ResourceManager::Get()->Request(ctx, req));
write_vars.push_back(requested.back().var);
break;
-#if MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 7
+#if MXNET_USE_CUDNN == 1
case ResourceRequest::kCuDNNDropoutDesc:
requested.push_back(ResourceManager::Get()->Request(ctx, req));
write_vars.push_back(requested.back().var);
break;
-#endif // MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 7
+#endif // MXNET_USE_CUDNN == 1
default:
LOG(FATAL) << "resource type not yet supported";
}
diff --git a/src/operator/bilinear_sampler.cu b/src/operator/bilinear_sampler.cu
index 03734a6..fab1433 100644
--- a/src/operator/bilinear_sampler.cu
+++ b/src/operator/bilinear_sampler.cu
@@ -27,9 +27,9 @@
#include "./bilinear_sampler-inl.h"
#include
#include "../common/cuda_utils.h"
-#if MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 5
+#if MXNET_USE_CUDNN == 1
#include "./cudnn_bilinear_sampler-inl.h"
-#endif // MXNET_USE_CUDNN && CUDNN_MAJOR
+#endif // MXNET_USE_CUDNN
namespace mshadow {
namespace cuda {
@@ -228,7 +228,7 @@ namespace op {
template<>
Operator* CreateOp(BilinearSamplerParam param, int dtype) {
Operator *op = NULL;
-#if MXNET_USE_CUDNN == 1 && CUDNN_MAJOR >= 5
+#if MXNET_USE_CUDNN == 1
MSHADOW_REAL_TYPE_SWITCH(dtype, DType, {
if (param.cudnn_off.has_value() && param.cudnn_off.value()) {
op = new BilinearSamplerOp(param);
@@ -240,7 +240,7 @@ Operator* CreateOp(BilinearSamplerParam param, int
dtype) {
MSHADOW_REAL_TYPE_SWITCH(dtype, DType, {
op = new BilinearSamplerOp(param);
})
-#endif // MXNET_USE_CUDNN && CUDNN_MAJOR
+#endif // MXNET_USE_CUDNN
return op;
}
diff --git a/src/operator/cudnn_bilinear_sampler-inl.h
b/src/operator/c
[incubator-mxnet] branch master updated (f0c69f5 -> 776d9c4)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git.
from f0c69f5 Add missing default axis value to symbol.squeeze op (#15707)
add 776d9c4 add deconv in TRT subgraph (#15666)
No new revisions were added by this update.
Summary of changes:
src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h | 19 -
src/operator/subgraph/tensorrt/nnvm_to_onnx.cc | 46 --
src/operator/subgraph/tensorrt/tensorrt-inl.h | 2 +
...nsorrt_batchnorm.py => test_tensorrt_deconv.py} | 12 +++---
4 files changed, 58 insertions(+), 21 deletions(-)
copy tests/python/tensorrt/{test_tensorrt_batchnorm.py =>
test_tensorrt_deconv.py} (84%)
[incubator-mxnet] branch master updated (7bc3db8 -> 6b22aa4)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7bc3db8 Bump Scala version to 1.6 (#15660) add 6b22aa4 handle fix_gamma in tensorrt subgraph conversion correctly (#15645) No new revisions were added by this update. Summary of changes: src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h | 21 ++-- src/operator/subgraph/tensorrt/nnvm_to_onnx.cc| 29 +- src/operator/subgraph/tensorrt/tensorrt.cc| 2 +- tests/python/tensorrt/test_tensorrt_batchnorm.py | 65 +++ 4 files changed, 109 insertions(+), 8 deletions(-) create mode 100644 tests/python/tensorrt/test_tensorrt_batchnorm.py
[incubator-mxnet] branch master updated (858e171 -> 0e0c7f3)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 858e171 1.5.0 news (#15137) add 0e0c7f3 fix LinearRegressionOutput with empty label (#15620) No new revisions were added by this update. Summary of changes: cpp-package/example/CMakeLists.txt | 4 +++ cpp-package/example/test_regress_label.cpp | 56 ++ cpp-package/tests/ci_test.sh | 3 ++ src/operator/regression_output-inl.h | 3 +- 4 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 cpp-package/example/test_regress_label.cpp
[incubator-mxnet] branch KellenSunderland-patch-2 created (now 6c38f3e)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch KellenSunderland-patch-2 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at 6c38f3e Small typo fixes in batch_norm-inl.h No new revisions were added by this update.
[incubator-mxnet] branch master updated (554b196 -> 6a564be)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 554b196 Rebase #13757 to master (#15189) add 6a564be cuda/cuDNN lib version checking. Force cuDNN v7 usage. (#15449) No new revisions were added by this update. Summary of changes: docs/faq/env_var.md | 10 + include/mxnet/resource.h | 8 ++-- src/common/cuda_utils.cc | 105 +++ src/common/cuda_utils.h | 27 src/resource.cc | 22 +- 5 files changed, 158 insertions(+), 14 deletions(-) create mode 100644 src/common/cuda_utils.cc
[incubator-mxnet] branch master updated (5078853 -> fc54781)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 5078853 remove comments from nano instructions (#15433) add fc54781 enable TensorRT integration with cpp api (#15335) No new revisions were added by this update. Summary of changes: cpp-package/example/inference/README.md| 14 +++ .../example/inference/imagenet_inference.cpp | 104 ++- cpp-package/include/mxnet-cpp/MxNetCpp.h | 1 + cpp-package/include/mxnet-cpp/contrib.h| 115 + cpp-package/include/mxnet-cpp/symbol.h | 17 +++ cpp-package/include/mxnet-cpp/symbol.hpp | 35 +++ 6 files changed, 260 insertions(+), 26 deletions(-) create mode 100644 cpp-package/include/mxnet-cpp/contrib.h
[incubator-mxnet] branch master updated (7fe478a -> 51acd4d)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7fe478a Fix build_ccache_wrappers: (#14631) add 51acd4d [MXNET-1086] added sub and mul to ONNX->TensorRT conversion (#15344) No new revisions were added by this update. Summary of changes: CMakeLists.txt| 2 +- src/operator/subgraph/tensorrt/nnvm_to_onnx-inl.h | 12 src/operator/subgraph/tensorrt/nnvm_to_onnx.cc| 12 tests/python/tensorrt/test_ops.py | 68 +++ 4 files changed, 93 insertions(+), 1 deletion(-) create mode 100644 tests/python/tensorrt/test_ops.py
[incubator-mxnet] branch master updated: [MXNET-857] Add initial NVTX profiler implementation (#12328)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new b22ee95 [MXNET-857] Add initial NVTX profiler implementation (#12328)
b22ee95 is described below
commit b22ee951ae45f7d34b9ae79433f318db5b6bc5ac
Author: Kellen Sunderland
AuthorDate: Fri May 10 22:15:12 2019 -0700
[MXNET-857] Add initial NVTX profiler implementation (#12328)
* [MXNET-857] Enable CUDA NVTX extensions for profiler
These extensions mark readable ranges in the NVIDIA Visual Profiler which
helps
show correlations between kernel launches and graph node executions.
Example shown here:
https://user-images.githubusercontent.com/7443219/33946110-34296d18-e021-11e7-8d18-6d40b797405c.png
The additional information enabled is in the 'Markers and Ranges' row.
* [MXNET-857] Add initial NVTX profiler implementation
This commit removes NVTX headers from the Amalgamation build process,
but this is a CUDA/CMake only feature, so it's not relevant to
Amalagamation builds.
* [MXNET-857] Use macro for NVTX specific code
* [MXNET-857] Add integration test.
* Turn on NVTX by default in Unix.
* Fixed typos and added NTVX info to profiler.md
* Add NVTX example to profiling tutorial
* Add NVTX flags for make
---
CMakeLists.txt | 10
Makefile | 5 ++
amalgamation/amalgamation.py | 2 +-
cmake/Modules/FindNVTX.cmake | 38 +++
docs/api/python/profiler/profiler.md | 4 +-
docs/tutorials/python/profiler.md| 25 +-
docs/tutorials/python/profiler_nvprof.png| Bin 0 -> 235747 bytes
docs/tutorials/python/profiler_nvprof_zoomed.png | Bin 0 -> 254663 bytes
docs/tutorials/python/profiler_winograd.png | Bin 0 -> 75450 bytes
make/config.mk | 3 ++
src/profiler/nvtx.cc | 21
src/profiler/nvtx.h | 59 +++
src/profiler/profiler.h | 20
tests/python/profiling/simple_forward.py | 42
tests/python/profiling/test_nvtx.py | 52
15 files changed, 277 insertions(+), 4 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6f8c33b..896c7b7 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -290,6 +290,16 @@ if(USE_CUDA)
message(WARNING "Could not find NCCL libraries")
endif()
endif()
+ if(UNIX)
+find_package(NVTX)
+if(NVTX_FOUND)
+ include_directories(${NVTX_INCLUDE_DIRS})
+ list(APPEND mxnet_LINKER_LIBS ${NVTX_LIBRARIES})
+ add_definitions(-DMXNET_USE_NVTX=1)
+else()
+ message(WARNING "Could not find NVTX libraries")
+endif()
+ endif()
else()
add_definitions(-DMSHADOW_USE_CUDA=0)
endif()
diff --git a/Makefile b/Makefile
index df0fe88..b578683 100644
--- a/Makefile
+++ b/Makefile
@@ -106,6 +106,11 @@ ifeq ($(ENABLE_TESTCOVERAGE), 1)
LDFLAGS += --coverage
endif
+ifeq ($(USE_NVTX), 1)
+CFLAGS += -DMXNET_USE_NVTX=1
+LDFLAGS += -lnvToolsExt
+endif
+
ifeq ($(USE_TENSORRT), 1)
CFLAGS += -I$(ROOTDIR) -I$(TPARTYDIR)
-DONNX_NAMESPACE=$(ONNX_NAMESPACE) -DMXNET_USE_TENSORRT=1
LDFLAGS += -lprotobuf -pthread -lonnx -lonnx_proto -lnvonnxparser
-lnvonnxparser_runtime -lnvinfer -lnvinfer_plugin
diff --git a/amalgamation/amalgamation.py b/amalgamation/amalgamation.py
index e47ab6b..fef54aa 100644
--- a/amalgamation/amalgamation.py
+++ b/amalgamation/amalgamation.py
@@ -30,7 +30,7 @@ blacklist = [
'opencv2/opencv.hpp', 'sys/stat.h', 'sys/types.h', 'cuda.h',
'cuda_fp16.h', 'omp.h',
'onnx/onnx.pb.h', 'execinfo.h', 'packet/sse-inl.h', 'emmintrin.h',
'thrust/device_vector.h',
'cusolverDn.h', 'internal/concurrentqueue_internal_debug.h',
'relacy/relacy_std.hpp',
-'relacy_shims.h', 'ittnotify.h', 'shared_mutex'
+'relacy_shims.h', 'ittnotify.h', 'shared_mutex', 'nvToolsExt.h'
]
minimum = int(sys.argv[6]) if len(sys.argv) > 5 else 0
diff --git a/cmake/Modules/FindNVTX.cmake b/cmake/Modules/FindNVTX.cmake
new file mode 100644
index 000..bf05eae
--- /dev/null
+++ b/cmake/Modules/FindNVTX.cmake
@@ -0,0 +1,38 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the Lice
[incubator-mxnet] branch master updated (bde1b84 -> 1c874cf)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git.
from bde1b84 [int8] Add MobileNetV2_1.0 & ResNet18 Quantization (#14823)
add 1c874cf reformat trt to use subgraph API, add fp16 support (#14040)
No new revisions were added by this update.
Summary of changes:
3rdparty/onnx-tensorrt | 2 +-
ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt | 1 +
include/mxnet/c_api.h | 1 -
python/mxnet/contrib/tensorrt.py | 119 ++--
src/c_api/c_api_executor.cc| 86 +--
src/common/serialization.h | 318 ---
src/executor/exec_pass.h | 12 -
src/executor/graph_executor.cc | 10 +
src/executor/graph_executor.h | 1 +
src/executor/tensorrt_pass.cc | 596 -
src/executor/trt_graph_executor.cc | 443 ---
src/executor/trt_graph_executor.h | 111
src/operator/contrib/tensorrt-inl.h| 79 ---
src/operator/contrib/tensorrt.cc | 181 ---
src/operator/subgraph/build_subgraph.cc| 16 +-
.../tensorrt}/nnvm_to_onnx-inl.h | 131 ++---
.../{contrib => subgraph/tensorrt}/nnvm_to_onnx.cc | 281 ++
.../subgraph/tensorrt}/onnx_to_tensorrt.cc | 27 +-
.../subgraph/tensorrt}/onnx_to_tensorrt.h | 18 +-
src/operator/subgraph/tensorrt/tensorrt-inl.h | 240 +
src/operator/subgraph/tensorrt/tensorrt.cc | 336
.../{contrib => subgraph/tensorrt}/tensorrt.cu | 30 +-
tests/cpp/misc/serialization.cc| 68 ---
tests/python/tensorrt/lenet5_train.py | 2 +
tests/python/tensorrt/test_cvnets.py | 34 +-
tests/python/tensorrt/test_cycle.py| 69 ---
tests/python/tensorrt/test_resnet18.py | 42 +-
tests/python/tensorrt/test_tensorrt_lenet5.py | 91 ++--
tests/python/tensorrt/test_training_warning.py | 70 ---
29 files changed, 1012 insertions(+), 2403 deletions(-)
delete mode 100644 src/common/serialization.h
delete mode 100644 src/executor/tensorrt_pass.cc
delete mode 100644 src/executor/trt_graph_executor.cc
delete mode 100644 src/executor/trt_graph_executor.h
delete mode 100644 src/operator/contrib/tensorrt-inl.h
delete mode 100644 src/operator/contrib/tensorrt.cc
rename src/operator/{contrib => subgraph/tensorrt}/nnvm_to_onnx-inl.h (61%)
rename src/operator/{contrib => subgraph/tensorrt}/nnvm_to_onnx.cc (68%)
rename src/{executor => operator/subgraph/tensorrt}/onnx_to_tensorrt.cc (89%)
rename src/{executor => operator/subgraph/tensorrt}/onnx_to_tensorrt.h (88%)
create mode 100644 src/operator/subgraph/tensorrt/tensorrt-inl.h
create mode 100644 src/operator/subgraph/tensorrt/tensorrt.cc
rename src/operator/{contrib => subgraph/tensorrt}/tensorrt.cu (69%)
delete mode 100644 tests/cpp/misc/serialization.cc
delete mode 100644 tests/python/tensorrt/test_cycle.py
delete mode 100644 tests/python/tensorrt/test_training_warning.py
[incubator-mxnet] branch master updated: Fix spelling in threaded_engine_test (#14709)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 42e929b Fix spelling in threaded_engine_test (#14709)
42e929b is described below
commit 42e929bcb35d2d0a7cff4b09aa6e392976e75acc
Author: Kellen Sunderland
AuthorDate: Wed Apr 17 10:04:10 2019 -0700
Fix spelling in threaded_engine_test (#14709)
---
tests/cpp/engine/threaded_engine_test.cc | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/tests/cpp/engine/threaded_engine_test.cc
b/tests/cpp/engine/threaded_engine_test.cc
index 405f3b3..ef3aec1 100644
--- a/tests/cpp/engine/threaded_engine_test.cc
+++ b/tests/cpp/engine/threaded_engine_test.cc
@@ -42,7 +42,7 @@
* present the following workload
* n = reads.size()
* data[write] = (data[reads[0]] + ... data[reads[n]]) / n
- * std::this_thread::sleep_for(std::chrono::microsecons(time));
+ * std::this_thread::sleep_for(std::chrono::microseconds(time));
*/
struct Workload {
std::vector reads;
@@ -76,7 +76,7 @@ void GenerateWorkload(int num_workloads, int num_var,
/**
* evaluate a single workload
*/
-void EvaluateWorload(const Workload& wl, std::vector* data) {
+void EvaluateWorkload(const Workload& wl, std::vector* data) {
double tmp = 0;
for (int i : wl.reads) tmp += data->at(i);
data->at(wl.write) = tmp / (wl.reads.size() + 1);
@@ -88,9 +88,9 @@ void EvaluateWorload(const Workload& wl, std::vector*
data) {
/**
* evaluate a list of workload, return the time used
*/
-double EvaluateWorloads(const std::vector& workloads,
- mxnet::Engine* engine,
- std::vector* data) {
+double EvaluateWorkloads(const std::vector& workloads,
+ mxnet::Engine* engine,
+ std::vector* data) {
using namespace mxnet;
double t = dmlc::GetTime();
std::vector vars;
@@ -103,10 +103,10 @@ double EvaluateWorloads(const std::vector&
workloads,
for (const auto& wl : workloads) {
if (wl.reads.size() == 0) continue;
if (engine == NULL) {
- EvaluateWorload(wl, data);
+ EvaluateWorkload(wl, data);
} else {
auto func = [wl, data](RunContext ctx, Engine::CallbackOnComplete cb) {
-EvaluateWorload(wl, data); cb();
+EvaluateWorkload(wl, data); cb();
};
std::vector reads;
for (auto i : wl.reads) {
@@ -159,7 +159,7 @@ TEST(Engine, RandSumExpr) {
std::vector> data(num_engine);
for (int i = 0; i < num_engine; ++i) {
data[i].resize(num_var, 1.0);
- t[i] += EvaluateWorloads(workloads, engine[i], [i]);
+ t[i] += EvaluateWorkloads(workloads, engine[i], [i]);
}
for (int i = 1; i < num_engine; ++i) {
[incubator-mxnet] branch master updated: Reference engine from chunk via weak pointer (#14591)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 3e2f752 Reference engine from chunk via weak pointer (#14591)
3e2f752 is described below
commit 3e2f752d493b3819aa7ac60c58a7b6caa346e13a
Author: Kellen Sunderland
AuthorDate: Tue Apr 16 20:45:52 2019 -0700
Reference engine from chunk via weak pointer (#14591)
---
include/mxnet/ndarray.h | 22 +++--
src/ndarray/ndarray.cc | 24 ++-
tests/cpp/engine/engine_shutdown_test.cc | 41
3 files changed, 69 insertions(+), 18 deletions(-)
diff --git a/include/mxnet/ndarray.h b/include/mxnet/ndarray.h
index 13fb42c..05d3fa4 100644
--- a/include/mxnet/ndarray.h
+++ b/include/mxnet/ndarray.h
@@ -850,15 +850,20 @@ class NDArray {
mxnet::ShapeVector aux_shapes;
/*! \brief Reference to the storage to ensure proper destruct order */
std::shared_ptr storage_ref_;
+/*! \brief Reference to the engine to ensure we cleanup without calling a
destructed engine */
+std::weak_ptr engine_ref_;
-/*! \brief default cosntructor */
+
+/*! \brief default constructor */
Chunk() : static_data(true), delay_alloc(false),
- storage_ref_(Storage::_GetSharedRef()) {}
+ storage_ref_(Storage::_GetSharedRef()),
+ engine_ref_(Engine::_GetSharedRef()) {}
/*! \brief construct a new chunk */
Chunk(mxnet::TShape shape, Context ctx_, bool delay_alloc_, int dtype)
: static_data(false), delay_alloc(true), ctx(ctx_),
- storage_ref_(Storage::_GetSharedRef()) {
+ storage_ref_(Storage::_GetSharedRef()),
+ engine_ref_(Engine::_GetSharedRef()) {
storage_shape = shape;
if (shape_is_known(storage_shape)) {
shandle.size = shape.Size() * mshadow::mshadow_sizeof(dtype);
@@ -872,7 +877,8 @@ class NDArray {
Chunk(const TBlob , int dev_id)
: static_data(true), delay_alloc(false),
- storage_ref_(Storage::_GetSharedRef()) {
+ storage_ref_(Storage::_GetSharedRef()),
+ engine_ref_(Engine::_GetSharedRef()) {
CHECK(storage_type == kDefaultStorage);
var = Engine::Get()->NewVariable();
if (data.dev_mask() == cpu::kDevMask) {
@@ -890,7 +896,8 @@ class NDArray {
Chunk(int shared_pid, int shared_id, const mxnet::TShape& shape, int dtype)
: static_data(false), delay_alloc(false),
- storage_ref_(Storage::_GetSharedRef()) {
+ storage_ref_(Storage::_GetSharedRef()),
+ engine_ref_(Engine::_GetSharedRef()) {
var = Engine::Get()->NewVariable();
ctx = Context::CPUShared(0);
shandle.size = shape.Size() * mshadow::mshadow_sizeof(dtype);
@@ -906,7 +913,8 @@ class NDArray {
const mxnet::ShapeVector _shapes_)
: static_data(false), delay_alloc(delay_alloc_),
storage_type(storage_type_),
aux_types(aux_types_), ctx(ctx_), storage_shape(storage_shape_),
- aux_shapes(aux_shapes_), storage_ref_(Storage::_GetSharedRef()) {
+ aux_shapes(aux_shapes_), storage_ref_(Storage::_GetSharedRef()),
+ engine_ref_(Engine::_GetSharedRef()) {
shandle.ctx = ctx;
var = Engine::Get()->NewVariable();
// aux_handles always reflect the correct number of aux data
@@ -924,7 +932,7 @@ class NDArray {
Chunk(const NDArrayStorageType storage_type_, const TBlob ,
const std::vector _data, int dev_id)
: static_data(true), delay_alloc(false), storage_type(storage_type_),
- storage_ref_(Storage::_GetSharedRef()) {
+ storage_ref_(Storage::_GetSharedRef()),
engine_ref_(Engine::_GetSharedRef()) {
using namespace mshadow;
CHECK_NE(storage_type, kDefaultStorage);
// init var
diff --git a/src/ndarray/ndarray.cc b/src/ndarray/ndarray.cc
index f5aac36..eddfbcf 100644
--- a/src/ndarray/ndarray.cc
+++ b/src/ndarray/ndarray.cc
@@ -113,20 +113,22 @@ NDArray::Chunk::~Chunk() {
// We want to delete mkldnn memory after deleting the variable.
mem.mem = this->mkl_mem_;
#endif
- Engine::Get()->DeleteVariable([mem, skip_free](RunContext s) {
-if (skip_free == false) {
+ if (auto engine = engine_ref_.lock()) {
+engine->DeleteVariable([mem, skip_free](RunContext s) {
+ if (skip_free == false) {
#if MXNET_USE_MKLDNN == 1
- if (mem.mem) {
-CHECK_LE(mem.mem->GetSize(), mem.h.size);
-CHECK_EQ(mem.mem->GetDataHandle(), mem.h.dptr);
- }
+if (mem.mem) {
+ CHECK_LE(mem.mem->GetSize(), mem.h.size);
+ CHECK_EQ(mem.mem->GetDataHandle(), mem.h.dptr);
+}
#endif
- Storage::Get()->Free(mem.h);
- for (const auto& aux : mem.aux_h) {
-Storage::Get()->Fr
[incubator-mxnet] branch KellenSunderland-patch-1 created (now 1e7e5fe)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch KellenSunderland-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at 1e7e5fe Tweak the copy for the cudnn autotuning warning. This branch includes the following new commits: new 1e7e5fe Tweak the copy for the cudnn autotuning warning. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[incubator-mxnet] 01/01: Tweak the copy for the cudnn autotuning warning.
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch KellenSunderland-patch-1
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
commit 1e7e5fe4a370dc164140da132dc18a715aad3cc7
Author: Kellen Sunderland
AuthorDate: Thu Apr 11 18:58:01 2019 -0700
Tweak the copy for the cudnn autotuning warning.
---
src/operator/nn/cudnn/cudnn_algoreg-inl.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/operator/nn/cudnn/cudnn_algoreg-inl.h
b/src/operator/nn/cudnn/cudnn_algoreg-inl.h
index cef9d6f..3f2d24c 100644
--- a/src/operator/nn/cudnn/cudnn_algoreg-inl.h
+++ b/src/operator/nn/cudnn/cudnn_algoreg-inl.h
@@ -96,7 +96,7 @@ class CuDNNAlgoReg {
if (param.cudnn_tune.value() && reg_.size() % 50 == 0) {
LOG(INFO) << "Running performance tests to find the best convolution "
"algorithm, "
-"this can take a while... (setting env variable "
+"this can take a while... (set the environment variable "
"MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)";
if (reg_.size() >= 1000) {
// Many people are very concerned about this warning, so change the
warning once.
[incubator-mxnet] branch master updated: fix test_stn (#14063)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new d02beeb fix test_stn (#14063) d02beeb is described below commit d02beeb6e4bad35a046810baac759ae50e7fdceb Author: Sheng Zha AuthorDate: Sun Feb 3 19:39:49 2019 -0800 fix test_stn (#14063) --- tests/python/unittest/test_operator.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/python/unittest/test_operator.py b/tests/python/unittest/test_operator.py index 146836c..6489756 100644 --- a/tests/python/unittest/test_operator.py +++ b/tests/python/unittest/test_operator.py @@ -2530,7 +2530,8 @@ def test_flip(): @with_seed() def test_stn(): -np.set_printoptions(threshold=np.nan) +import sys +np.set_printoptions(threshold=sys.maxsize) num_filter = 2 # conv of loc net kernel = (3, 3) # conv of loc net num_hidden = 6 # fc of loc net
[incubator-mxnet] branch master updated: api change (#13903)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 5dc138d api change (#13903)
5dc138d is described below
commit 5dc138d95f6ac302e3f0e1c9dc9dcb774d83f69e
Author: Zhennan Qin
AuthorDate: Sat Jan 26 04:05:21 2019 +0800
api change (#13903)
---
include/mxnet/ndarray.h | 10 +---
src/ndarray/ndarray.cc | 39 ++---
src/operator/subgraph/mkldnn/mkldnn_conv.cc | 12 ++---
3 files changed, 40 insertions(+), 21 deletions(-)
diff --git a/include/mxnet/ndarray.h b/include/mxnet/ndarray.h
index 4ba13ca..5de42e1 100644
--- a/include/mxnet/ndarray.h
+++ b/include/mxnet/ndarray.h
@@ -694,9 +694,13 @@ class NDArray {
/*
* Create NDArray from mkldnn memory.
* mkldnn_mem The mkldnn memory to be managed.
- * static_data If true, mkldnn memory won't be freed on destruction.
*/
- explicit NDArray(const mkldnn::memory *mkldnn_mem, bool static_data = true);
+ explicit NDArray(const std::shared_ptr _mem);
+ /*
+ * Create NDArray from mkldnn memory descriptor.
+ * mem_pd The mkldnn memory descriptor to be created.
+ */
+ explicit NDArray(mkldnn::memory::primitive_desc mem_pd);
/*
* Test if the data is stored in one of special MKLDNN format.
*/
@@ -776,7 +780,7 @@ class NDArray {
/*!
* \ Fix mkldnn memory descriptor mismatch from NDArray.
*/
- void UpdateMKLDNNMemDesc();
+ void UpdateMKLDNNMemDesc(mkldnn::memory::format format);
#endif
/*!
diff --git a/src/ndarray/ndarray.cc b/src/ndarray/ndarray.cc
index 251bfb3..0f0fed2 100644
--- a/src/ndarray/ndarray.cc
+++ b/src/ndarray/ndarray.cc
@@ -168,16 +168,28 @@ nnvm::Symbol NDArray::get_autograd_symbol() const {
#if MXNET_USE_MKLDNN == 1
-NDArray::NDArray(const mkldnn::memory *mkldnn_mem, bool static_data)
+NDArray::NDArray(mkldnn::memory::primitive_desc mem_pd)
: storage_type_(kDefaultStorage), entry_({nullptr, 0, 0}) {
- auto mem_pd = mkldnn_mem->get_primitive_desc();
auto mem_desc = mem_pd.desc();
shape_ = TShape(mem_desc.data.dims, mem_desc.data.dims +
mem_desc.data.ndims);
dtype_ = get_mxnet_type(mem_desc.data.data_type);
- auto data = TBlob(mkldnn_mem->get_data_handle(), shape_, cpu::kDevMask,
dtype_);
- ptr_ = std::make_shared(data, 0);
+ ptr_ = std::make_shared(shape_, Context::CPU(), true, dtype_);
+ ptr_->CheckAndAlloc(mem_pd.get_size());
ptr_->mkl_mem_ = std::make_shared(mem_pd, ptr_->shandle.dptr);
- ptr_->static_data = static_data;
+}
+
+NDArray::NDArray(const std::shared_ptr _mem)
+: storage_type_(kDefaultStorage), entry_({nullptr, 0, 0}) {
+ auto mem_pd = mkldnn_mem->get_primitive_desc();
+ auto mem_desc = mem_pd.desc();
+ shape_ = TShape(mem_desc.data.dims, mem_desc.data.dims +
mem_desc.data.ndims);
+ dtype_ = get_mxnet_type(mem_desc.data.data_type);
+ ptr_ = std::make_shared(shape_, Context::CPU(), true, dtype_);
+ ptr_->shandle.dptr = mkldnn_mem->get_data_handle();
+ ptr_->shandle.size = mem_pd.get_size();
+ ptr_->delay_alloc = false;
+ ptr_->mkl_mem_ = std::make_shared(mkldnn_mem);
+ ptr_->static_data = true;
}
NDArray NDArray::MKLDNNDataReshape(const TShape ) const {
@@ -710,19 +722,16 @@ mkldnn::memory *NDArray::CreateMKLDNNData(const
mkldnn::memory::primitive_desc &
return ptr_->mkl_mem_->GetRaw();
}
-void NDArray::UpdateMKLDNNMemDesc() {
+void NDArray::UpdateMKLDNNMemDesc(mkldnn::memory::format format) {
const mkldnn::memory *mem = GetMKLDNNData();
auto mem_desc = mem->get_primitive_desc().desc();
auto this_dtype = get_mkldnn_type(dtype());
- if (this_dtype != mem_desc.data.data_type) {
-mkldnn::memory::desc data_md(
-mkldnn::memory::dims(mem_desc.data.dims,
- mem_desc.data.dims + mem_desc.data.ndims),
-this_dtype, static_cast(mem_desc.data.format));
-mkldnn::memory::primitive_desc pd(data_md, CpuEngine::Get()->get_engine());
-ptr_->mkl_mem_.reset(new MKLDNNMemory(pd, ptr_->shandle.dptr));
-MKLDNNStream::Get()->RegisterMem(ptr_->mkl_mem_->GetMem());
- }
+ mkldnn::memory::desc data_md(
+ mkldnn::memory::dims(mem_desc.data.dims, mem_desc.data.dims +
mem_desc.data.ndims),
+ this_dtype, format);
+ mkldnn::memory::primitive_desc pd(data_md, CpuEngine::Get()->get_engine());
+ ptr_->mkl_mem_.reset(new MKLDNNMemory(pd, ptr_->shandle.dptr));
+ MKLDNNStream::Get()->RegisterMem(ptr_->mkl_mem_->GetMem());
}
#endif
diff --git a/src/operator/subgraph/mkldnn/mkldnn_conv.cc
b/src/operator/subgraph/mkldnn/mkldnn_conv.cc
index 65e0e5c..6a8feae 100644
--- a/src/operator/subgraph/mkldnn/mkldnn_conv.cc
+++ b/src/operator/subgraph/mkldnn/mkldnn_conv.cc
@@ -261,8 +261,12 @@ void SgMKLDNNConvOperator::Forwa
[incubator-mxnet] branch master updated: Update autoencoder example (#12933)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 402e985 Update autoencoder example (#12933) 402e985 is described below commit 402e985b1482bac6e8b89896bab18024e3a9fcfc Author: Thomas Delteil AuthorDate: Wed Jan 23 13:29:05 2019 -0800 Update autoencoder example (#12933) * Fixing the autoencoder example * adding pointer to VAE * fix typos * Update README.md * Updating notebook * Update after comments * Update README.md * Update README.md * Retrigger build * Updates after review --- example/autoencoder/README.md | 24 +- example/autoencoder/autoencoder.py | 206 .../autoencoder/convolutional_autoencoder.ipynb| 543 + example/autoencoder/data.py| 34 -- example/autoencoder/mnist_sae.py | 100 example/autoencoder/model.py | 78 --- example/autoencoder/solver.py | 151 -- 7 files changed, 557 insertions(+), 579 deletions(-) diff --git a/example/autoencoder/README.md b/example/autoencoder/README.md index 7efa30a..960636c 100644 --- a/example/autoencoder/README.md +++ b/example/autoencoder/README.md @@ -1,16 +1,20 @@ -# Example of Autencoder +# Example of a Convolutional Autoencoder -Autoencoder architecture is often used for unsupervised feature learning. This [link](http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/) contains an introduction tutorial to autoencoders. This example illustrates a simple autoencoder using stack of fully-connected layers for both encoder and decoder. The number of hidden layers and size of each hidden layer can be customized using command line arguments. +Autoencoder architectures are often used for unsupervised feature learning. This [link](http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/) contains an introduction tutorial to autoencoders. This example illustrates a simple autoencoder using a stack of convolutional layers for both the encoder and the decoder. -## Training Stages -This example uses a two-stage training. In the first stage, each layer of encoder and its corresponding decoder are trained separately in a layer-wise training loop. In the second stage the entire autoencoder network is fine-tuned end to end. + + + +([Diagram source](https://towardsdatascience.com/autoencoders-introduction-and-implementation-3f40483b0a85)) + + +The idea of an autoencoder is to learn to use bottleneck architecture to encode the input and then try to decode it to reproduce the original. By doing so, the network learns to effectively compress the information of the input, the resulting embedding representation can then be used in several domains. For example as featurized representation for visual search, or in anomaly detection. ## Dataset -The dataset used in this example is [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. This example uses scikit-learn module to download this dataset. -## Simple autoencoder example -mnist_sae.py: this example uses a simple auto-encoder architecture to encode and decode MNIST images with size of 28x28 pixels. It contains several command line arguments. Pass -h (or --help) to view all available options. To start the training on CPU (use --gpu option for training on GPU) using default options: +The dataset used in this example is [FashionMNIST](https://github.com/zalandoresearch/fashion-mnist) dataset. + +## Variational Autoencoder + +You can check an example of variational autoencoder [here](https://gluon.mxnet.io/chapter13_unsupervised-learning/vae-gluon.html) -``` -python mnist_sae.py -``` diff --git a/example/autoencoder/autoencoder.py b/example/autoencoder/autoencoder.py deleted file mode 100644 index 47931e5..000 --- a/example/autoencoder/autoencoder.py +++ /dev/null @@ -1,206 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language g
[incubator-mxnet] branch v1.4.x updated: fix bipartite match memory corruption (#13900)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a commit to branch v1.4.x in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/v1.4.x by this push: new bf86caa fix bipartite match memory corruption (#13900) bf86caa is described below commit bf86caa886cfc2e06dde131ed3c290559d418cac Author: Kellen Sunderland AuthorDate: Wed Jan 16 14:35:18 2019 -0800 fix bipartite match memory corruption (#13900) --- src/operator/contrib/bounding_box-inl.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/operator/contrib/bounding_box-inl.h b/src/operator/contrib/bounding_box-inl.h index 8e96346..031dd95 100644 --- a/src/operator/contrib/bounding_box-inl.h +++ b/src/operator/contrib/bounding_box-inl.h @@ -785,7 +785,7 @@ void BipartiteMatchingForward(const nnvm::NodeAttrs& attrs, .get_with_shape(Shape2(batch_size, col), s); Shape<1> sort_index_shape = Shape1(dshape.Size()); index_t workspace_size = sort_index_shape.Size(); -workspace_size += ((sort_index_shape.Size() * sizeof(int32_t) - 1) / sizeof(DType)) * 2; +workspace_size += (sort_index_shape.Size() * 2 * sizeof(int32_t) - 1) / sizeof(DType) + 1; Tensor workspace = ctx.requested[0] .get_space_typed(Shape1(workspace_size), s); Tensor scores_copy(workspace.dptr_,
[incubator-mxnet] branch master updated: [MXNET-703] Update to TensorRT 5, ONNX IR 3. Fix inference bugs. (#13310)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new cc15d9a [MXNET-703] Update to TensorRT 5, ONNX IR 3. Fix inference
bugs. (#13310)
cc15d9a is described below
commit cc15d9af70ec9e27b69d7e442be90be3bbc56c6e
Author: Kellen Sunderland
AuthorDate: Tue Jan 15 22:14:46 2019 -0800
[MXNET-703] Update to TensorRT 5, ONNX IR 3. Fix inference bugs. (#13310)
* [MXNET-703] Install CUDA 10 compatible cmake
This works around a CUDA 10 cmake issue documented here:
https://github.com/clab/dynet/issues/1457
This fix is temporary; once an updated cmake package is published to
Ubuntu's
package repo it may be reverted.
* [MXNET-703] Update to TensorRT 5 ONNX IR 3. Fix inference bugs.
* [MXNET-703] Describe onnx opsets and major version
---
3rdparty/onnx-tensorrt | 2 +-
CMakeLists.txt | 1 +
ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt | 2 +-
ci/docker/install/tensorrt.sh | 4 +-
ci/docker/install/ubuntu_core.sh | 19 ++--
ci/docker/runtime_functions.sh | 34 +++---
ci/jenkins/Jenkins_steps.groovy| 2 +-
src/executor/onnx_to_tensorrt.cc | 2 +-
src/executor/tensorrt_pass.cc | 2 +-
src/executor/trt_graph_executor.cc | 2 +-
src/operator/contrib/nnvm_to_onnx-inl.h| 2 +-
src/operator/contrib/nnvm_to_onnx.cc | 138 +
tests/python/tensorrt/test_resnet18.py | 68
13 files changed, 180 insertions(+), 98 deletions(-)
diff --git a/3rdparty/onnx-tensorrt b/3rdparty/onnx-tensorrt
index 3d8ee04..f1c7aa6 16
--- a/3rdparty/onnx-tensorrt
+++ b/3rdparty/onnx-tensorrt
@@ -1 +1 @@
-Subproject commit 3d8ee049970e81ff4935cc7f36b653c0b27bcbbc
+Subproject commit f1c7aa63d88d8d8ef70490f2ebb6b33f7450218b
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3e3de20..23609e5 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -187,6 +187,7 @@ if(USE_TENSORRT)
include_directories(${ONNX_PATH})
include_directories(3rdparty/onnx-tensorrt/)
include_directories(3rdparty/)
+ include_directories(3rdparty/onnx-tensorrt/third_party/onnx/)
add_definitions(-DMXNET_USE_TENSORRT=1)
add_definitions(-DONNX_NAMESPACE=onnx)
diff --git a/ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt
b/ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt
index 255da31..f484411 100644
--- a/ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt
+++ b/ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt
@@ -18,7 +18,7 @@
#
# Dockerfile to run MXNet on Ubuntu 16.04 for CPU
-FROM nvidia/cuda:9.0-cudnn7-devel
+FROM nvidia/cuda:10.0-cudnn7-devel
WORKDIR /work/deps
diff --git a/ci/docker/install/tensorrt.sh b/ci/docker/install/tensorrt.sh
index 61e73ef..1950cad 100755
--- a/ci/docker/install/tensorrt.sh
+++ b/ci/docker/install/tensorrt.sh
@@ -26,7 +26,7 @@ pip3 install gluoncv==0.2.0
pushd .
cd ..
apt-get update
-apt-get install -y automake libtool
+apt-get install -y automake libtool zip
git clone --recursive -b 3.5.1.1 https://github.com/google/protobuf.git
cd protobuf
./autogen.sh
@@ -41,7 +41,7 @@ popd
# Install TensorRT
echo "TensorRT build enabled. Installing TensorRT."
-wget -qO tensorrt.deb
https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvinfer-runtime-trt-repo-ubuntu1604-4.0.1-ga-cuda9.0_1-1_amd64.deb
+wget -qO tensorrt.deb
https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvinfer-runtime-trt-repo-ubuntu1604-5.0.2-ga-cuda10.0_1-1_amd64.deb
dpkg -i tensorrt.deb
apt-get update
apt-get install -y --allow-downgrades libnvinfer-dev
diff --git a/ci/docker/install/ubuntu_core.sh b/ci/docker/install/ubuntu_core.sh
index 4382aa6..fc903e5 100755
--- a/ci/docker/install/ubuntu_core.sh
+++ b/ci/docker/install/ubuntu_core.sh
@@ -22,6 +22,10 @@
set -ex
apt-get update || true
+
+# Avoid interactive package installers such as tzdata.
+export DEBIAN_FRONTEND=noninteractive
+
apt-get install -y \
apt-transport-https \
build-essential \
@@ -41,10 +45,11 @@ apt-get install -y \
unzip \
wget
-
-# Ubuntu 14.04
-if [[ $(lsb_release -r | grep 14.04) ]]; then
-apt-get install -y cmake3
-else
-apt-get install -y cmake
-fi
+# Note: we specify an exact cmake version to work around a cmake 3.10 CUDA 10
issue.
+# Reference: https://github.com/clab/dynet/issues/1457
+mkdir /opt/cmake && cd /opt/cmake
+wget -nv https://cmake.org/files/v3.12/cmake-3.12.4-Linux-x86_64.sh
+sh cmake-3.12.4-Linux-x86_64.sh --prefix=/opt/cmake --skip-license
+ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake
+rm cmake-3.12.4-Linux-x86_64.sh
+cmake --version
diff
[incubator-mxnet] branch master updated: Temporarily disable website testing (#13887)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 9d42812 Temporarily disable website testing (#13887)
9d42812 is described below
commit 9d4281271c871a938f1ac4ee55b218872031963d
Author: Kellen Sunderland
AuthorDate: Tue Jan 15 15:16:26 2019 -0800
Temporarily disable website testing (#13887)
---
ci/jenkins/Jenkinsfile_website | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/ci/jenkins/Jenkinsfile_website b/ci/jenkins/Jenkinsfile_website
index acdd2be..95ea02b 100644
--- a/ci/jenkins/Jenkinsfile_website
+++ b/ci/jenkins/Jenkinsfile_website
@@ -37,10 +37,11 @@ core_logic: {
custom_steps.compile_unix_cpu_openblas()
])
- utils.parallel_stage('Deploy', [
-custom_steps.docs_website(),
-custom_steps.docs_julia()
- ])
+ // Disable website generation due to flakyness
https://github.com/apache/incubator-mxnet/issues/13833
+ //utils.parallel_stage('Deploy', [
+ // custom_steps.docs_website(),
+ // custom_steps.docs_julia()
+ //])
}
,
failure_handler: {
[incubator-mxnet] branch master updated: Fix launch bounds in spatial transformer (#13188)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 0faa5b7 Fix launch bounds in spatial transformer (#13188)
0faa5b7 is described below
commit 0faa5b72c8a94f53219b3da3144892ac415d5ea4
Author: Przemyslaw Tredak
AuthorDate: Mon Jan 14 14:33:17 2019 -0800
Fix launch bounds in spatial transformer (#13188)
* Fix launch bounds in spatial transformer
* Adding explanation in comment.
---
src/operator/spatial_transformer.cu | 43 +++--
1 file changed, 32 insertions(+), 11 deletions(-)
diff --git a/src/operator/spatial_transformer.cu
b/src/operator/spatial_transformer.cu
index 33dbe3e..fd330bd 100644
--- a/src/operator/spatial_transformer.cu
+++ b/src/operator/spatial_transformer.cu
@@ -35,12 +35,23 @@ template
__device__ bool between(DType value, int lowerBound, int upperBound) {
return (value >= lowerBound && value <= upperBound);
}
+
template
-__global__ void BilinearSamplingForwardKernel(const int i_c, const int i_h,
- const int i_w, const DType* data,
- const DType* grid, const int o_n,
- const int o_c, const int o_h,
- const int o_w, DType* out) {
+__global__ void
+/*
+ * In order to not generate the code that uses too many
+ * registers (resulting in too many resources requested
+ * error) we need to tell the compiler that we will be
+ * launching this kernel with cuda::kMaxThreadsPerBlock
+ * threads per block. Setting __launch_bounds__ ensures
+ * that such configuration can always be launched.
+ */
+__launch_bounds__(cuda::kMaxThreadsPerBlock, 1)
+BilinearSamplingForwardKernel(const int i_c, const int i_h,
+ const int i_w, const DType* data,
+ const DType* grid, const int o_n,
+ const int o_c, const int o_h,
+ const int o_w, DType* out) {
for (int index = (blockIdx.x + blockIdx.y * gridDim.x) * blockDim.x +
threadIdx.x;
index < o_n * o_c * o_h * o_w;
index += blockDim.x * gridDim.x * gridDim.y) {
@@ -77,13 +88,23 @@ __global__ void BilinearSamplingForwardKernel(const int
i_c, const int i_h,
}
}
+/*
+ * In order to not generate the code that uses too many
+ * registers (resulting in too many resources requested
+ * error) we need to tell the compiler that we will be
+ * launching this kernel with cuda::kMaxThreadsPerBlock
+ * threads per block. Setting __launch_bounds__ ensures
+ * that such configuration can always be launched.
+ */
template
-__global__ void BilinearSamplingBackwardKernel(const int i_c, const int i_h,
- const int i_w, const DType* grad,
- const DType* data, const int o_n,
- const int o_c, const int o_h,
- const int o_w, DType* g_input,
- DType* grid_src) {
+__global__ void
+__launch_bounds__(cuda::kMaxThreadsPerBlock, 1)
+BilinearSamplingBackwardKernel(const int i_c, const int i_h,
+ const int i_w, const DType* grad,
+ const DType* data, const int o_n,
+ const int o_c, const int o_h,
+ const int o_w, DType* g_input,
+ DType* grid_src) {
for (int index = (blockIdx.x + blockIdx.y * gridDim.x) * blockDim.x +
threadIdx.x;
index < o_n * o_h * o_w;
index += blockDim.x * gridDim.x * gridDim.y) {
[incubator-mxnet] branch master updated: remove attributs clear on TRT nodes for GetOptimizedSymbol (#13703)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 1f151fb remove attributs clear on TRT nodes for GetOptimizedSymbol
(#13703)
1f151fb is described below
commit 1f151fbbc00751a590ec2377377003b421c45dec
Author: Caenorst
AuthorDate: Mon Jan 7 21:10:16 2019 -0500
remove attributs clear on TRT nodes for GetOptimizedSymbol (#13703)
---
src/executor/trt_graph_executor.cc | 9 +
1 file changed, 1 insertion(+), 8 deletions(-)
diff --git a/src/executor/trt_graph_executor.cc
b/src/executor/trt_graph_executor.cc
index 65dbb29..ec35fee 100644
--- a/src/executor/trt_graph_executor.cc
+++ b/src/executor/trt_graph_executor.cc
@@ -407,14 +407,7 @@ nnvm::Symbol TrtGraphExecutor::GetOptimizedSymbol() {
Symbol ret;
ret.outputs = std::vector(graph_.outputs.begin(),
graph_.outputs.begin() +
num_forward_outputs_);
- ret = ret.Copy();
- static const Op* trt_op = Op::Get("_trt_op");
- DFSVisit(ret.outputs, [](const nnvm::NodePtr n) {
-if (n->op() == trt_op) {
- n->attrs.dict.clear();
-}
- });
- return ret;
+ return ret.Copy();
}
Executor *TrtGraphExecutor::TensorRTBind(nnvm::Symbol symbol,
[incubator-mxnet] branch master updated: CMake: Enable installation of cpp-package headers (#13339)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new bff0fdc CMake: Enable installation of cpp-package headers (#13339)
bff0fdc is described below
commit bff0fdce13057ecd6d8b79c0c3d4dbefd5b85bd8
Author: Moritz Maxeiner
AuthorDate: Mon Dec 31 23:55:55 2018 +0100
CMake: Enable installation of cpp-package headers (#13339)
* Allow CMake based installation of cpp-package
* Add installation of missing nnvm headers
* Add documentation as to where public headers will be installed
---
CMakeLists.txt | 5 +
cpp-package/CMakeLists.txt | 2 ++
2 files changed, 7 insertions(+)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3b8bbd2..20ade5d 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -732,7 +732,12 @@ install(TARGETS ${MXNET_INSTALL_TARGETS}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
)
+# NOTE: Public headers will be installed into
${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_INCLUDEDIR}, see
+# https://cmake.org/cmake/help/v3.0/variable/CMAKE_INSTALL_PREFIX.html
+# https://cmake.org/cmake/help/v3.0/module/GNUInstallDirs.html
+
install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
+install(DIRECTORY 3rdparty/tvm/nnvm/include/ DESTINATION
${CMAKE_INSTALL_INCLUDEDIR})
if (INSTALL_EXAMPLES)
install(DIRECTORY example DESTINATION
${CMAKE_INSTALL_DATADIR}/${PROJECT_NAME})
endif()
diff --git a/cpp-package/CMakeLists.txt b/cpp-package/CMakeLists.txt
index f7fbc77..5d29772 100644
--- a/cpp-package/CMakeLists.txt
+++ b/cpp-package/CMakeLists.txt
@@ -20,4 +20,6 @@ if(USE_CPP_PACKAGE)
add_subdirectory(example)
endif()
+ install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
+
endif()
[incubator-mxnet] branch master updated: Add timeout/retry logic to docker cache download (#13573)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 63fe849 Add timeout/retry logic to docker cache download (#13573)
63fe849 is described below
commit 63fe84992360c421a5f95fe8adb304afcccdd10e
Author: Jose Luis Contreras
AuthorDate: Fri Dec 21 16:53:26 2018 +0100
Add timeout/retry logic to docker cache download (#13573)
* Added timeout/retry (linear backoff) to docker cache download
* Units changed, as time.sleep takes seconds as argument
* Improved error handling
* Using retry decorator
* Added retry decorator to _login_dockerhub method
* Fixed wrong import
---
ci/docker_cache.py | 51 +--
1 file changed, 25 insertions(+), 26 deletions(-)
diff --git a/ci/docker_cache.py b/ci/docker_cache.py
index fe1882a..70dd51c 100755
--- a/ci/docker_cache.py
+++ b/ci/docker_cache.py
@@ -30,12 +30,15 @@ import argparse
import sys
import subprocess
import json
-import time
from typing import *
import build as build_util
+from util import retry
DOCKERHUB_LOGIN_NUM_RETRIES = 5
DOCKERHUB_RETRY_SECONDS = 5
+DOCKER_CACHE_NUM_RETRIES = 3
+DOCKER_CACHE_TIMEOUT_MINS = 5
+
def build_save_containers(platforms, registry, load_cache) -> int:
"""
@@ -107,6 +110,8 @@ def _upload_image(registry, docker_tag, image_id) -> None:
subprocess.check_call(push_cmd)
+@retry(target_exception=subprocess.CalledProcessError,
tries=DOCKERHUB_LOGIN_NUM_RETRIES,
+ delay_s=DOCKERHUB_RETRY_SECONDS)
def _login_dockerhub():
"""
Login to the Docker Hub account
@@ -114,30 +119,19 @@ def _login_dockerhub():
"""
dockerhub_credentials = _get_dockerhub_credentials()
-for i in range(DOCKERHUB_LOGIN_NUM_RETRIES):
-logging.info('Logging in to DockerHub')
-# We use password-stdin instead of --password to avoid leaking
passwords in case of an error.
-# This method will produce the following output:
-# > WARNING! Your password will be stored unencrypted in
/home/jenkins_slave/.docker/config.json.
-# > Configure a credential helper to remove this warning. See
-# >
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
-# Since we consider the restricted slaves a secure environment, that's
fine. Also, using this will require
-# third party applications which would need a review first as well.
-p = subprocess.run(['docker', 'login', '--username',
dockerhub_credentials['username'], '--password-stdin'],
- stdout=subprocess.PIPE,
input=str.encode(dockerhub_credentials['password']))
-logging.info(p.stdout)
-if p.returncode != 0:
-logging.error('Error logging in to DockerHub')
-logging.error(p.stderr)
-
-# Linear backoff
-time.sleep(1000 * DOCKERHUB_RETRY_SECONDS * (i + 1))
-else:
-logging.info('Successfully logged in to DockerHub')
-break
-else:
-logging.error('DockerHub login not possible after %d retries,
aborting', DOCKERHUB_LOGIN_NUM_RETRIES)
-raise Exception('Unable to log in to DockerHub')
+logging.info('Logging in to DockerHub')
+# We use password-stdin instead of --password to avoid leaking passwords
in case of an error.
+# This method will produce the following output:
+# > WARNING! Your password will be stored unencrypted in
/home/jenkins_slave/.docker/config.json.
+# > Configure a credential helper to remove this warning. See
+# >
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
+# Since we consider the restricted slaves a secure environment, that's
fine. Also, using this will require
+# third party applications which would need a review first as well.
+p = subprocess.run(['docker', 'login', '--username',
dockerhub_credentials['username'], '--password-stdin'],
+ stdout=subprocess.PIPE,
input=str.encode(dockerhub_credentials['password']))
+logging.info(p.stdout)
+logging.info('Successfully logged in to DockerHub')
+
def _logout_dockerhub():
"""
@@ -149,6 +143,8 @@ def _logout_dockerhub():
logging.info('Successfully logged out of DockerHub')
+@retry(target_exception=subprocess.TimeoutExpired,
tries=DOCKER_CACHE_NUM_RETRIES,
+ delay_s=DOCKERHUB_RETRY_SECONDS)
def load_docker_cache(registry, docker_tag) -> None:
"""
Load the precompiled docker cache from the registry
@@ -163,7 +159,10 @@ def load_docker_cache(registry, docker_tag) -> None:
logging.info('Loading Docker cache for %s from %s', docker_tag, regist
[incubator-mxnet] branch master updated: [MXNET-1252][1 of 2] Decouple NNVM to ONNX from NNVM to TenosrRT conversion (#13659)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new f85b17b [MXNET-1252][1 of 2] Decouple NNVM to ONNX from NNVM to
TenosrRT conversion (#13659)
f85b17b is described below
commit f85b17b8dd647f8d286ebc51cd2e582b2a829d41
Author: Haohuan Wang
AuthorDate: Wed Dec 19 10:39:01 2018 -0800
[MXNET-1252][1 of 2] Decouple NNVM to ONNX from NNVM to TenosrRT conversion
(#13659)
---
src/executor/tensorrt_pass.cc | 8 +++
src/operator/contrib/nnvm_to_onnx-inl.h | 42 +
src/operator/contrib/nnvm_to_onnx.cc| 34 +-
src/operator/contrib/tensorrt-inl.h | 38 ++---
src/operator/contrib/tensorrt.cc| 28 ++
src/operator/contrib/tensorrt.cu| 2 +-
6 files changed, 76 insertions(+), 76 deletions(-)
diff --git a/src/executor/tensorrt_pass.cc b/src/executor/tensorrt_pass.cc
index b5fc8d1..d26704c 100644
--- a/src/executor/tensorrt_pass.cc
+++ b/src/executor/tensorrt_pass.cc
@@ -324,10 +324,10 @@ nnvm::NodePtr ConvertNnvmGraphToOnnx(const nnvm::Graph ,
std::unordered_map*
const params_map) {
auto p = nnvm::Node::Create();
p->attrs.op = nnvm::Op::Get("_trt_op");
- op::TRTParam trt_param = op::nnvm_to_onnx::ConvertNnvmGraphToOnnx(g,
params_map);
- p->attrs.dict["serialized_output_map"] = trt_param.serialized_output_map;
- p->attrs.dict["serialized_input_map"] = trt_param.serialized_input_map;
- p->attrs.dict["serialized_onnx_graph"] = trt_param.serialized_onnx_graph;
+ op::ONNXParam onnx_param = op::nnvm_to_onnx::ConvertNnvmGraphToOnnx(g,
params_map);
+ p->attrs.dict["serialized_output_map"] = onnx_param.serialized_output_map;
+ p->attrs.dict["serialized_input_map"] = onnx_param.serialized_input_map;
+ p->attrs.dict["serialized_onnx_graph"] = onnx_param.serialized_onnx_graph;
if (p->op()->attr_parser != nullptr) {
p->op()->attr_parser(&(p->attrs));
}
diff --git a/src/operator/contrib/nnvm_to_onnx-inl.h
b/src/operator/contrib/nnvm_to_onnx-inl.h
index 58f88b0..011ffe6 100644
--- a/src/operator/contrib/nnvm_to_onnx-inl.h
+++ b/src/operator/contrib/nnvm_to_onnx-inl.h
@@ -37,7 +37,6 @@
#include
#include
-#include
#include
#include
@@ -49,13 +48,48 @@
#include
#include
-#include "./tensorrt-inl.h"
#include "../operator_common.h"
#include "../../common/utils.h"
#include "../../common/serialization.h"
namespace mxnet {
namespace op {
+
+namespace nnvm_to_onnx {
+enum class TypeIO { Inputs = 0, Outputs = 1 };
+using NameToIdx_t = std::map;
+using InferenceTuple_t = std::tuple;
+using InferenceMap_t = std::map;
+} // namespace nnvm_to_onnx
+
+struct ONNXParam : public dmlc::Parameter {
+ std::string serialized_onnx_graph;
+ std::string serialized_input_map;
+ std::string serialized_output_map;
+ nnvm_to_onnx::NameToIdx_t input_map;
+ nnvm_to_onnx::InferenceMap_t output_map;
+ ::onnx::ModelProto onnx_pb_graph;
+
+ ONNXParam() {}
+
+ ONNXParam(const ::onnx::ModelProto& onnx_graph,
+ const nnvm_to_onnx::InferenceMap_t& input_map,
+ const nnvm_to_onnx::NameToIdx_t& output_map) {
+common::Serialize(input_map, _input_map);
+common::Serialize(output_map, _output_map);
+onnx_graph.SerializeToString(_onnx_graph);
+ }
+
+DMLC_DECLARE_PARAMETER(ONNXParam) {
+DMLC_DECLARE_FIELD(serialized_onnx_graph)
+.describe("Serialized ONNX graph");
+DMLC_DECLARE_FIELD(serialized_input_map)
+.describe("Map from inputs to topological order as input.");
+DMLC_DECLARE_FIELD(serialized_output_map)
+.describe("Map from outputs to order in g.outputs.");
+ }
+};
+
namespace nnvm_to_onnx {
using namespace nnvm;
@@ -76,7 +110,7 @@ void ConvertConstant(GraphProto* const graph_proto,
const std::string& node_name,
std::unordered_map* const shared_buffer);
-void ConvertOutput(op::tensorrt::InferenceMap_t* const trt_output_map,
+void ConvertOutput(op::nnvm_to_onnx::InferenceMap_t* const trt_output_map,
GraphProto* const graph_proto,
const std::unordered_map::iterator&
out_iter,
const std::string& node_name,
@@ -133,7 +167,7 @@ void ConvertElementwiseAdd(NodeProto *node_proto,
const nnvm::IndexedGraph ,
const array_view );
-TRTParam ConvertNnvmGraphToOnnx(
+ONNXParam ConvertNnvmGraphToOnnx(
const nnvm::Graph ,
std::unordered_map *const shared_buffer);
diff --git a/src/operator/contrib/nnvm_to_onnx.cc
b/src/operator/cont
[incubator-mxnet] branch revert-13532-jetson_fix created (now 58e9d01)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch revert-13532-jetson_fix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at 58e9d01 Revert "Fix Jetson compilation (#13532)" No new revisions were added by this update.
[incubator-mxnet] branch master updated (aa240cb -> 48e25c4)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from aa240cb add build fix for Scala/Java build (#13655) add 48e25c4 Fix Jetson compilation (#13532) No new revisions were added by this update. Summary of changes: ci/docker/Dockerfile.build.jetson | 1 + ci/jenkins/Jenkinsfile_edge | 2 +- make/crosscompile.jetson.mk | 4 ++-- 3 files changed, 4 insertions(+), 3 deletions(-)
[incubator-mxnet] branch master updated: Fix incorrect delete in MXExecutorReshape exception handling (#13376)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 67bbb49 Fix incorrect delete in MXExecutorReshape exception handling
(#13376)
67bbb49 is described below
commit 67bbb4906400379d6ffb6d4d086ab41aa31d289d
Author: Taliesin Beynon
AuthorDate: Sat Dec 15 01:36:39 2018 +0200
Fix incorrect delete in MXExecutorReshape exception handling (#13376)
* Fix bad delete.
Delete the pointed-to handle on cleanup, not the location of the handle
itself. Also don't delete it if we didn't set it in the first place.
* Remove unusued 'exec' var from MXExecutorBindEX.
---
src/c_api/c_api_executor.cc | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/src/c_api/c_api_executor.cc b/src/c_api/c_api_executor.cc
index 1f936b1..e2e53c7 100644
--- a/src/c_api/c_api_executor.cc
+++ b/src/c_api/c_api_executor.cc
@@ -148,8 +148,6 @@ int MXExecutorBindEX(SymbolHandle symbol_handle,
NDArrayHandle *aux_states,
ExecutorHandle shared_exec,
ExecutorHandle *out) {
- Executor* exec = nullptr;
-
API_BEGIN();
nnvm::Symbol *symb = static_cast(symbol_handle);
Context ctx = Context::Create(static_cast(dev_type),
dev_id);
@@ -181,7 +179,7 @@ int MXExecutorBindEX(SymbolHandle symbol_handle,
*out = Executor::Bind(*symb, ctx, ctx_map, in_args_vec,
arg_grad_vec, grad_req_vec, aux_states_vec,
reinterpret_cast(shared_exec));
- API_END_HANDLE_ERROR(delete exec);
+ API_END();
}
/*!
@@ -558,8 +556,11 @@ int MXExecutorReshape(int partial_shaping,
NDArrayHandle** aux_states,
ExecutorHandle shared_exec,
ExecutorHandle *out) {
+ Executor* new_exec = nullptr;
+
MXAPIThreadLocalEntry *ret = MXAPIThreadLocalStore::Get();
API_BEGIN();
+ *out = nullptr; // ensure we can know whether to free executor on early
abort
// create shape map for in_args and aux_states
std::unordered_map kwargs(num_provided_arg_shapes);
for (mx_uint i = 0; i < num_provided_arg_shapes; ++i) {
@@ -581,8 +582,9 @@ int MXExecutorReshape(int partial_shaping,
std::vector aux_state_vec;
Executor* exec = static_cast(shared_exec);
- *out = exec->Reshape(partial_shaping, allow_up_sizing, ctx, ctx_map, kwargs,
+ new_exec = exec->Reshape(partial_shaping, allow_up_sizing, ctx, ctx_map,
kwargs,
_arg_vec, _grad_vec, _state_vec);
+ *out = new_exec;
ret->ret_handles.clear();
ret->ret_handles.reserve(in_arg_vec.size()+arg_grad_vec.size()+aux_state_vec.size());
@@ -623,7 +625,7 @@ int MXExecutorReshape(int partial_shaping,
*aux_states = &(ret->ret_handles[nd_idx]);
nd_idx = ret->ret_handles.size();
}
- API_END_HANDLE_ERROR(delete out);
+ API_END_HANDLE_ERROR(delete new_exec);
}
int MXExecutorGetOptimizedSymbol(ExecutorHandle handle,
[incubator-mxnet] branch master updated: Be more explicit about the exit status of the container (#13425)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new d6e42d8 Be more explicit about the exit status of the container
(#13425)
d6e42d8 is described below
commit d6e42d8c13d717e9e56027713b3af8698ef36554
Author: Pedro Larroy <928489+lar...@users.noreply.github.com>
AuthorDate: Wed Nov 28 00:07:16 2018 +0100
Be more explicit about the exit status of the container (#13425)
---
ci/build.py | 14 +++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/ci/build.py b/ci/build.py
index 628f49e..acf277d 100755
--- a/ci/build.py
+++ b/ci/build.py
@@ -326,6 +326,10 @@ def container_run(platform: str,
wait_result = container.wait(timeout=container_wait_s)
logging.info("Container exit status: %s", wait_result)
ret = wait_result.get('StatusCode', 200)
+if ret != 0:
+logging.error("Container exited with an error ")
+else:
+logging.info("Container exited with success ")
except Exception as e:
logging.exception(e)
ret = 150
@@ -384,11 +388,15 @@ def script_name() -> str:
""":returns: script name with leading paths removed"""
return os.path.split(sys.argv[0])[1]
-
-def main() -> int:
+def config_logging():
+import time
logging.getLogger().setLevel(logging.INFO)
logging.getLogger("requests").setLevel(logging.WARNING)
-logging.basicConfig(format='{}: %(asctime)-15s
%(message)s'.format(script_name()))
+logging.basicConfig(format='{}: %(asctime)sZ %(levelname)s
%(message)s'.format(script_name()))
+logging.Formatter.converter = time.gmtime
+
+def main() -> int:
+config_logging()
logging.info("MXNet container based build tool.")
log_environment()
[incubator-mxnet] branch master updated: Added proper default value in cpp-package for optional (#13415)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 48bfc8d Added proper default value in cpp-package for optional
(#13415)
48bfc8d is described below
commit 48bfc8dd834557767ec77bb184e9ee0d147d4cc5
Author: Przemyslaw Tredak
AuthorDate: Mon Nov 26 19:23:58 2018 -0800
Added proper default value in cpp-package for optional (#13415)
---
cpp-package/scripts/OpWrapperGenerator.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/cpp-package/scripts/OpWrapperGenerator.py
b/cpp-package/scripts/OpWrapperGenerator.py
index 1b5f8b5..c26c370 100644
--- a/cpp-package/scripts/OpWrapperGenerator.py
+++ b/cpp-package/scripts/OpWrapperGenerator.py
@@ -138,6 +138,8 @@ class Arg:
self.defaultString = 'Shape(' + self.defaultString[1:-1] + ")"
elif self.type == 'dmlc::optional':
self.defaultString = self.type + '(' + self.defaultString + ')'
+elif self.type == 'dmlc::optional':
+self.defaultString = self.type + '(' + self.defaultString + ')'
elif typeString.startswith('caffe-layer-parameter'):
self.defaultString = 'textToCaffeLayerParameter(' +
self.MakeCString(self.defaultString) + ')'
hasCaffe = True
[incubator-mxnet] branch master updated (14ae5d4 -> f1de8e5)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 14ae5d4 Fix repeated typo in mxnet_op.h (#13406) add f1de8e5 Use dynamic omp schedule for sparse dot with large matrix (#13398) No new revisions were added by this update. Summary of changes: src/operator/mxnet_op.h | 31 +++ src/operator/tensor/dot-inl.h | 23 +++ 2 files changed, 50 insertions(+), 4 deletions(-)
[incubator-mxnet] branch master updated: Correct shapes of images in cifar10 and cifar100 (#13348)
This is an automated email from the ASF dual-hosted git repository. kellen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new afa4c3a Correct shapes of images in cifar10 and cifar100 (#13348) afa4c3a is described below commit afa4c3a85f0684e0dbdb097c9ffb4b5d14f9ef9d Author: Masaki Samejima AuthorDate: Mon Nov 26 13:14:29 2018 +0900 Correct shapes of images in cifar10 and cifar100 (#13348) * Correct shapes of images in cifar10 and cifar100 cifar10 and cifar100 have 3 channels * Retrigger build --- python/mxnet/gluon/data/vision/datasets.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/mxnet/gluon/data/vision/datasets.py b/python/mxnet/gluon/data/vision/datasets.py index 267aa1e..12ef7e1 100644 --- a/python/mxnet/gluon/data/vision/datasets.py +++ b/python/mxnet/gluon/data/vision/datasets.py @@ -128,7 +128,7 @@ class FashionMNIST(MNIST): class CIFAR10(dataset._DownloadedDataset): """CIFAR10 image classification dataset from https://www.cs.toronto.edu/~kriz/cifar.html -Each sample is an image (in 3D NDArray) with shape (32, 32, 1). +Each sample is an image (in 3D NDArray) with shape (32, 32, 3). Parameters -- @@ -190,7 +190,7 @@ class CIFAR10(dataset._DownloadedDataset): class CIFAR100(CIFAR10): """CIFAR100 image classification dataset from https://www.cs.toronto.edu/~kriz/cifar.html -Each sample is an image (in 3D NDArray) with shape (32, 32, 1). +Each sample is an image (in 3D NDArray) with shape (32, 32, 3). Parameters --
[incubator-mxnet] branch master updated: Replace mxnetci dockcross with public dockcross due to missing image (#13402)
This is an automated email from the ASF dual-hosted git repository.
kellen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new b7b0d3f Replace mxnetci dockcross with public dockcross due to
missing image (#13402)
b7b0d3f is described below
commit b7b0d3f52b7c55f02c76aa5cf969ce88419b267e
Author: Marco de Abreu
AuthorDate: Mon Nov 26 02:37:00 2018 +0100
Replace mxnetci dockcross with public dockcross due to missing image
(#13402)
* Replace mxnetci dockcross with public dockcross due to missing image
* Remove source lists change
* Disable Jetson
* Move to mxnetcipinned
---
Jenkinsfile | 20 ++--
ci/docker/Dockerfile.build.android_armv7 | 2 +-
ci/docker/Dockerfile.build.android_armv8 | 2 +-
ci/docker/Dockerfile.build.armv6 | 2 +-
ci/docker/Dockerfile.build.armv7 | 2 +-
ci/docker/Dockerfile.build.armv8 | 6 +++---
ci/docker/Dockerfile.build.jetson| 6 +++---
ci/jenkins/Jenkinsfile_edge | 2 +-
8 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 79392b4..015ca81 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -428,16 +428,16 @@ core_logic: {
}
}
},
-'NVidia Jetson / ARMv8':{
- node(NODE_LINUX_CPU) {
-ws('workspace/build-jetson-armv8') {
- timeout(time: max_time, unit: 'MINUTES') {
-utils.init_git()
-utils.docker_run('jetson', 'build_jetson', false)
- }
-}
- }
-},
+//'NVidia Jetson / ARMv8':{
+// node(NODE_LINUX_CPU) {
+//ws('workspace/build-jetson-armv8') {
+// timeout(time: max_time, unit: 'MINUTES') {
+//utils.init_git()
+//utils.docker_run('jetson', 'build_jetson', false)
+// }
+//}
+// }
+//},
'ARMv7':{
node(NODE_LINUX_CPU) {
ws('workspace/build-ARMv7') {
diff --git a/ci/docker/Dockerfile.build.android_armv7
b/ci/docker/Dockerfile.build.android_armv7
index 799e29c..c601fc5 100644
--- a/ci/docker/Dockerfile.build.android_armv7
+++ b/ci/docker/Dockerfile.build.android_armv7
@@ -18,7 +18,7 @@
#
# Dockerfile to build MXNet for Android ARMv7
-FROM mxnetci/dockcross-linux-base:08212018
+FROM mxnetcipinned/dockcross-base:11262018
MAINTAINER Pedro Larroy "pllar...@amazon.com"
# The cross-compiling emulator
diff --git a/ci/docker/Dockerfile.build.android_armv8
b/ci/docker/Dockerfile.build.android_armv8
index 2c2c71c..60376b8 100644
--- a/ci/docker/Dockerfile.build.android_armv8
+++ b/ci/docker/Dockerfile.build.android_armv8
@@ -18,7 +18,7 @@
#
# Dockerfile to build MXNet for Android ARM64/ARMv8
-FROM mxnetci/dockcross-linux-base:08212018
+FROM mxnetcipinned/dockcross-base:11262018
MAINTAINER Pedro Larroy "pllar...@amazon.com"
RUN apt-get update && apt-get install -y \
diff --git a/ci/docker/Dockerfile.build.armv6 b/ci/docker/Dockerfile.build.armv6
index 78071fa..6f16d8c 100644
--- a/ci/docker/Dockerfile.build.armv6
+++ b/ci/docker/Dockerfile.build.armv6
@@ -18,7 +18,7 @@
#
# Dockerfile to build MXNet for ARMv6
-FROM mxnetci/dockcross-linux-armv6:08212018
+FROM mxnetcipinned/dockcross-linux-armv6:11262018
ENV ARCH armv6l
ENV HOSTCC gcc
diff --git a/ci/docker/Dockerfile.build.armv7 b/ci/docker/Dockerfile.build.armv7
index 9a23a5d..5f02234 100644
--- a/ci/docker/Dockerfile.build.armv7
+++ b/ci/docker/Dockerfile.build.armv7
@@ -18,7 +18,7 @@
#
# Dockerfile to build MXNet for Android ARMv7
-FROM mxnetci/dockcross-linux-armv7:09182018
+FROM mxnetcipinned/dockcross-linux-armv7:11262018
ENV ARCH armv7l
ENV HOSTCC gcc
diff --git a/ci/docker/Dockerfile.build.armv8 b/ci/docker/Dockerfile.build.armv8
index 46cc229..27bd425 100644
--- a/ci/docker/Dockerfile.build.armv8
+++ b/ci/docker/Dockerfile.build.armv8
@@ -18,7 +18,7 @@
#
# Dockerfile to build MXNet for ARM64/ARMv8
-FROM mxnetci/dockcross-linux-arm64:08212018
+FROM mxnetcipinned/dockcross-linux-arm64:11262018
ENV ARCH aarch64
ENV HOSTCC gcc
@@ -27,8 +27,8 @@ ENV TARGET ARMV8
WORKDIR /work/deps
# gh issue #11567 https://github.com/apache/incubator-mxnet/issues/11567
-RUN sed -i '\#deb http://cdn-fastly.deb.debian.org/debian-security
jessie/updates main#d' /etc/apt/sources.list
-RUN sed -i 's/cdn-fastly.//' /etc/apt/sources.list
+#RUN sed -i '\#deb http://cdn-fastly.deb.debian.org/debian-security
jessie/updates main#d' /etc/apt/sources.list
+#RUN sed -i 's/cdn-fastly.//' /etc/apt/sources.list
COPY install/ubuntu_arm.sh /work/
RUN /work/ubuntu_arm.sh
diff --git a/ci/docker/Dockerfile.build.jetson
b/ci/docker/Dockerfile.build.jetson
index 15518cd..d128ebc 100644
--- a/ci/docker/Dockerfile.build.jetson
+++ b/ci/docker/Dockerfile.build.jetson
@@ -22,15 +22,15 @@
FROM nvidia/cuda:9.0
