[incubator-mxnet] branch v1.5.x updated: fix lstm layer with projection save params (#17266) (#17286)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.5.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.5.x by this push:
new 3b2bd99 fix lstm layer with projection save params (#17266) (#17286)
3b2bd99 is described below
commit 3b2bd999ea5293dfacd5008e459ea3c7271b9faa
Author: Frank Liu
AuthorDate: Mon Feb 3 13:01:50 2020 -0800
fix lstm layer with projection save params (#17266) (#17286)
Co-authored-by: Sheng Zha
---
python/mxnet/gluon/rnn/rnn_layer.py | 2 +-
tests/python/gpu/test_gluon_gpu.py | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/python/mxnet/gluon/rnn/rnn_layer.py
b/python/mxnet/gluon/rnn/rnn_layer.py
index b3cc596..11d4581 100644
--- a/python/mxnet/gluon/rnn/rnn_layer.py
+++ b/python/mxnet/gluon/rnn/rnn_layer.py
@@ -124,7 +124,7 @@ class _RNNLayer(HybridBlock):
def _collect_params_with_prefix(self, prefix=''):
if prefix:
prefix += '.'
-pattern = re.compile(r'(l|r)(\d)_(i2h|h2h)_(weight|bias)\Z')
+pattern = re.compile(r'(l|r)(\d)_(i2h|h2h|h2r)_(weight|bias)\Z')
def convert_key(m, bidirectional): # for compatibility with old
parameter format
d, l, g, t = [m.group(i) for i in range(1, 5)]
if bidirectional:
diff --git a/tests/python/gpu/test_gluon_gpu.py
b/tests/python/gpu/test_gluon_gpu.py
index fc65029..d6070d6 100644
--- a/tests/python/gpu/test_gluon_gpu.py
+++ b/tests/python/gpu/test_gluon_gpu.py
@@ -137,6 +137,8 @@ def test_lstmp():
check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, bidirectional=True,
dropout=0.5, projection_size=5),
mx.nd.ones((8, 3, 20)),
[mx.nd.ones((4, 3, 5)), mx.nd.ones((4, 3, 10))],
run_only=True, ctx=ctx)
+lstm_layer.save_parameters('gpu_tmp.params')
+lstm_layer.load_parameters('gpu_tmp.params')
@with_seed()
[incubator-mxnet] branch master updated (05af5c4 -> 634f95e)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 05af5c4 [BUGFIX] Fix race condition in kvstore.pushpull (#17007) add 634f95e swap wget to curl in Scala scripts (#17041) No new revisions were added by this update. Summary of changes: scala-package/core/scripts/get_cifar_data.sh | 2 +- scala-package/core/scripts/get_mnist_data.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
[incubator-mxnet] branch master updated (05af5c4 -> 634f95e)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 05af5c4 [BUGFIX] Fix race condition in kvstore.pushpull (#17007) add 634f95e swap wget to curl in Scala scripts (#17041) No new revisions were added by this update. Summary of changes: scala-package/core/scripts/get_cifar_data.sh | 2 +- scala-package/core/scripts/get_mnist_data.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
[incubator-mxnet] branch master updated (10903e8 -> 8dd7051)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 10903e8 Fix ndarray indexing bug (#16895) add 8dd7051 Downgrade the cublas version for nightly gpu (#16985) No new revisions were added by this update. Summary of changes: ci/docker/Dockerfile.build.ubuntu_base_gpu| 4 ci/docker/Dockerfile.build.ubuntu_nightly_gpu | 4 2 files changed, 8 insertions(+)
[incubator-mxnet] branch master updated (10903e8 -> 8dd7051)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 10903e8 Fix ndarray indexing bug (#16895) add 8dd7051 Downgrade the cublas version for nightly gpu (#16985) No new revisions were added by this update. Summary of changes: ci/docker/Dockerfile.build.ubuntu_base_gpu| 4 ci/docker/Dockerfile.build.ubuntu_nightly_gpu | 4 2 files changed, 8 insertions(+)
[incubator-mxnet] branch master updated (7713a43 -> 32a9baa)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7713a43 Update TensorRT tutorial to build-from-source. (#14860) add 32a9baa Adding user guidelines for using MXNet built with Large Tensor Support (#16894) No new revisions were added by this update. Summary of changes: .../src/pages/api/faq/large_tensor_support.md | 287 + 1 file changed, 287 insertions(+) create mode 100644 docs/static_site/src/pages/api/faq/large_tensor_support.md
[incubator-mxnet] branch master updated (7713a43 -> 32a9baa)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7713a43 Update TensorRT tutorial to build-from-source. (#14860) add 32a9baa Adding user guidelines for using MXNet built with Large Tensor Support (#16894) No new revisions were added by this update. Summary of changes: .../src/pages/api/faq/large_tensor_support.md | 287 + 1 file changed, 287 insertions(+) create mode 100644 docs/static_site/src/pages/api/faq/large_tensor_support.md
[incubator-mxnet] branch master updated (7713a43 -> 32a9baa)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7713a43 Update TensorRT tutorial to build-from-source. (#14860) add 32a9baa Adding user guidelines for using MXNet built with Large Tensor Support (#16894) No new revisions were added by this update. Summary of changes: .../src/pages/api/faq/large_tensor_support.md | 287 + 1 file changed, 287 insertions(+) create mode 100644 docs/static_site/src/pages/api/faq/large_tensor_support.md
[incubator-mxnet] branch master updated (7713a43 -> 32a9baa)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7713a43 Update TensorRT tutorial to build-from-source. (#14860) add 32a9baa Adding user guidelines for using MXNet built with Large Tensor Support (#16894) No new revisions were added by this update. Summary of changes: .../src/pages/api/faq/large_tensor_support.md | 287 + 1 file changed, 287 insertions(+) create mode 100644 docs/static_site/src/pages/api/faq/large_tensor_support.md
[incubator-mxnet] branch v1.6.x updated: updating MXNet version to 1.6.0 in base.h for C APIs (#16906)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch v1.6.x in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/v1.6.x by this push: new 121739a updating MXNet version to 1.6.0 in base.h for C APIs (#16906) 121739a is described below commit 121739a1656d42970c747956f995b52f9f6d31c2 Author: Rohit Kumar Srivastava AuthorDate: Tue Nov 26 13:04:00 2019 -0800 updating MXNet version to 1.6.0 in base.h for C APIs (#16906) --- include/mxnet/base.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/mxnet/base.h b/include/mxnet/base.h index 02dd204..90b36ab 100644 --- a/include/mxnet/base.h +++ b/include/mxnet/base.h @@ -73,7 +73,7 @@ /*! \brief major version */ #define MXNET_MAJOR 1 /*! \brief minor version */ -#define MXNET_MINOR 5 +#define MXNET_MINOR 6 /*! \brief patch version */ #define MXNET_PATCH 0 /*! \brief mxnet version */
[incubator-mxnet] branch revert-16304-cifix updated (c8a1bd7 -> 4aa5c78)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a change to branch revert-16304-cifix
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git.
discard c8a1bd7 asd
discard c720335 upgrade pytest
discard 28557c1 add ubuntu onnx fix
discard 463c55d Revert "add mkl installation temp fix (#16304)"
add a8181dd [MXNET-1430] julia: implement context.gpu_memory_info (#16324)
add 9dc0ab8 global numpy shape flag (#16335)
add cfe9e50 Skipping installing nightly test (#16418)
add a2018ba cuDNN non-persistant bidirectional RNN dgrad sync fix (#16391)
add 56e1bef Adds PyPI CD Pipeline (#16190)
add 88521ff upgrade the pytest version (#16429)
add 6ce323f [DOC] fix installation selector wrong history (#16381)
add 1ab4c95 New ops for RCNN + old ops improvements for RCNN (#16215)
add e484f72 Beta build (#16411)
add 243ade9 [WIP] Improving Python Docs API (#16392)
add 4aa5c78 Revert "add mkl installation temp fix (#16304)"
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c8a1bd7)
\
N -- N -- N refs/heads/revert-16304-cifix (4aa5c78)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
cd/Jenkinsfile_cd_pipeline | 4 +
cd/Jenkinsfile_utils.groovy| 73
cd/README.md | 8 +
cd/python/pypi/Jenkins_pipeline.groovy | 78
cd/python/pypi/README.md | 43 ++
cd/python/pypi/pypi_package.sh | 60 +++
cd/python/pypi/pypi_publish.py | 87
ci/docker/runtime_functions.sh | 48 ++-
ci/jenkins/Jenkins_steps.groovy| 43 +-
...enkinsfile_website => Jenkinsfile_website_beta} | 23 +-
docs/python_docs/README.md | 116 +-
docs/python_docs/_static/autodoc.js| 49 +++
docs/python_docs/_static/mxnet.css | 7 +
docs/python_docs/environment.yml | 1 +
docs/python_docs/python/Makefile | 3 +-
docs/python_docs/python/README.md | 130 --
docs/python_docs/python/api/advanced/index.rst | 74
.../python/api/advanced/mxnet.executor_manager.rst | 38 --
.../python/api/advanced/mxnet.kvstore_server.rst | 36 --
.../python/api/advanced/mxnet.test_utils.rst | 91
.../index.rst} | 10 +-
.../autograd/index.rst}| 13 +-
docs/python_docs/python/api/contrib/index.rst | 88
.../io/index.rst} | 10 +-
.../ndarray/index.rst} | 10 +-
.../mxnet.random.rst => contrib/onnx/index.rst}| 12 +-
.../quantization/index.rst}| 13 +-
.../symbol/index.rst} | 10 +-
.../tensorboard/index.rst} | 13 +-
.../tensorrt/index.rst}| 13 +-
.../mxnet.random.rst => contrib/text/index.rst}| 11 +-
.../python_docs/python/api/gluon-related/index.rst | 111 -
.../python/api/gluon-related/mxnet.autograd.rst| 38 --
.../python/api/gluon-related/mxnet.image.rst | 99 -
.../python/api/gluon-related/mxnet.initializer.rst | 58 ---
.../python/api/gluon-related/mxnet.io.rst | 52 ---
.../api/gluon-related/mxnet.kvstore.KVStore.rst| 61 ---
.../api/gluon-related/mxnet.lr_scheduler.rst | 31 --
.../python/api/gluon-related/mxnet.metric.rst | 57 ---
.../python/api/gluon-related/mxnet.optimizer.rst | 55 ---
.../python/api/gluon-related/mxnet.profiler.rst| 54 ---
.../python/api/gluon-related/mxnet.recordio.rst| 43 --
.../mxnet.random.rst => gluon/block.rst} | 13 +-
.../{mxnet.gluon.Constant.rst => constant.rst} | 10 +-
.../{mxnet.gluon.contrib.rst => contrib/index.rst} | 17 +-
.../mxnet.random.rst => gluon/hybrid_block.rst}| 13 +-
docs/python_docs/python/api/gluon/index.rst| 45 +-
.../mxnet.rtc.rst => gluon/loss/index.rst} | 25 +-
.../index.rst} | 30 +-
.../python/api/gluon/mxnet.gluon.HybridBlock.rst | 40 --
.../python/api/gluo
[incubator-mxnet] branch revert-16304-cifix updated (c720335 -> c8a1bd7)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-16304-cifix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from c720335 upgrade pytest add c8a1bd7 asd No new revisions were added by this update. Summary of changes: ci/docker/install/ubuntu_onnx.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[incubator-mxnet] branch revert-16304-cifix updated (28557c1 -> c720335)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-16304-cifix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 28557c1 add ubuntu onnx fix add c720335 upgrade pytest No new revisions were added by this update. Summary of changes: ci/docker/install/ubuntu_onnx.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[incubator-mxnet] branch revert-16304-cifix updated (463c55d -> 28557c1)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-16304-cifix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 463c55d Revert "add mkl installation temp fix (#16304)" add 28557c1 add ubuntu onnx fix No new revisions were added by this update. Summary of changes: ci/docker/install/ubuntu_onnx.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[incubator-mxnet] branch revert-16304-cifix updated (e58b42f -> 463c55d)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-16304-cifix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. discard e58b42f Revert "add mkl installation temp fix (#16304)" add 8096421 Embedding gradient performance optimization on GPU (#16355) add 2c81a71 Change mailing list url in footer to point to instructions about how to subscribe instead (#16384) add 2127f75 Add instructions to report a security vulnerability (#16383) add 09285c8 Implements ldexp. (#15845) add 2df3282 Numpy Operators: Inner, Outer, vdot (#15846) add 295fc14 Numpy det and slogdet operators (#15861) add 4940ec0 Fix random op signature add df4125a update NEWS.md and README.md (#16385) add 0bace55 fix choice signature add ec766d5 add raise test for shape add d5666ed Round and sign straight-through-estimators C operators. (#16373) add 15ea40d Add boolean ndarray (#15940) add 1d0d1e6 Faster Transpose 2D (#16104) add 9ff644b Fix windows flakiness (#16415) add 463c55d Revert "add mkl installation temp fix (#16304)" This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (e58b42f) \ N -- N -- N refs/heads/revert-16304-cifix (463c55d) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: 3rdparty/mshadow/mshadow/base.h| 62 ++- CMakeLists.txt | 6 +- Makefile | 6 +- NEWS.md| 24 + README.md | 1 + ci/docker/runtime_functions.sh | 4 +- contrib/tvmop/__init__.py | 1 + contrib/tvmop/compile.py | 41 +- .../pycocotools => contrib/tvmop/core}/__init__.py | 2 +- contrib/tvmop/core/fromnumeric.py | 63 +++ contrib/tvmop/core/umath.py| 122 + contrib/tvmop/opdef.py | 6 +- contrib/tvmop/utils.py | 16 +- docs/static_site/src/_includes/footer.html | 3 +- docs/static_site/src/pages/api/faq/security.md | 17 + include/mxnet/tensor_blob.h| 1 + python/mxnet/initializer.py| 8 +- python/mxnet/ndarray/ndarray.py| 6 + python/mxnet/ndarray/numpy/_op.py | 399 +++- python/mxnet/ndarray/numpy/random.py | 22 +- python/mxnet/numpy/multiarray.py | 501 ++--- python/mxnet/numpy/random.py | 12 +- python/mxnet/numpy/utils.py| 7 +- python/mxnet/numpy_op_signature.py | 7 +- python/mxnet/symbol/numpy/_symbol.py | 426 -- python/mxnet/symbol/numpy/random.py| 18 +- python/mxnet/test_utils.py | 17 + src/ndarray/ndarray.cc | 2 +- src/ndarray/ndarray_function.cc| 9 + src/ndarray/ndarray_function.cu| 10 +- src/operator/contrib/boolean_mask.cc | 7 +- src/operator/contrib/boolean_mask.cu | 4 +- src/operator/contrib/stes_op.cc| 84 src/operator/contrib/stes_op.cu| 43 ++ src/operator/contrib/stes_op.h | 33 ++ src/operator/mshadow_op.h | 11 + src/operator/mxnet_op.h| 16 + src/operator/numpy/np_broadcast_reduce_op.h| 21 +- src/operator/numpy/np_broadcast_reduce_op_value.cc | 71 +++ src/operator/numpy/np_elemwise_broadcast_op.cc | 284 +++- src/operator/numpy/np_elemwise_broadcast_op.cu | 19 + src/operator/operator_tune.cc | 28 +- src/operator/random/sample_op.cc | 2 - .../tensor/elemwise_binary_broadcast_op_logic.cc | 6 - .../tensor/elemwise_binary_scalar_op_logic.cc | 6 - src/operator/tensor/elemwise_unary_op.h| 2 +- src/operator/tensor/indexing_op.cu | 233 ++ src/operator/tensor/init_op.h | 10 +- src/operator/
[incubator-mxnet] branch revert-16304-cifix updated (eff50d5 -> e58b42f)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-16304-cifix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. discard eff50d5 Revert "add mkl installation temp fix (#16304)" add 09ae7df remove redundant branch name (#16372) add 626fc32 Disable Pylint false error in numpy_op_signature (#16370) add 916fbf2 boolean_mask_assign operator for future boolean indexing (#16361) add e58b42f Revert "add mkl installation temp fix (#16304)" This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (eff50d5) \ N -- N -- N refs/heads/revert-16304-cifix (e58b42f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: ci/Jenkinsfile_utils.groovy | 4 +- python/mxnet/numpy_op_signature.py | 8 +- src/operator/numpy/np_boolean_mask_assign.cc | 270 +++ src/operator/numpy/np_boolean_mask_assign.cu | 229 +++ src/operator/numpy/np_broadcast_reduce_op.h | 9 + tests/python/unittest/test_numpy_op.py | 36 6 files changed, 550 insertions(+), 6 deletions(-) create mode 100644 src/operator/numpy/np_boolean_mask_assign.cc create mode 100644 src/operator/numpy/np_boolean_mask_assign.cu
[incubator-mxnet] branch revert-16304-cifix created (now eff50d5)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-16304-cifix in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at eff50d5 Revert "add mkl installation temp fix (#16304)" No new revisions were added by this update.
[incubator-mxnet] branch master updated (01ca278 -> 943bed5)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 01ca278 [Gluon] Support None argument in HybridBlock (#16280) add 943bed5 add mkl installation temp fix (#16304) No new revisions were added by this update. Summary of changes: ci/docker/install/ubuntu_mkl.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[incubator-mxnet] branch master updated (2d86c70 -> 196d1f4)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 2d86c70 Port ops from np branch (#16018) add 196d1f4 [MXNET-1399] multiclass-mcc metric enhancements (#14874) No new revisions were added by this update. Summary of changes: python/mxnet/metric.py | 6 +- tests/python/unittest/test_metric.py | 7 +++ 2 files changed, 12 insertions(+), 1 deletion(-)
[incubator-mxnet] branch master updated (c310763 -> 7bc3db8)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from c310763 update profiler tutorial (#15580) add 7bc3db8 Bump Scala version to 1.6 (#15660) No new revisions were added by this update. Summary of changes: .../core/src/main/scala/org/apache/mxnet/NDArray.scala | 2 +- .../src/main/scala/org/apache/mxnet/NDArrayCollector.scala | 12 ++-- scala-package/pom.xml| 2 +- 3 files changed, 8 insertions(+), 8 deletions(-)
[incubator-mxnet] branch master updated (9777717 -> 300cb69)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 917 website build for julia: fix path to be static (#15554) add 300cb69 some minor typos/clarifications (#15538) No new revisions were added by this update. Summary of changes: scala-package/README.md | 28 ++-- 1 file changed, 14 insertions(+), 14 deletions(-)
[incubator-mxnet] branch master updated: fix the bug on Scala Sparse (#15500)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 6191dd7 fix the bug on Scala Sparse (#15500)
6191dd7 is described below
commit 6191dd7083cac7574f8172a4a062bb9408cc7621
Author: Lanking
AuthorDate: Wed Jul 10 15:22:07 2019 -0700
fix the bug on Scala Sparse (#15500)
* fix the bug on sparse tensor
* update jenkins
---
ci/docker/runtime_functions.sh | 16
ci/jenkins/Jenkins_steps.groovy | 4 ++--
.../core/src/main/scala/org/apache/mxnet/NDArray.scala | 4 +++-
3 files changed, 13 insertions(+), 11 deletions(-)
diff --git a/ci/docker/runtime_functions.sh b/ci/docker/runtime_functions.sh
index 0fef2a0..424ab2c 100755
--- a/ci/docker/runtime_functions.sh
+++ b/ci/docker/runtime_functions.sh
@@ -952,13 +952,6 @@ unittest_ubuntu_python3_quantization_gpu() {
nosetests-3.4 $NOSE_COVERAGE_ARGUMENTS $NOSE_TIMER_ARGUMENTS --with-xunit
--xunit-file nosetests_quantization_gpu.xml --verbose
tests/python/quantization_gpu
}
-unittest_ubuntu_cpu_scala() {
-set -ex
-scala_prepare
-cd scala-package
-mvn -B integration-test
-}
-
unittest_centos7_cpu_scala() {
set -ex
cd /work/mxnet
@@ -1156,12 +1149,19 @@ integrationtest_ubuntu_cpu_dist_kvstore() {
../../tools/launch.py -n 3 --launcher local python test_server_profiling.py
}
+integrationtest_ubuntu_cpu_scala() {
+set -ex
+scala_prepare
+cd scala-package
+mvn -B verify -DskipTests=false
+}
+
integrationtest_ubuntu_gpu_scala() {
set -ex
scala_prepare
cd scala-package
export SCALA_TEST_ON_GPU=1
-mvn -B integration-test -DskipTests=false
+mvn -B verify -DskipTests=false
}
integrationtest_ubuntu_gpu_dist_kvstore() {
diff --git a/ci/jenkins/Jenkins_steps.groovy b/ci/jenkins/Jenkins_steps.groovy
index 668d2f7..4eb84b6 100644
--- a/ci/jenkins/Jenkins_steps.groovy
+++ b/ci/jenkins/Jenkins_steps.groovy
@@ -941,7 +941,7 @@ def test_unix_scala_cpu() {
ws('workspace/ut-scala-cpu') {
timeout(time: max_time, unit: 'MINUTES') {
utils.unpack_and_init('cpu', mx_lib, true)
-utils.docker_run('ubuntu_cpu', 'unittest_ubuntu_cpu_scala', false)
+utils.docker_run('ubuntu_cpu', 'integrationtest_ubuntu_cpu_scala',
false)
utils.publish_test_coverage()
}
}
@@ -955,7 +955,7 @@ def test_unix_scala_mkldnn_cpu(){
ws('workspace/ut-scala-mkldnn-cpu') {
timeout(time: max_time, unit: 'MINUTES') {
utils.unpack_and_init('mkldnn_cpu', mx_mkldnn_lib, true)
-utils.docker_run('ubuntu_cpu', 'unittest_ubuntu_cpu_scala', false)
+utils.docker_run('ubuntu_cpu', 'integrationtest_ubuntu_cpu_scala',
false)
utils.publish_test_coverage()
}
}
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 1b7b31b..717120b 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -956,7 +956,9 @@ class NDArray private[mxnet](private[mxnet] val handle:
NDArrayHandle,
DType(mxDtype.value)
}
- val sparseFormat: SparseFormat = {
+ // This is a optimization on the SparseFormat checking
+ // TODO: In some cases, the checking on Sparse is invalid (-1)
+ lazy val sparseFormat: SparseFormat = {
val mxSF = new RefInt
checkCall(_LIB.mxNDArrayGetStorageType(handle, mxSF))
SparseFormat(mxSF.value)
[incubator-mxnet] branch master updated: Add Sparse NDArray support for Scala (#15378)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 09c71bf Add Sparse NDArray support for Scala (#15378)
09c71bf is described below
commit 09c71bf3144b09a28b9d09d33703a3dcbf4ca9a5
Author: Lanking
AuthorDate: Mon Jul 8 11:48:30 2019 -0700
Add Sparse NDArray support for Scala (#15378)
* add Sparse Support
* add imperative invoke sparse support
* add retain method and comments
* add getData method
* add Sparse NDIter test
* remove debug line
---
.../src/main/scala/org/apache/mxnet/DType.scala| 17 +-
.../src/main/scala/org/apache/mxnet/Executor.scala | 9 +-
.../src/main/scala/org/apache/mxnet/LibInfo.scala | 27 ++-
.../src/main/scala/org/apache/mxnet/NDArray.scala | 65 ++-
.../main/scala/org/apache/mxnet/SparseFormat.scala | 25 +++
.../scala/org/apache/mxnet/SparseNDArray.scala | 196 +
.../test/scala/org/apache/mxnet/NDArraySuite.scala | 16 ++
.../org/apache/mxnet/SparseNDArraySuite.scala | 93 ++
.../main/native/org_apache_mxnet_native_c_api.cc | 75 +++-
.../main/native/org_apache_mxnet_native_c_api.h| 48 -
10 files changed, 543 insertions(+), 28 deletions(-)
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/DType.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/DType.scala
index f3a8e8e..1d5cc28 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/DType.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/DType.scala
@@ -24,26 +24,17 @@ object DType extends Enumeration {
val Float16 = Value(2, "float16")
val UInt8 = Value(3, "uint8")
val Int32 = Value(4, "int32")
+ val Int8 = Value(5, "int8")
+ val Int64 = Value(6, "int64")
val Unknown = Value(-1, "unknown")
private[mxnet] def numOfBytes(dtype: DType): Int = {
dtype match {
- case DType.UInt8 => 1
+ case DType.UInt8 | DType.Int8 => 1
case DType.Int32 => 4
case DType.Float16 => 2
case DType.Float32 => 4
- case DType.Float64 => 8
+ case DType.Float64 | DType.Int64 => 8
case DType.Unknown => 0
}
}
- private[mxnet] def getType(dtypeStr: String): DType = {
-dtypeStr match {
- case "UInt8" => DType.UInt8
- case "Int32" => DType.Int32
- case "Float16" => DType.Float16
- case "Float32" => DType.Float32
- case "Float64" => DType.Float64
- case _ => throw new IllegalArgumentException(
-s"DType: $dtypeStr not found! please set it in DType.scala")
-}
- }
}
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
index b0fae0f..6365f9c 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
@@ -159,7 +159,14 @@ class Executor private[mxnet](private[mxnet] val handle:
ExecutorHandle,
private def getOutputs: Array[NDArray] = {
val ndHandles = ArrayBuffer[NDArrayHandle]()
checkCall(_LIB.mxExecutorOutputs(handle, ndHandles))
-ndHandles.toArray.map(new NDArray(_, addToCollector = false))
+ndHandles.toArray.map(ele => {
+val nd = new NDArray(ele, addToCollector = false)
+if (nd.isSparse) {
+ nd.asInstanceOf[SparseNDArray]
+}
+nd
+ }
+)
}
/**
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
index 640ecf5..0ee6476 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
@@ -31,13 +31,14 @@ private[mxnet] class LibInfo {
@native def mxListAllOpNames(names: ListBuffer[String]): Int
@native def nnGetOpHandle(opName: String, opHandle: RefLong): Int
// NDArray
- @native def mxImperativeInvoke(creator: FunctionHandle,
+ @native def mxImperativeInvokeEx(creator: FunctionHandle,
inputs: Array[NDArrayHandle],
outputsGiven: Array[NDArrayHandle],
outputs: ArrayBuffer[NDArrayHandle],
numParams: Int,
paramKeys: Array[String],
- paramVals: Array[String]): Int
+ paramVals: Array[String],
+ outStype: ArrayBuffer[Int]): Int
@native def mxNDArrayFree(handle: NDArrayHandle): I
[incubator-mxnet] branch v1.4.x updated: Exclude external dependencies from MXNet JAR (and fix Clojure build) (#15361)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.4.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.4.x by this push:
new e368ea2 Exclude external dependencies from MXNet JAR (and fix Clojure
build) (#15361)
e368ea2 is described below
commit e368ea24895b92f7e6b43fc0a51b3b0131193f12
Author: tklein23
AuthorDate: Mon Jul 1 23:51:32 2019 +0200
Exclude external dependencies from MXNet JAR (and fix Clojure build)
(#15361)
---
CONTRIBUTORS.md| 1 +
contrib/clojure-package/project.clj| 9 +-
scala-package/assembly/linux-x86_64-cpu/pom.xml| 36 ++
.../src/main/assembly/assembly.xml | 8 +
scala-package/assembly/linux-x86_64-gpu/pom.xml| 36 ++
.../src/main/assembly/assembly.xml | 8 +
scala-package/assembly/osx-x86_64-cpu/pom.xml | 36 ++
.../osx-x86_64-cpu/src/main/assembly/assembly.xml | 8 +
scala-package/assembly/pom.xml | 1 +
scala-package/core/pom.xml | 5 ---
scala-package/macros/pom.xml | 6
scala-package/pom.xml | 5 +++
12 files changed, 147 insertions(+), 12 deletions(-)
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 8ca8a89..51157ca 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -209,6 +209,7 @@ List of Contributors
* [Rahul Padmanabhan](https://github.com/rahul3)
* [Yuxi Hu](https://github.com/yuxihu)
* [Harsh Patel](https://github.com/harshp8l)
+* [Thoralf Klein](https://github.com/tklein23)
Label Bot
-
diff --git a/contrib/clojure-package/project.clj
b/contrib/clojure-package/project.clj
index 821da0e..ebc8256 100644
--- a/contrib/clojure-package/project.clj
+++ b/contrib/clojure-package/project.clj
@@ -34,7 +34,14 @@
[org.clojure/tools.logging "0.4.0"]
[org.apache.logging.log4j/log4j-core "2.8.1"]
[org.apache.logging.log4j/log4j-api "2.8.1"]
- [org.slf4j/slf4j-log4j12 "1.7.25" :exclusions
[org.slf4j/slf4j-api]]]
+ [org.slf4j/slf4j-log4j12 "1.7.25"]
+ [org.slf4j/slf4j-api "1.7.5"]
+
+ [org.scala-lang/scala-library "2.11.8"]
+ [org.scala-lang/scala-reflect "2.11.8"]
+ [org.scala-lang.modules/scala-parser-combinators_2.11 "1.0.4"]
+]
+
:pedantic? :skip
:plugins [[lein-codox "0.10.6" :exclusions [org.clojure/clojure]]
[lein-cloverage "1.0.10" :exclusions [org.clojure/clojure]]
diff --git a/scala-package/assembly/linux-x86_64-cpu/pom.xml
b/scala-package/assembly/linux-x86_64-cpu/pom.xml
index 6177676..ba2c030 100644
--- a/scala-package/assembly/linux-x86_64-cpu/pom.xml
+++ b/scala-package/assembly/linux-x86_64-cpu/pom.xml
@@ -47,6 +47,42 @@
mxnet-infer_${scala.binary.version}
1.4.1-SNAPSHOT
+
+ org.scala-lang
+ scala-library
+ ${scala.version}
+
+
+ org.scala-lang
+ scala-reflect
+ ${scala.version}
+
+
+ commons-codec
+ commons-codec
+ 1.10
+
+
+ commons-io
+ commons-io
+ 2.1
+
+
+ org.apache.logging.log4j
+ log4j-core
+ 2.11.1
+ provided
+
+
+ org.slf4j
+ slf4j-api
+ 1.7.5
+
+
+ args4j
+ args4j
+ 2.0.29
+
diff --git
a/scala-package/assembly/linux-x86_64-cpu/src/main/assembly/assembly.xml
b/scala-package/assembly/linux-x86_64-cpu/src/main/assembly/assembly.xml
index 9f28706..9628db6 100644
--- a/scala-package/assembly/linux-x86_64-cpu/src/main/assembly/assembly.xml
+++ b/scala-package/assembly/linux-x86_64-cpu/src/main/assembly/assembly.xml
@@ -25,6 +25,14 @@
*:*:jar
+
+org.scala-lang:*
+org.scala-lang.modules:*
+commons-io:commons-io
+commons-codec:commons-codec
+org.slf4j:slf4j-api
+args4j:args4j
+
/
true
true
diff --git a/scala-package/assembly/linux-x86_64-gpu/pom.xml
b/scala-package/assembly/linux-x86_64-gpu/pom.xml
index 1209a15..12a3e96 100644
--- a/scala-package/assembly/linux-x86_64-gpu/pom.xml
+++ b/scala-package/assembly/linux-x86_64-gpu/pom.xml
@@ -47,6 +47,42 @@
mxnet-infer_${scala.binary.version}
1.4.1-SNAPSHOT
+
+ org.scala-lang
+ scala-library
+ ${scala.version}
+
+
+ org.scala-lang
+ scala-reflect
+ ${scala.version}
+
+
+ commons-codec
+ commons-codec
+ 1.10
+
+
+ commons-io
+ commons-io
+ 2.1
+
+
+ org.apache.logging.lo
[incubator-mxnet] branch master updated (e8f3e91 -> 8aaacde)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from e8f3e91 [Opperf] Make module/namespace of the operator parameterized (#15226) add 8aaacde Update sparse_retain Documentation (#15394) No new revisions were added by this update. Summary of changes: src/operator/tensor/sparse_retain.cc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[incubator-mxnet] branch master updated (d32a3da -> 29fea48)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git.
from d32a3da [MKLDNN] Fix quantized act and concat (#15209)
add 29fea48 Readd Java HelloWorld example for intellij tutorial (#15217)
No new revisions were added by this update.
Summary of changes:
docs/tutorials/java/mxnet_java_on_intellij.md | 2 +-
scala-package/mxnet-demo/java-demo/README.md | 3 ++-
.../{NDArrayOperation.java => HelloWorld.java} | 26 ++
3 files changed, 10 insertions(+), 21 deletions(-)
copy
scala-package/mxnet-demo/java-demo/src/main/java/mxnet/{NDArrayOperation.java
=> HelloWorld.java} (67%)
[incubator-mxnet] branch master updated: [MXNET-1416] Fix inception inference example for potential index out of range error. (#15179)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 745a41c [MXNET-1416] Fix inception inference example for potential
index out of range error. (#15179)
745a41c is described below
commit 745a41ca1a6d74a645911de8af46dece03db93ea
Author: Amol Lele <19983848+lelea...@users.noreply.github.com>
AuthorDate: Fri Jun 7 15:04:37 2019 -0700
[MXNET-1416] Fix inception inference example for potential index out of
range error. (#15179)
* Adding support to get the data from 1D NDArray.
* Added the error handling for index out of range.
---
cpp-package/example/inference/inception_inference.cpp | 4 +---
cpp-package/include/mxnet-cpp/ndarray.h | 6 ++
cpp-package/include/mxnet-cpp/ndarray.hpp | 13 -
3 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/cpp-package/example/inference/inception_inference.cpp
b/cpp-package/example/inference/inception_inference.cpp
index fa56001..4d7d5f4 100644
--- a/cpp-package/example/inference/inception_inference.cpp
+++ b/cpp-package/example/inference/inception_inference.cpp
@@ -302,13 +302,11 @@ void Predictor::PredictImage(const std::string&
image_file) {
// The output is available in executor->outputs.
auto array = executor->outputs[0].Copy(Context::cpu());
-
/*
* Find out the maximum accuracy and the index associated with that accuracy.
* This is done by using the argmax operator on NDArray.
*/
auto predicted = array.ArgmaxChannel();
-
/*
* Wait until all the previous write operations on the 'predicted'
* NDArray to be complete before we read it.
@@ -317,7 +315,7 @@ void Predictor::PredictImage(const std::string& image_file)
{
*/
predicted.WaitToRead();
- int best_idx = predicted.At(0, 0);
+ int best_idx = predicted.At(0);
float best_accuracy = array.At(0, best_idx);
if (output_labels.empty()) {
diff --git a/cpp-package/include/mxnet-cpp/ndarray.h
b/cpp-package/include/mxnet-cpp/ndarray.h
index 7953b61..c4d51c5 100644
--- a/cpp-package/include/mxnet-cpp/ndarray.h
+++ b/cpp-package/include/mxnet-cpp/ndarray.h
@@ -321,6 +321,12 @@ class NDArray {
*/
size_t Offset(size_t c, size_t h, size_t w) const;
/*!
+ * \brief return value of the element at (index)
+ * \param index position
+ * \return value of one dimensions array
+ */
+ mx_float At(size_t index) const;
+ /*!
* \brief return value of the element at (h, w)
* \param h height position
* \param w width position
diff --git a/cpp-package/include/mxnet-cpp/ndarray.hpp
b/cpp-package/include/mxnet-cpp/ndarray.hpp
index 283fff1..ed23c76 100644
--- a/cpp-package/include/mxnet-cpp/ndarray.hpp
+++ b/cpp-package/include/mxnet-cpp/ndarray.hpp
@@ -375,11 +375,15 @@ inline void NDArray::Save(const std::string _name,
}
inline size_t NDArray::Offset(size_t h, size_t w) const {
- return (h * GetShape()[1]) + w;
+ auto const shape = GetShape();
+ CHECK_EQ(shape.size(), 2) << "The NDArray needs to be 2 dimensional.";
+
+ return (h * shape[1]) + w;
}
inline size_t NDArray::Offset(size_t c, size_t h, size_t w) const {
auto const shape = GetShape();
+ CHECK_EQ(shape.size(), 3) << "The NDArray needs to be 3 dimensional.";
return h * shape[0] * shape[2] + w * shape[0] + c;
}
@@ -391,6 +395,13 @@ inline mx_float NDArray::At(size_t c, size_t h, size_t w)
const {
return GetData()[Offset(c, h, w)];
}
+inline mx_float NDArray::At(size_t index) const {
+ auto shape = GetShape();
+ CHECK_EQ(shape.size(), 1) << "The NDArray needs to be 1 dimensional.";
+ CHECK_LT(index, shape[0]) << "Specified index is out of range.";
+ return GetData()[index];
+}
+
inline size_t NDArray::Size() const {
size_t ret = 1;
for (auto : GetShape()) ret *= i;
[incubator-mxnet] branch master updated (77da1e2 -> eaacbaf)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git.
from 77da1e2 add cast_dtype option to load parameters (#15168)
add eaacbaf Fix Scala release (#15155)
No new revisions were added by this update.
Summary of changes:
scala-package/deploy/pom.xml | 50 +--
scala-package/externalPom/pom.xml | 152 +
.../src/main/deploy/deploy.xml | 0
scala-package/pom.xml | 2 +
4 files changed, 161 insertions(+), 43 deletions(-)
create mode 100644 scala-package/externalPom/pom.xml
rename scala-package/{deploy => externalPom}/src/main/deploy/deploy.xml (100%)
[incubator-mxnet] branch master updated (eecf07a -> 60342d6)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from eecf07a Update MKL-DNN submodule to v0.19 (#14783) add 60342d6 add KEY for zachgk (#14965) No new revisions were added by this update. Summary of changes: CONTRIBUTORS.md | 3 ++- KEYS| 59 + 2 files changed, 61 insertions(+), 1 deletion(-)
[incubator-mxnet] branch master updated: Fixes call to build ubuntu gpu in nightly tests (#14964)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 8b7e374 Fixes call to build ubuntu gpu in nightly tests (#14964)
8b7e374 is described below
commit 8b7e374d385df9f40dd0d375841aec230ef31e39
Author: perdasilva
AuthorDate: Wed May 15 22:14:54 2019 +0200
Fixes call to build ubuntu gpu in nightly tests (#14964)
---
tests/nightly/JenkinsfileForBinaries | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tests/nightly/JenkinsfileForBinaries
b/tests/nightly/JenkinsfileForBinaries
index 6545d59..89b96e2 100755
--- a/tests/nightly/JenkinsfileForBinaries
+++ b/tests/nightly/JenkinsfileForBinaries
@@ -35,8 +35,7 @@ core_logic: {
node(NODE_LINUX_CPU) {
ws('workspace/build-gpu') {
utils.init_git()
- //sh "ci/build.py --platform ubuntu_build_cuda
/work/runtime_functions.sh build_ubuntu_gpu_cuda91_cudnn7"
- utils.docker_run('ubuntu_build_cuda',
'build_ubuntu_gpu_cuda91_cudnn7', false)
+ utils.docker_run('ubuntu_build_cuda',
'build_ubuntu_gpu_cuda100_cudnn7', false)
utils.pack_lib('gpu', mx_lib)
}
}
[incubator-mxnet] branch master updated: Deprecate NDArrayCollector and instead use ResourceScope (#14780)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 4796851 Deprecate NDArrayCollector and instead use ResourceScope
(#14780)
4796851 is described below
commit 4796851b2fb2254899dd73e56dc9589666fb2d8c
Author: Zach Kimberg
AuthorDate: Fri May 10 17:12:09 2019 -0700
Deprecate NDArrayCollector and instead use ResourceScope (#14780)
---
.../src/main/scala/org/apache/mxnet/NDArray.scala | 14 +--
.../scala/org/apache/mxnet/NDArrayCollector.scala | 6 +++
.../scala/org/apache/mxnet/ResourceScope.scala | 46 +++---
.../scala/org/apache/mxnet/io/NDArrayIter.scala| 2 +-
.../benchmark/ScalaInferenceBenchmark.scala| 8 ++--
.../CNNTextClassification.scala| 6 +--
.../mxnetexamples/customop/ExampleCustomOp.scala | 4 +-
.../org/apache/mxnetexamples/gan/GanMnist.scala| 6 +--
.../imageclassifier/ImageClassifierExample.scala | 4 +-
.../objectdetector/SSDClassifierExample.scala | 4 +-
.../mxnetexamples/multitask/ExampleMultiTask.scala | 4 +-
.../mxnetexamples/neuralstyle/NeuralStyle.scala| 2 +-
.../neuralstyle/end2end/BoostInference.scala | 6 +--
.../neuralstyle/end2end/BoostTrain.scala | 6 +--
.../apache/mxnetexamples/rnn/LstmBucketing.scala | 2 +-
.../org/apache/mxnetexamples/rnn/TestCharRnn.scala | 2 +-
.../apache/mxnetexamples/rnn/TrainCharRnn.scala| 2 +-
.../CNNClassifierExampleSuite.scala| 2 +-
.../apache/mxnetexamples/gan/GanExampleSuite.scala | 2 +-
.../neuralstyle/NeuralStyleSuite.scala | 2 +-
.../apache/mxnetexamples/rnn/ExampleRNNSuite.scala | 2 +-
21 files changed, 72 insertions(+), 60 deletions(-)
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 3764f5a..4088801 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -741,10 +741,16 @@ object NDArray extends NDArrayBase {
*
*/
class NDArray private[mxnet](private[mxnet] val handle: NDArrayHandle,
- val writable: Boolean = true,
- addToCollector: Boolean = true) extends
NativeResource {
- if (addToCollector) {
-NDArrayCollector.collect(this)
+ val writable: Boolean) extends NativeResource {
+
+ @deprecated("Please use ResourceScope instead", "1.5.0")
+ def this(handle: NDArrayHandle,
+ writable: Boolean = true,
+ addToCollector: Boolean = true) {
+this(handle, writable)
+if (addToCollector) {
+ NDArrayCollector.collect(this)
+}
}
override def nativeAddress: CPtrAddress = handle
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/NDArrayCollector.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArrayCollector.scala
index 0b7f9af..0761481 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArrayCollector.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArrayCollector.scala
@@ -64,6 +64,7 @@ import scala.collection.mutable
* });
*
*/
+@deprecated("Please use ResourceScope instead", "1.5.0")
object NDArrayCollector {
private val logger = LoggerFactory.getLogger(classOf[NDArrayCollector])
@@ -75,12 +76,14 @@ object NDArrayCollector {
* Create a collector which will dispose the collected NDArrays
automatically.
* @return an auto-disposable collector.
*/
+ @deprecated("Please use ResourceScope instead", "1.5.0")
def auto(): NDArrayCollector = new NDArrayCollector(true)
/**
* Create a collector allows users to later dispose the collected NDArray
manually.
* @return a manually-disposable collector.
*/
+ @deprecated("Please use ResourceScope instead", "1.5.0")
@Experimental
def manual(): NDArrayCollector = new NDArrayCollector(false)
@@ -88,11 +91,13 @@ object NDArrayCollector {
* Collect the NDArrays into the collector of the current thread.
* @param ndArray NDArrays need to be collected.
*/
+ @deprecated("Please use ResourceScope instead", "1.5.0")
@varargs def collect(ndArray: NDArray*): Unit = {
currCollector.get().add(ndArray: _*)
}
}
+@deprecated("Please use ResourceScope instead", "1.5.0")
class NDArrayCollector private(private val autoDispose: Boolean = true,
private val doCollect: Boolean = true) {
// native ptr (handle) of the NDArray -> NDArray
@@ -142,6 +147,7 @@ class NDArrayCollector private(private val autoDispose:
Boolean = true,
* @return T
[incubator-mxnet] branch master updated (42ede50 -> a722db4)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 42ede50 rewrite test_custom_op_exc (#14878) add a722db4 [Dependency Update] Upgrade openssl to 1.1.1b (#14837) No new revisions were added by this update. Summary of changes: tools/dependencies/openssl.sh | 2 +- tools/staticbuild/build_lib.sh | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)
[incubator-mxnet] branch v1.4.x updated: [v1.4.1] Java bug-fix cherry pick (#14834)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.4.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.4.x by this push:
new 69515c2 [v1.4.1] Java bug-fix cherry pick (#14834)
69515c2 is described below
commit 69515c2f9b1ac6fd4b661d5411a97de968cf4e2e
Author: Lanking
AuthorDate: Mon Apr 29 16:03:36 2019 -0700
[v1.4.1] Java bug-fix cherry pick (#14834)
* clean up submodule (#14645)
* Scala/Java Predict API fix #14756 (#14804)
* add fix in the code
* add unit test
* update comments
* add fixes to code gen
---
.../scala/org/apache/mxnet/module/BaseModule.scala | 17 +-
.../java/org/apache/mxnet/javaapi/NDArrayTest.java | 4 +-
.../test/scala/org/apache/mxnet/ModuleSuite.scala | 28
.../scala/org/apache/mxnet/APIDocGenerator.scala | 184 ++---
4 files changed, 173 insertions(+), 60 deletions(-)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
index b73f4ad..73ccef2 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
@@ -247,11 +247,23 @@ abstract class BaseModule {
/**
* Run prediction and collect the outputs.
- * @param evalData
+ * @param evalData dataIter to do the Inference
* @param numBatch Default is -1, indicating running all the batches in the
data iterator.
* @param reset Default is `True`, indicating whether we should reset the
data iter before start
* doing prediction.
* @return The return value will be a list `[out1, out2, out3]`.
+ *The concatenation process will be like
+ *{{{
+ *outputBatches = [
+ * [a1, a2, a3], // batch a
+ * [b1, b2, b3] // batch b
+ *]
+ *result = [
+ * NDArray, // [a1, b1]
+ * NDArray, // [a2, b2]
+ * NDArray, // [a3, b3]
+ *]
+ *}}}
* Where each element is concatenation of the outputs for all the
mini-batches.
*/
def predict(evalData: DataIter, numBatch: Int = -1, reset: Boolean = true)
@@ -264,7 +276,8 @@ abstract class BaseModule {
s"in mini-batches (${out.size})." +
"Maybe bucketing is used?")
)
-val concatenatedOutput = outputBatches.map(out => NDArray.concatenate(out))
+val oBT = outputBatches.transpose
+val concatenatedOutput = oBT.map(out => NDArray.concatenate(out))
outputBatches.foreach(_.foreach(_.dispose()))
concatenatedOutput
}
diff --git
a/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
b/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
index 2659b78..5bbe8bb 100644
--- a/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
+++ b/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
@@ -71,7 +71,7 @@ public class NDArrayTest {
NDArray$ NDArray = NDArray$.MODULE$;
float[] arr = new float[]{1.0f, 2.0f, 3.0f};
NDArray nd = new NDArray(arr, new Shape(new int[]{3}), new
Context("cpu", 0));
-float result = NDArray.norm(NDArray.new normParam(nd))[0].toArray()[0];
+float result = NDArray.norm(new normParam(nd))[0].toArray()[0];
float cal = 0.0f;
for (float ele : arr) {
cal += ele * ele;
@@ -79,7 +79,7 @@ public class NDArrayTest {
cal = (float) Math.sqrt(cal);
assertTrue(Math.abs(result - cal) < 1e-5);
NDArray dotResult = new NDArray(new float[]{0}, new Shape(new
int[]{1}), new Context("cpu", 0));
-NDArray.dot(NDArray.new dotParam(nd, nd).setOut(dotResult));
+NDArray.dot(new dotParam(nd, nd).setOut(dotResult));
assertTrue(Arrays.equals(dotResult.toArray(), new float[]{14.0f}));
}
}
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
index 88e314e..e6ebfd3 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
@@ -23,6 +23,34 @@ import org.apache.mxnet.optimizer._
import org.apache.mxnet.io._
class ModuleSuite extends FunSuite with BeforeAndAfterAll {
+
+ class myModule(symbol : Symbol) extends Module (symbol) {
+override def predictEveryBatch(evalData: DataIter,
+ numBatch: Int = 1, reset: Boolean = true):
+IndexedSeq[IndexedSeq[NDArray]] = {
+ val data = IndexedSeq(
+NDArray.ones(Shape
[incubator-mxnet] branch master updated: Scala/Java Predict API fix #14756 (#14804)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 6cbc273 Scala/Java Predict API fix #14756 (#14804)
6cbc273 is described below
commit 6cbc273454d0e6c24307183eeedc5c3e33e2ccc1
Author: Lanking
AuthorDate: Fri Apr 26 20:40:02 2019 -0700
Scala/Java Predict API fix #14756 (#14804)
* add fix in the code
* add unit test
* update comments
---
.../scala/org/apache/mxnet/module/BaseModule.scala | 17 +++--
.../test/scala/org/apache/mxnet/ModuleSuite.scala | 28 ++
2 files changed, 43 insertions(+), 2 deletions(-)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
index 3be8e06..7fbdae5 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/module/BaseModule.scala
@@ -247,11 +247,23 @@ abstract class BaseModule {
/**
* Run prediction and collect the outputs.
- * @param evalData
+ * @param evalData dataIter to do the Inference
* @param numBatch Default is -1, indicating running all the batches in the
data iterator.
* @param reset Default is `True`, indicating whether we should reset the
data iter before start
* doing prediction.
* @return The return value will be a list `[out1, out2, out3]`.
+ *The concatenation process will be like
+ *{{{
+ *outputBatches = [
+ * [a1, a2, a3], // batch a
+ * [b1, b2, b3] // batch b
+ *]
+ *result = [
+ * NDArray, // [a1, b1]
+ * NDArray, // [a2, b2]
+ * NDArray, // [a3, b3]
+ *]
+ *}}}
* Where each element is concatenation of the outputs for all the
mini-batches.
*/
def predict(evalData: DataIter, numBatch: Int = -1, reset: Boolean = true)
@@ -264,7 +276,8 @@ abstract class BaseModule {
s"in mini-batches (${out.size})." +
"Maybe bucketing is used?")
)
-val concatenatedOutput = outputBatches.map(out => NDArray.concatenate(out))
+val oBT = outputBatches.transpose
+val concatenatedOutput = oBT.map(out => NDArray.concatenate(out))
outputBatches.foreach(_.foreach(_.dispose()))
concatenatedOutput
}
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
index 3e753a1..5aed01b 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/ModuleSuite.scala
@@ -23,6 +23,34 @@ import org.apache.mxnet.optimizer._
import org.apache.mxnet.io._
class ModuleSuite extends FunSuite with BeforeAndAfterAll {
+
+ class myModule(symbol : Symbol) extends Module (symbol) {
+override def predictEveryBatch(evalData: DataIter,
+ numBatch: Int = 1, reset: Boolean = true):
+IndexedSeq[IndexedSeq[NDArray]] = {
+ val data = IndexedSeq(
+NDArray.ones(Shape(1, 10, 1)),
+NDArray.ones(Shape(1, 10, 1)),
+NDArray.ones(Shape(1, 10, 4))
+ )
+ List.fill(numBatch)(data).toIndexedSeq
+}
+ }
+
+ test("predict") {
+val sym = Symbol.Variable("data")
+val mod = new myModule(sym)
+val dummyIter = new NDArrayIter(IndexedSeq(NDArray.ones(1)))
+var output = mod.predict(dummyIter, 1)
+require(output(0).shape == Shape(1, 10, 1))
+require(output(1).shape == Shape(1, 10, 1))
+require(output(2).shape == Shape(1, 10, 4))
+output = mod.predict(dummyIter, 2)
+require(output(0).shape == Shape(2, 10, 1))
+require(output(1).shape == Shape(2, 10, 1))
+require(output(2).shape == Shape(2, 10, 4))
+ }
+
test ("model dtype") {
val dType = DType.Float32
val dShape = Shape(3, 8, 7)
[incubator-mxnet] branch master updated: clean up submodule (#14645)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 003800c clean up submodule (#14645)
003800c is described below
commit 003800c50bdab98443f32e6dac4e227d9384a2ba
Author: Lanking
AuthorDate: Fri Apr 26 20:40:17 2019 -0700
clean up submodule (#14645)
---
.../java/org/apache/mxnet/javaapi/NDArrayTest.java | 4 ++--
.../mxnetexamples/javaapi/infer/bert/BertQA.java | 6 ++---
.../scala/org/apache/mxnet/APIDocGenerator.scala | 28 +-
.../src/main/java/mxnet/NDArrayCreation.java | 2 +-
.../src/main/java/mxnet/NDArrayOperation.java | 2 +-
5 files changed, 24 insertions(+), 18 deletions(-)
diff --git
a/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
b/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
index 86c7eb2..1b7042d 100644
--- a/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
+++ b/scala-package/core/src/test/java/org/apache/mxnet/javaapi/NDArrayTest.java
@@ -86,7 +86,7 @@ public class NDArrayTest {
NDArray$ NDArray = NDArray$.MODULE$;
float[] arr = new float[]{1.0f, 2.0f, 3.0f};
NDArray nd = new NDArray(arr, new Shape(new int[]{3}), new
Context("cpu", 0));
-float result = NDArray.norm(NDArray.new normParam(nd))[0].toArray()[0];
+float result = NDArray.norm(new normParam(nd))[0].toArray()[0];
float cal = 0.0f;
for (float ele : arr) {
cal += ele * ele;
@@ -94,7 +94,7 @@ public class NDArrayTest {
cal = (float) Math.sqrt(cal);
assertTrue(Math.abs(result - cal) < 1e-5);
NDArray dotResult = new NDArray(new float[]{0}, new Shape(new
int[]{1}), new Context("cpu", 0));
-NDArray.dot(NDArray.new dotParam(nd, nd).setOut(dotResult));
+NDArray.dot(new dotParam(nd, nd).setOut(dotResult));
assertTrue(Arrays.equals(dotResult.toArray(), new float[]{14.0f}));
}
}
diff --git
a/scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertQA.java
b/scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertQA.java
index b40a4e9..dd17b1d 100644
---
a/scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertQA.java
+++
b/scala-package/examples/src/main/java/org/apache/mxnetexamples/javaapi/infer/bert/BertQA.java
@@ -68,15 +68,15 @@ public class BertQA {
*/
static List postProcessing(NDArray result, List tokens) {
NDArray[] output = NDArray.split(
-NDArray.new splitParam(result, 2).setAxis(2));
+new splitParam(result, 2).setAxis(2));
// Get the formatted logits result
NDArray startLogits = output[0].reshape(new int[]{0, -3});
NDArray endLogits = output[1].reshape(new int[]{0, -3});
// Get Probability distribution
float[] startProb = NDArray.softmax(
-NDArray.new softmaxParam(startLogits))[0].toArray();
+new softmaxParam(startLogits))[0].toArray();
float[] endProb = NDArray.softmax(
-NDArray.new softmaxParam(endLogits))[0].toArray();
+new softmaxParam(endLogits))[0].toArray();
int startIdx = argmax(startProb);
int endIdx = argmax(endProb);
return tokens.subList(startIdx, endIdx + 1);
diff --git
a/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
b/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
index a5102d6..e939b2e 100644
--- a/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
+++ b/scala-package/macros/src/main/scala/org/apache/mxnet/APIDocGenerator.scala
@@ -152,7 +152,8 @@ private[mxnet] object APIDocGenerator extends GeneratorBase
with RandomHelpers {
def javaClassGen(FILE_PATH : String) : String = {
val notGenerated = Set("Custom")
val absClassFunctions = functionsToGenerate(false, false, true)
-val absFuncs = absClassFunctions.filterNot(ele =>
notGenerated.contains(ele.name))
+val (absFuncs, paramClassUncleaned) =
+ absClassFunctions.filterNot(ele => notGenerated.contains(ele.name))
.groupBy(_.name.toLowerCase).map(ele => {
/* Pattern matching for not generating deprecated method
* Group all method name in lowercase
@@ -166,7 +167,8 @@ private[mxnet] object APIDocGenerator extends GeneratorBase
with RandomHelpers {
}
}).map(absClassFunction => {
generateJavaAPISignature(absClassFunction)
- }).toSeq
+ }).toSeq.unzip
+val paramClass = paramClassUncleaned.filterNot(_.isEmpty)
val packageName = "NDArrayBase"
val packageDef = "package org.apache.mxne
[incubator-mxnet] branch master updated: julia/ndarray: fix flaky test cases for `clamp` (#14776)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 8cae72e julia/ndarray: fix flaky test cases for `clamp` (#14776)
8cae72e is described below
commit 8cae72eeadd3ab59de1b1ad81c202f6eb513e07e
Author: Iblis Lin
AuthorDate: Thu Apr 25 06:44:34 2019 +0800
julia/ndarray: fix flaky test cases for `clamp` (#14776)
ref: #14757
---
julia/test/unittest/ndarray.jl | 22 --
1 file changed, 12 insertions(+), 10 deletions(-)
diff --git a/julia/test/unittest/ndarray.jl b/julia/test/unittest/ndarray.jl
index 26695e8..638963f 100644
--- a/julia/test/unittest/ndarray.jl
+++ b/julia/test/unittest/ndarray.jl
@@ -886,18 +886,20 @@ function test_saveload()
end
function test_clamp()
- dims = rand_dims()
- @info("NDArray::clamp::dims = $dims")
-
- j_array, nd_array = rand_tensors(dims)
- clip_up = maximum(abs.(j_array)) / 2
- clip_down = 0
- clipped = clamp(nd_array, clip_down, clip_up)
+ @info("NDArray::clamp::dims")
+
+ A = [1 2 3;
+ 4 5 6;
+ 7 8 9.]
+ B = [3 3 3;
+ 4 5 6;
+ 7 8 8.]
+ x = NDArray(A)
+ y = clamp(x, 3., 8.)
# make sure the original array is not modified
- @test copy(nd_array) ≈ j_array
-
- @test all(clip_down .<= copy(clipped) .<= clip_up)
+ @test copy(x) ≈ A
+ @test copy(y) ≈ B
@info("NDArray::clamp!")
let
[incubator-mxnet] branch master updated: [MXNET-1385] Improved Scala Init and Macros warning messages (#14656)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 153d2f4 [MXNET-1385] Improved Scala Init and Macros warning messages
(#14656)
153d2f4 is described below
commit 153d2f4bf9611c9d4a8d2d1abaa17732b6e9038a
Author: Lanking
AuthorDate: Fri Apr 19 15:53:14 2019 -0700
[MXNET-1385] Improved Scala Init and Macros warning messages (#14656)
* 1st phase scala/java warning killers
* add more comments
* Capitialize all starting words of comments
* amend the rest of them
---
.../src/main/scala/org/apache/mxnet/Image.scala| 8 +++
.../src/main/scala/org/apache/mxnet/NDArray.scala | 5 ++
.../scala/org/apache/mxnet/javaapi/Image.scala | 25 +++
.../main/scala/org/apache/mxnet/init/Base.scala| 27 +++-
.../main/scala/org/apache/mxnet/init/LibInfo.scala | 30 +
.../scala/org/apache/mxnet/APIDocGenerator.scala | 73 ++--
.../scala/org/apache/mxnet/GeneratorBase.scala | 77 +-
.../main/scala/org/apache/mxnet/NDArrayMacro.scala | 61 ++---
.../main/scala/org/apache/mxnet/SymbolMacro.scala | 58 ++--
.../apache/mxnet/javaapi/JavaNDArrayMacro.scala| 15 -
.../org/apache/mxnet/utils/CToScalaUtils.scala | 12 +++-
.../apache/mxnet/utils/OperatorBuildUtils.scala| 32 -
12 files changed, 347 insertions(+), 76 deletions(-)
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
index 52e26ef..b54ecc0 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
@@ -38,6 +38,7 @@ object Image {
* @param flag Convert decoded image to grayscale (0) or color (1).
* @param to_rgb Whether to convert decoded image
* to mxnet's default RGB format (instead of opencv's default
BGR).
+* @param outNDArray to store the output
* @return NDArray in HWC format with DType [[DType.UInt8]]
*/
def imDecode(buf: Array[Byte], flag: Int,
@@ -57,6 +58,10 @@ object Image {
/**
* Same imageDecode with InputStream
* @param inputStream the inputStream of the image
+* @param flag Convert decoded image to grayscale (0) or color (1).
+* @param to_rgb Whether to convert decoded image
+* to mxnet's default RGB format (instead of opencv's default
BGR).
+* @param outNDArray to store the output
* @return NDArray in HWC format with DType [[DType.UInt8]]
*/
def imDecode(inputStream: InputStream, flag: Int = 1,
@@ -79,6 +84,7 @@ object Image {
* @param flag Convert decoded image to grayscale (0) or color (1).
* @param to_rgb Whether to convert decoded image to mxnet's default RGB
format
* (instead of opencv's default BGR).
+* @param outNDArray to store the output
* @return org.apache.mxnet.NDArray in HWC format with DType [[DType.UInt8]]
*/
def imRead(filename: String, flag: Option[Int] = None,
@@ -99,6 +105,7 @@ object Image {
* @param w Width of resized image.
* @param h Height of resized image.
* @param interp Interpolation method (default=cv2.INTER_LINEAR).
+* @param outNDArray to store the output
* @return org.apache.mxnet.NDArray
*/
def imResize(src: org.apache.mxnet.NDArray, w: Int, h: Int,
@@ -124,6 +131,7 @@ object Image {
* @param typeOf Filling type (default=cv2.BORDER_CONSTANT).
* @param value (Deprecated! Use ``values`` instead.) Fill with single
value.
* @param values Fill with value(RGB[A] or gray), up to 4 channels.
+* @param outNDArray to store the output
* @return org.apache.mxnet.NDArray
*/
def copyMakeBorder(src: org.apache.mxnet.NDArray, top: Int, bot: Int,
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 849f456..3764f5a 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -36,6 +36,11 @@ import scala.util.Try
*/
@AddNDArrayFunctions(false)
object NDArray extends NDArrayBase {
+ /**
+* method to convert NDArrayFunctionReturn to NDArray
+* @param ret the returned NDArray list
+* @return NDArray result
+*/
implicit def getFirstResult(ret: NDArrayFuncReturn): NDArray = ret(0)
private val logger = LoggerFactory.getLogger(classOf[NDArray])
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.scala
index f72223d..57a4850 100644
[incubator-mxnet] branch master updated: License Googletest and Appendix (#14687)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new dc48cd2 License Googletest and Appendix (#14687) dc48cd2 is described below commit dc48cd2a5a6460171bf9b842453866e731e6ff7d Author: Zach Kimberg AuthorDate: Fri Apr 19 10:00:47 2019 -0700 License Googletest and Appendix (#14687) * Update googletest due to licensing * Add common licenses to LICENSE appendix Add zachgk to contributors * Remove appendix apache license and fix some indentation --- 3rdparty/googletest | 2 +- CONTRIBUTORS.md | 1 + LICENSE | 42 +- 3 files changed, 43 insertions(+), 2 deletions(-) diff --git a/3rdparty/googletest b/3rdparty/googletest index ec44c6c..eb9225c 16 --- a/3rdparty/googletest +++ b/3rdparty/googletest @@ -1 +1 @@ -Subproject commit ec44c6c1675c25b9827aacd08c02433cccde7780 +Subproject commit eb9225ce361affe561592e0912320b9db84985d0 diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 5c5c217..9508f1e 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -236,6 +236,7 @@ List of Contributors * [Zhennan Qin](https://github.com/ZhennanQin) * [Zhiyuan Huang](https://github.com/huangzhiyuan) * [Zak Jost](https://github.com/zjost) +* [Zach Kimberg](https://github.com/zachgk) Label Bot - diff --git a/LICENSE b/LICENSE index 72fe08f..0673e0e 100644 --- a/LICENSE +++ b/LICENSE @@ -207,7 +207,7 @@ The Apache MXNET (incubating) project contains subcomponents with separate copyright notices and license terms. Your use of the source code for the these subcomponents is subject to the terms and conditions of the following -licenses. +licenses (full text in Appendix). === Apache-2.0 licenses @@ -706,3 +706,43 @@ FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + + === +Appendix + === + +MIT License + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 3-clause BSD + + === + +3-Clause BSD License + +Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR [...] + + ==
[incubator-mxnet] branch master updated: [MXNET-1287] Feat dep (#14668)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 5b6e25b [MXNET-1287] Feat dep (#14668)
5b6e25b is described below
commit 5b6e25bd1a9178468c6394222a60511663528fb6
Author: Zach Kimberg
AuthorDate: Fri Apr 19 09:59:58 2019 -0700
[MXNET-1287] Feat dep (#14668)
* Feature and Deprecation Warnings
* Address feature and deprecation Warnings (mostly provideData)
* Fix name search
---
.../scala/org/apache/mxnet/ExecutorManager.scala | 27 +++
.../main/scala/org/apache/mxnet/FeedForward.scala | 14 ++--
.../core/src/main/scala/org/apache/mxnet/IO.scala | 35 +++--
.../scala/org/apache/mxnet/MX_PRIMITIVES.scala | 2 +
.../src/main/scala/org/apache/mxnet/Symbol.scala | 55 +-
.../scala/org/apache/mxnet/io/MXDataIter.scala | 3 +-
.../scala/org/apache/mxnet/io/NDArrayIter.scala| 3 +-
.../org/apache/mxnet/io/PrefetchingIter.scala | 87 +++---
.../scala/org/apache/mxnet/module/BaseModule.scala | 3 +-
.../org/apache/mxnet/module/BucketingModule.scala | 6 +-
.../scala/org/apache/mxnet/module/Module.scala | 4 +-
.../org/apache/mxnet/module/SequentialModule.scala | 16 ++--
.../org/apache/mxnet/util/OptionConversion.scala | 2 +
.../src/test/scala/org/apache/mxnet/IOSuite.scala | 29
.../test/scala/org/apache/mxnet/ModuleSuite.scala | 4 +-
.../scala/org/apache/mxnet/train/ConvSuite.scala | 1 +
.../CNNTextClassification.scala| 2 +-
.../mxnetexamples/customop/ExampleCustomOp.scala | 2 +-
.../customop/ExampleCustomOpWithRtc.scala | 2 +-
.../datasets/SyntheticDataIter.scala | 2 +-
.../org/apache/mxnetexamples/module/MnistMlp.scala | 2 +-
.../mxnetexamples/module/SequentialModuleEx.scala | 2 +-
.../mxnetexamples/multitask/ExampleMultiTask.scala | 11 +--
.../mxnetexamples/neuralstyle/NeuralStyle.scala| 4 +-
.../neuralstyle/end2end/DataProcessing.scala | 4 +-
.../org/apache/mxnetexamples/rnn/BucketIo.scala| 4 +-
.../apache/mxnetexamples/rnn/TrainCharRnn.scala| 7 +-
.../benchmark/ScalaInferenceBenchmarkSuite.scala | 1 +
.../CNNClassifierExampleSuite.scala| 1 +
.../customop/CustomOpExampleSuite.scala| 1 +
.../apache/mxnetexamples/gan/GanExampleSuite.scala | 1 +
.../IMClassificationExampleSuite.scala | 1 +
.../ImageClassifierExampleSuite.scala | 1 +
.../ObjectDetectorExampleSuite.scala | 1 +
.../neuralstyle/NeuralStyleSuite.scala | 1 +
.../apache/mxnetexamples/rnn/ExampleRNNSuite.scala | 1 +
.../org/apache/mxnet/infer/ClassifierSuite.scala | 4 +-
.../scala/org/apache/mxnet/GeneratorBase.scala | 2 +-
scala-package/pom.xml | 4 +
.../apache/mxnet/spark/io/LabeledPointIter.scala | 4 +-
.../mxnet/spark/io/LongLivingDataBatch.scala | 3 +-
.../org/apache/mxnet/spark/io/PointIter.scala | 4 +-
.../org/apache/mxnet/spark/MXNetGeneralSuite.scala | 1 +
43 files changed, 212 insertions(+), 152 deletions(-)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/ExecutorManager.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/ExecutorManager.scala
index b13741b..d94b8fb 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/ExecutorManager.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/ExecutorManager.scala
@@ -395,8 +395,8 @@ private[mxnet] object ExecutorManager {
* @param paramNames Names of all trainable parameters.
* @param ctx List of devices for training (data parallel)
* @param slices Describes how the data parallel splits data into different
devices.
- * @param providedData training data shapes
- * @param providedLabel training label shapes
+ * @param providedDataDesc training data descriptions
+ * @param providedLabelDesc training label descriptions
* @param sharedGroup: DataParallelExecutorGroup
* An existing executor group, if to share parameters with
it.
*
@@ -404,8 +404,8 @@ private[mxnet] object ExecutorManager {
private class DataParallelExecutorGroup private(sym: Symbol,
argNames: IndexedSeq[String], paramNames:
Set[String],
ctx: Array[Context], private val slices:
Array[(Int, Int)],
-providedData: Map[String, Shape],
-providedLabel: Map[String, Shape],
+providedDataDesc: IndexedSeq[DataDesc],
+providedLabelDesc: IndexedSeq[DataDesc],
sharedGroup: DataParallelExecutorGroup) {
// make sure the architecture is valid
ExecutorManager.checkArguments
[incubator-mxnet] branch master updated: [MXNET-1377] Add static-dependencies licenses (#14726)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 818be02 [MXNET-1377] Add static-dependencies licenses (#14726)
818be02 is described below
commit 818be027f9bddd701b2cab9cae9df0e07fdf0ccc
Author: Lanking
AuthorDate: Fri Apr 19 09:57:26 2019 -0700
[MXNET-1377] Add static-dependencies licenses (#14726)
* add static-dependencies licenses
* add rat-exclude
* kill licenses copying
---
.../assembly/src/main/assembly/assembly.xml| 7 +-
.../nightly/apache_rat_license_check/rat-excludes | 1 +
tools/dependencies/LICENSE.binary.dependencies | 289 +
tools/dependencies/eigen.sh| 1 +
tools/staticbuild/build.sh | 7 +
tools/staticbuild/build_lib.sh | 7 -
6 files changed, 302 insertions(+), 10 deletions(-)
diff --git a/scala-package/assembly/src/main/assembly/assembly.xml
b/scala-package/assembly/src/main/assembly/assembly.xml
index 5a93136..b463ac1 100644
--- a/scala-package/assembly/src/main/assembly/assembly.xml
+++ b/scala-package/assembly/src/main/assembly/assembly.xml
@@ -66,10 +66,11 @@
lib/native
- ${MXNET_DIR}/3rdparty
+ ${MXNET_DIR}/licenses
-cub/LICENSE.TXT
-
mkldnn/external/mklml_mac_2019.0.1.20180928/license.txt
+LICENSE.binary.dependencies
+NOTICE
+LICENSE
.
diff --git a/tests/nightly/apache_rat_license_check/rat-excludes
b/tests/nightly/apache_rat_license_check/rat-excludes
index b3f64c2..c08a281 100755
--- a/tests/nightly/apache_rat_license_check/rat-excludes
+++ b/tests/nightly/apache_rat_license_check/rat-excludes
@@ -49,3 +49,4 @@ include/*
.*.json.ref
searchtools_custom.js
theme.conf
+LICENSE.binary.dependencies
diff --git a/tools/dependencies/LICENSE.binary.dependencies
b/tools/dependencies/LICENSE.binary.dependencies
new file mode 100644
index 000..c9202ba
--- /dev/null
+++ b/tools/dependencies/LICENSE.binary.dependencies
@@ -0,0 +1,289 @@
+
+Apache MXNET (incubating) Dependencies:
+
+The Apache MXNET (incubating) project contains dependencies with separate
copyright
+notices and license terms. Your use of these dependencies is subject to
the terms and conditions of the following licenses.
+
+
===
+MIT licenses
+
===
+
+1. cityhash - For details, see
https://github.com/google/cityhash/blob/master/COPYING
+ Copyright (c) 2011 Google, Inc.
+2. protobuf - For details, see
https://github.com/protocolbuffers/protobuf/blob/master/LICENSE
+Copyright 2008 Google Inc. All rights reserved.
+
+
===
+Mozilla Public License Version 2.0
+
===
+
+1. eigen - For details, see
https://github.com/eigenteam/eigen-git-mirror/blob/master/COPYING.MPL2
+Source code available at: https://github.com/eigenteam/eigen-git-mirror
+Copyright (c) 2006-2019 various contributors
+
+
===
+2-clause BSD licenses
+
===
+
+1. lz4 - For details, see https://github.com/lz4/lz4/blob/dev/LICENSE
+Copyright (c) 2011-2016, Yann Collet
+
+
===
+3-clause BSD licenses
+
===
+
+1. openblas - For details, see
https://github.com/xianyi/OpenBLAS/blob/develop/LICENSE
+Copyright (c) 2011-2014, The OpenBLAS Project
+
+2. opencv - For details, see
https://github.com/opencv/opencv/blob/master/LICENSE
+Copyright (C) 2000-2019, Intel Corporation, all rights reserved.
+Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
+Copyright (C) 2009-2016, NVIDIA Corporation, all rights reserved.
+Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights
reserved.
+Copyright (C) 2015-2016, OpenCV Foundation, all rights reserved.
+Copyright (C) 2015-2016, Itseez Inc., all rights reserved.
+
+
===
+Apache-2.0 licenses
+
===
+
+1. openssl - For details, see
https
[incubator-mxnet] branch master updated: Add vim-nox to ci/docker/install/ubuntu_core.sh (#14632)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new ff04de0 Add vim-nox to ci/docker/install/ubuntu_core.sh (#14632) ff04de0 is described below commit ff04de0a79a8f0c03948b304a02cfe25a7be0e00 Author: Pedro Larroy AuthorDate: Tue Apr 16 15:07:34 2019 -0700 Add vim-nox to ci/docker/install/ubuntu_core.sh (#14632) --- ci/docker/install/ubuntu_core.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/docker/install/ubuntu_core.sh b/ci/docker/install/ubuntu_core.sh index 61a4637..3cb806e 100755 --- a/ci/docker/install/ubuntu_core.sh +++ b/ci/docker/install/ubuntu_core.sh @@ -45,6 +45,7 @@ apt-get install -y \ software-properties-common \ sudo \ unzip \ +vim-nox \ wget # Use libturbojpeg package as it is correctly compiled with -fPIC flag
[incubator-mxnet] branch master updated: Fix scaladoc scalastyle violations in Infer package (#14671)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 6a93bda Fix scaladoc scalastyle violations in Infer package (#14671)
6a93bda is described below
commit 6a93bda8e9e92bc635f66e5a379566e52592b593
Author: Zach Kimberg
AuthorDate: Thu Apr 11 15:57:49 2019 -0700
Fix scaladoc scalastyle violations in Infer package (#14671)
---
.../scala/org/apache/mxnet/infer/Classifier.scala | 17 +
.../org/apache/mxnet/infer/ImageClassifier.scala | 36 +++---
.../org/apache/mxnet/infer/MXNetHandler.scala | 14
.../org/apache/mxnet/infer/ObjectDetector.scala| 19 +
.../scala/org/apache/mxnet/infer/Predictor.scala | 38 +-
.../mxnet/infer/javaapi/ObjectDetector.scala | 80 ++
.../org/apache/mxnet/infer/javaapi/Predictor.scala | 52 +++---
7 files changed, 187 insertions(+), 69 deletions(-)
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Classifier.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Classifier.scala
index bf65815..38fdc00 100644
--- a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Classifier.scala
+++ b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Classifier.scala
@@ -31,6 +31,7 @@ trait ClassifierBase {
/**
* Takes an array of floats and returns corresponding (Label, Score) tuples
+* @tparam T The Scala equivalent of the DType used for the input array and
return value
* @param inputIndexed sequence one-dimensional array of
floats/doubles
* @param topK (Optional) How many result (sorting based on the
last axis)
* elements to return. Default returns unsorted
output.
@@ -167,6 +168,12 @@ class Classifier(modelPathPrefix: String,
result.toIndexedSeq
}
+ /**
+* Gives the path to the standard location of the synset.txt file
+* @throws IllegalArgumentException Thrown when the file does not exist
+* @param modelPathPrefix The path to the model directory
+* @return The path to the synset.txt file
+*/
private[infer] def getSynsetFilePath(modelPathPrefix: String): String = {
val dirPath = modelPathPrefix.substring(0, 1 +
modelPathPrefix.lastIndexOf(File.separator))
val d = new File(dirPath)
@@ -179,6 +186,11 @@ class Classifier(modelPathPrefix: String,
s.getCanonicalPath
}
+ /**
+* Parses the labels from a synset file
+* @param synsetFilePath The path to the synset file. Can be gotten from
getSynsetFilePath
+* @return A IndexedSeq of each element in the file
+*/
private[infer] def readSynsetFile(synsetFilePath: String):
IndexedSeq[String] = {
val f = io.Source.fromFile(synsetFilePath)
try {
@@ -188,6 +200,11 @@ class Classifier(modelPathPrefix: String,
}
}
+ /**
+* Creates a predictor with the same modelPath, inputDescriptors, contexts,
+* and epoch as the classifier
+* @return The new Predictor
+*/
private[infer] def getPredictor(): PredictBase = {
new Predictor(modelPathPrefix, inputDescriptors, contexts, epoch)
}
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
index 99c0432..fb5f39f 100644
---
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
+++
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
@@ -66,6 +66,10 @@ class ImageClassifier(modelPathPrefix: String,
protected[infer] val height = inputShape(inputLayout.indexOf('H'))
protected[infer] val width = inputShape(inputLayout.indexOf('W'))
+ /**
+* Get the names and shapes that would be returns by a classify call
+* @return a list of (name, shape) tuples
+*/
def outputShapes: IndexedSeq[(String, Shape)] = predictor.outputShapes
/**
@@ -127,6 +131,19 @@ class ImageClassifier(modelPathPrefix: String,
result
}
+ /**
+* Creates a Classifier
+*
+* @param modelPathPrefixPath prefix from where to load the model
artifacts.
+* These include the symbol, parameters, and
synset.txt.
+* Example: file://model-dir/resnet-152
(containing
+* resnet-152-symbol.json,
resnet-152-.params, and synset.txt).
+* @param inputDescriptors Descriptors defining the input node names,
shape,
+* layout and type parameters
+* @param contexts Device contexts on which you want to run
inference; defaults to CPU
+* @param epoch Model epoch to load; defaults to 0
+* @return A Classifier to perform inference
[incubator-mxnet] branch master updated: [MXNET-1287] Up scala comp (#14667)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 26b14bc [MXNET-1287] Up scala comp (#14667)
26b14bc is described below
commit 26b14bca0ace33b0f2767a71f7dfed52cef9c631
Author: Zach Kimberg
AuthorDate: Thu Apr 11 13:21:31 2019 -0700
[MXNET-1287] Up scala comp (#14667)
* Upgrade scala compiler
* Fix scala version mismatch
* Fix no source root warnings
* Misc compiler fixes
---
.../assembly/src/main/assembly/assembly.xml| 6 +--
.../assembly/src/main/assembly/javadoc.xml | 2 +-
.../assembly/src/main/assembly/source.xml | 2 +-
scala-package/core/pom.xml | 5 +--
scala-package/deploy/src/main/deploy/deploy.xml| 2 +-
scala-package/examples/pom.xml | 4 --
scala-package/infer/pom.xml| 4 --
scala-package/macros/pom.xml | 4 --
scala-package/pom.xml | 46 --
scala-package/spark/pom.xml| 5 +++
10 files changed, 38 insertions(+), 42 deletions(-)
diff --git a/scala-package/assembly/src/main/assembly/assembly.xml
b/scala-package/assembly/src/main/assembly/assembly.xml
index 3a8e11b..5a93136 100644
--- a/scala-package/assembly/src/main/assembly/assembly.xml
+++ b/scala-package/assembly/src/main/assembly/assembly.xml
@@ -33,8 +33,8 @@
org.slf4j:slf4j-api
args4j:args4j
- /
- true
+ .
+ false
true
runtime
@@ -71,7 +71,7 @@
cub/LICENSE.TXT
mkldnn/external/mklml_mac_2019.0.1.20180928/license.txt
- /
+ .
diff --git a/scala-package/assembly/src/main/assembly/javadoc.xml
b/scala-package/assembly/src/main/assembly/javadoc.xml
index 176fa43..c6df96a 100644
--- a/scala-package/assembly/src/main/assembly/javadoc.xml
+++ b/scala-package/assembly/src/main/assembly/javadoc.xml
@@ -25,7 +25,7 @@
${rootdir}/core/target/site/scaladocs
- /
+ .
diff --git a/scala-package/assembly/src/main/assembly/source.xml
b/scala-package/assembly/src/main/assembly/source.xml
index c067861..1f004e8 100644
--- a/scala-package/assembly/src/main/assembly/source.xml
+++ b/scala-package/assembly/src/main/assembly/source.xml
@@ -29,7 +29,7 @@
**\/*.scala
- /
+ .
diff --git a/scala-package/core/pom.xml b/scala-package/core/pom.xml
index 5a28a11..b888b55 100644
--- a/scala-package/core/pom.xml
+++ b/scala-package/core/pom.xml
@@ -102,10 +102,6 @@
org.apache.maven.plugins
-maven-compiler-plugin
-
-
-org.apache.maven.plugins
maven-clean-plugin
3.1.0
@@ -141,6 +137,7 @@
+
org.apache.mxnet
diff --git a/scala-package/deploy/src/main/deploy/deploy.xml
b/scala-package/deploy/src/main/deploy/deploy.xml
index 7265e7d..f5b6248 100644
--- a/scala-package/deploy/src/main/deploy/deploy.xml
+++ b/scala-package/deploy/src/main/deploy/deploy.xml
@@ -38,7 +38,7 @@
org.scala-lang.modules
scala-parser-combinators_2.11
- 1.0.4
+ 1.0.5
org.scala-lang
diff --git a/scala-package/examples/pom.xml b/scala-package/examples/pom.xml
index d60782f..2575291 100644
--- a/scala-package/examples/pom.xml
+++ b/scala-package/examples/pom.xml
@@ -83,10 +83,6 @@
maven-jar-plugin
-org.apache.maven.plugins
-maven-compiler-plugin
-
-
net.alchim31.maven
scala-maven-plugin
diff --git a/scala-package/infer/pom.xml b/scala-package/infer/pom.xml
index ed90d80..81e9393 100644
--- a/scala-package/infer/pom.xml
+++ b/scala-package/infer/pom.xml
@@ -47,10 +47,6 @@
-org.apache.maven.plugins
-maven-compiler-plugin
-
-
org.scalatest
scalatest-maven-plugin
diff --git a/scala-package/macros/pom.xml b/scala-package/macros/pom.xml
index 91a13d0..cb84824 100644
--- a/scala-package/macros/pom.xml
+++ b/scala-package/macros/pom.xml
@@ -52,10 +52,6 @@
-org.apache.maven.plugins
-maven-compiler-plugin
-
-
org.scalatest
scalatest-maven-plugin
diff --git a/scala-package/pom.xml b/scala-package/pom.xml
index 62def16..44b784a 100644
--- a/scala-package/pom.xml
+++ b/scala-package/pom.xml
@@ -53,8 +53,11 @@
+1.7
2.11.8
+2.11
+
g++
$
${project.basedir}/..
@@ -252,9 +255,11 @@
maven-compiler-plugin
3.3
- 1.7
- 1.7
+ ${java.version}
+ ${java.version
[incubator-mxnet] branch master updated: Avoid secondary deployment of package to local (#14647)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new e701e71 Avoid secondary deployment of package to local (#14647) e701e71 is described below commit e701e7100c31b63e8f3b93008872606e921ec6ad Author: Zach Kimberg AuthorDate: Thu Apr 11 10:09:07 2019 -0700 Avoid secondary deployment of package to local (#14647) --- scala-package/deploy/pom.xml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scala-package/deploy/pom.xml b/scala-package/deploy/pom.xml index 74b5707..b4dd273 100644 --- a/scala-package/deploy/pom.xml +++ b/scala-package/deploy/pom.xml @@ -150,6 +150,9 @@ + +true +
[incubator-mxnet] branch master updated: Fix scalastyle (#14669)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 52e2e8e Fix scalastyle (#14669)
52e2e8e is described below
commit 52e2e8ed630886b5e8edce20b960fa7d62620287
Author: Zach Kimberg
AuthorDate: Thu Apr 11 10:07:52 2019 -0700
Fix scalastyle (#14669)
---
.../main/scala/org/apache/mxnet/FeedForward.scala | 15 +-
.../org/apache/mxnet/module/BucketingModule.scala | 54
.../scala/org/apache/mxnet/module/Module.scala | 153 ++---
.../org/apache/mxnet/module/SequentialModule.scala | 135 +-
.../org/apache/mxnetexamples/rnn/BucketIo.scala| 8 +-
.../org/apache/mxnet/spark/utils/Network.scala | 22 ++-
.../apache/mxnet/spark/SharedSparkContext.scala| 12 +-
7 files changed, 194 insertions(+), 205 deletions(-)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/FeedForward.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/FeedForward.scala
index 2ed9d8c..2b17655 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/FeedForward.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/FeedForward.scala
@@ -180,6 +180,7 @@ class FeedForward private(
// Initialize the predictor module for running prediction.
private def initPredictor(inputShapes: Map[String, Shape]): Unit = {
+var shouldInit = true
if (this.predExec != null) {
val (argShapes, _, _) = symbol.inferShape(inputShapes)
require(argShapes != null, "Shape inference failed." +
@@ -187,14 +188,16 @@ class FeedForward private(
s"and aux states ${symbol.listAuxiliaryStates()}")
val predShapes = this.predExec.argArrays.map(_.shape)
if (argShapes.sameElements(predShapes)) {
-return
+shouldInit = false
}
}
-// for now only use the first device
-val predExec = symbol.simpleBind(ctx(0), gradReq = "null", shapeDict =
inputShapes)
-predExec.copyParamsFrom(_argParams, _auxParams)
-ExecutorManager.checkArguments(symbol)
-this.predExec = predExec
+if(shouldInit) {
+ // for now only use the first device
+ val predExec = symbol.simpleBind(ctx(0), gradReq = "null", shapeDict =
inputShapes)
+ predExec.copyParamsFrom(_argParams, _auxParams)
+ ExecutorManager.checkArguments(symbol)
+ this.predExec = predExec
+}
}
// Initialize the iterator given input.
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/module/BucketingModule.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/module/BucketingModule.scala
index 1ac798e..41a6f69 100644
---
a/scala-package/core/src/main/scala/org/apache/mxnet/module/BucketingModule.scala
+++
b/scala-package/core/src/main/scala/org/apache/mxnet/module/BucketingModule.scala
@@ -173,14 +173,13 @@ class BucketingModule(symGen: AnyRef => (Symbol,
IndexedSeq[String], IndexedSeq[
allowMissing: Boolean = false,
forceInit: Boolean = false,
allowExtra: Boolean = false): Unit = {
-if (paramsInitialized && !forceInit) {
- return
+if (!paramsInitialized || forceInit) {
+ require(binded, "call bind before initializing the parameters")
+ this._currModule.initParams(initializer, argParams, auxParams,
+allowMissing, forceInit, allowExtra)
+ this.paramsDirty = false
+ this.paramsInitialized = true
}
-require(binded, "call bind before initializing the parameters")
-this._currModule.initParams(initializer, argParams, auxParams,
- allowMissing, forceInit, allowExtra)
-this.paramsDirty = false
-this.paramsInitialized = true
}
/**
@@ -218,28 +217,27 @@ class BucketingModule(symGen: AnyRef => (Symbol,
IndexedSeq[String], IndexedSeq[
if (this.binded) {
logger.warn("Already bound, ignoring bind()")
- return
-}
+} else {
+ require(sharedModule.isEmpty,
+"sharedModule for BucketingModule is not supported")
-require(sharedModule.isEmpty,
- "sharedModule for BucketingModule is not supported")
-
-this.forTraining = forTraining
-this.inputsNeedGrad = inputsNeedGrad
-this.binded = true
-
-val (sym, dNames, lNames) = this.symGen(this.defaultBucketKey)
-val module = new Module(sym, dNames, lNames, this.contexts,
- this.workLoadList, this.fixedParamNames)
-module.bind(dataShapes, labelShapes, forTraining, inputsNeedGrad,
- forceRebind = false, sharedModule = None, gradReq)
-this._currModule = module
-this._currBucketKey = this.defaultBucketKey
-this._buckets(this.defaultBucketKey) = module
-
-// copy back saved params, if already initialize
[incubator-mxnet] branch master updated: [MXNET-1287] Miscellaneous Scala warning fixes (#14658)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 8f0878c [MXNET-1287] Miscellaneous Scala warning fixes (#14658)
8f0878c is described below
commit 8f0878c9b424aa9a1f575d22ce9b20e4e4254b93
Author: Zach Kimberg
AuthorDate: Wed Apr 10 14:02:42 2019 -0700
[MXNET-1287] Miscellaneous Scala warning fixes (#14658)
* Remove printing during tests
* Fix macro annotation warnings
* Remove failing links from spark
* Return printed as debug logs
---
.../core/src/test/scala/org/apache/mxnet/ImageSuite.scala | 10 +-
.../src/test/scala/org/apache/mxnet/KVStoreSuite.scala| 3 ---
.../test/scala/org/apache/mxnet/ModelParallelSuite.scala | 5 -
.../src/test/scala/org/apache/mxnet/OperatorSuite.scala | 6 --
.../src/test/scala/org/apache/mxnet/SymbolSuite.scala | 14 --
.../src/main/scala/org/apache/mxnet/GeneratorBase.scala | 2 +-
.../src/main/scala/org/apache/mxnet/NDArrayMacro.scala| 15 ---
.../src/main/scala/org/apache/mxnet/SymbolMacro.scala | 15 ---
.../scala/org/apache/mxnet/javaapi/JavaNDArrayMacro.scala | 8 +++-
.../src/main/scala/org/apache/mxnet/spark/MXNDArray.scala | 2 +-
.../main/scala/org/apache/mxnet/spark/MXNetModel.scala| 2 +-
11 files changed, 27 insertions(+), 55 deletions(-)
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/ImageSuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/ImageSuite.scala
index d4cf35a..dca4ce0 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/ImageSuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/ImageSuite.scala
@@ -57,7 +57,7 @@ class ImageSuite extends FunSuite with BeforeAndAfterAll {
test("Test load image") {
val nd = Image.imRead(imLocation)
-logger.info(s"OpenCV load image with shape: ${nd.shape}")
+logger.debug(s"OpenCV load image with shape: ${nd.shape}")
require(nd.shape == Shape(576, 1024, 3), "image shape not Match!")
}
@@ -65,14 +65,14 @@ class ImageSuite extends FunSuite with BeforeAndAfterAll {
val url = new
URL("https://s3.amazonaws.com/model-server/inputs/Pug-Cookie.jpg;)
val inputStream = url.openStream
val nd = Image.imDecode(inputStream)
-logger.info(s"OpenCV load image with shape: ${nd.shape}")
+logger.debug(s"OpenCV load image with shape: ${nd.shape}")
require(nd.shape == Shape(576, 1024, 3), "image shape not Match!")
}
test("Test resize image") {
val nd = Image.imRead(imLocation)
val resizeIm = Image.imResize(nd, 224, 224)
-logger.info(s"OpenCV resize image with shape: ${resizeIm.shape}")
+logger.debug(s"OpenCV resize image with shape: ${resizeIm.shape}")
require(resizeIm.shape == Shape(224, 224, 3), "image shape not Match!")
}
@@ -94,7 +94,7 @@ class ImageSuite extends FunSuite with BeforeAndAfterAll {
val tempDirPath = System.getProperty("java.io.tmpdir")
val img = Image.toImage(resizeIm)
ImageIO.write(img, "png", new File(tempDirPath + "/inputImages/out.png"))
-logger.info(s"converted image stored in ${tempDirPath +
"/inputImages/out.png"}")
+logger.debug(s"converted image stored in ${tempDirPath +
"/inputImages/out.png"}")
}
test("Test draw Bounding box") {
@@ -107,7 +107,7 @@ class ImageSuite extends FunSuite with BeforeAndAfterAll {
Image.drawBoundingBox(buf, box, Some(names), fontSizeMult = Some(1.4f))
val tempDirPath = System.getProperty("java.io.tmpdir")
ImageIO.write(buf, "png", new File(tempDirPath + "/inputImages/out2.png"))
-logger.info(s"converted image stored in ${tempDirPath +
"/inputImages/out2.png"}")
+logger.debug(s"converted image stored in ${tempDirPath +
"/inputImages/out2.png"}")
for (coord <- box) {
val topLeft = buf.getRGB(coord("xmin"), coord("ymin"))
val downLeft = buf.getRGB(coord("xmin"), coord("ymax"))
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/KVStoreSuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/KVStoreSuite.scala
index 2e1b36c..3eb6141 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/KVStoreSuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/KVStoreSuite.scala
@@ -67,9 +67,6 @@ class KVStoreSuite extends FunSuite with BeforeAndAfterAll {
val kv = KVStore.create()
val updater = new MXKVStoreUpdater {
override def update(key: Int, input: NDArray, stored: NDArr
[incubator-mxnet] branch master updated: Comment out test_unix_python3_tensorrt_gpu step (#14642)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 1dafa1a Comment out test_unix_python3_tensorrt_gpu step (#14642)
1dafa1a is described below
commit 1dafa1a79dafcdcfbafe38f034c7285c5739e427
Author: Chance Bair
AuthorDate: Mon Apr 8 19:08:51 2019 +0200
Comment out test_unix_python3_tensorrt_gpu step (#14642)
---
ci/jenkins/Jenkinsfile_unix_gpu | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/ci/jenkins/Jenkinsfile_unix_gpu b/ci/jenkins/Jenkinsfile_unix_gpu
index 664e591..f6191de 100644
--- a/ci/jenkins/Jenkinsfile_unix_gpu
+++ b/ci/jenkins/Jenkinsfile_unix_gpu
@@ -50,7 +50,8 @@ core_logic: {
custom_steps.test_unix_python2_mkldnn_gpu(),
custom_steps.test_unix_python3_mkldnn_gpu(),
custom_steps.test_unix_python3_mkldnn_nocudnn_gpu(),
-custom_steps.test_unix_python3_tensorrt_gpu(),
+// Disabled temporarily for
https://github.com/apache/incubator-mxnet/issues/14626
+//custom_steps.test_unix_python3_tensorrt_gpu(),
custom_steps.test_unix_perl_gpu(),
custom_steps.test_unix_r_gpu(),
custom_steps.test_unix_cpp_gpu(),
[incubator-mxnet] branch v1.4.x updated: Memory fixes. Resolves #10867, and resolves #14080 (#14372) (#14586)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.4.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.4.x by this push:
new c15cbaf Memory fixes. Resolves #10867, and resolves #14080 (#14372)
(#14586)
c15cbaf is described below
commit c15cbafd11258672fbc6a32c0a40d489dd035c62
Author: Lanking
AuthorDate: Fri Apr 5 13:40:03 2019 -0700
Memory fixes. Resolves #10867, and resolves #14080 (#14372) (#14586)
* Fixes for memory leak when reshaping executor
* Fixed Adam Optimizer memory leak
* Cleanup for PR
* Added unit test for new ResourceScope method
* Removing import that was added by overzealous ide
* Add back in an import
* Added flags for executor to know whether or not it owns NDArrays for
disposal
* Moving to ResourceScope.using implementation
* Changes to make ResourceScope.using work with existing scope
* Updating ResourceScope to work with existing scopes via
usingIfScopeExists method
* Fix clojure unit tests
* Fixes to be compatibile with how clojure is using ResourceScope
* Removing some unnecessary changes
* Adding scope assertion in unit test
---
.../src/main/scala/org/apache/mxnet/Executor.scala | 47 --
.../main/scala/org/apache/mxnet/Optimizer.scala| 20 ++--
.../scala/org/apache/mxnet/ResourceScope.scala | 21 -
.../src/main/scala/org/apache/mxnet/Symbol.scala | 21 +++--
.../mxnet/module/DataParallelExecutorGroup.scala | 11 ++-
.../scala/org/apache/mxnet/optimizer/Adam.scala| 101 ++---
.../org/apache/mxnet/ResourceScopeSuite.scala | 33 +++
7 files changed, 165 insertions(+), 89 deletions(-)
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
index 85f45bc..aec4402 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Executor.scala
@@ -45,29 +45,47 @@ object Executor {
* @see Symbol.bind : to create executor
*/
class Executor private[mxnet](private[mxnet] val handle: ExecutorHandle,
- private[mxnet] val symbol: Symbol) extends
NativeResource {
- private[mxnet] var argArrays: Array[NDArray] = null
- private[mxnet] var gradArrays: Array[NDArray] = null
- private[mxnet] var auxArrays: Array[NDArray] = null
+ private[mxnet] val symbol: Symbol,
+ private[mxnet] var argArrays: Array[NDArray] =
null,
+ private[mxnet] var gradArrays: Array[NDArray] =
null,
+ private[mxnet] var auxArrays: Array[NDArray] =
null,
+ private var _ctx: Context = null,
+ private var _gradsReq: Iterable[_] = null,
+ private var _group2ctx: Map[String, Context] =
null
+ ) extends NativeResource {
+
val outputs: Array[NDArray] = getOutputs
protected var _argDict: Map[String, NDArray] = null
protected var _gradDict: Map[String, NDArray] = null
protected var _auxDict: Map[String, NDArray] = null
protected var monitorCallback: MXMonitorCallback = null
- private[mxnet] var _ctx: Context = null
- private[mxnet] var _gradsReq: Iterable[_] = null
- private[mxnet] var _group2ctx: Map[String, Context] = null
private val logger: Logger = LoggerFactory.getLogger(classOf[Executor])
+ private[mxnet] var ownsArgArrays = false
+ private[mxnet] var ownsGradArrays = false
+ private[mxnet] var ownsAuxArrays = false
+
override def nativeAddress: CPtrAddress = handle
override def nativeDeAllocator: (CPtrAddress => Int) = _LIB.mxExecutorFree
// cannot determine the off-heap size of this object
override val bytesAllocated: Long = 0
override val ref: NativeResourceRef = super.register()
+
override def dispose(): Unit = {
if (!super.isDisposed) {
super.dispose()
outputs.foreach(o => o.dispose())
+ // Symbol.bind clones symbol when creating the executor so we need to
dispose of the clone
+ symbol.dispose()
+ if (ownsArgArrays && argArrays != null) {argArrays.foreach(a =>
a.dispose())}
+ if (ownsGradArrays && gradArrays != null) {gradArrays.foreach(
+// Symbol will sometimes fill this with nulls so we've got to check
the elements too
+a => if (a != null) {a.dispose()})
+ }
+ if (ownsAuxArrays && auxArrays != null) {auxArrays.foreach(a =>
a.dispose())}
+ if (_argDict != null) {_argDict.foreach(a => a._2.dispose())}
+ if (_gradDict != null) {_gradDict.foreach(a => a._2.dispose())}
+ if (_auxDict
[incubator-mxnet] branch master updated: Add BERT QA Scala/Java example (#14592)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new d5d1d7a Add BERT QA Scala/Java example (#14592)
d5d1d7a is described below
commit d5d1d7ac417ed75b4c2c898d653a210edec046b9
Author: Lanking
AuthorDate: Fri Apr 5 09:26:25 2019 -0700
Add BERT QA Scala/Java example (#14592)
* add BertQA major code piece
* add scripts and bug fixes
* add integration test
* address comments
* address doc comments
---
.../scala/org/apache/mxnet/javaapi/Layout.scala| 34 +
scala-package/examples/pom.xml | 5 +
.../examples/scripts/infer/bert/get_bert_data.sh | 31 +
.../scripts/infer/bert/run_bert_qa_example.sh | 27
.../javaapi/infer/bert/BertDataParser.java | 126 ++
.../mxnetexamples/javaapi/infer/bert/BertQA.java | 148 +
.../mxnetexamples/javaapi/infer/bert/README.md | 103 ++
.../javaapi/infer/predictor/BertExampleTest.java | 71 ++
8 files changed, 545 insertions(+)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Layout.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Layout.scala
new file mode 100644
index 000..cfe290c
--- /dev/null
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Layout.scala
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.mxnet.javaapi
+
+/**
+ * Layout definition of DataDesc
+ * N Batch size
+ * C channels
+ * H Height
+ * W Weight
+ * T sequence length
+ * __undefined__ default value of Layout
+ */
+object Layout {
+ val UNDEFINED: String = org.apache.mxnet.Layout.UNDEFINED
+ val NCHW: String = org.apache.mxnet.Layout.NCHW
+ val NTC: String = org.apache.mxnet.Layout.NTC
+ val NT: String = org.apache.mxnet.Layout.NT
+ val N: String = org.apache.mxnet.Layout.N
+}
diff --git a/scala-package/examples/pom.xml b/scala-package/examples/pom.xml
index 07e4301..d60782f 100644
--- a/scala-package/examples/pom.xml
+++ b/scala-package/examples/pom.xml
@@ -145,5 +145,10 @@
slf4j-simple
1.7.5
+
+ com.google.code.gson
+ gson
+ 2.8.5
+
diff --git a/scala-package/examples/scripts/infer/bert/get_bert_data.sh
b/scala-package/examples/scripts/infer/bert/get_bert_data.sh
new file mode 100755
index 000..609aae2
--- /dev/null
+++ b/scala-package/examples/scripts/infer/bert/get_bert_data.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+
+MXNET_ROOT=$(cd "$(dirname $0)/../../.."; pwd)
+
+data_path=$MXNET_ROOT/scripts/infer/models/static-bert-qa/
+
+if [ ! -d "$data_path" ]; then
+ mkdir -p "$data_path"
+ curl
https://s3.us-east-2.amazonaws.com/mxnet-scala/scala-example-ci/BertQA/vocab.json
-o $data_path/vocab.json
+ curl
https://s3.us-east-2.amazonaws.com/mxnet-scala/scala-example-ci/BertQA/static_bert_qa-0002.params
-o $data_path/static_bert_qa-0002.params
+ curl
https://s3.us-east-2.amazonaws.com/mxnet-scala/scala-example-ci/BertQA/static_bert_qa-symbol.json
-o $data_path/static_bert_qa-symbol.json
+fi
diff --git a/scala-package/examples/scripts/infer/be
[incubator-mxnet] branch master updated: added note about cuda9.2 requirement (#14140)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 43f7c12 added note about cuda9.2 requirement (#14140) 43f7c12 is described below commit 43f7c12cfc5079d950200b14aec16eddd34455c5 Author: Sam Skalicky AuthorDate: Thu Apr 4 11:05:34 2019 -0700 added note about cuda9.2 requirement (#14140) --- scala-package/mxnet-demo/java-demo/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scala-package/mxnet-demo/java-demo/README.md b/scala-package/mxnet-demo/java-demo/README.md index cad52cb..3e742fc 100644 --- a/scala-package/mxnet-demo/java-demo/README.md +++ b/scala-package/mxnet-demo/java-demo/README.md @@ -32,6 +32,8 @@ This command will pick the default values specified in the [pom](https://github. Note: If you are planning to use GPU, please add `-Dmxnet.profile=linux-x86_64-gpu` +Note: The Maven package is built with CUDA 9.2. + ### Use customized version set You can use the following instruction as an alternative to achieve the same result: You may use `mvn package` to build the package,
[incubator-mxnet] branch master updated: [MXNET-1379] update reshape operator (#14600)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new b482a44 [MXNET-1379] update reshape operator (#14600)
b482a44 is described below
commit b482a44fa8cd932ed48d62bafadb11299c7cd381
Author: Lanking
AuthorDate: Wed Apr 3 10:07:38 2019 -0700
[MXNET-1379] update reshape operator (#14600)
* update reshape operator
* Satisfy the Lint God =v=
* update the jni header signature
---
.../core/src/main/scala/org/apache/mxnet/LibInfo.scala | 5 +++--
.../core/src/main/scala/org/apache/mxnet/NDArray.scala | 13 -
.../core/src/test/scala/org/apache/mxnet/NDArraySuite.scala | 8 ++--
.../native/src/main/native/org_apache_mxnet_native_c_api.cc | 13 +++--
.../native/src/main/native/org_apache_mxnet_native_c_api.h | 8
5 files changed, 32 insertions(+), 15 deletions(-)
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
index 20b6ed9..40fc095 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/LibInfo.scala
@@ -86,9 +86,10 @@ private[mxnet] class LibInfo {
@native def mxNDArrayAt(handle: NDArrayHandle,
idx: MXUint,
out: NDArrayHandleRef): Int
- @native def mxNDArrayReshape(handle: NDArrayHandle,
+ @native def mxNDArrayReshape64(handle: NDArrayHandle,
nDim: Int,
- dims: Array[Int],
+ dims: Array[Long],
+ reverse: Boolean,
reshapeHandle: NDArrayHandleRef): Int
@native def mxNDArraySyncCopyFromCPU(handle: NDArrayHandle,
source: Array[MXFloat],
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 915e4c6..ab42265 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -950,8 +950,19 @@ class NDArray private[mxnet](private[mxnet] val handle:
NDArrayHandle,
* @return a reshaped NDArray that shares memory with current one.
*/
def reshape(dims: Array[Int]): NDArray = {
+reshape(dims.map(_.toLong))
+ }
+
+ /**
+* Return a reshaped NDArray that shares memory with current one.
+* @param dims New shape.
+* @param reverse whether to inplace reshape
+* @return a reshaped NDArray that shares memory with current one.
+*/
+ def reshape(dims: Array[Long], reverse: Option[Boolean] = None): NDArray = {
val reshapeHandle = new NDArrayHandleRef
-checkCall(_LIB.mxNDArrayReshape(handle, dims.length, dims, reshapeHandle))
+checkCall(_LIB.mxNDArrayReshape64(handle,
+ dims.length, dims, reverse.getOrElse(false), reshapeHandle))
new NDArray(handle = reshapeHandle.value, writable = this.writable)
}
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
index 206094c..c2ef641 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
@@ -878,14 +878,18 @@ class NDArraySuite extends FunSuite with
BeforeAndAfterAll with Matchers {
}
test("reshape") {
-val arr = NDArray.array(Array(1f, 2f, 3f, 4f, 5f, 6f), shape = Shape(3, 2))
+var arr = NDArray.array(Array(1f, 2f, 3f, 4f, 5f, 6f), shape = Shape(3, 2))
-val arr1 = arr.reshape(Array(2, 3))
+var arr1 = arr.reshape(Array(2, 3))
assert(arr1.shape === Shape(2, 3))
assert(arr1.toArray === Array(1f, 2f, 3f, 4f, 5f, 6f))
arr.set(1f)
assert(arr1.toArray === Array(1f, 1f, 1f, 1f, 1f, 1f))
+
+arr = NDArray.ones(1, 384, 1)
+arr1 = arr.reshape(Array(0, -3))
+assert(arr1.shape === Shape(1, 384))
}
test("dispose deps") {
diff --git
a/scala-package/native/src/main/native/org_apache_mxnet_native_c_api.cc
b/scala-package/native/src/main/native/org_apache_mxnet_native_c_api.cc
index ea6e9c8..33e4cca 100644
--- a/scala-package/native/src/main/native/org_apache_mxnet_native_c_api.cc
+++ b/scala-package/native/src/main/native/org_apache_mxnet_native_c_api.cc
@@ -404,14 +404,15 @@ JNIEXPORT jint JNICALL
Java_org_apache_mxnet_LibInfo_mxNDArrayAt
return ret;
}
-JNIEXPORT jint JNICALL Java_org_apache_mxnet_LibInfo_mxNDArrayReshape
- (JNIEnv *env, jobject obj, jlong ndArrayPtr, jint ndim, jintArray dims,
jobject reshapedHandle) {
+JN
[incubator-mxnet] branch master updated: Adds context parameter to check_rnn_layer_forward calls in test_lstmp (#14529)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 67c10f9 Adds context parameter to check_rnn_layer_forward calls in
test_lstmp (#14529)
67c10f9 is described below
commit 67c10f954d923cffcb9a932210d9ff0a2010e20b
Author: perdasilva
AuthorDate: Wed Mar 27 08:35:44 2019 +0100
Adds context parameter to check_rnn_layer_forward calls in test_lstmp
(#14529)
---
tests/python/gpu/test_gluon_gpu.py | 15 ---
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/tests/python/gpu/test_gluon_gpu.py
b/tests/python/gpu/test_gluon_gpu.py
index 88b436a..9c7 100644
--- a/tests/python/gpu/test_gluon_gpu.py
+++ b/tests/python/gpu/test_gluon_gpu.py
@@ -88,7 +88,8 @@ def test_lstmp():
rtol, atol = 1e-2, 1e-2
batch_size, seq_len = 7, 11
input_size = 5
-lstm_input = mx.nd.uniform(shape=(seq_len, batch_size, input_size),
ctx=mx.gpu(0))
+ctx = mx.gpu(0)
+lstm_input = mx.nd.uniform(shape=(seq_len, batch_size, input_size),
ctx=ctx)
shapes = {'i2h_weight': (hidden_size*4, input_size),
'h2h_weight': (hidden_size*4, projection_size),
'i2h_bias': (hidden_size*4,),
@@ -101,8 +102,8 @@ def test_lstmp():
projection_size=projection_size,
input_size=input_size,
prefix='lstm0_l0_')
-lstm_layer.initialize(ctx=mx.gpu(0))
-lstm_cell.initialize(ctx=mx.gpu(0))
+lstm_layer.initialize(ctx=ctx)
+lstm_cell.initialize(ctx=ctx)
layer_params = lstm_layer.collect_params()
cell_params = lstm_cell.collect_params()
for k, v in weights.items():
@@ -121,14 +122,14 @@ def test_lstmp():
print('checking gradient for {}'.format('lstm0_l0_'+k))
assert_almost_equal(layer_grad.asnumpy(), cell_grad.asnumpy(),
rtol=rtol, atol=atol)
-check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, projection_size=5),
mx.nd.ones((8, 3, 20)))
-check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, projection_size=5,
bidirectional=True), mx.nd.ones((8, 3, 20)), [mx.nd.ones((4, 3, 5)),
mx.nd.ones((4, 3, 10))])
+check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, projection_size=5),
mx.nd.ones((8, 3, 20)), ctx=ctx)
+check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, projection_size=5,
bidirectional=True), mx.nd.ones((8, 3, 20)), [mx.nd.ones((4, 3, 5)),
mx.nd.ones((4, 3, 10))], ctx=ctx)
check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, dropout=0.5,
projection_size=5), mx.nd.ones((8, 3, 20)),
-run_only=True)
+run_only=True, ctx=ctx)
check_rnn_layer_forward(gluon.rnn.LSTM(10, 2, bidirectional=True,
dropout=0.5, projection_size=5),
mx.nd.ones((8, 3, 20)),
-[mx.nd.ones((4, 3, 5)), mx.nd.ones((4, 3, 10))],
run_only=True)
+[mx.nd.ones((4, 3, 5)), mx.nd.ones((4, 3, 10))],
run_only=True, ctx=ctx)
@with_seed()
[incubator-mxnet] branch master updated: [MXNET-1285] Draw bounding box with Scala/Java Image API (#14474)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 092af36 [MXNET-1285] Draw bounding box with Scala/Java Image API
(#14474)
092af36 is described below
commit 092af3601bba6a24201154c3185bd3d9f39677f7
Author: Lanking
AuthorDate: Tue Mar 26 11:38:40 2019 -0700
[MXNET-1285] Draw bounding box with Scala/Java Image API (#14474)
* new feature to draw bounding box
* add Java support
* add point wise verification
* cancel the check on top-left corner
* add this example to Java world and fixing bugs
---
scala-package/.gitignore | 1 +
.../src/main/scala/org/apache/mxnet/Image.scala| 54 ++
.../scala/org/apache/mxnet/javaapi/Image.scala | 46 --
.../java/org/apache/mxnet/javaapi/ImageTest.java | 20 +++-
.../test/scala/org/apache/mxnet/ImageSuite.scala | 21 +
.../javaapi/infer/objectdetector/README.md | 2 +-
.../infer/objectdetector/SSDClassifierExample.java | 41 ++--
.../org/apache/mxnet/infer/ObjectDetector.scala| 2 +-
.../mxnet/infer/javaapi/ObjectDetectorOutput.scala | 4 +-
.../infer/javaapi/ObjectDetectorOutputTest.java| 4 +-
10 files changed, 161 insertions(+), 34 deletions(-)
diff --git a/scala-package/.gitignore b/scala-package/.gitignore
index 9bf7851..dadc000 100644
--- a/scala-package/.gitignore
+++ b/scala-package/.gitignore
@@ -9,3 +9,4 @@ core/src/main/scala/org/apache/mxnet/SymbolBase.scala
core/src/main/scala/org/apache/mxnet/SymbolRandomAPIBase.scala
examples/scripts/infer/images/
examples/scripts/infer/models/
+examples/scripts/infer/objectdetector/boundingImage.png
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
index 0f756e2..52e26ef 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
@@ -17,6 +17,7 @@
package org.apache.mxnet
// scalastyle:off
+import java.awt.{BasicStroke, Color, Graphics2D}
import java.awt.image.BufferedImage
// scalastyle:on
import java.io.InputStream
@@ -182,4 +183,57 @@ object Image {
img
}
+ /**
+* Helper function to generate ramdom colors
+* @param transparency The transparency level
+* @return Color
+*/
+ private def randomColor(transparency: Option[Float] = Some(1.0f)) : Color = {
+new Color(
+ Math.random().toFloat, Math.random().toFloat, Math.random().toFloat,
+ transparency.get
+)
+ }
+
+ /**
+* Method to draw bounding boxes for an image
+* @param src Source of the buffered image
+* @param coordinate Contains Map of xmin, xmax, ymin, ymax
+* corresponding to top-left and down-right points
+* @param names The name set of the bounding box
+* @param stroke Thickness of the bounding box
+* @param fontSizeMult Font size multiplier
+* @param transparency Transparency of the bounding box
+*/
+ def drawBoundingBox(src: BufferedImage, coordinate: Array[Map[String, Int]],
+ names: Option[Array[String]] = None,
+ stroke : Option[Int] = Some(3),
+ fontSizeMult : Option[Float] = Some(1.0f),
+ transparency: Option[Float] = Some(1.0f)): Unit = {
+val g2d : Graphics2D = src.createGraphics()
+g2d.setStroke(new BasicStroke(stroke.get))
+// Increase the size of font
+val currentFont = g2d.getFont
+val newFont = currentFont.deriveFont(currentFont.getSize *
fontSizeMult.get)
+g2d.setFont(newFont)
+// Get font metrics to draw the font box
+val fm = g2d.getFontMetrics(newFont)
+for (idx <- coordinate.indices) {
+ val map = coordinate(idx)
+ g2d.setColor(randomColor(transparency).darker())
+ g2d.drawRect(map("xmin"), map("ymin"), map("xmax") - map("xmin"),
map("ymax") - map("ymin"))
+ // Write the name of the bounding box
+ if (names.isDefined) {
+val x = map("xmin") - stroke.get
+val y = map("ymin")
+val h = fm.getHeight
+val w = fm.charsWidth(names.get(idx).toCharArray, 0,
names.get(idx).length())
+g2d.fillRect(x, y - h, w, h)
+g2d.setColor(Color.WHITE)
+g2d.drawString(names.get(idx), x, y)
+ }
+}
+g2d.dispose()
+ }
+
}
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.scala
index 7d6f31e..f72223d 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.scala
+++ b/scala-package/c
[incubator-mxnet] branch v1.3.x updated: Fixes for data links (#14525)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.3.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.3.x by this push:
new e083577 Fixes for data links (#14525)
e083577 is described below
commit e083577872acbd5227be55f85d55b28fd6009351
Author: Anton Chernov
AuthorDate: Tue Mar 26 18:22:14 2019 +0100
Fixes for data links (#14525)
* Changed data.dmlc.ml -> data.mxnet.io
* Added retries for downloads
* Removed silent mode for explicit failure reporting
---
R-package/tests/testthat/get_data.R| 33 +---
R-package/vignettes/CatsDogsFinetune.Rmd | 4 +-
.../classifyRealImageWithPretrainedModel.Rmd | 2 +-
cpp-package/example/feature_extract/run.sh | 7 ++-
cpp-package/example/get_data.sh| 61 +++---
matlab/get_inception_model.sh | 2 +-
6 files changed, 77 insertions(+), 32 deletions(-)
diff --git a/R-package/tests/testthat/get_data.R
b/R-package/tests/testthat/get_data.R
index 2676b20..924d9df 100644
--- a/R-package/tests/testthat/get_data.R
+++ b/R-package/tests/testthat/get_data.R
@@ -1,3 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
GetMNIST_ubyte <- function() {
if (!dir.exists("data")) {
@@ -45,13 +61,16 @@ GetInception <- function() {
if (!dir.exists("model")) {
dir.create("model/")
}
- if (!file.exists('model/Inception-BN-0126.params')) {
-
download.file('http://data.dmlc.ml/models/imagenet/inception-bn/Inception-BN-0126.params',
- destfile = 'model/Inception-BN-0126.params')
- }
- if (!file.exists('model/Inception-BN-symbol.json')) {
-
download.file('http://data.dmlc.ml/models/imagenet/inception-bn/Inception-BN-symbol.json',
- destfile = 'model/Inception-BN-symbol.json')
+
+ if (!file.exists("model/Inception-BN-0126.params")) {
+download.file(
+
"http://data.mxnet.io/mxnet/models/imagenet/inception-bn/Inception-BN-0126.params?raw=true;,
+destfile = "model/Inception-BN-0126.params")
+ }
+ if (!file.exists("model/Inception-BN-symbol.json")) {
+download.file(
+
"http://data.mxnet.io/mxnet/models/imagenet/inception-bn/Inception-BN-symbol.json;,
+destfile = "model/Inception-BN-symbol.json")
}
}
diff --git a/R-package/vignettes/CatsDogsFinetune.Rmd
b/R-package/vignettes/CatsDogsFinetune.Rmd
index 680b5a3..726bb1a 100644
--- a/R-package/vignettes/CatsDogsFinetune.Rmd
+++ b/R-package/vignettes/CatsDogsFinetune.Rmd
@@ -162,13 +162,13 @@ val <- data$val
## Load pretrained model
-Here we use the pretrained model from http://data.dmlc.ml/models/imagenet/.
+Here we use the pretrained model from http://data.mxnet.io/mxnet/data/.
There are 1000 classes in imagenet,
and we need to replace the last fully connected layer with a new layer for 2
classes.
```{r}
-download.file('http://data.dmlc.ml/data/Inception.zip', destfile =
'Inception.zip')
+download.file('http://data.mxnet.io/mxnet/data/Inception.zip', destfile =
'Inception.zip')
unzip("Inception.zip")
inception_bn <- mx.model.load("./Inception-BN", iteration = 126)
diff --git a/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
b/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
index ff631e0..9cfdd5a 100644
--- a/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
+++ b/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
@@ -31,7 +31,7 @@ Make sure you unzip the pre-trained model in current folder.
And we can use the
loading function to load the model into R.
```{r}
-download.file('http://data.dmlc.ml/data/Inception.zip', destfile =
'Inception.zip')
+download.file('http://data.mxnet.io/mxnet/data/Inception.zip', destfile =
'Inception.zip')
unzip("Inception.zip")
model <- mx.model.load("Inception/Inception_BN", iteration = 39)
```
diff --git a/cpp-package/example/feature_extract/run.sh
b/cpp-package/example/f
[incubator-mxnet] branch master updated: Fixes for CI downloads (#14504)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new a9458ca Fixes for CI downloads (#14504)
a9458ca is described below
commit a9458caec285d3c28d16588fef6aef5e219c7472
Author: Anton Chernov
AuthorDate: Tue Mar 26 04:59:27 2019 +0100
Fixes for CI downloads (#14504)
* Fixes for data links
* Changed data.dmlc.ml -> data.mxnet.io
* Added retries for downloads
* Removed silent mode for explicit failure reporting
* Reworked download scripts
---
R-package/tests/testthat/get_data.R| 11 ++--
R-package/vignettes/CatsDogsFinetune.Rmd | 4 +-
.../classifyRealImageWithPretrainedModel.Rmd | 2 +-
cpp-package/example/feature_extract/run.sh | 7 ++-
cpp-package/example/get_data.sh| 61 +++---
.../inference/unit_test_inception_inference.sh | 4 +-
julia/models/Inception/get.sh | 2 +-
matlab/get_inception_model.sh | 2 +-
tools/dependencies/README.md | 2 +-
tools/dependencies/curl.sh | 4 +-
tools/dependencies/eigen.sh| 4 +-
tools/dependencies/libpng.sh | 4 +-
tools/dependencies/libtiff.sh | 4 +-
tools/dependencies/libturbojpeg.sh | 4 +-
tools/dependencies/libz.sh | 4 +-
tools/dependencies/lz4.sh | 4 +-
tools/dependencies/make_shared_dependencies.sh | 28 +-
tools/dependencies/openblas.sh | 4 +-
tools/dependencies/opencv.sh | 4 +-
tools/dependencies/openssl.sh | 4 +-
tools/dependencies/protobuf.sh | 4 +-
tools/dependencies/zmq.sh | 4 +-
22 files changed, 126 insertions(+), 45 deletions(-)
diff --git a/R-package/tests/testthat/get_data.R
b/R-package/tests/testthat/get_data.R
index 691131c..9bcacdb 100644
--- a/R-package/tests/testthat/get_data.R
+++ b/R-package/tests/testthat/get_data.R
@@ -55,13 +55,16 @@ GetInception <- function() {
if (!dir.exists("model")) {
dir.create("model/")
}
+
if (!file.exists("model/Inception-BN-0126.params")) {
-
download.file("http://data.dmlc.ml/models/imagenet/inception-bn/Inception-BN-0126.params;,
- destfile = "model/Inception-BN-0126.params")
+download.file(
+
"http://data.mxnet.io/mxnet/models/imagenet/inception-bn/Inception-BN-0126.params?raw=true;,
+destfile = "model/Inception-BN-0126.params")
}
if (!file.exists("model/Inception-BN-symbol.json")) {
-
download.file("http://data.dmlc.ml/models/imagenet/inception-bn/Inception-BN-symbol.json;,
- destfile = "model/Inception-BN-symbol.json")
+download.file(
+
"http://data.mxnet.io/mxnet/models/imagenet/inception-bn/Inception-BN-symbol.json;,
+destfile = "model/Inception-BN-symbol.json")
}
}
diff --git a/R-package/vignettes/CatsDogsFinetune.Rmd
b/R-package/vignettes/CatsDogsFinetune.Rmd
index 680b5a3..726bb1a 100644
--- a/R-package/vignettes/CatsDogsFinetune.Rmd
+++ b/R-package/vignettes/CatsDogsFinetune.Rmd
@@ -162,13 +162,13 @@ val <- data$val
## Load pretrained model
-Here we use the pretrained model from http://data.dmlc.ml/models/imagenet/.
+Here we use the pretrained model from http://data.mxnet.io/mxnet/data/.
There are 1000 classes in imagenet,
and we need to replace the last fully connected layer with a new layer for 2
classes.
```{r}
-download.file('http://data.dmlc.ml/data/Inception.zip', destfile =
'Inception.zip')
+download.file('http://data.mxnet.io/mxnet/data/Inception.zip', destfile =
'Inception.zip')
unzip("Inception.zip")
inception_bn <- mx.model.load("./Inception-BN", iteration = 126)
diff --git a/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
b/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
index ff631e0..9cfdd5a 100644
--- a/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
+++ b/R-package/vignettes/classifyRealImageWithPretrainedModel.Rmd
@@ -31,7 +31,7 @@ Make sure you unzip the pre-trained model in current folder.
And we can use the
loading function to load the model into R.
```{r}
-download.file('http://data.dmlc.ml/data/Inception.zip', destfile =
'Inception.zip')
+download.file('http://data.mxnet.io/mxnet/data/Inception.zip', destfile =
'Inception.zip')
unzip("Inception.zip")
model <- mx.model.load("Inception/Inception_BN", iteration = 39)
```
diff --git a/cpp-package/example/feature_extract/run.sh
b/cpp-pack
[incubator-mxnet] branch master updated: Added link to landing page for Java examples (#14481)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 95d4680 Added link to landing page for Java examples (#14481) 95d4680 is described below commit 95d468090896ba6746857a3e3dbd30e9bade3073 Author: Denis Davydenko AuthorDate: Wed Mar 20 17:49:17 2019 -0700 Added link to landing page for Java examples (#14481) --- example/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/example/README.md b/example/README.md index 18022be..0c9ac4d 100644 --- a/example/README.md +++ b/example/README.md @@ -87,6 +87,7 @@ If your tutorial depends on specific packages, simply add them to this provision * [MXNet C++ API](http://mxnet.incubator.apache.org/api/c++/index.html) - [C++ examples](https://github.com/apache/incubator-mxnet/tree/master/example/image-classification/predict-cpp) - Example code for using C++ interface, including NDArray, symbolic layer and models. * [MXNet Python API](http://mxnet.incubator.apache.org/api/python/index.html) +* [MXNet Java API](http://mxnet.incubator.apache.org/api/java/index.html) * [MXNet Scala API](http://mxnet.incubator.apache.org/api/scala/index.html) * [MXNet R API](http://mxnet.incubator.apache.org/api/r/index.html) * [MXNet Julia API](http://mxnet.incubator.apache.org/api/julia/index.html)
[incubator-mxnet] branch master updated: Enforce determinism for backwards compatibility checker (#14463)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new c31bb7e Enforce determinism for backwards compatibility checker
(#14463)
c31bb7e is described below
commit c31bb7e16195bf283ee19b951280f2b28e3c3597
Author: Zach Kimberg
AuthorDate: Tue Mar 19 11:39:04 2019 -0700
Enforce determinism for backwards compatibility checker (#14463)
---
.../model_backward_compat_checker.sh | 3 ++-
.../model_backwards_compatibility_check/train_mxnet_legacy_models.sh | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git
a/tests/nightly/model_backwards_compatibility_check/model_backward_compat_checker.sh
b/tests/nightly/model_backwards_compatibility_check/model_backward_compat_checker.sh
index 2338683..ba53146 100755
---
a/tests/nightly/model_backwards_compatibility_check/model_backward_compat_checker.sh
+++
b/tests/nightly/model_backwards_compatibility_check/model_backward_compat_checker.sh
@@ -27,4 +27,5 @@ cd tests/nightly/model_backwards_compatibility_check
echo `pwd`
echo '=='
-python model_backwards_compat_inference.py
\ No newline at end of file
+export MXNET_ENFORCE_DETERMINISM=1
+python model_backwards_compat_inference.py
diff --git
a/tests/nightly/model_backwards_compatibility_check/train_mxnet_legacy_models.sh
b/tests/nightly/model_backwards_compatibility_check/train_mxnet_legacy_models.sh
index 02d480d..9a1700f 100755
---
a/tests/nightly/model_backwards_compatibility_check/train_mxnet_legacy_models.sh
+++
b/tests/nightly/model_backwards_compatibility_check/train_mxnet_legacy_models.sh
@@ -25,6 +25,7 @@ run_models() {
echo '=='
echo "Running training files and preparing models"
echo '=='
+ export MXNET_ENFORCE_DETERMINISM=1
python model_backwards_compat_train.py
echo '=='
}
[incubator-mxnet] branch v1.3.x updated: Cherry-picks changes from #14336 to fix window ci pipeline (#14469)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.3.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.3.x by this push:
new 1fed192 Cherry-picks changes from #14336 to fix window ci pipeline
(#14469)
1fed192 is described below
commit 1fed192efedc50eb8c65b8f03015dd2887d68130
Author: perdasilva
AuthorDate: Tue Mar 19 18:08:48 2019 +0100
Cherry-picks changes from #14336 to fix window ci pipeline (#14469)
---
Jenkinsfile | 18 +-
ci/build_windows.py | 6 +++---
ci/windows/test_py2_cpu.ps1 | 8 +---
ci/windows/test_py2_gpu.ps1 | 12 +++-
ci/windows/test_py3_cpu.ps1 | 8 +---
ci/windows/test_py3_gpu.ps1 | 12 +++-
tests/python/gpu/test_operator_gpu.py | 2 +-
tests/requirements.txt| 4
8 files changed, 41 insertions(+), 29 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 0e4aa19..67926f8 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -317,9 +317,9 @@ core_logic: {
node(NODE_WINDOWS_CPU) {
timeout(time: max_time, unit: 'MINUTES') {
ws('workspace/build-cpu') {
-withEnv(['OpenBLAS_HOME=C:\\mxnet\\openblas',
'OpenCV_DIR=C:\\mxnet\\opencv_vc14', 'CUDA_PATH=C:\\CUDA\\v8.0']) {
+timeout(time: max_time, unit: 'MINUTES') {
utils.init_git_win()
- powershell 'python ci/build_windows.py -f WIN_CPU'
+ powershell 'py -3 ci/build_windows.py -f WIN_CPU'
stash includes: 'windows_package.7z', name: 'windows_package_cpu'
}
}
@@ -331,10 +331,10 @@ core_logic: {
node(NODE_WINDOWS_CPU) {
timeout(time: max_time, unit: 'MINUTES') {
ws('workspace/build-gpu') {
-withEnv(['OpenBLAS_HOME=C:\\mxnet\\openblas',
'OpenCV_DIR=C:\\mxnet\\opencv_vc14', 'CUDA_PATH=C:\\CUDA\\v8.0']) {
- utils.init_git_win()
- powershell 'python ci/build_windows.py -f WIN_GPU'
- stash includes: 'windows_package.7z', name: 'windows_package_gpu'
+timeout(time: max_time, unit: 'MINUTES') {
+utils.init_git_win()
+powershell 'py -3 ci/build_windows.py -f WIN_GPU'
+stash includes: 'windows_package.7z', name:
'windows_package_gpu'
}
}
}
@@ -344,9 +344,9 @@ core_logic: {
node(NODE_WINDOWS_CPU) {
timeout(time: max_time, unit: 'MINUTES') {
ws('workspace/build-gpu') {
-withEnv(['OpenBLAS_HOME=C:\\mxnet\\openblas',
'OpenCV_DIR=C:\\mxnet\\opencv_vc14',
'CUDA_PATH=C:\\CUDA\\v8.0','BUILD_NAME=vc14_gpu_mkldnn']) {
+ timeout(time: max_time, unit: 'MINUTES') {
utils.init_git_win()
- powershell 'python ci/build_windows.py -f WIN_GPU_MKLDNN'
+ powershell 'py -3 ci/build_windows.py -f WIN_GPU_MKLDNN'
stash includes: 'windows_package.7z', name:
'windows_package_gpu_mkldnn'
}
}
@@ -935,4 +935,4 @@ failure_handler: {
emailext body: 'Build for MXNet branch ${BRANCH_NAME} has broken. Please
view the build at ${BUILD_URL}', replyTo: '${EMAIL}', subject: '[BUILD FAILED]
Branch ${BRANCH_NAME} build ${BUILD_NUMBER}', to: '${EMAIL}'
}
}
-)
+)
\ No newline at end of file
diff --git a/ci/build_windows.py b/ci/build_windows.py
index 5eca58d..a811f98 100755
--- a/ci/build_windows.py
+++ b/ci/build_windows.py
@@ -232,11 +232,11 @@ def main():
if system == 'Windows':
logging.info("Detected Windows platform")
if 'OpenBLAS_HOME' not in os.environ:
-os.environ["OpenBLAS_HOME"] = "C:\\mxnet\\openblas"
+os.environ["OpenBLAS_HOME"] = "C:\\Program Files\\OpenBLAS-v0.2.19"
if 'OpenCV_DIR' not in os.environ:
-os.environ["OpenCV_DIR"] = "C:\\mxnet\\opencv_vc14"
+os.environ["OpenCV_DIR"] = "C:\\Program
Files\\OpenCV-v3.4.1\\build"
if 'CUDA_PATH' not in os.environ:
-os.environ["CUDA_PATH"] = "C:\\CUDA\\v8.0"
+os.environ["CUDA_PATH"] = "C:\\Program Files\\NVIDIA GPU Computing
Toolkit\\CUDA\\v9.2"
windows_build(args)
elif system == 'Linux' or system == 'Darwin':
diff --git a/ci/windows/test_py2_cpu.ps1 b/ci/windows/test_py2_cpu.ps1
index aa38b81..4cef0ea 100644
--- a/ci/windows/test_py2_cpu.ps1
+++ b/ci/windows/test_py2_cpu.ps1
@@ -19,8 +19,10 @@
$env:MXNET_LIBRARY_PATH=join-path $pwd.Path windows_package\lib\libmxnet.dll
$env:PYTHONPATH=join-path $pwd.Path windows_package\python
$env:MXNET_STORAGE_FALLBACK_LOG_VERBOSE=0
-c:\Anaconda3\envs\py2\Scripts\pip install -r tests\requirements.txt
-c:\Anaconda3\envs\py2
[incubator-mxnet] branch master updated: temporarily disable integ tests with a dependency on origami repo (#14448)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new a091d36 temporarily disable integ tests with a dependency on origami
repo (#14448)
a091d36 is described below
commit a091d363ce9d5b97f219b6cb1658b736dbbffc2e
Author: Andrew Ayres
AuthorDate: Fri Mar 15 23:35:24 2019 -0700
temporarily disable integ tests with a dependency on origami repo (#14448)
---
contrib/clojure-package/integration-tests.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/contrib/clojure-package/integration-tests.sh
b/contrib/clojure-package/integration-tests.sh
index 3f80ea5..5ae26e8 100755
--- a/contrib/clojure-package/integration-tests.sh
+++ b/contrib/clojure-package/integration-tests.sh
@@ -26,7 +26,7 @@ lein install
# then run through the examples
EXAMPLES_HOME=${MXNET_HOME}/contrib/clojure-package/examples
# use AWK pattern for blacklisting
-TEST_CASES=`find ${EXAMPLES_HOME} -name test | awk
'!/dontselect1|cnn-text-classification/'`
+TEST_CASES=`find ${EXAMPLES_HOME} -name test | awk
'!/dontselect1|cnn-text-classification|gan|neural-style|infer|pre-trained-models/'`
for i in $TEST_CASES ; do
cd ${i} && lein test
done
[incubator-mxnet] branch master updated: [MXNET-1226] add Docs update for MXNet Java (#14395)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 4432af1 [MXNET-1226] add Docs update for MXNet Java (#14395) 4432af1 is described below commit 4432af1f47a439517eff9a21bef23ef7ae5e4aa4 Author: Lanking AuthorDate: Wed Mar 13 13:07:19 2019 -0700 [MXNET-1226] add Docs update for MXNet Java (#14395) * add Docs update for MXNet Java * fixed based on comments * Update docs/install/java_setup.md Co-Authored-By: lanking520 * Update docs/tutorials/java/mxnet_java_on_intellij.md Co-Authored-By: lanking520 --- docs/install/index.md | 6 ++--- docs/install/java_setup.md| 32 ++- docs/tutorials/java/mxnet_java_on_intellij.md | 17 +++--- 3 files changed, 13 insertions(+), 42 deletions(-) diff --git a/docs/install/index.md b/docs/install/index.md index eab1c82..f1e959e 100644 --- a/docs/install/index.md +++ b/docs/install/index.md @@ -505,7 +505,7 @@ You can use the Maven packages defined in the following dependency to include MX You can use the Maven packages defined in the following dependency to include MXNet in your Java project. The Java API is provided as a subset of the Scala API and is intended for inference only. Please refer to the MXNet-Java setup guide for a detailed set of instructions to help you with the setup process. -https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.4.0-SNAPSHOT~~;>https://img.shields.io/badge/org.apache.mxnet-linux gpu-green.svg" alt="maven badge"/> +https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.4.0~~;>https://img.shields.io/badge/org.apache.mxnet-linux gpu-green.svg" alt="maven badge"/> ```html @@ -522,7 +522,7 @@ You can use the Maven packages defined in the following dependency to include MX You can use the Maven packages defined in the following dependency to include MXNet in your Java project. The Java API is provided as a subset of the Scala API and is intended for inference only. Please refer to the MXNet-Java setup guide for a detailed set of instructions to help you with the setup process. -https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.4.0-SNAPSHOT~~;>https://img.shields.io/badge/org.apache.mxnet-linux cpu-green.svg" alt="maven badge"/> +https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.4.0~~;>https://img.shields.io/badge/org.apache.mxnet-linux cpu-green.svg" alt="maven badge"/> ```html @@ -810,7 +810,7 @@ Not available at this time. You can use the Maven packages defined in the following dependency to include MXNet in your Java project. The Java API is provided as a subset of the Scala API and is intended for inference only. Please refer to the MXNet-Java setup guide for a detailed set of instructions to help you with the setup process. -https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.4.0-SNAPSHOT~~;>https://img.shields.io/badge/org.apache.mxnet-mac cpu-green.svg" alt="maven badge"/> +https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~1.4.0~~;>https://img.shields.io/badge/org.apache.mxnet-mac cpu-green.svg" alt="maven badge"/> ```html diff --git a/docs/install/java_setup.md b/docs/install/java_setup.md index 569f439..ce4a4ab 100644 --- a/docs/install/java_setup.md +++ b/docs/install/java_setup.md @@ -35,21 +35,15 @@ The following instructions are provided for macOS and Ubuntu. Windows is not yet brew update brew tap caskroom/versions brew cask install java8 -brew install opencv brew install maven ``` **Ubuntu Steps** -These scripts will install Maven and its dependencies. You will be running the Scala scripts because the MXNet-Java project has a dependency on the MXNet-Scala project. +Please run the following lines: ```bash -wget https://raw.githubusercontent.com/apache/incubator-mxnet/master/ci/docker/install/ubuntu_core.sh -wget https://raw.githubusercontent.com/apache/incubator-mxnet/master/ci/docker/install/ubuntu_scala.sh -chmod +x ubuntu_core.sh -chmod +x ubuntu_scala.sh -sudo ./ubuntu_core.sh -sudo ./ubuntu_scala.sh +sudo apt-get install openjdk-8-java maven ``` **Step 2.** Run the demo MXNet-Java project. @@ -58,25 +52,14 @@ Go to the [MXNet-Java demo project's README](https://github.com/apache/incubator Maven Repository -MXNet-Java can be easily included in your Maven managed project. The Java packages are currently available as nightly builds on Maven. Add the following Maven repository to your `pom.xml` to fetch the Java packages : - -```html - - - Apache Snapshot - http
[incubator-mxnet] branch master updated: Fix relative difference scala (#14417)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 82504ad Fix relative difference scala (#14417)
82504ad is described below
commit 82504adc63b91f7b7be9075f8110f417944ba413
Author: Dang Trung Kien
AuthorDate: Thu Mar 14 02:07:41 2019 +0800
Fix relative difference scala (#14417)
* Fix relative difference scala
* Increase number of cases for scala arange test
* Add cases where arange produces NDArray of [0]
* Remote whitespace
---
.../core/src/test/scala/org/apache/mxnet/CheckUtils.scala | 4 ++--
.../core/src/test/scala/org/apache/mxnet/NDArraySuite.scala| 7 +++
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/CheckUtils.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/CheckUtils.scala
index 1ddb292..7602b53 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/CheckUtils.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/CheckUtils.scala
@@ -21,13 +21,13 @@ object CheckUtils {
def reldiff(a: NDArray, b: NDArray): Float = {
val diff = NDArray.sum(NDArray.abs(a - b)).toScalar
val norm = NDArray.sum(NDArray.abs(a)).toScalar
-diff / norm
+if (diff < Float.MinPositiveValue) diff else diff / norm
}
def reldiff(a: Array[Float], b: Array[Float]): Float = {
val diff =
(a zip b).map { case (aElem, bElem) => Math.abs(aElem - bElem) }.sum
val norm: Float = a.reduce(Math.abs(_) + Math.abs(_))
-diff / norm
+if (diff < Float.MinPositiveValue) diff else diff / norm
}
}
diff --git
a/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
index 72a5974..206094c 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/NDArraySuite.scala
@@ -355,6 +355,13 @@ class NDArraySuite extends FunSuite with BeforeAndAfterAll
with Matchers {
val result3 = 0f to stop by 1f
val range3 = NDArray.arange(stop)
assert(CheckUtils.reldiff(result3.toArray, range3.toArray) <= 1e-4f)
+
+ val stop4 = Math.abs(stop)
+ val step4 = stop4 + Math.abs(scala.util.Random.nextFloat())
+ val result4 = (0.0 until stop4.toDouble by step4.toDouble)
+.flatMap(x => Array.fill[Float](repeat)(x.toFloat))
+ val range4 = NDArray.arange(stop4, step = step4, repeat = repeat)
+ assert(CheckUtils.reldiff(result4.toArray, range4.toArray) <= 1e-4f)
}
}
[incubator-mxnet] branch master updated: Optimizer MXKVStoreUpdater bug fix in serializeState method (#14337)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 12c41e6 Optimizer MXKVStoreUpdater bug fix in serializeState method
(#14337)
12c41e6 is described below
commit 12c41e6580da4dc41788a9bd3bed73e9dbb7e3db
Author: Satya Krishna Gorti
AuthorDate: Thu Mar 7 13:13:31 2019 -0500
Optimizer MXKVStoreUpdater bug fix in serializeState method (#14337)
---
CONTRIBUTORS.md | 1 +
.../core/src/main/scala/org/apache/mxnet/Optimizer.scala | 16 +++-
2 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 584940b..4e27189 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -215,6 +215,7 @@ List of Contributors
* [Zach Boldyga](https://github.com/zboldyga)
* [Gordon Reid](https://github.com/gordon1992)
* [Ming Yang](http://ufoym.com)
+* [Satya Krishna Gorti](https://github.com/satyakrishnagorti)
* [Neo Chien](https://github.com/cchung100m)
Label Bot
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Optimizer.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Optimizer.scala
index 1fb634c..da58976 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Optimizer.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Optimizer.scala
@@ -52,15 +52,13 @@ object Optimizer {
val out = new ObjectOutputStream(bos)
out.writeInt(states.size)
states.foreach { case (k, v) =>
-if (v != null) {
- out.writeInt(k)
- val stateBytes = optimizer.serializeState(v)
- if (stateBytes == null) {
-out.writeInt(0)
- } else {
-out.writeInt(stateBytes.length)
-out.write(stateBytes)
- }
+out.writeInt(k)
+val stateBytes = optimizer.serializeState(v)
+if (stateBytes == null) {
+ out.writeInt(0)
+} else {
+ out.writeInt(stateBytes.length)
+ out.write(stateBytes)
}
}
out.flush()
[incubator-mxnet] branch master updated: Fixes #14181, validate model output shape for ObjectDetector. (#14215)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new e703694 Fixes #14181, validate model output shape for ObjectDetector.
(#14215)
e703694 is described below
commit e70369437bea25b92bd5531b08fff92988b2ff02
Author: Frank Liu
AuthorDate: Thu Mar 7 10:13:09 2019 -0800
Fixes #14181, validate model output shape for ObjectDetector. (#14215)
---
.../org/apache/mxnet/infer/ImageClassifier.scala | 2 ++
.../org/apache/mxnet/infer/ObjectDetector.scala| 25 +++---
.../scala/org/apache/mxnet/infer/Predictor.scala | 7 ++
3 files changed, 31 insertions(+), 3 deletions(-)
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
index 3c80f92..99c0432 100644
---
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
+++
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ImageClassifier.scala
@@ -66,6 +66,8 @@ class ImageClassifier(modelPathPrefix: String,
protected[infer] val height = inputShape(inputLayout.indexOf('H'))
protected[infer] val width = inputShape(inputLayout.indexOf('W'))
+ def outputShapes: IndexedSeq[(String, Shape)] = predictor.outputShapes
+
/**
* To classify the image according to the provided model
*
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
index 7146156..e29f068 100644
---
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
+++
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
@@ -20,12 +20,13 @@ package org.apache.mxnet.infer
// scalastyle:off
import java.awt.image.BufferedImage
+import org.apache.mxnet.Shape
+
import scala.collection.parallel.mutable.ParArray
// scalastyle:on
import org.apache.mxnet.NDArray
import org.apache.mxnet.DataDesc
import org.apache.mxnet.Context
-import scala.collection.mutable.ListBuffer
/**
* The ObjectDetector class helps to run ObjectDetection tasks where the goal
@@ -174,7 +175,25 @@ class ObjectDetector(modelPathPrefix: String,
contexts: Array[Context] = Context.cpu(),
epoch: Option[Int] = Some(0)):
ImageClassifier = {
-new ImageClassifier(modelPathPrefix, inputDescriptors, contexts, epoch)
- }
+val imageClassifier: ImageClassifier =
+ new ImageClassifier(modelPathPrefix, inputDescriptors, contexts, epoch)
+
+val shapes: IndexedSeq[(String, Shape)] = imageClassifier.outputShapes
+if (shapes.length != inputDescriptors.length) {
+ throw new IllegalStateException(s"Invalid output shapes, expected:" +
+s" $inputDescriptors.length, actual: $shapes.length.")
+}
+shapes.map(_._2).foreach(shape => {
+ if (shape.length < 3) {
+throw new IllegalArgumentException("Invalid output shapes, the model
doesn't"
+ + " support object detection.")
+ }
+ if (shape.get(2) < 6) {
+throw new IllegalArgumentException("Invalid output shapes, the model
doesn't"
+ + " support object detection with bounding box.")
+ }
+})
+imageClassifier
+ }
}
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Predictor.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Predictor.scala
index 67692a3..66284c8 100644
--- a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Predictor.scala
+++ b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/Predictor.scala
@@ -56,6 +56,11 @@ private[infer] trait PredictBase {
*/
def predictWithNDArray(input: IndexedSeq[NDArray]): IndexedSeq[NDArray]
+ /**
+* Get model output shapes.
+* @return model output shapes.
+*/
+ def outputShapes: IndexedSeq[(String, Shape)]
}
/**
@@ -122,6 +127,8 @@ class Predictor(modelPathPrefix: String,
protected[infer] val mod = loadModule()
+ override def outputShapes: IndexedSeq[(String, Shape)] = mod.outputShapes
+
/**
* Takes input as IndexedSeq one dimensional arrays and creates the NDArray
needed for inference
* The array will be reshaped based on the input descriptors.
svn commit: r32768 - in /dev/incubator/mxnet: 1.4.0.rc0/ 1.4.0.rc1/ 1.4.0.rc2/ 1.4.0.rc3/
Author: lanking Date: Tue Mar 5 18:26:40 2019 New Revision: 32768 Log: remove all 1.4.0rc* Removed: dev/incubator/mxnet/1.4.0.rc0/ dev/incubator/mxnet/1.4.0.rc1/ dev/incubator/mxnet/1.4.0.rc2/ dev/incubator/mxnet/1.4.0.rc3/
[incubator-mxnet] branch revert-13162-java-api deleted (was d894e05)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch revert-13162-java-api in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. was d894e05 Revert "[MXNET-1198] MXNet Java API (#13162)" This change permanently discards the following revisions: discard d894e05 Revert "[MXNET-1198] MXNet Java API (#13162)" discard 811d846 Revert "enabling test_dropout after fixing flaky issue (#13276)" discard cc3c971 Revert "fix the flag (#13293)" discard 21952bf Revert "Made fixes to sparse.py and sparse.md (#13305)" discard 41855e8 Revert "Fix descriptions in scaladocs for macro ndarray/sybmol APIs (#13210)" discard 8688626 Revert "[Example] Gradcam- Fixing a link (#13307)" discard 46d95a1 Revert "Updated the Instructions for use of the label bot (#13192)" discard 9e42914 Revert "[MXNET-33] Enhance mkldnn pooling to support full convention (#11047)" discard 57e0f10 Revert "[MXNET-1213] add Cent OS build for Scala (#13279)" discard cf8beff Revert "fix file lock issue (#13296)" discard 63086a9 Revert "modify code for working in gpu context. (#13302)" discard 119825e Revert "[MXNET-860] Use modernized ranged loops where possible (#12356)" discard 51d5e45 Revert "[MXNET-483] C++ tests for mkldnn convolution/deconvolution operator (#11778)" discard f8e8e17 Revert "Fix ONNX export of keepdims param (#12924)" discard e9da9fa Revert "adding unittest for MKLDNN Softmax operator (#12884)" discard b114752 Revert "[MXNET-1215] Allow dynamic shape exists in imperative mode (#13283)" discard f4402ec Revert "Fix/link to mkl build (#13268)" discard ff9abeb Revert "Update Requests version to 2.20.0 to handle security risk (#13266)" discard e766253 Revert "Fix nightly test issue and add docs build settings for 1.3.x (#13323)"
[incubator-mxnet] branch master updated: 1.4 release (#14297)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 1363557 1.4 release (#14297) 1363557 is described below commit 13635579aec8083ce5c568496161f24fd2409545 Author: IvyBazan <45951687+ivyba...@users.noreply.github.com> AuthorDate: Mon Mar 4 11:47:02 2019 -0800 1.4 release (#14297) * version 1.4.0 release changes * Updated navbar order * updated 1.4 release announcement * Update options.js --- docs/_static/js/options.js | 5 +-- docs/_static/mxnet-theme/index.html | 6 ++-- docs/_static/mxnet-theme/navbar.html | 4 +-- docs/install/build_from_source.md| 2 +- docs/install/download.md | 1 + docs/install/index.md| 59 docs/install/ubuntu_setup.md | 2 +- 7 files changed, 58 insertions(+), 21 deletions(-) diff --git a/docs/_static/js/options.js b/docs/_static/js/options.js index f4fde4e..ec39776 100644 --- a/docs/_static/js/options.js +++ b/docs/_static/js/options.js @@ -1,8 +1,9 @@ + /*! * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file + * regarding copyright ownership. The ASF licenses this file * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at @@ -18,7 +19,7 @@ */ /* Installation page display functions for install selector */ -var versionSelect = defaultVersion = 'v1.3.1'; +var versionSelect = defaultVersion = 'v1.4.0'; var platformSelect= 'Linux'; var languageSelect = 'Python'; var processorSelect = 'CPU'; diff --git a/docs/_static/mxnet-theme/index.html b/docs/_static/mxnet-theme/index.html index 302b173..34f6758 100644 --- a/docs/_static/mxnet-theme/index.html +++ b/docs/_static/mxnet-theme/index.html @@ -23,9 +23,9 @@ -MXNet 1.3.1 Released -This release includes bug fixes, performance improvements, and documentation updates. -https://github.com/apache/incubator-mxnet/releases/tag/1.3.1;>Learn More +MXNet 1.4.0 Released +This release introduces the Java Inference API and Julia API, as well as Control Flow Operators, MKLDNN optimizations, and SVRG optimization. +https://github.com/apache/incubator-mxnet/releases/tag/1.4.0;>Learn More A 60-minute Gluon Crash Course diff --git a/docs/_static/mxnet-theme/navbar.html b/docs/_static/mxnet-theme/navbar.html index 50c3deb..d39582e 100644 --- a/docs/_static/mxnet-theme/navbar.html +++ b/docs/_static/mxnet-theme/navbar.html @@ -23,11 +23,11 @@ Python C++ Clojure +Java Julia Perl R Scala -Java @@ -77,11 +77,11 @@ Python C++ Clojure + Java Julia Perl R Scala - Java diff --git a/docs/install/build_from_source.md b/docs/install/build_from_source.md index fee4ae5..7b00b03 100644 --- a/docs/install/build_from_source.md +++ b/docs/install/build_from_source.md @@ -45,11 +45,11 @@ MXNet's newest and most popular API is Gluon. Gluon is built into the Python bin - [Python (includes Gluon)](../api/python/index.html) - [C++](../api/c++/index.html) - [Clojure](../api/clojure/index.html) +- [Java](../api/java/index.html) - [Julia](../api/julia/index.html) - [Perl](../api/perl/index.html) - [R](../api/r/index.html) - [Scala](../api/scala/index.html) -- [Java](../api/java/index.html) diff --git a/docs/install/download.md b/docs/install/download.md index 725cf5e..cf95c23 100644 --- a/docs/install/download.md +++ b/docs/install/download.md @@ -21,6 +21,7 @@ These source archives are generated from tagged releases. Updates and patches wi | Version | Source | PGP
[incubator-mxnet] branch master updated: MXNET-1302 Exclude commons-codec and commons-io from assembled JAR (#14000)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new c6b1fd5 MXNET-1302 Exclude commons-codec and commons-io from assembled JAR (#14000) c6b1fd5 is described below commit c6b1fd5a74152b64f92a8b76e1d28f4c0d4f00cd Author: Gordon Reid AuthorDate: Fri Mar 1 18:11:29 2019 + MXNET-1302 Exclude commons-codec and commons-io from assembled JAR (#14000) --- CONTRIBUTORS.md | 1 + scala-package/assembly/src/main/assembly/assembly.xml | 2 ++ scala-package/core/pom.xml| 5 - scala-package/deploy/src/main/deploy/deploy.xml | 10 ++ scala-package/macros/pom.xml | 5 - scala-package/pom.xml | 5 + 6 files changed, 18 insertions(+), 10 deletions(-) diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index caec6cf..caf61e8 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -212,6 +212,7 @@ List of Contributors * [Xiao Wang](https://github.com/BeyonderXX) * [Piyush Ghai](https://github.com/piyushghai) * [Zach Boldyga](https://github.com/zboldyga) +* [Gordon Reid](https://github.com/gordon1992) * [Ming Yang](http://ufoym.com) diff --git a/scala-package/assembly/src/main/assembly/assembly.xml b/scala-package/assembly/src/main/assembly/assembly.xml index eaa3018..4c3d173 100644 --- a/scala-package/assembly/src/main/assembly/assembly.xml +++ b/scala-package/assembly/src/main/assembly/assembly.xml @@ -28,6 +28,8 @@ org.scala-lang:* org.scala-lang.modules:* +commons-io:commons-io +commons-codec:commons-codec / true diff --git a/scala-package/core/pom.xml b/scala-package/core/pom.xml index c513d28..5a28a11 100644 --- a/scala-package/core/pom.xml +++ b/scala-package/core/pom.xml @@ -154,11 +154,6 @@ INTERNAL provided - - commons-io - commons-io - 2.1 - org.mockito diff --git a/scala-package/deploy/src/main/deploy/deploy.xml b/scala-package/deploy/src/main/deploy/deploy.xml index 8d24894..c5be7e2 100644 --- a/scala-package/deploy/src/main/deploy/deploy.xml +++ b/scala-package/deploy/src/main/deploy/deploy.xml @@ -45,5 +45,15 @@ scala-compiler SCALA_VERSION + + commons-codec + commons-codec + 1.10 + + + commons-io + commons-io + 2.1 + diff --git a/scala-package/macros/pom.xml b/scala-package/macros/pom.xml index d6dd020..91a13d0 100644 --- a/scala-package/macros/pom.xml +++ b/scala-package/macros/pom.xml @@ -36,11 +36,6 @@ INTERNAL provided - - commons-io - commons-io - 2.1 - diff --git a/scala-package/pom.xml b/scala-package/pom.xml index a782be8..62def16 100644 --- a/scala-package/pom.xml +++ b/scala-package/pom.xml @@ -395,6 +395,11 @@ 1.10 + commons-io + commons-io + 2.1 + + org.apache.logging.log4j log4j-core 2.11.1
[incubator-mxnet] branch master updated (7c617cc -> f0fedec)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 7c617cc pypi package description. manifest/setup.py update (#14255) add f0fedec Refactors USE_NVRTC setting to ENABLE_CUDA_RTC in pip make config files (#14250) No new revisions were added by this update. Summary of changes: make/pip/pip_darwin_cpu.mk | 2 +- make/pip/pip_darwin_mkl.mk | 2 +- make/pip/pip_linux_cpu.mk | 2 +- make/pip/pip_linux_cu100.mk| 2 +- make/pip/pip_linux_cu100mkl.mk | 2 +- make/pip/pip_linux_cu75.mk | 2 +- make/pip/pip_linux_cu75mkl.mk | 2 +- make/pip/pip_linux_cu80.mk | 2 +- make/pip/pip_linux_cu80mkl.mk | 2 +- make/pip/pip_linux_cu90.mk | 2 +- make/pip/pip_linux_cu90mkl.mk | 2 +- make/pip/pip_linux_cu91.mk | 2 +- make/pip/pip_linux_cu91mkl.mk | 2 +- make/pip/pip_linux_cu92.mk | 2 +- make/pip/pip_linux_cu92mkl.mk | 2 +- make/pip/pip_linux_mkl.mk | 2 +- 16 files changed, 16 insertions(+), 16 deletions(-)
[incubator-mxnet] tag 1.4.0.rc0 deleted (was c84bb78)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to tag 1.4.0.rc0 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. *** WARNING: tag 1.4.0.rc0 was deleted! *** was c84bb78 Add bug fix #13686 to release note (#13691) The revisions that were on this tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[incubator-mxnet] tag 1.4.0.rc2 deleted (was e999a46)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to tag 1.4.0.rc2 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. *** WARNING: tag 1.4.0.rc2 was deleted! *** was e999a46 Use CPUPinned context in ImageRecordIOParser2 (#13980) (#13990) The revisions that were on this tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[incubator-mxnet] tag 1.4.0.rc1 deleted (was 45a1554)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to tag 1.4.0.rc1 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. *** WARNING: tag 1.4.0.rc1 was deleted! *** was 45a1554 api change (#13905) The revisions that were on this tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[incubator-mxnet] tag 1.4.0.rc3 deleted (was a03d59e)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to tag 1.4.0.rc3 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. *** WARNING: tag 1.4.0.rc3 was deleted! *** was a03d59e Fix gtest build (#13926) The revisions that were on this tag are still contained in other references; therefore, this change does not discard any commits from the repository.
[incubator-mxnet] tag 1.4.0 created (now a03d59e)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to tag 1.4.0 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at a03d59e (commit) No new revisions were added by this update.
svn commit: r32684 - /release/incubator/mxnet/KEYS
Author: lanking Date: Wed Feb 27 22:46:58 2019 New Revision: 32684 Log: update key for Qing Lan Modified: release/incubator/mxnet/KEYS Modified: release/incubator/mxnet/KEYS == --- release/incubator/mxnet/KEYS (original) +++ release/incubator/mxnet/KEYS Wed Feb 27 22:46:58 2019 @@ -658,3 +658,93 @@ a4LYL628Ksuv1Yxn/Uhb5nDPxU5RKRDeogn07wta 3Os= =XL0V -END PGP PUBLIC KEY BLOCK- + +pub rsa2048 2019-02-15 [SC] [expires: 2021-02-14] + 0812952358B12DC30536E7E0C06916C3AB88ABFE +uid [ultimate] Qing Lan +sig 3C06916C3AB88ABFE 2019-02-15 Qing Lan +sub rsa2048 2019-02-15 [E] [expires: 2021-02-14] +sig C06916C3AB88ABFE 2019-02-15 Qing Lan + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQENBFwb8GEBCADjmi5ZXihPfPbLxhNpbD4HVdT7xGEQgfgkeTA4TdFyBP81zM0F +dTHPQOhHzPGkHrVwUPt3ir2hS46q3L8wni3VRkUU8KPbqbS/a7Wl7LHFFS0lU36J +3uQLElZOFITlaL1dl7cIv+c8xCfmOPlmEtNAPIB26sIM5qzc5l4xvNf1H0Oq0wo6 +VKCsYb4el4nys2U3UBYVQjGyBEwwemHQmFPKg6a2bc/2UhWn4Z+//g0hzIpYtT/S +jua16r5SHy6BUtFGWfU6LQIwmxqc0TNqRdkDU0QY0A+nT6cgx6ghp/qxoLOhgken +Uw8rutC/oFg2VzS9yVsJNrkbQq5Fl/Mz/wqNABEBAAG0HWxhbmtpbmcgPGxhbmtp +bmdAbGFua2luZy5uZXQ+iQFUBBMBCAA+FiEECukpOOn0VXTzKWFpZ3gsZdQlyvsF +Alwb8GECGwMFCQPCZwAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQZ3gsZdQl +yvtmiAgAnFxzWwPSoHZW9U8ejNtLXxqKmfrUVHDJKVVE0zZPEs9kEk95YZ99jqUS +LagTZwdQOGCJe6Fc9XESW5KdzvnwQyJKwGPCIYDVA/CRuGDbs+tY6HCZ7V1DMGYm +EJsQEzdqWKaNLEVNk1LB3uizEjTS3BIVaVZ0+uovv2joM1mgHX9obGZAZRpBm7zG +TsDwQMuAaWpgnh+BQ9b8O0m6Mdlqh4lAHJmqLdwIz3rD1SLKst0WenjVtyBE1k5t +gBSVfQeMThhcbPITUTLdhAmc27QrsJdDdsH42CVhIJMW6aQnBnEKE3QB+C6DzpTi +UIKqPg99PXmBc9BvlugrZRM4KYiqOLkBDQRcG/BhAQgAzS6PeCCIwImwqSRRnwta +R/7ZbjhMDbnbiHhBKjy08vt2tR8cRK8X7utVwd96u0b3a6J4k0VzMcDP7wBYiY0x +GDbEVsmeXxiqh3+Dv2R+/CBebbswlMJqG+imEaGAAA1bk8U8RrjxuKAUOlyVjEaG +m9flPKOo33SK7UTujfnpSZhA6s0wzRSMn3/26c6+/9vpo8pnqooiD+K7eMhIUfjB ++hFRDJxrEPg9pj3hCJhXA1aCIMrq8jjkm+8PyDQWdt9TdnHLD+b5X+DKNNBpZsIk +IxNTqxNNLan4JZ+70z5m68Lqy4EEHFoHwhn8IFjxOMiJ+6MuP32UdYmvUD3owc8y +nwARAQABiQE8BBgBCAAmFiEECukpOOn0VXTzKWFpZ3gsZdQlyvsFAlwb8GECGwwF +CQPCZwAACgkQZ3gsZdQlyvvYwQf/ctlGApkvSxmhKgCTPBCrudGDpO3QsEF+bR6y +w9WvyvniCt6t5M2A+QPOCqdBvNaP/4wP1H2XghKr9XZcTBlxAJc6pdwxQ6IH6SD8 +syEDVTp0oiQytuyWeNoTM9bOFsliCkjiAVphTDTuVrNEgqdnTMo2cDZgry6gwD0Q +1ZOejcb7kIjiThzaKmgpPnnDNkiK2j25fiNhpiIzEVryLKoXvyoj//C/p8lm9KRk +/2bzRm84uoRsUlrr6I5qizakoODkh5+DlzAaes1fv8ED11zzrlIZjh1vBBgDd3QT +jurgAMn1+WYXhnTVUw8+JjWbmZX4NqJJGZyozkF7rgzI3A9CPJkBDQRcLmPJAQgA +6P5AdfBp4QJKfJC96BjDIrXAFXTytD5wxBsyl1WUcq9vnkX6NZGEtTgsuR/F+FBt +Za7fuDRf+ZT5J7sFQbKFJKRvR8shg1dqnIsoztBD0ESC46trD/YZrisDb09JUXN+ +zvha7jUisA7/pVwfenQM8CpnZJiuKIDU5qCTUr7qOjds6g9aCtUSOK41xLAn1y9b +hKJko4xtjUQu6hXjRXXKltl1N0X5TQCw3sR105xTTWx9nAgFu7cc1uGGuOPCpSms +FPkXFMz+75ff9eXtnmpM+9ZxE8c0jTN7p3U16nd2spsQOqQd83nKb3AbfmWrUYC6 +lmiXGlyOqf8fXSSj/d1ZWwARAQABtB5RaW5nIExhbiA8bGFua2luZzUyMEBsaXZl +LmNvbT6JAVQEEwEIAD4WIQSEDPQiKrv4QoSvRxUvo3RD/I+Y+gUCXC5jyQIbAwUJ +A8JnAAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRAvo3RD/I+Y+vjlB/4h3trK +2wGFn396m/jqLqBGObCrOTQYJVycfKdndTEvzIm5/U1gU0Q2BzP25k3gYQjjYKDB +ZN5ETrvRa+NUHzK9p4L4mEQpyWI1WR7jJ19Qg3wfjXW2lOJ8hVvGDILKm7JHEOFb +xf8bUsIftAJ+eDs0dFkDEZT+VLGIXWghd236oUy/uYKL0nLofyFZL3IHiBRZkZQf +6nW+wuYlNXJ+Gy8ATUOKmqbhKScOYLT88nvd6XK8ht3agvATTiEZNa7uWeSOZqp/ +Sx10jUKvKif1L3KiRADb461zQdijx1VOpj3OZPKwCViag14TGXot/YsjAHLCROPg +RH7Ey+WHqMzrkI/EuQENBFwuY8kBCADG74gBceYTyUZbwP6y1xKElXacZ4r2aovT +xoyEXGGovkVjR30GT4XWpKBbqFk5S+QtRnJJGCMeVeKbruZc2Jv5cNe5WL8t20SU +shkBi89h2ZWOgw5c8h+5axjyJfPzWuq0PoXZZ0poFVuSojEFuQKotBWqalri3SKu +E7dUlPQ1JzCj60dX03Lqb2fZZEuwNTf3VwXZZHCgRKw0jhvlr0yfZVcIx4C1Vizo +rkmgZoKbTTkcjBwG4xU+QAwtZqtqGUcZsGwGRD/VPYoDwVknTWJN9JwGasM2gdtG +y0IvhjHfhahZXPyiDsUpkvHyrjaKUwF9qkNLlnmV0LxRJYfNYkk1ABEBAAGJATwE +GAEIACYWIQSEDPQiKrv4QoSvRxUvo3RD/I+Y+gUCXC5jyQIbDAUJA8JnAAAKCRAv +o3RD/I+Y+ql8CADZJxKXAEhBN6ao0+OE2RM/+IPUgIuW/B33KxYN/SewVaUeZvIe +Nu3TWLiFa4eTOjc2hNZxRmQRtpmX7faJaynuz/bsfdtdsscuWxNUUiF3Tdd9/Eon +eFaywH8as9uFwYLRmfdb5cvst97XofQOJBEfolOgyGOEEuEVxs6bFa43/SSYCliE +MjhxubT8NqQmIsHGQ4nltxCbmVZNKi7eAjms+QkWX5bqWMqXF5hJisagXvO1Jr75 +ncoq3Wp+X1b+9x8KDGCVd1F5+d0OksPrnNIQVOS8TMw6bJzXLw4yQni+1ZtYTw8p +9eG+gakLwBnBvqs+7HmuvPNELZX5SsKfwRYzmQENBFxnEhcBCADFLKeuIwb8XnYd +MsAEbQIyueHsON04/XJVvULVHlbpF7X4XyUU2aHxkydFN5yGtJZjE7/BXrPSeoK8 +0qtTblb7ZugUgnyhXaN97RqdD6Lv6kha6EFSylUG9Wyfi1Onr0zlgqQ5TwbaTbLJ +UuzN5d7UIPNFZBrWgvH6AC982M3kMGBWEPRiJgCs/k601oX84g6HbynuxaySts1W +N8mC1TjzVuTRq6Cbfr1RAkNcj3mz66DpZLnHsBUPuQ5u0gJTKZyEGRxtT02eXpMI +61eP+79hnAwqZds9BXwO+3cCx/x3qJRLNDpR4mPq9QqETkqTIQd6JPjWPRh0OQDK +300hDmizABEBAAG0HVFpbmcgTGFuIDxsYW5raW5nQGFwYWNoZS5vcmc+iQFUBBMB +CAA+FiEECBKVI1ixLcMFNufgwGkWw6uIq/4FAlxnEhcCGwMFCQPCZwAFCwkIBwIG +FQoJCAsCBBYCAwECHgECF4AACgkQwGkWw6uIq/5KDgf/S8XUCzm+aVJUAcx9BuQV +SzvG8C1Bf/XrYXcm3NxEOMejdkt8FAnaPt9dZM3U8CaMu1a90iiuLyvr3aJvb7/K +/WuaK79jFB+zz8bifQANXgwAQPiUjBU/8FNOjiEywaSYwIvy8SMo5lLs2edV4Y8g +J73BU/WID3aH/BK4sclAs3brNY5le3B2GiAXhqCWB567ZTlc6vpf1DBaivVX8J0H
svn commit: r32682 - /dev/incubator/mxnet/1.4.0/ /release/incubator/mxnet/1.4.0/
Author: lanking Date: Wed Feb 27 19:46:04 2019 New Revision: 32682 Log: 1.4.0 release moving from dev to release repo Added: release/incubator/mxnet/1.4.0/ - copied from r32681, dev/incubator/mxnet/1.4.0/ Removed: dev/incubator/mxnet/1.4.0/
svn commit: r32681 - in /dev/incubator/mxnet/1.4.0: ./ apache-mxnet-src-1.4.0-incubating.tar.gz apache-mxnet-src-1.4.0-incubating.tar.gz.asc apache-mxnet-src-1.4.0-incubating.tar.gz.sha512
Author: lanking Date: Wed Feb 27 19:45:01 2019 New Revision: 32681 Log: Add mxnet-1.4.0 Added: dev/incubator/mxnet/1.4.0/ dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz (with props) dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.asc dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.sha512 Added: dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz == Binary file - no diff available. Propchange: dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz -- svn:mime-type = application/octet-stream Added: dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.asc == --- dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.asc (added) +++ dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.asc Wed Feb 27 19:45:01 2019 @@ -0,0 +1,11 @@ +-BEGIN PGP SIGNATURE- + +iQEzBAABCAAdFiEECBKVI1ixLcMFNufgwGkWw6uIq/4FAlx251EACgkQwGkWw6uI +q/6J9wf/aZ3qeAdrvufo3VfeqVndMkAFcOGY1IS235KcPasWEKNkoAopKTckOH+S +NShFOStLBGhpwpFm0lY04eeuqN+dt6ZazaisZex7Ng+SrlcCXv+nMhoA+MNPs9ww +MKQJifyMwoJebYisrZ5ZXw8xTTZatwwYgDhKZp68J6DGwOtvHqjXJPRL7VW26Etj +kmCYRndc5EUXE6lTtsjf+nSFMgTmlKG7JHYoxf2IPJLE8vNVy3Njh19ZYRmCUEvM +xDmtPEcY97s3afw3OAVpv8rjn9FhtFxGFNejLZiWjqU8Mg0w3BZkomZ5U94AWlbR +8bPLtUKiy2Gj9PKJ7SWhwyEWtkhiAA== +=P/ny +-END PGP SIGNATURE- Added: dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.sha512 == --- dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.sha512 (added) +++ dev/incubator/mxnet/1.4.0/apache-mxnet-src-1.4.0-incubating.tar.gz.sha512 Wed Feb 27 19:45:01 2019 @@ -0,0 +1 @@ +86d67a1d9f4ae5b5cdfff104830f4279f87589c5156169e3b8fa8760dc3ddd8fd062865576c10cbe6c878dc4fc3844ce93bc4de3b6c40ee4e2a55168811cf8c0 apache-mxnet-src-1.4.0-incubating.tar.gz
[incubator-mxnet] branch master updated: Scala interpreter instructions (#14169)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new c45e534 Scala interpreter instructions (#14169)
c45e534 is described below
commit c45e53462da69975d180fe672a77f73c8b641460
Author: Zach Kimberg
AuthorDate: Tue Feb 19 12:36:47 2019 -0800
Scala interpreter instructions (#14169)
---
docs/install/scala_setup.md | 28
docs/tutorials/scala/char_lstm.md | 6 +-
2 files changed, 29 insertions(+), 5 deletions(-)
diff --git a/docs/install/scala_setup.md b/docs/install/scala_setup.md
index bc069a1..15a2def 100644
--- a/docs/install/scala_setup.md
+++ b/docs/install/scala_setup.md
@@ -20,6 +20,7 @@
The following instructions are provided for macOS and Ubuntu. Windows is not
yet available.
**Note:** If you use IntelliJ or a similar IDE, you may want to follow the
[MXNet-Scala on IntelliJ
tutorial](../tutorials/scala/mxnet_scala_on_intellij.html) instead of these
instructions.
+**Note:** Currently, we only support scala 2.11
@@ -114,6 +115,33 @@ mvn install
+## Interpreter
+
+To run the scala interpreter, first download and install scala 2.11.x (run
`scala -version` to make sure you have the right version installed.**
+
+### Installing the Interpreter
+
+**Ubuntu***
+
+```
+sudo apt-get install scala
+```
+
+**macOS***
+
+```
+brew install scala@2.11
+```
+
+Then, add scala to your path by following the instructions output by homebrew.
+
+### Running the Interpreter
+
+To run the interpreter, download the appropriate mxnet jar from [the maven
repository](https://search.maven.org/search?q=g:org.apache.mxnet) or build from
source following the instructions above.
+
+Then, run `scala -cp {path/to/mxnet-full_2.11-os-version.jar}` to start it.
+If you receive a "NumberFormatException" when running the interpreter, run
`export TERM=xterm-color` before starting the interpreter.
+
## Documentation
Scaladocs are generated as part of the docs build pipeline. You can find them
published in the [Scala
API](http://mxnet.incubator.apache.org/api/scala/index.html) section of the
website or by going to the [scaladocs
output](https://mxnet.incubator.apache.org/api/scala/docs/index.html#org.apache.mxnet.package)
directly.
diff --git a/docs/tutorials/scala/char_lstm.md
b/docs/tutorials/scala/char_lstm.md
index 972661b..aca08dc 100644
--- a/docs/tutorials/scala/char_lstm.md
+++ b/docs/tutorials/scala/char_lstm.md
@@ -71,11 +71,7 @@ In this tutorial, you will accomplish the following:
## Prerequisites
-To complete this tutorial, you need:
-
-- MXNet. See the instructions for your operating system in [Setup and
Installation](http://mxnet.io/install/index.html)
-- [Scala 2.11.8](https://www.scala-lang.org/download/2.11.8.html)
-- [Maven 3](https://maven.apache.org/install.html)
+To complete this tutorial, setup and run the scala interpreter by following
the
[instructions](https://mxnet.incubator.apache.org/install/scala_setup.html#interpreter).
## Download the Data
[incubator-mxnet] branch master updated: Add tutorial on how to use build from source jar (#14197)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 49c311c Add tutorial on how to use build from source jar (#14197) 49c311c is described below commit 49c311cb2377093bcc13abdb3845401b789892ee Author: Jake Lee AuthorDate: Wed Feb 20 04:35:51 2019 +0800 Add tutorial on how to use build from source jar (#14197) * add tutorial * fix the typo --- scala-package/README.md | 31 +++ 1 file changed, 31 insertions(+) diff --git a/scala-package/README.md b/scala-package/README.md index 8322ab2..c7d0cec 100644 --- a/scala-package/README.md +++ b/scala-package/README.md @@ -179,6 +179,37 @@ mvn deploy -Pstaging Examples & Usage --- +Assuming you use `mvn install`, you can find the `mxnet-full_scala_version-INTERNAL.jar` e.g. `mxnet-full_2.11-INTERNAL.jar` under the path `incubator-mxnet/scala-package/assembly/target`. + +Adding the following configuration in `pom.xml` +```HTML + + org.apache.mxnet + mxnet-full_2.11-INTERNAL + 1.5.0 + system + path_to_jar/mxnet-full_2.11-INTERNAL.jar + +``` +If you have following error message +``` +Error: A JNI error has occurred, please check your installation and try again +Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/mxnet/NDArray +at java.lang.Class.getDeclaredMethods0(Native Method) +at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) +at java.lang.Class.privateGetMethodRecursive(Class.java:3048) +at java.lang.Class.getMethod0(Class.java:3018) +at java.lang.Class.getMethod(Class.java:1784) +at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) +at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) +Caused by: java.lang.ClassNotFoundException: org.apache.mxnet.NDArray +at java.net.URLClassLoader.findClass(URLClassLoader.java:381) +at java.lang.ClassLoader.loadClass(ClassLoader.java:424) +at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) +at java.lang.ClassLoader.loadClass(ClassLoader.java:357) +``` +Please make sure your $CLASSPATH is able to find `mxnet-full_scala_version-INTERNAL.jar`. + - To set up the Scala Project using IntelliJ IDE on macOS follow the instructions [here](https://mxnet.incubator.apache.org/tutorials/scala/mxnet_scala_on_intellij.html). - Several examples on using the Scala APIs are provided in the [Scala Examples Folder](https://github.com/apache/incubator-mxnet/tree/master/scala-package/examples/)
[incubator-mxnet] branch master updated: add quantization example to readme (#14186)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new bada8a1 add quantization example to readme (#14186) bada8a1 is described below commit bada8a1961f0da7f01cd9e61d03f280c48083f1b Author: Xinyu Chen AuthorDate: Tue Feb 19 02:57:22 2019 +0800 add quantization example to readme (#14186) --- example/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/example/README.md b/example/README.md index cd49cf9..18022be 100644 --- a/example/README.md +++ b/example/README.md @@ -128,6 +128,7 @@ If your tutorial depends on specific packages, simply add them to this provision * [Numpy Operator Customization](numpy-ops) - Examplea on quick customize new ops with Numpy * [Profiling](profiler) - generate profiling results in json files * [Python How To](python-howto) - a variety of Python examples +* [Quantization and Calibration Examples](quantization) - examples of quantizing a FP32 model to INT8 and performing low-precision inference with Intel MKL-DNN on CPU or cuDNN on GPU * [R-CNN](rcnn) - R-CNN with distributed implementation and data parallelization * [Recommender Systems](recommenders) - examples of how to build various kinds of recommender systems * [Reinforcement Learning](reinforcement-learning) - a variety of reinforcement learning examples
[incubator-mxnet] tag 1.4.0.rc3 created (now a03d59e)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to tag 1.4.0.rc3 in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. at a03d59e (commit) No new revisions were added by this update.
svn commit: r32522 - in /dev/incubator/mxnet/1.4.0.rc3: ./ apache-mxnet-src-1.4.0.rc3-incubating.tar.gz apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.asc apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.sh
Author: lanking Date: Fri Feb 15 21:08:57 2019 New Revision: 32522 Log: Add mxnet-1.4.0.rc3 Added: dev/incubator/mxnet/1.4.0.rc3/ dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz (with props) dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.asc dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.sha512 Added: dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz == Binary file - no diff available. Propchange: dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz -- svn:mime-type = application/octet-stream Added: dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.asc == --- dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.asc (added) +++ dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.asc Fri Feb 15 21:08:57 2019 @@ -0,0 +1,11 @@ +-BEGIN PGP SIGNATURE- + +iQEzBAABCAAdFiEECBKVI1ixLcMFNufgwGkWw6uIq/4FAlxnHbsACgkQwGkWw6uI +q/5f3wf/fM/42LnOMBfBIBA2u6/xrjzOB4r8YGYPZsPnaq1nQsLZLmLHXyCfsbQP +cQe2g5/AkouWbLkiDIrFh2p3KBYwXFT9DTF6cx4SbyV7CFj0Lnk3iG8Cwm0nlM2s +fTvOpXqyQp9PAdITd9ByTnkU+TKQQogCa2hxOzPeWgTW7K7HNv7CsGjRVZ3FChXO +heLuwxSSSv6XWtdV47jmocoiwagnY+6edRfMw52A4+pcc77epP8T9a4zUo6vIAGG +flFf1u0xQKJdKdCC4/UcGpKs4S94T+5RppAcV1+fCY28KTzTIFO29ns1P/dP1MJE +q6+pysOWskJdqgQ0IPN6ualLvHcdmA== +=XD2b +-END PGP SIGNATURE- Added: dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.sha512 == --- dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.sha512 (added) +++ dev/incubator/mxnet/1.4.0.rc3/apache-mxnet-src-1.4.0.rc3-incubating.tar.gz.sha512 Fri Feb 15 21:08:57 2019 @@ -0,0 +1 @@ +977b4048541375c8c70865464927d83b9f8824cc58bfbfc52bcbfb6eab13f5fae79a491812b6c2c080357bb93312b31941f4fac8fa386fda18bc5d9f558930f8 apache-mxnet-src-1.4.0.rc3-incubating.tar.gz
svn commit: r32521 - /dev/incubator/mxnet/KEYS
Author: lanking Date: Fri Feb 15 20:00:06 2019 New Revision: 32521 Log: update keys file for Qing Lan Modified: dev/incubator/mxnet/KEYS Modified: dev/incubator/mxnet/KEYS == --- dev/incubator/mxnet/KEYS (original) +++ dev/incubator/mxnet/KEYS Fri Feb 15 20:00:06 2019 @@ -717,3 +717,93 @@ S+td6k7QO2oVhdRDFWt8XesaIk9/2iEnZ+MAalCE hwYZBxZQLw== =yxZo -END PGP PUBLIC KEY BLOCK- + +pub rsa2048 2019-02-15 [SC] [expires: 2021-02-14] + 0812952358B12DC30536E7E0C06916C3AB88ABFE +uid [ultimate] Qing Lan +sig 3C06916C3AB88ABFE 2019-02-15 Qing Lan +sub rsa2048 2019-02-15 [E] [expires: 2021-02-14] +sig C06916C3AB88ABFE 2019-02-15 Qing Lan + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQENBFwb8GEBCADjmi5ZXihPfPbLxhNpbD4HVdT7xGEQgfgkeTA4TdFyBP81zM0F +dTHPQOhHzPGkHrVwUPt3ir2hS46q3L8wni3VRkUU8KPbqbS/a7Wl7LHFFS0lU36J +3uQLElZOFITlaL1dl7cIv+c8xCfmOPlmEtNAPIB26sIM5qzc5l4xvNf1H0Oq0wo6 +VKCsYb4el4nys2U3UBYVQjGyBEwwemHQmFPKg6a2bc/2UhWn4Z+//g0hzIpYtT/S +jua16r5SHy6BUtFGWfU6LQIwmxqc0TNqRdkDU0QY0A+nT6cgx6ghp/qxoLOhgken +Uw8rutC/oFg2VzS9yVsJNrkbQq5Fl/Mz/wqNABEBAAG0HWxhbmtpbmcgPGxhbmtp +bmdAbGFua2luZy5uZXQ+iQFUBBMBCAA+FiEECukpOOn0VXTzKWFpZ3gsZdQlyvsF +Alwb8GECGwMFCQPCZwAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQZ3gsZdQl +yvtmiAgAnFxzWwPSoHZW9U8ejNtLXxqKmfrUVHDJKVVE0zZPEs9kEk95YZ99jqUS +LagTZwdQOGCJe6Fc9XESW5KdzvnwQyJKwGPCIYDVA/CRuGDbs+tY6HCZ7V1DMGYm +EJsQEzdqWKaNLEVNk1LB3uizEjTS3BIVaVZ0+uovv2joM1mgHX9obGZAZRpBm7zG +TsDwQMuAaWpgnh+BQ9b8O0m6Mdlqh4lAHJmqLdwIz3rD1SLKst0WenjVtyBE1k5t +gBSVfQeMThhcbPITUTLdhAmc27QrsJdDdsH42CVhIJMW6aQnBnEKE3QB+C6DzpTi +UIKqPg99PXmBc9BvlugrZRM4KYiqOLkBDQRcG/BhAQgAzS6PeCCIwImwqSRRnwta +R/7ZbjhMDbnbiHhBKjy08vt2tR8cRK8X7utVwd96u0b3a6J4k0VzMcDP7wBYiY0x +GDbEVsmeXxiqh3+Dv2R+/CBebbswlMJqG+imEaGAAA1bk8U8RrjxuKAUOlyVjEaG +m9flPKOo33SK7UTujfnpSZhA6s0wzRSMn3/26c6+/9vpo8pnqooiD+K7eMhIUfjB ++hFRDJxrEPg9pj3hCJhXA1aCIMrq8jjkm+8PyDQWdt9TdnHLD+b5X+DKNNBpZsIk +IxNTqxNNLan4JZ+70z5m68Lqy4EEHFoHwhn8IFjxOMiJ+6MuP32UdYmvUD3owc8y +nwARAQABiQE8BBgBCAAmFiEECukpOOn0VXTzKWFpZ3gsZdQlyvsFAlwb8GECGwwF +CQPCZwAACgkQZ3gsZdQlyvvYwQf/ctlGApkvSxmhKgCTPBCrudGDpO3QsEF+bR6y +w9WvyvniCt6t5M2A+QPOCqdBvNaP/4wP1H2XghKr9XZcTBlxAJc6pdwxQ6IH6SD8 +syEDVTp0oiQytuyWeNoTM9bOFsliCkjiAVphTDTuVrNEgqdnTMo2cDZgry6gwD0Q +1ZOejcb7kIjiThzaKmgpPnnDNkiK2j25fiNhpiIzEVryLKoXvyoj//C/p8lm9KRk +/2bzRm84uoRsUlrr6I5qizakoODkh5+DlzAaes1fv8ED11zzrlIZjh1vBBgDd3QT +jurgAMn1+WYXhnTVUw8+JjWbmZX4NqJJGZyozkF7rgzI3A9CPJkBDQRcLmPJAQgA +6P5AdfBp4QJKfJC96BjDIrXAFXTytD5wxBsyl1WUcq9vnkX6NZGEtTgsuR/F+FBt +Za7fuDRf+ZT5J7sFQbKFJKRvR8shg1dqnIsoztBD0ESC46trD/YZrisDb09JUXN+ +zvha7jUisA7/pVwfenQM8CpnZJiuKIDU5qCTUr7qOjds6g9aCtUSOK41xLAn1y9b +hKJko4xtjUQu6hXjRXXKltl1N0X5TQCw3sR105xTTWx9nAgFu7cc1uGGuOPCpSms +FPkXFMz+75ff9eXtnmpM+9ZxE8c0jTN7p3U16nd2spsQOqQd83nKb3AbfmWrUYC6 +lmiXGlyOqf8fXSSj/d1ZWwARAQABtB5RaW5nIExhbiA8bGFua2luZzUyMEBsaXZl +LmNvbT6JAVQEEwEIAD4WIQSEDPQiKrv4QoSvRxUvo3RD/I+Y+gUCXC5jyQIbAwUJ +A8JnAAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRAvo3RD/I+Y+vjlB/4h3trK +2wGFn396m/jqLqBGObCrOTQYJVycfKdndTEvzIm5/U1gU0Q2BzP25k3gYQjjYKDB +ZN5ETrvRa+NUHzK9p4L4mEQpyWI1WR7jJ19Qg3wfjXW2lOJ8hVvGDILKm7JHEOFb +xf8bUsIftAJ+eDs0dFkDEZT+VLGIXWghd236oUy/uYKL0nLofyFZL3IHiBRZkZQf +6nW+wuYlNXJ+Gy8ATUOKmqbhKScOYLT88nvd6XK8ht3agvATTiEZNa7uWeSOZqp/ +Sx10jUKvKif1L3KiRADb461zQdijx1VOpj3OZPKwCViag14TGXot/YsjAHLCROPg +RH7Ey+WHqMzrkI/EuQENBFwuY8kBCADG74gBceYTyUZbwP6y1xKElXacZ4r2aovT +xoyEXGGovkVjR30GT4XWpKBbqFk5S+QtRnJJGCMeVeKbruZc2Jv5cNe5WL8t20SU +shkBi89h2ZWOgw5c8h+5axjyJfPzWuq0PoXZZ0poFVuSojEFuQKotBWqalri3SKu +E7dUlPQ1JzCj60dX03Lqb2fZZEuwNTf3VwXZZHCgRKw0jhvlr0yfZVcIx4C1Vizo +rkmgZoKbTTkcjBwG4xU+QAwtZqtqGUcZsGwGRD/VPYoDwVknTWJN9JwGasM2gdtG +y0IvhjHfhahZXPyiDsUpkvHyrjaKUwF9qkNLlnmV0LxRJYfNYkk1ABEBAAGJATwE +GAEIACYWIQSEDPQiKrv4QoSvRxUvo3RD/I+Y+gUCXC5jyQIbDAUJA8JnAAAKCRAv +o3RD/I+Y+ql8CADZJxKXAEhBN6ao0+OE2RM/+IPUgIuW/B33KxYN/SewVaUeZvIe +Nu3TWLiFa4eTOjc2hNZxRmQRtpmX7faJaynuz/bsfdtdsscuWxNUUiF3Tdd9/Eon +eFaywH8as9uFwYLRmfdb5cvst97XofQOJBEfolOgyGOEEuEVxs6bFa43/SSYCliE +MjhxubT8NqQmIsHGQ4nltxCbmVZNKi7eAjms+QkWX5bqWMqXF5hJisagXvO1Jr75 +ncoq3Wp+X1b+9x8KDGCVd1F5+d0OksPrnNIQVOS8TMw6bJzXLw4yQni+1ZtYTw8p +9eG+gakLwBnBvqs+7HmuvPNELZX5SsKfwRYzmQENBFxnEhcBCADFLKeuIwb8XnYd +MsAEbQIyueHsON04/XJVvULVHlbpF7X4XyUU2aHxkydFN5yGtJZjE7/BXrPSeoK8 +0qtTblb7ZugUgnyhXaN97RqdD6Lv6kha6EFSylUG9Wyfi1Onr0zlgqQ5TwbaTbLJ +UuzN5d7UIPNFZBrWgvH6AC982M3kMGBWEPRiJgCs/k601oX84g6HbynuxaySts1W +N8mC1TjzVuTRq6Cbfr1RAkNcj3mz66DpZLnHsBUPuQ5u0gJTKZyEGRxtT02eXpMI +61eP+79hnAwqZds9BXwO+3cCx/x3qJRLNDpR4mPq9QqETkqTIQd6JPjWPRh0OQDK +300hDmizABEBAAG0HVFpbmcgTGFuIDxsYW5raW5nQGFwYWNoZS5vcmc+iQFUBBMB +CAA+FiEECBKVI1ixLcMFNufgwGkWw6uIq/4FAlxnEhcCGwMFCQPCZwAFCwkIBwIG +FQoJCAsCBBYCAwECHgECF4AACgkQwGkWw6uIq/5KDgf/S8XUCzm+aVJUAcx9BuQV +SzvG8C1Bf/XrYXcm3NxEOMejdkt8FAnaPt9dZM3U8CaMu1a90iiuLyvr3aJvb7/K +/WuaK79jFB+zz8bifQANXgwAQPiUjBU/8FNOjiEywaSYwIvy8SMo5lLs2edV4Y8g +J73BU/WID3aH/BK4sclAs3brNY5le3B2GiAXhqCWB567ZTlc6vpf1DBaivVX8J0H
[incubator-mxnet] branch master updated: Fix jar path and add missing ones for spark jobs (#14020)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 8416c56 Fix jar path and add missing ones for spark jobs (#14020)
8416c56 is described below
commit 8416c563f2bc673f15bf890ea13be2a32afdf3ca
Author: Ashutosh Dwivedi
AuthorDate: Fri Feb 15 07:00:31 2019 +0530
Fix jar path and add missing ones for spark jobs (#14020)
* Fix jar path and add missing ones for spark jobs
Fix path of jars / add missing jars in spark job
remove print, reduce clutter
* fixes scalastyle violations
* exclude all of javadoc, sources, bundle, and src while searching for jars
* simplfied the exclude experession
---
.../apache/mxnet/spark/SharedSparkContext.scala| 42 +++---
1 file changed, 22 insertions(+), 20 deletions(-)
diff --git
a/scala-package/spark/src/test/scala/org/apache/mxnet/spark/SharedSparkContext.scala
b/scala-package/spark/src/test/scala/org/apache/mxnet/spark/SharedSparkContext.scala
index 2efd181..6d36ca5 100644
---
a/scala-package/spark/src/test/scala/org/apache/mxnet/spark/SharedSparkContext.scala
+++
b/scala-package/spark/src/test/scala/org/apache/mxnet/spark/SharedSparkContext.scala
@@ -80,30 +80,27 @@ trait SharedSparkContext extends FunSuite with
BeforeAndAfterEach with BeforeAnd
System.getProperty("user.dir")
}
- private def getJarFilePath(root: String): String = {
-for (platform <- List("linux-x86_64-cpu", "linux-x86_64-gpu",
"osx-x86_64-cpu")) {
- val jarFiles = new File(s"$root/$platform/target/").listFiles(new
FileFilter {
-override def accept(pathname: File) = {
- pathname.getAbsolutePath.endsWith(".jar") &&
-!pathname.getAbsolutePath.contains("javadoc") &&
-!pathname.getAbsolutePath.contains("sources")
-}
- })
- if (jarFiles != null && jarFiles.nonEmpty) {
-return jarFiles.head.getAbsolutePath
+ private def findJars(root: String): Array[File] = {
+val excludedSuffixes = List("bundle", "src", "javadoc", "sources")
+new File(root).listFiles(new FileFilter {
+ override def accept(pathname: File) = {
+pathname.getAbsolutePath.endsWith(".jar") &&
+ excludedSuffixes.forall(!pathname.getAbsolutePath.contains(_))
}
+})
+ }
+
+ private def getJarFilePath(root: String): String = {
+val jarFiles = findJars(s"$root/target/")
+if (jarFiles != null && jarFiles.nonEmpty) {
+ jarFiles.head.getAbsolutePath
+} else {
+ null
}
-null
}
private def getSparkJar: String = {
-val jarFiles = new File(s"$composeWorkingDirPath/target/").listFiles(new
FileFilter {
- override def accept(pathname: File) = {
-pathname.getAbsolutePath.endsWith(".jar") &&
- !pathname.getAbsolutePath.contains("javadoc") &&
- !pathname.getAbsolutePath.contains("sources")
- }
-})
+val jarFiles = findJars(s"$composeWorkingDirPath/target/")
if (jarFiles != null && jarFiles.nonEmpty) {
jarFiles.head.getAbsolutePath
} else {
@@ -111,6 +108,9 @@ trait SharedSparkContext extends FunSuite with
BeforeAndAfterEach with BeforeAnd
}
}
+ private def getNativeJars(root: String): String =
+new File(root).listFiles().map(_.toPath).mkString(",")
+
protected def buildLeNet(): MXNet = {
val workingDir = composeWorkingDirPath
val assemblyRoot = s"$workingDir/../assembly"
@@ -130,6 +130,8 @@ trait SharedSparkContext extends FunSuite with
BeforeAndAfterEach with BeforeAnd
protected def buildMlp(): MXNet = {
val workingDir = composeWorkingDirPath
val assemblyRoot = s"$workingDir/../assembly"
+val nativeRoot = s"$workingDir/../native/target/lib"
+
new MXNet()
.setBatchSize(128)
.setLabelName("softmax_label")
@@ -139,7 +141,7 @@ trait SharedSparkContext extends FunSuite with
BeforeAndAfterEach with BeforeAnd
.setNumEpoch(10)
.setNumServer(1)
.setNumWorker(numWorkers)
- .setExecutorJars(s"${getJarFilePath(assemblyRoot)},$getSparkJar")
+
.setExecutorJars(s"${getJarFilePath(assemblyRoot)},$getSparkJar,${getNativeJars(nativeRoot)}")
.setJava("java")
.setTimeout(0)
}
[incubator-mxnet] branch v1.4.x updated: Fix gtest build (#13926)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch v1.4.x in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/v1.4.x by this push: new a03d59e Fix gtest build (#13926) a03d59e is described below commit a03d59ed867ba334d78d61246a1090cd1868f5da Author: Zhennan Qin AuthorDate: Fri Feb 15 02:40:30 2019 +0800 Fix gtest build (#13926) --- tests/cpp/unittest.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/cpp/unittest.mk b/tests/cpp/unittest.mk index 746ee2f..bca0389 100644 --- a/tests/cpp/unittest.mk +++ b/tests/cpp/unittest.mk @@ -25,6 +25,8 @@ GTEST_SRCS_ = $(GTEST_DIR)/src/*.cc $(GTEST_DIR)/src/*.h $(GTEST_HEADERS) GTEST_HEADERS = $(GTEST_DIR)/include/gtest/*.h \ $(GTEST_DIR)/include/gtest/internal/*.h +GTEST_FLAGS=-Wall -Wshadow -DGTEST_HAS_PTHREAD=1 -fexceptions -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -std=c++11 + TEST_CFLAGS = -Itests/cpp/include -Isrc $(CFLAGS) TEST_LDFLAGS = $(LDFLAGS) -Llib -lmxnet @@ -36,7 +38,7 @@ endif .PHONY: runtest testclean gtest-all.o : $(GTEST_SRCS_) - $(CXX) $(CPPFLAGS) -I$(GTEST_INC) -I$(GTEST_DIR) $(CXXFLAGS) -c $(GTEST_DIR)/src/gtest-all.cc + $(CXX) $(CPPFLAGS) $(GTEST_FLAGS) -I$(GTEST_INC) -I$(GTEST_DIR) $(CXXFLAGS) -c $(GTEST_DIR)/src/gtest-all.cc gtest.a : gtest-all.o $(AR) $(ARFLAGS) $@ $^
[incubator-mxnet] branch master updated: Updated the MLP test to accept the number of epochs. Reduced the epochs in ci_test.sh to shorten the CI build time (#14149)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 518cd40 Updated the MLP test to accept the number of epochs. Reduced
the epochs in ci_test.sh to shorten the CI build time (#14149)
518cd40 is described below
commit 518cd408a2c1d82dd5342268dfc80c41e427b765
Author: Amol Lele <19983848+lelea...@users.noreply.github.com>
AuthorDate: Thu Feb 14 09:55:09 2019 -0800
Updated the MLP test to accept the number of epochs. Reduced the epochs in
ci_test.sh to shorten the CI build time (#14149)
---
cpp-package/example/mlp.cpp | 7 +++
cpp-package/tests/ci_test.sh | 10 +-
2 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/cpp-package/example/mlp.cpp b/cpp-package/example/mlp.cpp
index cc16f53..c3760fd 100644
--- a/cpp-package/example/mlp.cpp
+++ b/cpp-package/example/mlp.cpp
@@ -49,7 +49,7 @@ void OutputAccuracy(mx_float* pred, mx_float* target) {
std::cout << "Accuracy: " << right / 128.0 << std::endl;
}
-void MLP() {
+void MLP(int max_epoch) {
auto sym_x = Symbol::Variable("X");
auto sym_label = Symbol::Variable("label");
@@ -144,7 +144,6 @@ void MLP() {
grad_req_type, aux_states);
std::cout << "Training" << std::endl;
- int max_epoch = 15000;
mx_float learning_rate = 0.0001;
for (int epoch_num = 0; epoch_num < max_epoch; ++epoch_num) {
exe->Forward(true);
@@ -173,8 +172,8 @@ void MLP() {
}
int main(int argc, char** argv) {
- MLP();
+ int max_epoch = argc > 1 ? strtol(argv[1], NULL, 10) : 15000;
+ MLP(max_epoch);
MXNotifyShutdown();
return 0;
}
-
diff --git a/cpp-package/tests/ci_test.sh b/cpp-package/tests/ci_test.sh
index 7abdef4..18fabea 100755
--- a/cpp-package/tests/ci_test.sh
+++ b/cpp-package/tests/ci_test.sh
@@ -25,22 +25,22 @@ ls -l ../../lib/
./get_data.sh
cp ../../build/cpp-package/example/lenet .
-./lenet 10
+./lenet 1
cp ../../build/cpp-package/example/alexnet .
./alexnet 1
cp ../../build/cpp-package/example/lenet_with_mxdataiter .
-./lenet_with_mxdataiter 5
+./lenet_with_mxdataiter 1
cp ../../build/cpp-package/example/resnet .
-./resnet 5
+./resnet 1
cp ../../build/cpp-package/example/inception_bn .
-./inception_bn 5
+./inception_bn 1
cp ../../build/cpp-package/example/mlp .
-./mlp
+./mlp 150
cp ../../build/cpp-package/example/mlp_cpu .
./mlp_cpu
[incubator-mxnet] branch master updated: update the version name (#14076)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new d684c59 update the version name (#14076) d684c59 is described below commit d684c59049970794e8f7365f58b03e13801b44a3 Author: Jake Lee AuthorDate: Thu Feb 7 08:16:18 2019 +0800 update the version name (#14076) --- scala-package/README.md | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/scala-package/README.md b/scala-package/README.md index 3859e5f..be0fc41 100644 --- a/scala-package/README.md +++ b/scala-package/README.md @@ -84,7 +84,7 @@ Also, add the dependency which corresponds to your platform to the ```dependenci org.apache.mxnet mxnet-full_2.11-linux-x86_64-gpu - [1.5.0,) + [1.5.0-SNAPSHOT,) ``` @@ -96,7 +96,7 @@ Also, add the dependency which corresponds to your platform to the ```dependenci org.apache.mxnet mxnet-full_2.11-linux-x86_64-cpu - [1.5.0,) + [1.5.0-SNAPSHOT,) ``` @@ -107,11 +107,11 @@ Also, add the dependency which corresponds to your platform to the ```dependenci org.apache.mxnet mxnet-full_2.11-osx-x86_64-cpu - [1.5.0,) + [1.5.0-SNAPSHOT,) ``` -**Note:** ```[1.5.0,)<\version>``` indicates that we will fetch packages with version 1.5.0 or higher. This will always ensure that the pom.xml is able to fetch the latest and greatest jar files from Maven Snapshot repository. +**Note:** ```[1.5.0-SNAPSHOT,)``` indicates that we will fetch packages with version 1.5.0 or higher. This will always ensure that the pom.xml is able to fetch the latest and greatest jar files from Maven Snapshot repository. Build From Source -
[incubator-mxnet] branch master updated: [MXNET-1178] updating scala docs (#14070)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 8191585 [MXNET-1178] updating scala docs (#14070)
8191585 is described below
commit 819158559e15287b910647e22387486e04817fa6
Author: Piyush Ghai
AuthorDate: Wed Feb 6 16:13:46 2019 -0800
[MXNET-1178] updating scala docs (#14070)
* updating scala docs
* Addressed PR feedback
---
.../src/main/scala/org/apache/mxnet/Context.scala | 17 +-
.../src/main/scala/org/apache/mxnet/NDArray.scala | 36 --
.../org/apache/mxnet/infer/ObjectDetector.scala| 3 +-
3 files changed, 31 insertions(+), 25 deletions(-)
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Context.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Context.scala
index ab44f43..b04cd31 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Context.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Context.scala
@@ -38,11 +38,12 @@ object Context {
}
/**
- * Constructing a context.
-
- * @param deviceTypeName {'cpu', 'gpu'} String representing the device type
- * @param deviceId (default=0) The device id of the device, needed for GPU
- */
+ * Constructing a context which is used to specify the device and device type
that will
+ * be utilized by the engine.
+ *
+ * @param deviceTypeName {'cpu', 'gpu'} String representing the device type
+ * @param deviceId (default=0) The device id of the device, needed for GPU
+ */
class Context(deviceTypeName: String, val deviceId: Int = 0) extends
Serializable {
val deviceTypeid: Int = Context.devstr2type(deviceTypeName)
@@ -61,9 +62,9 @@ class Context(deviceTypeName: String, val deviceId: Int = 0)
extends Serializabl
}
/**
- * Return device type of current context.
- * @return device_type
- */
+* Return device type of current context.
+* @return device_type
+*/
def deviceType: String = Context.devtype2str(deviceTypeid)
override def toString: String = {
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 4324b3d..ca2e986 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -728,12 +728,15 @@ object NDArray extends NDArrayBase {
}
/**
- * NDArray object in mxnet.
- * NDArray is basic ndarray/Tensor like data structure in mxnet.
- *
- * WARNING: it is your responsibility to clear this object through dispose().
- *
- */
+ * NDArray object in mxnet.
+ * NDArray is basic ndarray/Tensor like data structure in mxnet.
+ *
+ * NOTE: NDArray is stored in native memory. Use NDArray in a
try-with-resources() construct
+ * or a [[org.apache.mxnet.ResourceScope]] in a try-with-resource to have them
+ * automatically disposed. You can explicitly control the lifetime of NDArray
+ * by calling dispose manually. Failure to do this will result in leaking
native memory.
+ *
+ */
class NDArray private[mxnet](private[mxnet] val handle: NDArrayHandle,
val writable: Boolean = true,
addToCollector: Boolean = true) extends
NativeResource {
@@ -775,21 +778,22 @@ class NDArray private[mxnet](private[mxnet] val handle:
NDArrayHandle,
}
/**
- * Dispose all NDArrays who help to construct this array.
- * e.g. (a * b + c).disposeDeps() will dispose a, b, c (including their
deps) and a * b
- * @return this array
- */
+* Dispose all NDArrays who help to construct this array.
+* e.g. (a * b + c).disposeDeps() will dispose a, b, c (including their
deps) and a * b
+* @return this NDArray
+*/
def disposeDeps(): NDArray = {
disposeDepsExcept()
}
/**
- * Dispose all NDArrays who help to construct this array, excepts those in
the arguments.
- * e.g. (a * b + c).disposeDepsExcept(a, b)
- * will dispose c and a * b.
- * Note that a, b's dependencies will not be disposed either.
- * @return this array
- */
+* Dispose all NDArrays who help to construct this array, excepts those in
the arguments.
+* e.g. (a * b + c).disposeDepsExcept(a, b)
+* will dispose c and a * b.
+* Note that a, b's dependencies will not be disposed either.
+* @param arrs array of NDArrays
+* @return this array
+*/
def disposeDepsExcept(arrs: NDArray*): NDArray = {
if (dependencies != null) {
val excepts = mutable.HashSet.empty[Long]
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/ObjectDetector.scala
index 78b237a..7146156 100644
---
a/scala-package/infer/src/main/scala/org/apache
[incubator-mxnet] branch master updated: Add maven wraper to scala project. (#13702)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 9e14f14 Add maven wraper to scala project. (#13702)
9e14f14 is described below
commit 9e14f148a8818fe30d5e93683da33737953c
Author: Frank Liu
AuthorDate: Tue Feb 5 19:01:11 2019 -0800
Add maven wraper to scala project. (#13702)
---
scala-package/.mvn/wrapper/.gitignore | 2 +
.../.mvn/wrapper/MavenWrapperDownloader.java | 110
.../.mvn/wrapper/maven-wrapper.properties | 1 +
scala-package/mvnw | 286 +
scala-package/mvnw.cmd | 161
5 files changed, 560 insertions(+)
diff --git a/scala-package/.mvn/wrapper/.gitignore
b/scala-package/.mvn/wrapper/.gitignore
new file mode 100644
index 000..576738f
--- /dev/null
+++ b/scala-package/.mvn/wrapper/.gitignore
@@ -0,0 +1,2 @@
+maven-wrapper.jar
+
diff --git a/scala-package/.mvn/wrapper/MavenWrapperDownloader.java
b/scala-package/.mvn/wrapper/MavenWrapperDownloader.java
new file mode 100755
index 000..fa4f7b4
--- /dev/null
+++ b/scala-package/.mvn/wrapper/MavenWrapperDownloader.java
@@ -0,0 +1,110 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+*/
+
+import java.net.*;
+import java.io.*;
+import java.nio.channels.*;
+import java.util.Properties;
+
+public class MavenWrapperDownloader {
+
+/**
+ * Default URL to download the maven-wrapper.jar from, if no 'downloadUrl'
is provided.
+ */
+private static final String DEFAULT_DOWNLOAD_URL =
+
"https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.4.2/maven-wrapper-0.4.2.jar;;
+
+/**
+ * Path to the maven-wrapper.properties file, which might contain a
downloadUrl property to
+ * use instead of the default one.
+ */
+private static final String MAVEN_WRAPPER_PROPERTIES_PATH =
+".mvn/wrapper/maven-wrapper.properties";
+
+/**
+ * Path where the maven-wrapper.jar will be saved to.
+ */
+private static final String MAVEN_WRAPPER_JAR_PATH =
+".mvn/wrapper/maven-wrapper.jar";
+
+/**
+ * Name of the property which should be used to override the default
download url for the wrapper.
+ */
+private static final String PROPERTY_NAME_WRAPPER_URL = "wrapperUrl";
+
+public static void main(String args[]) {
+System.out.println("- Downloader started");
+File baseDirectory = new File(args[0]);
+System.out.println("- Using base directory: " +
baseDirectory.getAbsolutePath());
+
+// If the maven-wrapper.properties exists, read it and check if it
contains a custom
+// wrapperUrl parameter.
+File mavenWrapperPropertyFile = new File(baseDirectory,
MAVEN_WRAPPER_PROPERTIES_PATH);
+String url = DEFAULT_DOWNLOAD_URL;
+if(mavenWrapperPropertyFile.exists()) {
+FileInputStream mavenWrapperPropertyFileInputStream = null;
+try {
+mavenWrapperPropertyFileInputStream = new
FileInputStream(mavenWrapperPropertyFile);
+Properties mavenWrapperProperties = new Properties();
+
mavenWrapperProperties.load(mavenWrapperPropertyFileInputStream);
+url =
mavenWrapperProperties.getProperty(PROPERTY_NAME_WRAPPER_URL, url);
+} catch (IOException e) {
+System.out.println("- ERROR loading '" +
MAVEN_WRAPPER_PROPERTIES_PATH + "'");
+} finally {
+try {
+if(mavenWrapperPropertyFileInputStream != null) {
+mavenWrapperPropertyFileInputStream.close();
+}
+} catch (IOException e) {
+// Ignore ...
+}
+}
+}
+System.out.println("- Downloading from: : " + url);
+
+File outputFile = new File(baseDirectory.getAbsolutePath(),
MAVEN_WRAPPER_JAR_PATH
[incubator-mxnet] branch master updated: Now passing DType of Label downstream to Label's DataDesc object (#14038)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new f95e794 Now passing DType of Label downstream to Label's DataDesc
object (#14038)
f95e794 is described below
commit f95e7949dcd96ca2a5a140dbcff16dd344b45d19
Author: Piyush Ghai
AuthorDate: Thu Jan 31 17:38:26 2019 -0800
Now passing DType of Label downstream to Label's DataDesc object (#14038)
---
.../core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala | 6 --
scala-package/core/src/test/scala/org/apache/mxnet/IOSuite.scala| 3 ++-
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala
index e690abb..b205bbe 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/io/NDArrayIter.scala
@@ -63,7 +63,8 @@ class NDArrayIter(data: IndexedSeq[(DataDesc, NDArray)],
dataName: String = "data", labelName: String = "label") {
this(IO.initDataDesc(data, allowEmpty = false, dataName,
if (data == null || data.isEmpty) MX_REAL_TYPE else data(0).dtype,
Layout.UNDEFINED),
- IO.initDataDesc(label, allowEmpty = true, labelName, MX_REAL_TYPE,
Layout.UNDEFINED),
+ IO.initDataDesc(label, allowEmpty = true, labelName,
+if (label == null || label.isEmpty) MX_REAL_TYPE else label(0).dtype,
Layout.UNDEFINED),
dataBatchSize, shuffle, lastBatchHandle)
}
@@ -175,7 +176,8 @@ class NDArrayIter(data: IndexedSeq[(DataDesc, NDArray)],
private def _padData(ndArray: NDArray): NDArray = {
val padNum = cursor + dataBatchSize - numData
val shape = Shape(dataBatchSize) ++ ndArray.shape.slice(1,
ndArray.shape.size)
-val newArray = NDArray.zeros(shape)
+// The new NDArray has to be created such that it inherits dtype from the
passed in array
+val newArray = NDArray.zeros(shape, dtype = ndArray.dtype)
NDArrayCollector.auto().withScope {
val batch = ndArray.slice(cursor, numData)
val padding = ndArray.slice(0, padNum)
diff --git a/scala-package/core/src/test/scala/org/apache/mxnet/IOSuite.scala
b/scala-package/core/src/test/scala/org/apache/mxnet/IOSuite.scala
index d3969b0..698a2b5 100644
--- a/scala-package/core/src/test/scala/org/apache/mxnet/IOSuite.scala
+++ b/scala-package/core/src/test/scala/org/apache/mxnet/IOSuite.scala
@@ -237,7 +237,7 @@ class IOSuite extends FunSuite with BeforeAndAfterAll {
val shape0 = Shape(Array(1000, 2, 2))
val data = IndexedSeq(NDArray.ones(shape0), NDArray.zeros(shape0))
val shape1 = Shape(Array(1000, 1))
-val label = IndexedSeq(NDArray.ones(shape1))
+val label = IndexedSeq(NDArray.ones(shape1, dtype = DType.Int32))
val batchData0 = NDArray.ones(Shape(Array(128, 2, 2)))
val batchData1 = NDArray.zeros(Shape(Array(128, 2, 2)))
val batchLabel = NDArray.ones(Shape(Array(128, 1)))
@@ -254,6 +254,7 @@ class IOSuite extends FunSuite with BeforeAndAfterAll {
assert(tBatch.data(0).toArray === batchData0.toArray)
assert(tBatch.data(1).toArray === batchData1.toArray)
assert(tBatch.label(0).toArray === batchLabel.toArray)
+ assert(tBatch.label(0).dtype == DType.Int32)
}
assert(batchCount === nBatch0)
[incubator-mxnet] branch master updated: [MXNET-1180] Java Image API (#13807)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 9a3e4a0 [MXNET-1180] Java Image API (#13807)
9a3e4a0 is described below
commit 9a3e4a02ded9c6d2c304557140dac0c9991d507e
Author: Lanking
AuthorDate: Thu Jan 31 10:55:12 2019 -0800
[MXNET-1180] Java Image API (#13807)
* add java example
* add test and change PredictorExample
* add image change
* Add minor fixes
* add License
* add predictor Example tests
* fix the issue with JUnit test
* Satisfy Lint God ʕ •ᴥ•ʔ
* update the pom file config
* update documentation
* add simplified methods
---
scala-package/core/pom.xml | 6 --
.../src/main/scala/org/apache/mxnet/Image.scala| 6 +-
.../src/main/scala/org/apache/mxnet/NDArray.scala | 6 +-
.../scala/org/apache/mxnet/javaapi/Image.scala | 114 +
.../java/org/apache/mxnet/javaapi/ImageTest.java | 67
scala-package/examples/pom.xml | 1 +
.../javaapi/infer/predictor/PredictorExample.java | 88 ++--
.../main/scala/org/apache/mxnetexamples/Util.scala | 4 +-
.../infer/predictor/PredictorExampleTest.java | 67
scala-package/infer/pom.xml| 8 --
.../apache/mxnet/javaapi/JavaNDArrayMacro.scala| 4 +-
scala-package/pom.xml | 6 ++
12 files changed, 274 insertions(+), 103 deletions(-)
diff --git a/scala-package/core/pom.xml b/scala-package/core/pom.xml
index 7264c39..4de65c0 100644
--- a/scala-package/core/pom.xml
+++ b/scala-package/core/pom.xml
@@ -139,12 +139,6 @@
provided
- junit
- junit
- 4.11
- test
-
-
commons-io
commons-io
2.1
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
index 77881ab..0f756e2 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/Image.scala
@@ -37,7 +37,7 @@ object Image {
* @param flag Convert decoded image to grayscale (0) or color (1).
* @param to_rgb Whether to convert decoded image
* to mxnet's default RGB format (instead of opencv's default
BGR).
-* @return NDArray in HWC format
+* @return NDArray in HWC format with DType [[DType.UInt8]]
*/
def imDecode(buf: Array[Byte], flag: Int,
to_rgb: Boolean,
@@ -56,7 +56,7 @@ object Image {
/**
* Same imageDecode with InputStream
* @param inputStream the inputStream of the image
-* @return NDArray in HWC format
+* @return NDArray in HWC format with DType [[DType.UInt8]]
*/
def imDecode(inputStream: InputStream, flag: Int = 1,
to_rgb: Boolean = true,
@@ -78,7 +78,7 @@ object Image {
* @param flag Convert decoded image to grayscale (0) or color (1).
* @param to_rgb Whether to convert decoded image to mxnet's default RGB
format
* (instead of opencv's default BGR).
-* @return org.apache.mxnet.NDArray in HWC format
+* @return org.apache.mxnet.NDArray in HWC format with DType [[DType.UInt8]]
*/
def imRead(filename: String, flag: Option[Int] = None,
to_rgb: Option[Boolean] = None,
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 5c345f2..4324b3d 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -97,9 +97,11 @@ object NDArray extends NDArrayBase {
case ndArr: Seq[NDArray @unchecked] =>
if (ndArr.head.isInstanceOf[NDArray]) (ndArr.toArray,
ndArr.toArray.map(_.handle))
else throw new IllegalArgumentException(
- "Unsupported out var type, should be NDArray or subclass of
Seq[NDArray]")
+ s"""Unsupported out ${output.getClass} type,
+ | should be NDArray or subclass of
Seq[NDArray]""".stripMargin)
case _ => throw new IllegalArgumentException(
-"Unsupported out var type, should be NDArray or subclass of
Seq[NDArray]")
+s"""Unsupported out ${output.getClass} type,
+ | should be NDArray or subclass of Seq[NDArray]""".stripMargin)
}
} else {
(null, null)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/javaapi/Image.sc
[incubator-mxnet] branch master updated: [MXNET-1232] fix demo and add Eclipse support (#13979)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new c4080de [MXNET-1232] fix demo and add Eclipse support (#13979)
c4080de is described below
commit c4080def0ef36b4f69ae0927b31ced6dcba6f9df
Author: Lanking
AuthorDate: Mon Jan 28 13:34:40 2019 -0800
[MXNET-1232] fix demo and add Eclipse support (#13979)
* fix demo and add Eclipse support
* fix on docs
* fix typo
* Update docs/install/java_setup.md
Co-Authored-By: lanking520
* add fixes in docs
---
ci/docker/runtime_functions.sh| 4 +-
docs/install/java_setup.md| 13 +--
scala-package/mxnet-demo/java-demo/Makefile | 52 -
scala-package/mxnet-demo/java-demo/README.md | 30 --
scala-package/mxnet-demo/java-demo/pom.xml| 38 ++
scala-package/mxnet-demo/scala-demo/Makefile | 56 ---
scala-package/mxnet-demo/scala-demo/README.md | 21 +-
scala-package/mxnet-demo/scala-demo/pom.xml | 25
8 files changed, 98 insertions(+), 141 deletions(-)
diff --git a/ci/docker/runtime_functions.sh b/ci/docker/runtime_functions.sh
index f763137..d97da64 100755
--- a/ci/docker/runtime_functions.sh
+++ b/ci/docker/runtime_functions.sh
@@ -1241,7 +1241,7 @@ nightly_tutorial_test_ubuntu_python2_gpu() {
nightly_java_demo_test_cpu() {
set -ex
cd /work/mxnet/scala-package/mxnet-demo/java-demo
-make java_ci_demo
+mvn -Pci-nightly install
bash bin/java_sample.sh
bash bin/run_od.sh
}
@@ -1249,7 +1249,7 @@ nightly_java_demo_test_cpu() {
nightly_scala_demo_test_cpu() {
set -ex
cd /work/mxnet/scala-package/mxnet-demo/scala-demo
-make scala_ci_demo
+mvn -Pci-nightly install
bash bin/demo.sh
bash bin/run_im.sh
}
diff --git a/docs/install/java_setup.md b/docs/install/java_setup.md
index 0075e92..ea4bb50 100644
--- a/docs/install/java_setup.md
+++ b/docs/install/java_setup.md
@@ -85,6 +85,13 @@ Also, add the dependency which corresponds to your platform
to the `dependencies
The official Java Packages will be released with the release of MXNet 1.4 and
will be available on [MXNet Maven package
repository](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.mxnet%22).
+### Eclipse IDE Support
+You can convert your existing Maven project to a project that can run in
Eclipse by:
+```
+mvn eclipse:eclipse
+```
+This can be done once you have your maven project properly configured.
+
## Source
The previously mentioned setup with Maven is recommended. Otherwise, the
following instructions for macOS and Ubuntu are provided for reference only:
@@ -99,11 +106,11 @@ The previously mentioned setup with Maven is recommended.
Otherwise, the followi
Build Java from an Existing MXNet Installation
-If you have already built MXNet **from source** and are looking to setup Java
from that point, you may simply run the following from the MXNet source root:
+If you have already built MXNet **from source** and are looking to setup Java
from that point, you may simply run the following from the MXNet
`scala-package` folder:
```
-make scalapkg
-make scalainstall
+mvn package
+mvn install
```
This will install both the Java Inference API and the required MXNet-Scala
package.
diff --git a/scala-package/mxnet-demo/java-demo/Makefile
b/scala-package/mxnet-demo/java-demo/Makefile
deleted file mode 100644
index 3811d75..000
--- a/scala-package/mxnet-demo/java-demo/Makefile
+++ /dev/null
@@ -1,52 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-SCALA_VERSION_PROFILE := 2.11
-
-ifeq ($(OS),Windows_NT)
- UNAME_S := Windows
-else
- UNAME_S := $(shell uname -s)
-endif
-
-ifeq ($(UNAME_S), Windows)
- # TODO: currently scala package does not support windows
- SCALA_PKG_PROFILE := windows
-else
- ifeq ($(UNAME_S), Darwin)
- SCALA_PKG_PROFILE := osx-x86_64-cpu
- else
- SCALA_PKG_PRO
[incubator-mxnet] branch master updated: Update scala-package gitignore configuration. (#13962)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 7b9779a Update scala-package gitignore configuration. (#13962) 7b9779a is described below commit 7b9779a8c9884370e6a84da9675d49b23f35b4a7 Author: Frank Liu AuthorDate: Mon Jan 28 11:28:48 2019 -0800 Update scala-package gitignore configuration. (#13962) --- .gitignore | 35 ++- scala-package/.gitignore | 1 + 2 files changed, 11 insertions(+), 25 deletions(-) diff --git a/.gitignore b/.gitignore index fc79926..d989888 100644 --- a/.gitignore +++ b/.gitignore @@ -63,11 +63,9 @@ __pycache__ *.states *.json *.d -build cmake-build* data recommonmark -deps # R *.Rcheck @@ -96,6 +94,8 @@ input.txt* # Jetbrain .idea +.gradle +*.iml # ctags tags @@ -104,28 +104,14 @@ tags cscope.out cscope.files -# Scala package -*.class -scala-package/*/target/ -scala-package/*/*/target/ -*.scala_dependencies -*.worksheet -*.idea -*.iml -*.classpath -*.project -*.settings -!scala-package/*/bin -*.bak -*/node_modules/ - # Eclipse project config .project .cproject +.classpath +.settings .pydevproject CMakeFiles cmake_install.cmake -lib # Visual Studio Code .vscode @@ -145,13 +131,12 @@ tools/pip_package/mxnet.egg-info tools/pip_package/mxnet # temporary path for building dependencies when building wheel -./deps/ -./staticdeps/ -bld -./tmp/* -*.jar -target -bin/im2rec +deps/ +staticdeps/ +tmp/ +build/ +lib/ +bin/ model/ # VTune diff --git a/scala-package/.gitignore b/scala-package/.gitignore index 22b12b3..9bf7851 100644 --- a/scala-package/.gitignore +++ b/scala-package/.gitignore @@ -1,3 +1,4 @@ +target/ .flattened-pom.xml core/src/main/scala/org/apache/mxnet/NDArrayAPIBase.scala core/src/main/scala/org/apache/mxnet/NDArrayBase.scala
[incubator-mxnet] branch master updated: [MXNET-1000] get Ndarray real value and form it from a NDArray (#12690)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 0f334ae [MXNET-1000] get Ndarray real value and form it from a
NDArray (#12690)
0f334ae is described below
commit 0f334aecf569c2f0ed5a279798e0ca58c4d143dc
Author: Lanking
AuthorDate: Fri Jan 25 12:01:02 2019 -0800
[MXNET-1000] get Ndarray real value and form it from a NDArray (#12690)
* add visualize
* adding Any type input to form NDArray
* fix bug and add tests
* add a toString method
* add Visualize Util and migrate visualize structure to there
* update with tests
* refactor code
* fix the minor issue
* add multiple types support
* add changes on names and tests
* make code elegant and improve readability
---
.../scala/org/apache/mxnet/MX_PRIMITIVES.scala | 6 ++
.../src/main/scala/org/apache/mxnet/NDArray.scala | 112 -
.../test/scala/org/apache/mxnet/NDArraySuite.scala | 82 +++
3 files changed, 199 insertions(+), 1 deletion(-)
diff --git
a/scala-package/core/src/main/scala/org/apache/mxnet/MX_PRIMITIVES.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/MX_PRIMITIVES.scala
index cb97885..3a51222 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/MX_PRIMITIVES.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/MX_PRIMITIVES.scala
@@ -82,4 +82,10 @@ object MX_PRIMITIVES {
implicit def MX_DoubleToDouble(d: MX_Double) : Double = d.data
+ def isValidMxPrimitiveType(num : Any) : Boolean = {
+num match {
+ case valid @ (_: Float | _: Double) => true
+ case _ => false
+}
+ }
}
diff --git a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
index 163ed26..5c345f2 100644
--- a/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
+++ b/scala-package/core/src/main/scala/org/apache/mxnet/NDArray.scala
@@ -28,6 +28,7 @@ import scala.collection.mutable
import scala.collection.mutable.{ArrayBuffer, ListBuffer}
import scala.language.implicitConversions
import scala.ref.WeakReference
+import scala.util.Try
/**
* NDArray Object extends from NDArrayBase for abstract function signatures
@@ -510,6 +511,61 @@ object NDArray extends NDArrayBase {
}
/**
+* Create a new NDArray based on the structure of source Array
+* @param sourceArr Array[Array...Array[MX_PRIMITIVE_TYPE]...]
+* @param ctx context like to pass in
+* @return an NDArray with the same shape of the input
+* @throws IllegalArgumentException if the data type is not valid
+*/
+ def toNDArray(sourceArr: Array[_], ctx : Context = null) : NDArray = {
+val shape = shapeGetter(sourceArr)
+val container = new Array[Any](shape.product)
+flattenArray(sourceArr, container, 0, container.length - 1)
+val finalArr = container(0) match {
+ case f: Float => array(container.map(_.asInstanceOf[Float]),
Shape(shape), ctx)
+ case d: Double => array(container.map(_.asInstanceOf[Double]),
Shape(shape), ctx)
+ case _ => throw new IllegalArgumentException(
+s"Unsupported type ${container(0).getClass}, please check
MX_PRIMITIVES for valid types")
+}
+finalArr
+ }
+
+ private def shapeGetter(sourceArr : Any) : ArrayBuffer[Int] = {
+sourceArr match {
+// e.g : Array[Double] the inner layer
+ case arr: Array[_] if MX_PRIMITIVES.isValidMxPrimitiveType(arr(0)) => {
+ArrayBuffer[Int](arr.length)
+ }
+// e.g : Array[Array...[]]
+ case arr: Array[_] => {
+var arrBuffer = new ArrayBuffer[Int]()
+if (!arr.isEmpty) arrBuffer = shapeGetter(arr(0))
+for (idx <- arr.indices) {
+ require(arrBuffer == shapeGetter(arr(idx)))
+}
+arrBuffer.insert(0, arr.length)
+arrBuffer
+ }
+ case _ => throw new IllegalArgumentException(s"Wrong type passed:
${sourceArr.getClass}")
+}
+ }
+
+ private def flattenArray(sourceArr : Any, arr : Array[Any],
+start : Int, end : Int) : Unit = {
+sourceArr match {
+ case arrValid: Array[_] if
MX_PRIMITIVES.isValidMxPrimitiveType(arrValid(0)) => {
+for (i <- arrValid.indices) arr(start + i) = arrValid(i)
+ }
+ case arrAny: Array[_] => {
+val fragment = (end - start + 1) / arrAny.length
+for (i <- arrAny.indices)
+ flattenArray(arrAny(i), arr, start + i * fragment, start + (i + 1) *
fragment)
+ }
+ case _ => throw new IllegalArgumentException(s"Wrong type passed:
${sourceArr.getClass}")
+}
+ }
+
+ /**
* Returns evenly spaced values within a
[incubator-mxnet] branch master updated: [MXNET-1293] Adding Iterables instead of List to method signature for infer APIs in Java (#13977)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 24412df [MXNET-1293] Adding Iterables instead of List to method
signature for infer APIs in Java (#13977)
24412df is described below
commit 24412df84934b57f6e5ed7ac135bc6fe5402cff2
Author: Piyush Ghai
AuthorDate: Thu Jan 24 11:36:55 2019 -0800
[MXNET-1293] Adding Iterables instead of List to method signature for infer
APIs in Java (#13977)
* Added Iterables as input type instead of List in Predictor for Java
* Added Iterables to ObjectDetector API
* Added tests for Predictor API
* Added tests for ObjectDetector
---
.../mxnet/infer/javaapi/ObjectDetector.scala | 10
.../org/apache/mxnet/infer/javaapi/Predictor.scala | 12 -
.../mxnet/infer/javaapi/ObjectDetectorTest.java| 25 +++
.../apache/mxnet/infer/javaapi/PredictorTest.java | 29 +++---
4 files changed, 62 insertions(+), 14 deletions(-)
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/ObjectDetector.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/ObjectDetector.scala
index 3014f8d..05334e4 100644
---
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/ObjectDetector.scala
+++
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/ObjectDetector.scala
@@ -44,8 +44,8 @@ import scala.language.implicitConversions
*/
class ObjectDetector private[mxnet] (val objDetector:
org.apache.mxnet.infer.ObjectDetector){
- def this(modelPathPrefix: String, inputDescriptors:
java.util.List[DataDesc], contexts:
- java.util.List[Context], epoch: Int)
+ def this(modelPathPrefix: String, inputDescriptors:
java.lang.Iterable[DataDesc], contexts:
+ java.lang.Iterable[Context], epoch: Int)
= this {
val informationDesc =
JavaConverters.asScalaIteratorConverter(inputDescriptors.iterator)
.asScala.toIndexedSeq map {a => a: org.apache.mxnet.DataDesc}
@@ -79,7 +79,7 @@ class ObjectDetector private[mxnet] (val objDetector:
org.apache.mxnet.infer.Obj
* @return List of list of tuples of
* (class, [probability, xmin, ymin, xmax, ymax])
*/
- def objectDetectWithNDArray(input: java.util.List[NDArray], topK: Int):
+ def objectDetectWithNDArray(input: java.lang.Iterable[NDArray], topK: Int):
java.util.List[java.util.List[ObjectDetectorOutput]] = {
val ret =
objDetector.objectDetectWithNDArray(convert(input.asScala.toIndexedSeq),
Some(topK))
(ret map {a => (a map {e => new ObjectDetectorOutput(e._1,
e._2)}).asJava}).asJava
@@ -92,7 +92,7 @@ class ObjectDetector private[mxnet] (val objDetector:
org.apache.mxnet.infer.Obj
* @param topK Number of result elements to return, sorted by
probability
* @return List of list of tuples of (class, probability)
*/
- def imageBatchObjectDetect(inputBatch: java.util.List[BufferedImage], topK:
Int):
+ def imageBatchObjectDetect(inputBatch: java.lang.Iterable[BufferedImage],
topK: Int):
java.util.List[java.util.List[ObjectDetectorOutput]] = {
val ret = objDetector.imageBatchObjectDetect(inputBatch.asScala,
Some(topK))
(ret map {a => (a map {e => new ObjectDetectorOutput(e._1,
e._2)}).asJava}).asJava
@@ -122,7 +122,7 @@ object ObjectDetector {
org.apache.mxnet.infer.ImageClassifier.bufferedImageToPixels(resizedImage,
inputImageShape)
}
- def loadInputBatch(inputImagePaths: java.util.List[String]):
java.util.List[BufferedImage] = {
+ def loadInputBatch(inputImagePaths: java.lang.Iterable[String]):
java.util.List[BufferedImage] = {
org.apache.mxnet.infer.ImageClassifier
.loadInputBatch(inputImagePaths.asScala.toList).toList.asJava
}
diff --git
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/Predictor.scala
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/Predictor.scala
index 146fe93..6c0871f 100644
---
a/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/Predictor.scala
+++
b/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi/Predictor.scala
@@ -40,8 +40,8 @@ import scala.collection.JavaConverters._
// JavaDoc description of class to be updated in
https://issues.apache.org/jira/browse/MXNET-1178
class Predictor private[mxnet] (val predictor:
org.apache.mxnet.infer.Predictor){
- def this(modelPathPrefix: String, inputDescriptors: java.util.List[DataDesc],
- contexts: java.util.List[Context], epoch: Int)
+ def this(modelPathPrefix: String, inputDescriptors:
java.lang.Iterable[DataDesc],
+ contexts: java.lang.Iterable[Context], epoch: Int)
= this {
val
[incubator-mxnet] branch v1.4.x updated: [v1.4.x] Support populating errors back to MXNet engine in callback (#13932)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch v1.4.x
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/v1.4.x by this push:
new 191f50ab [v1.4.x] Support populating errors back to MXNet engine in
callback (#13932)
191f50ab is described below
commit 191f50abc37e000133da8fe1c45bda3a53c3c22a
Author: Yuxi Hu
AuthorDate: Fri Jan 18 15:53:35 2019 -0800
[v1.4.x] Support populating errors back to MXNet engine in callback (#13932)
* add an optional error_msg in engine on_complete callbcak
* use dmlc::Error struct to make error population extendable
---
include/mxnet/engine.h| 8
src/engine/naive_engine.cc| 3 ++-
src/engine/threaded_engine.cc | 8 ++--
src/engine/threaded_engine.h | 3 ++-
4 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/include/mxnet/engine.h b/include/mxnet/engine.h
index e02b995..408a70a 100644
--- a/include/mxnet/engine.h
+++ b/include/mxnet/engine.h
@@ -74,15 +74,15 @@ class CallbackOnComplete {
public:
// use implicit copy and assign
/*! \brief involve the callback */
- inline void operator()() const {
-(*callback_)(engine_, param_);
+ inline void operator()(const dmlc::Error* error = nullptr) const {
+(*callback_)(engine_, param_, error);
}
private:
/*! \brief engine can see content of callback */
friend class ::mxnet::Engine;
/*! \brief the real callback */
- void (*callback_)(Engine *, void *);
+ void (*callback_)(Engine *, void *, const dmlc::Error *);
/*! \brief the engine class passed to callback */
Engine* engine_;
/*! \brief the parameter set on callback */
@@ -275,7 +275,7 @@ class MXNET_API Engine {
* \param param the paramter passed to callback.
*/
inline CallbackOnComplete CreateCallback(
- void (*callback)(Engine *, void *), void *param) {
+ void (*callback)(Engine *, void *, const dmlc::Error *), void *param) {
CallbackOnComplete ret;
ret.callback_ = callback;
ret.engine_ = this;
diff --git a/src/engine/naive_engine.cc b/src/engine/naive_engine.cc
index daff530..05b72d2 100644
--- a/src/engine/naive_engine.cc
+++ b/src/engine/naive_engine.cc
@@ -208,7 +208,8 @@ class NaiveEngine final : public Engine {
private:
// callback to oncomplete
- static void OnComplete(Engine *engine, void *param) {
+ static void OnComplete(Engine *engine, void *param,
+ const dmlc::Error* error) {
static_cast(engine)->req_completed_ = true;
}
// whether action is completed
diff --git a/src/engine/threaded_engine.cc b/src/engine/threaded_engine.cc
index 3a7587f..6a60040 100644
--- a/src/engine/threaded_engine.cc
+++ b/src/engine/threaded_engine.cc
@@ -478,10 +478,14 @@ inline void ThreadedEngine::ThrowException(ThreadedVar*
threaded_var) {
return;
}
-void ThreadedEngine::OnCompleteStatic(
-Engine *engine, void *opr_block_) {
+void ThreadedEngine::OnCompleteStatic(Engine *engine, void *opr_block_,
+ const dmlc::Error* error) {
OprBlock *opr_block = static_cast(opr_block_);
ThreadedOpr *threaded_opr = opr_block->opr;
+ if (error != nullptr) {
+auto ex_p = std::make_exception_ptr(*error);
+threaded_opr->opr_exception = std::make_shared(ex_p);
+ }
if (opr_block->profiling && threaded_opr->opr_name) {
// record operator end timestamp
opr_block->opr_profile->stop();
diff --git a/src/engine/threaded_engine.h b/src/engine/threaded_engine.h
index ccfd09d..fae120d 100644
--- a/src/engine/threaded_engine.h
+++ b/src/engine/threaded_engine.h
@@ -465,7 +465,8 @@ class ThreadedEngine : public Engine {
}
}
- static void OnCompleteStatic(Engine *engine, void *threaded_opr);
+ static void OnCompleteStatic(Engine *engine, void *threaded_opr,
+ const dmlc::Error* error);
/*! \brief append an operator to bulk */
inline void BulkAppend(SyncFn exec_fn, Context exec_ctx,
std::vector const& const_vars,
[incubator-mxnet] branch master updated: Java install info update (#13912)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new f3774c3 Java install info update (#13912) f3774c3 is described below commit f3774c38df31d9ce19e3c9af95a1574458151315 Author: IvyBazan <45951687+ivyba...@users.noreply.github.com> AuthorDate: Thu Jan 17 16:02:44 2019 -0800 Java install info update (#13912) * updated java dependency * update to duplicated java cpu * java gpu update * Updated java dependency version information --- docs/install/index.md | 38 -- 1 file changed, 4 insertions(+), 34 deletions(-) diff --git a/docs/install/index.md b/docs/install/index.md index 319e72a..f509f7d 100644 --- a/docs/install/index.md +++ b/docs/install/index.md @@ -479,9 +479,7 @@ You can use the Maven packages defined in the following dependency to include MX org.apache.mxnet mxnet-full_2.11-linux-x86_64-gpu -system -1.4.0 - /system/path/to/jar/mxnet-full_2.11-linux-x86_64-gpu-1.4.0-SNAPSHOT.jar +[1.4.0, ) ``` @@ -498,9 +496,7 @@ You can use the Maven packages defined in the following dependency to include MX org.apache.mxnet mxnet-full_2.11-linux-x86_64-cpu -system -1.4.0 - /system/path/to/jar/mxnet-full_2.11-linux-x86_64-cpu-1.4.0-SNAPSHOT.jar +[1.4.0, ) ``` @@ -781,10 +777,8 @@ You can use the Maven packages defined in the following dependency to include MX ```html org.apache.mxnet -mxnet-full_2.11-osx-x86_64-cpu -system -1.4.0 - /system/path/to/jar/mxnet-full_2.11-osx-x86_64-cpu-1.4.0-SNAPSHOT.jar +mxnet-full_2.11-linux-x86_64-cpu +[1.4.0, ) ``` @@ -797,30 +791,6 @@ Not available at this time. - - - -You can use the Maven packages defined in the following `dependency` to include MXNet in your Clojure project. To maximize leverage, the Clojure package has been built on the existing Scala package. Please refer to the MXNet-Scala setup guide for a detailed set of instructions to help you with the setup process that is required to use the Clojure dependency. - -https://mvnrepository.com/artifact/org.apache.mxnet.contrib.clojure/clojure-mxnet-osx-cpu;>https://img.shields.io/badge/org.apache.mxnet-mac cpu-green.svg" alt="maven badge"/> - -```html - -org.apache.mxnet.contrib.clojure -clojure-mxnet-osx-cpu - -``` - - - - -Not available at this time. - - - - - -
[incubator-mxnet] branch master updated: Jenkins nightly maven with static build script and gpu (#13767)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 13f7c58 Jenkins nightly maven with static build script and gpu
(#13767)
13f7c58 is described below
commit 13f7c58ef77afdfd7ff30c9c35c76100f44ea158
Author: Zach Kimberg
AuthorDate: Tue Jan 15 18:12:23 2019 -0800
Jenkins nightly maven with static build script and gpu (#13767)
---
.gitignore | 1 +
.../docker/Dockerfile.publish.ubuntu1604_cpu | 33 ++--
.../docker/Dockerfile.publish.ubuntu1604_gpu | 33 ++--
ci/docker/install/ubuntu_base.sh | 4 +
ci/docker/install/ubuntu_scala.sh | 3 -
ci/publish/Jenkinsfile | 27 +--
ci/publish/scala/build.sh | 4 +-
ci/publish/scala/deploy.sh | 16 +-
ci/publish/scala/fullDeploy.sh | 0
ci/publish/scala/test.sh | 9 +-
make/maven/maven_linux_cu92.mk | 185 +
scala-package/deploy/pom.xml | 2 +-
.../apache/mxnetexamples/rnn/ExampleRNNSuite.scala | 11 +-
scala-package/init/pom.xml | 7 +
scala-package/packageTest/README.md| 10 +-
scala-package/packageTest/examples/pom.xml | 51 +-
scala-package/packageTest/pom.xml | 2 +-
scala-package/pom.xml | 32 +++-
tools/dependencies/cityhash.sh | 8 +-
tools/dependencies/curl.sh | 8 +-
tools/dependencies/eigen.sh| 6 +-
tools/dependencies/libpng.sh | 8 +-
tools/dependencies/libtiff.sh | 8 +-
tools/dependencies/libturbojpeg.sh | 8 +-
tools/dependencies/libz.sh | 8 +-
tools/dependencies/lz4.sh | 8 +-
tools/dependencies/make_shared_dependencies.sh | 2 +
tools/dependencies/openblas.sh | 9 +-
tools/dependencies/opencv.sh | 8 +-
tools/dependencies/openssl.sh | 8 +-
tools/dependencies/protobuf.sh | 9 +-
tools/dependencies/zmq.sh | 9 +-
tools/setup_gpu_build_tools.sh | 2 +
tools/staticbuild/build.sh | 68
tools/{build => staticbuild}/build_lib.sh | 24 +--
tools/{build => staticbuild}/build_wheel.sh| 0
36 files changed, 520 insertions(+), 111 deletions(-)
diff --git a/.gitignore b/.gitignore
index 7eb8e7d..fc79926 100644
--- a/.gitignore
+++ b/.gitignore
@@ -146,6 +146,7 @@ tools/pip_package/mxnet
# temporary path for building dependencies when building wheel
./deps/
+./staticdeps/
bld
./tmp/*
*.jar
diff --git a/tools/build/build_wheel.sh
b/ci/docker/Dockerfile.publish.ubuntu1604_cpu
old mode 100755
new mode 100644
similarity index 60%
copy from tools/build/build_wheel.sh
copy to ci/docker/Dockerfile.publish.ubuntu1604_cpu
index a796341..df284be
--- a/tools/build/build_wheel.sh
+++ b/ci/docker/Dockerfile.publish.ubuntu1604_cpu
@@ -1,5 +1,4 @@
-#!/usr/bin/env bash
-
+# -*- mode: dockerfile -*-
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@@ -16,16 +15,28 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+#
+# Dockerfile to build and run MXNet on Ubuntu 16.04 for CPU
+
+FROM ubuntu:16.04
+
+WORKDIR /work/deps
+
+COPY install/ubuntu_base.sh /work/
+RUN /work/ubuntu_base.sh
+
+COPY install/ubuntu_python.sh /work/
+RUN /work/ubuntu_python.sh
-# This script builds the wheel for binary distribution and performs sanity
check.
+COPY install/ubuntu_scala.sh /work/
+RUN /work/ubuntu_scala.sh
-cd mxnet-build
-echo $(git rev-parse HEAD) >> python/mxnet/COMMIT_HASH
-cd -
+ARG USER_ID=0
+ARG GROUP_ID=0
+COPY install/ubuntu_adduser.sh /work/
+RUN /work/ubuntu_adduser.sh
-# Make wheel for testing
-python setup.py bdist_wheel
+COPY runtime_functions.sh /work/
-wheel_name=$(ls -t dist | head -n 1)
-pip install -U --user --force-reinstall dist/$wheel_name
-python sanity_test.py
+WORKDIR /work/mxnet
+ENV LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib
diff --git a/tools/build/build_wheel.sh
b/ci/docker/Dockerfile.publish.ubuntu1604_gpu
old mode 100755
new mode 100644
similarity index 59%
copy from tools/build/build_wheel.sh
copy to ci/docker/Dockerfile.publish.ubuntu1604_gpu
index a796341..2a1f859
--- a/tools/build/build_wheel.sh
+++
[incubator-mxnet] branch master updated: Fixed java benchmark failing error by fixing the classpath (#13891)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git The following commit(s) were added to refs/heads/master by this push: new 19764ac Fixed java benchmark failing error by fixing the classpath (#13891) 19764ac is described below commit 19764acba3929e3a5dd4d4981a4319764a4fbf4a Author: Piyush Ghai AuthorDate: Tue Jan 15 16:11:35 2019 -0800 Fixed java benchmark failing error by fixing the classpath (#13891) --- scala-package/examples/scripts/benchmark/run_java_inference_bm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scala-package/examples/scripts/benchmark/run_java_inference_bm.sh b/scala-package/examples/scripts/benchmark/run_java_inference_bm.sh index f426dda..c62a743 100644 --- a/scala-package/examples/scripts/benchmark/run_java_inference_bm.sh +++ b/scala-package/examples/scripts/benchmark/run_java_inference_bm.sh @@ -20,7 +20,7 @@ set -e MXNET_ROOT=$(cd "$(dirname $0)/../../../.."; pwd) -CLASS_PATH=$MXNET_ROOT/scala-package/assembly/target/*:$MXNET_ROOT/scala-package/examples/target/* +CLASS_PATH=$MXNET_ROOT/scala-package/assembly/target/*:$MXNET_ROOT/scala-package/examples/target/*:$MXNET_ROOT/scala-package/examples/target/classes/lib/* java -Xmx8G -Dmxnet.traceLeakedObjects=true -cp $CLASS_PATH \ org.apache.mxnetexamples.javaapi.benchmark.JavaBenchmark $@
[incubator-mxnet] branch v1.4.x updated (84beaf1 -> a9bdeb5)
This is an automated email from the ASF dual-hosted git repository. lanking pushed a change to branch v1.4.x in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git. from 84beaf1 Fix incorrect delete in MXExecutorReshape exception handling (#13376) (#13824) add a9bdeb5 License fixed with copyrights on 1.4.x release branch (#13856) No new revisions were added by this update. Summary of changes: 3rdparty/googletest| 2 +- LICENSE| 552 - R-package/LICENSE | 2 +- contrib/clojure-package/LICENSE| 2 +- docs/_static/selectlang.js | 21 +- perl-package/AI-MXNet-Gluon-Contrib/META.yml | 17 + perl-package/AI-MXNet-Gluon-ModelZoo/META.yml | 17 + perl-package/AI-MXNet/META.yml | 17 + perl-package/AI-MXNetCAPI/META.yml | 17 + perl-package/AI-NNVMCAPI/META.yml | 17 + readthedocs.yml| 17 + snapcraft.yaml | 17 + .../nightly/apache_rat_license_check/rat-excludes | 14 +- tools/license_header.py| 25 +- 14 files changed, 470 insertions(+), 267 deletions(-)
[incubator-mxnet] branch master updated: fix the fetching GPU problem (#13889)
This is an automated email from the ASF dual-hosted git repository.
lanking pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new 634d527 fix the fetching GPU problem (#13889)
634d527 is described below
commit 634d527a955b59f62728467451508fa30152aa48
Author: Lanking
AuthorDate: Tue Jan 15 16:02:31 2019 -0500
fix the fetching GPU problem (#13889)
---
scala-package/deploy/pom.xml | 1 +
scala-package/pom.xml| 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/scala-package/deploy/pom.xml b/scala-package/deploy/pom.xml
index c51aa9a..542bf6c 100644
--- a/scala-package/deploy/pom.xml
+++ b/scala-package/deploy/pom.xml
@@ -19,6 +19,7 @@
+${project.parent.basedir}/..
mxnet-full_2.11-${platform}-${flavor}
1.5.0-SNAPSHOT
apache.snapshots.https
diff --git a/scala-package/pom.xml b/scala-package/pom.xml
index 6665e95..d39075f 100644
--- a/scala-package/pom.xml
+++ b/scala-package/pom.xml
@@ -131,7 +131,7 @@
bash
- -c 'mkdir -p ${project.build.directory}; if
[[ $(ldd ${MXNET_DIR}/lib/libmxnet.so | grep libcuda.so | wc -l) == "0" ]];
then echo flavor=cpu ${project.build.directory}/flavor.properties; else
echo flavor=gpu ${project.build.directory}/flavor.properties;
fi'
+ -c 'mkdir -p ${project.build.directory}; if
[[ $(ldd ${MXNET_DIR}/lib/libmxnet.so | grep libcuda | wc -l) == "0" ]]; then
echo flavor=cpu ${project.build.directory}/flavor.properties; else echo
flavor=gpu ${project.build.directory}/flavor.properties;
fi'
