[GitHub] wkcn commented on issue #8569: The conflict between MXNet and OpenCV

[GitBox]

wkcn commented on issue #8569: The conflict between MXNet and OpenCV
URL: 
https://github.com/apache/incubator-mxnet/issues/8569#issuecomment-342399903
 
 
   I think the reason is that `gperftools` or `jemalloc` replaces **the memory 
allocator** including `malloc`, however `python-opencv` uses the default 
allocator. 
   
   There are some shared pointer between MXNet and OpenCV, but it's **not 
available** to free the memories different allocators(gperftools, jemallo, 
glibc) allocated.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] starimpact opened a new issue #8571: advise:set "is_train" of operator be "false" when its parameter gradient is null

[GitBox]

starimpact opened a new issue #8571: advise:set "is_train" of operator be 
"false" when its parameter gradient is null
URL: https://github.com/apache/incubator-mxnet/issues/8571
 
 
   if we fix the parameters of an operator in the proceeding of training, (that 
means we do not learn the parameters), then its operator should be in an 
"is_train=false" mode. the "forward" should use the inference code in it.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #8569: The conflict between MXNet and OpenCV

[GitBox]

szha commented on issue #8569: The conflict between MXNet and OpenCV
URL: 
https://github.com/apache/incubator-mxnet/issues/8569#issuecomment-342396930
 
 
   I've seen the same issue when building with cuda9 + cudnn7 with gperftools 
option on.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] sbodenstein commented on issue #7931: MKL-DNN integration: request for reviews

[GitBox]

sbodenstein commented on issue #7931: MKL-DNN integration: request for reviews
URL: https://github.com/apache/incubator-mxnet/pull/7931#issuecomment-342396676
 
 
   @ykim362: thanks! I saw that. 
   
   Btw: do you have any estimate for when this PR will be ready?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] 315386775 opened a new issue #8570: How to reduce the memory when load the trained model?

[GitBox]

315386775 opened a new issue #8570: How to reduce the memory when load the 
trained model?
URL: https://github.com/apache/incubator-mxnet/issues/8570
 
 
   ## Description
   I used the VGG16 base network. The faster-rcnn occupy 2G memory in fact, but 
when loading model need around 6G memory.
   How to reduce the memory when load the trained model?
   
   ## What have you tried to solve it?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wkcn opened a new issue #8569: The conflict between MXNet and OpenCV

[GitBox]

wkcn opened a new issue #8569: The conflict between MXNet and OpenCV
URL: https://github.com/apache/incubator-mxnet/issues/8569
 
 
   Hi, there.
   
   I found the reason of the conflict between MXNet and OpenCV.
   
   ## Environment info
   Operation System: Arch Linux 4.13.6
   MXNet: 3f37577 (Date:   Tue Nov 7 02:13:07 2017 +0800)
   OpenCV: 3.3.1
   Python: 2.7.14/3.6.3
   GCC: 6.3.1 20170109
   Build config: `make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas`
   
   ## Steps to reproduce
   1. I built the MXNet core shared library with `make -j $(nproc) USE_OPENCV=1 
USE_BLAS=openblas`
   `make/config.mk` is default.
   The building was successful. 
   2. Then I was going to install the MXNet Python binding.
   ```bash
cd python
sudo python setup.py install
   ```
   
   It showed the error that:
   
   `*** Error in python': free(): invalid pointer:0x55ec46fe1520 *** `
   
   ## What I have tried to solve it
   I deleted all `"import cv2"` in `$(MXNET_PATH)/python/mxnet/{recordio.py, 
image/{detection.py, image.py}}`
   
   Then I made two tests in the folder `$(MXNET_PATH)/python/`.
   
   ```bash
   ?  python git:(master) ? python 
   Python 3.6.3 (default, Oct 24 2017, 14:48:20) 
   [GCC 7.2.0] on linux
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import mxnet
   >>> import cv2
   *** Error in `python': free(): invalid pointer: 0x564c7470d520 ***
   [1]116917 abort (core dumped)  python
   
   ?  python git:(master) ? python
   Python 3.6.3 (default, Oct 24 2017, 14:48:20) 
   [GCC 7.2.0] on linux
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import cv2
   >>> import mxnet
   [src/tcmalloc.cc:283] Attempt to free invalid pointer 0x5568689f8fc0 
   [1]116946 abort (core dumped)  python
   ```
   
   `src/tcmalloc.cc` is the code of **gperftools**.
   So I think there is a conflict between `gperftools` and `opencv2`.
   
   I set `USE_GPERFTOOLS = 0` and `USE_JEMALLOC = 0` in 
`$(MXNET_PATH)/make/config.mk`, and rebuild MXNet.
   The problem is solved.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #8553: Engine reserves cores from OMP. Set some defaults for dynamic and recursion

[GitBox]

piiswrong closed pull request #8553: Engine reserves cores from OMP.  Set some 
defaults for dynamic and recursion
URL: https://github.com/apache/incubator-mxnet/pull/8553
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 63bc8d740b..539515b3a2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -185,10 +185,15 @@ endif()
 
 # ---[ jemalloc
 if(USE_JEMALLOC)
+  if(USE_GPERFTOOLS)
+message(ERROR "Only one of USE_JEMALLOC and USE_GPERFTOOLS can be defined 
at once")
+  endif()
   find_package(JeMalloc)
   if(JEMALLOC_FOUND)
 message(STATUS "Using JEMalloc malloc")
 add_definitions(-DUSE_JEMALLOC)
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${ALT_MALLOC_FLAGS}")
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${ALT_MALLOC_FLAGS}")
 include_directories(${JEMALLOC_INCLUDE_DIRS})
 set(mxnet_LINKER_LIBS ${mxnet_LINKER_LIBS} ${JEMALLOC_LIBRARIES})
   endif()
diff --git a/src/engine/openmp.cc b/src/engine/openmp.cc
index a605f977b6..be7885ba75 100644
--- a/src/engine/openmp.cc
+++ b/src/engine/openmp.cc
@@ -29,44 +29,67 @@ namespace engine {
 #define ARCH_IS_INTEL_X86
 #endif
 
+static inline bool is_env_set(const char *var) {
+  return dmlc::GetEnv(var, INT_MIN) == INT_MIN;
+}
+
 OpenMP *OpenMP::Get() {
   static OpenMP openMP;
   return 
 }
 
 OpenMP::OpenMP()
-  : omp_num_threads_set_in_environment(dmlc::GetEnv("OMP_NUM_THREADS", 
INT_MIN) == INT_MIN) {
+  : omp_num_threads_set_in_environment(is_env_set("OMP_NUM_THREADS")) {
 #ifdef _OPENMP
-  if (!omp_num_threads_set_in_environment) {
-omp_set_nested(true);
-omp_set_dynamic(false);
-  }
   const int max = dmlc::GetEnv("MXNET_OMP_MAX_THREADS", INT_MIN);
   if (max != INT_MIN) {
 omp_thread_max_ = max;
   } else {
+if (!omp_num_threads_set_in_environment) {
+  omp_thread_max_ = omp_get_num_procs();
 #ifdef ARCH_IS_INTEL_X86
-omp_thread_max_ = omp_get_num_procs() >> 1;
+  omp_thread_max_ >>= 1;
 #endif
+  omp_set_num_threads(omp_thread_max_);
+} else {
+  omp_thread_max_ = omp_get_max_threads();
+  }
   }
+  omp_set_nested(dmlc::GetEnv("OMP_NESTED", false));
+  omp_set_dynamic(dmlc::GetEnv("OMP_DYNAMIC", false));
 #else
   enabled_ = false;
   omp_thread_max_ = 1;
 #endif
 }
 
-int OpenMP::GetRecommendedOMPThreadCount() const {
+void OpenMP::set_reserve_cores(int cores) {
+  CHECK_GE(cores, 0);
+  reserve_cores_ = cores;
+#ifdef _OPENMP
+  if (reserve_cores_ >= omp_thread_max_) {
+omp_set_num_threads(1);
+  } else {
+omp_set_num_threads(omp_thread_max_ - reserve_cores_);
+  }
+#endif
+}
+
+int OpenMP::GetRecommendedOMPThreadCount(bool exclude_reserved) const {
 #ifdef _OPENMP
   if (omp_num_threads_set_in_environment) {
 return omp_get_max_threads();
   }
   if (enabled_) {
-#ifdef ARCH_IS_INTEL_X86
-// x86 does hyperthreading, but do to cache issues, it's faster to only 
use # true CPUs
-const int thread_count = omp_get_max_threads() >> 1;
-#else
-const int thread_count = omp_get_max_threads();
-#endif
+int thread_count = omp_get_max_threads();
+if (exclude_reserved) {
+  if (reserve_cores_ >= thread_count) {
+thread_count = 1;
+  } else {
+thread_count -= reserve_cores_;
+  }
+}
+// Check that OMP doesn't suggest more than our 'omp_thread_max_' value
 if (!omp_thread_max_ || thread_count < omp_thread_max_) {
   return thread_count;
 }
@@ -78,6 +101,8 @@ int OpenMP::GetRecommendedOMPThreadCount() const {
 #endif
 }
 
+OpenMP *__init_omp__ = OpenMP::Get();
+
 }  // namespace engine
 }  // namespace mxnet
 
diff --git a/src/engine/openmp.h b/src/engine/openmp.h
index 8b995a6357..02e73c0955 100644
--- a/src/engine/openmp.h
+++ b/src/engine/openmp.h
@@ -36,7 +36,7 @@ class OpenMP {
* \brief Get the recommended number of OMP threads to use given the current 
context
* \return Recommended number of OMP threads to use in a parallel operation
*/
-  int GetRecommendedOMPThreadCount() const;
+  int GetRecommendedOMPThreadCount(bool exclude_reserved = true) const;
 
   /*!
* \brief Set whether clients of this class receive pro-OMP behavior guidance
@@ -57,8 +57,19 @@ class OpenMP {
   int thread_max() const { return omp_thread_max_; }
 
   /*!
+   * \brief Reserve cores to be excluded from OMP regions
+   * \param cores Number of cores to be excluded from OMP region usage
+   */
+  void set_reserve_cores(int cores);
+  /*!
+   * \brief Get number of cores to be excluded from OMP regions
+   * \return Number of cores to be excluded from OMP regions
+   */
+  int reserve_cores() const { return reserve_cores_; }
+
+  /*!
* \brief Get the OpenMP object's singleton pointer
-   * \return
+   * \return Singleton OpenMP object pointer
*/
   static 

[incubator-mxnet] branch master updated: Engine reserves cores from OMP. Set some defaults for dynamic and recursion (#8553)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 5c84c30  Engine reserves cores from OMP.  Set some defaults for 
dynamic and recursion (#8553)
5c84c30 is described below

commit 5c84c30cfffbee1fdb70d6544743b55775ecf1e6
Author: Chris Olivier 
AuthorDate: Mon Nov 6 21:43:38 2017 -0800

Engine reserves cores from OMP.  Set some defaults for dynamic and 
recursion (#8553)

* Engine reserves cores from OMP.  Set some defaults for dynamic and 
recursion unless environment variables are set.

* Pull some generic unit testing stuff from tuner branch
Also, something with gperftools got missed in a CMakeLists.txt merge at 
some point.

* lint
---
 CMakeLists.txt|  5 +++
 src/engine/openmp.cc  | 51 ---
 src/engine/openmp.h   | 19 --
 src/engine/threaded_engine_perdevice.cc   | 29 ---
 src/initialize.cc |  5 +++
 src/operator/dropout-inl.h| 22 ++--
 src/operator/mxnet_op.h   |  1 +
 src/operator/operator_common.h|  6 +++-
 src/operator/slice_channel-inl.h  |  1 -
 tests/cpp/include/test_util.h | 22 ++--
 tests/cpp/misc/memory_test.cc | 29 +--
 tests/cpp/unittest.mk |  2 +-
 tests/python/unittest/test_sparse_operator.py | 14 +++-
 13 files changed, 150 insertions(+), 56 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 63bc8d7..539515b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -185,10 +185,15 @@ endif()
 
 # ---[ jemalloc
 if(USE_JEMALLOC)
+  if(USE_GPERFTOOLS)
+message(ERROR "Only one of USE_JEMALLOC and USE_GPERFTOOLS can be defined 
at once")
+  endif()
   find_package(JeMalloc)
   if(JEMALLOC_FOUND)
 message(STATUS "Using JEMalloc malloc")
 add_definitions(-DUSE_JEMALLOC)
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${ALT_MALLOC_FLAGS}")
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${ALT_MALLOC_FLAGS}")
 include_directories(${JEMALLOC_INCLUDE_DIRS})
 set(mxnet_LINKER_LIBS ${mxnet_LINKER_LIBS} ${JEMALLOC_LIBRARIES})
   endif()
diff --git a/src/engine/openmp.cc b/src/engine/openmp.cc
index a605f97..be7885b 100644
--- a/src/engine/openmp.cc
+++ b/src/engine/openmp.cc
@@ -29,44 +29,67 @@ namespace engine {
 #define ARCH_IS_INTEL_X86
 #endif
 
+static inline bool is_env_set(const char *var) {
+  return dmlc::GetEnv(var, INT_MIN) == INT_MIN;
+}
+
 OpenMP *OpenMP::Get() {
   static OpenMP openMP;
   return 
 }
 
 OpenMP::OpenMP()
-  : omp_num_threads_set_in_environment(dmlc::GetEnv("OMP_NUM_THREADS", 
INT_MIN) == INT_MIN) {
+  : omp_num_threads_set_in_environment(is_env_set("OMP_NUM_THREADS")) {
 #ifdef _OPENMP
-  if (!omp_num_threads_set_in_environment) {
-omp_set_nested(true);
-omp_set_dynamic(false);
-  }
   const int max = dmlc::GetEnv("MXNET_OMP_MAX_THREADS", INT_MIN);
   if (max != INT_MIN) {
 omp_thread_max_ = max;
   } else {
+if (!omp_num_threads_set_in_environment) {
+  omp_thread_max_ = omp_get_num_procs();
 #ifdef ARCH_IS_INTEL_X86
-omp_thread_max_ = omp_get_num_procs() >> 1;
+  omp_thread_max_ >>= 1;
 #endif
+  omp_set_num_threads(omp_thread_max_);
+} else {
+  omp_thread_max_ = omp_get_max_threads();
+  }
   }
+  omp_set_nested(dmlc::GetEnv("OMP_NESTED", false));
+  omp_set_dynamic(dmlc::GetEnv("OMP_DYNAMIC", false));
 #else
   enabled_ = false;
   omp_thread_max_ = 1;
 #endif
 }
 
-int OpenMP::GetRecommendedOMPThreadCount() const {
+void OpenMP::set_reserve_cores(int cores) {
+  CHECK_GE(cores, 0);
+  reserve_cores_ = cores;
+#ifdef _OPENMP
+  if (reserve_cores_ >= omp_thread_max_) {
+omp_set_num_threads(1);
+  } else {
+omp_set_num_threads(omp_thread_max_ - reserve_cores_);
+  }
+#endif
+}
+
+int OpenMP::GetRecommendedOMPThreadCount(bool exclude_reserved) const {
 #ifdef _OPENMP
   if (omp_num_threads_set_in_environment) {
 return omp_get_max_threads();
   }
   if (enabled_) {
-#ifdef ARCH_IS_INTEL_X86
-// x86 does hyperthreading, but do to cache issues, it's faster to only 
use # true CPUs
-const int thread_count = omp_get_max_threads() >> 1;
-#else
-const int thread_count = omp_get_max_threads();
-#endif
+int thread_count = omp_get_max_threads();
+if (exclude_reserved) {
+  if (reserve_cores_ >= thread_count) {
+thread_count = 1;
+  } else {
+thread_count -= reserve_cores_;
+  }
+}
+// Check that OMP doesn't suggest more than our 'omp_thread_max_' value
 if (!omp_thread_max_ || thread_count < omp_thread_max_) {
   return thread_count;
 }
@@ -78,6 +101,8 @@ int 

[GitHub] jeremiedb commented on issue #8545: Incorrect results from R 3.4.2 in MNIST

[GitBox]

jeremiedb commented on issue #8545: Incorrect results from R 3.4.2 in MNIST
URL: 
https://github.com/apache/incubator-mxnet/issues/8545#issuecomment-342380889
 
 
   There's effectively a bug introduced in 0.12 that caused the training issue: 
the random initializers had to be changed which, which should be solved when 
the pull request is merged. In the meantime, 0.11 is safer! 
   
   I haven't encountered installation issues with DiagrammeR for a while, if 
you have a reproducible example please provide it so a fix can be made. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] fallingdust commented on issue #8546: What's the equivalent of np.where()?

[GitBox]

fallingdust commented on issue #8546: What's the equivalent of np.where()?
URL: 
https://github.com/apache/incubator-mxnet/issues/8546#issuecomment-342380154
 
 
   @zhreshold 
   I can fully understand that np.where is not supported by mx.symbol, but why 
not mx.ndarray? Since we can _slice_ the ndarray by the value of data, why not 
_take_?
   And would you please give an example on how to get the cls_scores according 
to labels != -1?
   ```
   keep = np.where(labels != -1)[0]
   cls_scores = cls_scores[keep]
   ```
   Thx!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] fallingdust commented on issue #8546: What's the equivalent of np.where()?

[GitBox]

fallingdust commented on issue #8546: What's the equivalent of np.where()?
URL: 
https://github.com/apache/incubator-mxnet/issues/8546#issuecomment-342380154
 
 
   @zhreshold 
   I can fully understand that np.where is not supported by mx.symbol, but why 
not mx.ndarray? We can slice the ndarray by the value of data.
   And would you please give an example on how to get the cls_scores according 
to labels != -1?
   `
   keep = np.where(labels != -1)[0]
   cls_scores = cls_scores[keep]
   `
   Thx!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] fallingdust commented on issue #8546: What's the equivalent of np.where()?

[GitBox]

fallingdust commented on issue #8546: What's the equivalent of np.where()?
URL: 
https://github.com/apache/incubator-mxnet/issues/8546#issuecomment-342380154
 
 
   @zhreshold 
   I can fully understand that np.where is not supported by mx.symbol, but why 
not mx.ndarray? We can slice the ndarray by the value of data.
   And would you please give an example on how to get the cls_scores according 
to labels != -1?
   ```
   keep = np.where(labels != -1)[0]
   cls_scores = cls_scores[keep]
   ```
   Thx!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] fallingdust commented on issue #8546: What's the equivalent of np.where()?

[GitBox]

fallingdust commented on issue #8546: What's the equivalent of np.where()?
URL: 
https://github.com/apache/incubator-mxnet/issues/8546#issuecomment-342380154
 
 
   @zhreshold 
   I can fully understand that np.where is not supported by mx.symbol, but why 
not mx.ndarray? We can slice the ndarray by the value of data.
   And would you please give an example on how to get the cls_scores according 
to labels != -1?
   `
   keep = np.where(labels != -1)[0]
   
   cls_scores = cls_scores[keep]
   `
   Thx!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] fallingdust commented on issue #8546: What's the equivalent of np.where()?

[GitBox]

fallingdust commented on issue #8546: What's the equivalent of np.where()?
URL: 
https://github.com/apache/incubator-mxnet/issues/8546#issuecomment-342380154
 
 
   @zhreshold 
   I can fully understand that np.where is not supported by mx.symbol, but why 
not mx.ndarray? We can slice the ndarray by the value of data.
   And would you please give an example on how to get the cls_scores according 
to labels != -1?
   `
   keep = np.where(labels != -1)[0]
   cls_scores = cls_scores[keep]
   `
   Thx!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #8553: Engine reserves cores from OMP. Set some defaults for dynamic and recursion

[GitBox]

piiswrong commented on a change in pull request #8553: Engine reserves cores 
from OMP.  Set some defaults for dynamic and recursion
URL: https://github.com/apache/incubator-mxnet/pull/8553#discussion_r149274414
 
 

 ##
 File path: src/engine/threaded_engine_perdevice.cc
 ##
 @@ -113,8 +113,8 @@ class ThreadedEnginePerDevice : public ThreadedEngine {
 if (is_copy) {
   auto ptr =
   gpu_copy_workers_.Get(ctx.dev_id, [this, ctx, is_copy, nthread]() {
-// Signify to kernel that GPU is being used,  no Kernel Launch OMP 
(temporary behavior)
-OpenMP::Get()->set_enabled(false);
+// Signify to kernel that GPU is being used, so reserve cores as 
necessary
+OpenMP::Get()->set_reserve_cores(GetReserveCoreCount(true));
 
 Review comment:
   ok


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #8553: Engine reserves cores from OMP. Set some defaults for dynamic and recursion

[GitBox]

piiswrong commented on a change in pull request #8553: Engine reserves cores 
from OMP.  Set some defaults for dynamic and recursion
URL: https://github.com/apache/incubator-mxnet/pull/8553#discussion_r149274352
 
 

 ##
 File path: src/engine/threaded_engine_perdevice.cc
 ##
 @@ -234,6 +234,27 @@ class ThreadedEnginePerDevice : public ThreadedEngine {
 }
   }
 
+  /*!
+   * \brief Get number of cores this engine should reserve for its own use
+   * \param using_gpu Whether there is GPU usage
+   * \return number of cores that this engine wishes to be reserved
+   * \note Testing found no degradation of performance using these values
+   *   running cifar10 with resnet50 on various GPU systems,
+   *   including AWS p2.16xlarge, which has 16 GPU's
+   */
+  int GetReserveCoreCount(const bool using_gpu) const {
+int reserve = 0;
+if (using_gpu) {
+  // Save at least one for GPU tasks
+  ++reserve;
+  // If we have 8 or more real cores, reserve another core for GPU tasks
 
 Review comment:
   ok


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong closed pull request #8561: Fix unused var warning in release build (only used in assert on debug?

[GitBox]

piiswrong closed pull request #8561: Fix unused var warning in release build 
(only used in assert on debug?
URL: https://github.com/apache/incubator-mxnet/pull/8561
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/src/operator/roi_pooling.cc b/src/operator/roi_pooling.cc
index 9af00bb450..a7bd60872a 100644
--- a/src/operator/roi_pooling.cc
+++ b/src/operator/roi_pooling.cc
@@ -52,7 +52,6 @@ inline void ROIPoolForward(const Tensor ,
   const int pooled_width_ = out.size(3);
 
   const int num_rois = bbox.size(0);
-  const int batch_size = data.size(0);
   const int data_size = data.size(1) * data.size(2) * data.size(3);
   // For each ROI R = [batch_index x1 y1 x2 y2]: max pool over R
   for (int n = 0; n < num_rois; ++n) {
@@ -62,7 +61,7 @@ inline void ROIPoolForward(const Tensor ,
 int roi_end_w = round(bottom_rois[3] * spatial_scale_);
 int roi_end_h = round(bottom_rois[4] * spatial_scale_);
 assert(roi_batch_ind >= 0);
-assert(roi_batch_ind < batch_size);
+assert(roi_batch_ind < data.size(0) /* batch size */);
 
 // force malformed ROIs to be 1 * 1
 int roi_height = max(roi_end_h - roi_start_h + 1, 1);


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: Fix unused var warning in release build (only used in assert on debug builds) (#8561)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new cf2545f  Fix unused var warning in release build (only used in assert 
on debug builds) (#8561)
cf2545f is described below

commit cf2545f0d64a90a137f92f1d02b47972ea28f5e1
Author: Chris Olivier 
AuthorDate: Mon Nov 6 21:21:19 2017 -0800

Fix unused var warning in release build (only used in assert on debug 
builds) (#8561)
---
 src/operator/roi_pooling.cc | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/src/operator/roi_pooling.cc b/src/operator/roi_pooling.cc
index 9af00bb..a7bd608 100644
--- a/src/operator/roi_pooling.cc
+++ b/src/operator/roi_pooling.cc
@@ -52,7 +52,6 @@ inline void ROIPoolForward(const Tensor ,
   const int pooled_width_ = out.size(3);
 
   const int num_rois = bbox.size(0);
-  const int batch_size = data.size(0);
   const int data_size = data.size(1) * data.size(2) * data.size(3);
   // For each ROI R = [batch_index x1 y1 x2 y2]: max pool over R
   for (int n = 0; n < num_rois; ++n) {
@@ -62,7 +61,7 @@ inline void ROIPoolForward(const Tensor ,
 int roi_end_w = round(bottom_rois[3] * spatial_scale_);
 int roi_end_h = round(bottom_rois[4] * spatial_scale_);
 assert(roi_batch_ind >= 0);
-assert(roi_batch_ind < batch_size);
+assert(roi_batch_ind < data.size(0) /* batch size */);
 
 // force malformed ROIs to be 1 * 1
 int roi_height = max(roi_end_h - roi_start_h + 1, 1);

-- 
To stop receiving notification emails like this one, please contact
['"comm...@mxnet.apache.org" '].

[GitHub] piiswrong commented on a change in pull request #8566: optimize broadcast

[GitBox]

piiswrong commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149274245
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   It should be.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #8566: optimize broadcast

[GitBox]

piiswrong commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149274186
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_broadcast_op.h
 ##
 @@ -133,13 +133,35 @@ inline int BinaryBroadcastShapeCompact(const TShape& 
lshape, const TShape& rshap
   return j;
 }
 
+namespace mxnet_op {
+
+template
+struct binary_broadcast_kernel {
+  MSHADOW_XINLINE static void Map(int base, int length, OpReqType req,
+  const Shape& lstride, const 
Shape& rstride,
+  const Shape& oshape, DType* lhs, 
DType* rhs,
+  DType* out, int lsize, int rsize) {
+Shape coord = unravel(base, oshape);
+index_t lidx = dot(coord, lstride);
+index_t ridx = dot(coord, rstride);
+KERNEL_ASSIGN(out[base], req, OP::Map(lhs[lidx], rhs[ridx]));
+// starts from 1 to avoid extra inc at end of loop
+for (int i = 1; i < length; ++i) {
+  inc(, oshape, , lstride, , rstride);
+  KERNEL_ASSIGN(out[base+i], req, OP::Map(lhs[lidx], rhs[ridx]));
 
 Review comment:
   Yes, we can switch before launch. But that doubles compile time (and 
possibly binary size). So we should only do it if there is visible performance 
difference


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lilhope commented on issue #8568: How to adding an axes in ndarray?

[GitBox]

lilhope commented on issue #8568: How to adding an axes in ndarray?
URL: 
https://github.com/apache/incubator-mxnet/issues/8568#issuecomment-342369061
 
 
   Thanks @solin319 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lilhope closed issue #8568: How to adding an axes in ndarray?

[GitBox]

lilhope closed issue #8568: How to adding an axes in ndarray?
URL: https://github.com/apache/incubator-mxnet/issues/8568
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] solin319 commented on issue #8568: How to adding an axes in ndarray?

[GitBox]

solin319 commented on issue #8568: How to adding an axes in ndarray?
URL: 
https://github.com/apache/incubator-mxnet/issues/8568#issuecomment-342365694
 
 
   mx.nd.expand_dims
   
http://mxnet.incubator.apache.org/api/python/ndarray/ndarray.html#mxnet.ndarray.expand_dims


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] lilhope opened a new issue #8568: How to adding an axes in ndarray?

[GitBox]

lilhope opened a new issue #8568: How to adding an axes in ndarray?
URL: https://github.com/apache/incubator-mxnet/issues/8568
 
 
   I want to add an axes in a ndarray. for example `(3,4,5,5)` to 
`(1,3,4,5,5)`. **I don't know the ndarray shape in advance** I refer to the 
`reshape api`, but can't understand the `-4` means.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 closed pull request #8567: Prep for 0.12.1: Version Updates

[GitBox]

cjolivier01 closed pull request #8567: Prep for 0.12.1: Version Updates
URL: https://github.com/apache/incubator-mxnet/pull/8567
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/NEWS.md b/NEWS.md
index 666b5d88e6..0a7c018f4e 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -1,5 +1,17 @@
 MXNet Change Log
 
+## 0.12.1
+### Bug-fixes
+  - Added GPU support for the `syevd` operator which ensures that there is GPU 
support for all linalg-operators.
+  - Bugfix for `syevd` on CPU such that it works for `float32`.
+  - Fixed API call when `OMP_NUM_THREADS` environment variable is set. 
+  - Fixed `MakeNonlossGradNode` bug.
+  - Fixed bug related to passing `dtype` to `array()`. 
+  - Fixed some minor bugs for sparse distributed training.
+  - Fixed a bug on `Slice` accessing uninitialized memory in `param.begin` in 
the file `matrix_op-inl.h`. 
+  - Fixed `gluon.data.RecordFileDataset`.
+  - Fixed a bug that caused `autograd` to crash on some networks.
+  
 ## 0.12.0
 ### Performance
   - Added full support for NVIDIA Volta GPU Architecture and CUDA 9. Training 
CNNs is up to 3.5x faster than Pascal when using float16 precision.
diff --git a/R-package/DESCRIPTION b/R-package/DESCRIPTION
index 4c768245b3..3d57ea876f 100644
--- a/R-package/DESCRIPTION
+++ b/R-package/DESCRIPTION
@@ -1,7 +1,7 @@
 Package: mxnet
 Type: Package
 Title: MXNet: A Flexible and Efficient Machine Learning Library for 
Heterogeneous Distributed Systems
-Version: 0.12.0
+Version: 0.12.1
 Date: 2017-06-27
 Author: Tianqi Chen, Qiang Kou, Tong He
 Maintainer: Qiang Kou 
diff --git a/README.md b/README.md
index fc252a7a72..0326412541 100644
--- a/README.md
+++ b/README.md
@@ -22,6 +22,7 @@ deep learning systems, and interesting insights of DL systems 
for hackers.
 
 What's New
 --
+* [Version 0.12.1 
Release](https://github.com/apache/incubator-mxnet/releases/tag/0.12.1) - MXNet 
0.12.1 Patch Release.
 * [Version 0.12.0 
Release](https://github.com/apache/incubator-mxnet/releases/tag/0.12.0) - MXNet 
0.12.0 Release.
 * [Version 0.11.0 
Release](https://github.com/apache/incubator-mxnet/releases/tag/0.11.0) - MXNet 
0.11.0 Release.
 * [Apache Incubator](http://incubator.apache.org/projects/mxnet.html) - We are 
now an Apache Incubator project.
diff --git a/docs/build_version_doc/build_all_version.sh 
b/docs/build_version_doc/build_all_version.sh
index 2d33bd72c4..3162acd85a 100755
--- a/docs/build_version_doc/build_all_version.sh
+++ b/docs/build_version_doc/build_all_version.sh
@@ -21,7 +21,7 @@
 # Built files are stored in $built
 # Version numbers are stored in $tag_list.
 # Version numbers are ordered from latest to old and final one is master.
-tag_list="0.12.0 0.11.0 master"
+tag_list="0.12.1 0.12.0 0.11.0 master"
 
 mxnet_url="https://github.com/apache/incubator-mxnet.git;
 mxnet_folder="apache_mxnet"
diff --git a/include/mxnet/base.h b/include/mxnet/base.h
index 1c02ff185b..cceee70ffd 100644
--- a/include/mxnet/base.h
+++ b/include/mxnet/base.h
@@ -112,7 +112,7 @@
 /*! \brief minor version */
 #define MXNET_MINOR 12
 /*! \brief patch version */
-#define MXNET_PATCH 0
+#define MXNET_PATCH 1
 /*! \brief mxnet version */
 #define MXNET_VERSION (MXNET_MAJOR*1 + MXNET_MINOR*100 + MXNET_PATCH)
 /*! \brief helper for making version number */
diff --git a/python/mxnet/libinfo.py b/python/mxnet/libinfo.py
index d8d49a0ddb..d4d100e12d 100644
--- a/python/mxnet/libinfo.py
+++ b/python/mxnet/libinfo.py
@@ -61,4 +61,4 @@ def find_lib_path():
 
 
 # current version
-__version__ = "0.12.0"
+__version__ = "0.12.1"
diff --git a/scala-package/assembly/linux-x86_64-cpu/pom.xml 
b/scala-package/assembly/linux-x86_64-cpu/pom.xml
index 5e3e326368..f15a7e315d 100644
--- a/scala-package/assembly/linux-x86_64-cpu/pom.xml
+++ b/scala-package/assembly/linux-x86_64-cpu/pom.xml
@@ -6,7 +6,7 @@
   
 ml.dmlc.mxnet
 mxnet-full-parent_2.11
-0.12.0-SNAPSHOT
+0.12.1-SNAPSHOT
 ../pom.xml
   
 
@@ -18,12 +18,12 @@
 
   ml.dmlc.mxnet
   mxnet-core_${scala.binary.version}
-  0.12.0-SNAPSHOT
+  0.12.1-SNAPSHOT
 
 
   ml.dmlc.mxnet
   libmxnet-scala-linux-x86_64-cpu
-  0.12.0-SNAPSHOT
+  0.12.1-SNAPSHOT
   so
 
   
diff --git a/scala-package/assembly/linux-x86_64-gpu/pom.xml 
b/scala-package/assembly/linux-x86_64-gpu/pom.xml
index 6f3fc13974..81e4d1ec59 100644
--- a/scala-package/assembly/linux-x86_64-gpu/pom.xml
+++ b/scala-package/assembly/linux-x86_64-gpu/pom.xml
@@ -6,7 +6,7 @@
   
 ml.dmlc.mxnet
 mxnet-full-parent_2.11
-0.12.0-SNAPSHOT
+0.12.1-SNAPSHOT
 ../pom.xml
   
 
@@ -18,12 +18,12 @@
 
   ml.dmlc.mxnet
   mxnet-core_${scala.binary.version}
-  0.12.0-SNAPSHOT
+  0.12.1-SNAPSHOT
 
   

[incubator-mxnet] branch v0.12.0 updated: Prep for 0.12.1: Version Updates (#8567)

[cjolivier01]

This is an automated email from the ASF dual-hosted git repository.

cjolivier01 pushed a commit to branch v0.12.0
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/v0.12.0 by this push:
 new e0c7906  Prep for 0.12.1: Version Updates (#8567)
e0c7906 is described below

commit e0c7906693f0c79b0ce34a4d777c26a6bf1903c1
Author: mbaijal <30911248+mbai...@users.noreply.github.com>
AuthorDate: Mon Nov 6 19:04:04 2017 -0800

Prep for 0.12.1: Version Updates (#8567)

* Final Changes for 0.12.1

* Prep 0.12.1: Changes

* Initial Changes to NEWS.md
---
 NEWS.md | 12 
 R-package/DESCRIPTION   |  2 +-
 README.md   |  1 +
 docs/build_version_doc/build_all_version.sh |  2 +-
 include/mxnet/base.h|  2 +-
 python/mxnet/libinfo.py |  2 +-
 scala-package/assembly/linux-x86_64-cpu/pom.xml |  6 +++---
 scala-package/assembly/linux-x86_64-gpu/pom.xml |  6 +++---
 scala-package/assembly/osx-x86_64-cpu/pom.xml   |  6 +++---
 scala-package/assembly/pom.xml  |  2 +-
 scala-package/core/pom.xml  |  6 +++---
 scala-package/examples/pom.xml  |  4 ++--
 scala-package/init-native/linux-x86_64/pom.xml  |  4 ++--
 scala-package/init-native/osx-x86_64/pom.xml|  4 ++--
 scala-package/init-native/pom.xml   |  2 +-
 scala-package/init/pom.xml  |  2 +-
 scala-package/macros/pom.xml|  6 +++---
 scala-package/native/linux-x86_64-cpu/pom.xml   |  4 ++--
 scala-package/native/linux-x86_64-gpu/pom.xml   |  4 ++--
 scala-package/native/osx-x86_64-cpu/pom.xml |  4 ++--
 scala-package/native/pom.xml|  2 +-
 scala-package/pom.xml   |  2 +-
 scala-package/spark/pom.xml |  4 ++--
 setup-utils/install-mxnet-osx-python.sh |  2 +-
 snapcraft.yaml  |  2 +-
 25 files changed, 53 insertions(+), 40 deletions(-)

diff --git a/NEWS.md b/NEWS.md
index 666b5d8..0a7c018 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -1,5 +1,17 @@
 MXNet Change Log
 
+## 0.12.1
+### Bug-fixes
+  - Added GPU support for the `syevd` operator which ensures that there is GPU 
support for all linalg-operators.
+  - Bugfix for `syevd` on CPU such that it works for `float32`.
+  - Fixed API call when `OMP_NUM_THREADS` environment variable is set. 
+  - Fixed `MakeNonlossGradNode` bug.
+  - Fixed bug related to passing `dtype` to `array()`. 
+  - Fixed some minor bugs for sparse distributed training.
+  - Fixed a bug on `Slice` accessing uninitialized memory in `param.begin` in 
the file `matrix_op-inl.h`. 
+  - Fixed `gluon.data.RecordFileDataset`.
+  - Fixed a bug that caused `autograd` to crash on some networks.
+  
 ## 0.12.0
 ### Performance
   - Added full support for NVIDIA Volta GPU Architecture and CUDA 9. Training 
CNNs is up to 3.5x faster than Pascal when using float16 precision.
diff --git a/R-package/DESCRIPTION b/R-package/DESCRIPTION
index 4c76824..3d57ea8 100644
--- a/R-package/DESCRIPTION
+++ b/R-package/DESCRIPTION
@@ -1,7 +1,7 @@
 Package: mxnet
 Type: Package
 Title: MXNet: A Flexible and Efficient Machine Learning Library for 
Heterogeneous Distributed Systems
-Version: 0.12.0
+Version: 0.12.1
 Date: 2017-06-27
 Author: Tianqi Chen, Qiang Kou, Tong He
 Maintainer: Qiang Kou 
diff --git a/README.md b/README.md
index fc252a7..0326412 100644
--- a/README.md
+++ b/README.md
@@ -22,6 +22,7 @@ deep learning systems, and interesting insights of DL systems 
for hackers.
 
 What's New
 --
+* [Version 0.12.1 
Release](https://github.com/apache/incubator-mxnet/releases/tag/0.12.1) - MXNet 
0.12.1 Patch Release.
 * [Version 0.12.0 
Release](https://github.com/apache/incubator-mxnet/releases/tag/0.12.0) - MXNet 
0.12.0 Release.
 * [Version 0.11.0 
Release](https://github.com/apache/incubator-mxnet/releases/tag/0.11.0) - MXNet 
0.11.0 Release.
 * [Apache Incubator](http://incubator.apache.org/projects/mxnet.html) - We are 
now an Apache Incubator project.
diff --git a/docs/build_version_doc/build_all_version.sh 
b/docs/build_version_doc/build_all_version.sh
index 2d33bd7..3162acd 100755
--- a/docs/build_version_doc/build_all_version.sh
+++ b/docs/build_version_doc/build_all_version.sh
@@ -21,7 +21,7 @@
 # Built files are stored in $built
 # Version numbers are stored in $tag_list.
 # Version numbers are ordered from latest to old and final one is master.
-tag_list="0.12.0 0.11.0 master"
+tag_list="0.12.1 0.12.0 0.11.0 master"
 
 mxnet_url="https://github.com/apache/incubator-mxnet.git;
 mxnet_folder="apache_mxnet"
diff --git a/include/mxnet/base.h b/include/mxnet/base.h
index 1c02ff1..cceee70 100644
--- a/include/mxnet/base.h
+++ b/include/mxnet/base.h
@@ -112,7 +112,7 @@
 

[GitHub] cjolivier01 commented on issue #8567: Prep for 0.12.1: Version Updates

[GitBox]

cjolivier01 commented on issue #8567: Prep for 0.12.1: Version Updates
URL: https://github.com/apache/incubator-mxnet/pull/8567#issuecomment-342361112
 
 
   Not for this PR, but we really should have a script that updates all version 
numbers in one shot.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mbaijal commented on a change in pull request #8567: Prep for 0.12.1: Version Updates

[GitBox]

mbaijal commented on a change in pull request #8567: Prep for 0.12.1: Version 
Updates
URL: https://github.com/apache/incubator-mxnet/pull/8567#discussion_r149259307
 
 

 ##
 File path: NEWS.md
 ##
 @@ -1,5 +1,17 @@
 MXNet Change Log
 
+## 0.12.1
+### Bug-fixes
+  - Added GPU support for the `syevd` operator which ensures that there is GPU 
support for all linalg-operators.
+  - Bugfix for `syevd` on CPU such that it works for `float32`.
+  - Fixed API call when `OMP_NUM_THREADS` environment variable is set. 
+  - Fixed `MakeNonlossGradNode` bug.
+  - Fixed bug related to passing `dtype` to `array()`. 
+  - Fixed some minor bugs for sparse distributed training.
+  - Fixed a bug on `Slice` accessing uninitialized memory in `param.begin` in 
the file `matrix_op-inl.h`. 
+  - Fixed `gluon.data.RecordFileDataset`.
+  - Fixed a bug that caused `autograd` to crash on some networks.
+  
 
 Review comment:
   @cjolivier01 @eric-haibin-lin 
   Please Review!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149253284
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -345,6 +394,13 @@ __global__ void mxnet_generic_kernel(int N, Args... args) 
{
   }
 }
 
+template
+__global__ void mxnet_generic_kernel_ex(int N, Args... args) {
+  for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < N; i += blockDim.x * 
gridDim.x) {
 
 Review comment:
   Can this calculation "blockDim.x * gridDim.x" be moved outside of the loop 
so it doesn't execute so often (assuming the compiler doesn't somehow know 
blockDim.x and gridDim.x won't change, and optimize it out)? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] kkk669 commented on issue #8525: how to make the IOS library?

[GitBox]

kkk669 commented on issue #8525: how to make the IOS library?
URL: 
https://github.com/apache/incubator-mxnet/issues/8525#issuecomment-342346145
 
 
   Why don't you use Core ML instead of MXNet iOS library? Core ML runs on GPU, 
but MXNet probably doesn't. Some models can be converted into Core ML format 
using [the official 
converter](https://github.com/apache/incubator-mxnet/tree/master/tools/coreml).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149252660
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_broadcast_op.h
 ##
 @@ -133,13 +133,35 @@ inline int BinaryBroadcastShapeCompact(const TShape& 
lshape, const TShape& rshap
   return j;
 }
 
+namespace mxnet_op {
+
+template
+struct binary_broadcast_kernel {
+  MSHADOW_XINLINE static void Map(int base, int length, OpReqType req,
+  const Shape& lstride, const 
Shape& rstride,
+  const Shape& oshape, DType* lhs, 
DType* rhs,
+  DType* out, int lsize, int rsize) {
+Shape coord = unravel(base, oshape);
+index_t lidx = dot(coord, lstride);
+index_t ridx = dot(coord, rstride);
+KERNEL_ASSIGN(out[base], req, OP::Map(lhs[lidx], rhs[ridx]));
+// starts from 1 to avoid extra inc at end of loop
+for (int i = 1; i < length; ++i) {
+  inc(, oshape, , lstride, , rstride);
+  KERNEL_ASSIGN(out[base+i], req, OP::Map(lhs[lidx], rhs[ridx]));
 
 Review comment:
   Is there any way to get req into the template parameter to avoid executing a 
switch statement for every piece of data?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149251885
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   Sure. 
   
   Btw, if the actual number of OMP cores that run the for loop differ from the 
omp_cores value, will the calculation still be correct (albeit a little less 
optimal)?  It looks like it'll be ok, but just to be sure.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149251885
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   Sure. 
   
   Btw, if the actual number of OMP cores that run the for loop differ from the 
omp_cores value, will the calculation still be correct (albeit a little less 
optimal)?
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong commented on a change in pull request #8566: optimize broadcast

[GitBox]

piiswrong commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149249010
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   launch 2D maybe?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mbaijal opened a new pull request #8567: Prep for 0.12.1: Version Updates

[GitBox]

mbaijal opened a new pull request #8567: Prep for 0.12.1: Version Updates
URL: https://github.com/apache/incubator-mxnet/pull/8567
 
 
   ## Description ##
   MERGE INTO RELEASE BRANCH v0.12.0 ONLY. 
   1. Version Updates
   2. Added 0.12.1 to the Whats New in README.md. This tag does not exist yet
   3. NEWS.md is not updated yet.
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage
   - [ ] For user-facing API changes, API doc string has been updated.
   - [ ] To my best knowledge, examples are either not affected by this change, 
or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Intersting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] reminisce commented on issue #8566: optimize broadcast

[GitBox]

reminisce commented on issue #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#issuecomment-342334740
 
 
   Sorry, just realized that I was looking at a different PR but thinking about 
the CR of another one.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #7366: About distribute trainning, How can I set different gpu device id on different worker?

[GitBox]

szha closed issue #7366: About distribute trainning, How can I set different 
gpu device id on different worker?
URL: https://github.com/apache/incubator-mxnet/issues/7366
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #7370: Error when trying to build docker image with GPU and S3 support

[GitBox]

szha closed issue #7370: Error when trying to build docker image with GPU and 
S3 support
URL: https://github.com/apache/incubator-mxnet/issues/7370
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #7370: Error when trying to build docker image with GPU and S3 support

[GitBox]

szha commented on issue #7370: Error when trying to build docker image with GPU 
and S3 support
URL: 
https://github.com/apache/incubator-mxnet/issues/7370#issuecomment-342333967
 
 
   This issue is closed due to lack of activity in the last 90 days. Feel free 
to ping me to reopen if this is still an active issue. Thanks!
   Also, do please check out our [forum](https://discuss.mxnet.io/) (and 
[Chinese version](https://discuss.gluon.ai/)) for general "how-to" questions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149242330
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   I have same name function in my tuner branch :(


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149242047
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   (look familiar)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149239749
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -199,6 +199,37 @@ MSHADOW_XINLINE Shape calc_stride(const 
Shape& shape) {
   return stride;
 }
 
+/* Increment coordinates and modify index */
+template
+MSHADOW_XINLINE void inc(Shape* coord, const Shape& shape,
+ index_t* idx, const Shape& stride) {
+  ++(*coord)[ndim-1];
+  *idx += stride[ndim-1];
+  #pragma unroll
+  for (int i = ndim - 1; i > 0 && (*coord)[i] >= shape[i]; --i) {
+(*coord)[i] -= shape[i];
+++(*coord)[i-1];
+*idx = *idx + stride[i-1] - shape[i] * stride[i];
 
 Review comment:
   curious: why not += ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149239749
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -199,6 +199,37 @@ MSHADOW_XINLINE Shape calc_stride(const 
Shape& shape) {
   return stride;
 }
 
+/* Increment coordinates and modify index */
+template
+MSHADOW_XINLINE void inc(Shape* coord, const Shape& shape,
+ index_t* idx, const Shape& stride) {
+  ++(*coord)[ndim-1];
+  *idx += stride[ndim-1];
+  #pragma unroll
+  for (int i = ndim - 1; i > 0 && (*coord)[i] >= shape[i]; --i) {
+(*coord)[i] -= shape[i];
+++(*coord)[i-1];
+*idx = *idx + stride[i-1] - shape[i] * stride[i];
 
 Review comment:
   curious: why not += ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on a change in pull request #8566: optimize broadcast

[GitBox]

cjolivier01 commented on a change in pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566#discussion_r149240168
 
 

 ##
 File path: src/operator/mxnet_op.h
 ##
 @@ -334,6 +365,24 @@ struct Kernel {
 }
 #endif
   }
+
+  template
+  inline static void LaunchEx(mshadow::Stream *s, const int N, Args... 
args) {
 
 Review comment:
   ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] piiswrong opened a new pull request #8566: optimize broadcast

[GitBox]

piiswrong opened a new pull request #8566: optimize broadcast
URL: https://github.com/apache/incubator-mxnet/pull/8566
 
 
   @reminisce @cjolivier01 
   
   This can also be used to optimize slice, stack, etc.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ykim362 commented on issue #8532: mxnet-mkl (v0.12.0) crash when using (conda-installed) numpy with MKL

[GitBox]

ykim362 commented on issue #8532: mxnet-mkl (v0.12.0) crash when using 
(conda-installed) numpy with MKL
URL: 
https://github.com/apache/incubator-mxnet/issues/8532#issuecomment-342320754
 
 
   @fhieber Could you provide your numpy (conda) version?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] chsin opened a new issue #8563: feature extraction results in different output for same input

[GitBox]

chsin opened a new issue #8563: feature extraction results in different output 
for same input
URL: https://github.com/apache/incubator-mxnet/issues/8563
 
 
   ## Description
   I get different outputs from the caffenet fc7 layer for the same input. I 
don't remember where I downloaded the model, but it was either through 
http://data.dmlc.ml/mxnet/models/ or the older mxnet model repo.
   
   ## Environment info (Required)
   maxOS Sierra, v10.12.6
   Python 2.7.10
   
   ## Minimum reproducible example
   put this into test_fe_sym.py  where [model_dir] is where 
caffenet-.params and caffenet-symbol.json is.
   ``` python
   import mxnet as mx
   import numpy as np
   
   print 'mxnet version', mx.__version__
   print 'numpy version', np.__version__
   
   mod_name = "[mod_dir]/caffenet" 
   sym, arg_params, aux_params = mx.model.load_checkpoint(mod_name, 0)
   all_layers = sym.get_internals()
   fe_sym = all_layers['fc7_output']
   
   print fe_sym.get_internals()
   
   concat_args = {key:arg_params[key] for key in arg_params.keys()}
   mod = mx.mod.Module(symbol=fe_sym, data_names=['data'], label_names=[])
   mod.bind(data_shapes=[('data',(1, 3, 224, 224))])
   mod.init_params(arg_params = concat_args)
   
   mx_img = mx.nd.array(np.zeros((1,3,224,224)))
   batch = mx.io.DataBatch([mx_img], [])
   
   for _ in range(10):
 mod.forward(batch)
 print mod.get_outputs()[0]
   ```
   
   This is what I get when I run it in terminal:
   ```
   >> python test_fe_sym.py 
   mxnet version 0.11.0
   numpy version 1.13.3
   [16:14:24] src/nnvm/legacy_json_util.cc:190: Loading symbol saved by 
previous version v0.8.0. Attempting to upgrade...
   [16:14:24] src/nnvm/legacy_json_util.cc:198: Symbol successfully upgraded!
   
   
   [[-1.93128979 -2.29720759 -2.70292187 ..., -2.25644875 -3.05303383
 -1.76401949]]
   
   
   [[-1.80183637 -2.68599343 -2.66020608 ..., -1.50269926 -2.29078102
 -2.43784142]]
   
   
   [[-1.28073776 -2.04497075 -3.16189289 ..., -1.27064598 -3.24989033
 -2.34601569]]
   
   
   [[-1.86191022 -2.48778987 -2.61905193 ..., -2.04039574 -2.48587728
 -2.52999163]]
   
   
   [[-2.63889694 -2.77093887 -3.30290985 ..., -2.15549564 -2.78999281
 -2.12462115]]
   
   
   [[-1.32172596 -2.2919054  -2.7314136  ..., -1.93443453 -2.81845975
 -2.4142251 ]]
   
   
   [[-1.30976808 -2.26464558 -3.43015289 ..., -1.98673713 -2.11255097
 -2.5531776 ]]
   
   
   [[-2.22697783 -2.39356375 -2.24538589 ..., -2.7480278  -2.95472836
 -2.87552905]]
   
   
   [[-2.72385931 -2.22037935 -3.52920914 ..., -2.34241581 -2.9798
 -2.02456665]]
   
   
   [[-2.73990583 -1.87716341 -3.2182374  ..., -2.65652514 -2.66080451
 -2.56173849]]
   
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] EsraaRagaa opened a new issue #8562: Number of output from the mx.sym.SoftmaxOutput

[GitBox]

EsraaRagaa opened a new issue #8562: Number of output from the 
mx.sym.SoftmaxOutput
URL: https://github.com/apache/incubator-mxnet/issues/8562
 
 
   ## I am training a model using CNN to classify images between class 1 and 0, 
but after prediction, I get number of classes equal to number of nodes in the 
fully connected layer which precedes the softmax layer
   
   # This is the code I use for building the model:
   data = mx.sym.var('data')
   conv1 = mx.sym.Convolution(data=data, kernel=(3,3), num_filter=6)
   relu1 = mx.sym.Activation(data=conv1, act_type="relu")
   pool1 = mx.sym.Pooling(data=relu1, pool_type="max", kernel=(2,2), 
stride=(2,2))
   conv2 = mx.sym.Convolution(data=pool1, kernel=(6,6), num_filter=12)
   relu2 = mx.sym.Activation(data=conv2, act_type="relu")
   pool2 = mx.sym.Pooling(data=relu2, pool_type="max", kernel=(2,2), 
stride=(2,2))
   flatten = mx.sym.flatten(data=pool2)
   fc1 = mx.symbol.FullyConnected(data=flatten, num_hidden=48)
   lenet = mx.sym.SoftmaxOutput(data=fc1,name='softmax')
   lenet_model = mx.mod.Module(symbol=lenet, context=mx.cpu())
   lenet_model.fit(train_iterator,
 .
   num_epoch=10)
   
   # The solutions I tried to use to fix the problem:
   
   1- 
   lables = mx.sym.Variable('softmax_label')
   lenet = mx.sym.SoftmaxOutput(data=fc1,label=lables,name='softmax')
   * result:  did not fix the problem, still after prediciton the number 
probability coresponding to the classes=48
   
   2- 
   lables = mx.sym.Variable('softmax_label')
   lenet = mx.sym.SoftmaxOutput(data=fc1,label=lables, preserve_shape=True, 
name='softmax')
   * result:  did not fix the problem, still after prediciton the number 
probability coresponding to the classes=48
   3- 
   lables = mx.sym.Variable('softmax_label')
   lenet = mx.sym.SoftmaxOutput(data=fc1,label=lables,
  preserve_shape=True,
  multi_output=True, 
name='softmax')
   * result:  did not fix the problem, still after prediciton the number 
probability coresponding to the classes=48
   
   4- adding an extra hidden layer after fc1and pass it to the softmax
   extra_hidden_2nods = mx.symbol.FullyConnected(data=fc1, num_hidden=2)
   lenet = mx.sym.SoftmaxOutput(data=extra_hidden_2nods, name='softmax')
   * result:  fixed the problem but this is not what I want
   
   
   ## Environment info (Required)
   I am using the last version of mxnet, python, anaconda
   
   Thanks in advance


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] analog-cbarber commented on issue #8312: Gradient function not returning enough gradients

[GitBox]

analog-cbarber commented on issue #8312: Gradient function not returning enough 
gradients
URL: 
https://github.com/apache/incubator-mxnet/issues/8312#issuecomment-342278506
 
 
   Is there a workaround?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhreshold commented on issue #8546: What's the equivalent of np.where()?

[GitBox]

zhreshold commented on issue #8546: What's the equivalent of np.where()?
URL: 
https://github.com/apache/incubator-mxnet/issues/8546#issuecomment-342269299
 
 
   numpy.where is cannot be fully supported because it's output shape is 
dependent on the values of data which is beyond the capability of inter_shape.
   In some circumstances, you can have a work-around using nd.arange and 
nd.take/pick operations.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhreshold commented on issue #8549: What's the equivalent of tf.matmul in tensorflow

[GitBox]

zhreshold commented on issue #8549: What's the equivalent of tf.matmul in 
tensorflow
URL: 
https://github.com/apache/incubator-mxnet/issues/8549#issuecomment-342268590
 
 
   it's `nd/sym.batch_dot`
   You will need to stack 32x32 into batch_size and set `transpose_a == True`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 commented on issue #8553: Engine reserves cores from OMP. Set some defaults for dynamic and recursion

[GitBox]

cjolivier01 commented on issue #8553: Engine reserves cores from OMP.  Set some 
defaults for dynamic and recursion
URL: https://github.com/apache/incubator-mxnet/pull/8553#issuecomment-342257766
 
 
   this was done with a rebase, I don't know why the other commit is showing up


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] cjolivier01 opened a new pull request #8561: Fix unused var warning in release build (only used in assert on debug?

[GitBox]

cjolivier01 opened a new pull request #8561: Fix unused var warning in release 
build (only used in assert on debug?
URL: https://github.com/apache/incubator-mxnet/pull/8561
 
 
   ? builds)
   
   ## Description ##
   (Brief description on what this PR is about)
   
   ## Checklist ##
   ### Essentials ###
   - [ ] Passed code style checking (`make lint`)
   - [ ] Changes are complete (i.e. I finished coding on this PR)
   - [ ] All changes have test coverage
   - [ ] For user-facing API changes, API doc string has been updated.
   - [ ] To my best knowledge, examples are either not affected by this change, 
or have been fixed to be compatible with this change
   
   ### Changes ###
   - [ ] Feature1, tests, (and when applicable, API doc)
   - [ ] Feature2, tests, (and when applicable, API doc)
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be 
made.
   - Intersting edge cases to note here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-mxnet] branch master updated: * [cpp-package] fix for issue #7725 (#8551)

[jxie]

This is an automated email from the ASF dual-hosted git repository.

jxie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git


The following commit(s) were added to refs/heads/master by this push:
 new 3f37577  * [cpp-package] fix for issue #7725 (#8551)
3f37577 is described below

commit 3f3757741416a927d63f2204b6ff9f302bba83d6
Author: Ade 
AuthorDate: Tue Nov 7 02:13:07 2017 +0800

* [cpp-package] fix for issue #7725 (#8551)

* remove some unused include (MxNetCpp.h already included op.h)
* use same mnist data path avoiding duplication
---
 cpp-package/example/alexnet.cpp   |  3 +-
 cpp-package/example/charRNN.cpp   |  2 -
 cpp-package/example/googlenet.cpp | 12 +++---
 cpp-package/example/inception_bn.cpp  |  4 +-
 cpp-package/example/lenet.cpp |  4 +-
 cpp-package/example/lenet_with_mxdataiter.cpp | 57 +++
 cpp-package/example/mlp.cpp   |  3 +-
 cpp-package/example/mlp_cpu.cpp   |  8 ++--
 cpp-package/example/resnet.cpp|  4 +-
 9 files changed, 55 insertions(+), 42 deletions(-)

diff --git a/cpp-package/example/alexnet.cpp b/cpp-package/example/alexnet.cpp
index 4194b5b..dd5d2b4 100644
--- a/cpp-package/example/alexnet.cpp
+++ b/cpp-package/example/alexnet.cpp
@@ -23,8 +23,7 @@
 #include 
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace std;
 using namespace mxnet::cpp;
diff --git a/cpp-package/example/charRNN.cpp b/cpp-package/example/charRNN.cpp
index f5fff85..218d11e 100644
--- a/cpp-package/example/charRNN.cpp
+++ b/cpp-package/example/charRNN.cpp
@@ -43,8 +43,6 @@
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
 
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
 
 using namespace std;
 using namespace mxnet::cpp;
diff --git a/cpp-package/example/googlenet.cpp 
b/cpp-package/example/googlenet.cpp
index ac0585e..fe5dea6 100644
--- a/cpp-package/example/googlenet.cpp
+++ b/cpp-package/example/googlenet.cpp
@@ -22,10 +22,8 @@
 #include 
 #include 
 #include 
-
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace mxnet::cpp;
 
@@ -159,8 +157,8 @@ int main(int argc, char const *argv[]) {
 train_iter.Reset();
 while (train_iter.Next()) {
   auto data_batch = train_iter.GetDataBatch();
-  args_map["data"] = data_batch.data.Copy(Context::gpu());
-  args_map["data_label"] = data_batch.label.Copy(Context::gpu());
+  data_batch.data.CopyTo(_map["data"]);
+  data_batch.label.CopyTo(_map["data_label"]);
   NDArray::WaitAll();
   exec->Forward(true);
   exec->Backward();
@@ -174,8 +172,8 @@ int main(int argc, char const *argv[]) {
 val_iter.Reset();
 while (val_iter.Next()) {
   auto data_batch = val_iter.GetDataBatch();
-  args_map["data"] = data_batch.data.Copy(Context::gpu());
-  args_map["data_label"] = data_batch.label.Copy(Context::gpu());
+  data_batch.data.CopyTo(_map["data"]);
+  data_batch.label.CopyTo(_map["data_label"]);
   NDArray::WaitAll();
   exec->Forward(false);
   NDArray::WaitAll();
diff --git a/cpp-package/example/inception_bn.cpp 
b/cpp-package/example/inception_bn.cpp
index de21aad..e6f4790 100644
--- a/cpp-package/example/inception_bn.cpp
+++ b/cpp-package/example/inception_bn.cpp
@@ -19,13 +19,11 @@
 
 /*!
  */
-#include 
 #include 
 #include 
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace mxnet::cpp;
 
diff --git a/cpp-package/example/lenet.cpp b/cpp-package/example/lenet.cpp
index 05cc451..4c5a1f1 100644
--- a/cpp-package/example/lenet.cpp
+++ b/cpp-package/example/lenet.cpp
@@ -19,14 +19,12 @@
 
 /*!
  */
-#include 
 #include 
 #include 
 #include 
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace std;
 using namespace mxnet::cpp;
diff --git a/cpp-package/example/lenet_with_mxdataiter.cpp 
b/cpp-package/example/lenet_with_mxdataiter.cpp
index 077f556..04f5cbc 100644
--- a/cpp-package/example/lenet_with_mxdataiter.cpp
+++ b/cpp-package/example/lenet_with_mxdataiter.cpp
@@ -19,14 +19,12 @@
 
 /*!
  */
-#include 
-#include 
 #include 
 #include 
 #include 
+#include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace std;
 using namespace mxnet::cpp;
@@ -89,15 +87,15 @@ int main(int argc, char const *argv[]) {
   args_map["fc2_b"] = 0;
 
   auto train_iter = MXDataIter("MNISTIter")
-  .SetParam("image", "./train-images-idx3-ubyte")
-  .SetParam("label", "./train-labels-idx1-ubyte")
+  .SetParam("image", "./mnist_data/train-images-idx3-ubyte")
+  

[GitHub] piiswrong closed pull request #8551: * [cpp-package] fix cpp issue

[GitBox]

piiswrong closed pull request #8551: * [cpp-package] fix cpp issue
URL: https://github.com/apache/incubator-mxnet/pull/8551
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/cpp-package/example/alexnet.cpp b/cpp-package/example/alexnet.cpp
index 4194b5bae9..dd5d2b4b06 100644
--- a/cpp-package/example/alexnet.cpp
+++ b/cpp-package/example/alexnet.cpp
@@ -23,8 +23,7 @@
 #include 
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace std;
 using namespace mxnet::cpp;
diff --git a/cpp-package/example/charRNN.cpp b/cpp-package/example/charRNN.cpp
index f5fff853cb..218d11efc9 100644
--- a/cpp-package/example/charRNN.cpp
+++ b/cpp-package/example/charRNN.cpp
@@ -43,8 +43,6 @@
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
 
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
 
 using namespace std;
 using namespace mxnet::cpp;
diff --git a/cpp-package/example/googlenet.cpp 
b/cpp-package/example/googlenet.cpp
index ac0585e81a..fe5dea6a1f 100644
--- a/cpp-package/example/googlenet.cpp
+++ b/cpp-package/example/googlenet.cpp
@@ -22,10 +22,8 @@
 #include 
 #include 
 #include 
-
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace mxnet::cpp;
 
@@ -159,8 +157,8 @@ int main(int argc, char const *argv[]) {
 train_iter.Reset();
 while (train_iter.Next()) {
   auto data_batch = train_iter.GetDataBatch();
-  args_map["data"] = data_batch.data.Copy(Context::gpu());
-  args_map["data_label"] = data_batch.label.Copy(Context::gpu());
+  data_batch.data.CopyTo(_map["data"]);
+  data_batch.label.CopyTo(_map["data_label"]);
   NDArray::WaitAll();
   exec->Forward(true);
   exec->Backward();
@@ -174,8 +172,8 @@ int main(int argc, char const *argv[]) {
 val_iter.Reset();
 while (val_iter.Next()) {
   auto data_batch = val_iter.GetDataBatch();
-  args_map["data"] = data_batch.data.Copy(Context::gpu());
-  args_map["data_label"] = data_batch.label.Copy(Context::gpu());
+  data_batch.data.CopyTo(_map["data"]);
+  data_batch.label.CopyTo(_map["data_label"]);
   NDArray::WaitAll();
   exec->Forward(false);
   NDArray::WaitAll();
diff --git a/cpp-package/example/inception_bn.cpp 
b/cpp-package/example/inception_bn.cpp
index de21aadea9..e6f47904e0 100644
--- a/cpp-package/example/inception_bn.cpp
+++ b/cpp-package/example/inception_bn.cpp
@@ -19,13 +19,11 @@
 
 /*!
  */
-#include 
 #include 
 #include 
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace mxnet::cpp;
 
diff --git a/cpp-package/example/lenet.cpp b/cpp-package/example/lenet.cpp
index 05cc4517fe..4c5a1f1165 100644
--- a/cpp-package/example/lenet.cpp
+++ b/cpp-package/example/lenet.cpp
@@ -19,14 +19,12 @@
 
 /*!
  */
-#include 
 #include 
 #include 
 #include 
 #include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace std;
 using namespace mxnet::cpp;
diff --git a/cpp-package/example/lenet_with_mxdataiter.cpp 
b/cpp-package/example/lenet_with_mxdataiter.cpp
index 077f556225..04f5cbca3a 100644
--- a/cpp-package/example/lenet_with_mxdataiter.cpp
+++ b/cpp-package/example/lenet_with_mxdataiter.cpp
@@ -19,14 +19,12 @@
 
 /*!
  */
-#include 
-#include 
 #include 
 #include 
 #include 
+#include 
 #include "mxnet-cpp/MxNetCpp.h"
-// Allow IDE to parse the types
-#include "../include/mxnet-cpp/op.h"
+
 
 using namespace std;
 using namespace mxnet::cpp;
@@ -89,15 +87,15 @@ int main(int argc, char const *argv[]) {
   args_map["fc2_b"] = 0;
 
   auto train_iter = MXDataIter("MNISTIter")
-  .SetParam("image", "./train-images-idx3-ubyte")
-  .SetParam("label", "./train-labels-idx1-ubyte")
+  .SetParam("image", "./mnist_data/train-images-idx3-ubyte")
+  .SetParam("label", "./mnist_data/train-labels-idx1-ubyte")
   .SetParam("batch_size", batch_size)
   .SetParam("shuffle", 1)
   .SetParam("flat", 0)
   .CreateDataIter();
   auto val_iter = MXDataIter("MNISTIter")
-  .SetParam("image", "./t10k-images-idx3-ubyte")
-  .SetParam("label", "./t10k-labels-idx1-ubyte")
+  .SetParam("image", "./mnist_data/t10k-images-idx3-ubyte")
+  .SetParam("label", "./mnist_data/t10k-labels-idx1-ubyte")
   .CreateDataIter();
 
   Optimizer* opt = OptimizerRegistry::Find("ccsgd");
@@ -111,35 +109,62 @@ int main(int argc, char const *argv[]) {
   auto *exec = lenet.SimpleBind(Context::gpu(), args_map);
   auto arg_names = lenet.ListArguments();
 
+  // Create metrics
+  Accuracy train_acc, val_acc;
+
   for (int iter = 0; 

[GitHub] reminisce commented on issue #8558: slice operator supporting arbitrary values of step

[GitBox]

reminisce commented on issue #8558: slice operator supporting arbitrary values 
of step
URL: https://github.com/apache/incubator-mxnet/pull/8558#issuecomment-342223803
 
 
   @cjolivier01 I tried replacing infer shape and forward kernel with the same 
ones as in mshadow, but there is still about 10% difference. That means there 
is some difference between the runs of mshadow and Kernel::Launch. Any insights?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] reminisce commented on a change in pull request #8558: slice operator supporting arbitrary values of step

[GitBox]

reminisce commented on a change in pull request #8558: slice operator 
supporting arbitrary values of step
URL: https://github.com/apache/incubator-mxnet/pull/8558#discussion_r149144180
 
 

 ##
 File path: src/operator/tensor/matrix_op-inl.h
 ##
 @@ -620,6 +626,227 @@ void SliceEx(const nnvm::NodeAttrs& attrs,
   }
 }
 
+template
+inline void GetIndexRange(const SliceParam& param,
+  const TShape& dshape,
+  common::StaticArray* begin,
+  common::StaticArray* end,
+  common::StaticArray* step) {
+  CHECK_NE(dshape.ndim(), 0U);
+  CHECK_NE(dshape.Size(), 0U);
+  CHECK_LE(param.begin.ndim(), dshape.ndim())
+<< "Slicing axis exceeds data dimensions";
+  CHECK_LE(param.end.ndim(), dshape.ndim())
+<< "Slicing axis exceeds data dimensions";
+  CHECK_EQ(param.begin.ndim(), param.end.ndim())
+<< "begin and end must have the same length";
+  CHECK_EQ(ndim, dshape.ndim())
+<< "Static array size=" << ndim
+<< " is not equal to data shape ndim=" << dshape.ndim();
+
+  if (param.step.has_value()) {
+CHECK_EQ(param.step.value().ndim(), param.begin.ndim())
+  << "step and begin must have the same length";
+  }
+
+  for (index_t i = 0; i < param.begin.ndim(); ++i) {
+int b = 0, e = dshape[i], s = 1;
+const int len = dshape[i];
+if (param.step.has_value()) {
+  const auto& opt_step_val = param.step.value()[i];
+  if (opt_step_val.has_value()) {
+s = opt_step_val.value();
+CHECK_NE(s, 0) << "slice op step[" << i << "] cannot be 0";
+  }
+}
+
+if (param.begin[i].has_value()) {
+  b = param.begin[i].value();
+  if (b < 0) {
+b += len;
+CHECK_GE(b, 0) << "slicing with begin[" << i << "]="
+   << b - len << " exceeds limit of " << len;
+  }
+} else if (s < 0) {
+  b = len - 1;
+}
+CHECK_LT(b, len) << "slicing with begin[" << i << "]="
+ << b << " exceends limit of " << len;
+
+if (param.end[i].has_value()) {
+  e = param.end[i].value();
+  if (e < 0) {
+e += len;
+CHECK_GE(e, 0) << "slicing with end[" << i << "]="
+   << e - len << " exceeds limit of " << len;
+  }
+} else if (s < 0) {
+  e = -1;
+}
+CHECK_LE(e, len) << "slicing with end[" << i << "]="
 
 Review comment:
   It's not called if check is successful. See the code:
   
https://github.com/dmlc/dmlc-core/blob/595d02c0e87be8a0846700462b6f45f1b1031e39/include/dmlc/logging.h#L89
   All the << calls happen only when `_check_err` has a non-empty string stored.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jongchyisu commented on issue #8552: Problem in mx.io.ImageRecordIter

[GitBox]

jongchyisu commented on issue #8552: Problem in mx.io.ImageRecordIter
URL: 
https://github.com/apache/incubator-mxnet/issues/8552#issuecomment-342211556
 
 
   I see, thanks for the explanation!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ptrendx commented on issue #8552: Problem in mx.io.ImageRecordIter

[GitBox]

ptrendx commented on issue #8552: Problem in mx.io.ImageRecordIter
URL: 
https://github.com/apache/incubator-mxnet/issues/8552#issuecomment-342210318
 
 
   `reset` function used here has 2 slightly different behaviors due to that 
feature of the iterator - the first time you call it iterator knows it hit the 
end of epoch and internally already reset itself (to get those overflow images 
from the next epoch) and does not do anything else (besides setting its 
internal end of epoch indicator to `false`) - this means that this epoch will 
not include those overflow images. When you call it the second time iterator is 
in the middle of epoch and so resets itself completely - this makes the epoch 
give you all of the images, including those that were used at the end of the 
previous epoch.
   
   As I said, if you would prefer different behavior (not taking images from 
the next epoch and so having each epoch have the same number of images, you can 
use the `round_batch = False` option of the iterator).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jongchyisu commented on issue #8552: Problem in mx.io.ImageRecordIter

[GitBox]

jongchyisu commented on issue #8552: Problem in mx.io.ImageRecordIter
URL: 
https://github.com/apache/incubator-mxnet/issues/8552#issuecomment-342207466
 
 
   I see, but why do I need to reset twice to avoid exception error?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ptrendx commented on issue #8552: Problem in mx.io.ImageRecordIter

[GitBox]

ptrendx commented on issue #8552: Problem in mx.io.ImageRecordIter
URL: 
https://github.com/apache/incubator-mxnet/issues/8552#issuecomment-342206112
 
 
   The variable number of batches per epoch is kind of intentional. What 
happens here is basically: if the number of images is not divisible by the 
batch size, then iterator needs to fill the last batch with something after it 
used all of the images. By default it uses images from the next epoch (you can 
change that behavior by setting `round_batch` option of the iterator to 
`False`). Then, during the next epoch, it does not use those images (otherwise 
those images would be overrepresented in the training process), which in some 
cases may result in that epoch having 1 less batch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] anxiaoxi45 commented on issue #5475: Difference between ImageIter and ImageRecordIter

[GitBox]

anxiaoxi45 commented on issue #5475: Difference between ImageIter  and 
ImageRecordIter
URL: 
https://github.com/apache/incubator-mxnet/issues/5475#issuecomment-342192348
 
 
   @mli @piiswrong @tqchen This seems like a bug.  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] fmcarlucci opened a new issue #8560: Upsampling: lack of arguments crash the kernel

[GitBox]

fmcarlucci opened a new issue #8560: Upsampling: lack of arguments crash the 
kernel
URL: https://github.com/apache/incubator-mxnet/issues/8560
 
 
   Python 2.7, mxnet v '0.11.0'
   
   Very simply, if sym.UpSampling is not called with the appropriate 
parameters, MxNet crashes without meaningful error messages.
   
   This will crash:
   `mx.sym.UpSampling(data=body, scale=4, sample_type='bilinear', 
workspace=workspace)`
   
   But this is fine:
   `mx.sym.UpSampling(data=body, scale=4, num_filter=3, sample_type='bilinear', 
workspace=workspace)`
   
   Difference being that I added the num_filter argument 
   
   
   A full network example
   
   ```
   data = mx.sym.Variable(name='data')
   body = mx.sym.BatchNorm(data=data, fix_gamma=True, eps=2e-5, 
momentum=bn_mom, name='deco_bn_data')
   body = mx.sym.Convolution(data=body, num_filter=64, kernel=(7, 7), 
stride=(2,2), pad=(3, 3), 
 no_bias=True, name="deco_conv0", 
workspace=workspace)
   body = mx.sym.BatchNorm(data=body, fix_gamma=False, eps=2e-5, 
momentum=bn_mom, name='deco_bn0')
   body = mx.sym.LeakyReLU(data=body, act_type="leaky", slope=0.02, 
name='deco_lrelu0')
   body = mx.sym.Pooling(data=body, kernel=(3, 3), stride=(2,2), pad=(1,1), 
pool_type='max', name='deco_pooling')
   body = mx.sym.Convolution(data=body, num_filter=3, kernel=(1, 1), 
stride=(1,1), pad=(0, 0), 
 no_bias=True, name="deco_toRGB", 
workspace=workspace)
   body = mx.sym.UpSampling(data=body, scale=4, num_filter=3, 
sample_type='bilinear', workspace=workspace)
   test_net = mx.sym.Convolution(data=body, num_filter=1, kernel=(4, 4), 
stride=(3,3), pad=(0, 0))
   test_net = mx.symbol.FullyConnected(data=test_net, num_hidden=10, name='fc1')
   test_net = mx.symbol.SoftmaxOutput(data=test_net, name='softmax')
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KellenSunderland commented on a change in pull request #8555: Update PR & Issue Template

[GitBox]

KellenSunderland commented on a change in pull request #8555: Update PR & Issue 
Template
URL: https://github.com/apache/incubator-mxnet/pull/8555#discussion_r149082980
 
 

 ##
 File path: .github/PULL_REQUEST_TEMPLATE.md
 ##
 @@ -6,7 +6,7 @@
 - [ ] Passed code style checking (`make lint`)
 - [ ] Changes are complete (i.e. I finished coding on this PR)
 - [ ] All changes have test coverage
-- [ ] For user-facing API changes, API doc string has been updated.
+- [ ] For user-facing API changes, API doc string has been updated. For new 
C++ functions in header files, their functionalities and arguments are 
well-documented. 
 - [ ] To my best knowledge, examples are either not affected by this change, 
or have been fixed to be compatible with this change
 
 ### Changes ###
 
 Review comment:
   Let's also fix the typo below.  Intersting -> Interesting


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] larroy commented on issue #8526: Ci test randomness2

[GitBox]

larroy commented on issue #8526: Ci test randomness2
URL: https://github.com/apache/incubator-mxnet/pull/8526#issuecomment-342153353
 
 
   I like this approach, but I would like to have fixed seeds by default 
instead of setting an environment var, mostly for the cognitive load on test 
runners, we already have issues running tests on different platforms and all, 
if we add on top that not defining this variable is going to cause flaky tests 
or random failures, I think it's better to have it the other way around.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha closed issue #7350: Multi-Training-Task on the same GPU card

[GitBox]

szha closed issue #7350: Multi-Training-Task on the same GPU card
URL: https://github.com/apache/incubator-mxnet/issues/7350
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] szha commented on issue #7350: Multi-Training-Task on the same GPU card

[GitBox]

szha commented on issue #7350: Multi-Training-Task on the same GPU card
URL: 
https://github.com/apache/incubator-mxnet/issues/7350#issuecomment-342133913
 
 
   This issue is closed due to lack of activity in the last 90 days. Feel free 
to ping me to reopen if this is still an active issue. Thanks!
   Also, do please check out our [forum](https://discuss.mxnet.io/) (and 
[Chinese version](https://discuss.gluon.ai/)) for general "how-to" questions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ZiyueHuang commented on a change in pull request #8540: div_scalar for sparse

[GitBox]

ZiyueHuang commented on a change in pull request #8540: div_scalar for sparse
URL: https://github.com/apache/incubator-mxnet/pull/8540#discussion_r149063166
 
 

 ##
 File path: src/operator/tensor/elemwise_binary_scalar_op_basic.cc
 ##
 @@ -143,10 +143,19 @@ 
MXNET_OPERATOR_REGISTER_BINARY_SCALAR(_backward_mul_scalar)
 .set_attr("FComputeEx", BinaryScalarOp::ComputeEx);
 
 MXNET_OPERATOR_REGISTER_BINARY_SCALAR(_div_scalar)
+.set_attr("FInferStorageType", BinaryScalarStorageType)
 
 Review comment:
   Added :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ZiyueHuang commented on a change in pull request #8550: csr ndarray iter with shuffle

[GitBox]

ZiyueHuang commented on a change in pull request #8550: csr ndarray iter with 
shuffle
URL: https://github.com/apache/incubator-mxnet/pull/8550#discussion_r149058565
 
 

 ##
 File path: python/mxnet/io.py
 ##
 @@ -630,22 +631,25 @@ def __init__(self, data, label=None, batch_size=1, 
shuffle=False,
 super(NDArrayIter, self).__init__(batch_size)
 
 if ((_has_instance(data, CSRNDArray) or _has_instance(label, 
CSRNDArray)) and
-(shuffle or last_batch_handle != 'discard')):
+(last_batch_handle != 'discard')):
 raise NotImplementedError("`NDArrayIter` only supports 
``CSRNDArray``" \
-  " with `shuffle` set to `False`" \
-  " and `last_batch_handle` set to 
`discard`.")
+  " with `last_batch_handle` set to 
`discard`.")
 self.data = _init_data(data, allow_empty=False, default_name=data_name)
 self.label = _init_data(label, allow_empty=True, 
default_name=label_name)
 
 self.idx = np.arange(self.data[0][1].shape[0])
 # shuffle data
 if shuffle:
 np.random.shuffle(self.idx)
-self.data = [(k, array(v.asnumpy()[self.idx], v.context))
+self.data = [(k, sparse_array(v.asscipy()[self.idx], v.context)
 
 Review comment:
   Done :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] roggiezhang-nv closed issue #8494: Autograd bug in mxnet-cu80: 0.12

[GitBox]

roggiezhang-nv closed issue #8494: Autograd bug in mxnet-cu80: 0.12
URL: https://github.com/apache/incubator-mxnet/issues/8494
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] roggiezhang-nv commented on issue #8494: Autograd bug in mxnet-cu80: 0.12

[GitBox]

roggiezhang-nv commented on issue #8494: Autograd bug in mxnet-cu80: 0.12
URL: 
https://github.com/apache/incubator-mxnet/issues/8494#issuecomment-342100827
 
 
   Interesting. This is solved since 0.12.1. Close this as fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KiddoZhu commented on issue #833: [Discussion] FP16 Support

[GitBox]

KiddoZhu commented on issue #833: [Discussion] FP16 Support
URL: https://github.com/apache/incubator-mxnet/issues/833#issuecomment-342089105
 
 
   @tqchen How can I use cast<> in my code? I cannot find any definition of 
cast in mshadow.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] KiddoZhu commented on issue #833: [Discussion] FP16 Support

[GitBox]

KiddoZhu commented on issue #833: [Discussion] FP16 Support
URL: https://github.com/apache/incubator-mxnet/issues/833#issuecomment-342089105
 
 
   @tqchen How can I use cast<> in my code? I cannot find a definition of cast 
in mshadow.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] zhangxiao-ustc commented on issue #7519: cmake -DUSE_DIST_KVSTORE=1 . ERROR

[GitBox]

zhangxiao-ustc commented on issue #7519: cmake -DUSE_DIST_KVSTORE=1 . ERROR
URL: 
https://github.com/apache/incubator-mxnet/issues/7519#issuecomment-342080378
 
 
   Has this problem been solved? I got the same error log. 
   with standard v0.11.0 branch


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services