[apache/incubator-mxnet] [RFC] Custom subgraph property part2 (#17236)
## Description Request for comments on the next PR for enhancing custom subgraph property support Heres some suggestions from the initial PR (Part 1): Heres some suggestions from the initial PR (Part 1): - docs, readme, tutorial - dynamic graph passes ## References - initial PR (Part 1): #17034 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17236
Re: CD with windows need a special jenkins slave machine like restricted-utility
Regarding visual studio 2019: It seems we currently support Visual Studio 2015? Is there anything that Visual Studio 2015 can't do? If so, code and documentation should also be updated based on the new minimum version. On Tue, 2020-01-07 at 14:19 +0800, shiwen hu wrote: > it need visual studio 2019, cuda 9.0 9.2 10.0 10.1 10.2, > cmake 3.16.2,jom,opencv,openblas. > What do I need to do? Who should I contact?
[VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
Dear MXNet community, This is the vote to release Apache MXNet (incubating) version 1.6.0. Voting starts today and will close on Friday 1/10/2020 23:59 PST. Link to release notes: https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+notes Link to release candidate: https://github.com/apache/incubator-mxnet/releases/tag/1.6.0.rc1 Link to source and signatures on apache dist server: https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.6.0.rc1/ The differences comparing to previous release candidate 1.6.0.rc0: * Fix for RNN gradient calculation for MKLDNN ([v1.6.x] Cherry-pick MKL-DNN Rnn operator enhancements to v1.6.x (#17225)) * Fix for Windows CMake build (Backport #16980 #17031 #17018 #17019 to 1.6 branch (#17213)) * CPU counterpart to contrib multihead attention operators (Interleaved MHA for CPU path (#17138) (#17211)) * Fix for #16060 (fix norm sparse fallback (#17149)) * Fix for inconsistent names in estimator API (fix parameter names in the estimator api (#17051) (#17162)) * Fixes for OpenMP (Backport 3rdparty/openmp fixes (#17193)) * Fix for pointwise fusion speed for large networks (which was the reason of -1 in the vote for rc0) as well as fixes for nondeterminism in sum of squares operator and trainer parameter order (Backport #17002, #17068 and #17114 to 1.6 branch (#17137)) Please remember to TEST first before voting accordingly: +1 = approve +0 = no opinion -1 = disapprove (provide reason) Best regards, Przemyslaw Tredak
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
+1 Build from source on Ubuntu 18 with CUDA/CUDNN/NCCL on and verified it works with Horovod 0.18.2 On Tue, Jan 7, 2020 at 9:55 AM Przemysław Trędak wrote: > Dear MXNet community, > > This is the vote to release Apache MXNet (incubating) version 1.6.0. > Voting starts today and will close on Friday 1/10/2020 23:59 PST. > > Link to release notes: > https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+notes > > Link to release candidate: > https://github.com/apache/incubator-mxnet/releases/tag/1.6.0.rc1 > > Link to source and signatures on apache dist server: > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.6.0.rc1/ > > The differences comparing to previous release candidate 1.6.0.rc0: > * Fix for RNN gradient calculation for MKLDNN ([v1.6.x] Cherry-pick > MKL-DNN Rnn operator enhancements to v1.6.x (#17225)) > * Fix for Windows CMake build (Backport #16980 #17031 #17018 #17019 to 1.6 > branch (#17213)) > * CPU counterpart to contrib multihead attention operators (Interleaved > MHA for CPU path (#17138) (#17211)) > * Fix for #16060 (fix norm sparse fallback (#17149)) > * Fix for inconsistent names in estimator API (fix parameter names in the > estimator api (#17051) (#17162)) > * Fixes for OpenMP (Backport 3rdparty/openmp fixes (#17193)) > * Fix for pointwise fusion speed for large networks (which was the reason > of -1 in the vote for rc0) as well as fixes for nondeterminism in sum of > squares operator and trainer parameter order (Backport #17002, #17068 and > #17114 to 1.6 branch (#17137)) > > > Please remember to TEST first before voting accordingly: > +1 = approve > +0 = no opinion > -1 = disapprove (provide reason) > > > Best regards, > Przemyslaw Tredak >
[apache/incubator-mxnet] [MXNet 2.0] [RFC] Distribution Module (#17240)
## Description In MxNet 2.0, we would like to provide a distribution module, analogous to Pytorch distribution. The main difference from theirs is that we use numpy op and it allows hybridization. The current project code can be seen from https://github.com/xidulu/incubator-mxnet/tree/distribution_dev/python/mxnet/gluon/probability. The basic skeleton divides into following parts: 1. Stochastic `HybridBlock` and `HybridSequential`: they build upon gluon `HybridBlock` and `HybridSequential` and allows adding extra loss to each layer. 2. Distribution class: it implements a variety of functionalities including `prob`, `log_prob`, `sample`, `broadcast_to`, `mean`, `variance`, etc. 3. KL divergence: `kl_divergence(p, q)` function searches over registered KL divergence functions and performs computation. 4. Transform: transform one distribution to another invertible distribution. 5. Independent: reinterprets some of the batch dims of a distribution as event dims. Two features that is currently either not supported or kind of broken in MxNet will be very useful to this projects: symbolic shape and control flow. At the moment, we will implement most of distribution in frontend. We will move the computation to backend when new numpy probability ops such as `chisquare`, `dirichlet` and `multivariate_normal` are introduced into MxNet. ## References - https://pytorch.org/docs/stable/distributions.html - https://docs.scipy.org/doc/numpy-1.14.1/reference/routines.random.html @xidulu @szha @leezu @haojin2 -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17240
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
Correction: it was built from source on Ubuntu 16.04 On Tue, Jan 7, 2020 at 11:42 AM Lin Yuan wrote: > +1 > > Build from source on Ubuntu 18 with CUDA/CUDNN/NCCL on and verified it > works with Horovod 0.18.2 > > On Tue, Jan 7, 2020 at 9:55 AM Przemysław Trędak > wrote: > >> Dear MXNet community, >> >> This is the vote to release Apache MXNet (incubating) version 1.6.0. >> Voting starts today and will close on Friday 1/10/2020 23:59 PST. >> >> Link to release notes: >> https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+notes >> >> Link to release candidate: >> https://github.com/apache/incubator-mxnet/releases/tag/1.6.0.rc1 >> >> Link to source and signatures on apache dist server: >> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.6.0.rc1/ >> >> The differences comparing to previous release candidate 1.6.0.rc0: >> * Fix for RNN gradient calculation for MKLDNN ([v1.6.x] Cherry-pick >> MKL-DNN Rnn operator enhancements to v1.6.x (#17225)) >> * Fix for Windows CMake build (Backport #16980 #17031 #17018 #17019 to >> 1.6 branch (#17213)) >> * CPU counterpart to contrib multihead attention operators (Interleaved >> MHA for CPU path (#17138) (#17211)) >> * Fix for #16060 (fix norm sparse fallback (#17149)) >> * Fix for inconsistent names in estimator API (fix parameter names in the >> estimator api (#17051) (#17162)) >> * Fixes for OpenMP (Backport 3rdparty/openmp fixes (#17193)) >> * Fix for pointwise fusion speed for large networks (which was the reason >> of -1 in the vote for rc0) as well as fixes for nondeterminism in sum of >> squares operator and trainer parameter order (Backport #17002, #17068 and >> #17114 to 1.6 branch (#17137)) >> >> >> Please remember to TEST first before voting accordingly: >> +1 = approve >> +0 = no opinion >> -1 = disapprove (provide reason) >> >> >> Best regards, >> Przemyslaw Tredak >> >
Re: CD with windows need a special jenkins slave machine like restricted-utility
I'm putting some efforts on the side to improve the state of this: If you want to help: https://github.com/apache/incubator-mxnet/pull/17206 https://github.com/aiengines/ci/tree/master/windows Which of the cuda versions you listed it needs, I did some work on the side to update VS and cmake to 3.16.2 you can test the scripts in the windows folder above by using the three scripts in the windows folder in a fresh windows instance. The older CMake version has a bug which introduces a newline in the path and renders everything unusable, I installed 3.16.2 but needs to be added to the path by the install script. You can start a fresh gpu instance with this AMI: aws ssm get-parameter --name /aws/service/ami-windows-latest/Windows_Server-2019-English-Full-Base Once this is working, we can update the AMI from CI. Also this needs to be adjusted with the new VS 2019 https://github.com/apache/incubator-mxnet/blob/master/ci/build_windows.py#L42 To update cuda and nv driver, this two bundles should be added to the script https://github.com/aiengines/ci/blob/master/windows/windows_deps_headless_installer.py https://windows-post-install.s3-us-west-2.amazonaws.com/cuda.zip https://windows-post-install.s3-us-west-2.amazonaws.com/nv_driver_418.81.zip Send PRs if you want to collaborate. Pedro. On Tue, Jan 7, 2020 at 6:13 AM Lausen, Leonard wrote: > Regarding visual studio 2019: It seems we currently support Visual Studio > 2015? > Is there anything that Visual Studio 2015 can't do? If so, code and > documentation should also be updated based on the new minimum version. > > On Tue, 2020-01-07 at 14:19 +0800, shiwen hu wrote: > > it need visual studio 2019, cuda 9.0 9.2 10.0 10.1 10.2, > > cmake 3.16.2,jom,opencv,openblas. > > What do I need to do? Who should I contact? >
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
+1 Build from source on Ubuntu with CUDA/CUDNN/MKLDNN and tested with keras-mxnet. Unit tests passed and example works on CPU/GPU. Best Regards Lai On Tue, Jan 7, 2020 at 11:49 AM Lin Yuan wrote: > Correction: it was built from source on Ubuntu 16.04 > > On Tue, Jan 7, 2020 at 11:42 AM Lin Yuan wrote: > > > +1 > > > > Build from source on Ubuntu 18 with CUDA/CUDNN/NCCL on and verified it > > works with Horovod 0.18.2 > > > > On Tue, Jan 7, 2020 at 9:55 AM Przemysław Trędak > > wrote: > > > >> Dear MXNet community, > >> > >> This is the vote to release Apache MXNet (incubating) version 1.6.0. > >> Voting starts today and will close on Friday 1/10/2020 23:59 PST. > >> > >> Link to release notes: > >> https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+notes > >> > >> Link to release candidate: > >> https://github.com/apache/incubator-mxnet/releases/tag/1.6.0.rc1 > >> > >> Link to source and signatures on apache dist server: > >> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.6.0.rc1/ > >> > >> The differences comparing to previous release candidate 1.6.0.rc0: > >> * Fix for RNN gradient calculation for MKLDNN ([v1.6.x] Cherry-pick > >> MKL-DNN Rnn operator enhancements to v1.6.x (#17225)) > >> * Fix for Windows CMake build (Backport #16980 #17031 #17018 #17019 to > >> 1.6 branch (#17213)) > >> * CPU counterpart to contrib multihead attention operators (Interleaved > >> MHA for CPU path (#17138) (#17211)) > >> * Fix for #16060 (fix norm sparse fallback (#17149)) > >> * Fix for inconsistent names in estimator API (fix parameter names in > the > >> estimator api (#17051) (#17162)) > >> * Fixes for OpenMP (Backport 3rdparty/openmp fixes (#17193)) > >> * Fix for pointwise fusion speed for large networks (which was the > reason > >> of -1 in the vote for rc0) as well as fixes for nondeterminism in sum of > >> squares operator and trainer parameter order (Backport #17002, #17068 > and > >> #17114 to 1.6 branch (#17137)) > >> > >> > >> Please remember to TEST first before voting accordingly: > >> +1 = approve > >> +0 = no opinion > >> -1 = disapprove (provide reason) > >> > >> > >> Best regards, > >> Przemyslaw Tredak > >> > > >
Re: [apache/incubator-mxnet] [MXNet 2.0] [RFC] Distribution Module (#17240)
> Two features that is currently either not supported or kind of broken in > MxNet will be very useful to this projects: symbolic shape and control flow. More specifically, control flow helps perform parameter validity check: https://github.com/apache/incubator-mxnet/issues/16868 -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/17240#issuecomment-571845540
Re: CD with windows need a special jenkins slave machine like restricted-utility
> > I personally encountered the problem that 2015 can't compile in high > version cuda. But I can't remember the details. We can continue to use 2015 > until we encounter problems. >
