RE: [Announcement] New Committer - Sam Skalicky

[Chen, Ciyong]

Congratulations Sam! 
Those are great features to MXNet.

-Original Message-
From: Zhao, Patric  
Sent: Thursday, July 30, 2020 8:40 AM
To: dev@mxnet.incubator.apache.org
Subject: RE: [Announcement] New Committer - Sam Skalicky

Congratulations, Sam, thanks all of your great works in MXNet 

> -Original Message-
> From: Chaitanya Bapat 
> Sent: Thursday, July 30, 2020 1:12 AM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: [Announcement] New Committer - Sam Skalicky
> 
> Congratulations Sam! Well deserved!
> 
> On Wed, 29 Jul 2020 at 08:05, Marco de Abreu 
> wrote:
> 
> > Welcome!
> >
> > -Marco
> >
> > On Wed, Jul 29, 2020, 4:58 PM sandeep krishnamurthy < 
> > sandeep.krishn...@gmail.com> wrote:
> >
> > > Hello all,
> > >
> > > Please join me in welcoming Sam Skalicky(@samskalicky) as a new 
> > > committer of Apache MXNet (incubating)!
> > >
> > > Sam has made a number of contributions to this project such as 
> > > SubGraphs, Custom Ops, Accelerator APIs, along with several other 
> > > operator implementations and bug fixes. Sam has been actively 
> > > engaging in PR reviews, dev@ list discussions and helping the 
> > > project and fellow contributors.
> > >
> > > Sam, thank you for all your contributions and looking forward to 
> > > more support!
> > >
> > > Welcome, Sam!
> > >
> > > --
> > > Sandeep Krishnamurthy
> > >
> >
> 
> 
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
> 
> [image: https://www.linkedin.com//in/chaibapat25]
> [image:
> https://www.facebook.com/chaibapat]
> [image:
> https://twitter.com/ChaiBapchya] [image:
> https://www.linkedin.com//in/chaibapat25]
> 

RE: assimilation of mshadow into the MXNet codebase

[Chen, Ciyong]

Hi Justin,

We're doing 1.7.0 source release recently, and the vote on dev@ was passed, 
vote thread [1], result thread[2].
Seems the discussion on mshadow donation is still not finalized, may I know if 
you have any concern to proceed the current release under DISCLAIMER-WIP? 

Thanks,
-Ciyong

[1] 
https://lists.apache.org/thread.html/r525a961a10f69bdfb255c64f0be0589bb70efdd880c1be87c81c0c06%40%3Cdev.mxnet.apache.org%3E
[2] 
https://lists.apache.org/thread.html/rbd53614ca01f714d00097a02d906895211336a14ce0e083865cf5144%40%3Cdev.mxnet.apache.org%3E


-Original Message-
From: Sheng Zha  
Sent: Thursday, July 23, 2020 3:29 PM
To: Justin Mclean 
Cc: d...@mxnet.apache.org; Wall Michael ; Bob Paulin 
; wei...@apache.org; jason...@apache.org
Subject: Re: assimilation of mshadow into the MXNet codebase

Hi,

No, I don’t think we used ICLAs for mshadow before.

Out of the 42 people who made more than 1 commit or more than 10 lines of code 
change to mshadow, 26 signed ICLA with Apache (and additionally one member is 
unfortunately deceased...). Would this be a better criteria as “the major 
ones”? I wasn’t part of the initial code donation or the initial PPMC group, so 
apologies if the questions were silly.

I think the rest of the commits are manageable so that I could do a revert and 
rework for those commits if/when necessary.

Regards,
Sheng

> On Jul 22, 2020, at 11:50 PM, Justin Mclean  wrote:
> 
> Hi,
> 
>> Thanks for clarifying. All contributors who made more than 10 commits to 
>> msahdow before are committers of MXNet, so their ICLAs should already be on 
>> file: tqchen, bingxu, eric.xie, sxjscience, mli, yajiedesign [1]. If you 
>> think this is OK, one of the mentors or I can start the notification.
> 
> 
> What about the other 60 contributors? More than 10 commits is not a line I 
> would feel comfortable with. You need to be able to account for the IP 
> provenance of every line of code, just like in your initial code donation. It 
> would probably be best to make a list all contributors and if they have an 
> ICLA or not. Did the mshadow project use ICLAs? If so that may also help.
> 
> Thanks,
> Justin

RE: [RESULTS] [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc1

[Chen, Ciyong]

Thanks for your kindly reminder Macro, I will send out the vote on general@ 
later.

Regards,
-Ciyong

-Original Message-
From: Marco de Abreu  
Sent: Thursday, July 23, 2020 3:44 PM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org; Bob Paulin ; Henri Yandell 
; Jason Dai ; Markus Weimer 
; Michael Wall 
Subject: Re: [RESULTS] [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc1

Thanks! Please don't forget the release vote on incubator.

-Marco

On Thu, Jul 23, 2020, 9:37 AM Chen, Ciyong  wrote:

> Dear MXNet community,
>
> I'm happy to announce the results of the vote.
> This vote passes with 10 +1 votes (3 binding) and no 0 or -1 votes.
>
> +1 votes
> * Sheng Zha / binding
> * Tao Lv / binding
> * Zhi Zhang / binding
> * Aston Zhang
> * Patric Zhao
> * Skalicky Sam
> * Karan Jariwala
> * Chaitanya Bapat
> * Kshitij Kalambarkar
> * Patrick Mu
>
> 0 votes
> * No votes
>
> -1 votes
> * No votes
>
> Vote thread can be found here [1]. The list of members can be found 
> here [2].
> I'll continue with the release process and the release announcement 
> will follow in the next few days.
>
> Best regards,
> Ciyong Chen
>
> [1]
> https://lists.apache.org/thread.html/r525a961a10f69bdfb255c64f0be0589b
> b70efdd880c1be87c81c0c06%40%3Cdev.mxnet.apache.org%3E
> [2] http://incubator.apache.org/projects/mxnet.html
>
>

[RESULTS] [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc1

[Chen, Ciyong]

Dear MXNet community,

I'm happy to announce the results of the vote.
This vote passes with 10 +1 votes (3 binding) and no 0 or -1 votes.

+1 votes
* Sheng Zha / binding
* Tao Lv / binding
* Zhi Zhang / binding
* Aston Zhang
* Patric Zhao
* Skalicky Sam
* Karan Jariwala
* Chaitanya Bapat
* Kshitij Kalambarkar
* Patrick Mu

0 votes
* No votes

-1 votes
* No votes

Vote thread can be found here [1]. The list of members can be found here [2].
I'll continue with the release process and the release announcement will follow 
in the next few days.

Best regards,
Ciyong Chen

[1] 
https://lists.apache.org/thread.html/r525a961a10f69bdfb255c64f0be0589bb70efdd880c1be87c81c0c06%40%3Cdev.mxnet.apache.org%3E
[2] http://incubator.apache.org/projects/mxnet.html

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc1

[Chen, Ciyong]

Thanks Aston, Patric for the vote.

Hi Community,

I would like to call for action to test/validate/vote for the release candidate 
(1.7.0.rc1).
As we've not reached the quorum, I would like to extend the voting process to 
July 22, 23:59:59 PST.
Please prepare your time and provide feedback if you've tried with the 
pre-released code base, thanks!

Best Regards,
Ciyong

-Original Message-
From: Zhao, Patric  
Sent: Monday, July 20, 2020 11:36 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org; Bob Paulin ; Henri Yandell 
; Jason Dai ; Markus Weimer 
; Michael Wall 
Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc1

+1

Passed the performance benchmarking for CPU tests and no regression is found.


> -Original Message-
> From: Aston Zhang 
> Sent: Sunday, July 19, 2020 1:45 PM
> To: dev@mxnet.incubator.apache.org
> Cc: d...@mxnet.apache.org; Bob Paulin ; Henri Yandell 
> ; Jason Dai ; Markus Weimer 
> ; Michael Wall 
> Subject: Re: [VOTE] Release Apache MXNet (incubating) version 
> 1.7.0.rc1
> 
> +1
> Passed d2l-en v0.14.1: 
> https://github.com/d2l-ai/d2l-en/releases/tag/v0.14.1
> 
> On Thu, Jul 16, 2020 at 2:34 AM Chen, Ciyong  wrote:
> 
> > Dear MXNet community,
> >
> > This is the vote to release Apache MXNet (incubating) version 1.7.0.
> > Voting will start 16th July 23:59:59 PST and close on 19th July
> > 23:59:59 PST.
> >
> > Link to release notes:
> > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+note
> > s
> >
> > Link to release candidate:
> > https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc1
> >
> > Link to source and signatures on apache dist server:
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc1
> >
> > Please remember to TEST first before voting accordingly:
> > +1 = approve
> > +0 = no opinion
> > -1 = disapprove (provide reason)
> >
> > Here's the changes comparing to 1.7.0.rc0:
> >
> >   *   Revert "Fix memory leaks in Gluon (#18328) (#18358) (#18692)
> >   *   revise activations (#18700)
> >   *   Fix the monitor_callback invalid issue during calibration with
> > variable input shapes (#18632) (#18703)
> >
> >
> > Best regards,
> > Ciyong Chen
> >

[VOTE] Release Apache MXNet (incubating) version 1.7.0.rc1

[Chen, Ciyong]

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start 16th July 23:59:59 PST and close on 19th July 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc1

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc1

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Here's the changes comparing to 1.7.0.rc0:

  *   Revert "Fix memory leaks in Gluon (#18328) (#18358) (#18692)
  *   revise activations (#18700)
  *   Fix the monitor_callback invalid issue during calibration with variable 
input shapes (#18632) (#18703)


Best regards,
Ciyong Chen

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Hi MXNet Community,

I am canceling this vote as there's an issue which broke the Gluon CV Yolo and 
AutoGluon functionality.
Thanks Ziyi and Xingjian to root cause and fix the issue[1]. 

And the new code base will involve another two fixes (for Gluon activation[2] 
and monitor fix[3] respectively). 
I will update the artifacts and start a new vote for rc1 in the following days.

Thanks for everyone's help! Please let me know if there's any other issue with 
1.7.0.

[1] https://github.com/apache/incubator-mxnet/pull/18692
[2] https://github.com/apache/incubator-mxnet/pull/18700
[3] https://github.com/apache/incubator-mxnet/pull/18703

Thanks,
-Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Tuesday, July 14, 2020 10:13 AM
To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Thanks all for the effort to double check the performance status and the 
valuable comments, then let's not taking it as a blocker and moving forward 
with the 1.7.0 release process.

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Tuesday, July 14, 2020 4:41 AM
To: dev@mxnet.incubator.apache.org; lau...@apache.org; d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

That’s a good point, 1.6 did have a performance regression since it dropped 
MKLML to simplify build an fix licensing. 2.0 will have performance degradation 
too in favor of new features. Clearly the community is focusing on features 
rather than performance, at least we're consistent :-)

I would prefer we move forward with the 1.7.0 release and consider performance 
fixes for 1.7.1 (like we did for 1.3.1/1.4.1)

Sam

On 7/13/20, 1:36 PM, "Leonard Lausen"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



One of the selling points of MXNet is (or used to be) speed and having 
multiple
releases in series with speed regressions may not be acceptable to users 
that
adopted MXNet based on the speed advantage. Should we vote on a 1.7 Beta 
release
and only vote on 1.7 final release once the regressions have been fixed?

On Mon, 2020-07-13 at 19:33 +, Patrick Mu wrote:
> It happens only on CPU, and I did more runs and found that the runtime
> fluctuates very badly, but the average regression is ~10%.
>
>
> Through the previous benchmarks I also found some worse regression 
comparing
> 1.6 to 1.5 like inception inference on CPU and those regression was not
> caught.
>
> My 2-cent is it might not be a blocker for the release, and we can have 
room
> for improvement for upcoming 2.0 and 1.7.1 if necessary
>
> Ziyi
    >
> On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote:
> > Thanks Ziyi,
> >
> > May I know which platform did you notice the performance regression, 
CPU or
> > GPU? ~20% regression would be a large gap.
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Patrick Mu 
> > Sent: Monday, July 13, 2020 4:13 PM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
> >
> > Hi Ciyong,
> >
> > I have reverted the commit, and I am able to train Yolov3 with no 
problem.
> >
> > However I also noticed there is a ~20% regression in 1.7 comparing with 
1.6
> > in inference Yolov3 with Module API, so we are going to discuss 
tomorrow if
> > that would be an issue for 1.7.
> >
> > Thanks,
> > Ziyi
> >
> > On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote:
> > > Hi Ziyi, Xingjian,
> > >
> > > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > > I just did a quick try by reverting the
> > > https://github.com/apache/incubator-mxnet/pull/18358, then the 
behavior is
> > > same as 1.6.0 with the cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > >
> > > Considering there's many end-users using Gluon based API/models, and
> > > introducing a new patch to fix this issue could be risky, so I agree 
that
> > > reverting this PR (#18358) might be the best option for the 1.7.0 
release.
> > > But I'm considering is there any other test cases to cover this 
feature,
> > > which could be helpful to track this kind of code changes in future, 
or
> > > can you help to verify if this r

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Thanks all for the effort to double check the performance status and the 
valuable comments, then let's not taking it as a blocker and moving forward 
with the 1.7.0 release process.

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Tuesday, July 14, 2020 4:41 AM
To: dev@mxnet.incubator.apache.org; lau...@apache.org; d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

That’s a good point, 1.6 did have a performance regression since it dropped 
MKLML to simplify build an fix licensing. 2.0 will have performance degradation 
too in favor of new features. Clearly the community is focusing on features 
rather than performance, at least we're consistent :-)

I would prefer we move forward with the 1.7.0 release and consider performance 
fixes for 1.7.1 (like we did for 1.3.1/1.4.1)

Sam

On 7/13/20, 1:36 PM, "Leonard Lausen"  wrote:

CAUTION: This email originated from outside of the organization. Do not 
click links or open attachments unless you can confirm the sender and know the 
content is safe.



One of the selling points of MXNet is (or used to be) speed and having 
multiple
releases in series with speed regressions may not be acceptable to users 
that
adopted MXNet based on the speed advantage. Should we vote on a 1.7 Beta 
release
and only vote on 1.7 final release once the regressions have been fixed?

On Mon, 2020-07-13 at 19:33 +, Patrick Mu wrote:
> It happens only on CPU, and I did more runs and found that the runtime
> fluctuates very badly, but the average regression is ~10%.
>
>
> Through the previous benchmarks I also found some worse regression 
comparing
> 1.6 to 1.5 like inception inference on CPU and those regression was not
> caught.
>
> My 2-cent is it might not be a blocker for the release, and we can have 
room
> for improvement for upcoming 2.0 and 1.7.1 if necessary
>
> Ziyi
    >
> On 2020/07/13 08:40:32, "Chen, Ciyong"  wrote:
> > Thanks Ziyi,
> >
> > May I know which platform did you notice the performance regression, 
CPU or
> > GPU? ~20% regression would be a large gap.
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Patrick Mu 
> > Sent: Monday, July 13, 2020 4:13 PM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
> >
> > Hi Ciyong,
> >
> > I have reverted the commit, and I am able to train Yolov3 with no 
problem.
> >
> > However I also noticed there is a ~20% regression in 1.7 comparing with 
1.6
> > in inference Yolov3 with Module API, so we are going to discuss 
tomorrow if
> > that would be an issue for 1.7.
> >
> > Thanks,
> > Ziyi
> >
> > On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote:
> > > Hi Ziyi, Xingjian,
> > >
> > > Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> > > I just did a quick try by reverting the
> > > https://github.com/apache/incubator-mxnet/pull/18358, then the 
behavior is
> > > same as 1.6.0 with the cases in the gist (
> > > https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> > >
> > > Considering there's many end-users using Gluon based API/models, and
> > > introducing a new patch to fix this issue could be risky, so I agree 
that
> > > reverting this PR (#18358) might be the best option for the 1.7.0 
release.
> > > But I'm considering is there any other test cases to cover this 
feature,
> > > which could be helpful to track this kind of code changes in future, 
or
> > > can you help to verify if this revert do resolve the broken issue at 
your
> > > side?
> > >
> > > > Thus, the real issue is: Should we supporting pickling a Gluon 
Block? If
> > > > not, should we support combining multiprocessing.pool with the Gluon
> > > > Block?
> > > Seems it's more like a new feature for MXNet Gluon Block, probably we 
can
> > > make it available in the next patch/minor release?
> > >
> > > Thanks,
> > > -Ciyong
> > >
> > > -Original Message-
> > > From: Xingjian SHI 
> > > Sent: Saturday, July 11, 2020 4:27 AM
> > > To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> > > Subject: Re: [VOTE] Release Apache MXNet (incubating) version 
1.7.0.rc0
 

RE: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Thanks Ziyi,

May I know which platform did you notice the performance regression, CPU or 
GPU? ~20% regression would be a large gap.

Thanks,
-Ciyong

-Original Message-
From: Patrick Mu  
Sent: Monday, July 13, 2020 4:13 PM
To: d...@mxnet.apache.org
Subject: Re: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Hi Ciyong,

I have reverted the commit, and I am able to train Yolov3 with no problem.

However I also noticed there is a ~20% regression in 1.7 comparing with 1.6 in 
inference Yolov3 with Module API, so we are going to discuss tomorrow if that 
would be an issue for 1.7.

Thanks,
Ziyi

On 2020/07/13 02:19:28, "Chen, Ciyong"  wrote: 
> Hi Ziyi, Xingjian,
> 
> Thanks for reporting the issues from GluonCV/AutoGluon perspective.
> I just did a quick try by reverting the 
> https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is 
> same as 1.6.0 with the cases in the gist 
> (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> 
> Considering there's many end-users using Gluon based API/models, and 
> introducing a new patch to fix this issue could be risky, so I agree that 
> reverting this PR (#18358) might be the best option for the 1.7.0 release.
> But I'm considering is there any other test cases to cover this feature, 
> which could be helpful to track this kind of code changes in future, or can 
> you help to verify if this revert do resolve the broken issue at your side?
> 
> > Thus, the real issue is: Should we supporting pickling a Gluon Block? If 
> > not, should we support combining multiprocessing.pool with the Gluon Block?
> Seems it's more like a new feature for MXNet Gluon Block, probably we can 
> make it available in the next patch/minor release?
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Xingjian SHI  
> Sent: Saturday, July 11, 2020 4:27 AM
> To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
> Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Thanks Ziyi,
> 
> I've discovered the same issue when I'm trying to use AutoGluon with 1.7.0rc0 
> and would like to share my finding:
> 
> Basically, I don't think Gluon Block is designed to be pickleble. But 
> pickling do work for some cases in the old version:
> 
> I've included two cases in the gist 
> (https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).
> 
> - Case1: we construct a gluon block, hybridize it and feed one NDArray to 
> help initialize the block. After that, it will no longer be pickleble. 
> - Case2: we just construct a gluon block and it will be pickleble in 1.6.0, 
> but won't be pickleble in 1.7.0.
> 
> Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
> should we support combining multiprocessing.pool with the Gluon Block? For 
> reference, PyTorch supports pickling the nn.Module as shown in: 
> https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and also 
> in the doc 
> (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 
> 
> Best,
> Xingjian
> 
> 
> On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:
> 
> Hi Ciyong, 
> 
> I just discovered an issue with the 1.7, which causes the Yolo training 
> with latest Gluon CV Yolo to fail.
> 
> The PR that causes the failure is 
> https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
> blocks of Gluon to fix a memory leak issue.
> 
> Talked with Leonard, the author of the PR, and he said he found the root 
> cause, but patching that PR would modifies those Gluon basic blocks further, 
> which might be risky towards existing models and various customer models.
> 
> So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x 
> and 2.0, meaning that the 1.7 won't have memory usage optimized by that 
> feature.
> 
> I'd like to hear what you think about this issue.
> 
> Thanks,
> Ziyi
> 
> 
> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> > Hi Community,
> > 
> > I would like to call for action to test/validate/vote for the release 
> candidate (1.7.0.rc0)
> > As there's not any voting result during the scheduled time window, I 
> would like to extend the time windows to July 13, 23:59:59 PST.
> > Please prepare your time and provide feedback if you've tried with the 
> pre-release code bases, thanks!
> > 
> > Best regards,
> > Ciyong
> > 
> > -Original Message-
> > From: Chen, Ciyong  
> > Sent: Monday, July 6, 2020 10:48 PM
> > To: d...@mxnet.apache.org
> > Cc: Bob Paulin ;

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Thanks Macro for raising up the concern of license and Sheng for the 
clarification, the current release process is only targeting source release.

Regards,
-Ciyong

-Original Message-
From: Marco de Abreu  
Sent: Monday, July 13, 2020 4:51 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Okay, thanks for the clarification!

-Marco

On Sun, Jul 12, 2020, 5:58 PM Sheng Zha  wrote:

> Hi Marco,
>
> Since the license issues apply to binary distribution, we should still 
> be able to make official source releases.
>
> Regards,
> Sheng
>
> > On Jul 12, 2020, at 1:10 AM, Marco de Abreu 
> > 
> wrote:
> >
> > Are we in the position to make a release given that we have open 
> > license issues with the ipmc and Apache board? I want to avoid 
> > giving the impression that we are ignoring their requests - my 
> > current understanding is that we are non compliant.
> >
> > -Marco
> >
> >> On Sat, Jul 11, 2020, 9:46 AM Tong He  wrote:
> >>
> >> My +1 on the R binding.
> >>
> >> Tested with
> >>
> >> - Build from source
> >> - Install the R package and check it passed all tests.
> >>
> >>> On 2020/07/10 18:31:27, Patrick Mu  wrote:
> >>> Hi Ciyong,
> >>>
> >>> I just discovered an issue with the 1.7, which causes the Yolo 
> >>> training
> >> with latest Gluon CV Yolo to fail.
> >>>
> >>> The PR that causes the failure is
> >> https://github.com/apache/incubator-mxnet/pull/18358, which 
> >> modifies basic blocks of Gluon to fix a memory leak issue.
> >>>
> >>> Talked with Leonard, the author of the PR, and he said he found 
> >>> the
> root
> >> cause, but patching that PR would modifies those Gluon basic blocks 
> >> further, which might be risky towards existing models and various
> customer
> >> models.
> >>>
> >>> So my 2-cents is reverting this PR in 1.7, and try patching the PR 
> >>> in
> >> 1.x and 2.0, meaning that the 1.7 won't have memory usage optimized 
> >> by
> that
> >> feature.
> >>>
> >>> I'd like to hear what you think about this issue.
> >>>
> >>> Thanks,
> >>> Ziyi
> >>>
> >>>
> >>> On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote:
> >>>> Hi Community,
> >>>>
> >>>> I would like to call for action to test/validate/vote for the 
> >>>> release
> >> candidate (1.7.0.rc0)
> >>>> As there's not any voting result during the scheduled time 
> >>>> window, I
> >> would like to extend the time windows to July 13, 23:59:59 PST.
> >>>> Please prepare your time and provide feedback if you've tried 
> >>>> with the
> >> pre-release code bases, thanks!
> >>>>
> >>>> Best regards,
> >>>> Ciyong
> >>>>
> >>>> -----Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:48 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell 
> >>>> ;
> >> Jason Dai ; Markus Weimer ; 
> >> Michael Wall 
> >>>> Subject: RE: [VOTE] Release Apache MXNet (incubating) version
> 1.7.0.rc0
> >>>>
> >>>> For the language bindings and windows platform, may I have your
> >> support to help verify these features? Thanks!
> >>>>
> >>>> @lanking520 to help verify the Scala/Java @gigasquid to help 
> >>>> verify
> >> the Clojure
> >>>> @hetong007 to help verify the R
> >>>> @yajiedesign to help verify the windows platform
> >>>>
> >>>> Best regards,
> >>>> Ciyong Chen
> >>>>
> >>>> -Original Message-
> >>>> From: Chen, Ciyong 
> >>>> Sent: Monday, July 6, 2020 10:39 PM
> >>>> To: d...@mxnet.apache.org
> >>>> Cc: Bob Paulin ; Henri Yandell 
> >>>> ;
> >> Jason Dai ; Markus Weimer ; 
> >> Michael Wall 
> >>>> Subject: [VOTE] Release Apache MXNet (incubating) version 
> >>>> 1.7.0.rc0
> >>>>
> >>>> Dear MXNet community,
> >>>>
> >>>> This is the vote to release Apache MXNet (incubating) version 1.7.0.
>

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Hi Ziyi, Xingjian,

Thanks for reporting the issues from GluonCV/AutoGluon perspective.
I just did a quick try by reverting the 
https://github.com/apache/incubator-mxnet/pull/18358, then the behavior is same 
as 1.6.0 with the cases in the gist 
(https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).

Considering there's many end-users using Gluon based API/models, and 
introducing a new patch to fix this issue could be risky, so I agree that 
reverting this PR (#18358) might be the best option for the 1.7.0 release.
But I'm considering is there any other test cases to cover this feature, which 
could be helpful to track this kind of code changes in future, or can you help 
to verify if this revert do resolve the broken issue at your side?

> Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
> should we support combining multiprocessing.pool with the Gluon Block?
Seems it's more like a new feature for MXNet Gluon Block, probably we can make 
it available in the next patch/minor release?

Thanks,
-Ciyong

-Original Message-
From: Xingjian SHI  
Sent: Saturday, July 11, 2020 4:27 AM
To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Thanks Ziyi,

I've discovered the same issue when I'm trying to use AutoGluon with 1.7.0rc0 
and would like to share my finding:

Basically, I don't think Gluon Block is designed to be pickleble. But pickling 
do work for some cases in the old version:

I've included two cases in the gist 
(https://gist.github.com/sxjscience/944066c82e566f1b89b01fa226678890).

- Case1: we construct a gluon block, hybridize it and feed one NDArray to help 
initialize the block. After that, it will no longer be pickleble. 
- Case2: we just construct a gluon block and it will be pickleble in 1.6.0, but 
won't be pickleble in 1.7.0.

Thus, the real issue is: Should we supporting pickling a Gluon Block? If not, 
should we support combining multiprocessing.pool with the Gluon Block? For 
reference, PyTorch supports pickling the nn.Module as shown in: 
https://gist.github.com/sxjscience/90b812a66d445e759c55eedc3ef93668 and also in 
the doc (https://pytorch.org/tutorials/beginner/saving_loading_models.html). 

Best,
Xingjian


On 7/10/20, 11:31 AM, "Patrick Mu"  wrote:

Hi Ciyong, 

I just discovered an issue with the 1.7, which causes the Yolo training 
with latest Gluon CV Yolo to fail.

The PR that causes the failure is 
https://github.com/apache/incubator-mxnet/pull/18358, which modifies  basic 
blocks of Gluon to fix a memory leak issue.

Talked with Leonard, the author of the PR, and he said he found the root 
cause, but patching that PR would modifies those Gluon basic blocks further, 
which might be risky towards existing models and various customer models.

So my 2-cents is reverting this PR in 1.7, and try patching the PR in 1.x 
and 2.0, meaning that the 1.7 won't have memory usage optimized by that feature.

I'd like to hear what you think about this issue.

Thanks,
Ziyi


On 2020/07/10 06:18:02, "Chen, Ciyong"  wrote: 
> Hi Community,
> 
> I would like to call for action to test/validate/vote for the release 
candidate (1.7.0.rc0)
> As there's not any voting result during the scheduled time window, I 
would like to extend the time windows to July 13, 23:59:59 PST.
> Please prepare your time and provide feedback if you've tried with the 
pre-release code bases, thanks!
> 
> Best regards,
> Ciyong
    > 
> -Original Message-
> From: Chen, Ciyong  
> Sent: Monday, July 6, 2020 10:48 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; 
Jason Dai ; Markus Weimer ; Michael 
Wall 
> Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> For the language bindings and windows platform, may I have your support 
to help verify these features? Thanks!
> 
> @lanking520 to help verify the Scala/Java @gigasquid to help verify the 
Clojure
> @hetong007 to help verify the R
> @yajiedesign to help verify the windows platform
> 
    > Best regards,
> Ciyong Chen
> 
> -Original Message-
> From: Chen, Ciyong 
> Sent: Monday, July 6, 2020 10:39 PM
> To: d...@mxnet.apache.org
> Cc: Bob Paulin ; Henri Yandell ; 
Jason Dai ; Markus Weimer ; Michael 
Wall 
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0
> 
> Dear MXNet community,
> 
> This is the vote to release Apache MXNet (incubating) version 1.7.0. 
Voting will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.
> 
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes
> 
> Link

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Hi Community,

I would like to call for action to test/validate/vote for the release candidate 
(1.7.0.rc0)
As there's not any voting result during the scheduled time window, I would like 
to extend the time windows to July 13, 23:59:59 PST.
Please prepare your time and provide feedback if you've tried with the 
pre-release code bases, thanks!

Best regards,
Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Monday, July 6, 2020 10:48 PM
To: d...@mxnet.apache.org
Cc: Bob Paulin ; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

For the language bindings and windows platform, may I have your support to help 
verify these features? Thanks!

@lanking520 to help verify the Scala/Java @gigasquid to help verify the Clojure
@hetong007 to help verify the R
@yajiedesign to help verify the windows platform

Best regards,
Ciyong Chen

-Original Message-
From: Chen, Ciyong 
Sent: Monday, July 6, 2020 10:39 PM
To: d...@mxnet.apache.org
Cc: Bob Paulin ; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Additional notes:

  *   There was an issue and discussion[1] regarding on a few numpy operators 
failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.

[1] https://github.com/apache/incubator-mxnet/issues/18600

Best regards,
Ciyong Chen

RE: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

For the language bindings and windows platform, may I have your support to help 
verify these features? Thanks!

@lanking520 to help verify the Scala/Java
@gigasquid to help verify the Clojure
@hetong007 to help verify the R
@yajiedesign to help verify the windows platform

Best regards,
Ciyong Chen

-Original Message-
From: Chen, Ciyong  
Sent: Monday, July 6, 2020 10:39 PM
To: d...@mxnet.apache.org
Cc: Bob Paulin ; Henri Yandell ; Jason Dai 
; Markus Weimer ; Michael Wall 

Subject: [VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0<https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0/>

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Additional notes:

  *   There was an issue and discussion[1] regarding on a few numpy operators 
failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.

[1] https://github.com/apache/incubator-mxnet/issues/18600

Best regards,
Ciyong Chen

[VOTE] Release Apache MXNet (incubating) version 1.7.0.rc0

[Chen, Ciyong]

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version 1.7.0. Voting 
will start July 6, 23:59:59 PST and close on July 9, 23:59:59 PST.

Link to release notes:
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+notes

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/1.7.0.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.7.0.rc0

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)

Additional notes:

  *   There was an issue and discussion[1] regarding on a few numpy operators 
failed due to numpy 1.19.0 released on Jun 20, 2020, which exists in all 
branches (works with numpy <= 1.18.5). As numpy operator is still an 
experimental feature in 1.7.0 release and mainly targeting in MXNet 2.0 
release, so I decided to not block the voting and instead let the Community 
decide whether this is a blocker for the release.

[1] https://github.com/apache/incubator-mxnet/issues/18600

Best regards,
Ciyong Chen

RE: Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi dev,

Thanks everyone for your great support to backport the necessary fixes into 
1.7.x and identify & remove the (potential) block issue.
Today we've tagged the 1.7.0.rc0 for the upcoming 1.7.0 release, thanks for the 
help from @Tao.

The artifacts will be uploaded later, and we'll move forward with the rest of 
release process.
Again, thanks for your patience.

Thanks,
-Ciyong

-Original Message-
From: sandeep krishnamurthy  
Sent: Tuesday, June 30, 2020 11:07 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: Updates for 1.7.0 minor release

I agree with marking numpy operators being marked as experimental and going 
with v1.7 given numpy is still in progress and mainly targeted beginning v2.0. 
And, v1.7 has several significant features such as accelerator APIs.

On Mon, 29 Jun 2020, 7:51 pm Chen, Ciyong,  wrote:

> Hi Chai,
>
> We've finalized the multiple license header issue and merged the 
> necessary modification according to the dev@ discussion result.
> But @Leonard reported a numpy version issue in [1], which is about the 
> UT failure of numpy operators, as well as some other numpy issue in [2].
> Which is under discussion so far.
>
> @dev
> As the numpy operator is still in active development, there could be 
> more defects/bugs as including more new functionalities/features in 
> v1.7. Thus it's uncertain about how longer it will take to backport 
> these numpy bug fixes/features from master to v1.7, I suggest to mark 
> numpy operator as experimental feature in v1.7 release, and decide a 
> cut off day (24h or 48h) to include the fixes that are available, and 
> moving the 1.7 release process forward, what do you think?
>
> Thanks,
> -Ciyong
> [1]
> https://github.com/apache/incubator-mxnet/issues/18600#issuecomment-64
> 9712182 [2] https://github.com/apache/incubator-mxnet/issues/18641
>
> -Original Message-
> From: Chaitanya Bapat 
> Sent: Tuesday, June 30, 2020 1:45 AM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: Updates for 1.7.0 minor release
>
> Hey Ciyong,
>
> Any update from the ASF mentors/legal team re: multiple license header 
> issue?
> I can see the PR for checking Valid license header merged:
> https://github.com/apache/incubator-mxnet/pull/18478
> So if we get the multiple license header issue fixed, we can get 1.7.0 
> release going..
>
> Are we blocked somewhere?
> Thanks
> Chai
>
>
> On Sat, 13 Jun 2020 at 06:32, Chen, Ciyong  wrote:
>
> > Hi Leonard,
> >
> > Thanks for your confirmation on the build issue. As it's not a 
> > blocker for
> > 1.7 release now, then we can consider to backport the fix to 1.7.x 
> > branch when it's ready.
> > The only remaining item is how to deal with the multiple license 
> > header now, thank you for helping on this
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Leonard Lausen 
> > Sent: Saturday, June 13, 2020 1:10 AM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: Updates for 1.7.0 minor release
> >
> > Thank you Ciyong. After further investigation, the build issue is 
> > not as severe as initially claimed on Github. I checked the 
> > high-water memory usage during single-process build: It's 2.7GB on 
> > master. On 1.7 release, high-level usage is 2.2GB. This is much more 
> > acceptable than the previously claimed >16GB usage and thus not a 
> > blocking issue from my perspective. I'll later also report the numbers for 
> > 1.5 and 1.6.
> >
> > Fixing the respective implementations to be more compiler-friendly 
> > would still be good.
> >
> > Looking at the parallel-build high-level memory usage on a 96 core 
> > machine, I saw a 45% memory usage increase during build from 1.5 to 1.7.
> >
> > Best regards
> > Leonard
> >
> >
> > On Fri, 2020-06-12 at 02:09 +, Chen, Ciyong wrote:
> > > Hi Chai,
> > >
> > > Sorry for the late update.
> > >
> > > Recently, several bug fixes [4] including numpy operator/batchnorm 
> > > gradient/LSTM CPU gradient/CI/CD/license issues were back-ported 
> > > into
> > v1.7.x.
> > > So far, there's one build issue and two license issues being tracked.
> > > 1) build issue #18501 (It costs over 16GB memory to 
> > > compile indexing_op.o), which @leezu stated it's a blocker for the 
> > > release[1].
> > > 2) license issue: multiple license header issue[2] is 
> > > under discussion; no valid apache license header issue[3] is 
> > > identified, and I'm working on the PR as @szha suggested.
> > >
> > > If the 

RE: Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi Chai,

We've finalized the multiple license header issue and merged the necessary 
modification according to the dev@ discussion result.
But @Leonard reported a numpy version issue in [1], which is about the UT 
failure of numpy operators, as well as some other numpy issue in [2].
Which is under discussion so far.

@dev
As the numpy operator is still in active development, there could be more 
defects/bugs as including more new functionalities/features in v1.7. Thus it's 
uncertain about how longer it will take to backport these numpy bug 
fixes/features from master to v1.7, I suggest to mark numpy operator as 
experimental feature in v1.7 release, and decide a cut off day (24h or 48h) to 
include the fixes that are available, and moving the 1.7 release process 
forward, what do you think?

Thanks,
-Ciyong
[1] 
https://github.com/apache/incubator-mxnet/issues/18600#issuecomment-649712182
[2] https://github.com/apache/incubator-mxnet/issues/18641

-Original Message-
From: Chaitanya Bapat  
Sent: Tuesday, June 30, 2020 1:45 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: Updates for 1.7.0 minor release

Hey Ciyong,

Any update from the ASF mentors/legal team re: multiple license header issue?
I can see the PR for checking Valid license header merged:
https://github.com/apache/incubator-mxnet/pull/18478
So if we get the multiple license header issue fixed, we can get 1.7.0 release 
going..

Are we blocked somewhere?
Thanks
Chai


On Sat, 13 Jun 2020 at 06:32, Chen, Ciyong  wrote:

> Hi Leonard,
>
> Thanks for your confirmation on the build issue. As it's not a blocker 
> for
> 1.7 release now, then we can consider to backport the fix to 1.7.x 
> branch when it's ready.
> The only remaining item is how to deal with the multiple license 
> header now, thank you for helping on this
>
> Thanks,
> -Ciyong
>
> -Original Message-
> From: Leonard Lausen 
> Sent: Saturday, June 13, 2020 1:10 AM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: Updates for 1.7.0 minor release
>
> Thank you Ciyong. After further investigation, the build issue is not 
> as severe as initially claimed on Github. I checked the high-water 
> memory usage during single-process build: It's 2.7GB on master. On 1.7 
> release, high-level usage is 2.2GB. This is much more acceptable than 
> the previously claimed >16GB usage and thus not a blocking issue from 
> my perspective. I'll later also report the numbers for 1.5 and 1.6.
>
> Fixing the respective implementations to be more compiler-friendly 
> would still be good.
>
> Looking at the parallel-build high-level memory usage on a 96 core 
> machine, I saw a 45% memory usage increase during build from 1.5 to 1.7.
>
> Best regards
> Leonard
>
>
> On Fri, 2020-06-12 at 02:09 +, Chen, Ciyong wrote:
> > Hi Chai,
> >
> > Sorry for the late update.
> >
> > Recently, several bug fixes [4] including numpy operator/batchnorm 
> > gradient/LSTM CPU gradient/CI/CD/license issues were back-ported 
> > into
> v1.7.x.
> > So far, there's one build issue and two license issues being tracked.
> > 1) build issue #18501 (It costs over 16GB memory to compile 
> > indexing_op.o), which @leezu stated it's a blocker for the release[1].
> > 2) license issue: multiple license header issue[2] is under 
> > discussion; no valid apache license header issue[3] is identified, 
> > and I'm working on the PR as @szha suggested.
> >
> > If the community can help to expedite the item of [1] and [2], it 
> > will be great helpful.
> > Once we've completed the above items and no more other critical 
> > issues, it's ok to cut the rc0.
> >
> > Thanks for your patients.
> >
> > Thanks,
> > -Ciyong
> >
> > [1]
> > https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-
> > 64
> > 2785535
> > [2]
> > https://github.com/apache/incubator-mxnet/issues/17329#issuecomment-
> > 64
> > 1311199
> > [3]
> > https://github.com/apache/incubator-mxnet/pull/18478#issuecomment-64
> > 24
> > 62904
> > [4] PR list:
> > #18358/#18339/#18311/#18352/#18456/#18316/#18482/#18502/#18517/#1846
> > 4
> >
> >
> >
> > -Original Message-
> > From: Chaitanya Bapat 
> > Sent: Friday, June 12, 2020 1:34 AM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: RE: Updates for 1.7.0 minor release
> >
> > Hey Ciyong,
> >
> > Since the last discussion, the GPU memory regression PR has been
> reverted.
> > Is there any update for when the rc0 for 1.7 will be cut?
> > Can the community help expedite the process in any way?
> >
> > Thanks
> 

RE: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: [MENTORS] PPMC case-by-case decision for major modifications of third-party work guidance

[Chen, Ciyong]

Thanks Leonard for the confirmation, I will update the related files based on 
the consensus. 

Regards,
-Ciyong

-Original Message-
From: Leonard Lausen  
Sent: Wednesday, June 24, 2020 2:24 AM
To: dev@mxnet.incubator.apache.org; gene...@incubator.apache.org
Subject: Re: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: 
[MENTORS] PPMC case-by-case decision for major modifications of third-party 
work guidance

Hi Ciyong,

the consensus passed, so we should proceed according to the consensus.

Thank you
Leonard

On Tue, 2020-06-23 at 09:04 +, Chen, Ciyong wrote:
> Hi all,
> 
> I'm wondering if there's any further concerns for this "72 hours lazy 
> consensus"?
> Shall we continue with the option of "I believe PPMC would prefer to 
> put the ASF header on top of the file (ie. 2 headers)"
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Leonard Lausen 
> Sent: Tuesday, June 16, 2020 7:06 AM
> To: dev@mxnet.incubator.apache.org; gene...@incubator.apache.org
> Subject: Re: [DISCUSS] When to add Apache Headers to Third Party Code 
> [WAS]
> Re: [MENTORS] PPMC case-by-case decision for major modifications of 
> third- party work guidance
> 
> Thank you everyone for your valuable advice.
> 
> > so if you did want to avoid including the license in your releases 
> > you would either need to rely on the file as an external dependency 
> > or completely reimplement the functionality not deriving it from 
> > this file.
> 
> Including the BSD-3 style license in releases wouldn't be a problem, 
> as it's compatible with Apache License 2. As there are substantial 
> changes, I believe PPMC would prefer to put the ASF header on top of 
> the file (ie. 2 headers) [72 hours lazy consensus if there are no 
> concerns]. We still need to declare all the numpy einsum derived files 
> in the LICENSE and fix the inconsistency that ASF header was removed 
> in src/operator/numpy/np_einsum_op-inl.h but remains in 
> src/operator/numpy/np_einsum_path_op-inl.h
> 
> Related: As PPMC strives to provide partial API compatibility with 
> NumPy in MXNet 2 based on the NumPy Array Function Protocol [1], could 
> you clarify if these MXNet operators should be considered derived from 
> NumPy (thus warranting the BSD-3 style license headers) solely based 
> on integrating with the NumPy API and providing compatible operators? 
> Or only (as in the einsum case above), if the actual implementation 
> was derived from NumPy's implementation. I believe it's the latter, but 
> please clarify if that's wrong.
> 
> Should ASF update the "Do not add the standard Apache License header 
> to the top of third-party source files." at [2]? This sentence was the 
> motivation to open this discussion thread, and according to the 
> current consensus here is "incomplete". How about adding an "unless 
> the third-party source file contains major modifications by ASF" to clarify?
> 
> Thank you
> Leonard
> 
> [1]: https://numpy.org/neps/nep-0018-array-function-protocol.html
> [2]: https://www.apache.org/legal/src-headers.html#3party
> 
> On Mon, 2020-06-15 at 09:36 -0400, John D. Ament wrote:
> > On Sat, Jun 13, 2020 at 2:19 PM Bob Paulin  wrote:
> > 
> > > Hi,
> > > 
> > > I agree there does not appear to be consensus on when it's 
> > > appropriate to add Apache License Headers to Third Party code 
> > > across projects.  Here is Justin's email that request the Apache 
> > > Headers removed [1]
> > > 
> > > 
> > > 
> > > - file copyright  NumPy Developers [6] this file look to 
> > > incorrectly have an ASF header on it 
> > > 6. ./src/operator/numpy/np_einsum_path_op-inl.h
> > > 
> > > 
> > > We want to make the choice that will be most sustainable for the 
> > > project and most correct for the situation.
> > > 
> > > Based on the emails I linked in the prior email it does seem like 
> > > the cases where dual headers are appropriate is when there are 
> > > Major Modifications.  In the case of
> > > 
> > > np_einsum_path_op-inl.h
> > > 
> > > The file is derived from the implementation in Numpy [2].  If the 
> > > implementation in Numpy changes will this file change?  If so then 
> > > the community will be tasked with continuing to re-port the 
> > > changes over that is always based on Numpy so it may be more 
> > > appropriate to just keep the Numpy license.
> > > 
> > > Will MXNet likely evolve this file in a way that it's no longer 
> > > resembles the Numpy implementation (Maj

RE: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: [MENTORS] PPMC case-by-case decision for major modifications of third-party work guidance

[Chen, Ciyong]

Hi all,

I'm wondering if there's any further concerns for this "72 hours lazy 
consensus"?
Shall we continue with the option of "I believe PPMC would prefer to put the 
ASF header on top of the file (ie. 2 headers)"

Thanks,
-Ciyong

-Original Message-
From: Leonard Lausen  
Sent: Tuesday, June 16, 2020 7:06 AM
To: dev@mxnet.incubator.apache.org; gene...@incubator.apache.org
Subject: Re: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: 
[MENTORS] PPMC case-by-case decision for major modifications of third-party 
work guidance

Thank you everyone for your valuable advice.

> so if you did want to avoid including the license in your releases you 
> would either need to rely on the file as an external dependency or 
> completely reimplement the functionality not deriving it from this 
> file.

Including the BSD-3 style license in releases wouldn't be a problem, as it's 
compatible with Apache License 2. As there are substantial changes, I believe 
PPMC would prefer to put the ASF header on top of the file (ie. 2 headers) [72 
hours lazy consensus if there are no concerns]. We still need to declare all 
the numpy einsum derived files in the LICENSE and fix the inconsistency that 
ASF header was removed in src/operator/numpy/np_einsum_op-inl.h but remains in 
src/operator/numpy/np_einsum_path_op-inl.h

Related: As PPMC strives to provide partial API compatibility with NumPy in 
MXNet 2 based on the NumPy Array Function Protocol [1], could you clarify if 
these MXNet operators should be considered derived from NumPy (thus warranting 
the BSD-3 style license headers) solely based on integrating with the NumPy API 
and providing compatible operators? Or only (as in the einsum case above), if 
the actual implementation was derived from NumPy's implementation. I believe 
it's the latter, but please clarify if that's wrong.

Should ASF update the "Do not add the standard Apache License header to the top 
of third-party source files." at [2]? This sentence was the motivation to open 
this discussion thread, and according to the current consensus here is 
"incomplete". How about adding an "unless the third-party source file contains 
major modifications by ASF" to clarify?

Thank you
Leonard

[1]: https://numpy.org/neps/nep-0018-array-function-protocol.html
[2]: https://www.apache.org/legal/src-headers.html#3party

On Mon, 2020-06-15 at 09:36 -0400, John D. Ament wrote:
> On Sat, Jun 13, 2020 at 2:19 PM Bob Paulin  wrote:
> 
> > Hi,
> > 
> > I agree there does not appear to be consensus on when it's 
> > appropriate to add Apache License Headers to Third Party code across 
> > projects.  Here is Justin's email that request the Apache Headers 
> > removed [1]
> > 
> > 
> > 
> > - file copyright  NumPy Developers [6] this file look to incorrectly 
> > have an ASF header on it 
> > 6. ./src/operator/numpy/np_einsum_path_op-inl.h
> > 
> > 
> > We want to make the choice that will be most sustainable for the 
> > project and most correct for the situation.
> > 
> > Based on the emails I linked in the prior email it does seem like 
> > the cases where dual headers are appropriate is when there are Major 
> > Modifications.  In the case of
> > 
> > np_einsum_path_op-inl.h
> > 
> > The file is derived from the implementation in Numpy [2].  If the 
> > implementation in Numpy changes will this file change?  If so then 
> > the community will be tasked with continuing to re-port the changes 
> > over that is always based on Numpy so it may be more appropriate to 
> > just keep the Numpy license.
> > 
> > Will MXNet likely evolve this file in a way that it's no longer 
> > resembles the Numpy implementation (Major Modification)?  If so it 
> > may be better to keep the Apache Header as going forward the file 
> > will represent the work of the MXNet community not that of Numpy.
> > 
> 
> Keeping the (what appears to be) BSD-3 style license is perfectly fine 
> and is in fact what the NumPy license says to do.  We would only 
> change the license from the NumPy license to ALv2 if an SGA or ICLA is 
> received from all contributors historically on this file.  No matter 
> how drastic of modifications the MxNet community makes to it, it would 
> always be NumPy licensed; so if you did want to avoid including the 
> license in your releases you would either need to rely on the file as 
> an external dependency or completely reimplement the functionality not 
> deriving it from this file.  Whether or not the MxNet community 
> imports upstream changes or not is up to them, but either way you have a 
> derived work here.
> 
> John
> 
> 
> > Hopefully the above helps clarify my perspective on how to determine 
> > case by case.  I don't see the dual license state as the simpler 
> > case in all situations.  I don't believe you would have to get the 
> > original committer to relicense the file to you in order to remove 
> > the additional license.  I believe the PPMC has all the authority it 
> > needs to 

RE: [Lazy consensus] Turn on Branch Protection for Release branches and v1.x

[Chen, Ciyong]

+1, as v1.x and release branches are still moving forward

-Ciyong

-Original Message-
From: Tao Lv  
Sent: Monday, June 22, 2020 12:55 PM
To: dev@mxnet.incubator.apache.org
Subject: Re: [Lazy consensus] Turn on Branch Protection for Release branches 
and v1.x

+1

On Sat, Jun 20, 2020 at 7:01 AM Sheng Zha  wrote:

> Hi,
>
> I'd like to propose that we turn on branch protection and CI check 
> enforcement that's consistent with our master branch for all release 
> branches and v1.x. So far we only have branch protection and enforce 
> CI checks on master branch, while allowing force push without CI 
> checks on release branches.
>
> Feel free to reply if you have questions or concerns, or otherwise I 
> will wait for 96hrs before proceeding with changes.
>
> -sz
>

RE: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: [MENTORS] PPMC case-by-case decision for major modifications of third-party work guidance

[Chen, Ciyong]

Thanks a lot for your valuable input Bob, John, Justin, Leonard.

As it’s still not finalized on how to handle such dual license issue from the 
discussion.
In addition, Justin stated that converting the code from one program language 
to another one should **NOT** be considered as a major modification.
And based on the statement #3 and #4 from 
https://www.apache.org/legal/src-headers.html#3party
> 3.Do not add the standard Apache License header to the top of third-party 
> source files.
> 4.Minor modifications/additions to third-party source files should typically 
> be licensed under the same terms as the rest of the rest of the third-party 
> source for convenience.

So it seems more appropriate to remove the ASF header and just keep the Numpy 
license header and claim it at the top level LICENSE, or do we need to vote on 
the two options as Bob stated below, thanks!
>1) Numpy License Headers Only
> 2) Apache Header with Numpy License Header (keep the license header as is now)

Best Regards,
-Ciyong

From: Bob Paulin 
Sent: Monday, June 15, 2020 11:38 PM
To: dev@mxnet.incubator.apache.org; Chen, Ciyong ; 
lau...@apache.org; d...@mxnet.apache.org; gene...@incubator.apache.org
Subject: Re: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: 
[MENTORS] PPMC case-by-case decision for major modifications of third-party 
work guidance


Hi,

I should be more clear.  The 2 options in the case below is

1) Numpy License Headers Only

2) Apache Header with Numpy License Header

Re-reading my original reply does sound like I'm saying the Numpy license 
should be removed in the case for the Apache License Headers from the file.  
This was not my intent.  John states it more clearly in his reply that removal 
of the Numpy License requires additional steps.



- Bob
On 6/15/2020 3:05 AM, Chen, Ciyong wrote:

Hi Bob, Leonard,



Thanks for the elaboration/guideline on the dual license issue.

May I have the conclusion as below based on Bob’s direction/suggestion:





  *   If there’s no any different opinion or objection,  keep either origin 
Numpy license or ASF license but not dual, which depends on how MXNet’s source 
file evolves when the origin Numpy files changes? And the PPMC has all the 
authority to change the file like removing the additional license if needed.



Please correct me if I mis-understand anything, and help to determine the best 
appropriate way to handle such case. Thanks!



Best Regards,

-Ciyong



From: Bob Paulin <mailto:b...@bobpaulin.com>

Sent: Sunday, June 14, 2020 2:20 AM

To: lau...@apache.org<mailto:lau...@apache.org>; 
d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>; 
gene...@incubator.apache.org<mailto:gene...@incubator.apache.org>

Subject: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: 
[MENTORS] PPMC case-by-case decision for major modifications of third-party 
work guidance





Hi,



I agree there does not appear to be consensus on when it's appropriate to add 
Apache License Headers to Third Party code across projects.  Here is Justin's 
email that request the Apache Headers removed [1]







- file copyright  NumPy Developers [6] this file look to incorrectly have an 
ASF header on it







6. ./src/operator/numpy/np_einsum_path_op-inl.h







We want to make the choice that will be most sustainable for the project and 
most correct for the situation.



Based on the emails I linked in the prior email it does seem like the cases 
where dual headers are appropriate is when there are Major Modifications.  In 
the case of



np_einsum_path_op-inl.h



The file is derived from the implementation in Numpy [2].  If the 
implementation in Numpy changes will this file change?  If so then the 
community will be tasked with continuing to re-port the changes over that is 
always based on Numpy so it may be more appropriate to just keep the Numpy 
license.



Will MXNet likely evolve this file in a way that it's no longer resembles the 
Numpy implementation (Major Modification)?  If so it may be better to keep the 
Apache Header as going forward the file will represent the work of the MXNet 
community not that of Numpy.



Hopefully the above helps clarify my perspective on how to determine case by 
case.  I don't see the dual license state as the simpler case in all 
situations.  I don't believe you would have to get the original committer to 
relicense the file to you in order to remove the additional license.  I believe 
the PPMC has all the authority it needs to change the file.  I'd be interested 
to hear if this is a position that the rest of the Mentors/Incubator agree 
with.  I know Hen has been involved in some of the conversations in support of 
dual licenses.  Has this ever required escalation to an actual Lawyer in Legal? 
 Or have these determinations been low enough risk that we are comfortable with 
our PMC making best effort decisions based on the ASF guidelines?

RE: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: [MENTORS] PPMC case-by-case decision for major modifications of third-party work guidance

[Chen, Ciyong]

Hi Bob, Leonard,

Thanks for the elaboration/guideline on the dual license issue.
May I have the conclusion as below based on Bob’s direction/suggestion:


  *   If there’s no any different opinion or objection,  keep either origin 
Numpy license or ASF license but not dual, which depends on how MXNet’s source 
file evolves when the origin Numpy files changes? And the PPMC has all the 
authority to change the file like removing the additional license if needed.

Please correct me if I mis-understand anything, and help to determine the best 
appropriate way to handle such case. Thanks!

Best Regards,
-Ciyong

From: Bob Paulin 
Sent: Sunday, June 14, 2020 2:20 AM
To: lau...@apache.org; d...@mxnet.apache.org; gene...@incubator.apache.org
Subject: [DISCUSS] When to add Apache Headers to Third Party Code [WAS] Re: 
[MENTORS] PPMC case-by-case decision for major modifications of third-party 
work guidance


Hi,

I agree there does not appear to be consensus on when it's appropriate to add 
Apache License Headers to Third Party code across projects.  Here is Justin's 
email that request the Apache Headers removed [1]



- file copyright  NumPy Developers [6] this file look to incorrectly have an 
ASF header on it



6. ./src/operator/numpy/np_einsum_path_op-inl.h



We want to make the choice that will be most sustainable for the project and 
most correct for the situation.

Based on the emails I linked in the prior email it does seem like the cases 
where dual headers are appropriate is when there are Major Modifications.  In 
the case of

np_einsum_path_op-inl.h

The file is derived from the implementation in Numpy [2].  If the 
implementation in Numpy changes will this file change?  If so then the 
community will be tasked with continuing to re-port the changes over that is 
always based on Numpy so it may be more appropriate to just keep the Numpy 
license.

Will MXNet likely evolve this file in a way that it's no longer resembles the 
Numpy implementation (Major Modification)?  If so it may be better to keep the 
Apache Header as going forward the file will represent the work of the MXNet 
community not that of Numpy.

Hopefully the above helps clarify my perspective on how to determine case by 
case.  I don't see the dual license state as the simpler case in all 
situations.  I don't believe you would have to get the original committer to 
relicense the file to you in order to remove the additional license.  I believe 
the PPMC has all the authority it needs to change the file.  I'd be interested 
to hear if this is a position that the rest of the Mentors/Incubator agree 
with.  I know Hen has been involved in some of the conversations in support of 
dual licenses.  Has this ever required escalation to an actual Lawyer in Legal? 
 Or have these determinations been low enough risk that we are comfortable with 
our PMC making best effort decisions based on the ASF guidelines?



- Bob



[1] 
https://lists.apache.org/thread.html/rb83ff64bdac464df2f0cf2fe8fb4c6b9d3b8fa62b645763dc606045f%40%3Cgeneral.incubator.apache.org%3E

[2] https://github.com/numpy/numpy/blob/master/numpy/core/einsumfunc.py
On 6/12/2020 7:20 PM, Leonard Lausen wrote:

Thank you Bob for the elaboration. PPMC would like to minimize complexity,

that's why we ask for your recommendation.



If it's easiest to just keep the original license header, we can do that. Do we

need the contributor to re-license their contribution, or is the contribution

already available under both licenses as both license headers were included by

the contributor and the ASF header can simply be deleted?



Reading through the threads you referenced, there does not seem to be a strong

consensus in the ASF about how to handle this situation. For example, quoting

Roman Shaposhnik [2] in support of just putting 2 License Headers for

simplicity:



Hm. This is tricky, now that I re-read the language of the ASF license

header I'm not sure anymore. I *think* the language there should allow

you to slap said header on a compatible license code.



Besides, the alternative is much messier: every time somebody touches

that file he/she needs to decide whether it is time for an ASF header

or not.



I *think* (but I'd love for old-timers to chime in and correct me) that #3-5

were written from though-shall-not-fork-communities perspective.

Can we follow this approach (keep 2 License headers) for simplicity (assuming

removal of ASF header will require extra steps)?



With respect to einsumfunc.py [5] vs np_einsum_op.cc [6] if this is in

fact a port where the behavior was copied/derived directly from numpy I

could see that as supporting Justin's case that the Apache header should

be removed.  However that is just my opinion.

Which email of Justin are you referring to?



Best regards

Leonard





[1]: http://www.apache.org/legal/src-headers.html#purpose

[2]:

RE: Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi Leonard,

Thanks for your confirmation on the build issue. As it's not a blocker for 1.7 
release now, then we can consider to backport the fix to 1.7.x branch when it's 
ready.
The only remaining item is how to deal with the multiple license header now, 
thank you for helping on this

Thanks,
-Ciyong

-Original Message-
From: Leonard Lausen  
Sent: Saturday, June 13, 2020 1:10 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: Updates for 1.7.0 minor release

Thank you Ciyong. After further investigation, the build issue is not as severe 
as initially claimed on Github. I checked the high-water memory usage during 
single-process build: It's 2.7GB on master. On 1.7 release, high-level usage is 
2.2GB. This is much more acceptable than the previously claimed >16GB usage and 
thus not a blocking issue from my perspective. I'll later also report the 
numbers for 1.5 and 1.6.

Fixing the respective implementations to be more compiler-friendly would still 
be good.

Looking at the parallel-build high-level memory usage on a 96 core machine, I 
saw a 45% memory usage increase during build from 1.5 to 1.7.

Best regards
Leonard


On Fri, 2020-06-12 at 02:09 +0000, Chen, Ciyong wrote:
> Hi Chai,
> 
> Sorry for the late update.
> 
> Recently, several bug fixes [4] including numpy operator/batchnorm 
> gradient/LSTM CPU gradient/CI/CD/license issues were back-ported into v1.7.x.
> So far, there's one build issue and two license issues being tracked.
> 1) build issue #18501 (It costs over 16GB memory to compile 
> indexing_op.o), which @leezu stated it's a blocker for the release[1].
> 2) license issue: multiple license header issue[2] is under 
> discussion; no valid apache license header issue[3] is identified, and 
> I'm working on the PR as @szha suggested.
> 
> If the community can help to expedite the item of [1] and [2], it will 
> be great helpful.
> Once we've completed the above items and no more other critical 
> issues, it's ok to cut the rc0.
> 
> Thanks for your patients.
> 
> Thanks,
> -Ciyong
> 
> [1]
> https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-64
> 2785535
> [2]
> https://github.com/apache/incubator-mxnet/issues/17329#issuecomment-64
> 1311199
> [3]
> https://github.com/apache/incubator-mxnet/pull/18478#issuecomment-6424
> 62904
> [4] PR list:
> #18358/#18339/#18311/#18352/#18456/#18316/#18482/#18502/#18517/#18464
> 
> 
> 
> -Original Message-
> From: Chaitanya Bapat 
> Sent: Friday, June 12, 2020 1:34 AM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: RE: Updates for 1.7.0 minor release
> 
> Hey Ciyong,
> 
> Since the last discussion, the GPU memory regression PR has been reverted.
> Is there any update for when the rc0 for 1.7 will be cut?
> Can the community help expedite the process in any way?
> 
> Thanks
> Chai
> 
> On Wed, 13 May 2020 at 18:28, Chen, Ciyong  wrote:
> 
> > Hi Ziyi,
> > 
> > Thanks for reaching me for the known/found issue in the upcoming 
> > release, let's fix all these potential issues before dropping the 
> > rc0 tag 
> > I'll ask help from Tao to merge the PR.
> > 
> > Thanks,
> > -Ciyong
> > 
> > -Original Message-
> > From: Patrick Mu 
> > Sent: Thursday, May 14, 2020 8:58 AM
> > To: d...@mxnet.apache.org
> > Subject: Re: RE: Updates for 1.7.0 minor release
> > 
> > Hi Ciyong,
> > 
> > We found a GPU memory usage regression issue triggered by PR 
> > https://github.com/apache/incubator-mxnet/pull/17767, which was 
> > pushed to both 2.0, 1.x and 1.7 branches
> > 
> > I have reverted this commit in 2.0, but we should revert this in 1.x 
> > and
> > 1.7 branches. I have made a reverting PR on 1.x 
> > https://github.com/apache/incubator-mxnet/pull/18309.
> > 
> > I am thinking if you can help to merge the reverting into 1.x and 
> > 1.7 before making the rc0 tag?
> > 
> > Thanks,
> > Ziyi
> > 
> > On 2020/05/12 00:58:22, "Chen, Ciyong"  wrote:
> > > Hi Chai,
> > > 
> > > Thanks a lot for your kindly help to fix this 
> > > I will continue the rest steps of release process.
> > > 
> > > Thanks,
> > > -Ciyong
> > > 
> > > -Original Message-
> > > From: Chaitanya Bapat 
> > > Sent: Tuesday, May 12, 2020 8:14 AM
> > > To: dev@mxnet.incubator.apache.org
> > > Subject: Re: Updates for 1.7.0 minor release
> > > 
> > > Hello Ciyong,
> > > 
> > > With the https://github.com/apache/incubator-mxnet/pull/18261
> > > merged,
> > nightly pipeline passes for 1.7.x So

RE: RE: Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi Chai,

Sorry for the late update.

Recently, several bug fixes [4] including numpy operator/batchnorm 
gradient/LSTM CPU gradient/CI/CD/license issues were back-ported into v1.7.x.
So far, there's one build issue and two license issues being tracked.
1) build issue #18501 (It costs over 16GB memory to compile 
indexing_op.o), which @leezu stated it's a blocker for the release[1].
2) license issue: multiple license header issue[2] is under discussion; 
no valid apache license header issue[3] is identified, and I'm working on the 
PR as @szha suggested.

If the community can help to expedite the item of [1] and [2], it will be great 
helpful.
Once we've completed the above items and no more other critical issues, it's ok 
to cut the rc0.

Thanks for your patients.

Thanks,
-Ciyong

[1] 
https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-642785535
[2] 
https://github.com/apache/incubator-mxnet/issues/17329#issuecomment-641311199
[3] https://github.com/apache/incubator-mxnet/pull/18478#issuecomment-642462904
[4] PR list: 
#18358/#18339/#18311/#18352/#18456/#18316/#18482/#18502/#18517/#18464



-Original Message-
From: Chaitanya Bapat  
Sent: Friday, June 12, 2020 1:34 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: RE: Updates for 1.7.0 minor release

Hey Ciyong,

Since the last discussion, the GPU memory regression PR has been reverted.
Is there any update for when the rc0 for 1.7 will be cut?
Can the community help expedite the process in any way?

Thanks
Chai

On Wed, 13 May 2020 at 18:28, Chen, Ciyong  wrote:

> Hi Ziyi,
>
> Thanks for reaching me for the known/found issue in the upcoming 
> release, let's fix all these potential issues before dropping the rc0 
> tag 
> I'll ask help from Tao to merge the PR.
>
> Thanks,
> -Ciyong
>
> -Original Message-
> From: Patrick Mu 
> Sent: Thursday, May 14, 2020 8:58 AM
> To: d...@mxnet.apache.org
> Subject: Re: RE: Updates for 1.7.0 minor release
>
> Hi Ciyong,
>
> We found a GPU memory usage regression issue triggered by PR 
> https://github.com/apache/incubator-mxnet/pull/17767, which was pushed 
> to both 2.0, 1.x and 1.7 branches
>
> I have reverted this commit in 2.0, but we should revert this in 1.x 
> and
> 1.7 branches. I have made a reverting PR on 1.x 
> https://github.com/apache/incubator-mxnet/pull/18309.
>
> I am thinking if you can help to merge the reverting into 1.x and 1.7 
> before making the rc0 tag?
>
> Thanks,
> Ziyi
>
> On 2020/05/12 00:58:22, "Chen, Ciyong"  wrote:
> > Hi Chai,
> >
> > Thanks a lot for your kindly help to fix this 
> > I will continue the rest steps of release process.
> >
> > Thanks,
> > -Ciyong
> >
> > -Original Message-
> > From: Chaitanya Bapat 
> > Sent: Tuesday, May 12, 2020 8:14 AM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: Updates for 1.7.0 minor release
> >
> > Hello Ciyong,
> >
> > With the https://github.com/apache/incubator-mxnet/pull/18261 
> > merged,
> nightly pipeline passes for 1.7.x So as far as the 2 nightly test 
> pipelines are concerned [NightlyTests and NightlyTestsForBinaries] 
> 1.7.x is good to go!
> >
> > Thanks,
> > Chai
> >
> > On Sun, 10 May 2020 at 04:53, Chen, Ciyong 
> wrote:
> >
> > > Hi MXNet Community,
> > >
> > > Here's some updates after the code freeze.
> > > 1. Nightly tests[1] and nightly binaries tests[2] were enabled, 
> > > many thanks to Chaitanya who helped to create and activate these 
> > > jobs for v1.7.x branch.
> > > 2. A nightly test failure (incorrect with_seed path) was fixed by 
> > > Chaitanya [3] 3. A bug fix for external graph pass by Sam [4] 4.
> > > Recently, there's another failed cased (test_large_vector.test_nn) 
> > > in nightly test[5], and Chaitanya is helping to address this 
> > > issue[6]
> > >
> > > I'll keep monitoring the nightly test before making a rc0 tag.
> > > Please let me know if you have any other issues that should be 
> > > included/fixed in this release.
> > >
> > > Thanks,
> > > -Ciyong
> > >
> > > ---
> > > [1]
> > > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Nig
> > > ht
> > > ly
> > > Tests/job/v1.7.x/
> > > [2]
> > > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Nig
> > > ht
> > > ly
> > > TestsForBinaries/job/v1.7.x/ [3]
> > > https://github.com/apache/incubator-mxnet/pull/18220
> > > [4] https://github.com/apache/incubator-mxnet/pull/18237
> > > [5]
> 

RE: RE: Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi Ziyi,

Thanks for reaching me for the known/found issue in the upcoming release, let's 
fix all these potential issues before dropping the rc0 tag 
I'll ask help from Tao to merge the PR.

Thanks,
-Ciyong

-Original Message-
From: Patrick Mu  
Sent: Thursday, May 14, 2020 8:58 AM
To: d...@mxnet.apache.org
Subject: Re: RE: Updates for 1.7.0 minor release

Hi Ciyong,

We found a GPU memory usage regression issue triggered by PR 
https://github.com/apache/incubator-mxnet/pull/17767, which was pushed to both 
2.0, 1.x and 1.7 branches

I have reverted this commit in 2.0, but we should revert this in 1.x and 1.7 
branches. I have made a reverting PR on 1.x 
https://github.com/apache/incubator-mxnet/pull/18309.

I am thinking if you can help to merge the reverting into 1.x and 1.7 before 
making the rc0 tag?

Thanks,
Ziyi

On 2020/05/12 00:58:22, "Chen, Ciyong"  wrote: 
> Hi Chai,
> 
> Thanks a lot for your kindly help to fix this 
> I will continue the rest steps of release process.
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Chaitanya Bapat 
> Sent: Tuesday, May 12, 2020 8:14 AM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: Updates for 1.7.0 minor release
> 
> Hello Ciyong,
> 
> With the https://github.com/apache/incubator-mxnet/pull/18261 merged, nightly 
> pipeline passes for 1.7.x So as far as the 2 nightly test pipelines are 
> concerned [NightlyTests and NightlyTestsForBinaries] 1.7.x is good to go!
> 
> Thanks,
> Chai
> 
> On Sun, 10 May 2020 at 04:53, Chen, Ciyong  wrote:
> 
> > Hi MXNet Community,
> >
> > Here's some updates after the code freeze.
> > 1. Nightly tests[1] and nightly binaries tests[2] were enabled, many 
> > thanks to Chaitanya who helped to create and activate these jobs for 
> > v1.7.x branch.
> > 2. A nightly test failure (incorrect with_seed path) was fixed by 
> > Chaitanya [3] 3. A bug fix for external graph pass by Sam [4] 4.
> > Recently, there's another failed cased (test_large_vector.test_nn) 
> > in nightly test[5], and Chaitanya is helping to address this 
> > issue[6]
> >
> > I'll keep monitoring the nightly test before making a rc0 tag.
> > Please let me know if you have any other issues that should be 
> > included/fixed in this release.
> >
> > Thanks,
> > -Ciyong
> >
> > ---
> > [1]
> > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Night
> > ly
> > Tests/job/v1.7.x/
> > [2]
> > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Night
> > ly
> > TestsForBinaries/job/v1.7.x/ [3]
> > https://github.com/apache/incubator-mxnet/pull/18220
> > [4] https://github.com/apache/incubator-mxnet/pull/18237
> > [5]
> > http://jenkins.mxnet-ci.amazon-ml.com/job/NightlyTestsForBinaries/jo
> > b/ v1.7.x/2/execution/node/232/log/ [6]
> > https://github.com/apache/incubator-mxnet/pull/18261
> >
> >
> > -Original Message-
> > From: Chen, Ciyong 
> > Sent: Sunday, April 26, 2020 3:29 PM
> > To: dev@mxnet.incubator.apache.org
> > Cc: Marco de Abreu 
> > Subject: Code freeze for 1.7.0 minor release
> >
> > Hi MXNet Community,
> >
> > Code freeze for 1.7.0 minor release is in effect (last commit: 38e6634)!
> > Which means there're no more NEW features going to be accepted for 
> > this release.
> >
> > Many thanks to everyone who helped submitting/back porting/reviewing 
> > the PRs targeting this release.
> > I've created a draft Release Notes for 1.7.0 release[1], please take 
> > a review, any comments/suggestions are highly appreciated.
> >
> > Currently, the nightly test pipeline [2][3] for v1.7.x is not 
> > triggered, cc @Marco de Abreu  > marco.g.ab...@gmail.com> to help take a look.
> > I will keep monitoring the nightly test result for the current code 
> > base, and continue to go through the rest of releasing process.
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Note
> > s
> > [2]
> > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Night
> > ly
> > Tests/job/v1.7.x/
> > [3]
> > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Night
> > ly
> > TestsForBinaries/job/v1.7.x/
> >
> >
> > Thanks,
> > -Ciyong
> >
> >
> 
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
> 
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image: 
> https://www.facebook.com/chaibapat]
> <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>
> 

RE: Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi Chai,

Thanks a lot for your kindly help to fix this 
I will continue the rest steps of release process.

Thanks,
-Ciyong

-Original Message-
From: Chaitanya Bapat  
Sent: Tuesday, May 12, 2020 8:14 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: Updates for 1.7.0 minor release

Hello Ciyong,

With the https://github.com/apache/incubator-mxnet/pull/18261 merged, nightly 
pipeline passes for 1.7.x So as far as the 2 nightly test pipelines are 
concerned [NightlyTests and NightlyTestsForBinaries] 1.7.x is good to go!

Thanks,
Chai

On Sun, 10 May 2020 at 04:53, Chen, Ciyong  wrote:

> Hi MXNet Community,
>
> Here's some updates after the code freeze.
> 1. Nightly tests[1] and nightly binaries tests[2] were enabled, many 
> thanks to Chaitanya who helped to create and activate these jobs for 
> v1.7.x branch.
> 2. A nightly test failure (incorrect with_seed path) was fixed by 
> Chaitanya [3] 3. A bug fix for external graph pass by Sam [4] 4. 
> Recently, there's another failed cased (test_large_vector.test_nn) in 
> nightly test[5], and Chaitanya is helping to address this issue[6]
>
> I'll keep monitoring the nightly test before making a rc0 tag.
> Please let me know if you have any other issues that should be 
> included/fixed in this release.
>
> Thanks,
> -Ciyong
>
> ---
> [1]
> http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Nightly
> Tests/job/v1.7.x/
> [2]
> http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Nightly
> TestsForBinaries/job/v1.7.x/ [3] 
> https://github.com/apache/incubator-mxnet/pull/18220
> [4] https://github.com/apache/incubator-mxnet/pull/18237
> [5]
> http://jenkins.mxnet-ci.amazon-ml.com/job/NightlyTestsForBinaries/job/
> v1.7.x/2/execution/node/232/log/ [6] 
> https://github.com/apache/incubator-mxnet/pull/18261
>
>
> -Original Message-
> From: Chen, Ciyong 
> Sent: Sunday, April 26, 2020 3:29 PM
> To: dev@mxnet.incubator.apache.org
> Cc: Marco de Abreu 
> Subject: Code freeze for 1.7.0 minor release
>
> Hi MXNet Community,
>
> Code freeze for 1.7.0 minor release is in effect (last commit: 38e6634)!
> Which means there're no more NEW features going to be accepted for 
> this release.
>
> Many thanks to everyone who helped submitting/back porting/reviewing 
> the PRs targeting this release.
> I've created a draft Release Notes for 1.7.0 release[1], please take a 
> review, any comments/suggestions are highly appreciated.
>
> Currently, the nightly test pipeline [2][3] for v1.7.x is not 
> triggered, cc @Marco de Abreu  marco.g.ab...@gmail.com> to help take a look.
> I will keep monitoring the nightly test result for the current code 
> base, and continue to go through the rest of releasing process.
>
> [1] 
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Notes
> [2]
> http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Nightly
> Tests/job/v1.7.x/
> [3]
> http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/Nightly
> TestsForBinaries/job/v1.7.x/
>
>
> Thanks,
> -Ciyong
>
>

--
*Chaitanya Prakash Bapat*
*+1 (973) 953-6299*

[image: https://www.linkedin.com//in/chaibapat25]
<https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat]
<https://www.facebook.com/chaibapchya>[image:
https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
https://www.linkedin.com//in/chaibapat25]
<https://www.linkedin.com//in/chaibapchya/>

Updates for 1.7.0 minor release

[Chen, Ciyong]

Hi MXNet Community,

Here's some updates after the code freeze.
1. Nightly tests[1] and nightly binaries tests[2] were enabled, many thanks to 
Chaitanya who helped to create and activate these jobs for v1.7.x branch.
2. A nightly test failure (incorrect with_seed path) was fixed by Chaitanya [3]
3. A bug fix for external graph pass by Sam [4]
4. Recently, there's another failed cased (test_large_vector.test_nn) in 
nightly test[5], and Chaitanya is helping to address this issue[6]

I'll keep monitoring the nightly test before making a rc0 tag.
Please let me know if you have any other issues that should be included/fixed 
in this release.

Thanks,
-Ciyong

---
[1] 
http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/NightlyTests/job/v1.7.x/
[2] 
http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/NightlyTestsForBinaries/job/v1.7.x/
[3] https://github.com/apache/incubator-mxnet/pull/18220
[4] https://github.com/apache/incubator-mxnet/pull/18237
[5] 
http://jenkins.mxnet-ci.amazon-ml.com/job/NightlyTestsForBinaries/job/v1.7.x/2/execution/node/232/log/
[6] https://github.com/apache/incubator-mxnet/pull/18261


-Original Message-
From: Chen, Ciyong  
Sent: Sunday, April 26, 2020 3:29 PM
To: dev@mxnet.incubator.apache.org
Cc: Marco de Abreu 
Subject: Code freeze for 1.7.0 minor release

Hi MXNet Community,

Code freeze for 1.7.0 minor release is in effect (last commit: 38e6634)!
Which means there're no more NEW features going to be accepted for this release.

Many thanks to everyone who helped submitting/back porting/reviewing the PRs 
targeting this release.
I've created a draft Release Notes for 1.7.0 release[1], please take a review, 
any comments/suggestions are highly appreciated.

Currently, the nightly test pipeline [2][3] for v1.7.x is not triggered, cc 
@Marco de Abreu <mailto:marco.g.ab...@gmail.com> to 
help take a look.
I will keep monitoring the nightly test result for the current code base, and 
continue to go through the rest of releasing process.

[1] https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Notes
[2] 
http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/NightlyTests/job/v1.7.x/
[3] 
http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/NightlyTestsForBinaries/job/v1.7.x/


Thanks,
-Ciyong

Code freeze for 1.7.0 minor release

[Chen, Ciyong]

Hi MXNet Community,

Code freeze for 1.7.0 minor release is in effect (last commit: 38e6634)!
Which means there're no more NEW features going to be accepted for this release.

Many thanks to everyone who helped submitting/back porting/reviewing the PRs 
targeting this release.
I've created a draft Release Notes for 1.7.0 release[1], please take a review, 
any comments/suggestions are highly appreciated.

Currently, the nightly test pipeline [2][3] for v1.7.x is not triggered, cc 
@Marco de Abreu  to 
help take a look.
I will keep monitoring the nightly test result for the current code base, and 
continue to go through the rest of releasing process.

[1] https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Notes
[2] 
http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/NightlyTests/job/v1.7.x/
[3] 
http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/NightlyTestsForBinaries/job/v1.7.x/


Thanks,
-Ciyong

RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

[Chen, Ciyong]

Hi MXNet community,

As there're still some PRs (which targeting v1.7.0 release) pending on review 
or CI, I would suggest to extend the code freeze day to April 25 PST.
Please let me know if you have any questions or suggestion.
Thanks for your patience.

Thanks,
-Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Saturday, April 18, 2020 8:23 PM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi dev,

Please note that v1.7.x branch is rebased to the latest v1.x branch (4cf2ad39b) 
with Tao's help, which includes several PRs merged into v1.x recently.
Please cherry-pick or reach me if you want more PRs in v1.7.x.
And the upcoming code freeze will be April 20th PST, please let me know if you 
need a time extension.

Can I get some help from CI team to help creating the 
NightlyTests/NightlyTestForBinaries task for v1.7.x branch? Thanks!

Thanks,
-Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Thursday, April 16, 2020 10:11 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi Sam,

Thanks for pointing out this, I will track this PR before doing the code rebase 
(April 17 PST).

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Thursday, April 16, 2020 12:55 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi Ciyong,

I took a look at the 1.7 branch and noticed some PRs were missing for MXNet 
Extensions including fixes and enhancements for custom ops (sparse, RNG, 
subgraph) and custom subgraph properties. I created a PR to backport these PRs: 
https://github.com/apache/incubator-mxnet/pull/18063 and updated the wiki page: 
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan+and+Status.
 All PRs have already been merged on master branch, and I expect the backport 
PR to succeed sometime today.

Thanks for organizing this release!
Sam

On Apr 10, 2020, at 7:54 PM, Chen, Ciyong 
mailto:ciyong.c...@intel.com>> wrote:

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.



Hi Prezmek and all,

Sure, no problem. I saw there're some more PRs would like to be included in 
v1.7.0 release and doing backport to v1.x branch recently.
So I propose to postpone the code rebase for a week (April 17th), assume most 
of the backport PR will be available on v1.x branch at that time.

Feel free to add any other comments.

Thanks,
-Ciyong

-Original Message-
From: Przemysław Trędak mailto:ptre...@apache.org>>
Sent: Saturday, April 11, 2020 6:17 AM
To: d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

I added GPU vectorization of elementwise ops to the 1.7 scope, but will need a 
small extension - the PR should be ready to merge to master and cherry-picked 
to 1.7 early next week.

Thanks
Przemek

On 2020/04/09 08:27:58, "Chen, Ciyong" 
mailto:ciyong.c...@intel.com>> wrote:
Hi dev,

Just a remainder for you to do the backport of new features/enhancement/bug 
fixes from master branch to v1.x branch, as we're going to do the code rebase 
(rebase v1.7.0 to latest v1.x) on April 10th.
Or please let me know if you need a time extension.

Thanks,
-Ciyong

-Original Message-
From: Tao Lv mailto:ta...@apache.org>>
Sent: Tuesday, April 7, 2020 2:21 PM
To: dev@mxnet.incubator.apache.org<mailto:dev@mxnet.incubator.apache.org>
Cc: d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>
Subject: Re: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi dev,

Because we have master branch for 2.0 development and v1.x branch for 1.x 
development, please make sure your new features, bug fixes and improvements are 
properly picked to v1.x branch before the code rebase, if you want them to be 
included into v1.7 release.

thanks,
-tao

On Tue, Apr 7, 2020 at 4:08 AM Chaitanya Bapat 
mailto:chai.ba...@gmail.com>> wrote:

Hi Ciyong,
Awesome news!
Thanks for driving 1.7.0 release!
Looking forward to collaboration.

Cheers,
Chai

On Mon, 6 Apr 2020 at 02:51, Chen, Ciyong 
mailto:ciyong.c...@intel.com>> wrote:

Hi MXNet Community,



As the 1.6.0 release is done (many thanks to Przemek) for a while, it's time to 
prepare for the next minor release of MXNet 1.7.0.



Thanks to Patric (pengzhao-intel@github) who had created the GitHub issue[1] to 
discuss and collect the major features for 1.7.0. Please add any 
features/enhancements/bugfixes that you want included there.



I (ciyongch@github) would like to be the release manager for the upcoming
1.7.0 release. It's my first time to drive a release for MXNet, really thanks 
to Tao(TaoLv

RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

[Chen, Ciyong]

Hi dev,

Please note that v1.7.x branch is rebased to the latest v1.x branch (4cf2ad39b) 
with Tao's help, which includes several PRs merged into v1.x recently.
Please cherry-pick or reach me if you want more PRs in v1.7.x.
And the upcoming code freeze will be April 20th PST, please let me know if you 
need a time extension.

Can I get some help from CI team to help creating the 
NightlyTests/NightlyTestForBinaries task for v1.7.x branch? Thanks!

Thanks,
-Ciyong

-Original Message-
From: Chen, Ciyong  
Sent: Thursday, April 16, 2020 10:11 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi Sam,

Thanks for pointing out this, I will track this PR before doing the code rebase 
(April 17 PST).

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Thursday, April 16, 2020 12:55 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi Ciyong,

I took a look at the 1.7 branch and noticed some PRs were missing for MXNet 
Extensions including fixes and enhancements for custom ops (sparse, RNG, 
subgraph) and custom subgraph properties. I created a PR to backport these PRs: 
https://github.com/apache/incubator-mxnet/pull/18063 and updated the wiki page: 
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan+and+Status.
 All PRs have already been merged on master branch, and I expect the backport 
PR to succeed sometime today.

Thanks for organizing this release!
Sam

On Apr 10, 2020, at 7:54 PM, Chen, Ciyong 
mailto:ciyong.c...@intel.com>> wrote:

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.



Hi Prezmek and all,

Sure, no problem. I saw there're some more PRs would like to be included in 
v1.7.0 release and doing backport to v1.x branch recently.
So I propose to postpone the code rebase for a week (April 17th), assume most 
of the backport PR will be available on v1.x branch at that time.

Feel free to add any other comments.

Thanks,
-Ciyong

-Original Message-
From: Przemysław Trędak mailto:ptre...@apache.org>>
Sent: Saturday, April 11, 2020 6:17 AM
To: d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

I added GPU vectorization of elementwise ops to the 1.7 scope, but will need a 
small extension - the PR should be ready to merge to master and cherry-picked 
to 1.7 early next week.

Thanks
Przemek

On 2020/04/09 08:27:58, "Chen, Ciyong" 
mailto:ciyong.c...@intel.com>> wrote:
Hi dev,

Just a remainder for you to do the backport of new features/enhancement/bug 
fixes from master branch to v1.x branch, as we're going to do the code rebase 
(rebase v1.7.0 to latest v1.x) on April 10th.
Or please let me know if you need a time extension.

Thanks,
-Ciyong

-Original Message-
From: Tao Lv mailto:ta...@apache.org>>
Sent: Tuesday, April 7, 2020 2:21 PM
To: dev@mxnet.incubator.apache.org<mailto:dev@mxnet.incubator.apache.org>
Cc: d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>
Subject: Re: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi dev,

Because we have master branch for 2.0 development and v1.x branch for 1.x 
development, please make sure your new features, bug fixes and improvements are 
properly picked to v1.x branch before the code rebase, if you want them to be 
included into v1.7 release.

thanks,
-tao

On Tue, Apr 7, 2020 at 4:08 AM Chaitanya Bapat 
mailto:chai.ba...@gmail.com>> wrote:

Hi Ciyong,
Awesome news!
Thanks for driving 1.7.0 release!
Looking forward to collaboration.

Cheers,
Chai

On Mon, 6 Apr 2020 at 02:51, Chen, Ciyong 
mailto:ciyong.c...@intel.com>> wrote:

Hi MXNet Community,



As the 1.6.0 release is done (many thanks to Przemek) for a while, it's time to 
prepare for the next minor release of MXNet 1.7.0.



Thanks to Patric (pengzhao-intel@github) who had created the GitHub issue[1] to 
discuss and collect the major features for 1.7.0. Please add any 
features/enhancements/bugfixes that you want included there.



I (ciyongch@github) would like to be the release manager for the upcoming
1.7.0 release. It's my first time to drive a release for MXNet, really thanks 
to Tao(TaoLv@github) who will guide me to go through the process and already 
helped to create the Release plan and status page[2] on cwiki.
I will keep updating the issues and PRs mentioned in issues[1] to the cwiki 
page[2] to track the release process.



We're proposing to do the release in May, and rebase the code of v1.7.x to 
latest v1.x on April 10th PST, the code freeze date will be April 20th PST.
Please let me know whether this timeline is suitable to you or not, or want an 
extension. So your tim

RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

[Chen, Ciyong]

Hi Sam,

Thanks for pointing out this, I will track this PR before doing the code rebase 
(April 17 PST).

Thanks,
-Ciyong

-Original Message-
From: Skalicky, Sam  
Sent: Thursday, April 16, 2020 12:55 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi Ciyong,

I took a look at the 1.7 branch and noticed some PRs were missing for MXNet 
Extensions including fixes and enhancements for custom ops (sparse, RNG, 
subgraph) and custom subgraph properties. I created a PR to backport these PRs: 
https://github.com/apache/incubator-mxnet/pull/18063 and updated the wiki page: 
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan+and+Status.
 All PRs have already been merged on master branch, and I expect the backport 
PR to succeed sometime today.

Thanks for organizing this release!
Sam

On Apr 10, 2020, at 7:54 PM, Chen, Ciyong 
mailto:ciyong.c...@intel.com>> wrote:

CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.



Hi Prezmek and all,

Sure, no problem. I saw there're some more PRs would like to be included in 
v1.7.0 release and doing backport to v1.x branch recently.
So I propose to postpone the code rebase for a week (April 17th), assume most 
of the backport PR will be available on v1.x branch at that time.

Feel free to add any other comments.

Thanks,
-Ciyong

-Original Message-
From: Przemysław Trędak mailto:ptre...@apache.org>>
Sent: Saturday, April 11, 2020 6:17 AM
To: d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

I added GPU vectorization of elementwise ops to the 1.7 scope, but will need a 
small extension - the PR should be ready to merge to master and cherry-picked 
to 1.7 early next week.

Thanks
Przemek

On 2020/04/09 08:27:58, "Chen, Ciyong" 
mailto:ciyong.c...@intel.com>> wrote:
Hi dev,

Just a remainder for you to do the backport of new features/enhancement/bug 
fixes from master branch to v1.x branch, as we're going to do the code rebase 
(rebase v1.7.0 to latest v1.x) on April 10th.
Or please let me know if you need a time extension.

Thanks,
-Ciyong

-Original Message-
From: Tao Lv mailto:ta...@apache.org>>
Sent: Tuesday, April 7, 2020 2:21 PM
To: dev@mxnet.incubator.apache.org<mailto:dev@mxnet.incubator.apache.org>
Cc: d...@mxnet.apache.org<mailto:d...@mxnet.apache.org>
Subject: Re: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi dev,

Because we have master branch for 2.0 development and v1.x branch for 1.x 
development, please make sure your new features, bug fixes and improvements are 
properly picked to v1.x branch before the code rebase, if you want them to be 
included into v1.7 release.

thanks,
-tao

On Tue, Apr 7, 2020 at 4:08 AM Chaitanya Bapat 
mailto:chai.ba...@gmail.com>> wrote:

Hi Ciyong,
Awesome news!
Thanks for driving 1.7.0 release!
Looking forward to collaboration.

Cheers,
Chai

On Mon, 6 Apr 2020 at 02:51, Chen, Ciyong 
mailto:ciyong.c...@intel.com>> wrote:

Hi MXNet Community,



As the 1.6.0 release is done (many thanks to Przemek) for a while, it's time to 
prepare for the next minor release of MXNet 1.7.0.



Thanks to Patric (pengzhao-intel@github) who had created the GitHub issue[1] to 
discuss and collect the major features for 1.7.0. Please add any 
features/enhancements/bugfixes that you want included there.



I (ciyongch@github) would like to be the release manager for the upcoming
1.7.0 release. It's my first time to drive a release for MXNet, really thanks 
to Tao(TaoLv@github) who will guide me to go through the process and already 
helped to create the Release plan and status page[2] on cwiki.
I will keep updating the issues and PRs mentioned in issues[1] to the cwiki 
page[2] to track the release process.



We're proposing to do the release in May, and rebase the code of v1.7.x to 
latest v1.x on April 10th PST, the code freeze date will be April 20th PST.
Please let me know whether this timeline is suitable to you or not, or want an 
extension. So your timely response will be highly appreciated!



Feel free to add any other comments/suggestions.



Thanks,

Ciyong



[1] https://github.com/apache/incubator-mxnet/issues/16864

[2]

https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan
+a
nd+Status




--
*Chaitanya Prakash Bapat*
*+1 (973) 953-6299*

[image: https://www.linkedin.com//in/chaibapat25]
<https://github.com/ChaiBapchya>[image:
https://www.facebook.com/chaibapat
]
<https://www.facebook.com/chaibapchya>[image:
https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
https://www.linkedin.com//in/chaibapat25]
<https://www.linkedin.com//in/chaibapchya/>

RE: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

[Chen, Ciyong]

Hi Prezmek and all,

Sure, no problem. I saw there're some more PRs would like to be included in 
v1.7.0 release and doing backport to v1.x branch recently.
So I propose to postpone the code rebase for a week (April 17th), assume most 
of the backport PR will be available on v1.x branch at that time.

Feel free to add any other comments.

Thanks,
-Ciyong

-Original Message-
From: Przemysław Trędak  
Sent: Saturday, April 11, 2020 6:17 AM
To: d...@mxnet.apache.org
Subject: Re: RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

I added GPU vectorization of elementwise ops to the 1.7 scope, but will need a 
small extension - the PR should be ready to merge to master and cherry-picked 
to 1.7 early next week.

Thanks
Przemek 

On 2020/04/09 08:27:58, "Chen, Ciyong"  wrote: 
> Hi dev,
> 
> Just a remainder for you to do the backport of new features/enhancement/bug 
> fixes from master branch to v1.x branch, as we're going to do the code rebase 
> (rebase v1.7.0 to latest v1.x) on April 10th.
> Or please let me know if you need a time extension.
> 
> Thanks,
> -Ciyong
> 
> -Original Message-
> From: Tao Lv 
> Sent: Tuesday, April 7, 2020 2:21 PM
> To: dev@mxnet.incubator.apache.org
> Cc: d...@mxnet.apache.org
> Subject: Re: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 
> release
> 
> Hi dev,
> 
> Because we have master branch for 2.0 development and v1.x branch for 1.x 
> development, please make sure your new features, bug fixes and improvements 
> are properly picked to v1.x branch before the code rebase, if you want them 
> to be included into v1.7 release.
> 
> thanks,
> -tao
> 
> On Tue, Apr 7, 2020 at 4:08 AM Chaitanya Bapat  wrote:
> 
> > Hi Ciyong,
> > Awesome news!
> > Thanks for driving 1.7.0 release!
> > Looking forward to collaboration.
> >
> > Cheers,
> > Chai
> >
> > On Mon, 6 Apr 2020 at 02:51, Chen, Ciyong  wrote:
> >
> > >  Hi MXNet Community,
> > >
> > >
> > >
> > > As the 1.6.0 release is done (many thanks to Przemek) for a while, 
> > > it's time to prepare for the next minor release of MXNet 1.7.0.
> > >
> > >
> > >
> > > Thanks to Patric (pengzhao-intel@github) who had created the 
> > > GitHub issue[1] to discuss and collect the major features for 
> > > 1.7.0. Please add any features/enhancements/bugfixes that you want 
> > > included there.
> > >
> > >
> > >
> > > I (ciyongch@github) would like to be the release manager for the
> > upcoming
> > > 1.7.0 release. It's my first time to drive a release for MXNet, 
> > > really thanks to Tao(TaoLv@github) who will guide me to go through 
> > > the process and already helped to create the Release plan and 
> > > status page[2] on
> > cwiki.
> > > I will keep updating the issues and PRs mentioned in issues[1] to 
> > > the
> > cwiki
> > > page[2] to track the release process.
> > >
> > >
> > >
> > > We're proposing to do the release in May, and rebase the code of 
> > > v1.7.x
> > to
> > > latest v1.x on April 10th PST, the code freeze date will be April 
> > > 20th
> > PST.
> > > Please let me know whether this timeline is suitable to you or 
> > > not, or
> > want
> > > an extension. So your timely response will be highly appreciated!
> > >
> > >
> > >
> > > Feel free to add any other comments/suggestions.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Ciyong
> > >
> > >
> > >
> > > [1] https://github.com/apache/incubator-mxnet/issues/16864
> > >
> > > [2]
> > >
> > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan
> > +a
> > nd+Status
> > >
> > >
> > >
> >
> > --
> > *Chaitanya Prakash Bapat*
> > *+1 (973) 953-6299*
> >
> > [image: https://www.linkedin.com//in/chaibapat25]
> > <https://github.com/ChaiBapchya>[image: 
> > https://www.facebook.com/chaibapat
> > ]
> > <https://www.facebook.com/chaibapchya>[image:
> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> > https://www.linkedin.com//in/chaibapat25]
> > <https://www.linkedin.com//in/chaibapchya/>
> >
> 

RE: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

[Chen, Ciyong]

Hi dev,

Just a remainder for you to do the backport of new features/enhancement/bug 
fixes from master branch to v1.x branch, as we're going to do the code rebase 
(rebase v1.7.0 to latest v1.x) on April 10th.
Or please let me know if you need a time extension.

Thanks,
-Ciyong

-Original Message-
From: Tao Lv  
Sent: Tuesday, April 7, 2020 2:21 PM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: [Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

Hi dev,

Because we have master branch for 2.0 development and v1.x branch for 1.x 
development, please make sure your new features, bug fixes and improvements are 
properly picked to v1.x branch before the code rebase, if you want them to be 
included into v1.7 release.

thanks,
-tao

On Tue, Apr 7, 2020 at 4:08 AM Chaitanya Bapat  wrote:

> Hi Ciyong,
> Awesome news!
> Thanks for driving 1.7.0 release!
> Looking forward to collaboration.
>
> Cheers,
> Chai
>
> On Mon, 6 Apr 2020 at 02:51, Chen, Ciyong  wrote:
>
> >  Hi MXNet Community,
> >
> >
> >
> > As the 1.6.0 release is done (many thanks to Przemek) for a while, 
> > it's time to prepare for the next minor release of MXNet 1.7.0.
> >
> >
> >
> > Thanks to Patric (pengzhao-intel@github) who had created the GitHub 
> > issue[1] to discuss and collect the major features for 1.7.0. Please 
> > add any features/enhancements/bugfixes that you want included there.
> >
> >
> >
> > I (ciyongch@github) would like to be the release manager for the
> upcoming
> > 1.7.0 release. It's my first time to drive a release for MXNet, 
> > really thanks to Tao(TaoLv@github) who will guide me to go through 
> > the process and already helped to create the Release plan and status 
> > page[2] on
> cwiki.
> > I will keep updating the issues and PRs mentioned in issues[1] to 
> > the
> cwiki
> > page[2] to track the release process.
> >
> >
> >
> > We're proposing to do the release in May, and rebase the code of 
> > v1.7.x
> to
> > latest v1.x on April 10th PST, the code freeze date will be April 
> > 20th
> PST.
> > Please let me know whether this timeline is suitable to you or not, 
> > or
> want
> > an extension. So your timely response will be highly appreciated!
> >
> >
> >
> > Feel free to add any other comments/suggestions.
> >
> >
> >
> > Thanks,
> >
> > Ciyong
> >
> >
> >
> > [1] https://github.com/apache/incubator-mxnet/issues/16864
> >
> > [2]
> >
> https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan+a
> nd+Status
> >
> >
> >
>
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
>
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image: 
> https://www.facebook.com/chaibapat
> ]
> <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>
>

[Announce] Upcoming Apache MXNet (incubating) 1.7.0 release

[Chen, Ciyong]

 Hi MXNet Community,



As the 1.6.0 release is done (many thanks to Przemek) for a while, it's time to 
prepare for the next minor release of MXNet 1.7.0.



Thanks to Patric (pengzhao-intel@github) who had created the GitHub issue[1] to 
discuss and collect the major features for 1.7.0. Please add any 
features/enhancements/bugfixes that you want included there.



I (ciyongch@github) would like to be the release manager for the upcoming 1.7.0 
release. It's my first time to drive a release for MXNet, really thanks to 
Tao(TaoLv@github) who will guide me to go through the process and already 
helped to create the Release plan and status page[2] on cwiki. I will keep 
updating the issues and PRs mentioned in issues[1] to the cwiki page[2] to 
track the release process.



We're proposing to do the release in May, and rebase the code of v1.7.x to 
latest v1.x on April 10th PST, the code freeze date will be April 20th PST. 
Please let me know whether this timeline is suitable to you or not, or want an 
extension. So your timely response will be highly appreciated!



Feel free to add any other comments/suggestions.



Thanks,

Ciyong



[1] https://github.com/apache/incubator-mxnet/issues/16864

[2] 
https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+Plan+and+Status

RE: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1

[Chen, Ciyong]

+1

Build from source on CentOS 7.6 with GCC 4.8.5 with MKLDNN.
Unit tests passed and imagenet examples (with MKL-DNN subgraph backend) looked 
good on performance and accuracy in both FP32 and INT8 mode, RNN training 
worked.

Thanks,
-Ciyong

-Original Message-
From: Lai Wei  
Sent: Wednesday, January 8, 2020 8:56 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1

+1
Build from source on Ubuntu with CUDA/CUDNN/MKLDNN and tested with keras-mxnet.
Unit tests passed and example works on CPU/GPU.


Best Regards

Lai


On Tue, Jan 7, 2020 at 11:49 AM Lin Yuan  wrote:

> Correction: it was built from source on Ubuntu 16.04
>
> On Tue, Jan 7, 2020 at 11:42 AM Lin Yuan  wrote:
>
> > +1
> >
> > Build from source on Ubuntu 18 with CUDA/CUDNN/NCCL on and verified 
> > it works with Horovod 0.18.2
> >
> > On Tue, Jan 7, 2020 at 9:55 AM Przemysław Trędak 
> > 
> > wrote:
> >
> >> Dear MXNet community,
> >>
> >> This is the vote to release Apache MXNet (incubating) version 1.6.0.
> >> Voting starts today and will close on Friday 1/10/2020 23:59 PST.
> >>
> >> Link to release notes:
> >> https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+not
> >> es
> >>
> >> Link to release candidate:
> >> https://github.com/apache/incubator-mxnet/releases/tag/1.6.0.rc1
> >>
> >> Link to source and signatures on apache dist server:
> >> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.6.0.rc1/
> >>
> >> The differences comparing to previous release candidate 1.6.0.rc0:
> >> * Fix for RNN gradient calculation for MKLDNN ([v1.6.x] Cherry-pick 
> >> MKL-DNN Rnn operator enhancements to v1.6.x (#17225))
> >> * Fix for Windows CMake build (Backport #16980 #17031 #17018 #17019 
> >> to
> >> 1.6 branch (#17213))
> >> * CPU counterpart to contrib multihead attention operators 
> >> (Interleaved MHA for CPU path (#17138) (#17211))
> >> * Fix for #16060 (fix norm sparse fallback (#17149))
> >> * Fix for inconsistent names in estimator API (fix parameter names 
> >> in
> the
> >> estimator api (#17051) (#17162))
> >> * Fixes for OpenMP (Backport 3rdparty/openmp fixes (#17193))
> >> * Fix for pointwise fusion speed for large networks (which was the
> reason
> >> of -1 in the vote for rc0) as well as fixes for nondeterminism in 
> >> sum of squares operator and trainer parameter order (Backport 
> >> #17002, #17068
> and
> >> #17114 to 1.6 branch (#17137))
> >>
> >>
> >> Please remember to TEST first before voting accordingly:
> >> +1 = approve
> >> +0 = no opinion
> >> -1 = disapprove (provide reason)
> >>
> >>
> >> Best regards,
> >> Przemyslaw Tredak
> >>
> >
>

RE: Proposal for MXNet website improving

[Chen, Ciyong]

Hi Aaron,

Thanks for your valuable feedback.
I'll prepare to contribute this change and PR soon, and update the contents as 
suggested.

Regarding making "Performance" a Key Feature to replace with "Tools & 
Libraries", anything I need to take care when removing "Tools & Libraries" part?

Thanks!
-Ciyong

-Original Message-
From: Aaron Markham  
Sent: Saturday, December 21, 2019 4:14 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: Proposal for MXNet website improving

Hi Ciyong, thanks for the proposal.
I like your suggestions. Will you be submitting a PR?

Some feedback:

* Regarding changing the URLs, let's avoid that. We just had a lot of work 
trying to fix broken links.
* As far as changing the headings, sure, Tutorials and FAQs makes sense.
* Adding performance as a nav item - my preference and going from UX 
guidelines, is to keep the number of them down to less than five or six.
- What about making performance a Key Feature and highlighting that on the 
main page? I'd switch it with Tools & Libraries since Ecosystem is the next 
thing below.

Cheers,
Aaron

On Thu, Dec 19, 2019 at 2:03 AM Chen, Ciyong  wrote:
>
> Hi MXNet community,
>
> While doing search for MXNet from the official 
> website[https://mxnet.incubator.apache.org/], it's not that convenient to get 
> the recent/latest performance data, besides there's some mismatch between the 
> link and description in the current websites.
> We can also add some new contents (like distributed training via Horovod, and 
> AMP with bfloat16 data type) descriptions in FAQ section.
>
> So I propose to improve the current website structure from below 3 
> areas
>
> 1.Add a new Tab "Performance" in the header, and change 
> "Doc" to "Tutorials" according to the current contents.
>
> 2.Align description of FAQ section to the inner page.
>
> 3.FAQ list adjustment
>
> Please check the details via below link 
> https://drive.google.com/open?id=1gQrC1V1LeJH5NT6zRqBl8Ub2qSr1dc8O
>
> Suggestions and comments are highly appreciated.
>
> Thanks!
> -Ciyong
>

Proposal for MXNet website improving

[Chen, Ciyong]

Hi MXNet community,

While doing search for MXNet from the official 
website[https://mxnet.incubator.apache.org/], it's not that convenient to get 
the recent/latest performance data, besides there's some mismatch between the 
link and description in the current websites.
We can also add some new contents (like distributed training via Horovod, and 
AMP with bfloat16 data type) descriptions in FAQ section.

So I propose to improve the current website structure from below 3 areas

1.Add a new Tab "Performance" in the header, and change "Doc" 
to "Tutorials" according to the current contents.

2.Align description of FAQ section to the inner page.

3.FAQ list adjustment

Please check the details via below link
https://drive.google.com/open?id=1gQrC1V1LeJH5NT6zRqBl8Ub2qSr1dc8O

Suggestions and comments are highly appreciated.

Thanks!
-Ciyong

RE: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc0

[Chen, Ciyong]

Appreciate Tredak to push out voting for 1.6 release.

+1 as we've done lots of tests with expected performance in many different 
scenarios including both single-node and multi-node (horovod based), both FP32 
and INT8 precision on many topologies.

-Ciyong

-Original Message-
From: Zhao, Patric  
Sent: Tuesday, December 17, 2019 8:51 AM
To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
Subject: RE: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc0

Thanks, Tredak, I will add some words for the new feature in the release note.

+1 for voting because we have ran multiple time of tests in local and got the 
expected performance boost.

--Patric

> -Original Message-
> From: Przemysław Trędak 
> Sent: Tuesday, December 17, 2019 4:49 AM
> To: d...@mxnet.apache.org
> Subject: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc0
> 
> Dear MXNet community,
> 
> This is the vote to release Apache MXNet (incubating) version 1.6.0. 
> Voting starts now and will close on Friday, 20th December 2019 23:59:59 PST.
> 
> Link to release notes:
> https://cwiki.apache.org/confluence/display/MXNET/1.6.0+Release+notes
> 
> Link to release candidate:
> https://github.com/apache/incubator-mxnet/releases/tag/1.6.0.rc0
> 
> Link to source and signatures on apache dist server:
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.6.0.rc0/
> 
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
> 
> Additional notes:
>  - There was an issue[1] raised that 1.6.0.rc0 does not build with 
> clang on FreeBSD - I decided to not block the voting for this and 
> instead let the Community decide whether this is a blocker for the release.
>  - Patric Zhao and Tao Lv - could you help preparing a paragraph on 
> MKLDNN
> 1.0 update in the New features section in the release notes?
> 
> [1] https://github.com/apache/incubator-mxnet/issues/17076
> 
> Best regards,
> Przemyslaw Tredak

RE: RE: MXNet 1.6.0 release

[Chen, Ciyong]

Many thanks @ Przemek for leading 1.6 release process!
Can you share the latest status of 1.6 release as we don't see any updates for 
a while.
Any plan or ETA for the following voting?

Thanks!
-Ciyong

-Original Message-
From: Zhao, Patric  
Sent: Monday, November 18, 2019 2:10 PM
To: dev@mxnet.incubator.apache.org; d...@mxnet.apache.org
Subject: RE: RE: MXNet 1.6.0 release

Plan to cherry-pick below PR into 1.6.  Please take a review.

https://github.com/apache/incubator-mxnet/pull/16837

> -Original Message-
> From: Tan, Jonathan 
> Sent: Monday, November 18, 2019 9:43 AM
> To: d...@mxnet.apache.org
> Subject: Re: RE: MXNet 1.6.0 release
> 
> Hi MXNet Community,
> 
> I’ve been doing some testing on the performance of MXnet 1.6.x vs 
> 1.5.1 and I noticed some regression in training. You can find more details 
> here:
> https://github.com/apache/incubator-mxnet/issues/16845
> 
> Thanks,
> Jonathan
> 
> On 2019/11/14 02:41:26, "Zhao, Patric"
> mailto:p...@intel.com>> wrote:
> > @Przemek, what's the status of 1.6 release? >
> >
> >
> >
> > Do we have an ETA for the voting in dev@ and general@?>
> >
> >
> >
> > Thanks,>
> >
> >
> >
> > --Patric>
> >
> >
> >
> > > -Original Message->
> >
> > > From: Chaitanya Bapat mailto:ch...@gmail.com>>>
> >
> > > Sent: Friday, November 8, 2019 2:19 PM>
> >
> > > To:
> dev@mxnet.incubator.apache.org >>
> >
> > > Cc: d...@mxnet.apache.org>
> >
> > > Subject: Re: MXNet 1.6.0 release>
> >
> > > >
> >
> > > Thanks Przemyslaaw for leading and managing the 1.6 release!>
> >
> > > >
> >
> > > Moreover, thanks for clarifying the difference between code-freeze 
> > > and
> release>
> >
> > > candidate.>
> >
> > > >
> >
> > > Currently, log_softmax for Large Tensor would fail. It is fixed in 
> > > this PR by
> Hao ->
> >
> > > https://github.com/apache/incubator-mxnet/pull/16711>
> >
> > > It would be great to have that cherry-picked.>
> >
> > > Thanks>
> >
> > > Chai>
> >
> > > >
> >
> > > >
> >
> > > On Thu, 7 Nov 2019 at 17:33, Zhao, Patric
> mailto:pa...@intel.com>> wrote:>
> >
> > > >
> >
> > > > Thanks for the great efforts.>
> >
> > > >>
> >
> > > > I think below PR need to be backported to 1.6 for bugfix in 
> > > > large>
> >
> > > > tensor supports.>
> >
> > > > https://github.com/apache/incubator-mxnet/pull/16737>
> >
> > > >>
> >
> > > > --Patric>
> >
> > > >>
> >
> > > >>
> >
> > > > > -Original Message->
> >
> > > > > From: Przemysław Trędak
> mailto:pt...@apache.org>>>
> >
> > > > > Sent: Friday, November 8, 2019 5:46 AM>
> >
> > > > > To: d...@mxnet.apache.org>
> >
> > > > > Subject: Re: MXNet 1.6.0 release>
> >
> > > > >>
> >
> > > > > Dear MXNet Community,>
> >
> > > > >>
> >
> > > > > From talking to different Members of the Community, I realized 
> > > > > there>
> >
> > > > > is a misunderstanding of what "code freeze" actually means. 
> > > > > Let me>
> >
> > > > > try to>
> >
> > > > clear>
> >
> > > > > this confusion in this email.>
> >
> > > > >>
> >
> > > > > The code freeze does not mean "1.6 release is done, let's vote 
> > > > > on it>
> >
> > > > > and>
> >
> > > > ship>
> >
> > > > > it as-is". As some of You probably noticed, I did not tag a 
> > > > > RC0 yet.>
> >
> > > > That is>
> >
> > > > > because code freeze means "there are no more new features 
> > > > > going
> to>
> >
> > > > > be accepted in order to provide stable base for finding and 
> > > > > fixing>
> >
> > > > > bugs". I>
> >
> > > > know>
> >
> > > > > of a few showstopper issues that need to be tackled before a 
> > > > > release>
> >
> > > > > candidate can be made (mentioned in the previous email), so 
> > > > > tagging>
> >
> > > > > a release candidate would not really make sense.>
> >
> > > > >>
> >
> > > > > I would like to repeat my call for action to test the 
> > > > > release,>
> >
> > > > > create>
> >
> > > > issues and>
> >
> > > > > tag me on issues which need to be prioritized for the 1.6 
> > > > > release,>
> >
> > > > > as>
> >
> > > > well as>
> >
> > > > > help fixing those issues (the fixes will be cherry-picked to 
> > > > > 1.6.x>
> >
> > > > branch).>
> >
> > > > >>
> >
> > > > > Thank you>
> >
> > > > > Przemek>
> >
> > > > >>
> >
> > > > > On 2019/11/02 03:11:55, Przemys  aw Tr  dak
> mailto:pt...@apache.org>>>
> >
> > > > > wrote:>
> >
> > > > > > Dear MXNet Community,>
> >
> > > > > >>
> >
> > > > > > This morning I updated the 1.6.x branch and so the code 
> > > > > > freeze is>
> >
> > > > > > in>
> >
> > > > effect.>
> >
> > > > > I would like to thank everyone who helped in preparing and 
> > > > > reviewing>
> >
> > > > > pull requests to meet this deadline.>
> >
> > > > > >>
> >
> > > > > > Unfortunately, nightly tests do not currently pass (I 
> > > > > > created an>
> >
> > > > > > issue>
> >
> > > > about>
> >
> > > > > this: [1]). Another issue [2] was raised to my attention as>
> >
> > > > > potential>
> >

RE: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc1

[Chen, Ciyong]

dule: {0: 0.05, 82: 0.005001, 123: 0.0005, 300:
> > 0.0001} Epoch 0, Changed learning rate to 0.05 [22:49:42]
> > ../src/operator/nn/mkldnn/mkldnn_base.cc:74: Allocate
> > 147456 bytes with malloc directly
> > [22:49:42] ../src/operator/nn/mkldnn/mkldnn_base.cc:74: Allocate
> > 589824 bytes with malloc directly
> > [22:49:42] ../src/operator/nn/mkldnn/mkldnn_base.cc:74: Allocate
> > 2359296 bytes with malloc directly
> > [22:49:42] ../src/operator/nn/mkldnn/mkldnn_base.cc:74: Allocate
> > 9437184 bytes with malloc directly
> > Epoch 0, Batch 199, Speed=426.182733 Epoch 0, Duration=134.868458 
> > Epoch 0, Training accuracy=0.127238 Epoch 0, Validation 
> > accuracy=0.206388 Epoch 1, Batch 199, Speed=313.127156 Epoch 1, 
> > Duration=128.041775 Epoch 1, Training accuracy=0.182065 Epoch 1, 
> > Validation accuracy=0.202524 Epoch 2, Batch 199, Speed=410.931187 
> > Epoch 2, Duration=124.920588 Epoch 2, Training accuracy=0.202584 
> > Epoch 2, Validation accuracy=0.245693 Epoch 3, Batch 199, 
> > Speed=419.119335 Epoch 3, Duration=120.948349 Epoch 3, Training 
> > accuracy=0.235854 Epoch 3, Validation accuracy=0.291066 Epoch 4, 
> > Batch 199, Speed=430.473733 Epoch 4, Duration=130.181724 Epoch 4, 
> > Training accuracy=0.257773 Epoch 4, Validation accuracy=0.304988
> >
> > real11m7.356s
> > user406m9.910s
> > sys 14m18.349s
> > [23:00:49] ../src/io/iter_image_recordio_2.cc:172:
> > ImageRecordIOParser2:
> > /home/piotr/deeplearning-benchmark/data/cifar/train.rec, use 4 
> > threads for decoding..
> > [23:00:49] ../src/io/iter_image_recordio_2.cc:230: Load mean image 
> > from /home/piotr/deeplearning-benchmark/data/cifar/mean.bin
> > [23:00:49] ../src/io/iter_image_recordio_2.cc:248: Load mean image 
> > from /home/piotr/deeplearning-benchmark/data/cifar/mean.bin
> completed
> > [23:00:49] ../src/io/iter_image_recordio_2.cc:172:
> > ImageRecordIOParser2:
> > /home/piotr/deeplearning-benchmark/data/cifar/test.rec, use 4 
> > threads for decoding..
> > [23:00:49] ../src/io/iter_image_recordio_2.cc:230: Load mean image 
> > from /home/piotr/deeplearning-benchmark/data/cifar/mean.bin
> > [23:00:49] ../src/io/iter_image_recordio_2.cc:248: Load mean image 
> > from /home/piotr/deeplearning-benchmark/data/cifar/mean.bin
> completed
> > lr_schedule: {0: 0.05, 82: 0.005001, 123: 0.0005, 300:
> > 0.0001} Epoch 0, Changed learning rate to 0.05 Epoch 0, Batch 199,
> > Speed=348.618154 Epoch 0, Duration=146.469352 Epoch 0, Training
> > accuracy=0.124121 Epoch 0, Validation accuracy=0.167227 Epoch 1, 
> > Batch 199, Speed=452.790825 Epoch 1, Duration=130.199421 Epoch 1, 
> > Training
> > accuracy=0.183863 Epoch 1, Validation accuracy=0.237079 Epoch 2, 
> > Batch 199, Speed=451.406559 Epoch 2, Duration=126.320823 Epoch 2, 
> > Training
> > accuracy=0.214844 Epoch 2, Validation accuracy=0.244692 Epoch 3, 
> > Batch 199, Speed=403.161873 Epoch 3, Duration=125.331660 Epoch 3, 
> > Training
> > accuracy=0.243506 Epoch 3, Validation accuracy=0.301182 Epoch 4, 
> > Batch 199, Speed=450.826598 Epoch 4, Duration=126.426253 Epoch 4, 
> > Training
> > accuracy=0.266424 Epoch 4, Validation accuracy=0.311899
> >
> > real11m21.930s
> > user415m3.855s
> > sys 13m53.975s
> >
> > On Wed, Jun 26, 2019 at 3:50 PM Pedro Larroy 
> >  wrote:
> > >
> > > Hi Ciyong, thanks for trying to reproduce:
> > >
> > > I used this one:
> > > https://github.com/awslabs/deeplearning-
> benchmark/blob/master/dawnbe
> > > nch/cifar10.py
> > >
> > > Could you provide hardware and OS details?
> > >
> > > I will rerun and repost numbers in a few minutes.
> > >
> > > Pedro.
> > >
> > > On Wed, Jun 26, 2019 at 4:18 AM Chen, Ciyong 
> > > 
> wrote:
> > > >
> > > > Hi Pedro,
> > > >
> > > > I'm looking at this case, and using the script of 
> > > > "incubator-mxnet/example/image-classification/train_cifar10.py" 
> > > > to get
> the timing data, but seems there's not much difference between mxnet
> 1.4.1.rc0 and 1.5.0.rc1 on C5.18xlarge.
> > > >
> > > > Not sure if there's any difference in the python script, can you 
> > > > point me
> the link to get your script (cifar10.py)?
> > > > Or you can also have a try with MXNet's script 
> > > > (train_cifar10.py) and see
> the performance.
> > > >
> > > > Here's the command I used to c

RE: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc1

[Chen, Ciyong]

Hi Pedro,

I'm looking at this case, and using the script of 
"incubator-mxnet/example/image-classification/train_cifar10.py" to get
the timing data, but seems there's not much difference between mxnet 1.4.1.rc0 
and 1.5.0.rc1 on C5.18xlarge.

Not sure if there's any difference in the python script, can you point me the 
link to get your script (cifar10.py)?
Or you can also have a try with MXNet's script (train_cifar10.py) and see the 
performance.

Here's the command I used to collect the time: 
python train_cifar10.py --num-epoch=5

1) 1.5.0.rc1 (4d9667121ae6fb643f2a02ab15e25231ed756cde)
real9m4.880s
user333m13.340s
sys 14m36.100s

2) 1.4.1.rc0 (1a7199691f5cbc6012bb53eecbf884bed5ae6590)
real9m2.155s
user329m37.092s
sys 16m8.668s

-Ciyong


-Original Message-
From: Pedro Larroy [mailto:pedro.larroy.li...@gmail.com] 
Sent: Wednesday, June 26, 2019 6:28 AM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: [VOTE] Release Apache MXNet (incubating) version 1.5.0.rc1

Hi these were my build flags and system info:


--- # CMake configuration
USE_CUDA: "OFF" # Build with CUDA support
USE_OLDCMAKECUDA: "OFF" # Build with old cmake cuda
USE_NCCL: "OFF" # Use NVidia NCCL with CUDA
USE_OPENCV: "ON" # Build with OpenCV support
USE_OPENMP: "ON" # Build with Openmp support
USE_CUDNN: "ON" # Build with cudnn support) # one could set CUDNN_ROOT for 
search path
USE_SSE: "ON" # Build with x86 SSE instruction support IF NOT ARM
USE_F16C: "ON" # Build with x86 F16C instruction support) # autodetects support 
if "ON"
USE_LAPACK: "ON" # Build with lapack support
USE_MKL_IF_AVAILABLE: "ON" # Use MKL if found
USE_MKLML_MKL: "ON" # Use MKLDNN variant of MKL (if MKL found) IF 
USE_MKL_IF_AVAILABLE AND (NOT APPLE)
USE_MKLDNN: "ON" # Use MKLDNN variant of MKL (if MKL found) IF 
USE_MKL_IF_AVAILABLE AND (NOT APPLE)
USE_OPERATOR_TUNING: "ON" # Enable auto-tuning of operators IF NOT MSVC
USE_GPERFTOOLS: "ON" # Build with GPerfTools support (if found)
USE_JEMALLOC: "ON" # Build with Jemalloc support
USE_PROFILER: "ON" # Build with Profiler support
USE_DIST_KVSTORE: "OFF" # Build with DIST_KVSTORE support
USE_PLUGINS_WARPCTC: "OFF" # Use WARPCTC Plugins
USE_PLUGIN_CAFFE: "OFF" # Use Caffe Plugin
USE_CPP_PACKAGE: "OFF" # Build C++ Package
USE_MXNET_LIB_NAMING: "ON" # Use MXNet library naming conventions.
USE_GPROF: "OFF" # Compile with gprof (profiling) flag
USE_CXX14_IF_AVAILABLE: "OFF" # Build with C++14 if the compiler supports it
USE_VTUNE: "OFF" # Enable use of Intel Amplifier XE (VTune)) # one could set 
VTUNE_ROOT for search path
ENABLE_CUDA_RTC: "ON" # Build with CUDA runtime compilation support
BUILD_CPP_EXAMPLES: "ON" # Build cpp examples
INSTALL_EXAMPLES: "OFF" # Install the example source files.
USE_SIGNAL_HANDLER: "ON" # Print stack traces on segfaults.
USE_TENSORRT: "OFF" # Enable infeference optimization with TensorRT.
USE_ASAN: "OFF" # Enable Clang/GCC ASAN sanitizers.
ENABLE_TESTCOVERAGE: "OFF" # Enable compilation with test coverage metric output
CMAKE_BUILD_TYPE: "Release"
CMAKE_CUDA_COMPILER_LAUNCHER: "ccache"
CMAKE_C_COMPILER_LAUNCHER: "ccache"
CMAKE_CXX_COMPILER_LAUNCHER: "ccache"

commit 4d9667121ae6fb643f2a02ab15e25231ed756cde (HEAD, tag: 1.5.0.rc1,
upstream/v1.5.x)
commit 1a7199691f5cbc6012bb53eecbf884bed5ae6590 (HEAD, tag: 1.4.1.rc0,
upstream/v1.4.x)

curl http://169.254.169.254/latest/meta-data/instance-type
c5d.18xlarge


Version  : 3.6.7
Compiler : GCC 8.2.0
Build: ('default', 'Oct 22 2018 11:32:17')
Arch : ('64bit', 'ELF')
Pip Info---
Version  : 19.1.1
Directory: /home/piotr/mxnet_1.5/py3_venv/lib/python3.6/site-packages/pip
--MXNet Info---
Version  : 1.5.0
Directory: /home/piotr/mxnet_1.5/python/mxnet
Hashtag not found. Not installed from pre-built package.
--System Info--
Platform : Linux-4.15.0-1035-aws-x86_64-with-Ubuntu-18.04-bionic
system   : Linux
node : ip-172-31-63-171
release  : 4.15.0-1035-aws
version  : #37-Ubuntu SMP Mon Mar 18 16:15:14 UTC 2019
--Hardware Info--
machine  : x86_64
processor: x86_64
Architecture:x86_64
CPU op-mode(s):  32-bit, 64-bit
Byte Order:  Little Endian
CPU(s):  72
On-line CPU(s) list: 0-71
Thread(s) per core:  2
Core(s) per socket:  18
Socket(s):   2
NUMA node(s):2
Vendor ID:   GenuineIntel
CPU family:  6
Model:   85
Model name:  Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
Stepping:4
CPU MHz: 1326.446
BogoMIPS:6000.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:   32K
L1i cache:   32K
L2 cache:1024K
L3 cache:25344K
NUMA node0 CPU(s):   0-17,36-53
NUMA node1 CPU(s):   18-35,54-71
Flags:   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr