Hey Ciyong, Any update from the ASF mentors/legal team re: multiple license header issue? I can see the PR for checking Valid license header merged: https://github.com/apache/incubator-mxnet/pull/18478 So if we get the multiple license header issue fixed, we can get 1.7.0 release going..
Are we blocked somewhere? Thanks Chai On Sat, 13 Jun 2020 at 06:32, Chen, Ciyong <ciyong.c...@intel.com> wrote: > Hi Leonard, > > Thanks for your confirmation on the build issue. As it's not a blocker for > 1.7 release now, then we can consider to backport the fix to 1.7.x branch > when it's ready. > The only remaining item is how to deal with the multiple license header > now, thank you for helping on thisπ > > Thanks, > -Ciyong > > -----Original Message----- > From: Leonard Lausen <lau...@apache.org> > Sent: Saturday, June 13, 2020 1:10 AM > To: dev@mxnet.incubator.apache.org > Subject: Re: Updates for 1.7.0 minor release > > Thank you Ciyong. After further investigation, the build issue is not as > severe as initially claimed on Github. I checked the high-water memory > usage during single-process build: It's 2.7GB on master. On 1.7 release, > high-level usage is 2.2GB. This is much more acceptable than the previously > claimed >16GB usage and thus not a blocking issue from my perspective. I'll > later also report the numbers for 1.5 and 1.6. > > Fixing the respective implementations to be more compiler-friendly would > still be good. > > Looking at the parallel-build high-level memory usage on a 96 core > machine, I saw a 45% memory usage increase during build from 1.5 to 1.7. > > Best regards > Leonard > > > On Fri, 2020-06-12 at 02:09 +0000, Chen, Ciyong wrote: > > Hi Chai, > > > > Sorry for the late update. > > > > Recently, several bug fixes [4] including numpy operator/batchnorm > > gradient/LSTM CPU gradient/CI/CD/license issues were back-ported into > v1.7.x. > > So far, there's one build issue and two license issues being tracked. > > 1) build issue #18501 (It costs over 16GB memory to compile > > indexing_op.o), which @leezu stated it's a blocker for the release[1]. > > 2) license issue: multiple license header issue[2] is under > > discussion; no valid apache license header issue[3] is identified, and > > I'm working on the PR as @szha suggested. > > > > If the community can help to expedite the item of [1] and [2], it will > > be great helpful. > > Once we've completed the above items and no more other critical > > issues, it's ok to cut the rc0. > > > > Thanks for your patients. > > > > Thanks, > > -Ciyong > > > > [1] > > https://github.com/apache/incubator-mxnet/issues/18501#issuecomment-64 > > 2785535 > > [2] > > https://github.com/apache/incubator-mxnet/issues/17329#issuecomment-64 > > 1311199 > > [3] > > https://github.com/apache/incubator-mxnet/pull/18478#issuecomment-6424 > > 62904 > > [4] PR list: > > #18358/#18339/#18311/#18352/#18456/#18316/#18482/#18502/#18517/#18464 > > > > > > > > -----Original Message----- > > From: Chaitanya Bapat <chai.ba...@gmail.com> > > Sent: Friday, June 12, 2020 1:34 AM > > To: dev@mxnet.incubator.apache.org > > Subject: Re: RE: Updates for 1.7.0 minor release > > > > Hey Ciyong, > > > > Since the last discussion, the GPU memory regression PR has been > reverted. > > Is there any update for when the rc0 for 1.7 will be cut? > > Can the community help expedite the process in any way? > > > > Thanks > > Chai > > > > On Wed, 13 May 2020 at 18:28, Chen, Ciyong <ciyong.c...@intel.com> > wrote: > > > > > Hi Ziyi, > > > > > > Thanks for reaching me for the known/found issue in the upcoming > > > release, let's fix all these potential issues before dropping the > > > rc0 tag π > > > I'll ask help from Tao to merge the PR. > > > > > > Thanks, > > > -Ciyong > > > > > > -----Original Message----- > > > From: Patrick Mu <zm2...@columbia.edu> > > > Sent: Thursday, May 14, 2020 8:58 AM > > > To: d...@mxnet.apache.org > > > Subject: Re: RE: Updates for 1.7.0 minor release > > > > > > Hi Ciyong, > > > > > > We found a GPU memory usage regression issue triggered by PR > > > https://github.com/apache/incubator-mxnet/pull/17767, which was > > > pushed to both 2.0, 1.x and 1.7 branches > > > > > > I have reverted this commit in 2.0, but we should revert this in 1.x > > > and > > > 1.7 branches. I have made a reverting PR on 1.x > > > https://github.com/apache/incubator-mxnet/pull/18309. > > > > > > I am thinking if you can help to merge the reverting into 1.x and > > > 1.7 before making the rc0 tag? > > > > > > Thanks, > > > Ziyi > > > > > > On 2020/05/12 00:58:22, "Chen, Ciyong" <ciyong.c...@intel.com> wrote: > > > > Hi Chai, > > > > > > > > Thanks a lot for your kindly help to fix this π > > > > I will continue the rest steps of release process. > > > > > > > > Thanks, > > > > -Ciyong > > > > > > > > -----Original Message----- > > > > From: Chaitanya Bapat <chai.ba...@gmail.com> > > > > Sent: Tuesday, May 12, 2020 8:14 AM > > > > To: dev@mxnet.incubator.apache.org > > > > Subject: Re: Updates for 1.7.0 minor release > > > > > > > > Hello Ciyong, > > > > > > > > With the https://github.com/apache/incubator-mxnet/pull/18261 > > > > merged, > > > nightly pipeline passes for 1.7.x So as far as the 2 nightly test > > > pipelines are concerned [NightlyTests and NightlyTestsForBinaries] > > > 1.7.x is good to go! > > > > Thanks, > > > > Chai > > > > > > > > On Sun, 10 May 2020 at 04:53, Chen, Ciyong <ciyong.c...@intel.com> > > > wrote: > > > > > Hi MXNet Community, > > > > > > > > > > Here's some updates after the code freeze. > > > > > 1. Nightly tests[1] and nightly binaries tests[2] were enabled, > > > > > many thanks to Chaitanya who helped to create and activate these > > > > > jobs for v1.7.x branch. > > > > > 2. A nightly test failure (incorrect with_seed path) was fixed > > > > > by Chaitanya [3] 3. A bug fix for external graph pass by Sam [4] 4. > > > > > Recently, there's another failed cased > > > > > (test_large_vector.test_nn) in nightly test[5], and Chaitanya is > > > > > helping to address this issue[6] > > > > > > > > > > I'll keep monitoring the nightly test before making a rc0 tag. > > > > > Please let me know if you have any other issues that should be > > > > > included/fixed in this release. > > > > > > > > > > Thanks, > > > > > -Ciyong > > > > > > > > > > ----------- > > > > > [1] > > > > > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/N > > > > > ig > > > > > ht > > > > > ly > > > > > Tests/job/v1.7.x/ > > > > > [2] > > > > > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/N > > > > > ig > > > > > ht > > > > > ly > > > > > TestsForBinaries/job/v1.7.x/ [3] > > > > > https://github.com/apache/incubator-mxnet/pull/18220 > > > > > [4] https://github.com/apache/incubator-mxnet/pull/18237 > > > > > [5] > > > > > http://jenkins.mxnet-ci.amazon-ml.com/job/NightlyTestsForBinarie > > > > > s/ jo b/ v1.7.x/2/execution/node/232/log/ [6] > > > > > https://github.com/apache/incubator-mxnet/pull/18261 > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Chen, Ciyong <ciyong.c...@intel.com> > > > > > Sent: Sunday, April 26, 2020 3:29 PM > > > > > To: dev@mxnet.incubator.apache.org > > > > > Cc: Marco de Abreu <marco.g.ab...@gmail.com> > > > > > Subject: Code freeze for 1.7.0 minor release > > > > > > > > > > Hi MXNet Community, > > > > > > > > > > Code freeze for 1.7.0 minor release is in effect (last commit: > > > 38e6634)! > > > > > Which means there're no more NEW features going to be accepted > > > > > for this release. > > > > > > > > > > Many thanks to everyone who helped submitting/back > > > > > porting/reviewing the PRs targeting this release. > > > > > I've created a draft Release Notes for 1.7.0 release[1], please > > > > > take a review, any comments/suggestions are highly appreciated. > > > > > > > > > > Currently, the nightly test pipeline [2][3] for v1.7.x is not > > > > > triggered, cc @Marco de Abreu <marco.g.ab...@gmail.com><mailto: > > > > > marco.g.ab...@gmail.com> to help take a look. > > > > > I will keep monitoring the nightly test result for the current > > > > > code base, and continue to go through the rest of releasing > process. > > > > > > > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.7.0+Release+ > > > > > No > > > > > te > > > > > s > > > > > [2] > > > > > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/N > > > > > ig > > > > > ht > > > > > ly > > > > > Tests/job/v1.7.x/ > > > > > [3] > > > > > http://jenkins.mxnet-ci.amazon-ml.com/view/Nightly%20Tests/job/N > > > > > ig > > > > > ht > > > > > ly > > > > > TestsForBinaries/job/v1.7.x/ > > > > > > > > > > > > > > > Thanks, > > > > > -Ciyong > > > > > > > > > > > > > > > > > > -- > > > > *Chaitanya Prakash Bapat* > > > > *+1 (973) 953-6299* > > > > > > > > [image: https://www.linkedin.com//in/chaibapat25] > > > > <https://github.com/ChaiBapchya>[image: > > > > https://www.facebook.com/chaibapat] > > > > <https://www.facebook.com/chaibapchya>[image: > > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya > > > > [image: > > > > https://www.linkedin.com//in/chaibapat25] > > > > <https://www.linkedin.com//in/chaibapchya/> > > > > > > > > -- > > *Chaitanya Prakash Bapat* > > *+1 (973) 953-6299* > > > > [image: https://www.linkedin.com//in/chaibapat25] > > <https://github.com/ChaiBapchya>[image: > > https://www.facebook.com/chaibapat] > > <https://www.facebook.com/chaibapchya>[image: > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya > >[image: > > https://www.linkedin.com//in/chaibapat25] > > <https://www.linkedin.com//in/chaibapchya/> > > -- *Chaitanya Prakash Bapat* *+1 (973) 953-6299* [image: https://www.linkedin.com//in/chaibapat25] <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat] <https://www.facebook.com/chaibapchya>[image: https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image: https://www.linkedin.com//in/chaibapat25] <https://www.linkedin.com//in/chaibapchya/>
