Hi Tao, 5) is not a bug. It's just a large tensor support requirement. The PR was to fix a memory alignment issue introduced in master but not in 1.5.1 (since you did not cherry pick that PR). So, I have crossed out 5) in the doc and I don't think we need to mention it in release note.
Lin On Thu, Aug 29, 2019 at 8:12 AM Tao Lv <ta...@apache.org> wrote: > @Aaron, > Thank you for looking into these two issues. I have removed the #15609 from > the scope of 1.5.1. Please let me know if you have any update about #15608. > > @Lai, > I'm fine with the decision. License issue about MKL-DNN, cub and pybind is > moved to next release. > > @Sam, > I also removed the sidebar issue [3] from the scope of 1.5.1. Besides, I > notice one of your cherry picks is stopped by the CI. Please take a look at > it. Thanks. > > *Nice progress since the last update:* > 1. Per the discussion, we decided to remove #15609, the license issue about > MKL-DNN, cub and pybind, and the sidebar issue [3] from the scope of 1.5.1 > patch release; > 2. 3 fixes [4] [5] [6] were merged into the v1.5.x branch. > > *Opens (suggested owners are highlighted):* > 1. @Aaron is working on #15608 to see if we can have it in v1.5.x; > 2. Two cherry pick PRs [7] [8] cannot pass the CI. I have pinged the > authors to take a look at the CI failures. > 3. @Kellen proposed 5 fixes [9] for TensorRT but till now only 3 are picked > to v1.5.x. Please help to confirm if the other 2 are still needed. > 4. Sorry that I missed the proposal for fixing the nightly build [10] in > previous update. @Lai, can you help to confirm if it's still valid? > 5. @Lin please help to make a conclusion for the GPU OOM issue caused by > topk regression [11]. If it cannot be addressed on v1.5.x branch, I will > remove it from the scope of this release and mark it as a known issue in > the release note. > > Please find the details in > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status > . > > Thanks, > -tao > > [1] https://github.com/apache/incubator-mxnet/pull/15609 > [2] https://github.com/apache/incubator-mxnet/pull/15608 > [3] https://github.com/apache/incubator-mxnet/issues/15200 > [4] https://github.com/apache/incubator-mxnet/pull/16029 > [5] https://github.com/apache/incubator-mxnet/pull/16026 > [6] https://github.com/apache/incubator-mxnet/pull/16028 > [7] https://github.com/apache/incubator-mxnet/pull/15803 > [8] https://github.com/apache/incubator-mxnet/pull/16027 > [9] > > https://github.com/apache/incubator-mxnet/issues/15613#issuecomment-520688668 > [10] > > https://github.com/apache/incubator-mxnet/issues/15613#issuecomment-516937546 > [11] https://github.com/apache/incubator-mxnet/issues/15703 > > > > On Thu, Aug 29, 2019 at 1:06 AM Skalicky, Sam <sska...@amazon.com.invalid> > wrote: > > > Hi Tao, > > > > I just talked with Aaron, lets leave the sidebar issue for later. > > > > I created PRs in the v1.5.x branch to cherry pick the fixes into the > 1.5.1 > > release: > > https://github.com/apache/incubator-mxnet/pull/16027 > > https://github.com/apache/incubator-mxnet/pull/16028 > > > > Thanks for your work on this release! > > Sam > > > > On Aug 28, 2019, at 9:35 AM, Lai Wei <roywei...@gmail.com<mailto: > > roywei...@gmail.com>> wrote: > > > > Hi, > > > > Regrading the license issue[1], we still have item 3, 4, 5 left. > > I think it's better to remove them from 1.5.1 release scope and target > for > > 1.6.0 as it need more time and requires changes that should not go into > > patch release. > > > > > > [1] https://github.com/apache/incubator-mxnet/issues/15542 > > > > Best Regards > > > > Lai > > > > > > On Wed, Aug 28, 2019 at 9:20 AM Aaron Markham <aaron.s.mark...@gmail.com > > <mailto:aaron.s.mark...@gmail.com>> > > wrote: > > > > 5 no. Install page defaults to master so you don't need to pick it. > > 6 probably, but there might be other PRs needed. I'd check out the branch > > and attempt the install across platforms to be sure. > > > > On Wed, Aug 28, 2019, 08:55 Tao Lv <ta...@apache.org<mailto: > > ta...@apache.org>> wrote: > > > > Hi Aaron, > > > > They were proposed to be ported to v1.5.x at the beginning of the > > discussion but I didn't see any action for that. So I'm wondering if > > they're still needed. I asked for that in the last update on 8/20 but > > didn't get a response. > > > > If they're still needed, I hope someone who is more familiar with Julia > > frontend can help to cherry pick the commits to the v1.5.x branch. > > > > thanks, > > -tao > > > > On Wed, Aug 28, 2019 at 11:43 PM Aaron Markham < > > aaron.s.mark...@gmail.com<mailto:aaron.s.mark...@gmail.com>> > > wrote: > > > > I don't see any request for action on the Julia PRs: 5 or 6. > > We didn't put the change in right away because we wanted it to not > > break > > anything. But the changes are needed to make Julia setup more seamless. > > > > What "update" is needed? > > > > > > On Wed, Aug 28, 2019, 08:36 Tao Lv <ta...@apache.org<mailto: > > ta...@apache.org>> wrote: > > > > @Pedro, seems the issue is still open on the master branch. Do you > > still > > think we can have your fix on the 1.5.x branch? > > > > Progress since last update: > > 1. We received several more proposals in the github thread [1]. I > > humbly > > ask the reporters to pick the fixes to the v1.5.x. I will keep > > tracking > > the > > progress and the healthy status of the release branch. > > 2. Thanks to @Lai, the licence issue of julia cat image was fixed on > > the > > master branch and I opened a PR to pick it to v1.5.x [2]. > > 3. The GPU OOM issue was fixed on the master branch by @Lin [3] . But > > there > > is a problem with porting the fix to v1.5.x branch [4]. > > > > Opens: > > 1. https://github.com/apache/incubator-mxnet/pull/15803 still can > > not > > pass > > the CI; > > 2. Call for a update from julia folks about the back porting for [5] > > and > > [6] > > 3. License issue of cub and pybind is still open. @Lai opened a PR > > [7] > > to > > update cub submodule but seems it need more effort than just commit > > id > > update. I suspect that we cannot finish this work in 1.5.1 patch > > release. > > 4. Still no progress for the sidebar issue on web page [8]. > > 5. Call for a conclusion about fixing the GPU OOM issue in 1.5.1 > > > > Besides, I would like to ask if there is any preference for the > > release > > timeline of 1.5.1 patch release? Please share so I can propose the > > time > > for > > code freeze. > > > > Thanks, > > -tao > > > > [1] https://github.com/apache/incubator-mxnet/issues/15613. > > [2] https://github.com/apache/incubator-mxnet/pull/16026 > > [3] https://github.com/apache/incubator-mxnet/pull/15948 > > [4] https://github.com/apache/incubator-mxnet/pull/15999 > > [5] https://github.com/apache/incubator-mxnet/pull/15609 > > [6] https://github.com/apache/incubator-mxnet/pull/15608 > > [7] https://github.com/apache/incubator-mxnet/pull/15963 > > [8] https://github.com/apache/incubator-mxnet/issues/15200 > > > > On Wed, Aug 28, 2019 at 5:50 AM Pedro Larroy < > > pedro.larroy.li...@gmail.com<mailto:pedro.larroy.li...@gmail.com> > > > > wrote: > > > > Ok. I was just asking if we want this fix in 1.5.1 since it > > addresses > > crashes using multiprocessing. The problem with cherry picking is > > that > > the > > patch contains the dynamic load change which shouldn't impact > > anything > > else > > but is not supposed to go in a release branch. > > > > On Tue, Aug 27, 2019 at 1:19 PM Lin Yuan <apefor...@gmail.com<mailto: > > apefor...@gmail.com>> > > wrote: > > > > https://github.com/apache/incubator-mxnet/pull/15762 contains > > some > > unrelated changes which is being reverted. Please do not cherry > > pick > > it > > yet. > > > > On Mon, Aug 26, 2019 at 4:25 PM Pedro Larroy < > > pedro.larroy.li...@gmail.com<mailto:pedro.larroy.li...@gmail.com> > > > > wrote: > > > > There's a fix that I did which seems to still produce crashes > > in > > 1.5 > > for > > some users, which I got notice today and is fixed in master. > > > > Might be useful to put in 1.5.1: > > https://github.com/apache/incubator-mxnet/pull/15762 ? > > > > Pedro. > > > > On Tue, Aug 20, 2019 at 7:49 AM Tao Lv <ta...@apache.org<mailto: > > ta...@apache.org>> > > wrote: > > > > Hi dev, > > > > Here is an update for the 1.5.1 patch release. > > > > 1. Thanks for the effort from whole community, we have cherry > > picked > > a > > bunch of fixes to v1.5.x branch. So far, the branch looks > > healthy: > > > > > > > > > > > > > > > > > > > > > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity/ > > 2. https://github.com/apache/incubator-mxnet/pull/15803 > > cannot > > pass > > the > > CI; > > 3. I hope julia folks can take a look at the back porting for > > https://github.com/apache/incubator-mxnet/pull/15609 and > > https://github.com/apache/incubator-mxnet/pull/15608 - do we > > still > > need > > them? > > 4. License issue of cub and pybind is still not fixed. We > > also > > has > > a > > license issue of a cat image in julia examples. > > https://github.com/apache/incubator-mxnet/issues/15542 > > 5. Still no progress for the sidebar issue: > > https://github.com/apache/incubator-mxnet/issues/15200 > > 6. There is a GPU OOM issue in 1.5.0 release and already root > > caused > > by > > Lin: > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-mxnet/issues/15703#issuecomment-522780492 > > . > > We need decide whether we want to get it fixed in the 1.5.1 > > patch > > release. > > > > Please find details in > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status > > . > > > > Thanks, > > -tao > > > > On Mon, Aug 12, 2019 at 9:57 PM Zhao, Patric < > > patric.z...@intel.com> > > wrote: > > > > Thanks for the explanation, Marco & Tao. Sounds great! > > > > -----Original Message----- > > From: Tao Lv <ta...@apache.org> > > Sent: Monday, August 12, 2019 9:54 PM > > To: dev@mxnet.incubator.apache.org > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > Regarding the open issue, is there default code > > owner/maintainer? > > If > > so, he/she will be the right people to look into the > > issue. > > > > https://github.com/apache/incubator-mxnet/blob/master/CODEOWNERS > > > > > > I have no idea. But the CODEOWNERS is used to receive > > change > > notificaitons, > > not actually indicates the maintainer of a piece of code. > > > > Do we have regularly build, run, functionality and > > performance > > testing > > for > > this release? > > > > > > As Marco mentioned, build, run and functionality of > > v1.5.x > > branch > > are > > tracked > > automatically by the CI for each cherry pick pull request > > and > > the > > nightly tests > > here: > > http://jenkins.mxnet-ci.amazon- > > > > ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity > > . > > I see it's healthy so far. > > > > For performance, Shufan will track CPU performance with > > his > > test > > suite > > and > > send out the report once the branch is frozen. I'm not > > sure > > if > > there > > are > > any > > other performance tests. > > > > On Mon, Aug 12, 2019 at 9:36 PM Marco de Abreu > > <marco.g.ab...@gmail.com> > > wrote: > > > > Hi Patric, > > > > CI should automatically pick up the branch and validate > > it > > as > > usual. > > > > Best regards, > > Marco > > > > Zhao, Patric <patric.z...@intel.com> schrieb am Mo., > > 12. > > Aug. > > 2019, > > 15:22: > > > > It's great works, Tao 😊 > > > > Regarding the open issue, is there default code > > owner/maintainer? > > If > > so, he/she will be the right people to look into the > > issue. > > https://github.com/apache/incubator- > > mxnet/blob/master/CODEOWNERS > > > > Do we have regularly build, run, functionality and > > performance > > testing > > for > > this release? > > > > Thanks, > > > > --Patric > > > > -----Original Message----- > > From: Tao Lv <ta...@apache.org> > > Sent: Monday, August 12, 2019 8:59 PM > > To: dev@mxnet.incubator.apache.org > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > Update: > > > > We're cherry picking fixes from the master to the > > v1.5.x > > branch. > > Some > > of > > them are already merged. Please find details on the > > cwiki > > page: > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl > > an+a > > nd+Status > > > > > > There are still 3 opens: > > 1. Nightly test failure on CI ( > > > > https://github.com/apache/incubator-mxnet/issues/15374 > > ): > > The > > issue > > is > > still > > open. I'm wondering if it has been fixed or not. If > > not, > > is > > there > > anyone > > working on it? > > 2. Broken Sidebar on website API for master and > > 1.5.0 ( > > > > https://github.com/apache/incubator-mxnet/issues/15200 > > ): > > I > > don't > > see > > any > > progress on this issue? Do we still want to include > > it > > into > > 1.5.1 > > patch > > release? > > 3. License issues need to be fixed before 1.6 > > release ( > > > > https://github.com/apache/incubator-mxnet/issues/15542 > > ): > > Currently > > the license issue for code and images is partially > > fixed > > on > > the > > master > > branch and > > will be picked to v1.5.x soon. MKLML license issue > > is > > pushed > > out > > to 1.6 release. But license issue for cub and > > pybind > > is > > still > > open. > > > > Let me know if you any suggestion. Thanks for your > > support! > > > > -tao > > > > > > On Wed, Aug 7, 2019 at 11:03 PM Tao Lv < > > ta...@apache.org > > > > wrote: > > > > > > Update: > > > > Thanks to wkcn's report, Issue #15774 [1] and the > > fix > > #15751 > > [2] > > are added to the scope of 1.5.1 patch release. > > For issue #15703 [3], I'm still waiting from the > > response > > from > > the reporter. > > Issue #15431 [4] was closed as false positive > > report. > > I also included several MKL-DNN backend issues > > reported > > by > > mxnet > > users > > and downstream projects. They are already fixed > > on > > the > > master > > branch. > > > > Please kindly check the full list of issues need > > be > > included > > in > > the > > 1.5.1 patch release: > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl > > an+a > > nd+Status > > > > For issues which are already fixed on the master > > branch, > > we > > will > > start > > to cherry pick the fix commit to the v1.5.x > > branch. > > For > > issues > > which are still open, we will start to track the > > fix > > process. > > > > Thanks for your great support. Let me know if you > > have > > any > > questions or concerns. > > > > -tao > > > > [1] > > https://github.com/apache/incubator-mxnet/issues/15774 > > [2] > > https://github.com/apache/incubator-mxnet/pull/15751 > > [3] > > https://github.com/apache/incubator-mxnet/issues/15703 > > [4] > > https://github.com/apache/incubator-mxnet/issues/15431 > > > > > > On Tue, Aug 6, 2019 at 2:04 PM Tao Lv < > > ta...@apache.org> > > wrote: > > > > > > Per Sam's proposal [1], Issue #15737 [2] and the > > fix > > [3] > > are > > added > > to > > the scope of 1.5.1 patch release. > > > > A friendly reminder: the issue proposing will be > > closed > > before > > 11pm > > 8/7 CST (8am 8/7 PST). After that, we will start > > to > > cherry > > pick > > fixes > > to the v1.5.x branch. > > > > > > [1] > > https://github.com/apache/incubator- > > mxnet/issues/15613#issuecomment-5 > > 18430120 [2] > > > > https://github.com/apache/incubator-mxnet/issues/15737 > > [3] > > https://github.com/apache/incubator-mxnet/pull/15692 > > > > On Thu, Aug 1, 2019 at 4:24 PM Tao Lv < > > ta...@apache.org > > > > wrote: > > > > Hi Sandeep/Lai, > > > > Thank you for the prompt response! > > > > > > https://github.com/apache/incubator-mxnet/issues/15200 > > is > > added > > to > > the list to track the sidebar issue. > > > > On Thu, Aug 1, 2019 at 7:54 AM sandeep > > krishnamurthy > > < > > sandeep.krishn...@gmail.com> wrote: > > > > Thank you Tao and Shufan. > > Sidebar missing bug in API documentation is > > inconvenience > > for > > the > > user. > > It > > would great if we can fix it with 1.5.1 > > > > On Wed, Jul 31, 2019, 10:14 AM Lai Wei < > > roywei...@gmail.com > > > > wrote: > > > > Hi Tao, > > > > Thank you so much for driving it. Currently > > nightly > > test > > on > > tutorials are > > failing and it need to be fixed. [3] I have > > updated > > the > > issue[1] and cwiki.[2] > > > > [1] > > https://github.com/apache/incubator-mxnet/issues/15613 > > [2] > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl > > a > > n+and+Status > > [3] > > https://github.com/apache/incubator-mxnet/issues/15374 > > > > Best Regards > > > > Lai > > > > > > On Wed, Jul 31, 2019 at 8:04 AM Tao Lv < > > ta...@apache.org> > > wrote: > > > > Hi community, > > > > > > > > Thanks for the initiative from Sam > > (samskalicky@github > > ), > > we already > > have a > > discussion thread [1] on github about the > > defects > > and > > bugs exposed > > in the > > 1.5.0 release. > > > > Shufan (juliusshufan@github) and I > > (TaoLv@github) > > would > > like > > to > > manage > > the > > release of 1.5.1. This will be our first > > debut > > on > > the > > release > > process, > > your > > comments are always valuable. > > > > > > > > Per the SemVer 2.0 [2], MXNet 1.5.1 will > > be > > a > > patch > > release which > > contains > > backwards-compatible fixes only. > > > > I have created a page on cwiki [3] to > > track > > the > > release > > process > > and > > moved > > the issues and PRs mentioned in the github > > discussion > > thread > > to > > the > > page. > > > > > > > > Here I would like to ask the community to: > > > > (1) Raise any other defect or regression > > you > > identified > > in the > > 1.5.0 release. Please file a github issue > > for > > it > > and > > note > > the issue > > number in > > this thread; > > > > (2) Please comment with one sentence for > > why > > you > > think > > the issue is critical and must have in the > > 1.5.1 > > release; > > > > (3) If the issue is already fixed on > > master > > branch > > or > > already have > > a PR > > WIP, please also note the fix commit id or > > PR > > number; > > > > (4) If the issue is still open and there > > is > > no > > PR > > WIP, > > please > > indicate > > whether you'd be willing to help it out; > > > > (5) Feel free to comment if any other > > suggestion > > for > > the > > release. > > > > > > > > I suggest to keep this thread open for one > > week > > to > > collect enough information and proposals > > before > > we > > decide > > the timeline for the > > release. > > So > > your timely response will be highly > > appreciated! > > > > > > > > PS: Sorry to say that even as a committer, > > this > > is > > the > > first time > > for me > > to > > manage a release. So it would be great if > > an > > experienced > > committer > > can > > help > > to guide the process. > > > > > > > > -tao > > > > > > > > [1] > > > > https://github.com/apache/incubator-mxnet/issues/15613 > > > > [2] https://semver.org/ > > > > [3] > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl > > a > > n+and+Status > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
