Re: [VOTE] Release Apache MXNet (incubating) version 1.4.0.rc3

[Indhu]

+1

Built from source. Verified distributed training of Resnet50 works fine.

On Tue, Feb 19, 2019, 11:43 AM Sheng Zha  -[Y] Are release files in correct location?
> -[Y] Do release files have the word incubating in their name?
> -[Y] Are the digital signature and hashes correct?
> -[Y] Does DISCLAIMER file exist?
> -[Y] Do LICENSE and NOTICE files exists?
> -[Y] Is the LICENSE and NOTICE text correct?
> -[N] Is the NOTICE year correct?
> -[Y] Un-included software dependencies are not mentioned in LICENSE or
> NOTICE? (sz: did not finish checking)
> -[Y] License information is not mentioned in NOTICE?
> Is there any 3rd party code contained inside the release? If so:
> -[Y] Does the software have a compatible license?
> -[Y] Are all software licenses mentioned in LICENSE?
> -[Y] Is the full text of the licenses (or pointers to it) in LICENSE?
> Is any of this code Apache licensed? Do they have NOTICE files? If so:
> -[Y] Have relevant parts of those NOTICE files been added to this NOTICE
> file?
> -[Y] Do all source files have ASF headers?
> -[Y] Do the contents of the release match with what's tagged in version
> control?
> -[N] Are there any unexpected binary files in the release?
> -[Y] Can you compile from source? Are the instruction clear?
>
> +1 with the caveat:
> - NOTICE year was fixed on master but not on the release candidate. rc3
> still reads "2017-2018"
>
> -sz
>
> On 2019/02/19 00:19:52, Roshani Nagmote 
> wrote:
> > +1 Downloaded, installed on Ubuntu 16.04. Verified signatures.
> > Built from source with cuda enabled. Ran train_mnist.py test
> successfully.
> >
> > Thanks,
> > Roshani
> >
> > On Sun, Feb 17, 2019 at 12:13 PM Carin Meier 
> wrote:
> >
> > > +1 Downloaded and verified the signature on the tar. Built and tested
> the
> > > Scala/Clojure package
> > >
> > > On Sun, Feb 17, 2019 at 2:13 PM Qing Lan  wrote:
> > >
> > > > +1 (binding) on the release. Checked Mac + Linux (Ubuntu 16.04) build
> > > from
> > > > source successfully. Checked Scala build with no errors.
> > > >
> > > > On 2/15/19, 6:08 PM, "Piyush Ghai"  wrote:
> > > >
> > > > Dear MXNet community,
> > > >
> > > > I would like to propose a vote to release Apache MXNet
> (incubating)
> > > > version v1.4.0.
> > > > Voting will start today, Friday February 15th 6pm PST and will
> close
> > > > on Monday,
> > > > February 18th 6pm PST.
> > > >
> > > > Link to release notes:
> > > >
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.4.0+Release+Notes
> > > > <
> > > >
> > >
> https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+(incubating)+1.4.0+Release+Notes
> > > > >
> > > >
> > > > Link to release candidate 1.4.0.rc3:
> > > >  <
> https://github.com/apache/incubator-mxnet/releases/tag/1.4.0.rc3/>
> > > > https://github.com/apache/incubator-mxnet/releases/tag/1.4.0.rc3 <
> > > > https://github.com/apache/incubator-mxnet/releases/tag/1.4.0.rc3>/
> > > >
> > > > Link to source and signatures on apache dist server:
> > > >
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.4.0.rc3/ <
> > > > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.4.0.rc3/>
> > > >
> > > >
> > > > Please remember to TEST first before voting accordingly:
> > > > +1 = approve
> > > > +0 = no opinion
> > > > -1 = disapprove (provide reason)
> > > >
> > > >
> > > > Best regards,
> > > > Piyush
> > > >
> > > >
> > >
> >
>

Re: [VOTE] Release Apache MXNet (incubating) version 1.4.0.rc2

[Indhu]

+1

Build from source and tested few examples from the examples folder.

Thanks,
Indu



On Fri, Feb 1, 2019 at 6:21 PM Steffen Rochel 
wrote:

> Hi Sheng - thanks for the feedback.
> TVM notice  file is missing as the 1.4.x branch/v1.4.0 release is using TVM
> commit 0f053c8
> <
> https://github.com/dmlc/tvm/commit/0f053c82a747b4dcdf49570ec87c17e0067b7439
> >
>  from Oct 8, 2018, which didn't have the NOTICE file. IMHO, MXNet NOTICE
> file is consistent with release content.
> As the release started in 2018 I do think it is ok to move forward w/o
> update to 2019 IMHO.
>
> All -
> thanks to the committers/contributors (Tao, Aaron, Kellen, Aston, Yuxi) who
> tested and provided feedback - we have five +1 votes.
> As of today, Friday Feb 1st 2019 6pm PST we have two binding votes, one +1
> (Carin), one +0 (Sheng). The vote continues be open waiting for feedback
> from PMC members.
> Hope you can spare some time over the weekend to provide feedback.
>
> Regards,
> Steffen
>
> On Fri, Feb 1, 2019 at 12:44 AM Marco de Abreu 
> wrote:
>
> > Considering the release process has been started last year and the code
> tag
> > has also been based on last year, I'd say that it is not really a big
> deal.
> >
> > -Marco
> >
> > Am Fr., 1. Feb. 2019, 09:33 hat Sheng Zha 
> > geschrieben:
> >
> > > I found an awesome checklist for incubator releases [1] so I'm using it
> > > here:
> > >
> > > -[Y] Are release files in correct location?
> > > -[Y] Do release files have the word incubating in their name?
> > > -[Y] Are the digital signature and hashes correct?
> > > -[Y] Does DISCLAIMER file exist?
> > > -[Y] Do LICENSE and NOTICE files exists?
> > > -[N/A] Is the LICENSE and NOTICE text correct? (sz: did not finish
> > > checking)
> > > -[N] Is the NOTICE year correct?
> > > -[N/A] Un-included software dependencies are not mentioned in LICENSE
> or
> > > NOTICE? (sz: did not finish checking)
> > > -[Y] License information is not mentioned in NOTICE?
> > > Is there any 3rd party code contained inside the release? If so:
> > > -[Y] Does the software have a compatible license?
> > > -[Y] Are all software licenses mentioned in LICENSE?
> > > -[Y] Is the full text of the licenses (or pointers to it) in LICENSE?
> > > Is any of this code Apache licensed? Do they have NOTICE files? If so:
> > > -[N] Have relevant parts of those NOTICE files been added to this
> NOTICE
> > > file?
> > > TVM has Apache 2.0 license and its NOTICE hasn't been added to MXNet's
> > > NOTICE file.
> > > -[Y] Do all source files have ASF headers? (sz: enforced by license
> > > checker)
> > > -[Y] Do the contents of the release match with what's tagged in version
> > > control?
> > > -[N] Are there any unexpected binary files in the release?
> > > -[Y] Can you compile from source? Are the instruction clear?
> > >
> > > Is the issue minor?
> > > - Unsure. NOTICE year is wrong (it's 2019 now). TVM's NOTICE is missing
> > > from MXNet's NOTICE file.
> > > Could it possibly be fixed in the next release?
> > > - Yes
> > > I vote with:
> > > +0 not sure if it should be released. Could mentors advise if we should
> > fix
> > > them before release?
> > >
> > > [1] https://wiki.apache.org/incubator/IncubatorReleaseChecklist
> > >
> > >
> > > On Thu, Jan 31, 2019 at 10:56 PM Lv, Tao A  wrote:
> > >
> > > >
> > > > +1. Verified below items:
> > > >
> > > > 1. Checkout code from tag 1.4.0rc2 and build mkldnn backend
> > successfully
> > > > on both cpu and gpu w/ mkl and openblas
> > > > 2. ResNet50v1 FP32 performance looks good for both latency and
> > throughput
> > > > 3. Quantization script works well with ResNet50v1
> > > > 4. ResNet50v1 INT8 model accuracy looks good
> > > > 5. ResNet50v1 INT8 model performance speedup looks good for both
> > latency
> > > > and throughput
> > > >
> > > >
> > > > -Original Message-
> > > > From: kellen sunderland [mailto:kellen.sunderl...@gmail.com]
> > > > Sent: Friday, February 1, 2019 11:45 AM
> > > > To: dev@mxnet.incubator.apache.org
> > > > Subject: Re: [VOTE] Release Apache MXNet (incubating) version
> 1.4.0.rc2
> > > >
> > > > Great, thanks Steffen!  I added a few key files but missed that one.
> > > >
> > > > +1 from me.
> > > >
> > > > On Thu, Jan 31, 2019 at 9:35 AM Steffen Rochel <
> > steffenroc...@gmail.com>
> > > > wrote:
> > > >
> > > > > Kellen - Sergey, the 1.4.0 release co-manager signed the tar file.
> > > > > Please use his public key to validate the asc.
> > > > > I was able to validate:
> > > > >
> > > > > curl https://dist.apache.org/repos/dist/dev/incubator/mxnet/KEYS
> -o
> > > > > KEYS
> > > > >
> > > > > gpg --import KEYS
> > > > >
> > > > > gpg --verify apache-mxnet-src-1.4.0.rc2-incubating.tar.gz.asc
> > > > >
> > > > >
> > > > > output:
> > > > >
> > > > > gpg: assuming signed data in
> > > > 'apache-mxnet-src-1.4.0.rc2-incubating.tar.gz'
> > > > >
> > > > > gpg: Signature made Sat Jan 26 16:25:41 2019 PST
> > > > >
> > > > > gpg:using RSA key
> > > > 

Re: [VOTE] Release MXNet version 1.3.0.RC0

[Indhu]

+1

The release candidate looks good. I'm able to build and run basic models.

One the FP16 issue:

Like others have pointed out, releases on expensive in terms of time and
effort. There needs to be a high and more objective bar on what qualifies
as a release blocker to make sure we are not setting precedence for a lot
of release blockers in future.

I think a release blocker is justified only if there is a serious bug
discovered in one of the features included in the release or if there is a
regression. Given FP16 supports is not a new feature claimed in this
release and this is not a regression in this release candidate, I'm
inclined to release this candidate and include the FP16 fix in a subsequent
release.

Thanks,
Indu

On Wed, Sep 5, 2018 at 10:21 AM Aaron Markham 
wrote:

> 0 (non-binding) If we have a problem that blocks users, and a solution in
> hand... then we should fix it, but not at the expense of starting the
> release cycle again just for one fix. Users can cherry pick or build from
> master if they want the fix right away, right? I'd change my mind to -1 if
> this wasn't the case, with good reason, and if the user impact was critical
> to adoption or risks abandonment.
>
>
> On Wed, Sep 5, 2018 at 9:57 AM Roshani Nagmote 
> wrote:
>
> > I believe everyone here is working hard to make MXNet a better framework
> > for users. It's completely okay to have different opinions, we can decide
> > together if this issue is a blocker or not after voting time is over.
> >
> > As I mentioned before, voting will end at 7 pm today. So there is still
> > time to test the release. If there are any other issues anyone finds, I
> > will be happy to start the process again and work on RC1. For now, I want
> > to encourage everyone to utilize this time and vote. :)
> >
> > Thanks,
> > Roshani
> >
> > On Tue, Sep 4, 2018 at 10:35 PM sandeep krishnamurthy <
> > sandeep.krishn...@gmail.com> wrote:
> >
> > >1. As a Apache MXNet community member, I raised the concern of
> broken
> > >functionality for the user. I explained and provided the data points
> > on
> > > the
> > >issue, workaround and why I think it is important. If after all
> this,
> > > you
> > >think my vote is biased on my employer just because a user I quoted
> is
> > > from
> > >Amazon, this is more concerning to me on my voting abilities.
> > >2. My -1 no where undermines the huge amount of effort that goes
> > behind
> > >the scene for a release to happen. Great respect and recognition for
> > >everyone involved in all the releases of MXNet in the past and
> this. I
> > >voted on my judgement of what may be good for the users of MXNet.
> > >3. As pointed by Naveen & Chris, -1 are NOT veto. Feel free to
> decide
> > >and progress on the release as we already have >3 +1 in this thread.
> > >
> > >
> > > Best,
> > >
> > > Sandeep
> > >
> > > On Tue, Sep 4, 2018 at 8:29 PM Chris Olivier 
> > > wrote:
> > >
> > > > btw, there are no vetoes on package releases:
> > > >
> > > > VOTES ON PACKAGE RELEASES
> > > > 
> > > >
> > > > Votes on whether a package is ready to be released use majority
> > approval
> > > > 
> --
> > > i.e.
> > > > at least three PMC members must vote affirmatively for release, and
> > there
> > > > must be more positive than negative votes.Releases may not be vetoed.
> > > > Generally
> > > > the community will cancel the release vote if anyone identifies
> serious
> > > > problems, but in most cases the ultimate decision, lies with the
> > > individual
> > > > serving as release manager. The specifics of the process may vary
> from
> > > > project to project, but the 'minimum quorum of three +1 votes' rule
> is
> > > > universal.
> > > >
> > > > On Tue, Sep 4, 2018 at 7:12 PM Sheng Zha  wrote:
> > > >
> > > > > Thanks for sharing your opinions, Thomas. Your recognition and
> > respect
> > > of
> > > > > people's efforts on preparing the release candidate are certainly
> > > > > appreciated.
> > > > >
> > > > > Now that the vote is set to fail thanks to the veto, there will be
> > > plenty
> > > > > of opportunities to include those bug fixes, including the one Zhi
> > > > > mentioned [1], which was already merged in the master and yet chose
> > not
> > > > to
> > > > > block this release with [2]. I will be happy to work with Roshani
> to
> > > > > prepare another release candidate once ready.
> > > > >
> > > > > -sz
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/f02e952bec22c82cb00a6741390a78f55373311c97464997bb455a6c@%3Cdev.mxnet.apache.org%3E
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/85d3fcabb3437ba7f1af455cf69aa13eb3afd1ea1d1f6f891e9c339c@%3Cdev.mxnet.apache.org%3E
> > > > >
> > > > > On Tue, Sep 4, 2018 at 6:02 PM Thomas DELTEIL <
> > > 

Re: [VOTE] Subscribe dev@ to Github Activities

[Indhu]

Some mentors/contributors/committees feel that the amount of discussions in
dev list is too less given the amount of commits that happen and more
discussions need to happen in the dev list to grow the community.

In response some committees feel discussions actually happen in GitHub PRs.
If the policy says "if it didn't happen in dev, it didn't happen", let's
forward all GitHub discussions to dev so those discussions would count.
That's the motivation for this vote.

I think when people say there needs to be more discussions in the dev list,
I assume they mean the kind of discussions that happen *before* a PR is
created or even before someone starts working on anything. I don't think
people are asking an email for every activity on GitHub. The correct way to
address the problem would be for committees/contributors to stop
communicating in private channels (like Amazon or DMLC communication
channels) and do those discussions in the dev list instead.

Indu


On Wed, Jul 18, 2018, 5:51 AM Barber, Christopher <
christopher.bar...@analog.com> wrote:

> Can't people already subscribe to github notifications? I think it is safe
> to assume that developers are already smart enough to figure out how to do
> that if they want. What problem are you really trying to solve here?
>
> On 7/18/18, 4:49 AM, "Chris Olivier"  wrote:
>
> -1.  (changed from -0.9)
>
> seems more like a strategy (whether intentional or on accident) to
> *not*
> have design discussions on dev by flooding it with noise and then later
> claim it was discussed, even though you would have to sift through
> thousands of emails to find it.
>
>
>
> On Wed, Jul 18, 2018 at 12:42 AM Rahul Huilgol  >
> wrote:
>
> > I pulled up some more stats so we can make an informed decision.
> >
> > Here are some popular Apache projects and the number of emails to
> their
> > dev@
> > list in the last 30 days
> > Apache Flink: 540 mails
> > ​Apache Spark: 249 mails
> > Apache Hive: 481 mails
> > Apache HBase: 300 mails
> >
> > Current dev list for MXNet: 348 mails
> > Current commits list for MXNet: 5329 mails
> > Making the proposed dev list for MXNet to be ~5677 mails.
> >
> > Sheng, even going by your comments that 1 of of those 4 mails are
> relevant
> > for dev@, that's still a really high number of emails. (130 email
> lists
> > doesn't say anything if we ignore the actual number of emails in
> those
> > lists, especially when the 131st sends these many mails :) ). People
> are
> > already talking about setting up filters here. Doesn't that defeat
> the
> > purpose by making people filter out the discussion on Github? People
> can
> > subscribe to commits@ if they find it more convenient to follow
> Github
> > activity over email rather than Github.com.
> >
> > We should strive to maintain dev@ as a place for high quality
> discussion.
> > It's upto the contributors to bring up something to dev@ if they
> believe
> > it
> > deserves a focused discussion in the community. That discussion may
> be
> > started by the person who proposes code changes, or a reviewer who
> believes
> > that a particular code change warrants further discussion.
> >
> > Regards,
> > Rahul
> >
>
>
>

Re: [VOTE] Release MXNet version 1.2.1.RC1

[Indhu]

+1

Built from source. Tested few RNN samples on GPU. Works fine.

On Thu, Jul 12, 2018, 12:01 AM Haibin Lin  wrote:

> +1
> Built from source with cuda and dist kvstore. Ran dist_sync_kvstore.py
> nightly test and it passed.
>
> Best,
> Haibin
>
> On Wed, Jul 11, 2018 at 6:13 PM, Roshani Nagmote <
> roshaninagmo...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > Could you please test and vote for this release? Voting will end tomorrow
> > by 5:50 pm PDT.
> >
> > Thanks,
> > Roshani
> >
> > On Mon, Jul 9, 2018 at 4:53 PM Roshani Nagmote <
> roshaninagmo...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I would like to propose a vote to release Apache MXNet (incubating)
> > > version
> > > 1.2.1.RC1. Voting will start now (Monday, Jul 9th) and end at 5:50 PM
> > > PDT, Thursday, July 12th.
> > >
> > > Link to release candidate 1.2.1.rc1:
> > > *https://github.com/apache/incubator-mxnet/releases/tag/1.2.1.rc1
> > > *
> > >
> > > View this page, click on "Build from Source", and use the source code
> > > obtained from 1.2.1.rc1 tag:
> > > https://mxnet.incubator.apache.org/install/index.html
> > >
> > > (Note: The README.md points to the 1.2.1 tag and does not work at the
> > > moment.)
> > >
> > > Please remember to test first before voting accordingly:
> > >
> > > +1 = approve
> > > +0 = no opinion
> > > -1 = disapprove (provide reason)
> > >
> > > Thanks,
> > > Roshani
> > >
> >
>

Re: [VOTE] Release MXNet version 1.2.1.RC0 (Patch Release)

[Indhu]

+1

On Mon, Jun 18, 2018, 6:52 PM Anirudh  wrote:

> Hi,
>
> This is the vote to release Apache MXNet (incubating) version 1.2.1. Voting
> will start now and close Thursday June 21st 7:00 PM PDT.
>
> Link to release candidate 1.2.1.rc0:
>
> https://github.com/apache/incubator-mxnet/releases/tag/1.2.1.rc0
>
> View this page for installation instructions:
>
> https://mxnet.incubator.apache.org/install/index.html
>
> (Note: The README.md points to the 1.2.1 tag and does not work at the
> moment).
>
> Please remember to test first before voting accordingly.
>
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
>
> Anirudh
>

Re: Nightly tests README accurate?

[Indhu]

I was working on testing the notebooks from https://gluon.mxnet.io/ as part
of the nightly tests to make sure we notice any API change that could break
code in those notebooks.

The README in the nightly folder said the tests are being run on machines
with multiple GPUs. I was wondering if I have to run multiple notebooks in
parallel to efficiently utilize all GPUs. After reading your email, I'm
thinking I'll run most of the notebooks on P3.2xl and only the ones that
require multiple GPUs on P3.8xl. Thanks for the reply.


On Mon, Jun 18, 2018 at 2:24 PM Marco de Abreu
 wrote:

> In future (next few days) the nightlies will be running on our regular CI
> at http://jenkins.mxnet-ci.amazon-ml.com/
>
> On there, we support C5.18xlarge (Ubuntu, CentOS within Docker and
> Windows), G3.8xlarge (Ubuntu, CentOS within Docker and Windows), P3.2xlarge
> (Ubuntu, CentOS within docker) and (upcoming) P3.8xlarge (Ubuntu, CentOS
> within docker).
>
> What exact requirements do you have, Indhu? I'm happy to assist!
>
> -Marco
>
> On Mon, Jun 18, 2018 at 1:47 PM Indhu  wrote:
>
> > Thanks Meghna. Do you know what hardware the nightly tests are currently
> > running on? I'm adding some nightly tests. It will be useful to know.
> >
> >
> > On Mon, Jun 18, 2018 at 9:11 AM Meghna Baijal <
> meghnabaijal2...@gmail.com>
> > wrote:
> >
> > > Hi,
> > > This readme is incorrect and I am fixing this as a part of the nightly
> > > tests PR soon.
> > >
> > > Thanks,
> > > Meghna
> > >
> > > On Sun, Jun 17, 2018 at 7:05 AM kellen sunderland <
> > > kellen.sunderl...@gmail.com> wrote:
> > >
> > > > I'm not a commiter, but I would say this is clearly not correct.  I
> > would
> > > > propose removing the doc until someone has time to verify that the
> > steps
> > > > work correctly, and that the descriptions are accurate.
> > > >
> > > > On Thu, Jun 14, 2018 at 10:04 PM Indhu 
> > wrote:
> > > >
> > > > > Is the README
> > > > > <
> https://github.com/apache/incubator-mxnet/tree/master/tests/nightly
> > >
> > > > for
> > > > > the nightly tests accurate? For example,
> > > > >
> > > > > 1. Are tests being run on machines with Intel i7-4790 and 4 Nvidia
> > GTX
> > > > 970
> > > > > Tis?
> > > > > 2. Is http://ci.dmlc.ml/ the right place to look for build status?
> > > > > 3. Is the instruction to run on Jenkins correct?
> > > > >
> > > > > If not, what all needs to be changed in that page?
> > > > >
> > > >
> > >
> >
>

Re: Nightly tests README accurate?

[Indhu]

Thanks Meghna. Do you know what hardware the nightly tests are currently
running on? I'm adding some nightly tests. It will be useful to know.


On Mon, Jun 18, 2018 at 9:11 AM Meghna Baijal 
wrote:

> Hi,
> This readme is incorrect and I am fixing this as a part of the nightly
> tests PR soon.
>
> Thanks,
> Meghna
>
> On Sun, Jun 17, 2018 at 7:05 AM kellen sunderland <
> kellen.sunderl...@gmail.com> wrote:
>
> > I'm not a commiter, but I would say this is clearly not correct.  I would
> > propose removing the doc until someone has time to verify that the steps
> > work correctly, and that the descriptions are accurate.
> >
> > On Thu, Jun 14, 2018 at 10:04 PM Indhu  wrote:
> >
> > > Is the README
> > > <https://github.com/apache/incubator-mxnet/tree/master/tests/nightly>
> > for
> > > the nightly tests accurate? For example,
> > >
> > > 1. Are tests being run on machines with Intel i7-4790 and 4 Nvidia GTX
> > 970
> > > Tis?
> > > 2. Is http://ci.dmlc.ml/ the right place to look for build status?
> > > 3. Is the instruction to run on Jenkins correct?
> > >
> > > If not, what all needs to be changed in that page?
> > >
> >
>

Re: users@mxnet

[Indhu]

I prefer the discuss forum over email for following reasons:

1. It is easier for newcomers. People can login using Facebook, Twitter or
GitHub Id

2. The format is much more readable for people who search for something in
a search engine and land on the page.

3. Markdown support makes it easier to read code in the discussion.

4. Like button and marking a reply as answer signals the usefulness of an
answer.

That said, if a reasonable number of people like email lists better, I'm
not against it as far as it can co-exist along with the discuss forum.

Thanks,
Indu



On Fri, Jun 15, 2018, 11:23 PM Sergio Fernández  wrote:

> Thanks for your opinion, Tianqi. I still would love to listen others'
> opinion on the topic to really assert anything.
>
> On Fri, Jun 15, 2018, 21:41 Tianqi Chen  wrote:
>
> > Then who should represent the users who are using the forums but not the
> > mail-list? I personally think it is a bit abuse use of the term "Apache
> > way" to force our mind into the entire community... Maybe I am wrong..
> >
> > Tianqi
> >
> > On Fri, Jun 15, 2018 at 9:39 PM, Sergio Fernández 
> > wrote:
> >
> > > Well, I do respect what you discussed in that meetup, if course. But
> for
> > > those who weren't there, maybe the decision taken what a bit bias.
> > >
> > > In Apache we like to say that "if it didn't happen on the mailing list
> s,
> > > it didn't happen" ;-)
> > >
> > > Look like there are different feelings about this. Should I cast a
> VOTE?
> > >
> > >
> > > On Fri, Jun 15, 2018, 21:27 Tianqi Chen 
> > wrote:
> > >
> > > > I do think we are targeting all the community, but we must also agree
> > > that
> > > > the voice of users from the meetup is a representative sample of
> users'
> > > > demand, and it is important that we respect that.
> > > >
> > > > Tianqi
> > > >
> > > > On Fri, Jun 15, 2018 at 9:25 PM, Sergio Fernández  >
> > > > wrote:
> > > >
> > > > > Are we targeting just Seattle as our community? I really hope we
> are
> > > > > thinking a bit beyond that...
> > > > >
> > > > > On Fri, Jun 15, 2018, 21:22 Tianqi Chen 
> > > > wrote:
> > > > >
> > > > > > I remember last time during the mxnet meetup in Seattle, we did a
> > > > survey,
> > > > > > and most users preferred the current discuss forum. So I would
> say
> > we
> > > > > stick
> > > > > > with that given the user community prefers that
> > > > > >
> > > > > > Tianqi
> > > > > >
> > > > > > On Fri, Jun 15, 2018 at 9:13 PM, Sergio Fernández <
> > wik...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Then, if everybody agree, let's request the mailing list
> creation
> > > to
> > > > > > INFRA
> > > > > > > ;-)
> > > > > > >
> > > > > > > Marco, I wouldn't do that. Typically developers are also
> > subscribed
> > > > > > there,
> > > > > > > since they may be the most informed people for answering users'
> > > > > > questions.
> > > > > > > But the topics discussed there may not be of the interest for
> > pure
> > > > > > > development purposes. Some discussions will jump from users@
> to
> > > dev@
> > > > ,
> > > > > > but
> > > > > > > at a different level. So I wouldn't forward one mailing list to
> > the
> > > > > > other.
> > > > > > >
> > > > > > > On Fri, Jun 15, 2018, 21:01 Marco de Abreu
> > > > > > >  wrote:
> > > > > > >
> > > > > > > > I think nobody was opposed to it in the past, right?
> > > > > > > >
> > > > > > > > I'd propose that all emails automatically get copied to dev@
> > to
> > > > > ensure
> > > > > > > > high
> > > > > > > > visibility initially. What do you think?
> > > > > > > >
> > > > > > > > Sebastian  schrieb am Fr., 15. Juni 2018,
> > 20:51:
> > > > > > > >
> > > > > > > > > I have already proposed this many times in the past and
> would
> > > > > > strongly
> > > > > > > > > encourage it.
> > > > > > > > >
> > > > > > > > > -s
> > > > > > > > >
> > > > > > > > > On 15.06.2018 21:56, Sergio Fernández wrote:
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > is there any good reason why the podling doesn't have a
> > > users@
> > > > > > > mailing
> > > > > > > > > list
> > > > > > > > > > yet?
> > > > > > > > > >
> > > > > > > > > > Honestly speaking, I'm not a big fan of the other tools
> the
> > > > > podling
> > > > > > > is
> > > > > > > > > > using. Slack and Web forums a cool tools, and I used
> them a
> > > lot
> > > > > in
> > > > > > > > other
> > > > > > > > > > contexts. But when it comes to transparency and
> community,
> > > > > mailing
> > > > > > > > lists
> > > > > > > > > > play a crucial role in the Apache Way.
> > > > > > > > > >
> > > > > > > > > > Users are the most important asset a project can have.
> Even
> > > > more
> > > > > > than
> > > > > > > > > > developers, believe me. So I think it's time to create a
> > > users@
> > > > > > > > mailing
> > > > > > > > > > list for to helping MXNet grow its community beyong the
> > core
> > > > > team.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > >
> > > > > > > > >
> 

Nightly tests README accurate?

[Indhu]

Is the README
 for
the nightly tests accurate? For example,

1. Are tests being run on machines with Intel i7-4790 and 4 Nvidia GTX 970
Tis?
2. Is http://ci.dmlc.ml/ the right place to look for build status?
3. Is the instruction to run on Jenkins correct?

If not, what all needs to be changed in that page?

Re: [Launch Announcement] Keras 2 with Apache MXNet (incubating) backend

[Indhu]

This is great. Thanks to everybody involved.


On Tue, May 22, 2018, 9:15 AM sandeep krishnamurthy  wrote:

> Hello MXNet community,
>
> Keras users can now use the high-performance MXNet deep learning engine for
> the distributed training of convolutional neural networks (CNNs) and
> recurrent neural networks (RNNs). With an update of a few lines of code,
> Keras developers can increase training speed by using MXNet's multi-GPU
> distributed training capabilities. Saving an MXNet model is another
> valuable feature of the release. You can design in Keras, train with
> Keras-MXNet, and run inference in production, at-scale with MXNet.
>
> From our initial benchmarks, CNNs with Keras-MXNet is up to 3X faster on
> GPUs compared to the default backend. See the benchmark module
>  for
> more details.
>
> RNN support in this release is experimental with few known
> issues/unsupported functionalities. See using RNN with Keras-MXNet
> limitations and workarounds doc
> <
> https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md
> >
> for more details.
>
> See Release Notes
>  for
> more details on unsupported operators and known issues. We will continue
> our efforts in the future releases to close the gaps.
>
> Thank you for all the contributors - Lai Wei ,
> Karan
> Jariwala , Jiajie Chen
> , Kalyanee Chendke <
> https://github.com/kalyc>,
> Junyuan Xie 
>
> We welcome your contributions -
> https://github.com/awslabs/keras-apache-mxnet. Here is the issue with the
> list of operators to be implemented. Do check it out and create a PR -
> https://github.com/awslabs/keras-apache-mxnet/issues/18
>
> Best,
> Sandeep
>

Re: [QUICK VOTE] Release Apache MXNet(incubating) version 1.2.0.RC3

[Indhu]

+1 given this is the same as previous RC with couple of markdowns removed.


On Mon, May 14, 2018, 11:51 PM Anirudh  wrote:

> Hi all,
>
> As part of RC3 release, we have removed the docs (
>
> https://github.com/google/googletest/blob/ec44c6c1675c25b9827aacd08c02433cccde7780/googlemock/docs/DevGuide.md
> and
>
> https://github.com/google/googletest/blob/ec44c6c1675c25b9827aacd08c02433cccde7780/googletest/docs/DevGuide.md
> )
> with CC-BY license from the source tarball.
>
> I would like to propose a vote to release Apache MXNet (incubating) version
> 1.2.0.RC3. This is going to be a quick vote and we will close the vote once
> the quorum is reached.
>
> Link to release notes:
>
> https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.2.0+Release+Notes
>
> Link to release candidate 1.2.0.rc3:
> https://github.com/apache/incubator-mxnet/releases/tag/1.2.0.rc3
>
> Voting results for 1.2.0.rc2 in general:
>
> https://mail-archives.apache.org/mod_mbox/incubator-general/201805.mbox/%3CCAFVLAA-0c9_BfypLJasBraSpO%2B5dtnT1RHXGgyg5j9%3D89_GThw%40mail.gmail.com%3E
>
> View this page, click on "Build from Source" and use the source code
> obtained from 1.2.0.rc3 tag:
>
> https://mxnet.incubator.apache.org/install/index.html
>
> (Note: The README.md points to the 1.2.0 tag and does not work at the
> moment).
>
> Please remember to test first before voting accordingly:
>
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
>
> Anirudh
>

Re: MXNet 1.2 release question ?

[Indhu]

Justin's -1 does not block the release if we can get one more +1. Do we
have one more mentor or someone else who can +1 the release in general list?


On Mon, May 14, 2018 at 1:01 PM, Anirudh  wrote:

> Hi all,
>
> I am going to proceed with option 1. Please let me know now, if you have
> any concerns.
>
> Anirudh
>
> On Mon, May 14, 2018 at 10:11 AM, Anirudh  wrote:
>
> > Hi all,
> >
> > Our vote got blocked on general@ list. The main reason was the inclusion
> > of a file with category b license in non binary form:
> "3rdparty/googletest/
> > googlemock/docs/DevGuide.md".
> > This file needs to be removed from source.
> >
> > There are two options we have right now:
> >
> > 1. Remove the DevGuide.md from source and call for a new vote.
> > 2. Remove the DevGuide.md, fix some other license issues and also fix
> some
> > other issues: compilation issues on older versions of mac 10.10 or older,
> > push other fixes etc.
> >
> > The first option would happen much quicker, since there is no change in
> > source apart from removal of DevGuide.md file and we can close the vote
> > quicker when the quorum is reached.
> >
> > The second option will have to go through the full vote cycle again but
> > will fix some issues with previous RC.
> >
> > I am inclined to do 1 currently since I don't see the compilation issue
> on
> > older version of Mac as a critical one and we are already considerably
> late
> > with the release.
> >
> > I would like to know what you guys think.
> >
> > Anirudh
> >
>

Re: [VOTE] Release Apache MXNet(incubating) version 1.2.0.RC2

[Indhu]

+1

I've been using CUDA build from this branch (built from source) on Ubuntu
for couple of days now and I haven't seen any issue.

The flaky tests need to be fixed but this release need not be blocked for
that.


On Fri, May 4, 2018 at 11:32 AM, Haibin Lin 
wrote:

> I agree with Anirudh that the focus of the discussion should be limited to
> the release branch, not the master branch. Anything that breaks on master
> but works on release branch should not block the release itself.
>
>
> Best,
>
> Haibin
>
> On Fri, May 4, 2018 at 10:58 AM, Pedro Larroy <
> pedro.larroy.li...@gmail.com>
> wrote:
>
> > I see your point.
> >
> > I checked the failures on the v1.2.0 branch and I don't see segfaults,
> just
> > minor failures due to flaky tests.
> >
> > I will trigger it repeatedly a few times until Sunday to have a and
> change
> > my vote accordingly.
> >
> > http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/job/v1.2.0/
> > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/
> > incubator-mxnet/detail/v1.2.0/17/pipeline
> > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/
> > incubator-mxnet/detail/v1.2.0/15/pipeline/
> >
> >
> > Pedro.
> >
> > On Fri, May 4, 2018 at 7:16 PM, Anirudh  wrote:
> >
> > > Hi Pedro,
> > >
> > > Thank you for the suggestions. I will try to reproduce this without
> fixed
> > > seeds and also run it for a longer time duration.
> > > Having said that, running unit tests over and over for a couple of days
> > > will likely cause
> > > problems  because there around 42 open issues for flaky tests:
> > > https://github.com/apache/incubator-mxnet/issues?q=is%
> > > 3Aopen+is%3Aissue+label%3AFlaky
> > > Also, the release branch has diverged from master around 3 weeks back
> and
> > > it doesn't have many of the changes merged to the master.
> > > So, my question essentially is, what will be your benchmark to accept
> the
> > > release ?
> > > Is it that we run the test which you provided on 1.2 without fixed
> seeds
> > > and for a longer duration without failures ?
> > > Or is it that all unit tests should pass over a period of 2 days
> without
> > > issues. This may require fixing all of the flaky tests which would
> delay
> > > the release by considerable amount of time.
> > > Or is it something else ?
> > >
> > > Anirudh
> > >
> > >
> > > On Fri, May 4, 2018 at 4:49 AM, Pedro Larroy <
> > pedro.larroy.li...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Could you remove the fixed seeds and run it for a couple of hours
> with
> > an
> > > > additional loop?  Also I would suggest running the unit tests over
> and
> > > over
> > > > for a couple of days if possible.
> > > >
> > > >
> > > > Pedro.
> > > >
> > > > On Thu, May 3, 2018 at 8:33 PM, Anirudh 
> wrote:
> > > >
> > > > > Hi Pedro and Naveen,
> > > > >
> > > > > I am unable to reproduce this issue with MKLDNN on the master but
> not
> > > on
> > > > > the 1.2.RC2 branch.
> > > > >
> > > > > Did the following on 1.2.RC2 branch:
> > > > >
> > > > > make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_DIST_KVSTORE=0
> > > > > USE_CUDA=0 USE_CUDNN=0 USE_MKLDNN=1
> > > > > export MXNET_STORAGE_FALLBACK_LOG_VERBOSE=0
> > > > > export MXNET_TEST_SEED=11
> > > > > export MXNET_MODULE_SEED=812478194
> > > > > export MXNET_TEST_COUNT=1
> > > > > nosetests-2.7 -v tests/python/unittest/test_
> > > > module.py:test_forward_reshape
> > > > >
> > > > > Was able to do the 10k runs successfully.
> > > > >
> > > > > Anirudh
> > > > >
> > > > > On Thu, May 3, 2018 at 8:46 AM, Anirudh 
> > wrote:
> > > > >
> > > > > > Hi Pedro and Naveen,
> > > > > >
> > > > > > Is this issue reproducible when MXNet is built with USE_MKLDNN=0?
> > > > > > Also, there are a bunch of MKLDNN fixes that didn't go into the
> > > release
> > > > > > branch. Is this issue reproducible on the release branch ?
> > > > > > In my opinion, since we have marked MKLDNN as experimental
> feature
> > > for
> > > > > the
> > > > > > release, if it is confirmed to be a MKLDNN issue
> > > > > > we don't need to block the release on it.
> > > > > >
> > > > > > Anirudh
> > > > > >
> > > > > > On Thu, May 3, 2018 at 6:58 AM, Naveen Swamy  >
> > > > wrote:
> > > > > >
> > > > > >> Thanks for raising this issue Pedro.
> > > > > >>
> > > > > >> -1(binding)
> > > > > >>
> > > > > >> We were in a similar state for a while a year ago, a lot of
> effort
> > > > went
> > > > > to
> > > > > >> stabilize the tests and the CI. I have seen the PR builds are
> > > > > >> non-deterministic and you have to retry over and over (wasting
> > > > resources
> > > > > >> and time) and hope you get lucky.
> > > > > >>
> > > > > >> Look at the dashboard for master build
> > > > > >> http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-
> > > mxnet/job/master/
> > > > > >>
> > > > > >> -Naveen
> > > > > >>
> > > > > >> On Thu, May 3, 2018 at 5:11 AM, Pedro 

Re: [VOTE] Release Apache MXNet (incubating) version 1.2.0.RC0

[Indhu]

+1

Compiled from source on P3 instance. Tested the SSD example and some Gluon
examples.

On Wed, Apr 18, 2018, 7:40 PM Anirudh  wrote:

> Hi everyone,
>
> This is a vote to release Apache MXNet (incubating) version 1.2.0. Voting
> will start now (Wednesday, April 18th) and end at 7:40 PM PDT, Saturday,
> April 21st.
>
> Link to the release notes:
>
>
> https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.2.0+Release+Notes
>
> Link to the release candidate 1.2.0.rc0:
> https://github.com/apache/incubator-mxnet/releases/tag/1.2.0.rc0
>
> View this page, click on “Build from Source”, and use the source code
> obtained from the 1.2.0.rc0 tag:
> https://mxnet.incubator.apache.org/install/index.html
>
> (Note: The README.md points to the 1.2.0 tag and does not work at the
> moment.)
>
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
>
> Thanks,
>
> Anirudh
>

Re: blog for MXNet

[Indhu]

One option is to use a static website generator like Jekyll. Advantage is
that blog posts can be written as markdown, and checked into GitHub. It is
easy to collaborate. Access from China shouldn't be a problem too since we
can host it on a domain of our choice.


On Wed, Apr 11, 2018, 6:22 PM Zhao, Patric  wrote:

> FYI, China user can't access medium.com :(
>
> > -Original Message-
> > From: Anirudh Acharya [mailto:anirudhk...@gmail.com]
> > Sent: Thursday, April 12, 2018 6:31 AM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: blog for MXNet
> >
> > There is already an AWS Evangelist, Julien Simon, who has quite a few
> posts
> > about mxnet/gluon on medium - https://medium.com/@julsimon
> >
> >
> > Regards
> > Anirudh
> >
> > On Wed, Apr 11, 2018 at 3:27 PM, Sebastian Gutierrez <
> > sebast...@aiworkbox.com> wrote:
> >
> > > Aaron and Thomas
> > >
> > > Great ideas!
> > >
> > > One thing worth also considering is something like
> > >
> > > https://www.r-bloggers.com/
> > >
> > > What it does is serve as a blog aggregation service for all of the
> > > people who have blogged about r topics. Because of the central
> > > repository nature, it serves as a natural gathering point and allows
> > > people not using RSS (or similar technologies) to keep up to date with
> what is
> > happening.
> > >
> > > Another thing worth considering is a job board / site for MXNet full
> > > time / part time / remote jobs.  The data vis community has this free
> > > email list service
> > > https://groups.google.com/forum/m/#!forum/data-vis-jobs that's very
> > > community friendly and is a good place for people to gather to see job
> > > needs.
> > >
> > > All the best
> > > Sebastian Gutierrez
> > >
> > >
> > > On Wed, Apr 11, 2018 at 6:10 PM Thomas DELTEIL
> > > 
> > > wrote:
> > >
> > > > Thanks Aaron, I like medium, a lot of projects seems to be posting
> > > > their articles there, as you mentioned.
> > > >
> > > > Note that there is a newly created Chinese MXNet blog here:
> > > > https://zh.mxnet.io/blog/
> > > >
> > > > I would be happy to contribute to the blogs, if you want to add me
> > > > to the writer/editor list.
> > > >
> > > >
> > > >
> > > > Also, there is a mxnet subreddit r/mxnet which was created by
> > > > Sebastian
> > > > (thanks!) and I am now a moderator as well. Feel free to cross-post
> > > > any interesting content there! https://www.reddit.com/r/mxnet/
> > > > Please subscribe!
> > > >
> > > >
> > > > I will try to post one link a day at least, until I run out of links
> > > > ☺ We will also improve the look of it this week and add links to
> > > > relevant resources on the side bar, etc.
> > > >
> > > >
> > > >
> > > > All the best,
> > > >
> > > >
> > > > Thomas
> > > >
> > > >
> > > > 2018-04-11 14:45 GMT-07:00 Aaron Markham
> > :
> > > >
> > > > > Having a blog for MXNet would be very useful for conveying news,
> > > > > talking about features, demoing applications, and building
> awareness.
> > > > >
> > > > > Does anyone have particular preferences or recommendations on blog
> > > > > hosting or platform?
> > > > >
> > > > > I currently have editor access for an MXNet branded account on
> Medium.
> > > > > https://medium.com/mxnet
> > > > >
> > > > > There's nothing there at the moment, but at least with Medium we
> > > > > all could get started right away, and have a built-in syndication
> > > > > platform. Also, note that this is where the TensorFlow blog
> resides:
> > > > > https://medium.com/tensorflow
> > > > >
> > > > > Please make it known if you'd like to contribute, so you can get
> > > > > writer/editor access (to whichever platform we settle on.)
> > > > >
> > > > > Cheers,
> > > > > Aaron
> > > > >
> > > >
> > >
>

PR build failed because of git errors

[Indhu]

Hi,

Looks like PR #10039 build failed because of git errors. Here is the error
log:
http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/job/PR-10039/4/console.
Does someone know what could be happening here?

Build error:

Adding as 3rdparty/dlpack~7c28089749287f42ea8f41abd1358e6dbac54187 instead
Automatic merge failed; fix conflicts and then commit the result.

stderr:
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1990)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1958)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1954)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1592)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:692)
at
jenkins.plugins.git.MergeWithGitSCMExtension.decorateRevisionToBuild(MergeWithGitSCMExtension.java:122)
at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1068)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1161)
at
org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
at
org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:130)
at
org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:120)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:263)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

Thanks,
Indu

Re: [VOTE] tracking code changes with JIRA by associating pull requests

[Indhu]

+1 to the proposal


On Tue, Feb 27, 2018, 9:20 AM Nan Zhu  wrote:

> ideally,
>
> users report something fishy in github issue
>
> when confirmed that it is a bug or something to be improved, we should
> create JIRAs
>
> On Sun, Feb 25, 2018 at 9:31 AM, Chris Olivier 
> wrote:
>
> > i believe that JIRAs are “work items@, while github issues are more like
> > reporting. at least this is what Infra sort of claimed.
> >
> > On Sun, Feb 25, 2018 at 9:30 AM Chris Olivier 
> > wrote:
> >
> > > +1
> > >
> > >
> > > On Fri, Feb 23, 2018 at 9:56 AM Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > >> Hello Nan,
> > >>
> > >> Good suggestion!
> > >>
> > >> "hotfix which brings the broken build back to track" nitpicking, but I
> > >> wouldn't consider this a tiny fix. There should also be a jira that
> > >> reported the build being broken, so that shouldn't be a problem to
> link.
> > >>
> > >> Very good idea with the automated script!
> > >>
> > >> How would we handle permissions? Which actions are non-committers able
> > to
> > >> execute and in which cases would a committer be required?
> > >>
> > >> How would we treat GitHub issues in future? As a board for users to
> ask
> > >> usage questions?
> > >>
> > >> In order to improve user experience for new developers, I'd like to
> > >> suggest
> > >> that more experienced people might create jira tickets on behalf of
> > others
> > >> instead of telling them "please create a ticket". I think we all made
> > the
> > >> experience with people from it department who blocked every request
> > until
> > >> a
> > >> ticket was created and assigned :P
> > >>
> > >> Best regards,
> > >> Marco
> > >>
> > >> Am 23.02.2018 6:07 nachm. schrieb "CodingCat" :
> > >>
> > >> Hi, all
> > >>
> > >> To make the changes in code base more trackable,
> > >>
> > >> I would propose to link each PR with the a JIRA from now on:
> > >>
> > >> 1. most of PRs should be titled as [MXNET-???][MODULE_NAME] PR short
> > >> description, where MXNET-??? is the JIRA ID, MODULE_NAME can be
> python,
> > >> scala, symbol, etc.
> > >>
> > >> 2. only the very tiny fix, e.g. fix a typo, remove some unused
> > variables,
> > >> or the hotfix which brings the broken build back to track, can be
> filed
> > >> without JIRA ID in title,
> > >>
> > >> In JIRA side, to link the issue with an arrived PR, we can run a
> script
> > >> like https://github.com/apache/spark/blob/master/dev/github_
> > jira_sync.py
> > >> to
> > >> update JIRA status (can be run in Jenkins)
> > >>
> > >>
> > >> The benefits of applying such a flow include (but not limited to)
> > >>
> > >> 1. facilitate the release process:
> > >>
> > >> e.g., as long as tagging each JIRA with the correct affected version
> and
> > >> fixed version, the release manager can quickly identify what are the
> > >> changes applied in this version; or we can quickly identify whether
> > there
> > >> is any blocker issue in the JIRA
> > >>
> > >> 2. trackable changes
> > >>
> > >> any changes in MXNET can be quickly searched through JIRA or even
> > through
> > >> Google (JIRA looks like did something makes it more indexable by
> > Google),
> > >>
> > >>
> > >> If the vote gets pass,  the plan in the near horizon includes
> > >>
> > >> 1. update JIRA with the modules names
> > >>
> > >> 2. write runbook for release manager/committer for releasing new
> > >> version/merging patches, as well as contributors about the usage of
> JIRA
> > >>
> > >> 3. push the changes to the existing and coming PRs (this also needs
> the
> > >> collaboration across the whole committer team)
> > >>
> > >> The vote will end at 12:00 p.m. of March 2nd, 2018
> > >>
> > >> Best,
> > >>
> > >> Nan
> > >>
> > >
> >
>

Re: [VOTE] Release Apache MXNet (incubating) version 1.1.0.RC0

[Indhu]

+1 (binding)


On Tue, Jan 30, 2018 at 2:16 PM Chris Olivier  wrote:

> +1 (binding)
>
> On Sun, Jan 28, 2018 at 12:35 AM, Haibin Lin 
> wrote:
>
> > Update:
> >
> > Link to release candidate 1.1.0.rc0:
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.1.0.rc0/
> >
> > Link to the release tag:
> > https://github.com/apache/incubator-mxnet/tree/1.1.0.rc0
> >
> > Best,
> > Haibin
> >
> > On Sun, Jan 28, 2018 at 12:29 AM, Haibin Lin 
> > wrote:
> >
> > > Hi everyone,
> > >
> > > Given that most people on dev@ are in favor of a minor release (1.1.0)
> > > instead of a patch release due to API changes, I'd like to propose a
> vote
> > > to release Apache MXNet (incubating) 1.1.0. Voting will start now
> > (Sunday,
> > > January 28th) and end at 1pm Wednesday, January 31th PST.
> > >
> > > Link to release notes:
> > > https://cwiki.apache.org/confluence/display/MXNET/
> > > Apache+MXNet+%28incubating%29+1.1.0+Release+Notes
> > >
> > > Link to release candidate 1.1.0.rc0:
> > > https://github.com/apache/incubator-mxnet/releases/tag/1.1.0.rc0
> > >
> > > View this page and scroll down to “Build from Source” with source code
> > > obtained from the 1.1.0.rc0 tag:
> > > https://mxnet.incubator.apache.org/install/index.html
> > >
> > > (Note: The README.md points to the 1.1.0 tag and does not work at the
> > > moment.)
> > >
> > > Please remember to TEST first before voting accordingly:
> > > +1 = approve
> > > +0 = no opinion
> > > -1 = disapprove (provide reason)
> > >
> > > Best,
> > > Haibin
> > >
> > >
> >
>

Re: Commiter access to Jenkins Sevrer

[Indhu]

Objective is to make sure committers has access to CI. As far as that
objective is met, it shouldn't matter whether the auth mechanism uses
GitHub or Apache SSO.

If GitHub SSO is easier to implement, let's go ahead and do that unless
someone in the list objects.


On Fri, Jan 5, 2018 at 11:21 AM Chris Olivier  wrote:

> I am fine without a vote unless a vote is required?  Any objections,
> anyone?  You're sort of adding functionality here, not changing or
> restricting...  We can always change to Apache later.
>
> On Fri, Jan 5, 2018 at 11:18 AM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > I'd be in favour of GitHub. Shall we open a vote or would you like me to
> > create a POC with GitHub first and afterwards we can check if that's
> > enough?
> >
> > -Marco
> >
> > On Fri, Jan 5, 2018 at 8:13 PM, Chris Olivier 
> > wrote:
> >
> > > Apparently Apache supports OATH, so I am open to either.
> > > Good idea for the docker thing.
> > >
> > > On Fri, Jan 5, 2018 at 11:02 AM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > GitHub SSO allows the neat feature that login and permission can be
> > > > selected depending on the access rights a user has to a project.
> > Somebody
> > > > with write access (committers) would be get different permissions
> than
> > > > somebody with only read access.
> > > >
> > > > We could check back with Apache for SSO, but this would involve
> Apache
> > > > infra. We could put it up to a vote whether to use GitHub or Apache
> > SSO.
> > > >
> > > > In order to reproduce a build failure we have been thinking about
> > > changing
> > > > the ci_build.sh in such a way that it can be run manually without
> > > Jenkins.
> > > > The setup I took over binds the Jenkins work directory into the
> docker
> > > > containers and uses a few hacks which are hard to reproduce locally.
> We
> > > > plan to reengineer this script to make it easier to run manually.
> > > > But making the AMI public is a good idea! We plan to make the whole
> > > > infrastructure code (based on Terraform) completely public - at the
> > > moment
> > > > it's in a private repository as it contains credentials, but they
> will
> > be
> > > > moved to KMS soon. It would definitely be a good approach to just
> > supply
> > > > the AMI so everybody could recreate the environment in their own
> > account.
> > > >
> > > > -Marco
> > > >
> > > > Am 05.01.2018 7:51 nachm. schrieb "Chris Olivier" <
> > cjolivie...@gmail.com
> > > >:
> > > >
> > > > Well, login to the Jenkins server, I would imagine.
> > > >
> > > > github or Apache SSO (does Apache support OAUTH?) seems like a good
> > idea
> > > as
> > > > long as there's a way to not let everyone with a github account log
> in.
> > > >
> > > > Access to actual slave machines could be more restricted, I imagine.
> > > >
> > > > Eventually, a public current AMI for a build slave would be good in
> > order
> > > > to reproduce build or test problems that can't be reproduced locally.
> > > >
> > > > wdyt?
> > > >
> > > >
> > > >
> > > > On Fri, Jan 5, 2018 at 10:41 AM, Marco de Abreu <
> > > > marco.g.ab...@googlemail.com> wrote:
> > > >
> > > > > Would it be an acceptable solution if we add SSO or do you also
> want
> > > > access
> > > > > to the actual AWS account and all machines?
> > > > >
> > > > > Yes, the build jobs are automatically getting created for new
> > branches.
> > > > >
> > > > > -Marco
> > > > >
> > > > > Am 05.01.2018 7:35 nachm. schrieb "Marco de Abreu" <
> > > > > marco.g.ab...@googlemail.com>:
> > > > >
> > > > > I totally agree, this is not the way it should work in an Apache
> > > Project.
> > > > > It's running on an isengard account, meaning it is only accessible
> > for
> > > > > Amazon employees. The problem is that a compromised account could
> > cause
> > > > > damage up to 170,000$ per day. There are alarms in place to notice
> > > those
> > > > > cases, but we still have to be very careful. These high limits have
> > > been
> > > > > chosen due to auto scaling being added within the next week's.
> > > > >
> > > > > I'd be happy to introduce a committer into the CI process and all
> the
> > > > > necessary steps as well as granting them permission. The only
> > > restriction
> > > > > being that it has to be and Amazon employee and access to console,
> > > master
> > > > > and slave only being possible from the Corp network.
> > > > >
> > > > > There is no open ticket. What would you like to request?
> > > > >
> > > > > -Marco
> > > > >
> > > > >
> > > > > Am 05.01.2018 7:22 nachm. schrieb "Chris Olivier" <
> > > cjolivie...@gmail.com
> > > > >:
> > > > >
> > > > > Like John and other mentors were saying, it's not proper for CI to
> > be a
> > > > > closed/inaccessible environment.  Is it running on an Isengard
> > account
> > > or
> > > > > in PROD or CORP or just generic EC2?  I think that we should remedy
> > > this.
> > > > > It's very strange that no 

Breaking change to the model JSON file in 1.0.0 release

[Indhu]

After the 1.0.0 release, we heard from multiple users that the model JSON
file produced by 1.0.0 when a model is saved is not compatible with older
versions of MXNet. Since there was no such change documented in the release
notes, our first thought was that it must be an unintentional change.

We looked at the commit logs and found out that the change was indeed
intentional. Here is the PR that introduced the change:
https://github.com/dmlc/nnvm/pull/152

I think a breaking change like this must have been documented in the
release notes.

1. Given this happened, what actions must we take to make sure such changes
don't happen in the future without being documented in the release notes?
2. For users who ask, I assume we say the change was intentional and there
is no plan to roll it back. Please let me know if that is not correct.

Thanks,
Indu

Automatically translating Caffe code to MXNet

[Indhu]

One of the questions we frequently encounter in forums and emails is “I
have caffe code that does this. How do I do the same thing in MXNet”. While
it might be possible to go through the documentation and do the translation
layer by layer manually, we thought a tool that automatically does it will
be helpful for people who want to migrate from Caffe to MXNet. Here is the
first version of the tool:
https://github.com/apache/incubator-mxnet/tree/master/tools/caffe_translator.
Thanks to Aaron (@aaronmarkham), Madan (@madjam), Pracheer (@pracheer) and
Steffen (@srochel) for their help and feedback in building this.

Note that this is different from the caffe converter that already exists (
https://github.com/apache/incubator-mxnet/tree/master/tools/caffe_converter).
Caffe converter converts Caffe Model to MXNet model so that user can run
inference on MXNet or finetune on MXNet. The new translator on the other
hand helps users to migrate their Caffe code to MXNet and continue
development on MXNet. I think both tools are useful in different contexts.

Caffe Translator can currently translate the layers mentioned in the README
.
Obviously this is only a humble beginning and there is much more layers
that can be supported. Apart from adding support for more layers, following
are some things that can make the tool more useful:

- Eliminate the CaffeDataIter

dependency to translate data layers.
- Generate code that can do distributed training. This is one of main
reasons I might want to move my Caffe code to MXNet.
- Make the tool accessible through a website so that user can just copy
paste a Caffe code snippet in the website and get translated code instantly
without downloading and running anything locally.

Any help in these efforts is greatly appreciated. Also let me know if there
is some other improvement that could be done to make the tool more useful.

Thanks,
Indu

Re: [VOTE] Release Apache MXNet(incubating) version 1.0.0.rc0

[Indhu]

+1 (binding)

Built from source and tested with some example code.


On Fri, Nov 24, 2017, 4:00 PM Chris Olivier  wrote:

> This is the vote to release Apache MXNet (incubating) version 1.0.0.
>
> Voting will start now (Friday, November 24, 2017) and
>
> will remain open for 72 hours.
>
>
> Link to release notes:
>
> https://cwiki.apache.org/confluence/display/MXNET/
> Apache+MXNet+%28incubating%29+1.0+Release+Notes
>
>
> Link to release candidate 1.0.0.rc0:
>
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.0.0.rc0/
>
>
> View this page and scroll down to “Build from Source” to build this
> project:
>
> https://mxnet.incubator.apache.org/install/index.html
>
>
> The release tag can be found here:
>
> https://github.com/apache/incubator-mxnet/tree/1.0.0.rc0
>
> (Note: The README.md points to the 1.0.0 tag and does not work at the
> moment.)
>
>
> We are planning to have the MXNet version 1.0.0 release ready before the
> NIPS conference that starts on 04-Dec, 2017 and your timely response for
> the vote will be highly appreciated.
>
> Please make sure you TEST before you vote accordingly:
>
> +1 = approve
>
> +0 = no opinion
>
> -1 = disapprove (provide reason)
>

Re: [VOTE] Release Apache MXNet(incubating) version 0.12.1.rc0

[Indhu]

+1 (binding)

Built from source and ran some example code.


On Tue, Nov 7, 2017 at 7:48 PM Meghna Baijal 
wrote:

> This is the vote to release Apache MXNet (incubating) version 0.12.1.
>
> Voting will start now (Tuesday, November 7, 2017) and
>
> close Friday, November 10, 2017
>
>
> Link to release notes:
>
>
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+0.12.1+Release+Notes
>
>
> Link to release candidate 0.12.1.rc0:
>
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.12.1.rc0/
>
>
> View this page and scroll down to “Build from Source” to build this
> project:
>
> https://mxnet.incubator.apache.org/install/index.html
>
>
> The release tag can be found here:
>
> https://github.com/apache/incubator-mxnet/tree/0.12.1.rc0
>
> (Note: The README.md points to the 0.12.1 tag and does not work at the
> moment.)
>
>
>
> Please make sure you TEST before you vote accordingly:
>
>
> +1 = approve
>
>
> +0 = no opinion
>
>
> -1 = disapprove (provide reason)
>
>
>
> Thanks,
>
> Meghna Baijal
>

Re: Disable 'required status check' for master

[Indhu]

BTW, here is a separate thread where there is discussion happening on
improving the CI system:
https://lists.apache.org/thread.html/13e5ba08acd2f28a49f791c23f4baf97ba9e5bcdabe43aec598d1e19@%3Cdev.mxnet.apache.org%3E

While we will eventually get a faster CI system, it is important to disable
the checks for now to not block or slow down development until we have a
faster CI system.


On Tue, Oct 31, 2017 at 12:07 PM Indhu <indhubhara...@gmail.com> wrote:

> Hi,
>
> We have been having issues getting CI to work fast enough. Currently the
> CI is being the bottleneck to get commits in. This is severely impacting
> development. I propose we disable 'required status check' for the master
> branch so that we can development is not impacted. We can work on fixing
> the CI situation in parallel.
>
> Committer,
> Please vote if you are okay with disabling 'required status check' for
> master.
>
> Once we have enough votes, I'll request a mentor to file a ticket with
> infra.
>
> Thanks,
> Indu
>
>

Disable 'required status check' for master

[Indhu]

Hi,

We have been having issues getting CI to work fast enough. Currently the CI
is being the bottleneck to get commits in. This is severely impacting
development. I propose we disable 'required status check' for the master
branch so that we can development is not impacted. We can work on fixing
the CI situation in parallel.

Committer,
Please vote if you are okay with disabling 'required status check' for
master.

Once we have enough votes, I'll request a mentor to file a ticket with
infra.

Thanks,
Indu

Re: The Exchange Layer Support of MXNet

[Indhu]

Do we have a consensus yet on how to implement this?


On Fri, Oct 20, 2017 at 9:05 AM Tianqi Chen 
wrote:

> As with every change, there need to be steps. The python API of gluon is a
> first step toward moving API to the set of the new operator. I think
> exchange and compilation are next step it is relatively easy to do so
> without start implementing these ops (we canonicalize to core ops and
> convert them back to the ops we have).   The final step is to move the
> implementation toward the new ops, which can happen gradually.
>
> Tianqi
>
> On Fri, Oct 20, 2017 at 8:58 AM, Tianqi Chen 
> wrote:
>
> > The core ops do not mean implementation. They are merely a spec that we
> > want to agree on what attributes we need.
> >
> > - The current list of the proposal spec is here:
> > http://nnvm.tvmlang.org/top.html  (I think eventually MXNet's document
> > should have one such page)
> > - They are is modeled after numpy and mxnet/gluon and is used by the new
> > graph optimizations in nnvm compiler.
> > - There is priority in terms of support level, which indicates how
> > important they are when support.
> >
> > The major need for core ops is to be concise(smallest set),
> > expressive(covers most need), conventional (follows existing well adopted
> > API like numpy). The legacy ops have no problem in terms of expressive
> but
> > have issues with respect to conciseness and conventional.
> >
> > By drifting the central support to core ops and canonicalize other ops
> > into them, we can make optimization, exchange and other related problems
> > easier. It also gives clear external support of ApacheMXNet
> >
> > Tinaqi
> >
> > On Fri, Oct 20, 2017 at 8:50 AM, Chris Olivier 
> > wrote:
> >
> >> Thansk for that explanation!
> >> What is meant by the "core ops"?  Are these the gluon python ops?  I
> know
> >> what the "legacy ops" are -- things like BatchNormProp/BatchNormOp.
> WHat
> >> are the "new ones"?
> >>
> >> I apologize for not knowing this, but what are some of the "pain points"
> >> with the legacy ops?  Not in great detail, but would just like to get a
> >> little perspective.
> >>
> >> Thanks,
> >>
> >> -Chris
> >>
> >>
> >> On Fri, Oct 20, 2017 at 8:42 AM, Tianqi Chen 
> >> wrote:
> >>
> >> > First of all, I agree that supporting the format as an important step
> of
> >> > adding influence. We are going to do it in either options. The
> overhead
> >> of
> >> > engineering is not that much difference.  I also do not argue for
> >> specific
> >> > types of APIs(gluon vs symbolic) we should use.
> >> >
> >> >  MXNet Sym API contains two elements:
> >> >
> >> > - 1) The graph manipulation API
> >> > - 2) The operator defs
> >> >
> >> > The graph manipulation API has been serving us great and we should be
> >> keep
> >> > using it for whatever purposes we have. However, the operator
> >> definitions
> >> > has been involved for two years and there are quite a few pains in
> >> legacy
> >> > operator and attributes.  The new remodeled gluon API is much cleaner.
> >> > While there is a gap between the current operator def and gluon's API,
> >> we
> >> > want to shift defs toward gluon style operator defs (NOTE this do not
> >> mean
> >> > imperative only, just to make operator def align with numpy and
> gluon).
> >> >
> >> > What I am proposing is to have a clear document and list of "core
> >> operator
> >> > defs" we want to prioritize in terms of supporting exchange offers
> >> > optimization.  Having such document, gives us and others a clear
> >> reference
> >> > on what we need, and what is needed in order to give good external
> >> support
> >> > to ApacheMXNet.
> >> >
> >> > The MXNet/gluon or nnvm/top is such initial set of operator defs that
> is
> >> > proposed. By canonicalizing the current MXNet op defs into this "core
> >> set",
> >> > and using it as a center for talking to other exchange format gives
> the
> >> > advantage of the things we mentioned above.
> >> >
> >> > Going through the path have second major advantage, because it
> >> > enables(answering your question on what compilation mean)
> >> >   mxnet->core ops -> nnvm compiler-> bare metal(javascript, ARM,
> >> > AMDGPU, Metal)
> >> >
> >> > While in theory we can also do mxnet->onnx-> nnvm compiler, This is a
> >> more
> >> > twisted path as the exchange format do not preserve information and
> >> subject
> >> > to change. While going through the core ops offers bi-directional
> >> exchange
> >> > with mxnet, and directly doing that in memory.  The compilation path
> is
> >> not
> >> > offered if we do not go through the core ops, because the graph
> >> > optimization can take great advantage of a clear core set of
> operators.
> >> >
> >> > To facilitate discussion technically. I would suggest we break our
> >> points
> >> > down. I tried to do that in this email on things we agree on and have
> >> > opions. So we can 

Re: [VOTE] Release MXNet version 0.12.0.rc0

[Indhu]

+1


On Fri, Oct 20, 2017, 5:01 PM Meghna Baijal 
wrote:

> This is the vote to release Apache MXNet (incubating) version 0.12.0.
>
> Voting will start now (Friday, October 20, 2017 11:55PM UTC) and
>
> close Tuesday, October 24, 2017 11:55PM UTC.
>
>
> Link to release notes:
>
>
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+0.12.0+Release+Notes
>
>
> Link to release candidate 0.12.0.rc0:
>
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.12.0.rc0/
>
>
> View this page and scroll down to “Build from Source” to build this
> project:
>
> http://mxnet.incubator.apache.org/get_started/install.html
>
>
> The release tag can be found here:
>
> https://github.com/apache/incubator-mxnet/tree/0.12.0.rc
> 0
>
> (Note: The README.md points to the 0.12.0 tag and does not work at the
> moment.)
>
>
> Please make sure you TEST before you vote accordingly:
>
>
> +1 = approve
>
>
> +0 = no opinion
>
>
> -1 = disapprove (provide reason)
>
>
> Thanks,
>
> Meghna Baijal
>

[Important] Please rebase your PRs

[Indhu]

Hi,

The build problem mentioned below was fixed by changing
<https://github.com/apache/incubator-mxnet/commit/573a010879583885a0193e30dc0b8c848d80869b>
the Jenkinsfile. Note that your build could still fail if you have the old
Jenkinfile. To make sure your builds succeed, please rebase your code with
master.

Thanks,
Indu

On Mon, Oct 9, 2017 at 6:42 PM Indhu <indhubhara...@gmail.com> wrote:

> Hi,
>
> Some of you might have noticed that the CI builds were failing for reasons
> other than your changes. There was a problem in the build process because
> of which outputs from previous builds were not getting deleted and new
> builds were failing due to that. This issue has been rectified. Since the
> build slaves were dirty, I had to kill the builds that were running (which
> would have failed anyway).
>
> If you have a pending PR waiting for build, could you please kick off a
> new build. You can also send me an email with the PR number and I can kick
> off the build for you.
>
> I'm sorry about the inconvenience. I'll closely monitor the builds until
> the builds are back to normal state.
>
> Thanks,
> Indu
>
>

Cancelled builds

[Indhu]

Hi,

Some of you might have noticed that the CI builds were failing for reasons
other than your changes. There was a problem in the build process because
of which outputs from previous builds were not getting deleted and new
builds were failing due to that. This issue has been rectified. Since the
build slaves were dirty, I had to kill the builds that were running (which
would have failed anyway).

If you have a pending PR waiting for build, could you please kick off a new
build. You can also send me an email with the PR number and I can kick off
the build for you.

I'm sorry about the inconvenience. I'll closely monitor the builds until
the builds are back to normal state.

Thanks,
Indu

Incremental builds

[Indhu]

Hi,

We all have the experience of making a small change in a PR and waiting for
a couple of hours or more for the continuous integration builds to finish.
This is because at the moment, all builds (even if it only fixes a typo in
a comment) is built from scratch as a clean build. I was wondering if we
can do incremental builds instead for changes to PRs. So, a clean build
will still happen when a PR is submitted for the first time. But every time
a change is made to the PR, we should be able to do an incremental build
from the previous build that was done for that PR. This will drastically
reduce the waiting time for developers.

I have an idea to do this which involves creating a separate directory in
jenkins slaves for each PR and doing the build for that PR under its own
directory. These directories can be created in EFS and they can be deleted
when the PR is closed. I'm wondering if there is a better/easier way to do
this. Please let me know if you know of any.

Thanks,
Indu

mx.symbol.CaffeLoss with more than two inputs

[Indhu]

Following is smooth L1 Loss layer used in PVANet

:

layer {
name: "rpn_loss_bbox"
type: "SmoothL1Loss"
bottom: "rpn_bbox_pred"
bottom: "rpn_bbox_targets"
bottom: "rpn_bbox_inside_weights"
bottom: "rpn_bbox_outside_weights"
top: "rpn_loss_bbox"
include { phase: TRAIN }
loss_weight: 1
smooth_l1_loss_param { sigma: 3.0 }
}
This seems to be a custom loss layer that takes four inputs. I'm trying to
use this in MXNet using 'mx.symbol.CaffeLoss'. I see that 'CaffeLoss' only
takes 2 inputs (data and label). Does anybody know how I can I use
'CaffeLoss' to use the above loss layer in MXNet?

Thanks,
Indu