Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0

[Srivastava, Rohit Kumar]

+1 
build mxnet from source with large tensor support. Ran tests only for large 
array. All passed !

On 9/19/19, 2:58 PM, "Lai Wei"  wrote:

+1

build from source on GPU and tested with gluon estimator and latest
keras-mxnet.


Best Regards

Lai


On Thu, Sep 19, 2019 at 1:02 PM sandeep krishnamurthy <
sandeep.krishn...@gmail.com> wrote:

> Thank you Tao for leading this and all the community members for helping 
in
> this release.
>
>
> +1
>
>
> -[Y] Are release files in correct location?
>
> -[Y] Do release files have the word incubating in their name?
>
> -[Y] Are the digital signature and hashes correct?
>
> -[Y] Does DISCLAIMER file exist?
>
> -[Y] Do LICENSE and NOTICE files exists?
>
> -[Y] Is the LICENSE and NOTICE text correct?
>
> -[Y] Is the NOTICE year correct?
>
> -[Y] Un-included software dependencies are not mentioned in LICENSE or
> NOTICE?
>
> -[Y] License information is not mentioned in NOTICE?
>
> Is there any 3rd party code contained inside the release? If so:
>
> -[N] Does the software have a compatible license?
>
> -[Y] Are all software licenses mentioned in LICENSE?
>
> -[Y] Is the full text of the licenses (or pointers to it) in LICENSE?
>
> Is any of this code Apache licensed? Do they have NOTICE files? If so:
>
> -[Y] Have relevant parts of those NOTICE files been added to this NOTICE
>
> file?
>
> -[Y] Do all source files have ASF headers?
>
> -[Y] Do the contents of the release match with what's tagged in version
> control?
>
> -[N] Are there any unexpected binary files in the release?
>
> -[Y] Can you compile from source? Are the instruction clear?
>
>
> Except the license issue mentioned in this Github issue -
> https://github.com/apache/incubator-mxnet/issues/15542
>
>
> I was able to build from source on GPU(p3.2x EC2 instance) and run
> opperf-operator
> benchmark utilit
> y
> successfully
> with no regression compared to v1.5.0.
>
>
>
>
> On Thu, Sep 19, 2019 at 11:51 AM Anirudh Subramanian <
> anirudh2...@gmail.com>
> wrote:
>
> > +1
> >
> > Build from source with cmake and ran unittest for gluon and amp.
> >
> > Noticed that test_sync_batchnorm fails on p3.8xlarge (hidden by the CI
> > because passes on machines with 1 or 2 gpus).
> > I have opened an issue for the same
> > https://github.com/apache/incubator-mxnet/issues/16214 though I think
> its
> > not a blocker for this release.
> >
> > Anirudh
> >
> > On Thu, Sep 19, 2019 at 11:28 AM Chaitanya Bapat 
> > wrote:
> >
> > > +1
> > >
> > > Correctly built for GPU, CPU on Ubuntu 14.01 (10.1 Cuda for GPU)
> > > Ran image classification (resnet50+cifar10)
> > > Ran Operator Performance (opperf)
> > >
> > > On Thu, 19 Sep 2019 at 02:12, Tao Lv  wrote:
> > >
> > > > Hi community,
> > > >
> > > > Friendly reminder: it is less than 1.5 days remaining, so please 
take
> > > your
> > > > time to verify and vote.
> > > >
> > > > Thanks,
> > > > -tao
> > > >
> > > > On Thu, Sep 19, 2019 at 3:06 PM Lin Yuan 
> wrote:
> > > >
> > > > > +1
> > > > > Tested Horovod on GPU
> > > > >
> > > > > On Wed, Sep 18, 2019 at 6:16 AM Zhao, Patric <
> patric.z...@intel.com>
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Tested MKLDNN backend and everything looks great.
> > > > > >
> > > > > > > -Original Message-
> > > > > > > From: Qing Lan 
> > > > > > > Sent: Wednesday, September 18, 2019 2:20 AM
> > > > > > > To: dev@mxnet.incubator.apache.org
> > > > > > > Subject: Re: [VOTE] Release Apache MXNet (incubating) 
1.5.1.rc0
> > > > > > >
> > > > > > > +1 for Scala/Java test. Passed all tests for CPU/GPU build.
> > > > > > > Also tested build from source with static build.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Qing
> > > > > > > 
> > > > > > > From: Tao Lv 
> > > > > > > Sent: Tuesday, September 17, 2019 14:14
> > > > > > > To: dev@mxnet.incubator.apache.org <
> > dev@mxnet.incubator.apache.org
> > > >
> > > > > > > Subject: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> > > > > > >
> > > > > > > Dear MXNet community,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This is the 3-day vote to release Apache MXNet (incubating)
> > version
> > > > > > 1.5.1.
> > > > > > >
> > > > > > > Voting on dev@ will start September 17, 12:00pm (PST)  and
> close
> > > on
> > 

Re: [DISCUSS] CI Access Control

[Sheng Zha]

Hi Marco,

Thank you for sharing the insights. The discussion is intended for setting 
goals so that future design improvement to the CI can take these goals into 
consideration. Thus, while I fully recognize that there could be difficulty in 
implementation, I'd still like to confirm with the community if the outlined 
access control recommendation is at the right level.

To summarize your concerns:
- opening up access control should be conditioned on having good version 
control and roll-back mechanism to ease the operation burden from breakage, 
which is more likely given larger user base.
- upgrades to the system would be better managed as planned and collective 
efforts instead of adhoc tasks performed by uncoordinated individuals.

You also mentioned that "changes to the system should only be done by the 
administrators". It's exactly the intention of this thread is to define who 
would qualify as administrators. Currently, such qualification is opaque, and 
only happens within a group in Amazon.

On the other hand, this current way can, and already has caused friction. When 
this project's daily activity of validating and merging code is affected due to 
the system's instability, the community members have no choice but to wait for 
the issues to be resolved by the current system administrators. Other affected 
community members have no way to help even if they wish to.

Given the existing Apache project governance model, I'd recommend that the goal 
for CI access control be set so that committer and PMC member who wishes to be 
involved should have the right to help.

-sz

On 2019/09/17 12:49:20, Marco de Abreu  wrote: 
> Ah, with regards to #1 and #2: Currently, we don't have any plugins that
> control the actions of a single user and allows us to monitor and rate
> limit them. Just giving trigger permission (which is also tied with
> abort-permission if I recall correctly), would allow a malicious user to
> start a huge number of jobs and thus either create immense costs or bring
> down the system. Also, we'd have to check how we can restrict the trigger
> permission to specific jobs.
> 
> -Marco
> 
> On Tue, Sep 17, 2019 at 2:47 PM Marco de Abreu 
> wrote:
> 
> > Hi Sheng,
> >
> > will I'm in general all in favour of widening the access to distribute the
> > tasks, the situation around the CI system in particular is a bit more
> > difficult.
> >
> > As far as I know, the creation of the CI system is neither automated,
> > versioned nor backed up or safeguarded. This means that if somebody makes a
> > change that breaks something, we're left with a broken system we can't
> > recover from. Thus, I preferred it in the past to restrict the access as
> > much as possible (at least to Prod) to avoid these situations from
> > happening. While #1 and #2 are already possible today (we have two roles
> > for committers and regular users that allow this already), #3 and #4 come
> > with a significant risk for the stability of the system.
> >
> > As soon as a job is added or changed, a lot of things happen in Jenkins -
> > one of these tasks is the SCM scan which tries to determine the branches
> > the job should run on. For somebody who is inexperienced, the first pitfall
> > is that suddenly hundreds of jobs are being spawned which will certainly
> > overload Jenkins and render it unusable. There are a lot of tricks and I
> > could elaborate them, but basically the bottom line is that the
> > configuration interface of Jenkins is far from fail-proof and exposes a
> > significant risk if accessed by somebody who doesn't exactly know what
> > they're doing - speak, we would need to design some kind of training and
> > even that would not safeguard us from these fatal events.
> >
> > There's the whole security aspect around user-facing artifact generation
> > of CI/CD and the possibility of them being tampered, but I don't think I
> > have to elaborate that.
> >
> > With regards to #4 especially, I'd say that the risk of somebody just
> > upgrading the system or changing plugins inherits an even bigger risk.
> > Plugins are notoriously unsafe and system updates have also shown to not
> > really go like a breeze. I'd argue that changes to the system should only
> > be done by the administrators of it since they have a bigger overview over
> > all the things that are currently going on while also having the full
> > access (backups before making changes, SSH access, log access, metric
> > access, etc) to debug errors. In the end we shouldn't forget that this is a
> > productive system - usually, you'd have nobody being able to touch it at
> > all, but we're not in a perfect world, so I'd say we should restrict it to
> > a bare minimum in the form of admins.
> >
> > So while I certainly understand and encourage to distribute the access, I
> > don't feel comfortable widening the access to such a critical productive
> > system. It being down means that the GitHub development is fully halted,
> > which is really 

Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0

[sandeep krishnamurthy]

Thank you Tao for leading this and all the community members for helping in
this release.


+1


-[Y] Are release files in correct location?

-[Y] Do release files have the word incubating in their name?

-[Y] Are the digital signature and hashes correct?

-[Y] Does DISCLAIMER file exist?

-[Y] Do LICENSE and NOTICE files exists?

-[Y] Is the LICENSE and NOTICE text correct?

-[Y] Is the NOTICE year correct?

-[Y] Un-included software dependencies are not mentioned in LICENSE or
NOTICE?

-[Y] License information is not mentioned in NOTICE?

Is there any 3rd party code contained inside the release? If so:

-[N] Does the software have a compatible license?

-[Y] Are all software licenses mentioned in LICENSE?

-[Y] Is the full text of the licenses (or pointers to it) in LICENSE?

Is any of this code Apache licensed? Do they have NOTICE files? If so:

-[Y] Have relevant parts of those NOTICE files been added to this NOTICE

file?

-[Y] Do all source files have ASF headers?

-[Y] Do the contents of the release match with what's tagged in version
control?

-[N] Are there any unexpected binary files in the release?

-[Y] Can you compile from source? Are the instruction clear?


Except the license issue mentioned in this Github issue -
https://github.com/apache/incubator-mxnet/issues/15542


I was able to build from source on GPU(p3.2x EC2 instance) and run
opperf-operator
benchmark utilit
y
successfully
with no regression compared to v1.5.0.




On Thu, Sep 19, 2019 at 11:51 AM Anirudh Subramanian 
wrote:

> +1
>
> Build from source with cmake and ran unittest for gluon and amp.
>
> Noticed that test_sync_batchnorm fails on p3.8xlarge (hidden by the CI
> because passes on machines with 1 or 2 gpus).
> I have opened an issue for the same
> https://github.com/apache/incubator-mxnet/issues/16214 though I think its
> not a blocker for this release.
>
> Anirudh
>
> On Thu, Sep 19, 2019 at 11:28 AM Chaitanya Bapat 
> wrote:
>
> > +1
> >
> > Correctly built for GPU, CPU on Ubuntu 14.01 (10.1 Cuda for GPU)
> > Ran image classification (resnet50+cifar10)
> > Ran Operator Performance (opperf)
> >
> > On Thu, 19 Sep 2019 at 02:12, Tao Lv  wrote:
> >
> > > Hi community,
> > >
> > > Friendly reminder: it is less than 1.5 days remaining, so please take
> > your
> > > time to verify and vote.
> > >
> > > Thanks,
> > > -tao
> > >
> > > On Thu, Sep 19, 2019 at 3:06 PM Lin Yuan  wrote:
> > >
> > > > +1
> > > > Tested Horovod on GPU
> > > >
> > > > On Wed, Sep 18, 2019 at 6:16 AM Zhao, Patric 
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Tested MKLDNN backend and everything looks great.
> > > > >
> > > > > > -Original Message-
> > > > > > From: Qing Lan 
> > > > > > Sent: Wednesday, September 18, 2019 2:20 AM
> > > > > > To: dev@mxnet.incubator.apache.org
> > > > > > Subject: Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> > > > > >
> > > > > > +1 for Scala/Java test. Passed all tests for CPU/GPU build.
> > > > > > Also tested build from source with static build.
> > > > > >
> > > > > > Thanks,
> > > > > > Qing
> > > > > > 
> > > > > > From: Tao Lv 
> > > > > > Sent: Tuesday, September 17, 2019 14:14
> > > > > > To: dev@mxnet.incubator.apache.org <
> dev@mxnet.incubator.apache.org
> > >
> > > > > > Subject: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> > > > > >
> > > > > > Dear MXNet community,
> > > > > >
> > > > > >
> > > > > >
> > > > > > This is the 3-day vote to release Apache MXNet (incubating)
> version
> > > > > 1.5.1.
> > > > > >
> > > > > > Voting on dev@ will start September 17, 12:00pm (PST)  and close
> > on
> > > > > > September 20, 12:00pm (PST).
> > > > > >
> > > > > >
> > > > > >
> > > > > > 1) Link to release notes:
> > > > > >
> > > > > >
> > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Notes
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2) Link to release candidate:
> > > > > >
> > > > > > https://github.com/apache/incubator-mxnet/releases/tag/1.5.1.rc0
> > > > > >
> > > > > >
> > > > > >
> > > > > > 3) Link to source and signatures on Apache dist server:
> > > > > >
> > > > > >
> https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.5.1.rc0/
> > > > > >
> > > > > >
> > > > > >
> > > > > > Please remember to TEST first before voting accordingly:
> > > > > >
> > > > > > +1 = approve
> > > > > >
> > > > > > +0 = no opinion
> > > > > >
> > > > > > -1 = disapprove (provide reason)
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > -tao
> > > > >
> > > >
> > >
> >
> >
> > --
> > *Chaitanya Prakash Bapat*
> > *+1 (973) 953-6299*
> >
> > [image: https://www.linkedin.com//in/chaibapat25]
> > [image:
> https://www.facebook.com/chaibapat
> > ]
> > [image:
> > https://twitter.com/ChaiBapchya]  >[image:
> > 

Re: new website, docs code freeze

[Aaron Markham]

Thanks everyone. The PRs passed CI, but please continue holding off on
docs and CI edits. Unless there are any objections, I'd like to launch
the new website today.

On Wed, Sep 18, 2019 at 7:46 AM Aaron Markham  wrote:
>
> Hi everyone,
> The last two PRs [1][2] for the new website and docs have passed CI
> (finally). Please do not make changes to /docs or /ci until we get
> these approved and merged. Every time there's a merge conflict it has
> set us back a day or two while shepherding the PRs through CI again.
> Unless there are catastrophic issues discovered in a review, I
> recommend that we hold any patches or updates to the PRs to follow-up
> PRs.
>
> There are four steps to launch:
> 1. Once the PRs are approved, the plan is to merge 15885 to delete the
> old content first.
> 2. Then immediately merge 15883 to add in the new CI flows and updates
> to the content Thomas and I have already had merged in 15884 [3].
> 3. I will change the website validation Jenkins pipeline to point to
> the new pipeline.
> 4. I will change the website publishing Jenkins pipeline to point to
> its new pipeline as well. Once triggered, the old site will be
> replaced with the new one.
>
> Post launch we'll need to update the DNS for beta.mxnet.io to point to
> production, and there will likely be some redirect/.htaccess updates
> needed next week to assist with any deep linking and 404 issues that
> pop up.
>
> Cheers,
> Aaron
>
> [1] https://github.com/apache/incubator-mxnet/pull/15885
> [2] https://github.com/apache/incubator-mxnet/pull/15883
> [3] https://github.com/apache/incubator-mxnet/pull/15884

Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0

[Tao Lv]

Hi community,

Friendly reminder: it is less than 1.5 days remaining, so please take your
time to verify and vote.

Thanks,
-tao

On Thu, Sep 19, 2019 at 3:06 PM Lin Yuan  wrote:

> +1
> Tested Horovod on GPU
>
> On Wed, Sep 18, 2019 at 6:16 AM Zhao, Patric 
> wrote:
>
> > +1
> >
> > Tested MKLDNN backend and everything looks great.
> >
> > > -Original Message-
> > > From: Qing Lan 
> > > Sent: Wednesday, September 18, 2019 2:20 AM
> > > To: dev@mxnet.incubator.apache.org
> > > Subject: Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> > >
> > > +1 for Scala/Java test. Passed all tests for CPU/GPU build.
> > > Also tested build from source with static build.
> > >
> > > Thanks,
> > > Qing
> > > 
> > > From: Tao Lv 
> > > Sent: Tuesday, September 17, 2019 14:14
> > > To: dev@mxnet.incubator.apache.org 
> > > Subject: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> > >
> > > Dear MXNet community,
> > >
> > >
> > >
> > > This is the 3-day vote to release Apache MXNet (incubating) version
> > 1.5.1.
> > >
> > > Voting on dev@ will start September 17, 12:00pm (PST)  and close on
> > > September 20, 12:00pm (PST).
> > >
> > >
> > >
> > > 1) Link to release notes:
> > >
> > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Notes
> > >
> > >
> > >
> > > 2) Link to release candidate:
> > >
> > > https://github.com/apache/incubator-mxnet/releases/tag/1.5.1.rc0
> > >
> > >
> > >
> > > 3) Link to source and signatures on Apache dist server:
> > >
> > > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.5.1.rc0/
> > >
> > >
> > >
> > > Please remember to TEST first before voting accordingly:
> > >
> > > +1 = approve
> > >
> > > +0 = no opinion
> > >
> > > -1 = disapprove (provide reason)
> > >
> > >
> > >
> > > Thanks,
> > >
> > > -tao
> >
>

Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0

[Lin Yuan]

+1
Tested Horovod on GPU

On Wed, Sep 18, 2019 at 6:16 AM Zhao, Patric  wrote:

> +1
>
> Tested MKLDNN backend and everything looks great.
>
> > -Original Message-
> > From: Qing Lan 
> > Sent: Wednesday, September 18, 2019 2:20 AM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> >
> > +1 for Scala/Java test. Passed all tests for CPU/GPU build.
> > Also tested build from source with static build.
> >
> > Thanks,
> > Qing
> > 
> > From: Tao Lv 
> > Sent: Tuesday, September 17, 2019 14:14
> > To: dev@mxnet.incubator.apache.org 
> > Subject: [VOTE] Release Apache MXNet (incubating) 1.5.1.rc0
> >
> > Dear MXNet community,
> >
> >
> >
> > This is the 3-day vote to release Apache MXNet (incubating) version
> 1.5.1.
> >
> > Voting on dev@ will start September 17, 12:00pm (PST)  and close on
> > September 20, 12:00pm (PST).
> >
> >
> >
> > 1) Link to release notes:
> >
> > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Notes
> >
> >
> >
> > 2) Link to release candidate:
> >
> > https://github.com/apache/incubator-mxnet/releases/tag/1.5.1.rc0
> >
> >
> >
> > 3) Link to source and signatures on Apache dist server:
> >
> > https://dist.apache.org/repos/dist/dev/incubator/mxnet/1.5.1.rc0/
> >
> >
> >
> > Please remember to TEST first before voting accordingly:
> >
> > +1 = approve
> >
> > +0 = no opinion
> >
> > -1 = disapprove (provide reason)
> >
> >
> >
> > Thanks,
> >
> > -tao
>