Re: Breaking change to the model JSON file in 1.0.0 release

2017-12-12 Thread Bhavin Thaker
Marco: we were thinking on the lines of following Semantic versioning but
have not made the proposal to dev@ yet — plan to do that in a week or so.

Bhavin Thaker.

On Tue, Dec 12, 2017 at 4:04 PM Marco de Abreu 
wrote:

> Are we following semantic versioning https://semver.org/ for MXNet
> releases? If that's the case, 0.x was basically an unstable beta release
> and API changes could be expected at any case.
>
> But in general I totally agree, we should make users aware of the
> incompatibility. If it's just the rename, i propose a short printf if a
> file from an old version gets loaded and in case it contains the old name.
> Automatic conversion could have unknown side effects.
>
> -Marco
>
>
> Am 13.12.2017 12:58 vorm. schrieb "Indhu" :
>
> After the 1.0.0 release, we heard from multiple users that the model JSON
> file produced by 1.0.0 when a model is saved is not compatible with older
> versions of MXNet. Since there was no such change documented in the release
> notes, our first thought was that it must be an unintentional change.
>
> We looked at the commit logs and found out that the change was indeed
> intentional. Here is the PR that introduced the change:
> https://github.com/dmlc/nnvm/pull/152
>
> I think a breaking change like this must have been documented in the
> release notes.
>
> 1. Given this happened, what actions must we take to make sure such changes
> don't happen in the future without being documented in the release notes?
> 2. For users who ask, I assume we say the change was intentional and there
> is no plan to roll it back. Please let me know if that is not correct.
>
> Thanks,
> Indu
>


Re: Breaking change to the model JSON file in 1.0.0 release

2017-12-12 Thread Marco de Abreu
Are we following semantic versioning https://semver.org/ for MXNet
releases? If that's the case, 0.x was basically an unstable beta release
and API changes could be expected at any case.

But in general I totally agree, we should make users aware of the
incompatibility. If it's just the rename, i propose a short printf if a
file from an old version gets loaded and in case it contains the old name.
Automatic conversion could have unknown side effects.

-Marco


Am 13.12.2017 12:58 vorm. schrieb "Indhu" :

After the 1.0.0 release, we heard from multiple users that the model JSON
file produced by 1.0.0 when a model is saved is not compatible with older
versions of MXNet. Since there was no such change documented in the release
notes, our first thought was that it must be an unintentional change.

We looked at the commit logs and found out that the change was indeed
intentional. Here is the PR that introduced the change:
https://github.com/dmlc/nnvm/pull/152

I think a breaking change like this must have been documented in the
release notes.

1. Given this happened, what actions must we take to make sure such changes
don't happen in the future without being documented in the release notes?
2. For users who ask, I assume we say the change was intentional and there
is no plan to roll it back. Please let me know if that is not correct.

Thanks,
Indu


Breaking change to the model JSON file in 1.0.0 release

2017-12-12 Thread Indhu
After the 1.0.0 release, we heard from multiple users that the model JSON
file produced by 1.0.0 when a model is saved is not compatible with older
versions of MXNet. Since there was no such change documented in the release
notes, our first thought was that it must be an unintentional change.

We looked at the commit logs and found out that the change was indeed
intentional. Here is the PR that introduced the change:
https://github.com/dmlc/nnvm/pull/152

I think a breaking change like this must have been documented in the
release notes.

1. Given this happened, what actions must we take to make sure such changes
don't happen in the future without being documented in the release notes?
2. For users who ask, I assume we say the change was intentional and there
is no plan to roll it back. Please let me know if that is not correct.

Thanks,
Indu


Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread de Abreu, Marco
Thanks for looking into this, Chris! No hurries on that one, we’ll look into it 
next stage when we add new system- and build-configurations to the CI.

On 12.12.17, 19:12, "Chris Olivier"  wrote:

I am on vacation starting Thursday.

On Tue, Dec 12, 2017 at 9:49 AM kellen sunderland <
kellen.sunderl...@gmail.com> wrote:

> Absolutely, let's do an investigation and see if it's possible to
> virtualize.  Would you have time to look into it a bit further?
>
> On Tue, Dec 12, 2017 at 6:47 PM, Chris Olivier 
> wrote:
>
> > Don’t get me wrong, I’m not saying this Mac OS Jenkins solution is 
doable
> > but I feel like we should investigate because the payoff would be large.
> >
> >
> > On Tue, Dec 12, 2017 at 9:38 AM Chris Olivier 
> > wrote:
> >
> > > Apple’s Darwin OS Is recently open-sourced.
> > > https://github.com/PureDarwin/PureDarwin
> > >
> > > How to convert this into a non-GUI VM I am not sure but I am willing 
to
> > > bet that people have done it already.
> > >
> > > On Tue, Dec 12, 2017 at 9:16 AM kellen sunderland <
> > > kellen.sunderl...@gmail.com> wrote:
> > >
> > >> It might be technically possible, but I think it would violate the
> MacOS
> > >> license: http://store.apple.com/Catalog/US/Images/MacOSX.htm
> > >>
> > >> "2. Permitted License Uses and Restrictions.
> > >> A. This License allows you to install and use one copy of the Apple
> > >> Software on a single Apple-labeled computer at a time. This License
> does
> > >> not allow the Apple Software to exist on more than one computer at a
> > >> time,and you may not make the Apple Software available over a network
> > >> where
> > >> it could be used by multiple computers at the same time. You may make
> > one
> > >> copy of the Apple Software (excluding the Boot ROM code) in
> > >> machine-readable form for backup purposes only; provided that the
> backup
> > >> copy must include all copyright or other proprietary notices 
contained
> > on
> > >> the original. "
> > >>
> > >> I could be wrong though, does anyone know the details of MacOS
> > licensing /
> > >> virtualization?
> > >>
> > >> On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier  >
> > >> wrote:
> > >>
> > >> > googling seems to be full of running OSX (and even open-sourced
> > >> PureDarwin)
> > >> > in VMs. One could conceivably run a VM on an EC2 instance, right?
> > >> >
> > >> > On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
> > >> > kellen.sunderl...@gmail.com> wrote:
> > >> >
> > >> > > It would be ideal if we could cover OSX in Jenkins, but the only
> > >> solution
> > >> > > that I'm aware of would require physical machines to be the
> workers.
> > >> I
> > >> > > would be weakly opposed to having physical servers running on 
PRs.
> > >> The
> > >> > > downsides that I see in order of importance:
> > >> > >
> > >> > > -  We can't autoscale physical hardware.   If we find that the
> load
> > is
> > >> > too
> > >> > > high we have to buy more machines.
> > >> > > -  Security would be tricky, as they'd have to be connected to 
the
> > >> > internet
> > >> > > and then to our Jekins master instance.  Connecting via a wired
> > >> network
> > >> > > would probably not be possible on most corporate networks as 
these
> > >> > machines
> > >> > > are by definition running arbitrary code from the internet.  Many
> > >> > corporate
> > >> > > sites have public wifi that this machine could potentially 
connect
> > to,
> > >> > but
> > >> > > then our PRs start failing if the wifi disconnects temporarily.
> To
> > >> > connect
> > >> > > to the master we would need to setup a vpn solution with 
endpoints
> > in
> > >> our
> > >> > > vpc on AWS.  This is possible but would probably require a lot of
> > >> > security
> > >> > > work.
> > >> > > -  We can't just create a simple startup script or yaml file that
> is
> > >> > > checked into GitHub to manage the machine.  Someone will actually
> > >> have to
> > >> > > physically administer the machine, apply updates, etc. which will
> > make
> > >> > > community ownership difficult.
> > >> > >
> > >> > > Specific to an OSX build:
> > >> > > -  We can't virtualize OSX which means we'd only be able to cover
> > one
> > >> OSX
> > >> > > build environment per physical device.  We couldn't target a
> matrix
> > of
> > >> > OSX
> > >> > > and Xcode versions as in Travis.
> > >> > >
> > >> > > -Kellen
> > >> > >
> > >> > > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier <
> > cjolivie...@gmail.com
> > >> >
> > >> > > wrote:
> > >> > >
> > >> > > > So why Travis whe

Invite to apache-mxnet.slack.com request

2017-12-12 Thread Robert Hryniewicz
Thank you for the invitation to the ASF slack channel!

Could you please also invite me to the apache-mxnet.slack.com channel?

Thanks,

Robert Hryniewicz
Community & Data Evangelist | Data Scientist
(855) 846-7866 Ext: 94749




Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Chris Olivier
I am on vacation starting Thursday.

On Tue, Dec 12, 2017 at 9:49 AM kellen sunderland <
kellen.sunderl...@gmail.com> wrote:

> Absolutely, let's do an investigation and see if it's possible to
> virtualize.  Would you have time to look into it a bit further?
>
> On Tue, Dec 12, 2017 at 6:47 PM, Chris Olivier 
> wrote:
>
> > Don’t get me wrong, I’m not saying this Mac OS Jenkins solution is doable
> > but I feel like we should investigate because the payoff would be large.
> >
> >
> > On Tue, Dec 12, 2017 at 9:38 AM Chris Olivier 
> > wrote:
> >
> > > Apple’s Darwin OS Is recently open-sourced.
> > > https://github.com/PureDarwin/PureDarwin
> > >
> > > How to convert this into a non-GUI VM I am not sure but I am willing to
> > > bet that people have done it already.
> > >
> > > On Tue, Dec 12, 2017 at 9:16 AM kellen sunderland <
> > > kellen.sunderl...@gmail.com> wrote:
> > >
> > >> It might be technically possible, but I think it would violate the
> MacOS
> > >> license: http://store.apple.com/Catalog/US/Images/MacOSX.htm
> > >>
> > >> "2. Permitted License Uses and Restrictions.
> > >> A. This License allows you to install and use one copy of the Apple
> > >> Software on a single Apple-labeled computer at a time. This License
> does
> > >> not allow the Apple Software to exist on more than one computer at a
> > >> time,and you may not make the Apple Software available over a network
> > >> where
> > >> it could be used by multiple computers at the same time. You may make
> > one
> > >> copy of the Apple Software (excluding the Boot ROM code) in
> > >> machine-readable form for backup purposes only; provided that the
> backup
> > >> copy must include all copyright or other proprietary notices contained
> > on
> > >> the original. "
> > >>
> > >> I could be wrong though, does anyone know the details of MacOS
> > licensing /
> > >> virtualization?
> > >>
> > >> On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier  >
> > >> wrote:
> > >>
> > >> > googling seems to be full of running OSX (and even open-sourced
> > >> PureDarwin)
> > >> > in VMs. One could conceivably run a VM on an EC2 instance, right?
> > >> >
> > >> > On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
> > >> > kellen.sunderl...@gmail.com> wrote:
> > >> >
> > >> > > It would be ideal if we could cover OSX in Jenkins, but the only
> > >> solution
> > >> > > that I'm aware of would require physical machines to be the
> workers.
> > >> I
> > >> > > would be weakly opposed to having physical servers running on PRs.
> > >> The
> > >> > > downsides that I see in order of importance:
> > >> > >
> > >> > > -  We can't autoscale physical hardware.   If we find that the
> load
> > is
> > >> > too
> > >> > > high we have to buy more machines.
> > >> > > -  Security would be tricky, as they'd have to be connected to the
> > >> > internet
> > >> > > and then to our Jekins master instance.  Connecting via a wired
> > >> network
> > >> > > would probably not be possible on most corporate networks as these
> > >> > machines
> > >> > > are by definition running arbitrary code from the internet.  Many
> > >> > corporate
> > >> > > sites have public wifi that this machine could potentially connect
> > to,
> > >> > but
> > >> > > then our PRs start failing if the wifi disconnects temporarily.
> To
> > >> > connect
> > >> > > to the master we would need to setup a vpn solution with endpoints
> > in
> > >> our
> > >> > > vpc on AWS.  This is possible but would probably require a lot of
> > >> > security
> > >> > > work.
> > >> > > -  We can't just create a simple startup script or yaml file that
> is
> > >> > > checked into GitHub to manage the machine.  Someone will actually
> > >> have to
> > >> > > physically administer the machine, apply updates, etc. which will
> > make
> > >> > > community ownership difficult.
> > >> > >
> > >> > > Specific to an OSX build:
> > >> > > -  We can't virtualize OSX which means we'd only be able to cover
> > one
> > >> OSX
> > >> > > build environment per physical device.  We couldn't target a
> matrix
> > of
> > >> > OSX
> > >> > > and Xcode versions as in Travis.
> > >> > >
> > >> > > -Kellen
> > >> > >
> > >> > > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier <
> > cjolivie...@gmail.com
> > >> >
> > >> > > wrote:
> > >> > >
> > >> > > > So why Travis when we could possibly use Jenkins?
> > >> > > >
> > >> > > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> > >> > > > marco.g.ab...@googlemail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Yes that's correct, Chris.
> > >> > > > >
> > >> > > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
> > >> > > cjolivie...@gmail.com
> > >> > > > >:
> > >> > > > >
> > >> > > > > > A quick google search seems to indicate that Mac can be used
> > as
> > >> a
> > >> > > > Jenkins
> > >> > > > > > slave. Is this correct?
> > >> > > > > >
> > >> > > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> > >> > > > steffenroc...@gmail.com>
> > >> > > > > > wrote:
> > >> > > > > >

Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread kellen sunderland
Absolutely, let's do an investigation and see if it's possible to
virtualize.  Would you have time to look into it a bit further?

On Tue, Dec 12, 2017 at 6:47 PM, Chris Olivier 
wrote:

> Don’t get me wrong, I’m not saying this Mac OS Jenkins solution is doable
> but I feel like we should investigate because the payoff would be large.
>
>
> On Tue, Dec 12, 2017 at 9:38 AM Chris Olivier 
> wrote:
>
> > Apple’s Darwin OS Is recently open-sourced.
> > https://github.com/PureDarwin/PureDarwin
> >
> > How to convert this into a non-GUI VM I am not sure but I am willing to
> > bet that people have done it already.
> >
> > On Tue, Dec 12, 2017 at 9:16 AM kellen sunderland <
> > kellen.sunderl...@gmail.com> wrote:
> >
> >> It might be technically possible, but I think it would violate the MacOS
> >> license: http://store.apple.com/Catalog/US/Images/MacOSX.htm
> >>
> >> "2. Permitted License Uses and Restrictions.
> >> A. This License allows you to install and use one copy of the Apple
> >> Software on a single Apple-labeled computer at a time. This License does
> >> not allow the Apple Software to exist on more than one computer at a
> >> time,and you may not make the Apple Software available over a network
> >> where
> >> it could be used by multiple computers at the same time. You may make
> one
> >> copy of the Apple Software (excluding the Boot ROM code) in
> >> machine-readable form for backup purposes only; provided that the backup
> >> copy must include all copyright or other proprietary notices contained
> on
> >> the original. "
> >>
> >> I could be wrong though, does anyone know the details of MacOS
> licensing /
> >> virtualization?
> >>
> >> On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier 
> >> wrote:
> >>
> >> > googling seems to be full of running OSX (and even open-sourced
> >> PureDarwin)
> >> > in VMs. One could conceivably run a VM on an EC2 instance, right?
> >> >
> >> > On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
> >> > kellen.sunderl...@gmail.com> wrote:
> >> >
> >> > > It would be ideal if we could cover OSX in Jenkins, but the only
> >> solution
> >> > > that I'm aware of would require physical machines to be the workers.
> >> I
> >> > > would be weakly opposed to having physical servers running on PRs.
> >> The
> >> > > downsides that I see in order of importance:
> >> > >
> >> > > -  We can't autoscale physical hardware.   If we find that the load
> is
> >> > too
> >> > > high we have to buy more machines.
> >> > > -  Security would be tricky, as they'd have to be connected to the
> >> > internet
> >> > > and then to our Jekins master instance.  Connecting via a wired
> >> network
> >> > > would probably not be possible on most corporate networks as these
> >> > machines
> >> > > are by definition running arbitrary code from the internet.  Many
> >> > corporate
> >> > > sites have public wifi that this machine could potentially connect
> to,
> >> > but
> >> > > then our PRs start failing if the wifi disconnects temporarily.  To
> >> > connect
> >> > > to the master we would need to setup a vpn solution with endpoints
> in
> >> our
> >> > > vpc on AWS.  This is possible but would probably require a lot of
> >> > security
> >> > > work.
> >> > > -  We can't just create a simple startup script or yaml file that is
> >> > > checked into GitHub to manage the machine.  Someone will actually
> >> have to
> >> > > physically administer the machine, apply updates, etc. which will
> make
> >> > > community ownership difficult.
> >> > >
> >> > > Specific to an OSX build:
> >> > > -  We can't virtualize OSX which means we'd only be able to cover
> one
> >> OSX
> >> > > build environment per physical device.  We couldn't target a matrix
> of
> >> > OSX
> >> > > and Xcode versions as in Travis.
> >> > >
> >> > > -Kellen
> >> > >
> >> > > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier <
> cjolivie...@gmail.com
> >> >
> >> > > wrote:
> >> > >
> >> > > > So why Travis when we could possibly use Jenkins?
> >> > > >
> >> > > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> >> > > > marco.g.ab...@googlemail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Yes that's correct, Chris.
> >> > > > >
> >> > > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
> >> > > cjolivie...@gmail.com
> >> > > > >:
> >> > > > >
> >> > > > > > A quick google search seems to indicate that Mac can be used
> as
> >> a
> >> > > > Jenkins
> >> > > > > > slave. Is this correct?
> >> > > > > >
> >> > > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> >> > > > steffenroc...@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > +1 for #1 and #2
> >> > > > > > >
> >> > > > > > > I’m working on getting a MacPro to add to CI system.
> >> > > > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> >> > > > > > > kellen.sunderl...@gmail.com> wrote:
> >> > > > > > >
> >> > > > > > > > Background:  TravisCI is a startup providing managed
> >> continuous
> >> > > > > > > > integration services with GitHub

Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Chris Olivier
Don’t get me wrong, I’m not saying this Mac OS Jenkins solution is doable
but I feel like we should investigate because the payoff would be large.


On Tue, Dec 12, 2017 at 9:38 AM Chris Olivier  wrote:

> Apple’s Darwin OS Is recently open-sourced.
> https://github.com/PureDarwin/PureDarwin
>
> How to convert this into a non-GUI VM I am not sure but I am willing to
> bet that people have done it already.
>
> On Tue, Dec 12, 2017 at 9:16 AM kellen sunderland <
> kellen.sunderl...@gmail.com> wrote:
>
>> It might be technically possible, but I think it would violate the MacOS
>> license: http://store.apple.com/Catalog/US/Images/MacOSX.htm
>>
>> "2. Permitted License Uses and Restrictions.
>> A. This License allows you to install and use one copy of the Apple
>> Software on a single Apple-labeled computer at a time. This License does
>> not allow the Apple Software to exist on more than one computer at a
>> time,and you may not make the Apple Software available over a network
>> where
>> it could be used by multiple computers at the same time. You may make one
>> copy of the Apple Software (excluding the Boot ROM code) in
>> machine-readable form for backup purposes only; provided that the backup
>> copy must include all copyright or other proprietary notices contained on
>> the original. "
>>
>> I could be wrong though, does anyone know the details of MacOS licensing /
>> virtualization?
>>
>> On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier 
>> wrote:
>>
>> > googling seems to be full of running OSX (and even open-sourced
>> PureDarwin)
>> > in VMs. One could conceivably run a VM on an EC2 instance, right?
>> >
>> > On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
>> > kellen.sunderl...@gmail.com> wrote:
>> >
>> > > It would be ideal if we could cover OSX in Jenkins, but the only
>> solution
>> > > that I'm aware of would require physical machines to be the workers.
>> I
>> > > would be weakly opposed to having physical servers running on PRs.
>> The
>> > > downsides that I see in order of importance:
>> > >
>> > > -  We can't autoscale physical hardware.   If we find that the load is
>> > too
>> > > high we have to buy more machines.
>> > > -  Security would be tricky, as they'd have to be connected to the
>> > internet
>> > > and then to our Jekins master instance.  Connecting via a wired
>> network
>> > > would probably not be possible on most corporate networks as these
>> > machines
>> > > are by definition running arbitrary code from the internet.  Many
>> > corporate
>> > > sites have public wifi that this machine could potentially connect to,
>> > but
>> > > then our PRs start failing if the wifi disconnects temporarily.  To
>> > connect
>> > > to the master we would need to setup a vpn solution with endpoints in
>> our
>> > > vpc on AWS.  This is possible but would probably require a lot of
>> > security
>> > > work.
>> > > -  We can't just create a simple startup script or yaml file that is
>> > > checked into GitHub to manage the machine.  Someone will actually
>> have to
>> > > physically administer the machine, apply updates, etc. which will make
>> > > community ownership difficult.
>> > >
>> > > Specific to an OSX build:
>> > > -  We can't virtualize OSX which means we'd only be able to cover one
>> OSX
>> > > build environment per physical device.  We couldn't target a matrix of
>> > OSX
>> > > and Xcode versions as in Travis.
>> > >
>> > > -Kellen
>> > >
>> > > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier > >
>> > > wrote:
>> > >
>> > > > So why Travis when we could possibly use Jenkins?
>> > > >
>> > > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
>> > > > marco.g.ab...@googlemail.com>
>> > > > wrote:
>> > > >
>> > > > > Yes that's correct, Chris.
>> > > > >
>> > > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
>> > > cjolivie...@gmail.com
>> > > > >:
>> > > > >
>> > > > > > A quick google search seems to indicate that Mac can be used as
>> a
>> > > > Jenkins
>> > > > > > slave. Is this correct?
>> > > > > >
>> > > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
>> > > > steffenroc...@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > +1 for #1 and #2
>> > > > > > >
>> > > > > > > I’m working on getting a MacPro to add to CI system.
>> > > > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
>> > > > > > > kellen.sunderl...@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Background:  TravisCI is a startup providing managed
>> continuous
>> > > > > > > > integration services with GitHub integration and YAML based
>> > > > > > > configuration.
>> > > > > > > > TravisCI is one of the few CI providers that will build a
>> > variety
>> > > > of
>> > > > > > > > OSX/MacOS builds for software projects.  Their pricing
>> ranges
>> > > from
>> > > > > Free
>> > > > > > > > (for open source, 1 concurrent job, to $489 monthly for 10
>> > > > concurrent
>> > > > > > > jobs).
>> > > > > > > >
>> > > > > > > > Problem: We’ve had a few OSX build issues slip into M

Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Chris Olivier
Apple’s Darwin OS Is recently open-sourced.
https://github.com/PureDarwin/PureDarwin

How to convert this into a non-GUI VM I am not sure but I am willing to bet
that people have done it already.

On Tue, Dec 12, 2017 at 9:16 AM kellen sunderland <
kellen.sunderl...@gmail.com> wrote:

> It might be technically possible, but I think it would violate the MacOS
> license: http://store.apple.com/Catalog/US/Images/MacOSX.htm
>
> "2. Permitted License Uses and Restrictions.
> A. This License allows you to install and use one copy of the Apple
> Software on a single Apple-labeled computer at a time. This License does
> not allow the Apple Software to exist on more than one computer at a
> time,and you may not make the Apple Software available over a network where
> it could be used by multiple computers at the same time. You may make one
> copy of the Apple Software (excluding the Boot ROM code) in
> machine-readable form for backup purposes only; provided that the backup
> copy must include all copyright or other proprietary notices contained on
> the original. "
>
> I could be wrong though, does anyone know the details of MacOS licensing /
> virtualization?
>
> On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier 
> wrote:
>
> > googling seems to be full of running OSX (and even open-sourced
> PureDarwin)
> > in VMs. One could conceivably run a VM on an EC2 instance, right?
> >
> > On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
> > kellen.sunderl...@gmail.com> wrote:
> >
> > > It would be ideal if we could cover OSX in Jenkins, but the only
> solution
> > > that I'm aware of would require physical machines to be the workers.  I
> > > would be weakly opposed to having physical servers running on PRs.  The
> > > downsides that I see in order of importance:
> > >
> > > -  We can't autoscale physical hardware.   If we find that the load is
> > too
> > > high we have to buy more machines.
> > > -  Security would be tricky, as they'd have to be connected to the
> > internet
> > > and then to our Jekins master instance.  Connecting via a wired network
> > > would probably not be possible on most corporate networks as these
> > machines
> > > are by definition running arbitrary code from the internet.  Many
> > corporate
> > > sites have public wifi that this machine could potentially connect to,
> > but
> > > then our PRs start failing if the wifi disconnects temporarily.  To
> > connect
> > > to the master we would need to setup a vpn solution with endpoints in
> our
> > > vpc on AWS.  This is possible but would probably require a lot of
> > security
> > > work.
> > > -  We can't just create a simple startup script or yaml file that is
> > > checked into GitHub to manage the machine.  Someone will actually have
> to
> > > physically administer the machine, apply updates, etc. which will make
> > > community ownership difficult.
> > >
> > > Specific to an OSX build:
> > > -  We can't virtualize OSX which means we'd only be able to cover one
> OSX
> > > build environment per physical device.  We couldn't target a matrix of
> > OSX
> > > and Xcode versions as in Travis.
> > >
> > > -Kellen
> > >
> > > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier 
> > > wrote:
> > >
> > > > So why Travis when we could possibly use Jenkins?
> > > >
> > > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> > > > marco.g.ab...@googlemail.com>
> > > > wrote:
> > > >
> > > > > Yes that's correct, Chris.
> > > > >
> > > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
> > > cjolivie...@gmail.com
> > > > >:
> > > > >
> > > > > > A quick google search seems to indicate that Mac can be used as a
> > > > Jenkins
> > > > > > slave. Is this correct?
> > > > > >
> > > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> > > > steffenroc...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > +1 for #1 and #2
> > > > > > >
> > > > > > > I’m working on getting a MacPro to add to CI system.
> > > > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > > > > > > kellen.sunderl...@gmail.com> wrote:
> > > > > > >
> > > > > > > > Background:  TravisCI is a startup providing managed
> continuous
> > > > > > > > integration services with GitHub integration and YAML based
> > > > > > > configuration.
> > > > > > > > TravisCI is one of the few CI providers that will build a
> > variety
> > > > of
> > > > > > > > OSX/MacOS builds for software projects.  Their pricing ranges
> > > from
> > > > > Free
> > > > > > > > (for open source, 1 concurrent job, to $489 monthly for 10
> > > > concurrent
> > > > > > > jobs).
> > > > > > > >
> > > > > > > > Problem: We’ve had a few OSX build issues slip into MXNet
> > master
> > > in
> > > > > the
> > > > > > > > past few weeks.  We’ve previously had a Travis CI based
> testing
> > > > > system
> > > > > > > that
> > > > > > > > would have caught these issues.
> > > > > > > >
> > > > > > > > Proposals so far:
> > > > > > > >
> > > > > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity
> 

Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread kellen sunderland
It might be technically possible, but I think it would violate the MacOS
license: http://store.apple.com/Catalog/US/Images/MacOSX.htm

"2. Permitted License Uses and Restrictions.
A. This License allows you to install and use one copy of the Apple
Software on a single Apple-labeled computer at a time. This License does
not allow the Apple Software to exist on more than one computer at a
time,and you may not make the Apple Software available over a network where
it could be used by multiple computers at the same time. You may make one
copy of the Apple Software (excluding the Boot ROM code) in
machine-readable form for backup purposes only; provided that the backup
copy must include all copyright or other proprietary notices contained on
the original. "

I could be wrong though, does anyone know the details of MacOS licensing /
virtualization?

On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier 
wrote:

> googling seems to be full of running OSX (and even open-sourced PureDarwin)
> in VMs. One could conceivably run a VM on an EC2 instance, right?
>
> On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
> kellen.sunderl...@gmail.com> wrote:
>
> > It would be ideal if we could cover OSX in Jenkins, but the only solution
> > that I'm aware of would require physical machines to be the workers.  I
> > would be weakly opposed to having physical servers running on PRs.  The
> > downsides that I see in order of importance:
> >
> > -  We can't autoscale physical hardware.   If we find that the load is
> too
> > high we have to buy more machines.
> > -  Security would be tricky, as they'd have to be connected to the
> internet
> > and then to our Jekins master instance.  Connecting via a wired network
> > would probably not be possible on most corporate networks as these
> machines
> > are by definition running arbitrary code from the internet.  Many
> corporate
> > sites have public wifi that this machine could potentially connect to,
> but
> > then our PRs start failing if the wifi disconnects temporarily.  To
> connect
> > to the master we would need to setup a vpn solution with endpoints in our
> > vpc on AWS.  This is possible but would probably require a lot of
> security
> > work.
> > -  We can't just create a simple startup script or yaml file that is
> > checked into GitHub to manage the machine.  Someone will actually have to
> > physically administer the machine, apply updates, etc. which will make
> > community ownership difficult.
> >
> > Specific to an OSX build:
> > -  We can't virtualize OSX which means we'd only be able to cover one OSX
> > build environment per physical device.  We couldn't target a matrix of
> OSX
> > and Xcode versions as in Travis.
> >
> > -Kellen
> >
> > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier 
> > wrote:
> >
> > > So why Travis when we could possibly use Jenkins?
> > >
> > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> > > marco.g.ab...@googlemail.com>
> > > wrote:
> > >
> > > > Yes that's correct, Chris.
> > > >
> > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
> > cjolivie...@gmail.com
> > > >:
> > > >
> > > > > A quick google search seems to indicate that Mac can be used as a
> > > Jenkins
> > > > > slave. Is this correct?
> > > > >
> > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> > > steffenroc...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1 for #1 and #2
> > > > > >
> > > > > > I’m working on getting a MacPro to add to CI system.
> > > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > > > > > kellen.sunderl...@gmail.com> wrote:
> > > > > >
> > > > > > > Background:  TravisCI is a startup providing managed continuous
> > > > > > > integration services with GitHub integration and YAML based
> > > > > > configuration.
> > > > > > > TravisCI is one of the few CI providers that will build a
> variety
> > > of
> > > > > > > OSX/MacOS builds for software projects.  Their pricing ranges
> > from
> > > > Free
> > > > > > > (for open source, 1 concurrent job, to $489 monthly for 10
> > > concurrent
> > > > > > jobs).
> > > > > > >
> > > > > > > Problem: We’ve had a few OSX build issues slip into MXNet
> master
> > in
> > > > the
> > > > > > > past few weeks.  We’ve previously had a Travis CI based testing
> > > > system
> > > > > > that
> > > > > > > would have caught these issues.
> > > > > > >
> > > > > > > Proposals so far:
> > > > > > >
> > > > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity
> check
> > > on
> > > > > OSX.
> > > > > > > If we compile the program, and for example run C++ unit tests
> > we’re
> > > > > > > unlikely to run into problems with queued builds.  The total
> > build
> > > > time
> > > > > > > here should be less than 15 minutes.  Configuration should be
> > quite
> > > > > > simple
> > > > > > > and easy to maintain.  Error messages should also be obvious to
> > > > > > > contributors.
> > > > > > > 2) Run clang in Linux with our current CI.  Building with clang
> > > > should
> > > > > 

Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Chris Olivier
googling seems to be full of running OSX (and even open-sourced PureDarwin)
in VMs. One could conceivably run a VM on an EC2 instance, right?

On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
kellen.sunderl...@gmail.com> wrote:

> It would be ideal if we could cover OSX in Jenkins, but the only solution
> that I'm aware of would require physical machines to be the workers.  I
> would be weakly opposed to having physical servers running on PRs.  The
> downsides that I see in order of importance:
>
> -  We can't autoscale physical hardware.   If we find that the load is too
> high we have to buy more machines.
> -  Security would be tricky, as they'd have to be connected to the internet
> and then to our Jekins master instance.  Connecting via a wired network
> would probably not be possible on most corporate networks as these machines
> are by definition running arbitrary code from the internet.  Many corporate
> sites have public wifi that this machine could potentially connect to, but
> then our PRs start failing if the wifi disconnects temporarily.  To connect
> to the master we would need to setup a vpn solution with endpoints in our
> vpc on AWS.  This is possible but would probably require a lot of security
> work.
> -  We can't just create a simple startup script or yaml file that is
> checked into GitHub to manage the machine.  Someone will actually have to
> physically administer the machine, apply updates, etc. which will make
> community ownership difficult.
>
> Specific to an OSX build:
> -  We can't virtualize OSX which means we'd only be able to cover one OSX
> build environment per physical device.  We couldn't target a matrix of OSX
> and Xcode versions as in Travis.
>
> -Kellen
>
> On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier 
> wrote:
>
> > So why Travis when we could possibly use Jenkins?
> >
> > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> > marco.g.ab...@googlemail.com>
> > wrote:
> >
> > > Yes that's correct, Chris.
> > >
> > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
> cjolivie...@gmail.com
> > >:
> > >
> > > > A quick google search seems to indicate that Mac can be used as a
> > Jenkins
> > > > slave. Is this correct?
> > > >
> > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> > steffenroc...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 for #1 and #2
> > > > >
> > > > > I’m working on getting a MacPro to add to CI system.
> > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > > > > kellen.sunderl...@gmail.com> wrote:
> > > > >
> > > > > > Background:  TravisCI is a startup providing managed continuous
> > > > > > integration services with GitHub integration and YAML based
> > > > > configuration.
> > > > > > TravisCI is one of the few CI providers that will build a variety
> > of
> > > > > > OSX/MacOS builds for software projects.  Their pricing ranges
> from
> > > Free
> > > > > > (for open source, 1 concurrent job, to $489 monthly for 10
> > concurrent
> > > > > jobs).
> > > > > >
> > > > > > Problem: We’ve had a few OSX build issues slip into MXNet master
> in
> > > the
> > > > > > past few weeks.  We’ve previously had a Travis CI based testing
> > > system
> > > > > that
> > > > > > would have caught these issues.
> > > > > >
> > > > > > Proposals so far:
> > > > > >
> > > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity check
> > on
> > > > OSX.
> > > > > > If we compile the program, and for example run C++ unit tests
> we’re
> > > > > > unlikely to run into problems with queued builds.  The total
> build
> > > time
> > > > > > here should be less than 15 minutes.  Configuration should be
> quite
> > > > > simple
> > > > > > and easy to maintain.  Error messages should also be obvious to
> > > > > > contributors.
> > > > > > 2) Run clang in Linux with our current CI.  Building with clang
> > > should
> > > > > > take less than 10 minutes, should flush out a large subset of the
> > > > issues
> > > > > > we’ve seen with OSX, and be quite easy to maintain.
> > > > > > 3) Run full test-suites in TravisCI, equaling the level of
> coverage
> > > we
> > > > > > provide to Linux in Jenkins.  This could require us to subscribe
> > to a
> > > > > > monthly package with Travis to ensure our build queue doesn’t
> grow
> > to
> > > > an
> > > > > > unacceptable length.  It may also require a volunteer to setup
> and
> > > > > maintain
> > > > > > long-term.
> > > > > >
> > > > > > I’d +1 #1 and #2 as I think those should be low-cost,
> low-maintence
> > > > > > solutions that should catch the majority of the problems we’ve
> seen
> > > > thus
> > > > > > far.
> > > > > >
> > > > > > -Kellen
> > > > > >
> > > > >
> > > >
> > >
> >
>


Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread kellen sunderland
It would be ideal if we could cover OSX in Jenkins, but the only solution
that I'm aware of would require physical machines to be the workers.  I
would be weakly opposed to having physical servers running on PRs.  The
downsides that I see in order of importance:

-  We can't autoscale physical hardware.   If we find that the load is too
high we have to buy more machines.
-  Security would be tricky, as they'd have to be connected to the internet
and then to our Jekins master instance.  Connecting via a wired network
would probably not be possible on most corporate networks as these machines
are by definition running arbitrary code from the internet.  Many corporate
sites have public wifi that this machine could potentially connect to, but
then our PRs start failing if the wifi disconnects temporarily.  To connect
to the master we would need to setup a vpn solution with endpoints in our
vpc on AWS.  This is possible but would probably require a lot of security
work.
-  We can't just create a simple startup script or yaml file that is
checked into GitHub to manage the machine.  Someone will actually have to
physically administer the machine, apply updates, etc. which will make
community ownership difficult.

Specific to an OSX build:
-  We can't virtualize OSX which means we'd only be able to cover one OSX
build environment per physical device.  We couldn't target a matrix of OSX
and Xcode versions as in Travis.

-Kellen

On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier 
wrote:

> So why Travis when we could possibly use Jenkins?
>
> On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> marco.g.ab...@googlemail.com>
> wrote:
>
> > Yes that's correct, Chris.
> >
> > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier"  >:
> >
> > > A quick google search seems to indicate that Mac can be used as a
> Jenkins
> > > slave. Is this correct?
> > >
> > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> steffenroc...@gmail.com>
> > > wrote:
> > >
> > > > +1 for #1 and #2
> > > >
> > > > I’m working on getting a MacPro to add to CI system.
> > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > > > kellen.sunderl...@gmail.com> wrote:
> > > >
> > > > > Background:  TravisCI is a startup providing managed continuous
> > > > > integration services with GitHub integration and YAML based
> > > > configuration.
> > > > > TravisCI is one of the few CI providers that will build a variety
> of
> > > > > OSX/MacOS builds for software projects.  Their pricing ranges from
> > Free
> > > > > (for open source, 1 concurrent job, to $489 monthly for 10
> concurrent
> > > > jobs).
> > > > >
> > > > > Problem: We’ve had a few OSX build issues slip into MXNet master in
> > the
> > > > > past few weeks.  We’ve previously had a Travis CI based testing
> > system
> > > > that
> > > > > would have caught these issues.
> > > > >
> > > > > Proposals so far:
> > > > >
> > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity check
> on
> > > OSX.
> > > > > If we compile the program, and for example run C++ unit tests we’re
> > > > > unlikely to run into problems with queued builds.  The total build
> > time
> > > > > here should be less than 15 minutes.  Configuration should be quite
> > > > simple
> > > > > and easy to maintain.  Error messages should also be obvious to
> > > > > contributors.
> > > > > 2) Run clang in Linux with our current CI.  Building with clang
> > should
> > > > > take less than 10 minutes, should flush out a large subset of the
> > > issues
> > > > > we’ve seen with OSX, and be quite easy to maintain.
> > > > > 3) Run full test-suites in TravisCI, equaling the level of coverage
> > we
> > > > > provide to Linux in Jenkins.  This could require us to subscribe
> to a
> > > > > monthly package with Travis to ensure our build queue doesn’t grow
> to
> > > an
> > > > > unacceptable length.  It may also require a volunteer to setup and
> > > > maintain
> > > > > long-term.
> > > > >
> > > > > I’d +1 #1 and #2 as I think those should be low-cost, low-maintence
> > > > > solutions that should catch the majority of the problems we’ve seen
> > > thus
> > > > > far.
> > > > >
> > > > > -Kellen
> > > > >
> > > >
> > >
> >
>


Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Chris Olivier
So why Travis when we could possibly use Jenkins?

On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu 
wrote:

> Yes that's correct, Chris.
>
> Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" :
>
> > A quick google search seems to indicate that Mac can be used as a Jenkins
> > slave. Is this correct?
> >
> > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel 
> > wrote:
> >
> > > +1 for #1 and #2
> > >
> > > I’m working on getting a MacPro to add to CI system.
> > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > > kellen.sunderl...@gmail.com> wrote:
> > >
> > > > Background:  TravisCI is a startup providing managed continuous
> > > > integration services with GitHub integration and YAML based
> > > configuration.
> > > > TravisCI is one of the few CI providers that will build a variety of
> > > > OSX/MacOS builds for software projects.  Their pricing ranges from
> Free
> > > > (for open source, 1 concurrent job, to $489 monthly for 10 concurrent
> > > jobs).
> > > >
> > > > Problem: We’ve had a few OSX build issues slip into MXNet master in
> the
> > > > past few weeks.  We’ve previously had a Travis CI based testing
> system
> > > that
> > > > would have caught these issues.
> > > >
> > > > Proposals so far:
> > > >
> > > > 1) Use TravisCI in it’s free mode for a very minimal sanity check on
> > OSX.
> > > > If we compile the program, and for example run C++ unit tests we’re
> > > > unlikely to run into problems with queued builds.  The total build
> time
> > > > here should be less than 15 minutes.  Configuration should be quite
> > > simple
> > > > and easy to maintain.  Error messages should also be obvious to
> > > > contributors.
> > > > 2) Run clang in Linux with our current CI.  Building with clang
> should
> > > > take less than 10 minutes, should flush out a large subset of the
> > issues
> > > > we’ve seen with OSX, and be quite easy to maintain.
> > > > 3) Run full test-suites in TravisCI, equaling the level of coverage
> we
> > > > provide to Linux in Jenkins.  This could require us to subscribe to a
> > > > monthly package with Travis to ensure our build queue doesn’t grow to
> > an
> > > > unacceptable length.  It may also require a volunteer to setup and
> > > maintain
> > > > long-term.
> > > >
> > > > I’d +1 #1 and #2 as I think those should be low-cost, low-maintence
> > > > solutions that should catch the majority of the problems we’ve seen
> > thus
> > > > far.
> > > >
> > > > -Kellen
> > > >
> > >
> >
>


Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Marco de Abreu
Yes that's correct, Chris.

Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" :

> A quick google search seems to indicate that Mac can be used as a Jenkins
> slave. Is this correct?
>
> On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel 
> wrote:
>
> > +1 for #1 and #2
> >
> > I’m working on getting a MacPro to add to CI system.
> > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > kellen.sunderl...@gmail.com> wrote:
> >
> > > Background:  TravisCI is a startup providing managed continuous
> > > integration services with GitHub integration and YAML based
> > configuration.
> > > TravisCI is one of the few CI providers that will build a variety of
> > > OSX/MacOS builds for software projects.  Their pricing ranges from Free
> > > (for open source, 1 concurrent job, to $489 monthly for 10 concurrent
> > jobs).
> > >
> > > Problem: We’ve had a few OSX build issues slip into MXNet master in the
> > > past few weeks.  We’ve previously had a Travis CI based testing system
> > that
> > > would have caught these issues.
> > >
> > > Proposals so far:
> > >
> > > 1) Use TravisCI in it’s free mode for a very minimal sanity check on
> OSX.
> > > If we compile the program, and for example run C++ unit tests we’re
> > > unlikely to run into problems with queued builds.  The total build time
> > > here should be less than 15 minutes.  Configuration should be quite
> > simple
> > > and easy to maintain.  Error messages should also be obvious to
> > > contributors.
> > > 2) Run clang in Linux with our current CI.  Building with clang should
> > > take less than 10 minutes, should flush out a large subset of the
> issues
> > > we’ve seen with OSX, and be quite easy to maintain.
> > > 3) Run full test-suites in TravisCI, equaling the level of coverage we
> > > provide to Linux in Jenkins.  This could require us to subscribe to a
> > > monthly package with Travis to ensure our build queue doesn’t grow to
> an
> > > unacceptable length.  It may also require a volunteer to setup and
> > maintain
> > > long-term.
> > >
> > > I’d +1 #1 and #2 as I think those should be low-cost, low-maintence
> > > solutions that should catch the majority of the problems we’ve seen
> thus
> > > far.
> > >
> > > -Kellen
> > >
> >
>


Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Chris Olivier
A quick google search seems to indicate that Mac can be used as a Jenkins
slave. Is this correct?

On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel 
wrote:

> +1 for #1 and #2
>
> I’m working on getting a MacPro to add to CI system.
> On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> kellen.sunderl...@gmail.com> wrote:
>
> > Background:  TravisCI is a startup providing managed continuous
> > integration services with GitHub integration and YAML based
> configuration.
> > TravisCI is one of the few CI providers that will build a variety of
> > OSX/MacOS builds for software projects.  Their pricing ranges from Free
> > (for open source, 1 concurrent job, to $489 monthly for 10 concurrent
> jobs).
> >
> > Problem: We’ve had a few OSX build issues slip into MXNet master in the
> > past few weeks.  We’ve previously had a Travis CI based testing system
> that
> > would have caught these issues.
> >
> > Proposals so far:
> >
> > 1) Use TravisCI in it’s free mode for a very minimal sanity check on OSX.
> > If we compile the program, and for example run C++ unit tests we’re
> > unlikely to run into problems with queued builds.  The total build time
> > here should be less than 15 minutes.  Configuration should be quite
> simple
> > and easy to maintain.  Error messages should also be obvious to
> > contributors.
> > 2) Run clang in Linux with our current CI.  Building with clang should
> > take less than 10 minutes, should flush out a large subset of the issues
> > we’ve seen with OSX, and be quite easy to maintain.
> > 3) Run full test-suites in TravisCI, equaling the level of coverage we
> > provide to Linux in Jenkins.  This could require us to subscribe to a
> > monthly package with Travis to ensure our build queue doesn’t grow to an
> > unacceptable length.  It may also require a volunteer to setup and
> maintain
> > long-term.
> >
> > I’d +1 #1 and #2 as I think those should be low-cost, low-maintence
> > solutions that should catch the majority of the problems we’ve seen thus
> > far.
> >
> > -Kellen
> >
>


Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread Steffen Rochel
+1 for #1 and #2

I’m working on getting a MacPro to add to CI system.
On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
kellen.sunderl...@gmail.com> wrote:

> Background:  TravisCI is a startup providing managed continuous
> integration services with GitHub integration and YAML based configuration.
> TravisCI is one of the few CI providers that will build a variety of
> OSX/MacOS builds for software projects.  Their pricing ranges from Free
> (for open source, 1 concurrent job, to $489 monthly for 10 concurrent jobs).
>
> Problem: We’ve had a few OSX build issues slip into MXNet master in the
> past few weeks.  We’ve previously had a Travis CI based testing system that
> would have caught these issues.
>
> Proposals so far:
>
> 1) Use TravisCI in it’s free mode for a very minimal sanity check on OSX.
> If we compile the program, and for example run C++ unit tests we’re
> unlikely to run into problems with queued builds.  The total build time
> here should be less than 15 minutes.  Configuration should be quite simple
> and easy to maintain.  Error messages should also be obvious to
> contributors.
> 2) Run clang in Linux with our current CI.  Building with clang should
> take less than 10 minutes, should flush out a large subset of the issues
> we’ve seen with OSX, and be quite easy to maintain.
> 3) Run full test-suites in TravisCI, equaling the level of coverage we
> provide to Linux in Jenkins.  This could require us to subscribe to a
> monthly package with Travis to ensure our build queue doesn’t grow to an
> unacceptable length.  It may also require a volunteer to setup and maintain
> long-term.
>
> I’d +1 #1 and #2 as I think those should be low-cost, low-maintence
> solutions that should catch the majority of the problems we’ve seen thus
> far.
>
> -Kellen
>


[DISCUSS] Build OSX builds in CI (possibly with TravisCI).

2017-12-12 Thread kellen sunderland
Background:  TravisCI is a startup providing managed continuous integration 
services with GitHub integration and YAML based configuration.  TravisCI is one 
of the few CI providers that will build a variety of OSX/MacOS builds for 
software projects.  Their pricing ranges from Free (for open source, 1 
concurrent job, to $489 monthly for 10 concurrent jobs).  

Problem: We’ve had a few OSX build issues slip into MXNet master in the past 
few weeks.  We’ve previously had a Travis CI based testing system that would 
have caught these issues.

Proposals so far:

1) Use TravisCI in it’s free mode for a very minimal sanity check on OSX.  If 
we compile the program, and for example run C++ unit tests we’re unlikely to 
run into problems with queued builds.  The total build time here should be less 
than 15 minutes.  Configuration should be quite simple and easy to maintain.  
Error messages should also be obvious to contributors.
2) Run clang in Linux with our current CI.  Building with clang should take 
less than 10 minutes, should flush out a large subset of the issues we’ve seen 
with OSX, and be quite easy to maintain.
3) Run full test-suites in TravisCI, equaling the level of coverage we provide 
to Linux in Jenkins.  This could require us to subscribe to a monthly package 
with Travis to ensure our build queue doesn’t grow to an unacceptable length.  
It may also require a volunteer to setup and maintain long-term.

I’d +1 #1 and #2 as I think those should be low-cost, low-maintence solutions 
that should catch the majority of the problems we’ve seen thus far.

-Kellen


RE: [VOTE RESULT] Disable Appveyor

2017-12-12 Thread kellen sunderland
Thanks for the votes all.  As a summary we have:

Binding:
Chris Olivier +1
Indhu Bharathi +1

Non-Binding:
Kellen Sunderland +1
Marco de Abreu +1
Pedro Larroy +1
Asmus Hertzel +1
Daniel Bay +1

Looks like the vote has passed, and I believe Sheng has already opened a ticket 
to remove Appveyor (thanks Sheng and Seb).  
https://issues.apache.org/jira/browse/INFRA-15652

-Kellen
From: Pedro Larroy
Sent: Tuesday, December 12, 2017 4:51 AM
To: dev@mxnet.incubator.apache.org
Subject: Re: [VOTE] Disable Appveyor

+1

Queued for 16 hours:


https://ci.appveyor.com/project/ApacheSoftwareFoundation/incubator-mxnet/build/1.0.4199
https://github.com/apache/incubator-mxnet/pull/9016


On Fri, Dec 8, 2017 at 12:19 PM, Bay, Daniel  wrote:
> +1
>
> On 07.12.17, 23:41, "Indhu"  wrote:
>
> +1
>
> My build is waiting in queue for 13 hours now.
>
>
> On Thu, Dec 7, 2017, 11:26 AM kellen sunderland 
> 
> wrote:
>
> > Background:  Appveyor is a free-to-use (for opensource) CI system that
> > specializes in providing support for Windows environments.  Appveyor is
> > currently running a very simple windows build for MXNet.  Specifically 
> we
> > are generating build files via cmake, and then building with Visual 
> Studio
> > 14 with fairly standard build options.
> >
> > Problem: AppVeyor frequently slows down our release process and has been
> > made redundant by the new CI system's windows builds.
> >
> > Action: Let's remove AppVeyor and rely on the new CI's windows build
> > systems which has very similar functionality.
> >
> > Vote will close Monday Dec 11th and will rely on lazy consensus.
> >
> > -Kellen
> >
>
>