Re: Config for Scala Maven Package

[Naveen Swamy]

Anirudh, I have tried to fix and automate parts of the publish process and
written down the process. If everything works as expected & since it takes
building on 3 different platforms it could take 4-6 hours. We will also
continue to automate this further. This is excluding the time it needs to
fix various bugs if any.

On Mon, Jun 18, 2018 at 11:39 AM, Anirudh  wrote:

> Hi Naveen,
>
> Sorry I missed this. Can you please elaborate on how long it takes
> typically for publishing the scala packages to maven before the vote?
> I think if there is considerable churn in publishing these packages today,
> we should not wait for the package for the vote as we may sacrifice our big
> python user base
> getting the release early for the scala user base.
>
> Anirudh
>
>
> On Thu, May 31, 2018 at 7:27 AM, Naveen Swamy  wrote:
>
> > All, I have created a guide to maven publish process here;
> > https://cwiki.apache.org/confluence/display/MXNET/
> > MXNet-Scala+Release+Process
> > .
> > Going forward, I propose that we include the scala maven packages also in
> > our RC voting(I have updated the release-process already), I understand
> the
> > maven packages use the same code base that is sent for voting. I
> > encountered a quite bit of issues trying to publish the package, this
> could
> > because we never did this past and I have automated as much as possible.
> I
> > think there is value in getting these packages tested during the RC
> > process.
> >
> > Sandeep,
> > I am using what is available on AWS Deep Learning Base AMI which is 3.4
> and
> > CUDNN 7.0.5.
> >
> > Thanks, Naveen
> >
> > On Wed, May 23, 2018 at 8:57 AM, sandeep krishnamurthy <
> > sandeep.krishn...@gmail.com> wrote:
> >
> > > Hi Naveen,
> > >
> > > Great work.
> > > I was wondering about OpenCV and cuDNN
> > >
> > > Thanks,
> > > Sandeep
> > >
> > > On Mon, May 21, 2018 at 5:44 PM, Naveen Swamy 
> > wrote:
> > >
> > > > Hi Marco,
> > > > For MKL/MKLDNN --> I don't think we should build since its still
> > > > experimental. Its sad that MKLML was removed without a proper
> > discussion,
> > > > granted
> > > > I have not tested with CUDA 9.2, I will building on a AWS Ubuntu
> > instance
> > > > that is built with CUDA 9.0, I presume it should work fine.
> > > >
> > > >
> > > > On Mon, May 21, 2018 at 5:31 PM, Marco de Abreu <
> > > > marco.g.ab...@googlemail.com> wrote:
> > > >
> > > > > Hi Naveen,
> > > > >
> > > > > thank you for driving the releases for the Scala packages!
> > > > >
> > > > > For CPU, I'd propose add MKL/MKLDNN builds.
> > > > >
> > > > > For GPU, do we have to specify the minor version of CUDA up front
> or
> > > are
> > > > > they interchangeable? CUDA 9.2, for example, received some quite
> > > > > significant performance updates and it would be great if our users
> > > could
> > > > > use them.
> > > > >
> > > > > Best regards,
> > > > > Marco
> > > > >
> > > > > On Fri, May 18, 2018 at 8:28 PM, Naveen Swamy 
> > > > wrote:
> > > > >
> > > > > > I am working on publishing the Scala package from 1.2.0 release.
> > What
> > > > do
> > > > > > you recommend should be the configuration for the Scala Packages
> ?
> > > > > >
> > > > > > I am thinking of the below
> > > > > > For CPU build with
> > > > > > OPENBLAS
> > > > > > LAPACK
> > > > > >
> > > > > > For GPU build with
> > > > > > OPENBLAS
> > > > > > CUDA 9.0
> > > > > > NCCL
> > > > > >
> > > > > > Let me know if you concerns or recommendations.
> > > > > >
> > > > > > Thanks, Naveen
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Sandeep Krishnamurthy
> > >
> >
>

Re: Config for Scala Maven Package

[Anirudh]

Hi Naveen,

Sorry I missed this. Can you please elaborate on how long it takes
typically for publishing the scala packages to maven before the vote?
I think if there is considerable churn in publishing these packages today,
we should not wait for the package for the vote as we may sacrifice our big
python user base
getting the release early for the scala user base.

Anirudh


On Thu, May 31, 2018 at 7:27 AM, Naveen Swamy  wrote:

> All, I have created a guide to maven publish process here;
> https://cwiki.apache.org/confluence/display/MXNET/
> MXNet-Scala+Release+Process
> .
> Going forward, I propose that we include the scala maven packages also in
> our RC voting(I have updated the release-process already), I understand the
> maven packages use the same code base that is sent for voting. I
> encountered a quite bit of issues trying to publish the package, this could
> because we never did this past and I have automated as much as possible. I
> think there is value in getting these packages tested during the RC
> process.
>
> Sandeep,
> I am using what is available on AWS Deep Learning Base AMI which is 3.4 and
> CUDNN 7.0.5.
>
> Thanks, Naveen
>
> On Wed, May 23, 2018 at 8:57 AM, sandeep krishnamurthy <
> sandeep.krishn...@gmail.com> wrote:
>
> > Hi Naveen,
> >
> > Great work.
> > I was wondering about OpenCV and cuDNN
> >
> > Thanks,
> > Sandeep
> >
> > On Mon, May 21, 2018 at 5:44 PM, Naveen Swamy 
> wrote:
> >
> > > Hi Marco,
> > > For MKL/MKLDNN --> I don't think we should build since its still
> > > experimental. Its sad that MKLML was removed without a proper
> discussion,
> > > granted
> > > I have not tested with CUDA 9.2, I will building on a AWS Ubuntu
> instance
> > > that is built with CUDA 9.0, I presume it should work fine.
> > >
> > >
> > > On Mon, May 21, 2018 at 5:31 PM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > Hi Naveen,
> > > >
> > > > thank you for driving the releases for the Scala packages!
> > > >
> > > > For CPU, I'd propose add MKL/MKLDNN builds.
> > > >
> > > > For GPU, do we have to specify the minor version of CUDA up front or
> > are
> > > > they interchangeable? CUDA 9.2, for example, received some quite
> > > > significant performance updates and it would be great if our users
> > could
> > > > use them.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > On Fri, May 18, 2018 at 8:28 PM, Naveen Swamy 
> > > wrote:
> > > >
> > > > > I am working on publishing the Scala package from 1.2.0 release.
> What
> > > do
> > > > > you recommend should be the configuration for the Scala Packages ?
> > > > >
> > > > > I am thinking of the below
> > > > > For CPU build with
> > > > > OPENBLAS
> > > > > LAPACK
> > > > >
> > > > > For GPU build with
> > > > > OPENBLAS
> > > > > CUDA 9.0
> > > > > NCCL
> > > > >
> > > > > Let me know if you concerns or recommendations.
> > > > >
> > > > > Thanks, Naveen
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Sandeep Krishnamurthy
> >
>

Re: Config for Scala Maven Package

[Carin Meier]

Thanks for your work on this!

It's going to make it much easier for Scala, (and soon Clojure users), to
get up and running on MXNet.

- Carin

On Thu, May 31, 2018 at 10:27 AM, Naveen Swamy  wrote:

> All, I have created a guide to maven publish process here;
> https://cwiki.apache.org/confluence/display/MXNET/
> MXNet-Scala+Release+Process
> .
> Going forward, I propose that we include the scala maven packages also in
> our RC voting(I have updated the release-process already), I understand the
> maven packages use the same code base that is sent for voting. I
> encountered a quite bit of issues trying to publish the package, this could
> because we never did this past and I have automated as much as possible. I
> think there is value in getting these packages tested during the RC
> process.
>
> Sandeep,
> I am using what is available on AWS Deep Learning Base AMI which is 3.4 and
> CUDNN 7.0.5.
>
> Thanks, Naveen
>
> On Wed, May 23, 2018 at 8:57 AM, sandeep krishnamurthy <
> sandeep.krishn...@gmail.com> wrote:
>
> > Hi Naveen,
> >
> > Great work.
> > I was wondering about OpenCV and cuDNN
> >
> > Thanks,
> > Sandeep
> >
> > On Mon, May 21, 2018 at 5:44 PM, Naveen Swamy 
> wrote:
> >
> > > Hi Marco,
> > > For MKL/MKLDNN --> I don't think we should build since its still
> > > experimental. Its sad that MKLML was removed without a proper
> discussion,
> > > granted
> > > I have not tested with CUDA 9.2, I will building on a AWS Ubuntu
> instance
> > > that is built with CUDA 9.0, I presume it should work fine.
> > >
> > >
> > > On Mon, May 21, 2018 at 5:31 PM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com> wrote:
> > >
> > > > Hi Naveen,
> > > >
> > > > thank you for driving the releases for the Scala packages!
> > > >
> > > > For CPU, I'd propose add MKL/MKLDNN builds.
> > > >
> > > > For GPU, do we have to specify the minor version of CUDA up front or
> > are
> > > > they interchangeable? CUDA 9.2, for example, received some quite
> > > > significant performance updates and it would be great if our users
> > could
> > > > use them.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > On Fri, May 18, 2018 at 8:28 PM, Naveen Swamy 
> > > wrote:
> > > >
> > > > > I am working on publishing the Scala package from 1.2.0 release.
> What
> > > do
> > > > > you recommend should be the configuration for the Scala Packages ?
> > > > >
> > > > > I am thinking of the below
> > > > > For CPU build with
> > > > > OPENBLAS
> > > > > LAPACK
> > > > >
> > > > > For GPU build with
> > > > > OPENBLAS
> > > > > CUDA 9.0
> > > > > NCCL
> > > > >
> > > > > Let me know if you concerns or recommendations.
> > > > >
> > > > > Thanks, Naveen
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Sandeep Krishnamurthy
> >
>

Re: Config for Scala Maven Package

[Naveen Swamy]

All, I have created a guide to maven publish process here;
https://cwiki.apache.org/confluence/display/MXNET/MXNet-Scala+Release+Process
.
Going forward, I propose that we include the scala maven packages also in
our RC voting(I have updated the release-process already), I understand the
maven packages use the same code base that is sent for voting. I
encountered a quite bit of issues trying to publish the package, this could
because we never did this past and I have automated as much as possible. I
think there is value in getting these packages tested during the RC
process.

Sandeep,
I am using what is available on AWS Deep Learning Base AMI which is 3.4 and
CUDNN 7.0.5.

Thanks, Naveen

On Wed, May 23, 2018 at 8:57 AM, sandeep krishnamurthy <
sandeep.krishn...@gmail.com> wrote:

> Hi Naveen,
>
> Great work.
> I was wondering about OpenCV and cuDNN
>
> Thanks,
> Sandeep
>
> On Mon, May 21, 2018 at 5:44 PM, Naveen Swamy  wrote:
>
> > Hi Marco,
> > For MKL/MKLDNN --> I don't think we should build since its still
> > experimental. Its sad that MKLML was removed without a proper discussion,
> > granted
> > I have not tested with CUDA 9.2, I will building on a AWS Ubuntu instance
> > that is built with CUDA 9.0, I presume it should work fine.
> >
> >
> > On Mon, May 21, 2018 at 5:31 PM, Marco de Abreu <
> > marco.g.ab...@googlemail.com> wrote:
> >
> > > Hi Naveen,
> > >
> > > thank you for driving the releases for the Scala packages!
> > >
> > > For CPU, I'd propose add MKL/MKLDNN builds.
> > >
> > > For GPU, do we have to specify the minor version of CUDA up front or
> are
> > > they interchangeable? CUDA 9.2, for example, received some quite
> > > significant performance updates and it would be great if our users
> could
> > > use them.
> > >
> > > Best regards,
> > > Marco
> > >
> > > On Fri, May 18, 2018 at 8:28 PM, Naveen Swamy 
> > wrote:
> > >
> > > > I am working on publishing the Scala package from 1.2.0 release. What
> > do
> > > > you recommend should be the configuration for the Scala Packages ?
> > > >
> > > > I am thinking of the below
> > > > For CPU build with
> > > > OPENBLAS
> > > > LAPACK
> > > >
> > > > For GPU build with
> > > > OPENBLAS
> > > > CUDA 9.0
> > > > NCCL
> > > >
> > > > Let me know if you concerns or recommendations.
> > > >
> > > > Thanks, Naveen
> > > >
> > >
> >
>
>
>
> --
> Sandeep Krishnamurthy
>

Re: Config for Scala Maven Package

[Naveen Swamy]

Hi Marco,
For MKL/MKLDNN --> I don't think we should build since its still
experimental. Its sad that MKLML was removed without a proper discussion,
granted
I have not tested with CUDA 9.2, I will building on a AWS Ubuntu instance
that is built with CUDA 9.0, I presume it should work fine.


On Mon, May 21, 2018 at 5:31 PM, Marco de Abreu <
marco.g.ab...@googlemail.com> wrote:

> Hi Naveen,
>
> thank you for driving the releases for the Scala packages!
>
> For CPU, I'd propose add MKL/MKLDNN builds.
>
> For GPU, do we have to specify the minor version of CUDA up front or are
> they interchangeable? CUDA 9.2, for example, received some quite
> significant performance updates and it would be great if our users could
> use them.
>
> Best regards,
> Marco
>
> On Fri, May 18, 2018 at 8:28 PM, Naveen Swamy  wrote:
>
> > I am working on publishing the Scala package from 1.2.0 release. What do
> > you recommend should be the configuration for the Scala Packages ?
> >
> > I am thinking of the below
> > For CPU build with
> > OPENBLAS
> > LAPACK
> >
> > For GPU build with
> > OPENBLAS
> > CUDA 9.0
> > NCCL
> >
> > Let me know if you concerns or recommendations.
> >
> > Thanks, Naveen
> >
>

Re: Config for Scala Maven Package

[Marco de Abreu]

Hi Naveen,

thank you for driving the releases for the Scala packages!

For CPU, I'd propose add MKL/MKLDNN builds.

For GPU, do we have to specify the minor version of CUDA up front or are
they interchangeable? CUDA 9.2, for example, received some quite
significant performance updates and it would be great if our users could
use them.

Best regards,
Marco

On Fri, May 18, 2018 at 8:28 PM, Naveen Swamy  wrote:

> I am working on publishing the Scala package from 1.2.0 release. What do
> you recommend should be the configuration for the Scala Packages ?
>
> I am thinking of the below
> For CPU build with
> OPENBLAS
> LAPACK
>
> For GPU build with
> OPENBLAS
> CUDA 9.0
> NCCL
>
> Let me know if you concerns or recommendations.
>
> Thanks, Naveen
>

Config for Scala Maven Package

[Naveen Swamy]

I am working on publishing the Scala package from 1.2.0 release. What do
you recommend should be the configuration for the Scala Packages ?

I am thinking of the below
For CPU build with
OPENBLAS
LAPACK

For GPU build with
OPENBLAS
CUDA 9.0
NCCL

Let me know if you concerns or recommendations.

Thanks, Naveen