[BUILD FAILED] Branch master build 207

Build for MXNet branch master has broken. Please view the build at https://builds.apache.org/job/incubator-mxnet/job/master/207/

Re: Java API for MXNet

I use MXNet in production for a while, where I trained the model in Python and deploy it in Java env by calling a Scala object. I think that’s enough in my use case. class MXNetPredictor (prefix: String, epoch: Int, batchSize: Int) { val model = FeedForward.load(prefix, epoch) /** * pr

[VOTE] Release MXNet version 0.11.0.rc2

This is the vote to release Apache MXNet (incubating) version 0.11.0. Voting will start now (Thursday, August 17, 2017 12:37 AM UTC) and close Monday, August 21, 2017 12:37 AM UTC. Link to release notes: https://cwiki.apache.org/confluence/display/MXNET/v0.11.0+Release+Notes

Re: [VOTE][CANCELLED]Release MXNet version 0.11.0.rc1

We are closing this vote because there is a bug causing 20-30% performance degradation. We will address this issue and send out RC2 for another vote. > On Aug 14, 2017, at 7:25 PM, Meghna Baijal wrote: > > This is the vote to release Apache MXNet (incubating) version 0.11.0. > Voting will start

Re: Java API for MXNet

With Java API I mean a set of classes I can use from Java, I tried this with the current Scala API but wasn't very successful. Probably if you know a bit about Scala internals, you can figure it all out but this makes it kind of unpleasant to use. You don't necessarily need to write Java code to bu

[BUILD FAILED] Branch master build 206

Build for MXNet branch master has broken. Please view the build at https://builds.apache.org/job/incubator-mxnet/job/master/206/

Re: Turned off PR builds

I turned off PR builds to prioritize testing our v0.11.0 branch for cutting RC2. The following PR builds that were in progress, and any new PR's or updated PR's, will be affected and will pick back up once we cut the release and re-enable PR builds tonight: 7442 7495 7494 7496 7152 7082 7498 On M

Re: [VOTE] Release MXNet version 0.11.0.rc1

-1 Agree with Steffen. Regards, Dom > On Aug 16, 2017, at 1:45 PM, Steffen Rochel wrote: > > -1 (non-binding) > > 0.11 RC2 contains performance degradation fixed with > https://github.com/apache/incubator-mxnet/pull/7489 . This fix should be > included in 0.11 release. > > Steffen > >> On

Re: [VOTE] Release MXNet version 0.11.0.rc1

-1 (non-binding) 0.11 RC2 contains performance degradation fixed with https://github.com/apache/incubator-mxnet/pull/7489 . This fix should be included in 0.11 release. Steffen On Tue, Aug 15, 2017 at 11:44 AM Naveen Swamy wrote: > +1 > > built from source and validated on a P2.Xlarge EC2 Inst

Re: Java API for MXNet

It would be great if there could be some examples for the benefits of the proposed API. I understand that certain syntax may look unsatisfying when calling scala API from java, and the questions that should be asked are: 1. Whether it’s intolerable to the extent that a new language binding must

Re: Java API for MXNet

Hi, Joern, when you say "Java API " it's sharing scala impl or not? Best, Nan On Wed, Aug 16, 2017 at 12:46 PM, Joern Kottmann wrote: > Seems like we are all agree about the idea to add a Java API. > > Maybe it is just me, but it wouldn't at all make sense for me (OpenNLP > use case) to use t

Re: Java API for MXNet

@Sheng, yes in most of cases that's true...I once saw some issues when involving Any/AnyVal/AnyRef (If I remember correctly) Best, Nan On Wed, Aug 16, 2017 at 12:49 PM, Zha, Sheng wrote: > How does it work for java calling into scala-built jar? I had the > impression that only jvm is required

Re: Java API for MXNet

How does it work for java calling into scala-built jar? I had the impression that only jvm is required as dependency. Best regards, -sz On 8/16/17, 12:46 PM, "Joern Kottmann" wrote: Seems like we are all agree about the idea to add a Java API. Maybe it is just me, but it wouldn't

Re: Java API for MXNet

Seems like we are all agree about the idea to add a Java API. Maybe it is just me, but it wouldn't at all make sense for me (OpenNLP use case) to use the Java API when it requires a Scala dependency, because at that point I would be better of just using the Scala API, and ensure that the things I

[BUILD FAILED] Branch master build 203

Build for MXNet branch master has broken. Please view the build at https://builds.apache.org/job/incubator-mxnet/job/master/203/

Re: Java API for MXNet

+1 for Sandeep's suggestion On Wed, Aug 16, 2017 at 11:21 AM, YiZhi Liu wrote: > Agree with Sandeep, while I guess the performance won't change. But > yes, benchmark talks. > > Moreover, in Scala package we use macros to generate operators > automatically, which will require more efforts if we s

Re: Java API for MXNet

Agree with Sandeep, while I guess the performance won't change. But yes, benchmark talks. Moreover, in Scala package we use macros to generate operators automatically, which will require more efforts if we switch to pure Java. 2017-08-17 2:12 GMT+08:00 sandeep krishnamurthy : > The fastest way to

Re: Java API for MXNet

The fastest way to get Java binding is through building Java native wrappers on Scala package. Disadvantages would be: * *Bloated library size: *May not be suitable for users planning to use Java APIs in Android of such smaller systems. * *Performance:* Performance may not be as good as build

Re: Java API for MXNet

Great discussion! Can we consider a fully stable java api as a two step process: Coming up with a good usable java api and second, removing the scala dependencies. We can be quicker in going to market if we just do the first one for the time being: It's simple, relatively quick, w/ focus only o

Re: Java API for MXNet

What Nan and I worried about is the re-implementation of something like https://github.com/apache/incubator-mxnet/blob/master/scala-package/core/src/main/scala/ml/dmlc/mxnet/Model.scala#L246, and the executorManager, NDArray, KVStore ... it uses. the C API stays at the very low level. If this is

Re: Java API for MXNet

There will be a new scala version one day, and the story we had with going from 2.10 to 2.11 might just repeat. In the end if you make a dependency using scala you just end up making it for the currently popular scala versions. And that might be ok for projects with developers who are familiar with

Re: Java API for MXNet

+1 for using Swig or JavaCPP to generate a low-level API and then high-level Scala and Java API's can converge to use that layer. Scala diverging towards scripting and Java towards some other API (i.e. Deeplearning4j, which I have never used). Using a generated interface layer makes it much more

Re: Java API for MXNet

I don't think it will load two copies of the jar unless it is shaded... From: Nan Zhu Sent: Wednesday, August 16, 2017 9:45:02 AM To: dev@mxnet.incubator.apache.org Subject: Re: Java API for MXNet I don't think there will be problems under "11", did the user see

Re: Java API for MXNet

I don't think there will be problems under "11", did the user see concrete errors? Best, Nan On Wed, Aug 16, 2017 at 9:30 AM, YiZhi Liu wrote: > Hi Nan, > > Users have 2.11, but with a different minor version, will it cause > conflicts? > > 2017-08-17 0:19 GMT+08:00 Nan Zhu : > > Hi, Yizhi,

Re: Java API for MXNet

Hi Nan, Users have 2.11, but with a different minor version, will it cause conflicts? 2017-08-17 0:19 GMT+08:00 Nan Zhu : > Hi, Yizhi, > > You mean users have 2.10 env while we assemble 2.11 in it? > > Best, > > Nan > > On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu wrote: > >> Hi Joern, >> >> The p

Re: Java API for MXNet

Hi, Yizhi, You mean users have 2.10 env while we assemble 2.11 in it? Best, Nan On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu wrote: > Hi Joern, > > The point is that, the front is not a simple wrapper of c_api.h, as > you mentioned, which can be easily achieved by JavaCPP. > > I have noticed th

Re: Java API for MXNet

Hi, Jorn Thanks for providing the info My opinion is that, 1. MxNet current relies on Scala 2.11 (having some problems due to the spark package...I am fixing it), which is a stable release for most of packages...my personal experience hasn't involved a package which only provides 2.10 version in

Re: Java API for MXNet

Hi Joern, The point is that, the front is not a simple wrapper of c_api.h, as you mentioned, which can be easily achieved by JavaCPP. I have noticed the potential conflicts between the assembled scala library and the one in users' environment. Can we remove the scala library from the assembly jar

Re: Java API for MXNet

Hello, I personally had quite some issues with Scala dependencies in different versions and Spark, where one version is not compatible with the other version. Then you need to debug the dependency tree to find the places where the versions don't match. Every project which would like to use MXnet t

Re: Java API for MXNet

My two cents here: Start with something simple and share foundation with scala, focus on the API and usability. Once the java API get stabilized, we can then discuss if such issue arise. I feel maven is quite automatic in terms of handling scala deps and that won't create an issue except for em

Re: Java API for MXNet

I agree with Yizhi My major concern is the duplicate implementations, which are usually one of the major sources of bugs, especially with two languages which are naturally interactive (OK, Calling Scala from Java might need some more efforts). It is just like we provide C++ & C APIs of MxNet in tw

Re: Java API for MXNet

Really glad to see this discussion. Sandeep has been giving this some thought as well. Regards, Dom > On Aug 16, 2017, at 8:19 AM, YiZhi Liu wrote: > > Hi, > > If we build the Java API from the very beginning, i.e. the JNI part, > we have to rewrite the codes for training, predict, inferShap

Re: Java API for MXNet

Hi, If we build the Java API from the very beginning, i.e. the JNI part, we have to rewrite the codes for training, predict, inferShape, etc. It would be too heavy to maintain a totally new front language. As far as I see, I don't think Scala library dependency would be a big problem in most case

Re: Java API for MXNet

Hello, the approach which is taken by Spark is described here [1]. As far as I see this has the great disadvantage that the Java API would force Scala as a dependency onto the java users. For a library it is always a great advantage if it doesn't have many dependencies, or zero dependencies. In o

Re: Java API for MXNet

Hi Joern, I suggest to build Java API as a wrapper of Scala API, re-use most of the procedures. Referring to the Java API in Apache Spark. 2017-08-16 18:21 GMT+08:00 Joern Kottmann : > Hello all, > > I would like to propose the addition of a Java API to MXNet. > > There has been some previous wor

Java API for MXNet

Hello all, I would like to propose the addition of a Java API to MXNet. There has been some previous work done for the Scala API, and it makes sense to at least share the JNI layer between the two. The Java API probably should be aligned with the Python API (and others which exist already) with