Hi Joern, The point is that, the front is not a simple wrapper of c_api.h, as you mentioned, which can be easily achieved by JavaCPP.
I have noticed the potential conflicts between the assembled scala library and the one in users' environment. Can we remove the scala library from the assembly jar? @Nan It wouldn't be a problem since the scala libraries with same major version are compatible. 2017-08-16 23:49 GMT+08:00 Joern Kottmann <kottm...@gmail.com>: > Hello, > > I personally had quite some issues with Scala dependencies in > different versions and Spark, where one version is not compatible with > the other version. Then you need to debug the dependency tree to find > the places where the versions don't match. Every project which would > like to use MXnet then has to depend on Scala and might also get > conflicts if other dependencies depend on different Scala versions. > Probably something which will cause issues for some of your users. > Users who want to use Java might not be familiar with Scala dependency > problems and have a hard time resolving them by getting strange error > messages. > > The JNI layer could be generated with JavaCPP, then we would not need > to write/maintain the C and the jvm side for that our self. > A good example of JavaCPP and Scala usage is Apache Mahout [1]. > > Even if we don't use JavaCPP, the JNI layer should be easy to get into > a state where both can share it, the current Scala JNI layers LibInfo > classes could be converted to Java classes and would in most cases > require only minor changes in the Scala code. > > Jörn > > [1] https://github.com/apache/mahout/tree/master/viennacl/src/main > > On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcg...@gmail.com> wrote: >> I agree with Yizhi >> >> My major concern is the duplicate implementations, which are usually one of >> the major sources of bugs, especially with two languages which are >> naturally interactive (OK, Calling Scala from Java might need some more >> efforts). It is just like we provide C++ & C APIs of MxNet in two separated >> packages. >> >> About dependency problem, when you say "As far as I see this has the great >> disadvantage that the Java API would force Scala as a dependency onto the >> java users.", would you please give a concrete example causing critical >> issues? >> >> Best, >> >> Nan >> >> >> >> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javeli...@gmail.com> wrote: >> >>> Hi, >>> >>> If we build the Java API from the very beginning, i.e. the JNI part, >>> we have to rewrite the codes for training, predict, inferShape, etc. >>> It would be too heavy to maintain a totally new front language. >>> >>> As far as I see, I don't think Scala library dependency would be a big >>> problem in most cases, unless we are going to use it in embedded >>> devices. Could you illustrate some use-cases where you cannot involve >>> Scala dependencies? >>> >>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottm...@gmail.com>: >>> > Hello, >>> > >>> > the approach which is taken by Spark is described here [1]. >>> > >>> > As far as I see this has the great disadvantage that the Java API >>> > would force Scala as a dependency onto the java users. >>> > For a library it is always a great advantage if it doesn't have many >>> > dependencies, or zero dependencies. In our case it could be quite >>> > realistic to have a thin wrapper around the C API without needing any >>> > other dependencies (or only dependencies which can't be avoided). >>> > >>> > The JNI layer could easily be shared between the Java and Scala API. >>> > As far as I understand is the JNI layer in the Scala API anyway >>> > private and a change to it wouldn't require that the public part of >>> > the Scala API is changed. >>> > >>> > What do you think? >>> > >>> > Jörn >>> > >>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+API+Internals >>> > >>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javeli...@gmail.com> wrote: >>> >> Hi Joern, >>> >> >>> >> I suggest to build Java API as a wrapper of Scala API, re-use most of >>> >> the procedures. Referring to the Java API in Apache Spark. >>> >> >>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <jo...@apache.org>: >>> >>> Hello all, >>> >>> >>> >>> I would like to propose the addition of a Java API to MXNet. >>> >>> >>> >>> There has been some previous work done for the Scala API, and it makes >>> >>> sense to at least share the JNI layer between the two. >>> >>> >>> >>> The Java API probably should be aligned with the Python API (and >>> >>> others which exist already) with a few changes to give it a native >>> >>> Java feel. >>> >>> >>> >>> As far as I understand there are multiple people interested to work on >>> >>> this and it would be good to maybe come up with a written proposal on >>> >>> how things should be. >>> >>> >>> >>> My motivation is to get a Java API which can be used by Apache OpenNLP >>> >>> to solve various NLP tasks using Deep Learning based approaches and I >>> >>> am also interested to work on MXNet. >>> >>> >>> >>> Jörn >>> >> >>> >> >>> >> >>> >> -- >>> >> Yizhi Liu >>> >> DMLC member >>> >> Technical Manager >>> >> Qihoo 360 Inc, Shanghai, China >>> >>> >>> >>> -- >>> Yizhi Liu >>> DMLC member >>> Technical Manager >>> Qihoo 360 Inc, Shanghai, China >>> -- Yizhi Liu DMLC member Technical Manager Qihoo 360 Inc, Shanghai, China
