Hi Nan, Users have 2.11, but with a different minor version, will it cause conflicts?
2017-08-17 0:19 GMT+08:00 Nan Zhu <zhunanmcg...@gmail.com>: > Hi, Yizhi, > > You mean users have 2.10 env while we assemble 2.11 in it? > > Best, > > Nan > > On Wed, Aug 16, 2017 at 9:08 AM, YiZhi Liu <javeli...@gmail.com> wrote: > >> Hi Joern, >> >> The point is that, the front is not a simple wrapper of c_api.h, as >> you mentioned, which can be easily achieved by JavaCPP. >> >> I have noticed the potential conflicts between the assembled scala >> library and the one in users' environment. Can we remove the scala >> library from the assembly jar? @Nan It wouldn't be a problem since the >> scala libraries with same major version are compatible. >> >> 2017-08-16 23:49 GMT+08:00 Joern Kottmann <kottm...@gmail.com>: >> > Hello, >> > >> > I personally had quite some issues with Scala dependencies in >> > different versions and Spark, where one version is not compatible with >> > the other version. Then you need to debug the dependency tree to find >> > the places where the versions don't match. Every project which would >> > like to use MXnet then has to depend on Scala and might also get >> > conflicts if other dependencies depend on different Scala versions. >> > Probably something which will cause issues for some of your users. >> > Users who want to use Java might not be familiar with Scala dependency >> > problems and have a hard time resolving them by getting strange error >> > messages. >> > >> > The JNI layer could be generated with JavaCPP, then we would not need >> > to write/maintain the C and the jvm side for that our self. >> > A good example of JavaCPP and Scala usage is Apache Mahout [1]. >> > >> > Even if we don't use JavaCPP, the JNI layer should be easy to get into >> > a state where both can share it, the current Scala JNI layers LibInfo >> > classes could be converted to Java classes and would in most cases >> > require only minor changes in the Scala code. >> > >> > Jörn >> > >> > [1] https://github.com/apache/mahout/tree/master/viennacl/src/main >> > >> > On Wed, Aug 16, 2017 at 5:30 PM, Nan Zhu <zhunanmcg...@gmail.com> wrote: >> >> I agree with Yizhi >> >> >> >> My major concern is the duplicate implementations, which are usually >> one of >> >> the major sources of bugs, especially with two languages which are >> >> naturally interactive (OK, Calling Scala from Java might need some more >> >> efforts). It is just like we provide C++ & C APIs of MxNet in two >> separated >> >> packages. >> >> >> >> About dependency problem, when you say "As far as I see this has the >> great >> >> disadvantage that the Java API would force Scala as a dependency onto >> the >> >> java users.", would you please give a concrete example causing critical >> >> issues? >> >> >> >> Best, >> >> >> >> Nan >> >> >> >> >> >> >> >> On Wed, Aug 16, 2017 at 8:19 AM, YiZhi Liu <javeli...@gmail.com> wrote: >> >> >> >>> Hi, >> >>> >> >>> If we build the Java API from the very beginning, i.e. the JNI part, >> >>> we have to rewrite the codes for training, predict, inferShape, etc. >> >>> It would be too heavy to maintain a totally new front language. >> >>> >> >>> As far as I see, I don't think Scala library dependency would be a big >> >>> problem in most cases, unless we are going to use it in embedded >> >>> devices. Could you illustrate some use-cases where you cannot involve >> >>> Scala dependencies? >> >>> >> >>> 2017-08-16 22:13 GMT+08:00 Joern Kottmann <kottm...@gmail.com>: >> >>> > Hello, >> >>> > >> >>> > the approach which is taken by Spark is described here [1]. >> >>> > >> >>> > As far as I see this has the great disadvantage that the Java API >> >>> > would force Scala as a dependency onto the java users. >> >>> > For a library it is always a great advantage if it doesn't have many >> >>> > dependencies, or zero dependencies. In our case it could be quite >> >>> > realistic to have a thin wrapper around the C API without needing any >> >>> > other dependencies (or only dependencies which can't be avoided). >> >>> > >> >>> > The JNI layer could easily be shared between the Java and Scala API. >> >>> > As far as I understand is the JNI layer in the Scala API anyway >> >>> > private and a change to it wouldn't require that the public part of >> >>> > the Scala API is changed. >> >>> > >> >>> > What do you think? >> >>> > >> >>> > Jörn >> >>> > >> >>> > [1] https://cwiki.apache.org/confluence/display/SPARK/Java+ >> API+Internals >> >>> > >> >>> > On Wed, Aug 16, 2017 at 3:39 PM, YiZhi Liu <javeli...@gmail.com> >> wrote: >> >>> >> Hi Joern, >> >>> >> >> >>> >> I suggest to build Java API as a wrapper of Scala API, re-use most >> of >> >>> >> the procedures. Referring to the Java API in Apache Spark. >> >>> >> >> >>> >> 2017-08-16 18:21 GMT+08:00 Joern Kottmann <jo...@apache.org>: >> >>> >>> Hello all, >> >>> >>> >> >>> >>> I would like to propose the addition of a Java API to MXNet. >> >>> >>> >> >>> >>> There has been some previous work done for the Scala API, and it >> makes >> >>> >>> sense to at least share the JNI layer between the two. >> >>> >>> >> >>> >>> The Java API probably should be aligned with the Python API (and >> >>> >>> others which exist already) with a few changes to give it a native >> >>> >>> Java feel. >> >>> >>> >> >>> >>> As far as I understand there are multiple people interested to >> work on >> >>> >>> this and it would be good to maybe come up with a written proposal >> on >> >>> >>> how things should be. >> >>> >>> >> >>> >>> My motivation is to get a Java API which can be used by Apache >> OpenNLP >> >>> >>> to solve various NLP tasks using Deep Learning based approaches >> and I >> >>> >>> am also interested to work on MXNet. >> >>> >>> >> >>> >>> Jörn >> >>> >> >> >>> >> >> >>> >> >> >>> >> -- >> >>> >> Yizhi Liu >> >>> >> DMLC member >> >>> >> Technical Manager >> >>> >> Qihoo 360 Inc, Shanghai, China >> >>> >> >>> >> >>> >> >>> -- >> >>> Yizhi Liu >> >>> DMLC member >> >>> Technical Manager >> >>> Qihoo 360 Inc, Shanghai, China >> >>> >> >> >> >> -- >> Yizhi Liu >> DMLC member >> Technical Manager >> Qihoo 360 Inc, Shanghai, China >> -- Yizhi Liu DMLC member Technical Manager Qihoo 360 Inc, Shanghai, China
