Naveen, Thanks. I appreciate your help. I acknowledge there are some challenges in trying to bring in a new language package, but I think if we can make it work, it will be the best way for it to grow along with the MXNet project and be a good fit with its goals and ecosystem long term.
I added a confluence page of the general state of what is working and needs help. https://cwiki.apache.org/confluence/display/MXNET/Clojure+Package+Contribution+Needs. Feel free to edit. Chen, I added the code generator use on the page as well. I don't fully understand what you are proposing, but I would like to. If you could provide some more details to help me, that would be great. - Carin On Sun, Jun 10, 2018 at 8:00 AM, Chen HY <chenhy12...@gmail.com> wrote: > I personally think that is the reason the clojure package should use more > code generator. > It makes easier to build everything align with scala package. > > 2018-06-10 12:45 GMT+01:00 Naveen Swamy <mnnav...@gmail.com>: > > > I have called for contribution within my organization to help with the > > review, some of them have shown interest and hope to see them help with > the > > review. Carin, I learnt you are popular to have given great talks on > > Clojure. > > > > Could I please ask you document what functionality is working and what > > parts of it need contribution. > > > > In general I want to request a few things: > > Good API documentation, > > Examples, turn examples into integration tests. > > Tests, more tests > > Everything testable on CI. > > > > In the long term, We have to think how to maintain API compatibility and > > tests across language bindings. > > > > Thanks, Naveen > > > > > > On Sat, Jun 9, 2018 at 2:38 AM, YiZhi Liu <eazhi....@gmail.com> wrote: > > > > > Carin, > > > > > > Thank you for the great work. I'll do the review. As I have no > > > expertise in Clojure, it will really help to have people from Clojure > > > community reviewing it as well :) > > > > > > 2018-06-08 14:22 GMT-07:00 Carin Meier <carinme...@gmail.com>: > > > > A couple of questions came up in regard to the PR and the current > test > > > > suite state as well as the best way to review the PR since it is a > new > > > > language binding. > > > > > > > > In regards to the Clojure test suite, most of the Scala test suite > has > > > been > > > > ported over with the goal of having comparable coverage. I can go > > ahead a > > > > put in a coverage tool to make that it a bit more transparent. > > > > > > > > For reviewing, I have a couple people in the Clojure community that > are > > > > interested in collaborating in this project and I can ask them for > help > > > in > > > > reviewing the PR in some way if that is helpful too. > > > > > > > > I'm also open to other suggestions. > > > > > > > > Thanks, > > > > Carin > > > > > > > > On Fri, Jun 8, 2018 at 4:06 PM, Carin Meier <carinme...@gmail.com> > > > wrote: > > > > > > > >> Here is the PR https://github.com/apache/incubator-mxnet/pull/11205 > > > >> > > > >> I've ported in the content from the external github repo ( > > > >> https://github.com/gigasquid/clojure-mxnet), that has been tested > by > > > some > > > >> of the Clojure community, into the contrib directory. > > > >> > > > >> There is still lots more to do in relation to adding tests, > > benchmarks, > > > >> and increasing stability, but I thought this might be a good point > to > > > bring > > > >> it in initially so that the other work can be reviewed in smaller > > > chunks. > > > >> I also would like to get other people involved in making it better, > > so I > > > >> thought that having the base package in there would be a good > starting > > > >> point for collaboration. > > > >> > > > >> Feedback welcome. > > > >> > > > >> - Carin > > > >> > > > >> > > > >> On Tue, Jun 5, 2018 at 11:40 AM, Carin Meier <carinme...@gmail.com> > > > wrote: > > > >> > > > >>> Thanks everyone. I'll work on getting together a PR with your > > feedback > > > >>> and post it here. > > > >>> > > > >>> On Tue, Jun 5, 2018 at 4:05 AM, Chen HY <chenhy12...@gmail.com> > > wrote: > > > >>> > > > >>>> I would suggest using code generators in case upstream library > > adding > > > the > > > >>>> functions for arrays. > > > >>>> It seems that cpp binding is using a code generator and works > fine. > > > >>>> > > > >>>> 2018-06-05 7:59 GMT+01:00 Naveen Swamy <mnnav...@gmail.com>: > > > >>>> > > > >>>> > ~/mxnet/contrib/clojure-package good place for the code. > > > >>>> > > > > >>>> > the package name org.apache.mxnet.contrib.clojure ? do you need > > > mxnet > > > >>>> > again? > > > >>>> > > > > >>>> > I forgot to request to run some benchmarks and document. One of > > the > > > >>>> reasons > > > >>>> > users use MXNet is because of its performance and we want to > > ensure > > > >>>> that we > > > >>>> > maintain it across language bindings. > > > >>>> > > > > >>>> > Also invite your other clojure programmer buddies to the party > :) > > > >>>> > > > > >>>> > Thanks, Naveen > > > >>>> > > > > >>>> > > > > >>>> > On Mon, Jun 4, 2018 at 1:55 PM, Carin Meier < > carinme...@gmail.com > > > > > > >>>> wrote: > > > >>>> > > > > >>>> > > Oh right. That's not a problem, I wonder if something like > > > >>>> > > > > > >>>> > > org.apache.mxnet.contrib/clojure-mxnet > > > >>>> > > > > > >>>> > > would work? > > > >>>> > > > > > >>>> > > If this seems like it is the right direction, we could work > out > > > the > > > >>>> > details > > > >>>> > > in a PR. > > > >>>> > > > > > >>>> > > > > > >>>> > > On Mon, Jun 4, 2018 at 4:44 PM, Naveen Swamy < > > mnnav...@gmail.com> > > > >>>> wrote: > > > >>>> > > > > > >>>> > > > I agree with your assessment that we shouldn't need the user > > to > > > >>>> change > > > >>>> > > > their code. I am not sure if we can release under > > > >>>> > > org.apache.clojure-mxnet > > > >>>> > > > we might have to stick with our primary group id > > > org.apache.mxnet > > > >>>> and > > > >>>> > may > > > >>>> > > > be create a sub-package under it? any creative ideas? > > > >>>> > > > > > > >>>> > > > On Mon, Jun 4, 2018 at 1:29 PM, Carin Meier < > > > carinme...@gmail.com> > > > >>>> > > wrote: > > > >>>> > > > > > > >>>> > > > > Thanks for the feedback everyone. > > > >>>> > > > > > > > >>>> > > > > I agree on the contrib option. I think it's a great path > > > forward > > > >>>> and > > > >>>> > > > would > > > >>>> > > > > allow it time for feedback, contribution by others, and > > > >>>> > stabilization. > > > >>>> > > > > > > > >>>> > > > > If I'm understanding correctly, that would mean putting > the > > > >>>> source > > > >>>> > code > > > >>>> > > > in: > > > >>>> > > > > ~/mxnet/contrib/clojure-package > > > >>>> > > > > > > > >>>> > > > > and having the artifact jar named > > > >>>> > > > > `org.apache.contrib.clojure-mxnet/clojure-mxnet` > > > >>>> > > > > > > > >>>> > > > > I would recommend not having the individual namespaces of > > the > > > >>>> files > > > >>>> > > have > > > >>>> > > > > contrib embedded in them, so that if it graduates, users > > won't > > > >>>> have > > > >>>> > to > > > >>>> > > > > change their code, only the dependency. > > > >>>> > > > > > > > >>>> > > > > Please let me know if this is correct or if there are any > > > other > > > >>>> > ideas. > > > >>>> > > > > > > > >>>> > > > > - Carin > > > >>>> > > > > > > > >>>> > > > > > > > >>>> > > > > > > > >>>> > > > > > > > >>>> > > > > On Mon, Jun 4, 2018 at 4:03 PM, Naveen Swamy < > > > mnnav...@gmail.com > > > >>>> > > > > >>>> > > wrote: > > > >>>> > > > > > > > >>>> > > > > > I think that's a great idea to bring in under contrib > and > > we > > > >>>> can > > > >>>> > also > > > >>>> > > > get > > > >>>> > > > > > user feedback > > > >>>> > > > > > > > > >>>> > > > > > > On Jun 4, 2018, at 12:44 PM, sandeep krishnamurthy < > > > >>>> > > > > > sandeep.krishn...@gmail.com> wrote: > > > >>>> > > > > > > > > > >>>> > > > > > > Hi Carin, > > > >>>> > > > > > > > > > >>>> > > > > > > This is a commendable work. Thanks a lot for all the > > hard > > > and > > > >>>> > smart > > > >>>> > > > > work > > > >>>> > > > > > > you have put behind this :-) I think this will be a > > great > > > >>>> value > > > >>>> > > > > addition. > > > >>>> > > > > > > > > > >>>> > > > > > > If people are not sure about usage, can I suggest this > > > >>>> awesome > > > >>>> > work > > > >>>> > > > to > > > >>>> > > > > be > > > >>>> > > > > > > brought in "contrib" package? Invite and build the > > > community > > > >>>> > around > > > >>>> > > > > > > Clojure, stabilize and increase the coverage, and > based > > on > > > >>>> usage > > > >>>> > > and > > > >>>> > > > > > > development, graduate it to main stable support from > > > contrib. > > > >>>> > > > > > > > > > >>>> > > > > > > Suggestions and thoughts? > > > >>>> > > > > > > > > > >>>> > > > > > > Best, > > > >>>> > > > > > > Sandeep > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > On Mon, Jun 4, 2018 at 12:27 PM, Ivan Serdyuk < > > > >>>> > > > > > local.tourist.k...@gmail.com> > > > >>>> > > > > > > wrote: > > > >>>> > > > > > > > > > >>>> > > > > > >> Hello everyone. > > > >>>> > > > > > >> > > > >>>> > > > > > >> A small comment, about Scala API: main commiters are > > > hardly > > > >>>> > > > available, > > > >>>> > > > > > as > > > >>>> > > > > > >> for today. > > > >>>> > > > > > >> > > > >>>> > > > > > >> As for Clojure - I might suggest that it might be > > > possible > > > >>>> to > > > >>>> > > > enlight > > > >>>> > > > > > >> future work, for that package, for Clojure > developers. > > > >>>> > > > > > >> > > > >>>> > > > > > >>> On Mon, Jun 4, 2018 at 8:50 PM, Naveen Swamy < > > > >>>> > mnnav...@gmail.com > > > >>>> > > > > > > >>>> > > > > > wrote: > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> Hi Carin, > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> This is great effort and very nicely documented. > When > > I > > > >>>> > surveyed > > > >>>> > > at > > > >>>> > > > > my > > > >>>> > > > > > >> day > > > >>>> > > > > > >>> job on the interest for Scala packages there were a > > few > > > >>>> people > > > >>>> > > who > > > >>>> > > > > > >>> specifically were interested in the Clojure > packages, > > I > > > >>>> think > > > >>>> > > those > > > >>>> > > > > are > > > >>>> > > > > > >>> people who might have tried and understand the > > > complexity > > > >>>> of > > > >>>> > > making > > > >>>> > > > > > Scala > > > >>>> > > > > > >>> work in Clojure. > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> I am in support of a MXNet-Clojure package, I > request > > > you > > > >>>> to > > > >>>> > add > > > >>>> > > > more > > > >>>> > > > > > >> unit > > > >>>> > > > > > >>> tests and integration tests that can be ported to > CI. > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> Thanks for your efforts. > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> -Naveen > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> > > > >>>> > > > > > >>> On Fri, Jun 1, 2018 at 6:16 PM, Carin Meier < > > > >>>> > > carinme...@gmail.com> > > > >>>> > > > > > >> wrote: > > > >>>> > > > > > >>> > > > >>>> > > > > > >>>> It is always a good thing to consider the cost with > > the > > > >>>> > benefit. > > > >>>> > > > > I'll > > > >>>> > > > > > >> do > > > >>>> > > > > > >>> my > > > >>>> > > > > > >>>> best to explain what I see the tradeoffs to be. > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>> First, I wanted to clarify that it took significant > > > >>>> > development > > > >>>> > > > > effort > > > >>>> > > > > > >> to > > > >>>> > > > > > >>>> get the Clojure package and the interop working > > > properly > > > >>>> > despite > > > >>>> > > > my > > > >>>> > > > > > >>> simple > > > >>>> > > > > > >>>> looking design on the confluence page :) > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>> One of the advantages of MXNet over its competitors > > is > > > its > > > >>>> > many > > > >>>> > > > > > >> language > > > >>>> > > > > > >>>> support. The Clojure package would only increase > the > > > >>>> value of > > > >>>> > > this > > > >>>> > > > > > >>>> proposition and bring new users and growth into the > > > >>>> community. > > > >>>> > > > > > >>>> However, there is a cost associated with adding > this > > > >>>> language > > > >>>> > > > > support > > > >>>> > > > > > >> as > > > >>>> > > > > > >>>> you pointed out. > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>> Since the Clojure package right now is only reliant > > on > > > the > > > >>>> > Scala > > > >>>> > > > > jars > > > >>>> > > > > > >>> from > > > >>>> > > > > > >>>> Maven, it can exist outside the main project as an > > > >>>> independent > > > >>>> > > > repo > > > >>>> > > > > > >> but I > > > >>>> > > > > > >>>> think that would lessen the growth benefit both to > > the > > > >>>> Clojure > > > >>>> > > > > > >> community > > > >>>> > > > > > >>>> and to the MXNet community to not be included as a > > > first > > > >>>> class > > > >>>> > > > > > >> language. > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>> I believe having first class Clojure support in > MXNet > > > is > > > >>>> > > valuable, > > > >>>> > > > > but > > > >>>> > > > > > >>> the > > > >>>> > > > > > >>>> cost of that support is up to the community to > > decide. > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>> Is there a process for considering a new package in > > > MXNet? > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>> - Carin > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>>>> On Fri, Jun 1, 2018 at 5:51 PM, Chen HY < > > > >>>> > chenhy12...@gmail.com > > > >>>> > > > > > > >>>> > > > > > wrote: > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>> Have checked the issue and the confluence page, > but > > > still > > > >>>> > > > curious. > > > >>>> > > > > > >>>>> Clojure and Scala are both JVM based languages. > > > >>>> > > > > > >>>>> They, as well as many JVM based languages, can > share > > > >>>> their > > > >>>> > > class > > > >>>> > > > > and > > > >>>> > > > > > >>>> method > > > >>>> > > > > > >>>>> at a certain level. > > > >>>> > > > > > >>>>> Why should the community maintain two APIs for two > > > >>>> languages > > > >>>> > > with > > > >>>> > > > > can > > > >>>> > > > > > >>>> share > > > >>>> > > > > > >>>>> their packages with almost zero effort? > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>> 2018-06-01 21:58 GMT+01:00 Carin Meier < > > > >>>> carinme...@gmail.com > > > >>>> > >: > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>>> Hi all, > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> I've been working on a Clojure package for MXNet. > > > Since > > > >>>> > > Clojure > > > >>>> > > > is > > > >>>> > > > > > >> a > > > >>>> > > > > > >>>> JVM > > > >>>> > > > > > >>>>>> language, the package leverages the great work of > > the > > > >>>> > existing > > > >>>> > > > > > >> Scala > > > >>>> > > > > > >>>>>> package. > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> I would appreciate any feedback and testing. > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> Here is the original issue: > > > >>>> > > > > > >>>>>> https://github.com/apache/ > > > incubator-mxnet/issues/8971 > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> Architecture & Design: > > > >>>> > > > > > >>>>>> https://cwiki.apache.org/ > confluence/display/MXNET/ > > > >>>> > > MXNet+Clojure > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> and the github repo for rapid testing and issue > > > fixing > > > >>>> > before > > > >>>> > > of > > > >>>> > > > > > >>>> opening > > > >>>> > > > > > >>>>> an > > > >>>> > > > > > >>>>>> official PR https://github.com/gigasquid/ > > > clojure-mxnet > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> I'm also active in the slack channel so feel free > > to > > > >>>> ping me > > > >>>> > > > > there. > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>>> Thanks, > > > >>>> > > > > > >>>>>> Carin Meier > > > >>>> > > > > > >>>>>> > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>>> -- > > > >>>> > > > > > >>>>> Chen Hanyang 陈涵洋 > > > >>>> > > > > > >>>>> Software School Fudan University > > > >>>> > > > > > >>>>> +86-138-1881-7745 > > > >>>> > > > > > >>>>> > > > >>>> > > > > > >>>> > > > >>>> > > > > > >>> > > > >>>> > > > > > >> > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > > > > >>>> > > > > > > -- > > > >>>> > > > > > > Sandeep Krishnamurthy > > > >>>> > > > > > > > > >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > > >>>> > > > >>>> -- > > > >>>> Chen Hanyang 陈涵洋 > > > >>>> Software School Fudan University > > > >>>> +86-138-1881-7745 > > > >>>> > > > >>> > > > >>> > > > >> > > > > > > > > > > > > -- > > > Yizhi Liu > > > DMLC member > > > Amazon Web Services > > > Vancouver, Canada > > > > > > > > > -- > Chen Hanyang 陈涵洋 > Software School Fudan University > +86-138-1881-7745 >
