Re: mxnet Scala Convolution

HI YiZhi, 1. I looked at: scala-package/macros/src/main/scala/ml/dmlc/mxnet/SymbolMacro.scala scala-package/macros/src/main/scala/ml/dmlc/mxnet/NDArrayMacro.scala These files appear to dynamically register nnvm functions into NDArray/Symbol right? I would like the "javadocs" of the dynamically

Re: mxnet Scala Convolution

The javadoc cannot be added in this way. I'm afraid we have to write another 'javadoc generator' for these operators. 2017-10-19 7:53 GMT+00:00 TongKe Xue : > HI YiZhi, > > > 1. I looked at: > scala-package/macros/src/main/scala/ml/dmlc/mxnet/SymbolMacro.scala > scala-package/macros/src/main/scala

Re: mxnet Scala Convolution

You can retrieve the functions/operators by reflection. Or, when compiling the scala package, it prints all operators out. 2017-10-19 8:15 GMT+00:00 YiZhi Liu : > The javadoc cannot be added in this way. I'm afraid we have to write > another 'javadoc generator' for these operators. > > 2017-10-19

Re: mxnet Scala Convolution

You can retrieve the functions/operators by reflection. Or, when compiling the scala package, it prints all operators out. 2017-10-19 8:15 GMT+00:00 YiZhi Liu : > The javadoc cannot be added in this way. I'm afraid we have to write > another 'javadoc generator' for these operators. > > 2017-10-19

Re: mxnet Scala Convolution

Hi YiZhi, 1. I will read out the function specs from reflection. 2. Thanks for confirming that "other routes are not possible" -- saves me quite a bit of time searching. --TongKe On Thu, Oct 19, 2017 at 1:25 AM, YiZhi Liu wrote: > You can retrieve the functions/operators by reflection. Or, whe

Re: Request for suggestions- Supporting onnx in mxnet

Since there seems to be a difficulty to reach a consensus here, and this is a new area, maybe a good compromise would be to contribute this under /contrib as experimental, with whatever way Roshani thinks makes sense. Once there is code in place, and MXNet users and contributors are able to check

Re: Request for suggestions- Supporting onnx in mxnet

I'm speaking under my "MXNet contributor" hat. It will be sad that our new model format and compiler is not supported by our own contributors. It puts us in a bad position to reach out to outside to ask for support. If you really what to do it with the onnx <-> mxnet way, I suggest putting the co

Re: Request for suggestions- Supporting onnx in mxnet

Since this is a long discussion thread, I will summarize my point and again clarify some technical considerations here: Technical Reasoning: - Model exchange format like CoreML and ONNX are not lossless and complete. They are designed to an contain a core set of the minimum operators to support

Re: Request for suggestions- Supporting onnx in mxnet

Mu, You’re mentioning plans for a new model format and compiler, but I don’t recall seeing it shared/discussed on the dev list. Can you share these, so it is more accessible to folks to understand the plan and vision? Personally, I think it will be a shame to add ONNX support to MXNet, and hav

Re: Request for suggestions- Supporting onnx in mxnet

By new model format, I mean NNVM/TOP, and compiler I mean TVM. They are not discussed on the dev list yet, but we planned to open another thread later. If you check how coreml is converted, it asks users to pip install mxnet-to-coreml >From the users' viewpoint, it doesn't matter where the sourc

Re: Request for suggestions- Supporting onnx in mxnet

As for where the code should sit, we have seen onnx's support for caffe2 sitting on a separate repo. My suggestion would be put code under nnvm/top and migrate into mxnet eventually when the top components get into MXNet, hopefully by end of next month. I have elaborated my point in the last emai

Re: Request for suggestions- Supporting onnx in mxnet

Here are the detailed points(sorry for resenting it over again) Technical Reasoning: - Model exchange format like CoreML and ONNX are not lossless and complete. They are designed to an contain a core set of the minimum operators to support necessary inference tasks like ResNet, etc. So you canno

Re: Request for suggestions- Supporting onnx in mxnet

What I think I'm seeing here is that: * MXNet moved to Apache. * Some of the code it relied on (50% per the last release thread, but that may have been bombastic) remained at DMLC. * The MXNet community thinks one thing. * The DMLC community (which is a subset of the MXNet community that runs unde

Build cancellations

Hi all, We are having lots of problems with CI yesterday and still today. It is very likely that if you have a PR, the build will get cancelled in an effort to diagnose and remedy the situation. I apologize in advance for this. -Chris

Re: Request for suggestions- Supporting onnx in mxnet

One solution: The MXNet community forks the DMLC code it relies on into the MXNet codebase and moves on without being tied down by the decisions of a non-compatible community. +1 had a similar impression while following this discussion. -s

Re: Request for suggestions- Supporting onnx in mxnet

Hi Hen: It is sad to think DMLC adversarially in this matter. DMLC projects adopt apache way of doing things and we are planning moving more modules into Apache. All the discussion so far happens under the Apache manner and I do think that healthy discussion on critical design issues is importan

Re: Request for suggestions- Supporting onnx in mxnet

Why don't we just move all of these dmlc modules into the Apache repository right now and have the correct discussions on dev? What's the argument against this? IMHO, I thought that's what was going to be done originally. On Thu, Oct 19, 2017 at 12:14 PM, Tianqi Chen wrote: > Hi Hen: > > It is

Re: Request for suggestions- Supporting onnx in mxnet

Hi Henri, I don't understand why you labeled Tianqi and me as from "non-compatible DMLC community". We are the top-2 contributors for MXNet. Actually, among the participants in this thread, only both of us are listed as the committers . We are als

Re: Request for suggestions- Supporting onnx in mxnet

We thought there is a better way of doing this, by proposing nnvm as part of apache deep learning stack along with Mxnet. This is the reason why we did not simply move the repo over now Tianqi On Thu, Oct 19, 2017 at 12:43 PM Chris Olivier wrote: > Why don't we just move all of these dmlc module

Re: Request for suggestions- Supporting onnx in mxnet

I guess the whole discussion here is about - Who is 'We' in your email ? 'We thought there is a better way of doing this' It may just be misinterpretation or misunderstanding amongst folks here due to language barrier. On Thu, Oct 19, 2017 at 3:48 PM, Tianqi Chen wrote: > We thought there is

Re: Request for suggestions- Supporting onnx in mxnet

There is overhead for moving repos to Apache. For example, moving MXNet took about a few months. Also, we don't have enough evidence to convince the rest NNVM/TVM contributors that the benefits of moving to Apache are larger than the cost. So unlikely we can "move all DMCL modules right now". On T

Re: Request for suggestions- Supporting onnx in mxnet

...and please don't get me wrong, my English is horrendously poor. On Thu, Oct 19, 2017 at 3:53 PM, Suneel Marthi wrote: > I guess the whole discussion here is about - Who is 'We' in your email ? > > 'We thought there is a better way of doing this' > > It may just be misinterpretation or misund

Re: Request for suggestions- Supporting onnx in mxnet

I think "We" refer to "Tianqi and me" on this thread. On Thu, Oct 19, 2017 at 12:53 PM, Suneel Marthi wrote: > I guess the whole discussion here is about - Who is 'We' in your email ? > > 'We thought there is a better way of doing this' > > It may just be misinterpretation or misunderstanding a

Re: Request for suggestions- Supporting onnx in mxnet

I do have one request, please do not turn the technical discussion into political debate. If we need to start a debate, let us do that in another thread. I would very love to keep the discussion technical Tianqi On Thu, Oct 19, 2017 at 12:59 PM Mu Li wrote: > I think "We" refer to "Tianqi and me

Re: Request for suggestions- Supporting onnx in mxnet

Not at all, definitely wasn't the intent and let's keep this technical. Thanks for clarifying, because i have 20 slack messages from different folks who had the same question and i was just posting that on behalf of others. On Thu, Oct 19, 2017 at 4:02 PM, Tianqi Chen wrote: > I do have one re

Re: Request for suggestions- Supporting onnx in mxnet

This thread is long and forked, but I just wanted to re-iterate my proposal to have ONNX import/export implemented in MXNet /contrib as experimental. I think this is a good first step that hopefully allows MXNet users to easily leverage ONNX, but still leave a clear path to update the implementat

Re: Request for suggestions- Supporting onnx in mxnet

Again my recommendation is to go through mxnet/gluon (which in that case core operator set of NNVM) with the following technical reason: - Enjoy future compatibility and compilation pipeline that other frameworks do not have - Articulate Apache MXNet's need of core operators clearly to give Apache

Re: Request for suggestions- Supporting onnx in mxnet

Tianqi, Can you clarify your proposal to go through mxnet/gluon? - Are you proposing to implement “import ONNX” in Gluon by dynamically building the ONNX graph using Gluon API? - Gluon does not have a “save” API, you can only save weights, not the network. How can we export Gluon model to ONNX?

Re: Request for suggestions- Supporting onnx in mxnet

- We can implement onnx importer directly through gluon API - For the exporter, it is recommended to canonicalize them to gluon's op def, before converting into onnx. Such canonicalization utility is support via nnvm/top. - If we don't want to rely on the code in the nnvm/top(which I recommend at l

Re: Request for suggestions- Supporting onnx in mxnet

Essentially we want to have a clean set of core operators, that we can use for common exchange, and articulate our need of in-memory and exchange IR. We could build a python layer in ApacheMXNet code repo, (e.g. mxnet.contrib.top) to do so, and use this as a bridge to exchange everything. If that

DMLC [Was: Request for suggestions- Supporting onnx in mxnet]

It wasn't about the two of you specifically, it was that it seemed to me that we have two communities (DMLC and MXNet; communities not committers) having a difference of opinion. This is not a healthy thing. Apologies on the "Something is rotten"; it's an allusion to a quote from Shakespeare's Ham

Re: DMLC [Was: Request for suggestions- Supporting onnx in mxnet]

As a group DMLC members consist of committers to the DMLC projects, using the Apache License and act with the Apache model of comitter-ship. MXNet comes from DMLC community and is now ApacheMXNet. All its design decisions are made by the ApacheMXNet comitters. Of course many of the members and key

The Exchange Layer Support of MXNet

I will start forking the previous discussion and it has gone awry and I hope to start a pure technical discussion thread. I said in another email that we could do a vote to settle this issue. I now think that I was wrong and would like to apology for my rush proposal on this. I hope to reopen thi

RE: The Exchange Layer Support of MXNet

Just a quick question: Where to find the document to describe the standardized core operator set of Mxnet/gluon? Thanks, Haitao -Original Message- From: workc...@gmail.com [mailto:workc...@gmail.com] On Behalf Of Tianqi Chen Sent: Friday, October 20, 2017 11:43 AM To: dev@mxnet.incubator

ci mailing list?

Would it be useful to get an mxnet-ci mailing list setup so that it's easy to filter build failures and general conversation into different buckets? Hen

Re: ci mailing list?

There’s a Slack channel On Thu, Oct 19, 2017 at 9:59 PM Hen wrote: > Would it be useful to get an mxnet-ci mailing list setup so that it's easy > to filter build failures and general conversation into different buckets? > > Hen >