Thansk for that explanation! What is meant by the "core ops"? Are these the gluon python ops? I know what the "legacy ops" are -- things like BatchNormProp/BatchNormOp. WHat are the "new ones"?
I apologize for not knowing this, but what are some of the "pain points" with the legacy ops? Not in great detail, but would just like to get a little perspective. Thanks, -Chris On Fri, Oct 20, 2017 at 8:42 AM, Tianqi Chen <tqc...@cs.washington.edu> wrote: > First of all, I agree that supporting the format as an important step of > adding influence. We are going to do it in either options. The overhead of > engineering is not that much difference. I also do not argue for specific > types of APIs(gluon vs symbolic) we should use. > > MXNet Sym API contains two elements: > > - 1) The graph manipulation API > - 2) The operator defs > > The graph manipulation API has been serving us great and we should be keep > using it for whatever purposes we have. However, the operator definitions > has been involved for two years and there are quite a few pains in legacy > operator and attributes. The new remodeled gluon API is much cleaner. > While there is a gap between the current operator def and gluon's API, we > want to shift defs toward gluon style operator defs (NOTE this do not mean > imperative only, just to make operator def align with numpy and gluon). > > What I am proposing is to have a clear document and list of "core operator > defs" we want to prioritize in terms of supporting exchange offers > optimization. Having such document, gives us and others a clear reference > on what we need, and what is needed in order to give good external support > to ApacheMXNet. > > The MXNet/gluon or nnvm/top is such initial set of operator defs that is > proposed. By canonicalizing the current MXNet op defs into this "core set", > and using it as a center for talking to other exchange format gives the > advantage of the things we mentioned above. > > Going through the path have second major advantage, because it > enables(answering your question on what compilation mean) > mxnet->core ops -> nnvm compiler-> bare metal(javascript, ARM, > AMDGPU, Metal) > > While in theory we can also do mxnet->onnx-> nnvm compiler, This is a more > twisted path as the exchange format do not preserve information and subject > to change. While going through the core ops offers bi-directional exchange > with mxnet, and directly doing that in memory. The compilation path is not > offered if we do not go through the core ops, because the graph > optimization can take great advantage of a clear core set of operators. > > To facilitate discussion technically. I would suggest we break our points > down. I tried to do that in this email on things we agree on and have > opions. So we can find common ground and move forward. > > > Tianqi > > > > On 10/19/17, 20:43, "Tianqi Chen" <workc...@gmail.com on behalf of > > tqc...@cs.washington.edu> wrote: > > > > I will start forking the previous discussion and it has gone awry > and I > > hope to start a pure technical discussion thread. > > > > I said in another email that we could do a vote to settle this issue. > > I now > > think that I was wrong and would like to apology for my rush proposal > > on > > this. > > > > I hope to reopen this email thread to gain consensus on what we want. > > There > > has been express of concerns that the code should reside on > ApacheMXNet > > repo. I think that discussion is already over and at least I would > > want the > > code to be in the repo as long as we reach the consensus. > > > > The leftover point is how should we do it. There are two ways of > doing > > this > > > > 1) Doing it on top of existing Symbol API. > > 2) Moving most of the exchange layer on standardized core operator > set, > > namely mxnet/gluon spec. > > > > Both approaches are feasible. There is some advocation on which way > > might > > be simpler, but the additional effort of engineering won't be that > > much. > > The reason why I am seeing this decision so seriously is that it will > > affect how we can influence the design of the exchange format we act > > on, and > > how easily it is to integrate future features that are made > available. > > > > I am in favor of (2) because technically it gives us a clean future > > compatibility, offers compilation and articulates clearly what > > ApacheMXNet's stance on core operators. > > These things could bring more impact to the community in general. > > > > Tianqi > > > > > > > > >
