Hi Tianqi I thought a bit more about your comments and I think there is a simple way to address your concerns that satisfies both needs.
We can have a NodeAttributes template class which has a map of string to any as it's currenlty the case, so the graph can be used in the highly dynamic scenario that you are concerned about. Let me know what you think. Pedro. On Tue, May 14, 2019 at 5:50 PM Pedro Larroy <pedro.larroy.li...@gmail.com> wrote: > > Hi Tianqi > > Thanks for the quick response. > > Could you point to examples where graph.h is being exposed which would > not be possible with what I propose? I don't think my proposal is > having any impact in language bindings, and the way I describe it > doesn't affect having or not having higher language bindings. Please > elaborate so I can understand your concern. Maybe code examples where > the graph attributes are being changed from Python? I don't think we > have this on MXNet. This is such a core foundation for MXNet, that I > don't think we should compromise on it because other project not > directly related to MXNet might want to expose some untyped graph and > Node attributes. The current status makes maintaining the code very > painful and also is preventing desired features such as higher order > gradients to be developed. I have heard from you many times how speed > is critical for us to innovate in this quickly changing field. > > My proposal is limited to the graph and wouldn't change the way > operators are registered and arguments are processed for operators for > example. > > > Regarding the second point, the documentation about Relay in the web > which I found for example: > > https://docs.tvm.ai/dev/relay_add_op.html# > > Is somebody working on making Imperative::Backward use this API? this > would be a big change which I'm not aware of. And using an IR is of a > much bigger scope than the change I'm proposing here for example. > > I think I'm having difficulty understanding what are the arguments > here. I'm saying I need to change one piece of my car and what you are > selling me is a new vehicle here? Or your suggestion that we use > Relay for the graph passes in MXNet? > > I would like to see C++ code examples, Python examples are not > sufficient when we talk about the core MXNet. > > Pedro. > > > > > > > On Tue, May 14, 2019 at 5:39 PM Tianqi Chen <tqc...@cs.washington.edu> wrote: > > > > Thanks for the proposal. Let me share some of my thoughts: > > > > Specific comments on the proposal > > ----------------------------------------------- > > The heavy use of generic in the Graph type was a huge departure from > > type-erased data structure which was presented in the previous design. > > While we understand the advantage of typed language(more compile-time > > checking) and type-erased types(more dynamism) the heavy use of > > the template will actually make the project solely C++ focused, making it > > hard to expose intermediate(templatized) data structure to > > other languages like python/scala/clojure. > > > > While I fully understand some of the lessons taught in programming > > C++(reduce shared_ptr, more typing etc.) > > We need to think about the context of MXNet project and **the need to > > support multi-language as a first-class**. > > Some of the type-erased types are design trade-offs made to support these > > features, and we need to think more > > carefully instead of just applying "rules for C++" which may bring problems. > > > > Future of NNVM > > ---------------------- > > Given that this thread touched upon what we should do for better > > computational graph handling. I would recommend also to take a look at > > NNVMv2 -- relay. > > > > Relay addresses many of the wish-lists in the proposal already, such as > > operator fusion, high order gradient, offload to hardware, isolated > > compilation, deployment on edge and accelerators etc. > > Relay also address problems not yet being mentioned in the proposal, > > including control flow and dynamic runtime, automatic layout optimization > > etc. > > > > Tianqi > > > > On Tue, May 14, 2019 at 5:06 PM Sheng Zha <zhash...@apache.org> wrote: > > > > > Hi Pedro, > > > > > > Thanks for taking the inititaive. Skimming through the design doc, I > > > didn't see comparison with existing solutions such as relay in tvm, which > > > is already a dependency of mxnet already. Could you elaborate on > > > comparison > > > with existing solutions in the design doc too? > > > > > > -sz > > > > > > On 2019/05/14 23:49:30, Pedro Larroy <pedro.larroy.li...@gmail.com> > > > wrote: > > > > Hi dev@ > > > > > > > > As a result of my deep dives on the graph machinery I have created a > > > > new proposal to improve the operator graph in MXNet. > > > > > > > > This would mean superseding the use of NNVM Graph in MXNet and having > > > > a new implementation that we can use to simplify a lot of code and do > > > > powerful graph manipulation and passes such as operator fusion and > > > > other optimizations. > > > > > > > > As it would be a change with big impact and ramifications, your > > > > thoughts and feedback on the document would be highly appreciated so > > > > we can take potential future interesting use cases: > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/MXVM%3A+Operator+graph+2.0 > > > > > > > > Pedro. > > > > > > >
