Re: Request to join the Slack channel
Invite sent! On Tue, Sep 25, 2018 at 10:29 PM Vikas Kumar wrote: > Please send me an invite to join. > > Thanks > Vikas >
Request to join the Slack channel
Please send me an invite to join. Thanks Vikas
Re: Some feedback from MXNet Zhihu topic
Hi Kellen (and all), thank you for your reply. I think there is a lack of C++ documentation, and documentation could be improved. I will try and explain with examples the difficulty I have with the C++ API. The problem I face with the C++ documentation is that the only thing I see as documentation is Doxygen generated list of namespaces/classes/files. That is, the first thing I see when press the C++ API is this: [image: image.png] this doesn't tell me much about what is the underlying connection of classes, or how mxnet is designed etc (huge difference between what I see when I click on the python API). I could equally well go directly into the source code. The mxnet CPP Package, is a github repository that has all the source code and several c++ examples. When I look into them, I see examples of usage. They make sense as I read along, but the main barrier is that I do not have an underlying understanding, a diagram, of how things are connected. This is in contrast with the documentation of python, where, say for Gluon, I have "under the hood" examples, and I can look at the source code and understand how things are connected (most times, and up to a level). In addition, for python, along with the description of objects and methods, sometimes exist also examples. So, say for example I want to create a 2nd order gradient, with C++, modify the source code and expose it to python. Where can I find info/docs for that? Where is the differentiation defined in the C++ code? This is not evident from the documentation - at least not to me. In contrast, I can find easily the mxnet.autograd python package with custom function definitions. Let's see some other examples from C++ libaries, e.g. boost, it has full documentation for the various packages starting from "hello world" constructs most of the times, e.g. accumulators (something similar could exist for NDArray or Symbol): https://www.boost.org/doc/libs/1_68_0/doc/html/accumulators.html or boost C++ to python etc. Another example is the documentation of the Evolving Objects library ( http://eodev.sourceforge.net/), where they have a more "enriched" doxygen documentation (http://eodev.sourceforge.net/eo/doc/html/index.html), and a basic tutorial (although outdated - http://eodev.sourceforge.net/eo/tutorial/html/eoTutorial.html) that describes the underlying structure and binds the different objects together. I think one way that the documentation for C++ could be improved is: a) add a description of the underlying connection of the objects within mxnet library. An overall "big picture". b) add examples of modifying the C++ source code to add features. This perhaps could be the most useful for user contribution. There exist examples of designing custom layers in python with the gluon API. c) optional but most useful: add words, examples, like a document (built on the existing examples), that explains more the source code and classes used and "connects the dots" between the underlying structures, just like is done for python examples, or other C++ libraries examples. There are pages where they explain the source code as they go along, and then, in the end, there is a link to the full code. I understand that the intended usage of C++ code is to be callled from python wrappers, and perhaps this would not be a good time investment for the project. Again a huge thank you for this awesome library and all the work you've put into it. All the above comments are with the best intentions. Kind regards, Foivos On Tue, Sep 25, 2018 at 10:47 PM kellen sunderland < kellen.sunderl...@gmail.com> wrote: > Thanks for the detailed feedback Foivos. Just want to dig into the C++ > comment. > > Any more details you could give on how we could improve the readability. > Would modernizing the codebase and trying to provide consistent code style > help? In regards to documentation was it that it's mostly lacking, or that > the existing documentation could be improved (or both)? > > -Kellen > > On Mon, Sep 24, 2018 at 8:43 PM Foivos Diakogiannis < > phoevos.diakogian...@gmail.com> wrote: > > > Dear all, > > > > first my compliments on this great software, and thank you all for the > > effort you've put into this. > > > > I am a gluon API user, and I thought I should give some feedback to > > highlight some user-perspective issues. I am working in CSIRO and I am > > using gluon to write and deploy custom deep learning models for semantic > > segmentation/classification on CSIRO HPC facilities. I came into the deep > > learning world as of July 2017 (2nd postdoc, after astronomy), starting > > with Keras (great intro, but too simple/automated for my needs), moving > on > > to TF (complexity of C++, with the inconvenience of python performance + > > memory management was bad. On the plus side, great > documentation+community > > support, and of course great product overall, just not for me) and as of > > December 2017 I am using gluon exclusively since it solved the majority
Re: A New API for creating .rec files
This makes a lot of sense to me Anirudh. On Tue, Sep 25, 2018 at 11:38 AM Anirudh Acharya wrote: > Hi, > > During some recent MXNet user surveys, one of the user requests was to have > a im2rec API that will have similar functionality as the im2rec tool( > https://mxnet.incubator.apache.org/faq/recordio.html?highlight=im2rec). > The > advantage with the API would be that the user can access this functionality > from the PyPi package itself, instead of cloning the repo. > > I was thinking of converting the tool into an API call under the mx.io > package. I will send the API design shortly. I wanted to know what the > community thinks of this change. > > > Thanks > Anirudh Acharya >
A New API for creating .rec files
Hi, During some recent MXNet user surveys, one of the user requests was to have a im2rec API that will have similar functionality as the im2rec tool( https://mxnet.incubator.apache.org/faq/recordio.html?highlight=im2rec). The advantage with the API would be that the user can access this functionality from the PyPi package itself, instead of cloning the repo. I was thinking of converting the tool into an API call under the mx.io package. I will send the API design shortly. I wanted to know what the community thinks of this change. Thanks Anirudh Acharya
Re: Some feedback from MXNet Zhihu topic
Thanks for the detailed feedback Foivos. Just want to dig into the C++ comment. Any more details you could give on how we could improve the readability. Would modernizing the codebase and trying to provide consistent code style help? In regards to documentation was it that it's mostly lacking, or that the existing documentation could be improved (or both)? -Kellen On Mon, Sep 24, 2018 at 8:43 PM Foivos Diakogiannis < phoevos.diakogian...@gmail.com> wrote: > Dear all, > > first my compliments on this great software, and thank you all for the > effort you've put into this. > > I am a gluon API user, and I thought I should give some feedback to > highlight some user-perspective issues. I am working in CSIRO and I am > using gluon to write and deploy custom deep learning models for semantic > segmentation/classification on CSIRO HPC facilities. I came into the deep > learning world as of July 2017 (2nd postdoc, after astronomy), starting > with Keras (great intro, but too simple/automated for my needs), moving on > to TF (complexity of C++, with the inconvenience of python performance + > memory management was bad. On the plus side, great documentation+community > support, and of course great product overall, just not for me) and as of > December 2017 I am using gluon exclusively since it solved the majority of > my problems. > > Things I love about gluon: > 1. Great structured tutorials (https://gluon.mxnet.io/), like a book. In > fact, at the time of starting using Gluon, this was better (i.e. more > structured, with a beginning and an end) than pytorch documentation. > 2. Efficient code, both in speed and GPU memory consumption. > 3. With a push of a button (hybridize) I can go from research to > production. I get up to x3-4 speed up, this is a huge benefit and I don't > see other frameworks easily beating that, in the immediate future. > torch.jit is nowhere near the ease of use of hybridize() - not yet. > 4. I really value the effort/support given in discuss.mxnet.io forum. > Almost always when I have a problem I find a solution there, from experts. > This complements my knowledge/understanding on the code around the gluon > API. > 5. Super easy data parallel modeling. The examples in the tutorial make > life really easy. This made a huge difference for me, and it was the main > reason I switched from TF to gluon. > > Things I find difficult with gluon: > 1. Documentation is not in one place, so gluon-cv and gluon-nlp are things > I've learned they exist (and they have great examples) via twitter. These > should be on the main mxnet page, somewhere altogether (they should > actually be advertised). In addition, some times the examples are not > updated with the latest changes. For example, > mynet.collect_params().initialize(...) on the gluon "book" should now be > mynet.initialize(...), and several other examples on the same spirit. Also, > I don't see a clear definition/description of new methods when added to > know how to improve my code, in the release announcements. For example, > I've learned about the block.summary(*inputs) features by checking on the > pull requests. Yes, it exists on the official API documentation, and I am > used in going through all of every now and then. Can be done better. > 2. Not all custom architectures are easy to implement in a hybrid format. > For example, taking the shape of a layer and using this as information for > pooling layers (or other things) is not easy (without copying to cpu > first), and many times I have to implement many hacks to get around this > (for performance gains). For example, here: > https://discuss.mxnet.io/t/coordconv-layer/1394/4 Another example is the > pyramid scene parsing network, it took me a lot of time and many hacks to > hybridize it. > 3. The distributed examples are not yet fully functional. When one needs to > run distributed computing for increasing the batch size is OK-ish (under > SLURM manager, see this: > https://discuss.mxnet.io/t/distributed-training-questions/1269/6 ), but > when one wants to implement async SGD - at least for me - is still an open > problem. Of course, I completely understand that distributed training is > still very much a research project, and I am not sure if using a large > batch size is good for training (hence my effort to use async SGD). I've > read various opinions on research papers for this. At the moment I am using > distributed only for hyper parameter optimization, as I increase the batch > size (when necessary) with delayed gradient updates. > 4. No higher order gradient support. This is where pytorch is better, and > where I am forced to use it in my GAN experiments for gradient penalty > implementation ( https://github.com/apache/incubator-mxnet/issues/10002). > I > hope that this will change in the immediate future. It is my understanding > that a lot of effort goes into semi-supervised training techniques and my > gut feeling tells me that GANs are an important key ingredient to th
