The main trade off is that you can do it as long as the all the dependency is contained easy to build. If you start to involve things like cuda code you want to switch to formal build system. That is why I explained most usecases is for a small runtime component
On Sat, Sep 28, 2019 at 10:48 AM Chris Olivier <cjolivie...@apache.org> wrote: > This seems similar to sqlite which offers (maybe now by default?) a single > c file that is the whole DB engine and makes it quite attractive as a local > db option. > > I have no strong opinion on whether or not to keep amalgamation, but I > wasn’t actually aware of this use-case for mxnet and I recall how much I > liked sqlite for this. > > On Sat, Sep 28, 2019 at 10:17 AM Tianqi Chen <tqc...@cs.washington.edu> > wrote: > > > To give a complete picture. I will talk a bit about my experience on > using > > the all-in-one file approach and a few examples in the tvm project. > > > > Currently, tvm uses all-in-one build for its lightweight runtime in cases > > where we want a separate build system and do not want to bother to use > > CMake, this include > > > > - Standalone deployment: > > https://github.com/dmlc/tvm/tree/master/apps/howto_deploy > > - Android projects where we need to use NDK for cross compilation. > > - iOS where we need to use XCode. > > > > In all these cases, all-in-one build provide the benefit of easy to > > incorporate into the target project. However, we only use all-in-one > build > > for the "minimum runtime" that is necessary to run a model. But will not > > include components that compiles and optimizes the model as things will > get > > out of hand pretty quickly and in those cases, having a build system > > out-weights the gains of the single file. > > > > So if there is a minimum runtime that MXNet wants to incorporate into the > > users' env and the developers do not want to code up a CMake recipe for > > that, it is a possible route. And in that case, I would still suggest to > > move away from the current approach that create a single file but follow > > the approach in the above link. In both cases, the current amalgamation > is > > not fulfilling its function so I think it is fine to remove and add new > > ones with thoughts if necessary > > > > TQ > > > > On Sat, Sep 28, 2019 at 9:49 AM Tianqi Chen <tqc...@cs.washington.edu> > > wrote: > > > > > The main use of amalgamation(aka all in one file build) for cases where > > it > > > is hard to setup a Make system. Most user knows how to include a single > > > file into their project, but it is relatively harder to incorporate an > > > entire build system. > > > > > > As a result, all-in-one file runtime is still being quite widely used > and > > > I personally liked the approach, I just suggested that the current > > approach > > > may not be the best way to go and creates some maintenance burden. > > > > > > See the link of an example project that uses new all-in-one approach > that > > > i mentioned(which illustrates the usecase of all-in-one file as well) > > > https://github.com/dmlc/tvm/tree/master/apps/howto_deploy > > > > > > TQ > > > > > > On Sat, Sep 28, 2019 at 3:46 AM Marco de Abreu < > marco.g.ab...@gmail.com> > > > wrote: > > > > > >> Do we have a good knowledge and overview over all the use cases that > use > > >> Amalgamation? At least from my perspective I don't feel well informed > > >> about > > >> the blast radius. > > >> > > >> -Marco > > >> > > >> Junru Shao <junrushao1...@gmail.com> schrieb am Sa., 28. Sep. 2019, > > >> 09:14: > > >> > > >> > As Tianqi and Sheng mentioned, given the fact that we are able to do > > >> > deployment in a possibly better way (correct me if I was wrong), I > > would > > >> > love to +1 to Pedro’s proposal. > > >> > > > >> > In the meantime, as a healthy open source community, I also agree > with > > >> > Naveen’s point that we should do more homework for both our > developers > > >> and > > >> > customers. IMHO, for example, it would be super helpful if Pedro may > > >> bring > > >> > up some documentation describing what is the “best practice” of > using > > >> the > > >> > alternative of amalgamation, if our community agree to deprecate it. > > >> > > > >> > Thank you guys so much for the discussion! > > >> > > > >> > Junru > > >> > > > >> > On Fri, Sep 27, 2019 at 17:00 Tianqi Chen <tqc...@cs.washington.edu > > > > >> > wrote: > > >> > > > >> > > The original amalgamation tries to generate a single file for > > >> compilation > > >> > > via a script. This is largely un-necessary, having a file that > > >> include > > >> > the > > >> > > dependent files and compile that one is relatively easy and > > sometimes > > >> > more > > >> > > robust(without expanding everything into a single file). > > >> > > > > >> > > I think it might makes sense to deprecate and remove the current > one > > >> > given > > >> > > that it is no longer properly functioning. If someone is > interested, > > >> > create > > >> > > another deployment example that is more standalone without the > file > > >> > > expansion. Here is an reference of the "new style" used in the tvm > > >> > project > > >> > > https://github.com/dmlc/tvm/tree/master/apps/howto_deploy > > >> > > > > >> > > TQ > > >> > > > > >> > > On Fri, Sep 27, 2019 at 4:49 PM Sheng Zha <zhash...@apache.org> > > >> wrote: > > >> > > > > >> > > > As an alternative to amalgamation, could we simply ask users to > > >> > > statically > > >> > > > link to libmxnet.a, and then prune the symbol table to get rid > of > > >> the > > >> > > > binary of unused functions? Though I don't know the full context > > of > > >> > > > amalgamation, based on my knowledge on this feature I'm not > aware > > of > > >> > any > > >> > > > difference in the end result, compared to the code-inlining > > approach > > >> > that > > >> > > > amalgamation takes. > > >> > > > > > >> > > > -sz > > >> > > > > > >> > > > On 2019/09/12 17:29:02, Naveen Swamy <mnnav...@gmail.com> > wrote: > > >> > > > > so the original email suggesting to remove was after all > > >> self-serving > > >> > > :) > > >> > > > > > > >> > > > > let's encourage if someone wants to maintain and make use of > the > > >> > > original > > >> > > > > work and make it better. > > >> > > > > > > >> > > > > -1 to remove at this point > > >> > > > > > > >> > > > > P.S: I suggest to do some due diligence before bringing topics > > up > > >> for > > >> > > > > discussion. > > >> > > > > > > >> > > > > On Wed, Sep 11, 2019 at 8:10 AM Lv, Tao A <tao.a...@intel.com > > > > >> > wrote: > > >> > > > > > > >> > > > > > Sorry to chime in. > > >> > > > > > > > >> > > > > > There is a PR to fix amalgamation. I was pinged several > times > > to > > >> > > merge > > >> > > > it > > >> > > > > > but I don't think I have enough knowledge to do that. So it > > >> would > > >> > be > > >> > > > great > > >> > > > > > if someone from this thread can help to review. > > >> > > > > > > > >> > > > > > https://github.com/apache/incubator-mxnet/pull/15303 > > >> > > > > > > > >> > > > > > thanks, > > >> > > > > > -tao > > >> > > > > > > > >> > > > > > -----Original Message----- > > >> > > > > > From: Marco de Abreu <marco.g.ab...@gmail.com> > > >> > > > > > Sent: Wednesday, September 11, 2019 9:38 PM > > >> > > > > > To: dev@mxnet.incubator.apache.org > > >> > > > > > Subject: Re: [DISCUSS] Remove amalgamation > > >> > > > > > > > >> > > > > > Is Amalgamation only used on Android though? Are there any > > other > > >> > use > > >> > > > cases? > > >> > > > > > > > >> > > > > > -Marco > > >> > > > > > > > >> > > > > > Pedro Larroy <pedro.larroy.li...@gmail.com> schrieb am Mi., > > 11. > > >> > Sep. > > >> > > > 2019, > > >> > > > > > 11:57: > > >> > > > > > > > >> > > > > > > Hi Anirudh > > >> > > > > > > > > >> > > > > > > Appreciate your feedback and sorry if my email came across > > >> that > > >> > way > > >> > > > to > > >> > > > > > > you, I think you might miss some context. I don't think > > >> calling > > >> > > > > > > something hacky is anything bad and isn't supposed to be > the > > >> > topic > > >> > > of > > >> > > > > > > the discussion. It was reported as not working by users, > > hence > > >> > the > > >> > > > > > > original thread. It was a request for opinions from people > > who > > >> > > might > > >> > > > > > > actually have tried to work in Mxnet on Android. > > >> > > > > > > > > >> > > > > > > Thanks. > > >> > > > > > > > > >> > > > > > > Pedro. > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > On Tuesday, September 10, 2019, Anirudh Subramanian > > >> > > > > > > <anirudh2...@gmail.com > > >> > > > > > > > > > >> > > > > > > wrote: > > >> > > > > > > > Hi Pedro, > > >> > > > > > > > > > >> > > > > > > > I don't see anything "destructive" with Chris asking for > > >> > > > > > > > justification > > >> > > > > > > for > > >> > > > > > > > you calling something "hacky". The only email in this > > thread > > >> > > where > > >> > > > I > > >> > > > > > > > see > > >> > > > > > > ad > > >> > > > > > > > hominems and disrespectful comments is your email. > > >> > > > > > > > > > >> > > > > > > > On Sat, Sep 7, 2019, 10:18 PM Pedro Larroy > > >> > > > > > > > <pedro.larroy.li...@gmail.com > > >> > > > > > > > > > >> > > > > > > > wrote: > > >> > > > > > > > > > >> > > > > > > >> Apache mentors should have a look at these reincident > > >> > harassment > > >> > > > > > > >> and destructive behaviors which demotivate > contributions > > >> and > > >> > > take > > >> > > > > > > >> action. It takes only one bad apple to ruin a > community. > > >> > > > > > > >> > > >> > > > > > > >> The mobile solution that is known to work as of know is > > >> cross > > >> > > > > > > >> compiling with "ci/build.py -p build.android_armv8" or > > >> > > > > > > >> "build.android_armv7". The only advantage of > amalgamation > > >> is > > >> > to > > >> > > > > > > >> provide a smaller binary that we > > >> > > > > > > could > > >> > > > > > > >> accomplish with the C preprocessor. > > >> > > > > > > >> > > >> > > > > > > >> My technical contributions speak for themselves, > > including > > >> > > porting > > >> > > > > > > >> MXNet > > >> > > > > > > to > > >> > > > > > > >> Android and ARM and helping many users run MXNet in > > Jetson, > > >> > > > > > > >> Raspberry Pi and Android amongst many other topics. I > > have > > >> > never > > >> > > > > > > >> been disrespectful > > >> > > > > > > to > > >> > > > > > > >> anyone. I'm entitled to my own technical opinions about > > >> > > > > > > >> amalgamation or > > >> > > > > > > any > > >> > > > > > > >> other piece of code whatsoever, that's no personal > > >> disrespect > > >> > to > > >> > > > > > > >> anyone > > >> > > > > > > and > > >> > > > > > > >> perfectly valid. If you are not interested in this > > project > > >> > > > anymore, > > >> > > > > > > >> do > > >> > > > > > > us > > >> > > > > > > >> all a favor and stop trolling and being toxic. If you > > want > > >> my > > >> > > > > > > >> respect, > > >> > > > > > > step > > >> > > > > > > >> up your technical contributions, be positive and > > encourage > > >> > > others, > > >> > > > > > > >> this including commits, I haven't seen for many months, > > >> please > > >> > > be > > >> > > > > > > >> positive > > >> > > > > > > and > > >> > > > > > > >> constructive. This scorched-earth attitude is only > > >> reflecting > > >> > > bad > > >> > > > > > > >> on > > >> > > > > > > you. > > >> > > > > > > >> I'm certainly not interested in your ad-hominems or > > unasked > > >> > for > > >> > > > > > > technical > > >> > > > > > > >> advice, which to be honest, showing poor judgment and > > >> > > ignorance. > > >> > > > > > > >> Myself and others have come up with numbers, graphs, > > >> metrics > > >> > and > > >> > > > > > > >> arguments and have been met with dismissal, trolling > and > > >> > > > > > > >> sea-lioning. I have recieved your insults via public > and > > >> > private > > >> > > > > > > >> channels (such as linkedin) as have others. This is not > > ok > > >> and > > >> > > has > > >> > > > > > > >> to stop. If you have something personal against me or > > >> against > > >> > > your > > >> > > > > > > >> former employer, this is not the right place > > >> > > > > > > or > > >> > > > > > > >> forum. > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> On Fri, Sep 6, 2019 at 3:56 PM Chris Olivier > > >> > > > > > > >> <cjolivie...@gmail.com> > > >> > > > > > > >> wrote: > > >> > > > > > > >> > > >> > > > > > > >> > Hi Pedro, > > >> > > > > > > >> > > > >> > > > > > > >> > While I was not involved with amalgamation or its > > >> > development > > >> > > in > > >> > > > > > > >> > any > > >> > > > > > > way, > > >> > > > > > > >> > can you please refrain from referring to the work of > > >> others > > >> > > as a > > >> > > > > > > "hacky > > >> > > > > > > >> > solution"? This is derogatory slang and the > statement > > >> was > > >> > not > > >> > > > > > > supported > > >> > > > > > > >> > with any justification for such name-calling. > Someone > > >> > spent a > > >> > > > > > > >> > good > > >> > > > > > > deal > > >> > > > > > > >> of > > >> > > > > > > >> > time on this solution at some point in time and I am > > >> sure it > > >> > > > > > > >> > worked > > >> > > > > > > for > > >> > > > > > > >> its > > >> > > > > > > >> > purpose at that time -- I think it was used in the > > >> original > > >> > > > > > > >> > javascript > > >> > > > > > > >> port > > >> > > > > > > >> > as well, actually -- and it is disrespectful to call > > >> their > > >> > > > > > > >> > efforts "hacky". Please respect what came before. > > >> > > > > > > >> > > > >> > > > > > > >> > Thanks for understanding, > > >> > > > > > > >> > > > >> > > > > > > >> > -Chris > > >> > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > > >> > On Fri, Sep 6, 2019 at 3:07 PM Pedro Larroy < > > >> > > > > > > >> pedro.larroy.li...@gmail.com> > > >> > > > > > > >> > wrote: > > >> > > > > > > >> > > > >> > > > > > > >> > > Hi > > >> > > > > > > >> > > > > >> > > > > > > >> > > I would like to propose to remove amalgamation from > > >> MXNet > > >> > > and > > >> > > > > > > >> > > CI, > > >> > > > > > > users > > >> > > > > > > >> > > have reported that they couldn't use it > successfully > > in > > >> > > > > > > >> > > Android, and instead they were able to use the > cross > > >> > > compiled > > >> > > > > > > >> > > docker build > > >> > > > > > > >> > successfully. > > >> > > > > > > >> > > > > >> > > > > > > >> > > Any reason why we shouldn't remove this hacky > > solution? > > >> > > > > > > >> > > > > >> > > > > > > >> > > Pedro. > > >> > > > > > > >> > > > > >> > > > > > > >> > > > >> > > > > > > >> > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
