Re: Severe legal issues with releases on repository.apache.org
On Sat, May 9, 2020 at 10:50 PM Tianqi Chen wrote: > > Seems the conclusion so far is only release source through apache and > release the binary builds as third party(as a different community, a > company or individual) Yes, that is the precedent established in multiple projects. I think it might still be worthwhile to pursue an exception from nvidia, though. Do we have any nvidia employees on the list that can inquire about that? Markus
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc2
Hi, I was trying to follow the build instructions[0] on Ubuntu 18.04. However, I a stumped at step 2: `cp config/config.cmake config.cmake` The file `cmake.conf` does not seem to exist in the tarball on the dist sit. `find . -name "cmake.conf" -print` finds nothing. In fact, the `config` folder doesn't seem to exist in the tarball either. However, the file and folder do exist on GitHub[1]. Are the build instructions for a release different from the build from the repository? On a related note: It might make sense to package build instructions with the source release. Websites get updated to reflect current use, and it might be difficult for future users of this version of mxnet to piece together the build instructions. Thanks, Markus [0]: https://mxnet.apache.org/get_started/ubuntu_setup [1]: https://github.com/apache/incubator-mxnet/tree/master/config On Tue, Feb 4, 2020 at 3:05 PM Lausen, Leonard wrote: > > Using latest upstream jemalloc > https://github.com/leezu/mxnet/commit/fd4c78a635087f6164344da53a55ba2b67da2fd2 > fixes the issue. > > However, there were concerns that this commit relies on unreleased development > features of jemalloc (jemalloc cmake build system support) and we'll not merge > this commit until upstream releases cmake build system support in a release. > > In the meantime anyone is welcome to work on an equivalent patch based on the > custom build system in latest stable jemalloc. > > On Tue, 2020-02-04 at 22:46 +, Lausen, Leonard wrote: > > Bisect identifies > > https://github.com/apache/incubator-mxnet/commit/425319cb59904573bd3fe1b6fe0a7381eceb9bbd > > > > Thus this is an issue with jemalloc + llvm libopemnp. > > > > The correct reproducer for latest master branch is > > > > > > git clone --recursive https://github.com/apache/incubator-mxnet/ mxnet > > cd mxnet > > git checkout a726c406964b9cd17efa826738a662e09d973972 # workaround > > https://github.com/apache/incubator-mxnet/issues/17514 > > mkdir build; cd build; > > cmake -DUSE_CPP_PACKAGE=1 -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja > > -DUSE_CUDA=OFF -DUSE_JEMALLOC=ON .. > > ninja > > ./cpp-package/example/test_regress_label # run a 2-3 times to reproduce > > > > Let's move the discussion to about fixing the jemalloc, openmp > > incompatibility > > to https://github.com/apache/incubator-mxnet/issues/17043 > > > > > > > > @Chris, could you look into this issue as it only happens with LLVM OpenMP? > > > > > > > > @Przemek: For 1.6.0 releas notes I suggest include recommendation to set > > USE_JEMALLOC=OFF when compiling from source. > > > > This note should probably be added in any case, as building with > > USE_JEMALLOC=ON > > is broken on Ubuntu Ubuntu 18.10 and higher, as well as Debian Stable. > > > > Given these release notes, +1 for the release. > > > > > > Best regards > > Leonard > > > > On Tue, 2020-02-04 at 22:26 +, Lausen, Leonard wrote: > > > Actually below reproducer is wrong. The issue was apparently fixed on > > > master > > > recently. I'm running an automated bisect and will report the result > > > later. > > > > > > On Tue, 2020-02-04 at 21:44 +, Lausen, Leonard wrote: > > > > Hi Chris, > > > > > > > > you previously found and fixed a OMP race condition during fork at > > > > https://github.com/apache/incubator-mxnet/pull/17039 > > > > > > > > This time no forks are involved. Could you run the following reproducer > > > > on > > > > master branch: > > > > > > > > git clone --recursive https://github.com/apache/incubator-mxnet/ mxnet > > > > cd mxnet > > > > git checkout a726c406964b9cd17efa826738a662e09d973972 # workaround > > > > https://github.com/apache/incubator-mxnet/issues/17514 > > > > mkdir build; cd build; > > > > cmake -DUSE_CPP_PACKAGE=1 -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja > > > > -DUSE_CUDA=OFF .. > > > > ninja > > > > ./cpp-package/example/test_regress_label # run a 2-3 times to > > > > reproduce > > > > > > > > > > > > As you are OpenMP expert, you may be able to identify the root cause > > > > withe > > > > relative ease. > > > > > > > > Thank you, > > > > > > > > Leonard > > > > > > > > On Tue, 2020-02-04 at 11:06 -0800, Chris Olivier wrote: > > > > > When "fixing", please "fix" through actual root-cause analysis (use > > > > > gdb, > > > > > for instance) and not simply by guesswork and cutting out things which > > > > > probably aren't actually at fault (blaming an OMP library that's in > > > > > worldwide distribution int he billions should be treated with great > > > > > skepticism). > > > > > > > > > > On Tue, Feb 4, 2020 at 10:44 AM Lin Yuan wrote: > > > > > > > > > > > Pedro, > > > > > > > > > > > > While I agree with you we need to fix this usability issue, I don't > > > > > > think > > > > > > this is a release blocker as Przemek mentioned above. Could we fix > > > > > > this > > > > > > in > > > > > > the next minor release? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Lin > > > > > > > > > > > > On Tue, Feb
Re: Stop redistributing source code of 3rdparty dependencies to avoid licensing issues
On Mon, Jan 20, 2020 at 1:17 PM Lausen, Leonard wrote: > If I don't misremember, our mentor Markus Weimer suggested at KDD 2019 in a > conversation that it's easiest to stop bundling non-ASF 3rdparty code. You do not misremember :-) If possible, it is easiest for everyone involved to capture the dependencies in a clean way for the build system to download. That being said, I came to this conclusion in the relative sane space of Java / Maven projects. I believe that this isn't the state of the art in C++, and might be difficult to do. Has anyone looked into whether the dependencies are or could be made available e.g. as VCpkgs? [0] The other aspect to consider are situations where the code mxnet depends on is actually also managed by people in the mxnet community. I believe a lot of that was the case when mxnet started in the incubator from dmlc. For those cases, one could consider inverting the dependency relationship: The ASF / ASL is designed to be the universal donor of software. Hence, it is unlikely that depending on any ASF code would be a problem for current dependencies of the DMLC code. Markus [0]: https://github.com/Microsoft/vcpkg
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
> Shall we provide pip wheels for later release votes? Apache projects generally do not provide binaries as "signed off" release artifacts. That being said, binaries are commonly provided and marked as "convenience binaries". Hence, no vote is necessary on them, nor are they "official" releases. Markus
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
> Shall we provide pip wheels for later release votes? Apache projects generally do not provide binaries as "signed off" release artifacts. That being said, binaries are commonly provided and marked as "convenience binaries". Hence, no vote is necessary on them, nor are they "official" releases. Markus On Sat, Jan 11, 2020 at 4:17 PM Hao Jin wrote: > > +1 (binding) > Built from source and ran all the latest d2l notebooks without problems. > Hao > > On Fri, Jan 10, 2020 at 10:03 PM Jun Wu wrote: > > > +1 (binding) > > > > Built from source. Ran all the GPU tests and test_numpy*.py cpu tests > > without problems. > > > > On Fri, Jan 10, 2020 at 9:43 PM Skalicky, Sam > > wrote: > > > > > We can enable building nightlys for feature branches too. > > > > > > Sam > > > > > > > On Jan 10, 2020, at 7:48 PM, Lin Yuan wrote: > > > > > > > > We can release one cpu-mkl and one CUDA wheel for testing various > > > > applications. Other people can build from source if they want other > > > flavors > > > > > > > > Lin > > > > > > > >> On Fri, Jan 10, 2020 at 4:00 PM Karan Jariwala < > > > karankjariw...@gmail.com> > > > >> wrote: > > > >> > > > >> Yes, agree with your point. But we will be requiring many flavors of > > > pip > > > >> wheel. > > > >> > > > >> MKL/ without MKL > > > >> CUDA/ no CUDA > > > >> Linux/windows/Mac > > > >> > > > >> Thanks, > > > >> Karan > > > >> > > > >> On Fri, Jan 10, 2020 at 3:54 PM Haibin Lin > > > >> wrote: > > > >> > > > >>> Shall we provide pip wheels for later release votes? > > > >>> > > > >>> Not everyone knows how to build MXNet from source (and building from > > > >> source > > > >>> also takes very long). Providing a pip wheel would lower the bar for > > > >> users > > > >>> who wants to test MXNet and participate in voting. > > > >>> > > > >>> Best, > > > >>> Haibin > > > >>> > > > >>> On Fri, Jan 10, 2020 at 3:50 PM Haibin Lin > > > > > >>> wrote: > > > >>> > > > >>>> +1 > > > >>>> > > > >>>> Built from source with USE_CUDA=1 on Ubuntu. Run gluon-nlp unit > > tests > > > >> and > > > >>>> they passed. > > > >>>> > > > >>>> On Fri, Jan 10, 2020 at 3:18 PM Karan Jariwala < > > > >> karankjariw...@gmail.com > > > >>>> > > > >>>> wrote: > > > >>>> > > > >>>>> +1 > > > >>>>> > > > >>>>> Tested MXNet with and without MKL-DNN on Ubuntu 16.04 with Horovod > > > >>> 0.18.2. > > > >>>>> No regression seen between 1.5.1 and 1.6.0.rc1 when running > > > >>> horovod_MXNet > > > >>>>> integration test. > > > >>>>> > > > >>>>> > > > >>>>> Thanks, > > > >>>>> > > > >>>>> Karan > > > >>>>> > > > >>>>> On Fri, Jan 10, 2020 at 2:47 PM Markus Weimer > > > >>> wrote: > > > >>>>> > > > >>>>>> +1 (binding) > > > >>>>>> > > > >>>>>> I tested on Ubuntu 18.04 on the Windows Subsystem for Linux. > > > >>>>>> > > > >>>>>> Tested: > > > >>>>>> * Built from source using the instructions here [0] > > > >>>>>> * Ran the tests in `./build/tests/mxnet_unit_tests` > > > >>>>>> * SHA512 of the archive > > > >>>>>> > > > >>>>>> Not tested: > > > >>>>>> * Language bindings > > > >>>>>> * CUDA or other GPU acceleration > > > >>>>>> * LICENSE and compliance status > > > >>>>>> * Signature of the archive > > > >>>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > > >
Re: [VOTE] Release Apache MXNet (incubating) version 1.6.0.rc1
+1 (binding) I tested on Ubuntu 18.04 on the Windows Subsystem for Linux. Tested: * Built from source using the instructions here [0] * Ran the tests in `./build/tests/mxnet_unit_tests` * SHA512 of the archive Not tested: * Language bindings * CUDA or other GPU acceleration * LICENSE and compliance status * Signature of the archive
Re: MXNet Podling Report - October
Sorry about not signing in time. I messed up my email filters somehow and the reminder did not make it to my inbox. I'll fix that :) -- Markus On Wed, Oct 10, 2018 at 6:45 AM Bob Paulin wrote: > Hi, > > I reviewed the report but it appears to be final now so it doesn't look > like I can edit the wiki. No concerns. Consider it approved. > > - Bob > > On 10/9/2018 12:37 PM, Haibin Lin wrote: > > Hi Sebastian, Markus and Bob, > > Do you have time to review MXNet's podling report for October? Is there > any question or concern? Thank you. > > Best, > Haibin > > On Fri, Oct 5, 2018 at 5:49 AM Michael Wall wrote: > >> Looks good to me. Not sure that Justin is subscribed to the dev so >> including him explicitly here. Thanks Haibin. >> >> Mike >> >> On Thu, Oct 4, 2018 at 8:12 PM Haibin Lin >> wrote: >> >> > Hi Justin and Michael, >> > >> > I updated the report with the links to the tutorial summaries: >> > June - >> > >> > >> https://lists.apache.org/thread.html/52f88e9dc7a6a2a1dfa5ad41c469fe2cdd1209a0be2eb345bc2f9a96@%3Cuser.mxnet.apache.org%3E >> > July - >> > >> > >> https://lists.apache.org/thread.html/dea9184350f2fe87ce450722ead28072f763196045f39859190f83f8@%3Cuser.mxnet.apache.org%3E >> > August - >> https://discuss.mxnet.io/t/apache-mxnet-digest-august-2018/1863 >> > >> > Justin, the length of the permanent link is longer than 76 characters. >> > Would this be an issue? >> > >> > Best, >> > Haibin >> > >> > >> > >> > On Thu, Oct 4, 2018 at 1:16 PM Haibin Lin >> > wrote: >> > >> > > Hi Justin, >> > > >> > > Thanks for the notice. I've reformatted the MXNet section to have at >> most >> > > 76 characters per line. Sorry about the last minute update. >> > > >> > > Best, >> > > Haibin >> > > >> > > On Thu, Oct 4, 2018 at 12:59 PM Justin Mclean >> > wrote: >> > > >> > >> Hi, >> > >> >> > >> I noticed you have edited the report after the due date and have >> broken >> > >> the formatting a little after I formatted it. Each line must have a >> > maximum >> > >> of 76 characters, would you mind fixing your section of the report? >> > >> >> > >> Thanks, >> > >> Justin >> > >> >> > > >> > >> > >
Re: Suggestions on how to increase community involvement on Apache MXNet incubating?
I really like most of what has been said, and the below might be a bit of a repeat, but through a different lens. One key aspect to consider when thinking about participation in OSS projects is the journey a contributor makes, which starts before even using the software. Each of the steps of the journey is only traversed by a minuscule number of people, so it is a good idea to have as many people as possible lined up at the beginning of the process :) A quick guestimate of the journey for a future contributor looks something like this: 1) They have to know off the project, which can be addressed by conferences, talks at universities and such. 2) They have to want to become users. This is really difficult to make happen externally :) 3) They have to be successful in their first use. Tutorials help with that, and I think mxnet is doing a fantastic job there with the Gluon book. 4) They need to be able to "lurk" among other users: There needs to be a place where the users of the software come together and chat, complain and help. Specifically, there should be *one* such place. The exact mechanisms can differ, but I found that email lists still serve this use case well, as they are sticky: Once subscribed, people stay informed :) If we get people to this stage, we can be proud: We just gained contributors: They are helping each other out, advocate for the project and all. Now, the next step is for them to contribute to the code in addition to the community: 5) They have to know how to contribute, e.g. a bug report. And that first time they do it needs to be encouraging for future contributions. 6) They need to be able to observe the work as it happens, e.g. by lurking on the mailing list. This gives context on what the community thinks about, how it makes decisions, ... . Just like in RL, observing a community gives confidence on how to engage with it. 7) When they start thinking about contributing, the process to do so needs to be clear. If step 6) worked, this doesn't even have to be very formal. Certainly, there are steps and hurdles that I forgot. There is a reason whole PhD theses are written about participation in OSS. But thinking down the lines of the journey should help us identify actions to be taken. With that descriptive stuff being said, allow me to be prescriptive with some ideas: (1) Drop all communications channels besides the dev list.This greatly simplifies lurking. My $DAY_JOB doesn't give me a chance to really follow slack, for instance. But I read every email on this list. (2) For those of you collocated in the same institution: Force yourself to not talk about mxnet in the office or at lunch. Pretend not to be in the same room or even on the same continent. Move all discussions onto the list. This makes lurking easier, and, in my experience, improves the quality of the technical discussions. (3) Decide on one way for users to discuss among each other. Most Apache projects use email lists for it as well. Which, granted, is pretty old school. Thanks for reading this far, I did not have enough time to write less :) Markus On Thu, Jan 4, 2018 at 5:22 AM, Pedro Larroywrote: > Agreed with Chris and Jeff. Googling for MXNet roadmap is > enlightening. Also the communication channels are disperse. > I would remove suggesting "Ask questions" in issues in the README.md > because this encourages inflation of issues that are just questions. > > We should link to the slack channel or discuss or mailing list in the > README and be clear on how to use those communication channels. > > More visibility on epics / roadmap in something like Trello? Roadmap > tagged issues right not don't seem to be really maintained. > > Explicitly asking in a TODO file or in the README or the Wiki on how > other people can contribute. > Having a list of "low hanging fruit" issues for newcomers to pick and > get familiar contributing to the project. > Making a blog post about how to contribute to MXNet, can be as simple > as how to use CLion to contribute to the project and send a patch... > > Just my thoughts. > > Pedro. > > On Wed, Jan 3, 2018 at 9:55 PM, Chris Olivier > wrote: > > I suggest more transparency in the development process and requirement > > definitions and grooming. > > > > Backlog wish-list on public wiki, which link to public JIRA epics or > > stories, where people outside of Amazon can edit/comment on the items or > > add items themselves. > > > > > > On Wed, Jan 3, 2018 at 11:48 AM, Bhavin Thaker > > wrote: > > > >> Hi Team, > >> > >> Do you have any suggestions on how to increase community involvement and > >> contributions including code additions/changes, bug-fixes/enhancements, > >> documentation updates, tutorials, increased adoption, answering > questions > >> on Stackoverflow/discuss.mxnet.io, etc.? > >> > >> While I have asked multiple questions in the one question above, I am > >> looking for more
Re: Podling Report Reminder - January 2018
Done. -- Markus On Sun, Dec 31, 2017 at 4:31 PM, Suneel Marthi <smar...@apache.org> wrote: > Report's been filed - mentors please comment and sign-off > > On Sun, Dec 31, 2017 at 12:42 PM, Suneel Marthi <smar...@apache.org> > wrote: > > > I'll post the report to Incubator Wiki sometime today - mentors can then > > sign off. > > > > On Sun, Dec 31, 2017 at 12:38 PM, Markus Weimer <mar...@weimo.de> wrote: > > > >> Thanks for putting this together! I made a pass and left one comment. > >> Besides that, it looks good to me and documents immense progress. Great > >> job > >> :) > >> > >> Regarding the timeline for mentor sign-off, this is challenging. > >> > >> I believe the way that I sign off on the report is to log into the > >> incubator wiki and adding an `x` at the right spot. It isn't enough for > >> you > >> to copy & paste that from the Google doc. > >> > >> The timeline set here gives exactly 1 day for us to do that, which > >> $DAY_JOB > >> might make challenging. I should be able to make it this time, but can't > >> really promise that in the future. Any chance we can get this into the > >> wiki > >> earlier in the future? > >> > >> Thanks! > >> > >> Markus > >> > >> On Sat, Dec 30, 2017 at 9:39 PM, Suk Won Kim <kim.suk...@gmail.com> > >> wrote: > >> > >> > Hi all, > >> > Bhavin Thaker and I propose the attached podling report draft and > would > >> > like your review comments and contributions to the report by 02-Jan, > >> 2018, > >> > since the report is due on 03-Jan, 2018. Mentors, please signoff this > >> draft > >> > via email by 1/2/2018 since we plan to submit the report by 1/3. > >> > > >> > Please find the draft below. > >> > https://docs.google.com/document/d/1PGhs96klZB6DXhpK9_ > >> > biPh4-aCm8-bWwFzexnOW_GMA/edit > >> > > >> > > >> > > >> > > >> > On Sat, Dec 30, 2017 at 6:30 AM, <johndam...@apache.org> wrote: > >> > > >> > > Dear podling, > >> > > > >> > > This email was sent by an automated system on behalf of the Apache > >> > > Incubator PMC. It is an initial reminder to give you plenty of time > to > >> > > prepare your quarterly board report. > >> > > > >> > > The board meeting is scheduled for Wed, 17 January 2018, 10:30 am > PDT. > >> > > The report for your podling will form a part of the Incubator PMC > >> > > report. The Incubator PMC requires your report to be submitted 2 > weeks > >> > > before the board meeting, to allow sufficient time for review and > >> > > submission (Wed, January 03). > >> > > > >> > > Please submit your report with sufficient time to allow the > Incubator > >> > > PMC, and subsequently board members to review and digest. Again, the > >> > > very latest you should submit your report is 2 weeks prior to the > >> board > >> > > meeting. > >> > > > >> > > Thanks, > >> > > > >> > > The Apache Incubator PMC > >> > > > >> > > Submitting your Report > >> > > > >> > > -- > >> > > > >> > > Your report should contain the following: > >> > > > >> > > * Your project name > >> > > * A brief description of your project, which assumes no knowledge > of > >> > > the project or necessarily of its field > >> > > * A list of the three most important issues to address in the move > >> > > towards graduation. > >> > > * Any issues that the Incubator PMC or ASF Board might wish/need > to > >> be > >> > > aware of > >> > > * How has the community developed since the last report > >> > > * How has the project developed since the last report. > >> > > * How does the podling rate their own maturity. > >> > > > >> > > This should be appended to the Incubator Wiki page at: > >> > > > >> > > https://wiki.apache.org/incubator/January2018 > >> > > > >> > > Note: This is manually populated. You may need to wait a little > before > >> > > this page is created from a template. > >> > > > >> > > Mentors > >> > > --- > >> > > > >> > > Mentors should review reports for their project(s) and sign them off > >> on > >> > > the Incubator wiki page. Signing off reports shows that you are > >> > > following the project - projects that are not signed may raise > alarms > >> > > for the Incubator PMC. > >> > > > >> > > Incubator PMC > >> > > > >> > > >> > > >> > > >> > -- > >> > *Sukwon Kim* > >> > > >> > > > > >
Re: Podling Report Reminder - January 2018
On Sun, Dec 31, 2017 at 9:42 AM, Suneel Marthiwrote: > I'll post the report to Incubator Wiki sometime today - mentors can then > sign off. > Thanks! Markus
Re: [DISCUSSION] Adding labels to PRs
Option 2 works for us over in REEF. We are a bit (too) religious about it [0], but it creates really nice commit messages[1]. We require each commit message to start with a one line summary which names the JIRA in brackets and describes the change, e.g. `[REEF-1234] Added integration with mxnet`. The remainder of the commit message is valid markdown, and they all end in a block which contains explicit references to the JIRA and pull request: ``` JIRA: [REEF-1234](https://issues.apache.org/jira/browse/REEF-1234) Pull Request: This closes #1234 ``` The hope is that this structure will eventually proof useful in automated analysis. Then again, we haven't done that ever :) Markus [0]: https://cwiki.apache.org/confluence/display/REEF/Commit+Messages [1]: https://github.com/apache/reef/commits/master
Re: Podling Report Reminder - July 2017
On Wed, Jul 5, 2017 at 3:30 PM, Suneel Marthiwrote: > Thanks Dom, the report has been filed. > > Mentors, please sign-off on the report. Thanks and done! Markus
Re: Test
Yes, it came through. -- Markus On Sun, Jun 25, 2017 at 8:29 PM, Chris Olivierwrote: > Does this email work? I am having trouble subscribing
Re: Draft of March 2016 Podling Report
On 2017-03-01 22:13, Suneel Marthi wrote: Thanks Henry. Done, other mentors please go ahead and sign the report. Thanks for drafting! I signed. Markus