Thanks for finding these! Impala is more or less doing what we're doing, except it doesn't use submodules. One thing is that if you download their release (http://impala.apache.org/downloads.html) it's not clear where all the thirdparty dependencies went.
On Mon, Jan 9, 2017 at 12:00 PM, Hakan Memisoglu <hakanmemiso...@apache.org> wrote: > Hi Marc, > > Here are two Apache projects using C++: > > Impala: https://github.com/cloudera/Impala <https://github.com/cloudera/ > Impala> > > Kudu: https://github.com/cloudera/kudu <https://github.com/cloudera/kudu> > > Kudu has a download script for dependencies in third_party folder. > It seems to be a good example for our case. > > > On Jan 8, 2017, at 2:17 PM, Marc Spehlmann <spehl.apa...@gmail.com> > wrote: > > > > Regarding the IWYU script. It sounds safe then. If anything, maybe it > > belongs in another folder. ("tools/" ?) > > > > On Thu, Jan 5, 2017 at 3:07 PM, Julian Hyde <jh...@apache.org> wrote: > > > >> Better still, make a release depend only releases. Snapshots and even > >> commit IDs can disappear. > >> > >> And consider the state of the IP. In Apache and many other open source > >> projects, the IP is only guaranteed "clean" when it has been formally > >> released. > >> > >> Sorry this is all painful. But conversely, this is the value-add of > >> making a release. > >> > >> On Thu, Jan 5, 2017 at 11:57 AM, Harshad Deshmukh <hars...@cs.wisc.edu> > >> wrote: > >>>> Also, since you’re relying on the latest googletest, protobuf and re2 > >> your > >>>> release will work today and will break at some point in the future. > >> That’s > >>>> not OK. A release can depend only on releases, not snapshots or live > >>>> repositories. > >>> > >>> Thanks for pointing that. It is possible to fetch a particular release > >>> (based on its commit ID) of the submodule. We will add the release > >> commit ID > >>> to the git submodule file. > >>> > >>> > >>> > >>> On 01/05/2017 01:44 PM, Julian Hyde wrote: > >>>> > >>>> Git sub-modules are useful, I agree. But for a source release, the > goal > >> is > >>>> that someone should be able to download the source tar-ball and go: > >>>> > >>>> $ curl -O …/apache-quickstep-x.x-incubating.tar.gz > >>>> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz > >>>> $ cd apache-quickstep-x.x-incubating/build > >>>> $ ./download-thirdparty.sh > >>>> $ cmake etc. > >>>> > >>>> I tried to find other Apache projects that use git submodules and see > >> what > >>>> they do for source releases. Not much luck. Maybe someone else can > find > >>>> something. > >>>> > >>>> Also, since you’re relying on the latest googletest, protobuf and re2 > >> your > >>>> release will work today and will break at some point in the future. > >> That’s > >>>> not OK. A release can depend only on releases, not snapshots or live > >>>> repositories. > >>>> > >>>> Julian > >>>> > >>>> > >>>> > >>>>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <hars...@cs.wisc.edu> > >>>>> wrote: > >>>>> > >>>>> Thanks for the review Julian. > >>>>> > >>>>> For some of the third party libraries used in Quickstep (e.g. > >> googletest, > >>>>> protobuf and re2) we use the submodules feature of git. For such > >> libraries, > >>>>> the developer has to initialize the submodule only once, which pulls > >> code > >>>>> from the third party repo to the Quickstep third party directory. > >>>>> > >>>>> I don't know of a centralized repo for C++ projects. Does the git > >>>>> submodule method sound similar to the maven central approach you > >> mentioned? > >>>>> > >>>>> On 01/05/2017 12:43 PM, Julian Hyde wrote: > >>>>>> > >>>>>> I took a quick look at third_party and there don’t seem to be any > >>>>>> binaries in there. That’s good. You definitely cannot include > >> binaries in a > >>>>>> source release. > >>>>>> > >>>>>> The more you can remove from third_party, the better. It doesn’t > have > >> to > >>>>>> be done this release, but the less IP there is to review, the easier > >> for > >>>>>> everyone. > >>>>>> > >>>>>> Consider pulling from an external source the first time the > developer > >>>>>> builds in a sandbox, then apply patches. The patches will be the > only > >> thing > >>>>>> checked in to quickstep. (My expertise is in Java projects, which > >> these days > >>>>>> get their dependencies from a Maven repo such as Maven central; I > >> don’t know > >>>>>> whether there is an equivalent place to pull C and C++ source code. > >> Might be > >>>>>> worth a review of what other C and C++ based Apache projects do for > >> their > >>>>>> third-party dependencies.) > >>>>>> > >>>>>> Julian > >>>>>> > >>>>>> > >>>>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <spehl.apa...@gmail.com > > > >>>>>>> wrote: > >>>>>>> > >>>>>>> That seems to be the repo with the LLVM code for implementing > IWYU. I > >>>>>>> think > >>>>>>> what we have in our repo is scripts ontop of that library. I'm > >>>>>>> wondering > >>>>>>> where the scripts came from. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Marc > >>>>>>> > >>>>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh < > >> hars...@cs.wisc.edu> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hi Marc, > >>>>>>>> > >>>>>>>> How about this one for IWYU? > >>>>>>>> > >>>>>>>> https://github.com/include-what-you-use/include-what-you-use > >>>>>>>> /blob/master/LICENSE.TXT > >>>>>>>> > >>>>>>>> > >>>>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote: > >>>>>>>> > >>>>>>>>> I double checked the libraries in thirdy_party. They adhere to > >>>>>>>>> Apache's > >>>>>>>>> 3rd > >>>>>>>>> party requirement as they are all apache 2 or opensourced by > >> Google. > >>>>>>>>> The > >>>>>>>>> only issue I saw was that IWYU has no documentation. Anyone know > of > >>>>>>>>> its > >>>>>>>>> source? > >>>>>>>>> > >>>>>>>>> Library > >>>>>>>>> > >>>>>>>>> Ver > >>>>>>>>> > >>>>>>>>> License > >>>>>>>>> > >>>>>>>>> Notes > >>>>>>>>> > >>>>>>>>> benchmark > >>>>>>>>> > >>>>>>>>> Apache 2.0 > >>>>>>>>> > >>>>>>>>> cpplint > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> Header states that reuse is unconditional so long as the > copyright > >>>>>>>>> header > >>>>>>>>> stays intact. > >>>>>>>>> > >>>>>>>>> Farmhash > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING > >>>>>>>>> > >>>>>>>>> gflags > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING > >>>>>>>>> > >>>>>>>>> glog > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING > >>>>>>>>> > >>>>>>>>> gtest > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING > >> in > >>>>>>>>> subprojects. > >>>>>>>>> > >>>>>>>>> gperftools > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as COPYING file is preserved. > >>>>>>>>> > >>>>>>>>> iwyu > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> No license present > >>>>>>>>> > >>>>>>>>> linenoise > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as LICENSE file is preserved. > >>>>>>>>> > >>>>>>>>> protobuf > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as LICENSE file is preserved. > >>>>>>>>> > >>>>>>>>> RE2 > >>>>>>>>> > >>>>>>>>> Google > >>>>>>>>> > >>>>>>>>> No restrictions so long as LICENSE file is preserved. > >>>>>>>>> > >>>>>>>>> tmb > >>>>>>>>> > >>>>>>>>> Apache 2.0 > >>>>>>>>> > >>>>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal > >>>>>>>>> Software, > >>>>>>>>> Inc.) and is distributed under the same license terms. > >>>>>>>>> > >>>>>>>>> > >>>>>>>> -- > >>>>>>>> Thanks, > >>>>>>>> Harshad > >>>>>>>> > >>>>>>>> > >>>>> -- > >>>>> Thanks, > >>>>> Harshad > >>>>> > >>> > >>> -- > >>> Thanks, > >>> Harshad > >>> > >> > >
