Ok, if no other objections then we have no blockers for source release. There is a process to get the tar into Apache mirrors but to release on Github all we need to do it merge develop, wait for travis tests and tag the master. Github can be told to produce a source release tar and host it and the install.sh can be made to point to it.
Not sure what the ASF rules are regarding this so maybe the mentors can comment—specifically do we have to use the Apache mirror system? On Sep 5, 2016, at 4:34 PM, Pat Ferrel <p...@occamsmachete.com> wrote: On Sep 5, 2016, at 1:25 PM, Alex Merritt <emergentor...@apache.org> wrote: Agree we should go source only for this release. On Sep 5, 2016 1:10 PM, "Suneel Marthi" <suneel.mar...@gmail.com> wrote: > On Mon, Sep 5, 2016 at 2:55 PM, Andrew Purtell <andrew.purt...@gmail.com> > wrote: > >> I also don't have experience with SBT, apologies. I did do some poking >> around on Google and it looks like SBT is well behind Maven in providing >> this type of functionality out of the box or by third party plugin >> (sbt-assembly does some useful and interesting things but is focused >> exclusively on producing über jars). I think that's to be expected given >> the origin story. "Maven is huge and crufty and we want new and simple!" >> "Ok, let's make Simple Build Tool!" Fast forward. No longer simple. Not >> able to do a lot of what Maven can. Years of reinventing the wheel ahead, >> ahoy! Happy to be corrected. >> > > Heh, not to mention that Sbt is just not as flexible as maven in being able > to handle different phases and cycles of build and deployment. > > >> Doing what I've described looks achievable by programming what is needed >> in SBT's DSL. Source only releases for a while maybe? Or work up LICENSE >> and NOTICE files by hand and figure out how to break release builds if >> dependencies change and the metadata hasn't been updated by hand? >> >> I was wondering why Spark went with Maven for their build of reference. >> >> My little rant on SBT aside I am NOT suggesting you replace SBT with >> Maven. That would be in my opinion an unfortunate use of developer >> bandwidth better put to task getting the current software with current >> build system out the door in a first Apache release. >> > > +1 and we all seem to agree for a quick source-only first release. > > >>> On Sep 5, 2016, at 11:23 AM, Suneel Marthi <smar...@apache.org> wrote: >>> >>> Its easy to do what Andy is describing using maven's assembly plugin in >> the >>> maven world. I have no experience with sbt so can't speak to how it can >> be >>> done with Sbt and would defer that to the experts. >>> >>> We hit a similar issue with licenses in source and binary on the first >> Pirk >>> release last week. We finally decided to make a source-only first > release >>> while we r now working on fixing the binary license packaging for the >> next >>> release. >>> >>> >>> >>> On Mon, Sep 5, 2016 at 2:05 PM, Andrew Purtell < > andrew.purt...@gmail.com >>> >>> wrote: >>> >>>> It covers LICENSE and NOTICE file generation for both source and > binary >>>> releases, and inclusion of the resulting files in source archives, >> binary >>>> jars, and binary archives through integration with the maven build and >>>> assembly targets. >>>> >>>> Including the complete text of any given license in LICENSE is > important >>>> but only needs to be done once. You retain the copyright notice and >> mention >>>> of the license type per dependency. We are just talking about >>>> deduplicating, eg 100 full texts of the ASLv2 into one. >>>> >>>>> On Sep 5, 2016, at 10:45 AM, Pat Ferrel <p...@occamsmachete.com> > wrote: >>>>> >>>>> Thanks Andy. >>>>> >>>>> RE “Only need to include one entry with the complete text of a > license, >>>> everything else can just name the license.” So the copyright notice in >> the >>>> license is not important, only the license type? This is often the > only >>>> important difference in the license from one dep to another. >>>>> >>>>> It sounds like your automation covered LICENSE.txt creation? or just >>>> inclusion in the binary? >>>>> >>>>> >>>>>> On Sep 5, 2016, at 9:59 AM, Andrew Purtell < > andrew.purt...@gmail.com> >>>>> wrote: >>>>> >>>>> I won't weigh in on the question at hand but I'd like to make a > couple >>>> of clarifications for what it is worth: >>>>> >>>>>> This yielded 166 deps, so this implies we need to include 166 > licenses >>>> and copyright notices in LICENSE.txt. >>>>> >>>>> There are some available simplifications: >>>>> >>>>> - Only need to include one entry with the complete text of a license, >>>> everything else can just name the license. >>>>> >>>>> - Where there are multiple artifacts coming from a single project, > like >>>> Hadoop, only one entry for the project is needed. >>>>> >>>>>> Donald is looking at automating this but I’m personally dubious >>>>> >>>>> As I think I've mentioned before here we have successfully automated >>>> this for HBase (based on automation done by yet other Apache projects) >> so I >>>> hope you'll take my advice and evidence based assertion it can be > done. >>>> Caveat: we use maven not SBT as build framework. >>>>> >>>>> >>>>>> >>>>>> On Sep 5, 2016, at 9:43 AM, Pat Ferrel <p...@actionml.com> wrote: >>>>>> >>>>>> This weekend I tracked down all out deps, which required a few > scripts >>>> to process sbt output. This yielded 166 deps, so this implies we need > to >>>> include 166 licenses and copyright notices in LICENSE.txt. As I read > the >>>> Apache guidelines this should be the license that goes with the > version >> we >>>> include since the copyright owner of license may have changed in newer >>>> versions. >>>>>> >>>>>> This may be near impossible to maintain by hand if we have frequent >>>> dependency upgrades and frequent releases. Donald is looking at >> automating >>>> this but I’m personally dubious about this because it require all 166 >> deps >>>> have maintained their licenses in artifacts for all versions we might >> use. >>>>>> >>>>>> A source release requires that *only* the source included be > reflected >>>> in LICENSES.txt. This would be ~0, I think a couple things are > included. >>>>>> >>>>>> Several things lead me to favor a source-only release: >>>>>> 1) 166 licenses needed for binary ~0 needed for source—I’d rather we >>>> spend time on things that add more value >>>>>> 2) I have never used the binary release. Any version of a source >>>> download and `./make-dirstribution` works universally. >>>>>> 3) our install.sh now installs source and builds it for the user. > This >>>> is good because we can use the same script for unreleased -SNAPSHOT >>>> versions sitting in the `develop` branch. >>>>>> 4) outside of instructions for downloading and installing the binary >>>> that do not yet exist afaik, there would be no obvious way for the > user >> to >>>> get the binary. >>>>>> 5) indirectly any delay to release is getting to be a serious > problem. >>>> We haven’t had a well supported release from the main project since >> close >>>> on a year ago and work on new features is being delayed. >>>>>> 6) we can do a source only release now and be clean of the license >>>> issue as far as the IPMC is concerned. We can add binary when we have > a >>>> better answer to automation. In other words why hold the release for >> binary? >>>>>> >>>>>> Since this decision will affect the project for as long as it is in >>>> incubation. I’d like to see what others think. I believe we can > release >> now >>>> if we do source-only. >>>>>> >>>>>> Source only, or source & binary? >>>> >> >
