+1 Peter. Note that the current build system does not support java 8.
Maintaining a build system is a significant burden ( I know I had to implement Java 5 support). We will need a new build system for java 8, this looks like a step in that direction. In reality we need to adopt the build conventions used by Rio. Regards, Peter. ----- Original message ----- > > As discussion has settled somewhat, I would like to call another vote to > accept the latest patch described in > https://issues.apache.org/jira/browse/RIVER-432 > > The patch removes the archived jar files for Velocity and ASM and > replaces them with Apache Ivy scripts that download the jars from Maven > Central the first time a build is done. From then on, the jar files are > in Ivy’s repository (for more info, see http://ant.apache.org/ivy). > > Voting will remain open at least until 2000 UTC Feb 13, 2014. > > Cheers, > > Greg. > > > On Jan 3, 2014, at 12:57 PM, Greg Trasuk <tras...@stratuscom.com> wrote: > > > > > On Jan 3, 2014, at 5:25 AM, Simon IJskes - QCG <si...@qcg.nl> wrote: > > > > > In order to gain some time to discuss this first i will vote -1. > > > > > > First, we decided to NOT remove velocity builder. > > > > When I read the email chain, my impression was that we wanted to > > remove it (to quote you Sim, “To be honest, I hate it”), but there was > > a dependency on it in the ‘extras’ folder that was added in the trunk > > branch. As there is no ‘extras’ in the 2.2 branch, and that is what > > this patch applies to, I thought it was fair to remove > > VelocityConfigurationBuilder from the 2.2 branch. Perhaps we should > > revisit the ConfigurationBuilder approach in another thread. For now > > I’ll spin another patch that doesn’t remove > > VelocityConfigurationBuilder. > > > > > > > > Second, no need to remove the jars as specified in your own comments > > > on RIVER-432. > > > > > > Pulling in external jars at compile time, shall we start here? > > > > > > They are already in the svn. They are already in the build scripts. > > > What does this patch fix? No legal problems? > > > > > > > Apache policy is somewhat unclear on this point. One needs to examine > > the mailing lists for clues on what we should really do. I will argue > > that: > > > > 1 - The fundamental distribution model of Apache is source code, not > > binaries. 2 - Distributing binaries is tolerated but not encouraged. > > Since the svn repository can be seen as a distribution point, binaries > > in svn are also tolerated but not encouraged. 3 - Downloading > > dependency binaries at build time is technologically easy, provides > > the same guarantees as putting them in cvs, and avoids the question of > > effectively distributing someone else’s code. > > > > All these together suggest that although we’re technically OK to put > > dependency jars in a “-deps” package (note that the status quo _is_ > > unacceptable - at the very least, we need to restructure the > > dependencies into a “-deps” binary package), there is some policy > > uncertainty which we avoid totally by having dependencies downloaded > > from a known-good source at build time. > > > > Let’s examine these points: > > > > 1 - The fundamental distribution model of Apache is source code, not > > binaries. Apache began with httpd. Back in those days, “Open Source” > > software was distributed in source form only, simply because it was > > mostly intended for Unix systems (then later Linux). I recall the > > first release of Perl coming down as a ten-part uunet news message. > > Part of this distribution model was practical necessity - System > > differences made it necessary to compile your software on the hardware > > it was going to run on. Part of it was open-source philosophy. > > Having the source code meant that you could take a look at it and > > verify that it wasn’t hazardous to your operations. > > > > In any case, the way we use to use open source software was > > (“./configure; make; make install”). If the software had > > dependencies, you built them from source, for the same reasons. > > > > Now, as time has gone on, we’ve gotten used to having the JVM as a > > common runtime, and jar files as a common binary distribution medium. > > But the Apache Foundation’s mandate is to produce open source software > > that is freely usable under the Apache License. That means source > > code is at the heart of Apache, despite the rest of the world’s > > comfort with binaries. Hence Roy’s statements in (1): > > > > > Class files are not open source. Jar files filled with class files > > > are not open source. The fact that they are derived from open source > > > is applicable only to what we allow projects to be dependent upon, > > > not what we vote on as a release package. Release votes are on > > > verified open source artifacts. Binary packages are separate from > > > source packages. One cannot vote to approve a release containing a > > > mix of source and binary code because the binary is not open source > > > and cannot be verified to be safe for release (even if it was > > > derived from open source). > > > > > > I thought that was frigging obvious. Why do I need to write > > > documentation to explain something that is fundamental to the open > > > source definition? > > He’s talking about binary packages, not jar files in svn, but I read > > that (and many other emails) as a distaste for binary distributions. > > > > In fact, if you look at Apache httpd’s download page, it doesn’t > > appear that the Apache project publishes any Unix or Linux binaries. > > They leave that to other organizations. > > > > 2 - Distributing binaries is tolerated but not encouraged. Since the > > svn repository can be seen as a distribution point, binaries in svn > > are also tolerated but not encouraged. > > > > It’s hard to find a single reference that encapsulates this outlook, > > but that’s the impression I get from reading the various mailing > > lists. For instance, Sam Ruby says (2): > > > IMO, our projects release source. So, our projects should not > > > maintain object/binary artifacts in their svn release tree, > > > regardless of license (category a or b). > > There is some debate on whether the svn tree should be considered a > > distribution point. Incubator releases are regularly called out for > > not having “NOTICE” and “RELEASE” files at all reasonable checkout > > points in svn. [LEGAL-26] > > (https://issues.apache.org/jira/browse/LEGAL-26) concerns this and > > remains unresolved. > > > > Doug Cutting (3) says: > > > On Mon, Sep 16, 2013 at 2:50 AM, Stephen Connolly > > > <stephen.alan.conno...@gmail.com> wrote: > > > > * Source control is not an Apache distribution and hence we do not > > > > need to have LICENSE and NOTICE files in source control, it can be > > > > a nice convenience, but there is no *requirement*. > > > > > > I think perhaps you're looking for clear lines where things are > > > actually a bit fuzzy. Certainly releases are official distributions > > > and need LICENSE and NOTICE files. That line is clear. On the other > > > hand, we try to discourage folks from thinking that source control is > > > a distribution. Rather we wish it to be considered our shared > > > workspace, containing works in progress, not yet always ready for > > > distribution to folks outside the foundation. But, since we work in > > > public, folks from outside the foundation can see our shared > > > workspace and might occasionally mistake it for an official > > > distribution. We'd like them to still see a LICENSE and NOTICE > > > file. So it's not a hard-and-fast requirement that every tree that > > > can possibly be checked out have a LICENSE and NOTICE file at its > > > root, but it's a good practice for those trees that are likely to be > > > checked out have them, so that folks who might consume them are well > > > informed. > > Again, he’s not talking directly about jar files in svn, however I > > think his statement that “since we work in public, folks from outside > > the foundation can see our shared workspace and might occasionally > > mistake it for an official distribution” applies here. Fundamentally, > > the decision on how and where to distribute ‘velocity.jar’ rightly > > belongs with the Velocity group and I don’t think we ought to > > redistribute it. > > > > 3 - Downloading dependency binaries at build time is technologically > > easy, provides the same guarantees as putting them in cvs, and avoids > > the question of effectively distributing someone else’s code. > > > > There doesn’t seem to be clear policy in the ASF on this, as evidenced > > by the frequent debates on it, and the lack of documentation. I’ve > > tried to lay out an argument that having jars in svn is not encouraged > > by the ASF (really, it’s not in line with the ASF’s charter), even if > > it isn’t disallowed. You may disagree, and I won’t claim I’ve made a > > strong argument, simply because the policy isn’t clear. So instead of > > going through arguments that amount to differences of opinion on > > Apache policy, let’s use a technological solution that is simple, > > common, and avoids the question altogether, by automatically > > downloading the dependencies at build time. > > > > Projects that use Maven do this automatic download as standard > > practice (that’s what Maven does, and that’s what the Maven Central > > infrastructure is there to support). We don’t use Maven, which is > > fine (our customers have asked us to make our binaries available in > > Maven Central, and we’ve done that). Apache Ivy is a popular add-on > > to Apache Ant that provides similar dependency resolution to an > > Ant-based build. > > > > I was a little surprised how easy it was to persuade Ivy to get the > > required dependencies at build time. The “ivy.xml” file is 39 lines > > including the ASL header (which by the way I forgot to include in the > > patch - I’ll fix that). There are about 50 lines added to ‘build.xml’ > > to download Ivy and then download the required jar files > > > > So, given that the status-quo seems to be unacceptable (Roy talks > > about not having jar files in the open-source trees, only in “-deps” > > and “tools” trees), we have two options: > > > > (a) - restructure the svn repository and the build to allow a separate > > “-deps” distribution. This wouldn’t affect our binary distributions > > (note that I’m no longer using the term “binary release”), but to > > build from source, a user would have to download a separate archive, > > unpack it, and then copy those files into the directory that was > > unpacked from the source release. This option effectively still has > > us distributing dependent binaries, which is not the goal of the ASF, > > just with a disclaimer that says “this isn’t an ASF release, its just > > a binary distribution put together by a committer for your > > convenience, so don’t feel you should trust it too much”. > > > > (b) - use Ivy to get the jars from Maven Central automatically as part > > of the build. > > > > I think (b) is the option that causes the least hassle for our > > downstream consumers, and not much hassle for us. > > > > > > > Pulling external jars at compile time also makes it more difficult > > > to certify the software. In order to certify the software you need > > > to establish baseline that will be garanteed the same, even if you > > > pull it from the archive 10 years later. > > > > As I said above, Apache’s focus is creating software that can be built > > from source, not distributing binaries (note that QCG or another > > company might have a different focus, and is perfectly able to > > distribute binaries under the Apache license). I think a reasonably > > prudent user would ask “How can I trust the ‘velocity.jar’ that’s > > included in this binary?” And the answer would be either “because I > > built it from source and installed it in my corporate repository” > > (very cautious, but not unheard-of) or “It was published by the > > Velocity group to a trusted repository, Maven Central” (more common). > > > > If you look in the “ivy.xml” file you’ll see that the dependencies are > > specified using Maven-style “group-artifact-version” coordinates. Old > > versions are maintained in Maven Central forever. I suppose it’s > > possible that a publisher could convince Maven Central to remove a > > version for some reason (security or licensing problems perhaps), but > > then, would we want to be distributing that version in a “-deps” > > package? > > > > I agree that it’s not enough to just say “you need to download > > such-and-such jar”, hence the automatic download managed by “Ivy” from > > Maven Central. > > > > > It is not a high level project that builds on several frameworks. It > > > is a lowlevel system library. The stuff below the stack is minimal. > > > The number of jars we use is limited. Why bother? > > > > > > > In the currently released branches, the dependencies are limited to > > ASM and Velocity. Looking forward to the trunk branch and the > > qa_refactor branch, the number of external dependencies seem to be > > increasing (IMO I don’t like that, because I also view River as a low > > level system library, but I’m only one PMC member). We need to get in > > front of the problem before we start distributing large numbers of > > dependencies. > > > > This point rolls in with the general question of jar files in version > > control. I was always taught that version control was for source > > code, and putting binaries into version control was a bad idea. In > > addition, there are practical problems - with older systems like cvs, > > even doing an update or commit effectively downloads the binaries, > > which slows things down if there are large binary files. On newer > > distributed version control systems like git or Mercurial, the entire > > repository, including all versions of binary artifacts, comes down > > with the project checkout. Currently, we have one version of > > relatively few jar files in our repository, so it’s not a major issue. > > But it gets worse as time goes on. So I suggest we work out the > > technology now to avoid the problem. > > > > > Gr. Simon > > > > > > > Thanks for the questions, Sim. I hope you’ll come around to removing > > your ‘-1’. > > > > Cheers, > > > > Greg > > > > Footnotes > > —————— > > > > (1) - Roy Fielding - http://s.apache.org/roy-binary-deps-1 > > (2) - Sam Ruby - http://s.apache.org/r5 > > (3) - Doug Cutting - http://s.apache.org/GNP > > > > > On 02-01-14 18:22, Greg Trasuk wrote: > > > > > > > > Hello all: > > > > > > > > Please have a look at the patch mentioned below and cast a vote on > > > > it. > > > > > > > > The main idea is to remove the dependency jar files from the > > > > source distribution. As a side effect of using Ivy, it becomes > > > > reasonable to remove them from the svn archive as well. Also, the > > > > Velocity dependency was there to support the > > > > VelocityConfigurationBuilder. We had discussed removing that > > > > component, so rather than move that dependency to Ivy, I’ve > > > > removed VelocityConfigurationBuilder. > > > > > > > > It’s arguable whether the VelocityConfigurationBuider was part of > > > > the official Jini API (I see it as a utility, not API), so I don’t > > > > think this commit actually requires a vote. However, it does seem > > > > like a significant change to the build process that ought to be > > > > reviewed. So I propose to treat this as a “lazy consensus” vote, > > > > and will commit the change to the 2.2 branch if there are no > > > > objections in 72 hours (i.e. 1730UTC 20140105). > > > > > > > > At the same time, based on discussions over on > > > > gene...@incubator.apache.org, I’ll withdraw my assertion that we > > > > can’t have jars in svn. Those interested may want to check out > > > > the thread at > > > > http://mail-archives.apache.org/mod_mbox/incubator-general/201312.mbox/%3C01B04CC4-95B8-4A39-BC16-04BAA4269B65%40stratuscom.com%3E > > > > > > > > Cheers, > > > > > > > > Greg. > > > > > > > > On Jan 2, 2014, at 12:05 PM, Greg Trasuk (JIRA) <j...@apache.org> > > > > wrote: > > > > > > > > > > > > > > [ > > > > > https://issues.apache.org/jira/browse/RIVER-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > > > > > ] > > > > > > > > > > Greg Trasuk updated RIVER-432: > > > > > ------------------------------ > > > > > > > > > > Attachment: river-2_2_remove_jars.diff > > > > > > > > > > The attached patch for the 2.2 branch does the following: > > > > > - removes the 'asm' directory and 'tests/lib' directories which > > > > > currently contain the asm library, mockito, and junit jars. - > > > > > Modifies 'build.xml', 'common.xml', and adds 'ivy.xml' so that > > > > > the Apache Ivy ant plugin is downloaded at build time, and then > > > > > used to retrieve the libraries mentioned above from Maven > > > > > Central. This removes the need to have the jar files in svn. - > > > > > Removes (as per discussion > > > > > http://mail-archives.apache.org/mod_mbox/river-dev/201211.mbox/%3C509B99E3.6080800%40qcg.nl%3E) > > > > > the VelocityConfigBuilder, and associated Velocity jars. Note > > > > > that the 'extras' folder is not present in the 2.2 branch, so > > > > > Sim's last comments in the thread do not apply. > > > > > > > > > > > Jar files in svn and src distributions > > > > > > -------------------------------------- > > > > > > > > > > > > Key: RIVER-432 > > > > > > URL: https://issues.apache.org/jira/browse/RIVER-432 > > > > > > Project: River > > > > > > Issue Type: Bug > > > > > > Reporter: Greg Trasuk > > > > > > Attachments: river-2_2_remove_jars.diff > > > > > > > > > > > > > > > > > > Recent traffic on the incubator lists has pointed out that > > > > > > including jar files for dependencies in the subversion > > > > > > repository and the source distributions is against Apache > > > > > > policy. In River, the following libraries appear in the > > > > > > Subversion repository and the source distributions (these are > > > > > > from trunk, a smaller set appear in the 2.2 branch): > > > > > > animal-sniffer asm bouncy-castle dnsjava high-scale-lib > > > > > > rc-libs > > > > > > velocity > > > > > > They all have to go. What are we using them for? As I > > > > > > understand it, we were going to remove the > > > > > > VelocityConfigurationBuilder, so that's not a problem. Some > > > > > > of the others are available from Maven Central, so we can get > > > > > > them at build time using Ivy or another build tool. Which > > > > > > ones are actually required? And where did they come from? > > > > > > > > > > > > > > > > > > > > -- > > > > > This message was sent by Atlassian JIRA > > > > > (v6.1.5#6160) > > > > > > > > > > > > > -- > > > QCG, Software voor het MKB, 071-5890970, http://www.qcg.nl > > > Quality Consultancy Group b.v., Leiderdorp, Kvk Den Haag: 28088397 > > >
