I think I found a real release blocker. Currently we don't add license files to our shaded jars. For example the flink-shaded-include-yarn-0.9.0-milestone-1.jar shades hadoop code. This code also includes the `org.apache.util.bloom.*` classes. These classes are licensed under The European Commission project OneLab. We have a notice in the LICENSE file of our binary distribution but I think we also have to add them in the shaded jar. There might even be more code bundled as part of some shaded jars which I have not spotted yet.
Furthermore, I noticed that we list all Apache License dependencies in our LICENSE file of our binary distribution (which we don't have to do). However, we don't do it in our jars which contain for example guava and asm as shaded dependencies. Maybe we should be consistent here. But maybe I overlook something here and we don't have to do it. On Fri, Jun 12, 2015 at 10:29 AM Till Rohrmann <trohrm...@apache.org> wrote: > Well I think the initial idea was to keep the dist jar as small a possible > and therefore we did not include the libraries. I'm not sure whether we can > decide this here ad-hoc. If the community says that we shall include these > libraries then I can add them. But bear in mind that all of them have some > transitive dependencies which will be added as well. > > > On Fri, Jun 12, 2015 at 10:15 AM Márton Balassi <balassi.mar...@gmail.com> > wrote: > >> As for outstanding issues I think streaming is good to go as far as I >> know. >> I am personally against including all libraries - at least speaking for >> the >> streaming connectors. Robert, Stephan and myself had a detailed discussion >> on that some time ago and the disadvantage of having all the libraries in >> the distribution is the dependency mess that they pull. In this case I >> would rather add documentation on putting them in the user jar then. As >> for >> the other libraries they do not depend on so much external code, so +1 for >> putting them in. >> >> On Fri, Jun 12, 2015 at 10:02 AM, Maximilian Michels <m...@apache.org> >> wrote: >> >> > We should have a nightly cluster test for every library. Let's keep >> that in >> > mind for the future. Very nice find, Till! >> > >> > Since there were not objections, I cherry-picked the proposed commits >> from >> > the document to the release-0.9 branch. If I understand correctly, we >> can >> > create the new release candidate once Till has checked the licenses, >> Ufuk's >> > TableInput fix has been merged, and Fabian's web interface improvement >> are >> > in. Plus, we need to include all Flink libraries in flink-dist. Are you >> > going to fix that as well, Till? >> > >> > On Fri, Jun 12, 2015 at 9:53 AM, Ufuk Celebi <u...@apache.org> wrote: >> > >> > > >> > > On 12 Jun 2015, at 09:45, Till Rohrmann <trohrm...@apache.org> wrote: >> > > >> > > > Hi guys, >> > > > >> > > > I just noticed while testing the TableAPI on the cluster that it is >> not >> > > > part of the dist module. Therefore, programs using the TableAPI will >> > only >> > > > run when you put the TableAPI jar directly on the cluster or if you >> > > build a >> > > > fat jar including the TableAPI jar. This is nowhere documented. >> > > > Furthermore, this also applies to Gelly and FlinkML. >> > > >> > > I think all of these should be included in the fat jar. They are all >> > > highly advertized components. >> > > >> > > Very good catch, Till! I didn't get around to testing Table API on a >> > > cluster, yet. >> > >> >
