Thank you Matthias! I'm not sure where the "Downloads" section is the right place for this. I would actually put it under "Community", with a header "External Contributions" or something like this, but I'm not feeling strong about this :)
-Vasia. On 9 October 2015 at 15:29, Matthias J. Sax <mj...@apache.org> wrote: > I was not sure what we should add and was hoping for input from the > community. > > I am aware of the following projects we might want to add: > > - Zeppelin > - SAMOA > - Mahout > - Cascading (dataartisan repo) > - BigPetStore > - Gradoop > > > -Matthias > > > > On 10/09/2015 03:07 PM, Maximilian Michels wrote: > > Cool. Right now the list is empty. Do you already have a list you > > could include in the upcoming pull request? :) > > > > On Fri, Oct 9, 2015 at 2:29 PM, Matthias J. Sax <mj...@apache.org> > wrote: > >> Hi, > >> > >> I just started this. Please see > >> https://github.com/mjsax/flink-web/tree/flink-external-page > >> > >> I think, it is the best way to extend the "Downloads" page. I would also > >> add a link to this on the main page's "Getting Started" section. > >> > >> As a first try, I started like this: > >>> Third party packages > >>> > >>> This is a list of third party packages (ie, libraries, system > extensions, or examples) build for Flink. The Flink community only collects > links to those packages but does not maintain them. Thus, they do not > belong to the Apache Flink project and the community cannot give any > support for them. > >>> Package Name > >>> > >>> Available for Flink 0.8.x and 0.9.x > >>> > >>> Short description > >>> > >>> Please let us know, if we missed to list your package. Be aware, that > we might remove listed packages without notice. > >> > >> Can you please give me some input, what projects I should add initially? > >> > >> > >> -Matthias > >> > >> > >> On 10/08/2015 04:03 PM, Maximilian Michels wrote: > >>> IMHO we can do that. There should be a disclaimer that the third party > >>> software is not officially supported. > >>> > >>> On Thu, Oct 8, 2015 at 2:25 PM, Matthias J. Sax <mj...@apache.org> > wrote: > >>>> Should we add a new page at Flink project web page? > >>>> > >>>> On 10/08/2015 12:56 PM, Maximilian Michels wrote: > >>>>> +1 for your pragmatic approach, Vasia. A simple collection of third > >>>>> party software using Flink should be enough for now; of course, > >>>>> outside the Apache realm. > >>>>> > >>>>> On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park <chiwanp...@apache.org> > wrote: > >>>>>> +1 for Vasia’s suggestion. From a long-term perspective, the site > like Spark Packages [1] would be helpful to manage external contribution. > >>>>>> > >>>>>> [1] http://spark-packages.org > >>>>>> > >>>>>>> On Oct 8, 2015, at 12:28 PM, Matthias J. Sax <mj...@apache.org> > wrote: > >>>>>>> > >>>>>>> Thanks for the feedback. > >>>>>>> > >>>>>>> I think, the repository does not need to build on a single Flink > >>>>>>> release. From my point of view, there should be a single parent > module > >>>>>>> that contains *independent modules* for each extension/library > (there > >>>>>>> should be no "cross dependencies" between the modules and each > module > >>>>>>> can specify the flink dependencies it needs by itself). This make > is > >>>>>>> most flexible. And if a library works on an old release, it might > just > >>>>>>> stay there as is. If a library changes (due to Flink changes), it > might > >>>>>>> just be contained multiple times for different Flink releases. > >>>>>>> > >>>>>>> Each module should provide a short doc (README) that shows how to > use an > >>>>>>> integrate it with Flink. Thus, the responsibility goes to the > >>>>>>> contributor to maintain the library. If it breaks and is not > maintained > >>>>>>> any further, we can simple remove it. > >>>>>>> > >>>>>>> I agree, that the community might not be able to maintain those > >>>>>>> extension/libraries right now. I would put the responsibility > (more or > >>>>>>> less completely) on the contributor and delete project that do not > fix > >>>>>>> any more. > >>>>>>> > >>>>>>> @Vasia: a link to a library could be included in the README. If > anybody > >>>>>>> only wants to share a library but not contribute code, the parent > README > >>>>>>> could contain a list of additional links. > >>>>>>> > >>>>>>> > >>>>>>> -Matthias > >>>>>>> > >>>>>>> > >>>>>>> On 10/08/2015 12:15 PM, Vasiliki Kalavri wrote: > >>>>>>>> How about, for now, we simply create a page where we gather > links/short > >>>>>>>> descriptions of all these contributions > >>>>>>>> and let the maintenance and dependency management to the > tool/library > >>>>>>>> creators? > >>>>>>>> This way we will at least have these contributions in one place > and link to > >>>>>>>> them somewhere from the website. > >>>>>>>> > >>>>>>>> -Vasia. > >>>>>>>> > >>>>>>>> On 8 October 2015 at 12:06, Maximilian Michels <m...@apache.org> > wrote: > >>>>>>>> > >>>>>>>>> Hi Matthias, > >>>>>>>>> > >>>>>>>>> Thanks for bringing up this idea. Actually, it has been > discussed a > >>>>>>>>> couple of times on the mailing list whether we should have a > central > >>>>>>>>> place for third-party extensions/contributions/libraries. This > could > >>>>>>>>> either be something package-based or, like you proposed, another > >>>>>>>>> repository. > >>>>>>>>> > >>>>>>>>> An external place for contributions raises a couple of questions > >>>>>>>>> > >>>>>>>>> - Which version should the external contributions be based on? > >>>>>>>>> - How do we make sure, the extensions are continuously updated? > >>>>>>>>> (dedicated maintainers or automatic compatibility checks) > >>>>>>>>> - How do we easily plug-in the external modules into Flink? > >>>>>>>>> > >>>>>>>>> In the long term, we really need a solution for these questions. > The > >>>>>>>>> code base of Flink is growing and more and more packages go to > >>>>>>>>> flink-contrib/flink-staging. I would find something > packaged-based > >>>>>>>>> better than a repository. Quite frankly, momentarily, I think > >>>>>>>>> developing such a plugin system is out of scope for most Flink > >>>>>>>>> developers. At the current pace of Flink development, collecting > these > >>>>>>>>> contributions externally without properly maintaining them, > doesn't > >>>>>>>>> make much sense to me. > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> Max > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax < > mj...@apache.org> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> many people are building quite exiting stuff on top of Flink. > It is hard > >>>>>>>>>> to keep an good overview on what stuff is available and what > not. What > >>>>>>>>>> do you think about starting a second git repository > "flink-external" > >>>>>>>>>> that collects all those code? > >>>>>>>>>> > >>>>>>>>>> The ideas would be to collect stuff in a central point, such > that people > >>>>>>>>>> can access it easily and get an overview what is already > available (this > >>>>>>>>>> might also avoid duplicate development). It might also be a > good point > >>>>>>>>>> to show common patterns. In order to collect as much as > possible, the > >>>>>>>>>> contributing requirement (with respect to testing etc) could be > lower > >>>>>>>>>> than for Flink itself. > >>>>>>>>>> > >>>>>>>>>> For example, I recently started a small flink-clojure module > with a > >>>>>>>>>> simple word-count example to answer a question on SO. Including > this in > >>>>>>>>>> Flink would not be appropriate. However, for a flink-external > repro it > >>>>>>>>>> might be nice to have. > >>>>>>>>>> > >>>>>>>>>> What do you think about it? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -Matthias > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> Regards, > >>>>>> Chiwan Park > >>>>>> > >>>>>> > >>>>>> > >>>> > >> > >
