Thanks Matthias! I made a comment. Please open a pull request.
On Tue, Oct 27, 2015 at 10:37 PM, Matthias J. Sax <mj...@apache.org> wrote: > Just updated this. Improved the layout and added FastR project. > > https://github.com/mjsax/flink-web/tree/flink-external-page > > -Matthias > > On 10/27/2015 03:25 AM, Matthias J. Sax wrote: >> Hi, >> >> I updated the flink-external section on the Flink Web-Page: >> https://github.com/mjsax/flink-web/tree/flink-external-page >> >> The section is now located in "Contribute" page. The layout needs some >> refinement though... Some Project are "previews", ie, Flink support was >> announced but there is not information on the according project web >> pages. We might want to reach out to those people to see if we should >> include those project already or just add them later on. >> >> Please give feedback. >> >> >> -Matthias >> >> >> >> >> On 10/09/2015 03:34 PM, Maximilian Michels wrote: >>> Yes, Community is a better place. You can also add the Dataflow Runner >>> https://github.com/dataArtisans/flink-dataflow. >>> >>> On Fri, Oct 9, 2015 at 3:32 PM, Vasiliki Kalavri >>> <vasilikikala...@gmail.com> wrote: >>>> Thank you Matthias! >>>> >>>> I'm not sure where the "Downloads" section is the right place for this. >>>> I would actually put it under "Community", with a header "External >>>> Contributions" or something like this, but I'm not feeling strong about >>>> this :) >>>> >>>> -Vasia. >>>> >>>> >>>> On 9 October 2015 at 15:29, Matthias J. Sax <mj...@apache.org> wrote: >>>> >>>>> I was not sure what we should add and was hoping for input from the >>>>> community. >>>>> >>>>> I am aware of the following projects we might want to add: >>>>> >>>>> - Zeppelin >>>>> - SAMOA >>>>> - Mahout >>>>> - Cascading (dataartisan repo) >>>>> - BigPetStore >>>>> - Gradoop >>>>> >>>>> >>>>> -Matthias >>>>> >>>>> >>>>> >>>>> On 10/09/2015 03:07 PM, Maximilian Michels wrote: >>>>>> Cool. Right now the list is empty. Do you already have a list you >>>>>> could include in the upcoming pull request? :) >>>>>> >>>>>> On Fri, Oct 9, 2015 at 2:29 PM, Matthias J. Sax <mj...@apache.org> >>>>> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I just started this. Please see >>>>>>> https://github.com/mjsax/flink-web/tree/flink-external-page >>>>>>> >>>>>>> I think, it is the best way to extend the "Downloads" page. I would also >>>>>>> add a link to this on the main page's "Getting Started" section. >>>>>>> >>>>>>> As a first try, I started like this: >>>>>>>> Third party packages >>>>>>>> >>>>>>>> This is a list of third party packages (ie, libraries, system >>>>> extensions, or examples) build for Flink. The Flink community only >>>>> collects >>>>> links to those packages but does not maintain them. Thus, they do not >>>>> belong to the Apache Flink project and the community cannot give any >>>>> support for them. >>>>>>>> Package Name >>>>>>>> >>>>>>>> Available for Flink 0.8.x and 0.9.x >>>>>>>> >>>>>>>> Short description >>>>>>>> >>>>>>>> Please let us know, if we missed to list your package. Be aware, that >>>>> we might remove listed packages without notice. >>>>>>> >>>>>>> Can you please give me some input, what projects I should add initially? >>>>>>> >>>>>>> >>>>>>> -Matthias >>>>>>> >>>>>>> >>>>>>> On 10/08/2015 04:03 PM, Maximilian Michels wrote: >>>>>>>> IMHO we can do that. There should be a disclaimer that the third party >>>>>>>> software is not officially supported. >>>>>>>> >>>>>>>> On Thu, Oct 8, 2015 at 2:25 PM, Matthias J. Sax <mj...@apache.org> >>>>> wrote: >>>>>>>>> Should we add a new page at Flink project web page? >>>>>>>>> >>>>>>>>> On 10/08/2015 12:56 PM, Maximilian Michels wrote: >>>>>>>>>> +1 for your pragmatic approach, Vasia. A simple collection of third >>>>>>>>>> party software using Flink should be enough for now; of course, >>>>>>>>>> outside the Apache realm. >>>>>>>>>> >>>>>>>>>> On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park <chiwanp...@apache.org> >>>>> wrote: >>>>>>>>>>> +1 for Vasia’s suggestion. From a long-term perspective, the site >>>>> like Spark Packages [1] would be helpful to manage external contribution. >>>>>>>>>>> >>>>>>>>>>> [1] http://spark-packages.org >>>>>>>>>>> >>>>>>>>>>>> On Oct 8, 2015, at 12:28 PM, Matthias J. Sax <mj...@apache.org> >>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Thanks for the feedback. >>>>>>>>>>>> >>>>>>>>>>>> I think, the repository does not need to build on a single Flink >>>>>>>>>>>> release. From my point of view, there should be a single parent >>>>> module >>>>>>>>>>>> that contains *independent modules* for each extension/library >>>>> (there >>>>>>>>>>>> should be no "cross dependencies" between the modules and each >>>>> module >>>>>>>>>>>> can specify the flink dependencies it needs by itself). This make >>>>> is >>>>>>>>>>>> most flexible. And if a library works on an old release, it might >>>>> just >>>>>>>>>>>> stay there as is. If a library changes (due to Flink changes), it >>>>> might >>>>>>>>>>>> just be contained multiple times for different Flink releases. >>>>>>>>>>>> >>>>>>>>>>>> Each module should provide a short doc (README) that shows how to >>>>> use an >>>>>>>>>>>> integrate it with Flink. Thus, the responsibility goes to the >>>>>>>>>>>> contributor to maintain the library. If it breaks and is not >>>>> maintained >>>>>>>>>>>> any further, we can simple remove it. >>>>>>>>>>>> >>>>>>>>>>>> I agree, that the community might not be able to maintain those >>>>>>>>>>>> extension/libraries right now. I would put the responsibility >>>>> (more or >>>>>>>>>>>> less completely) on the contributor and delete project that do not >>>>> fix >>>>>>>>>>>> any more. >>>>>>>>>>>> >>>>>>>>>>>> @Vasia: a link to a library could be included in the README. If >>>>> anybody >>>>>>>>>>>> only wants to share a library but not contribute code, the parent >>>>> README >>>>>>>>>>>> could contain a list of additional links. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -Matthias >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 10/08/2015 12:15 PM, Vasiliki Kalavri wrote: >>>>>>>>>>>>> How about, for now, we simply create a page where we gather >>>>> links/short >>>>>>>>>>>>> descriptions of all these contributions >>>>>>>>>>>>> and let the maintenance and dependency management to the >>>>> tool/library >>>>>>>>>>>>> creators? >>>>>>>>>>>>> This way we will at least have these contributions in one place >>>>> and link to >>>>>>>>>>>>> them somewhere from the website. >>>>>>>>>>>>> >>>>>>>>>>>>> -Vasia. >>>>>>>>>>>>> >>>>>>>>>>>>> On 8 October 2015 at 12:06, Maximilian Michels <m...@apache.org> >>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Matthias, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for bringing up this idea. Actually, it has been >>>>> discussed a >>>>>>>>>>>>>> couple of times on the mailing list whether we should have a >>>>> central >>>>>>>>>>>>>> place for third-party extensions/contributions/libraries. This >>>>> could >>>>>>>>>>>>>> either be something package-based or, like you proposed, another >>>>>>>>>>>>>> repository. >>>>>>>>>>>>>> >>>>>>>>>>>>>> An external place for contributions raises a couple of questions >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Which version should the external contributions be based on? >>>>>>>>>>>>>> - How do we make sure, the extensions are continuously updated? >>>>>>>>>>>>>> (dedicated maintainers or automatic compatibility checks) >>>>>>>>>>>>>> - How do we easily plug-in the external modules into Flink? >>>>>>>>>>>>>> >>>>>>>>>>>>>> In the long term, we really need a solution for these questions. >>>>> The >>>>>>>>>>>>>> code base of Flink is growing and more and more packages go to >>>>>>>>>>>>>> flink-contrib/flink-staging. I would find something >>>>> packaged-based >>>>>>>>>>>>>> better than a repository. Quite frankly, momentarily, I think >>>>>>>>>>>>>> developing such a plugin system is out of scope for most Flink >>>>>>>>>>>>>> developers. At the current pace of Flink development, collecting >>>>> these >>>>>>>>>>>>>> contributions externally without properly maintaining them, >>>>> doesn't >>>>>>>>>>>>>> make much sense to me. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>> Max >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax < >>>>> mj...@apache.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> many people are building quite exiting stuff on top of Flink. >>>>> It is hard >>>>>>>>>>>>>>> to keep an good overview on what stuff is available and what >>>>> not. What >>>>>>>>>>>>>>> do you think about starting a second git repository >>>>> "flink-external" >>>>>>>>>>>>>>> that collects all those code? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The ideas would be to collect stuff in a central point, such >>>>> that people >>>>>>>>>>>>>>> can access it easily and get an overview what is already >>>>> available (this >>>>>>>>>>>>>>> might also avoid duplicate development). It might also be a >>>>> good point >>>>>>>>>>>>>>> to show common patterns. In order to collect as much as >>>>> possible, the >>>>>>>>>>>>>>> contributing requirement (with respect to testing etc) could be >>>>> lower >>>>>>>>>>>>>>> than for Flink itself. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> For example, I recently started a small flink-clojure module >>>>> with a >>>>>>>>>>>>>>> simple word-count example to answer a question on SO. Including >>>>> this in >>>>>>>>>>>>>>> Flink would not be appropriate. However, for a flink-external >>>>> repro it >>>>>>>>>>>>>>> might be nice to have. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What do you think about it? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -Matthias >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Chiwan Park >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>>> >> >
