I think it makes sense to keep the stateful functions code in a separate repository in the beginning as described. At a later point in time we could revisit this topic if we see that the split codebase becomes a problem or if there are other benefits such as better visibility.
For the website, we could keep them separate but put a prominent link from the Flink documentation to the stateful functions documentation. Cheers, Till On Wed, Oct 16, 2019 at 10:02 AM Timo Walther <twal...@apache.org> wrote: > Hi Stephan, > > +1 for keeping it in a separate repository for fast release cycles and > stability until it is mature enough. But we should definitely merge it > back to the core repo also for marketing reasons. > > IMHO side projects tend to be overlooked by the outside world even > though they are great technology. > > Would we still document the code in our main documentation or on a > separate website? > > Thanks, > Timo > > > On 16.10.19 09:15, Aljoscha Krettek wrote: > > I would keep statefun in a separate repo in the beginning, for the > reasons you mentioned. > > > > Best, > > Aljoscha > > > >> On 15. Oct 2019, at 23:40, Flavio Pompermaier <pomperma...@okkam.it> > wrote: > >> > >> Definitely on the same page..+1 to keep it in a separate repo (at least > >> until the cose becomes "stable" and widely adopted from the community) > >> > >> Il Mar 15 Ott 2019, 23:17 Stephan Ewen <se...@apache.org> ha scritto: > >> > >>> Hi Flink folks! > >>> > >>> After the positive reaction to the contribution proposal for Stateful > >>> Functions, I would like to kick off the discussion for the big > question: In > >>> which form should it go into Flink? > >>> > >>> Before jumping into the "repository" question directly, let's get some > >>> clarity on what would be our high-level goal with this project and the > >>> contribution. > >>> My thinking so far was: > >>> > >>> - Stateful Functions is a way for Flink and stream processing to > become > >>> applicable for more general application development. That is a chance > to > >>> grow our community to a new crowd of developers. > >>> > >>> - While adding this to Flink gives synergies with the runtime it > build on > >>> top of, it makes sense to offer the new developers a lightweight way > to get > >>> involved. Simple setup, easy contributions. > >>> > >>> - This is a new project, the API and many designs are not frozen at > this > >>> point and may still change heavily. > >>> To become really good, the project needs to still make a bunch of > >>> iterations (no pun intended) and change many things quickly. > >>> > >>> - The Stateful Functions project will likely try to release very > >>> frequently in its early days, to improve quickly and gather feedback > fast. > >>> Being bound to Flink core release cycle would hurt here. > >>> > >>> > >>> I believe that with all those goals, adding Stateful Functions to the > Flink > >>> core repository would not make sense. Flink core has processes that > make > >>> sense for an established project that needs to guarantee stability. > These > >>> processes are simply prohibitive for new projects to develop. > >>> In addition, the Flink main repository is gigantic, has a build system > and > >>> CI system that cannot handle the size of the project any more. Not the > best > >>> way to start expanding into a new community. > >>> > >>> In some sense, Stateful Functions could make sense as an independent > >>> project, but it is so tightly coupled to Flink right now that I think > an > >>> even better fit is a separate repository in Flink. > >>> Think Hive and Hadoop in the early days. That way, we get the synergy > >>> between the two (the same community drives them) while letting both > move at > >>> their own speed. > >>> It would somehow mean two closely related projects shepherded by the > same > >>> community. > >>> > >>> It might be possible at a later stage to either merge this into Flink > core > >>> (once Stateful Functions is more settled) or even spin this out as a > >>> standalone Apache project, if that is how the community develops. > >>> > >>> That is my main motivation. It is not driven primarily by > technicalities > >>> like code versioning and dependencies, but much rather by what is the > best > >>> setup to develop this as Flink's way to expand its community towards > new > >>> users from a different background. > >>> > >>> Curious to hear if that makes sense to you. > >>> > >>> Best, > >>> Stephan > >>> > >