Great initiative! I think mailing list discussions can be very hard to follow in retrospective. The Kafka community does a great job in summing up the discussions on individual wiki pages. We should do that as well!
On Tue, Jul 12, 2016 at 9:55 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > I'll make the changes to the list of things that we consider public "API" > and also add these to the FLIP Template. Then we should be good to go. I'll > convert the design docs I started to FLIPs in the next few days and will > also ask other people that have authored design docs to move them to FLIPs. > > On Fri, 8 Jul 2016 at 17:44 Aljoscha Krettek <aljos...@apache.org> wrote: > >> I was planning to ask people to do that once the process is in place, yes. >> >> Wouldn't it be "forward porting", though? :-) >> >> On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rmetz...@apache.org> wrote: >> >>> What about "backporting" some of the currently ongoing FLIPs to the new >>> process? >>> In particular those which are not yet fully implemented (Security, Mesos, >>> Key Groups, ...) ? >>> >>> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <aljos...@apache.org> >>> wrote: >>> >>> > Of course, this would make the list look like this: >>> > >>> > All of the following are public interfaces that people build around: >>> > - DataStream and DataSet API, including classes related to that, such >>> as >>> > StreamExecutionEnvironment >>> > - Classes marked with the @Public annotation >>> > - On-disk binary formats, such as checkpoints/savepoints >>> > - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts >>> > - Configuration >>> > - Monitoring >>> > >>> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <u...@apache.org> wrote: >>> > >>> > > What about the scripts? >>> > > >>> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljos...@apache.org >>> > >>> > > wrote: >>> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on >>> > the >>> > > > template you copied over. >>> > > > >>> > > > I propose to change the list of public interfaces to this: >>> > > > >>> > > > All of the following are public interfaces that people build around: >>> > > > - DataStream and DataSet API, including classes related to that, >>> such >>> > as >>> > > > StreamExecutionEnvironment >>> > > > - Classes marked with the @Public annotation >>> > > > - On-disk binary formats, such as checkpoints/savepoints >>> > > > - Configuration >>> > > > - Monitoring >>> > > > >>> > > > Cheers, >>> > > > Aljoscha >>> > > > >>> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org> >>> wrote: >>> > > > >>> > > >> Done. >>> > > >> >>> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some >>> things >>> > > >> we need to rework. Please have a look and propose changes. >>> > > >> >>> > > >> >>> > > >> >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals >>> > > >> >>> > > >> Same for FLIP Template >>> > > >> >>> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template >>> > > >> >>> > > >> >>> > > >> -Matthias >>> > > >> >>> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote: >>> > > >> > Hi, >>> > > >> > I got this reply from one of the Kafka committers: >>> > > >> > >>> > > >> > Thanks for sharing your intention to use a process similar to our >>> > KIP >>> > > >> > >>> > > >> > process. You are more than welcome to copy the structures and >>> docs >>> > > that >>> > > >> we >>> > > >> > >>> > > >> > have for the KIP process. :) >>> > > >> > >>> > > >> > Ismael >>> > > >> > >>> > > >> > >>> > > >> > So it seems we're good to go. @Matthias, since you are a Kafka >>> > > >> contributor, >>> > > >> > could you maybe copy the relevant docs from the Kafka wiki to our >>> > > wiki? >>> > > >> The >>> > > >> > source for this page: >>> > > >> > >>> > > >> >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals >>> > > >> > and >>> > > >> > the KIP template that is referenced there. If you want, I can >>> take >>> > it >>> > > >> from >>> > > >> > there and adapt it to Flink and then let everyone discuss it >>> again. >>> > > >> > >>> > > >> > >>> > > >> > Cheers, >>> > > >> > >>> > > >> > Aljoscha >>> > > >> > >>> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek < >>> aljos...@apache.org> >>> > > >> wrote: >>> > > >> > >>> > > >> >> I'll reach out to the Kafka community and ask if it's ok for us >>> to >>> > > >> "steal" >>> > > >> >> their process for this. >>> > > >> >> >>> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek < >>> aljos...@apache.org> >>> > > >> wrote: >>> > > >> >> >>> > > >> >>> @Matthias: Yes, this is the reason why I like the KIP process >>> and >>> > > why I >>> > > >> >>> said "The problem with these is that a) the comments on the >>> Google >>> > > >> Docs are >>> > > >> >>> not reflected in Jira and the mailing list. There has been some >>> > very >>> > > >> active >>> > > >> >>> discussion on some of the docs that most people would never >>> > > notice.". >>> > > >> >>> >>> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger < >>> rmetz...@apache.org> >>> > > >> wrote: >>> > > >> >>> >>> > > >> >>>> I also like the proposal. I think its an issue that Google >>> Docs >>> > > >> comments >>> > > >> >>>> are not reflected within ASF infra. Therefore, I'm +1 on >>> > discussing >>> > > >> the >>> > > >> >>>> proposals on the mailing list. >>> > > >> >>>> >>> > > >> >>>> I agree that we need to clean up our wiki. >>> > > >> >>>> >>> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax < >>> > mj...@apache.org >>> > > > >>> > > >> >>>> wrote: >>> > > >> >>>> >>> > > >> >>>>> Just to point out one thing about Kafka KIPs and using the >>> > project >>> > > >> >>>> wiki: >>> > > >> >>>>> >>> > > >> >>>>> The wiki contains the current state of the proposal, while >>> the >>> > > >> >>>>> discussion is covered over the dev-mailing list. IMHO, this >>> > makes >>> > > a >>> > > >> lot >>> > > >> >>>>> of sense, as people tend to follow the mailing list but not >>> wiki >>> > > >> >>>>> changes. Furthermore, the mailing list tracks the whole >>> > discussion >>> > > >> >>>>> history, while the proposal is kept in a clean state and thus >>> > > easy to >>> > > >> >>>> read. >>> > > >> >>>>> >>> > > >> >>>>> -Matthias >>> > > >> >>>>> >>> > > >> >>>>> >>> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote: >>> > > >> >>>>>> Jip, that's why I referenced the Kafka process which is >>> also in >>> > > >> their >>> > > >> >>>>> wiki. >>> > > >> >>>>>> >>> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> >>> > > wrote: >>> > > >> >>>>>> >>> > > >> >>>>>>> Yes, big +1 >>> > > >> >>>>>>> >>> > > >> >>>>>>> I had actually talked about the same thing with some >>> people as >>> > > >> well. >>> > > >> >>>>>>> >>> > > >> >>>>>>> I am currently sketching a few FLIPs for things, like >>> > > improvements >>> > > >> >>>> to >>> > > >> >>>>> the >>> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration >>> > > >> >>>>>>> >>> > > >> >>>>>>> >>> > > >> >>>>>>> One thing we should do here is to actually structure the >>> wiki >>> > a >>> > > bit >>> > > >> >>>> to >>> > > >> >>>>> make >>> > > >> >>>>>>> it easier to find information and proposals. >>> > > >> >>>>>>> >>> > > >> >>>>>>> >>> > > >> >>>>>>> >>> > > >> >>>>>>> >>> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi < >>> u...@apache.org> >>> > > >> wrote: >>> > > >> >>>>>>> >>> > > >> >>>>>>>> Hey Aljoscha, >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> thanks for this proposal. I've somehow missed it last >>> week. I >>> > > like >>> > > >> >>>> the >>> > > >> >>>>>>>> idea very much and agree with your assessment about the >>> > > problems >>> > > >> >>>> with >>> > > >> >>>>>>>> the Google Doc approach. >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> Regarding the process: I'm also in favour of adopting it >>> from >>> > > >> >>>> Kafka. I >>> > > >> >>>>>>>> would not expect any problems with this, but we can post a >>> > > quick >>> > > >> >>>> note >>> > > >> >>>>>>>> to their ML. >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> @Matthias: The name works for me. ;-) >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> – Ufuk >>> > > >> >>>>>>>> >>> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax < >>> > > >> >>>> mj...@apache.org> >>> > > >> >>>>>>>> wrote: >>> > > >> >>>>>>>>> FLIP ?? Really? :D >>> > > >> >>>>>>>>> >>> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip >>> > > >> >>>>>>>>> >>> > > >> >>>>>>>>> -Matthias >>> > > >> >>>>>>>>> >>> > > >> >>>>>>>>> >>> > > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote: >>> > > >> >>>>>>>>>> I'm proposing to add a formal process for how we deal >>> with >>> > > >> >>>> (major) >>> > > >> >>>>>>>>>> improvements to Flink and design docs. This has been >>> > > mentioned >>> > > >> >>>>> several >>> > > >> >>>>>>>>>> times recently but we never took any decisive action to >>> > > actually >>> > > >> >>>>>>>> implement >>> > > >> >>>>>>>>>> such a process so here we go. >>> > > >> >>>>>>>>>> >>> > > >> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have >>> > > design >>> > > >> >>>> docs >>> > > >> >>>>>>>> that we >>> > > >> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those >>> > > that he >>> > > >> >>>>> could >>> > > >> >>>>>>>> find >>> > > >> >>>>>>>>>> on the mailing list to the Flink wiki: >>> > > >> >>>>>>>>>> >>> > > >> >>>> >>> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home. >>> > > >> >>>>>>>> The >>> > > >> >>>>>>>>>> problem with these is that a) the comments on the Google >>> > Docs >>> > > >> >>>> are not >>> > > >> >>>>>>>>>> reflected in Jira and the mailing list. There has been >>> some >>> > > very >>> > > >> >>>>>>> active >>> > > >> >>>>>>>>>> discussion on some of the docs that most people would >>> never >>> > > >> >>>> notice. >>> > > >> >>>>>>> The >>> > > >> >>>>>>>>>> community therefore might seem less active than it >>> actually >>> > > is. >>> > > >> >>>> b) >>> > > >> >>>>> the >>> > > >> >>>>>>>>>> documents are not very discoverable, if we had a clearly >>> > > defined >>> > > >> >>>>> place >>> > > >> >>>>>>>>>> where we put them and also prominently link to this on >>> the >>> > > Flink >>> > > >> >>>>>>>> homepage >>> > > >> >>>>>>>>>> this would greatly help people that try to find out >>> about >>> > > >> current >>> > > >> >>>>>>>>>> developments. >>> > > >> >>>>>>>>>> >>> > > >> >>>>>>>>>> Kafka has a process like this: >>> > > >> >>>>>>>>>> >>> > > >> >>>>>>>> >>> > > >> >>>>>>> >>> > > >> >>>>> >>> > > >> >>>> >>> > > >> >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals >>> > > >> >>>>>>>> . >>> > > >> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We >>> could >>> > > >> either >>> > > >> >>>>>>> adapt >>> > > >> >>>>>>>>>> this for Flink or come up with our own process. Doing >>> the >>> > > former >>> > > >> >>>>> would >>> > > >> >>>>>>>> save >>> > > >> >>>>>>>>>> us a lot of time and I don't think the Kafka community >>> > would >>> > > >> >>>> mind us >>> > > >> >>>>>>>>>> copying their process. The subject also hints at this, >>> our >>> > > >> >>>> process >>> > > >> >>>>>>>> could be >>> > > >> >>>>>>>>>> called FLIP, for Flink Improvement Proposal. >>> > > >> >>>>>>>>>> >>> > > >> >>>>>>>>>> What do you think? Feedback is highly welcome. :-) >>> > > >> >>>>>>>>>> >>> > > >> >>>>>>>>>> Cheers, >>> > > >> >>>>>>>>>> Aljoscha >>> > > >> >>>>>>>>>> >>> > > >> >>>>>>>>> >>> > > >> >>>>>>>> >>> > > >> >>>>>>> >>> > > >> >>>>>> >>> > > >> >>>>> >>> > > >> >>>>> >>> > > >> >>>> >>> > > >> >>> >>> > > >> > >>> > > >> >>> > > >> >>> > > >>> > >>> >>
