Guozhang's idea is interesting. It does promote collaboration. Not all KIPs are created equally and anyone (including the person reviewing it) would be able to call out and request a stronger vote for it to go in.
I think that having 3 x +1 is good (sometimes) because it reduces the risk of things getting lost in the shuffle and missing changes, which does, will and always happen. We also need to discuss 0.8.3.0, 0.9.0.0 and such https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan it should all be KIP we think should be in where, right? - Joe Stein On Fri, Feb 6, 2015 at 2:59 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Here are my cents: > > We could use "lazy" as the default policy for KIP, i.e. : > > 1. Once a KIP is created at least one committer should engage and provide > feedbacks within a few days; > 2. If the committer(s) think it may be a bigger issue / change and requires > broader discussion, she(they) may call out to upgrade to "majority" or even > "consensus" and urge other committers to get involved. > > Hence it is committers' responsibility to guarantee step 1). > > Guozhang > > On Thu, Feb 5, 2015 at 11:24 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > Just wanted to add a few more comments on this: KIPs were suggested as > > a process to help reach early consensus on a major change or not so > > major (but tricky or backward incompatible) change in order to reduce > > the likelihood of multiple iterations and complete rewrites during > > code reviews (which is time-intensive for both the contributor and > > reviewers); as well as to reduce the likelihood of surprises (say, if > > a patch inadvertently changes a public API). So KIPs are intended to > > speed up development since a clear path is charted out and there is > > prior consensus on whether a feature and its design/implementation > > make sense or not. > > > > Obviously this breaks down if KIPs are not being actively discussed - > > again I think we can do much better here. I think we ended up with a > > backlog because as soon as the KIP wiki was started, a number of > > pre-existing jiras and discussions were moved there - all within a few > > days. Now that there are quite a few outstanding KIPs I think we just > > need to methodically work through those - preferably a couple at a > > time. I looked through the list and I think we should be able to > > resolve all of them relatively quickly if everyone is on board with > > this. > > > > > > Its probably more helpful for contributors if its "lazy" as in "no > > > > strong objections" . > > > > Gwen also suggested this and this also sounds ok to me as I wrote > > earlier - what do others think? This is important especially if > > majority in the community think if this less restrictive policy would > > spur and not hinder development - I'm not sure that it does. I > > completely agree that KIPs fail to a large degree as far as the > > original motivation goes if they require a lazy majority but the > > DISCUSS threads are stalled. IOW regardless of that discussion, I > > think we should rejuvenate some of those threads especially now that > > 0.8.2 is out of the way. > > > > Thanks, > > > > Joel > > > > On Thu, Feb 05, 2015 at 08:56:13PM -0800, Joel Koshy wrote: > > > I'm just thinking aloud - I don't know what a good number would be, and > > it > > > is just one possibility to streamline how KIPs are processed. It > largely > > > depends on how complex the proposals are. What would be concerning is > if > > > there are 10 different threads all dealing with large KIPs and no one > has > > > the time to give due diligence to each one and all those threads grind > > to a > > > halt due to confusion, incomplete context and misunderstandings. > > > > > > On Thursday, February 5, 2015, Harsha <ka...@harsha.io> wrote: > > > > > > > Joel, > > > > Having only 2 or 3 KIPS under active discussion is concerning. > > > > This will slow down development process as well. > > > > Having a turn-around time for a KIP is a good idea but what will > happen > > > > if it didn't received required votes within that time frame. > > > > Its probably more helpful for contributors if its "lazy" as in "no > > > > strong objections" . > > > > Just to make sure this is only for KIPs not for regular bug fixes > > right? > > > > Thanks, > > > > Harsha > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Feb 5, 2015, at 05:59 PM, Jiangjie Qin wrote: > > > > > Iąm having an impression that KIP is mostly for new features but > not > > for > > > > > bug fixes. But I agree with Joel that it might make sense to have > > some > > > > > big > > > > > patches, even if they are bug fixes, to follow the KIP like process > > which > > > > > is more strict. > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > On 2/5/15, 4:57 PM, "Gwen Shapira" <gshap...@cloudera.com > > <javascript:;>> > > > > wrote: > > > > > > > > > > >> > > > > > >> > > > > > >> Yes there are KIPs that are currently blocked on feedback/votes, > > but I > > > > > >> don't think it is an issue of not caring to comment vs having so > > many > > > > > >> KIPs and other code reviews in flight that people are just > > swamped. > > > > > >> > > > > > >> > > > > > >This is exactly my concern. > > > > > >Even now important patches and features have very long development > > and > > > > > >review cycles due to Kafka's small and very busy committer > > community. I > > > > > >feel that this change takes things in the wrong direction > > > > > > > > > > > > > > > > > >> Joel > > > > > >> > > > > > >> On Thu, Feb 05, 2015 at 04:19:54PM -0800, Gwen Shapira wrote: > > > > > >> > Isn't requiring 3 binding votes a bit overly strict here? We > are > > > > > >>talking > > > > > >> > about patches which in can be fixed, reverted, etc. Not > > releases, > > > > > >>which > > > > > >> > have legal implications. > > > > > >> > > > > > > >> > Why not go with usual definition: "lazy" = "No strong > > objections for > > > > > >>few > > > > > >> > days"? > > > > > >> > This means contributors will not be blocked on issues where no > > one > > > > > >>cares > > > > > >> to > > > > > >> > comment (and we had few of those). > > > > > >> > > > > > > >> > Gwen > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > On Thu, Feb 5, 2015 at 4:14 PM, Joel Koshy < > jjkosh...@gmail.com > > > > <javascript:;>> > > > > > >>wrote: > > > > > >> > > > > > > >> > > Sorry about this - I actually meant to suggest lazy > consensus > > > > (which > > > > > >> > > is a stronger requirement): "3 binding +1 votes and no > binding > > > > > >> > > vetoes." > > > > > >> > > > > > > > >> > > I have updated the wiki to lazy consensus - but can change > it > > back > > > > > >>if > > > > > >> > > there is a reasonable objection. > > > > > >> > > > > > > > >> > > On Thu, Feb 05, 2015 at 06:17:44PM -0500, Joe Stein wrote: > > > > > >> > > > +1 > > > > > >> > > > > > > > > >> > > > On Thu, Feb 5, 2015 at 6:16 PM, Neha Narkhede < > > > > n...@confluent.io <javascript:;>> > > > > > >> wrote: > > > > > >> > > > > > > > > >> > > > > Sounds good. > > > > > >> > > > > > > > > > >> > > > > On Thu, Feb 5, 2015 at 2:35 PM, Jay Kreps < > > > > jay.kr...@gmail.com <javascript:;>> > > > > > >> wrote: > > > > > >> > > > > > > > > > >> > > > > > None on my part. > > > > > >> > > > > > > > > > > >> > > > > > -Jay > > > > > >> > > > > > > > > > > >> > > > > > On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy > > > > > >><jjkosh...@gmail.com <javascript:;> > > > > > >> > > > > > > >> > > wrote: > > > > > >> > > > > > > > > > > >> > > > > > > One amendment I would like to bring up for > > consideration > > > > wrt > > > > > >> the > > > > > >> > > KIP > > > > > >> > > > > > > process (before we formally include it in our > > by-laws) is > > > > to > > > > > >> not > > > > > >> > > > > > > restrict the votes to be a lazy majority of the PMC, > > and > > > > to > > > > > >> instead > > > > > >> > > > > > > make it a lazy majority of committers. > > > > > >> > > > > > > > > > > > >> > > > > > > Our current requirement for code changes per our > > by-laws > > > > > >>are +1 > > > > > >> > > from a > > > > > >> > > > > > > committer (who is not the contributor) followed by > > lazy > > > > > >> approval. I > > > > > >> > > > > > > think a lazy majority vote for more significant code > > > > changes > > > > > >> > > (i.e., a > > > > > >> > > > > > > KIP) should be sufficient. > > > > > >> > > > > > > > > > > > >> > > > > > > Any objection to this? > > > > > >> > > > > > > > > > > > >> > > > > > > On Sun, Jan 18, 2015 at 10:31:08AM -0800, Jay Kreps > > wrote: > > > > > >> > > > > > > > Great! Sounds like everyone is on the same page > > > > > >> > > > > > > > > > > > > >> > > > > > > > - I created a template page to make things > > easier. If > > > > > >>you > > > > > >> do > > > > > >> > > > > > > Tools->Copy > > > > > >> > > > > > > > on this page you can just fill in the italic > > portions > > > > > >>with > > > > > >> > > your > > > > > >> > > > > > > details. > > > > > >> > > > > > > > - I retrofitted KIP-1 to match this formatting > > > > > >> > > > > > > > - I added the metadata section people asked for > > (a > > > > link > > > > > >> to the > > > > > >> > > > > > > > discussion, the JIRA, and the current status). > > Let's > > > > > >>make > > > > > >> > > sure we > > > > > >> > > > > > > remember > > > > > >> > > > > > > > to update the current status as things are > > figured > > > > out. > > > > > >> > > > > > > > - Let's keep the discussion on the mailing list > > > > rather > > > > > >> than > > > > > >> > > on the > > > > > >> > > > > > > wiki > > > > > >> > > > > > > > pages. It makes sense to do one or the other so > > all > > > > the > > > > > >> > > comments > > > > > >> > > > > are > > > > > >> > > > > > > in one > > > > > >> > > > > > > > place and I think prior experience is that the > > wiki > > > > > >> comments > > > > > >> > > are > > > > > >> > > > > the > > > > > >> > > > > > > worse > > > > > >> > > > > > > > way. > > > > > >> > > > > > > > > > > > > >> > > > > > > > I think it would be great do KIPs for some of the > > > > > >>in-flight > > > > > >> items > > > > > >> > > > > folks > > > > > >> > > > > > > > mentioned. > > > > > >> > > > > > > > > > > > > >> > > > > > > > -Jay > > > > > >> > > > > > > > > > > > > >> > > > > > > > On Sat, Jan 17, 2015 at 8:23 AM, Gwen Shapira < > > > > > >> > > gshap...@cloudera.com <javascript:;> > > > > > >> > > > > > > > > > > >> > > > > > > wrote: > > > > > >> > > > > > > > > > > > > >> > > > > > > > > +1 > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Will be happy to provide a KIP for the > > > > > >>multiple-listeners > > > > > >> > > patch. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Gwen > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > On Sat, Jan 17, 2015 at 8:10 AM, Joe Stein < > > > > > >> > > joe.st...@stealth.ly <javascript:;>> > > > > > >> > > > > > > wrote: > > > > > >> > > > > > > > > > +1 to everything we have been saying and where > > this > > > > > >>(has > > > > > >> > > settled > > > > > >> > > > > > > to)/(is > > > > > >> > > > > > > > > > settling to). > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > I am sure other folks have some more feedback > > and > > > > > >>think > > > > > >> we > > > > > >> > > should > > > > > >> > > > > > > try to > > > > > >> > > > > > > > > > keep this discussion going if need be. I am > > also a > > > > > >>firm > > > > > >> > > believer > > > > > >> > > > > of > > > > > >> > > > > > > form > > > > > >> > > > > > > > > > following function so kicking the tires some > to > > > > flesh > > > > > >> out the > > > > > >> > > > > > > details of > > > > > >> > > > > > > > > > this and have some organic growth with the > > process > > > > > >>will > > > > > >> be > > > > > >> > > > > healthy > > > > > >> > > > > > > and > > > > > >> > > > > > > > > > beneficial to the community. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > For my part, what I will do is open a few KIP > > based > > > > on > > > > > >> some > > > > > >> > > of > > > > > >> > > > > the > > > > > >> > > > > > > work I > > > > > >> > > > > > > > > > have been involved with for 0.8.3. Off the top > > of my > > > > > >>head > > > > > >> > > this > > > > > >> > > > > > would > > > > > >> > > > > > > > > > include 1) changes to re-assignment of > > partitions 2) > > > > > >> kafka > > > > > >> > > cli 3) > > > > > >> > > > > > > global > > > > > >> > > > > > > > > > configs 4) security white list black list by > ip > > 5) > > > > SSL > > > > > >> 6) We > > > > > >> > > > > > probably > > > > > >> > > > > > > > > will > > > > > >> > > > > > > > > > have lots of Security related KIPs and should > > treat > > > > > >>them > > > > > >> all > > > > > >> > > > > > > individually > > > > > >> > > > > > > > > > when the time is appropriate 7) Kafka on > Mesos. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > If someone else wants to jump in to start > > getting > > > > some > > > > > >> of the > > > > > >> > > > > > > security > > > > > >> > > > > > > > > KIP > > > > > >> > > > > > > > > > that we are going to have in 0.8.3 I think > that > > > > would > > > > > >>be > > > > > >> > > great > > > > > >> > > > > > (e.g. > > > > > >> > > > > > > > > > Multiple Listeners for Kafka Brokers). There > are > > > > also > > > > > >>a > > > > > >> few > > > > > >> > > other > > > > > >> > > > > > > > > tickets I > > > > > >> > > > > > > > > > can think of that are important to have in the > > code > > > > in > > > > > >> 0.8.3 > > > > > >> > > that > > > > > >> > > > > > > should > > > > > >> > > > > > > > > > have KIP also that I haven't really been > > involved > > > > in. > > > > > >>I > > > > > >> will > > > > > >> > > > > take a > > > > > >> > > > > > > few > > > > > >> > > > > > > > > > minutes and go through JIRA (one I can think > of > > like > > > > > >>auto > > > > > >> > > assign > > > > > >> > > > > id > > > > > >> > > > > > > that > > > > > >> > > > > > > > > is > > > > > >> > > > > > > > > > already committed I think) and ask for a KIP > if > > > > > >> appropriate > > > > > >> > > or > > > > > >> > > > > if I > > > > > >> > > > > > > feel > > > > > >> > > > > > > > > > that I can write it up (both from a time and > > > > > >> understanding > > > > > >> > > > > > > perspective) > > > > > >> > > > > > > > > do > > > > > >> > > > > > > > > > so. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > long story short, I encourage folks to start > > moving > > > > > >>ahead > > > > > >> > > with > > > > > >> > > > > the > > > > > >> > > > > > > KIP > > > > > >> > > > > > > > > for > > > > > >> > > > > > > > > > 0.8.3 as how we operate. any objections? > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > On Fri, Jan 16, 2015 at 2:40 PM, Guozhang > Wang < > > > > > >> > > > > wangg...@gmail.com <javascript:;> > > > > > >> > > > > > > > > > > > >> > > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > >> +1 on the idea, and we could mutually link > the > > KIP > > > > > >>wiki > > > > > >> page > > > > > >> > > > > with > > > > > >> > > > > > > the > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > >> created JIRA ticket (i.e. include the JIRA > > number > > > > on > > > > > >>the > > > > > >> > > page > > > > > >> > > > > and > > > > > >> > > > > > > the > > > > > >> > > > > > > > > KIP > > > > > >> > > > > > > > > >> url on the ticket description). > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> Regarding the KIP process, probably we do not > > need > > > > > >>two > > > > > >> phase > > > > > >> > > > > > > > > communication > > > > > >> > > > > > > > > >> of a [DISCUSS] followed by [VOTE], as Jay > said > > the > > > > > >> voting > > > > > >> > > should > > > > > >> > > > > > be > > > > > >> > > > > > > > > clear > > > > > >> > > > > > > > > >> while people discuss about that. > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> About who should trigger the process, I think > > the > > > > > >>only > > > > > >> > > involved > > > > > >> > > > > > > people > > > > > >> > > > > > > > > >> would be 1) when the patch is submitted / or > > even > > > > the > > > > > >> > > ticket is > > > > > >> > > > > > > created, > > > > > >> > > > > > > > > >> the assignee could choose to start the KIP > > process > > > > if > > > > > >> she > > > > > >> > > > > thought > > > > > >> > > > > > > it is > > > > > >> > > > > > > > > >> necessary; 2) the reviewer of the patch can > > also > > > > > >>suggest > > > > > >> > > > > starting > > > > > >> > > > > > > KIP > > > > > >> > > > > > > > > >> discussions. > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> On Fri, Jan 16, 2015 at 10:49 AM, Gwen > Shapira > > < > > > > > >> > > > > > > gshap...@cloudera.com <javascript:;>> > > > > > >> > > > > > > > > >> wrote: > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > +1 to Ewen's suggestions: Deprecation, > > status and > > > > > >> version. > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > > >> > Perhaps add the JIRA where the KIP was > > > > implemented > > > > > >>to > > > > > >> the > > > > > >> > > > > > > metadata. > > > > > >> > > > > > > > > >> > This will help tie things together. > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > > >> > On Fri, Jan 16, 2015 at 9:35 AM, Ewen > > > > > >>Cheslack-Postava > > > > > >> > > > > > > > > >> > <e...@confluent.io <javascript:;>> wrote: > > > > > >> > > > > > > > > >> > > I think adding a section about > deprecation > > > > would > > > > > >>be > > > > > >> > > > > helpful. A > > > > > >> > > > > > > good > > > > > >> > > > > > > > > >> > > fraction of the time I would expect the > > goal > > > > of a > > > > > >> KIP > > > > > >> > > is to > > > > > >> > > > > > fix > > > > > >> > > > > > > or > > > > > >> > > > > > > > > >> > replace > > > > > >> > > > > > > > > >> > > older functionality that needs continued > > > > support > > > > > >>for > > > > > >> > > > > > > compatibility, > > > > > >> > > > > > > > > but > > > > > >> > > > > > > > > >> > > should eventually be phased out. This > helps > > > > Kafka > > > > > >> devs > > > > > >> > > > > > > understand > > > > > >> > > > > > > > > how > > > > > >> > > > > > > > > >> > long > > > > > >> > > > > > > > > >> > > they'll end up supporting multiple > > versions of > > > > > >> features > > > > > >> > > and > > > > > >> > > > > > > helps > > > > > >> > > > > > > > > users > > > > > >> > > > > > > > > >> > > understand when they're going to have to > > make > > > > > >> updates to > > > > > >> > > > > their > > > > > >> > > > > > > > > >> > applications. > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > Less important but useful -- having a bit > > of > > > > > >> standard > > > > > >> > > > > metadata > > > > > >> > > > > > > like > > > > > >> > > > > > > > > >> PEPs > > > > > >> > > > > > > > > >> > > do. Two I think are important are status > > (if > > > > > >>someone > > > > > >> > > lands > > > > > >> > > > > on > > > > > >> > > > > > > the > > > > > >> > > > > > > > > KIP > > > > > >> > > > > > > > > >> > page, > > > > > >> > > > > > > > > >> > > can they tell whether this KIP was ever > > > > > >>completed?) > > > > > >> > > and/or > > > > > >> > > > > the > > > > > >> > > > > > > > > version > > > > > >> > > > > > > > > >> > the > > > > > >> > > > > > > > > >> > > KIP was first released in. > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > On Fri, Jan 16, 2015 at 9:20 AM, Joel > > Koshy < > > > > > >> > > > > > > jjkosh...@gmail.com <javascript:;>> > > > > > >> > > > > > > > > >> wrote: > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > >> I'm definitely +1 on the KIP concept. As > > Joe > > > > > >> > > mentioned, we > > > > > >> > > > > > are > > > > > >> > > > > > > > > already > > > > > >> > > > > > > > > >> > >> doing this in one form or the other. > > However, > > > > > >>IMO > > > > > >> it is > > > > > >> > > > > > fairly > > > > > >> > > > > > > ad > > > > > >> > > > > > > > > hoc > > > > > >> > > > > > > > > >> > >> - i.e., a combination of DISCUSS > threads, > > jira > > > > > >> > > comments, RB > > > > > >> > > > > > and > > > > > >> > > > > > > > > code > > > > > >> > > > > > > > > >> > >> comments, wikis and html documentation. > > In the > > > > > >> past I > > > > > >> > > have > > > > > >> > > > > > had > > > > > >> > > > > > > to > > > > > >> > > > > > > > > dig > > > > > >> > > > > > > > > >> > >> into a bunch of these to try and figure > > out > > > > why > > > > > >> > > something > > > > > >> > > > > was > > > > > >> > > > > > > > > >> > >> implemented a certain way. I think KIPs > > can > > > > > >>help a > > > > > >> lot > > > > > >> > > here > > > > > >> > > > > > > first > > > > > >> > > > > > > > > by > > > > > >> > > > > > > > > >> > >> providing guidelines on what to think > > about > > > > > >> > > (compatibility, > > > > > >> > > > > > new > > > > > >> > > > > > > > > APIs, > > > > > >> > > > > > > > > >> > >> etc.) when working through a major > > feature; > > > > and > > > > > >> second > > > > > >> > > by > > > > > >> > > > > > > becoming > > > > > >> > > > > > > > > a > > > > > >> > > > > > > > > >> > >> crisp source of truth documentation for > > new > > > > > >> releases. > > > > > >> > > > > E.g., > > > > > >> > > > > > > for > > > > > >> > > > > > > > > >> > >> feature X: see relevant KIPs: a, b, c, > > etc. > > > > > >> > > > > > > > > >> > >> > > > > > >> > > > > > > > > >> > >> On Thu, Jan 15, 2015 at 08:11:20PM > -0800, > > Jay > > > > > >>Kreps > > > > > >> > > wrote: > > > > > >> > > > > > > > > >> > >> > Hey Joe, > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > Yeah I guess the question is what is > the > > > > > >> definition > > > > > >> > > of > > > > > >> > > > > > > major? I > > > > > >> > > > > > > > > >> agree > > > > > >> > > > > > > > > >> > we > > > > > >> > > > > > > > > >> > >> > definitely don't want to generate a > > bunch of > > > > > >> > > paperwork. > > > > > >> > > > > We > > > > > >> > > > > > > have > > > > > >> > > > > > > > > >> enough > > > > > >> > > > > > > > > >> > >> > problems just getting all the > > contributions > > > > > >> reviewed > > > > > >> > > and > > > > > >> > > > > > > checked > > > > > >> > > > > > > > > in > > > > > >> > > > > > > > > >> > in a > > > > > >> > > > > > > > > >> > >> > timely fashion... > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > So obviously bug fixes would not apply > > here. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > I think it is also pretty clear that > big > > > > > >>features > > > > > >> > > should > > > > > >> > > > > > get > > > > > >> > > > > > > > > >> reviewed > > > > > >> > > > > > > > > >> > and > > > > > >> > > > > > > > > >> > >> > discussed. To pick on myself, for > > example, > > > > the > > > > > >> log > > > > > >> > > > > > compaction > > > > > >> > > > > > > > > work > > > > > >> > > > > > > > > >> was > > > > > >> > > > > > > > > >> > >> done > > > > > >> > > > > > > > > >> > >> > without enough public discussion about > > how > > > > it > > > > > >> worked > > > > > >> > > and > > > > > >> > > > > > why > > > > > >> > > > > > > > > >> (imho). I > > > > > >> > > > > > > > > >> > >> > hope/claim that enough rigour in > > thinking > > > > > >>about > > > > > >> > > use-cases > > > > > >> > > > > > and > > > > > >> > > > > > > > > >> > operations > > > > > >> > > > > > > > > >> > >> > and so on was done that it turned out > > well, > > > > > >>but > > > > > >> the > > > > > >> > > > > > > discussion > > > > > >> > > > > > > > > was > > > > > >> > > > > > > > > >> > just > > > > > >> > > > > > > > > >> > >> > between a few people with no real > public > > > > > >>output. > > > > > >> This > > > > > >> > > > > kind > > > > > >> > > > > > of > > > > > >> > > > > > > > > >> feature > > > > > >> > > > > > > > > >> > is > > > > > >> > > > > > > > > >> > >> > clearly a big change and something we > > should > > > > > >> discuss. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > If we limit ourselves to just the > public > > > > > >> contracts > > > > > >> > > the > > > > > >> > > > > KIP > > > > > >> > > > > > > > > >> introduces > > > > > >> > > > > > > > > >> > the > > > > > >> > > > > > > > > >> > >> > discussion would just be on the new > > configs > > > > > >>and > > > > > >> > > > > monitoring > > > > > >> > > > > > > > > without > > > > > >> > > > > > > > > >> > >> really a > > > > > >> > > > > > > > > >> > >> > discussion of the design and how it > > works > > > > > >>which > > > > > >> is > > > > > >> > > > > > obviously > > > > > >> > > > > > > > > closely > > > > > >> > > > > > > > > >> > >> > related. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > I don't think this should be more work > > > > > >>because in > > > > > >> > > > > practice > > > > > >> > > > > > > we are > > > > > >> > > > > > > > > >> > making > > > > > >> > > > > > > > > >> > >> > wiki pages for any big thing anyway. > So > > this > > > > > >> would > > > > > >> > > just > > > > > >> > > > > be > > > > > >> > > > > > a > > > > > >> > > > > > > > > >> > consistent > > > > > >> > > > > > > > > >> > >> way > > > > > >> > > > > > > > > >> > >> > of numbering and structuring these > > pages. > > > > This > > > > > >> would > > > > > >> > > also > > > > > >> > > > > > > give a > > > > > >> > > > > > > > > >> clear > > > > > >> > > > > > > > > >> > >> call > > > > > >> > > > > > > > > >> > >> > to action: "hey kafka people, come > read > > my > > > > > >>wiki > > > > > >> and > > > > > >> > > think > > > > > >> > > > > > > this > > > > > >> > > > > > > > > >> > through". > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > I actually thinking the voting aspect > is > > > > less > > > > > >> > > important. > > > > > >> > > > > I > > > > > >> > > > > > > think > > > > > >> > > > > > > > > it > > > > > >> > > > > > > > > >> is > > > > > >> > > > > > > > > >> > >> > generally clear when there is > agreement > > on > > > > > >> something > > > > > >> > > and > > > > > >> > > > > > > not. So > > > > > >> > > > > > > > > >> from > > > > > >> > > > > > > > > >> > my > > > > > >> > > > > > > > > >> > >> > point of view we could actually just > > > > eliminate > > > > > >> that > > > > > >> > > part > > > > > >> > > > > if > > > > > >> > > > > > > that > > > > > >> > > > > > > > > is > > > > > >> > > > > > > > > >> > too > > > > > >> > > > > > > > > >> > >> > formal, it just seemed like a good way > > to > > > > > >> formally > > > > > >> > > adopt > > > > > >> > > > > > > > > something. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > To address some of your comments from > > the > > > > > >>wiki: > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > 1. This doesn't inhibit someone coming > > along > > > > > >>and > > > > > >> > > putting > > > > > >> > > > > > up a > > > > > >> > > > > > > > > patch. > > > > > >> > > > > > > > > >> > It > > > > > >> > > > > > > > > >> > >> is > > > > > >> > > > > > > > > >> > >> > just that when they do if it is a big > > thing > > > > > >> > > introducing > > > > > >> > > > > new > > > > > >> > > > > > > > > >> > functionality > > > > > >> > > > > > > > > >> > >> > we would ask for a little discussion > on > > the > > > > > >>basic > > > > > >> > > > > > > > > feature/contracts > > > > > >> > > > > > > > > >> > prior > > > > > >> > > > > > > > > >> > >> > to code review. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > 2. We definitely definitely don't want > > > > people > > > > > >> > > generating > > > > > >> > > > > a > > > > > >> > > > > > > lot of > > > > > >> > > > > > > > > >> > these > > > > > >> > > > > > > > > >> > >> > things every time they have an idea > that > > > > they > > > > > >> aren't > > > > > >> > > > > going > > > > > >> > > > > > to > > > > > >> > > > > > > > > >> > implement. > > > > > >> > > > > > > > > >> > >> So > > > > > >> > > > > > > > > >> > >> > this is only applicable to things you > > > > > >>absolutely > > > > > >> will > > > > > >> > > > > check > > > > > >> > > > > > > in > > > > > >> > > > > > > > > code > > > > > >> > > > > > > > > >> > for. > > > > > >> > > > > > > > > >> > >> We > > > > > >> > > > > > > > > >> > >> > also don't want to be making proposals > > > > before > > > > > >> things > > > > > >> > > are > > > > > >> > > > > > > thought > > > > > >> > > > > > > > > >> > through, > > > > > >> > > > > > > > > >> > >> > which often requires writing the code. > > So I > > > > > >> think the > > > > > >> > > > > right > > > > > >> > > > > > > time > > > > > >> > > > > > > > > >> for a > > > > > >> > > > > > > > > >> > >> KIP > > > > > >> > > > > > > > > >> > >> > is when you are far enough along that > > you > > > > know > > > > > >> the > > > > > >> > > issues > > > > > >> > > > > > and > > > > > >> > > > > > > > > >> > tradeoffs > > > > > >> > > > > > > > > >> > >> but > > > > > >> > > > > > > > > >> > >> > not so far along that you are going to > > be > > > > > >>totally > > > > > >> > > opposed > > > > > >> > > > > > to > > > > > >> > > > > > > any > > > > > >> > > > > > > > > >> > change. > > > > > >> > > > > > > > > >> > >> > Sometimes that is prior to writing any > > code > > > > > >>and > > > > > >> > > sometimes > > > > > >> > > > > > not > > > > > >> > > > > > > > > until > > > > > >> > > > > > > > > >> > you > > > > > >> > > > > > > > > >> > >> are > > > > > >> > > > > > > > > >> > >> > practically done. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > The key problem I see this fixing is > > that > > > > > >>there > > > > > >> is > > > > > >> > > enough > > > > > >> > > > > > new > > > > > >> > > > > > > > > >> > development > > > > > >> > > > > > > > > >> > >> > happening that it is pretty hard for > > > > everyone > > > > > >>to > > > > > >> > > review > > > > > >> > > > > > every > > > > > >> > > > > > > > > line > > > > > >> > > > > > > > > >> of > > > > > >> > > > > > > > > >> > >> every > > > > > >> > > > > > > > > >> > >> > iteration of every patch. But all of > us > > > > > >>should be > > > > > >> > > fully > > > > > >> > > > > > > aware of > > > > > >> > > > > > > > > new > > > > > >> > > > > > > > > >> > >> > features, the ramifications, the new > > public > > > > > >> > > interfaces, > > > > > >> > > > > > etc. > > > > > >> > > > > > > If > > > > > >> > > > > > > > > we > > > > > >> > > > > > > > > >> > aren't > > > > > >> > > > > > > > > >> > >> > aware of that we can't really build a > > > > holistic > > > > > >> system > > > > > >> > > > > that > > > > > >> > > > > > is > > > > > >> > > > > > > > > >> > beautiful > > > > > >> > > > > > > > > >> > >> and > > > > > >> > > > > > > > > >> > >> > consistent across. So the idea is that > > if > > > > you > > > > > >> fully > > > > > >> > > > > review > > > > > >> > > > > > > the > > > > > >> > > > > > > > > KIPs > > > > > >> > > > > > > > > >> > you > > > > > >> > > > > > > > > >> > >> can > > > > > >> > > > > > > > > >> > >> > be sure that even if you don't know > > every > > > > new > > > > > >> line of > > > > > >> > > > > code, > > > > > >> > > > > > > you > > > > > >> > > > > > > > > know > > > > > >> > > > > > > > > >> > the > > > > > >> > > > > > > > > >> > >> > major changes coming in. > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > -Jay > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > On Thu, Jan 15, 2015 at 12:18 PM, Joe > > Stein > > > > < > > > > > >> > > > > > > > > joe.st...@stealth.ly <javascript:;>> > > > > > >> > > > > > > > > >> > >> wrote: > > > > > >> > > > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > >> > > Thanks Jay for kicking this off! I > > think > > > > the > > > > > >> > > confluence > > > > > >> > > > > > > page > > > > > >> > > > > > > > > you > > > > > >> > > > > > > > > >> > wrote > > > > > >> > > > > > > > > >> > >> up > > > > > >> > > > > > > > > >> > >> > > is a great start. > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > The KIP makes sense to me (at a > > minimum) > > > > if > > > > > >> there > > > > > >> > > is > > > > > >> > > > > > going > > > > > >> > > > > > > to > > > > > >> > > > > > > > > be > > > > > >> > > > > > > > > >> any > > > > > >> > > > > > > > > >> > >> > > "breaking change". This way Kafka > can > > > > > >>continue > > > > > >> to > > > > > >> > > grow > > > > > >> > > > > > and > > > > > >> > > > > > > > > blossom > > > > > >> > > > > > > > > >> > and > > > > > >> > > > > > > > > >> > >> we > > > > > >> > > > > > > > > >> > >> > > have a process in place if we are > > going to > > > > > >> release > > > > > >> > > a > > > > > >> > > > > > thorn > > > > > >> > > > > > > ... > > > > > >> > > > > > > > > and > > > > > >> > > > > > > > > >> > >> when we > > > > > >> > > > > > > > > >> > >> > > do it is *CLEAR* about what and why > > that > > > > is. > > > > > >> We can > > > > > >> > > > > > easily > > > > > >> > > > > > > > > >> document > > > > > >> > > > > > > > > >> > >> which > > > > > >> > > > > > > > > >> > >> > > KIPs where involved with this > release > > > > > >>(which I > > > > > >> > > think > > > > > >> > > > > > > should get > > > > > >> > > > > > > > > >> > >> committed > > > > > >> > > > > > > > > >> > >> > > afterwards somewhere so no chance of > > edit > > > > > >>after > > > > > >> > > > > release). > > > > > >> > > > > > > This > > > > > >> > > > > > > > > >> > >> approach I > > > > > >> > > > > > > > > >> > >> > > had been thinking about also allows > > > > changes > > > > > >>to > > > > > >> > > occur as > > > > > >> > > > > > > they do > > > > > >> > > > > > > > > >> now > > > > > >> > > > > > > > > >> > as > > > > > >> > > > > > > > > >> > >> long > > > > > >> > > > > > > > > >> > >> > > as they are backwards compatible. > > > > > >>Hopefully we > > > > > >> > > never > > > > > >> > > > > > need > > > > > >> > > > > > > a > > > > > >> > > > > > > > > KIP > > > > > >> > > > > > > > > >> but > > > > > >> > > > > > > > > >> > >> when > > > > > >> > > > > > > > > >> > >> > > we do the PMC can vote on it and > > folks can > > > > > >> read the > > > > > >> > > > > > release > > > > > >> > > > > > > > > notes > > > > > >> > > > > > > > > >> > with > > > > > >> > > > > > > > > >> > >> > > *CLEAR* understanding what is going > to > > > > break > > > > > >> their > > > > > >> > > > > > existing > > > > > >> > > > > > > > > >> > setup... at > > > > > >> > > > > > > > > >> > >> > > least that is how I have been > thinking > > > > about > > > > > >> it. > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > Let me know what you think about > this > > base > > > > > >> minimum > > > > > >> > > > > > > approach... > > > > > >> > > > > > > > > I > > > > > >> > > > > > > > > >> > hadn't > > > > > >> > > > > > > > > >> > >> > > really thought of the KIP for *ANY* > > "major > > > > > >> change" > > > > > >> > > and > > > > > >> > > > > I > > > > > >> > > > > > > have > > > > > >> > > > > > > > > to > > > > > >> > > > > > > > > >> > think > > > > > >> > > > > > > > > >> > >> more > > > > > >> > > > > > > > > >> > >> > > about that. I have some other > > comments for > > > > > >> minor > > > > > >> > > items > > > > > >> > > > > in > > > > > >> > > > > > > the > > > > > >> > > > > > > > > >> > >> confluence > > > > > >> > > > > > > > > >> > >> > > page I will make once I think more > > about > > > > > >>how I > > > > > >> feel > > > > > >> > > > > > having > > > > > >> > > > > > > a > > > > > >> > > > > > > > > KIP > > > > > >> > > > > > > > > >> for > > > > > >> > > > > > > > > >> > >> more > > > > > >> > > > > > > > > >> > >> > > than what I was thinking about > > already. > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > I do think we should have "major > > changes" > > > > go > > > > > >> > > through > > > > > >> > > > > > > > > confluence, > > > > > >> > > > > > > > > >> > >> mailing > > > > > >> > > > > > > > > >> > >> > > list discuss and JIRA but kind of > > feel we > > > > > >>have > > > > > >> been > > > > > >> > > > > doing > > > > > >> > > > > > > that > > > > > >> > > > > > > > > >> > already > > > > > >> > > > > > > > > >> > >> ... > > > > > >> > > > > > > > > >> > >> > > if there are cases where that isn't > > the > > > > > >>case we > > > > > >> > > should > > > > > >> > > > > > > > > highlight > > > > > >> > > > > > > > > >> and > > > > > >> > > > > > > > > >> > >> learn > > > > > >> > > > > > > > > >> > >> > > from them and formalize that more if > > need > > > > > >>be. > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > /******************************************* > > > > > >> > > > > > > > > >> > >> > > Joe Stein > > > > > >> > > > > > > > > >> > >> > > Founder, Principal Consultant > > > > > >> > > > > > > > > >> > >> > > Big Data Open Source Security LLC > > > > > >> > > > > > > > > >> > >> > > http://www.stealth.ly > > > > > >> > > > > > > > > >> > >> > > Twitter: @allthingshadoop < > > > > > >> > > > > > > > > >> http://www.twitter.com/allthingshadoop> > > > > > >> > > > > > > > > >> > >> > > > > > > > >>********************************************/ > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > On Thu, Jan 15, 2015 at 1:42 PM, Jay > > > > Kreps < > > > > > >> > > > > > > > > jay.kr...@gmail.com <javascript:;>> > > > > > >> > > > > > > > > >> > >> wrote: > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > The idea of KIPs came up in a > > previous > > > > > >> > > discussion but > > > > > >> > > > > > > there > > > > > >> > > > > > > > > was > > > > > >> > > > > > > > > >> no > > > > > >> > > > > > > > > >> > >> real > > > > > >> > > > > > > > > >> > >> > > > crisp definition of what they > were. > > Here > > > > > >>is > > > > > >> an > > > > > >> > > > > attempt > > > > > >> > > > > > at > > > > > >> > > > > > > > > >> > defining a > > > > > >> > > > > > > > > >> > >> > > > process: > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Propo > > > > > >>sals > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > >> > >> > > > The trick here is to have > something > > > > > >> light-weight > > > > > >> > > > > enough > > > > > >> > > > > > > that > > > > > >> > > > > > > > > it > > > > > >> > > > > > > > > >> > >> isn't a > > > > > >> > > > > > > > > >> > >> > > > hassle for small changes, but > > enough so > > > > > >>that > > > > > >> > > changes > > > > > >> > > > > > get > > > > > >> > > > > > > the > > > > > >> > > > > > > > > >> > >> eyeballs of > > > > > >> > > > > > > > > >> > >> > > > the committers and heavy users. > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > >> > >> > > > Thoughts? > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > >> > >> > > > -Jay > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > >> > > > > > > > > >> > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > -- > > > > > >> > > > > > > > > >> > > Thanks, > > > > > >> > > > > > > > > >> > > Ewen > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> -- > > > > > >> > > > > > > > > >> -- Guozhang > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > -- > > > > > >> > > > > Thanks, > > > > > >> > > > > Neha > > > > > >> > > > > > > > > > >> > > > > > > > >> > > -- > > > > > >> > > Joel > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > Sent from Gmail Mobile > > > > > > > -- > -- Guozhang >
