I'm looking through a couple of the KIP threads today and had the same issue; and thought it would be useful to do a status round-up of KIPs. We could incorporate status in the title itself (so we can quickly see it in the child-page list) but I just added a table to the top-level wiki. Hopefully that captures the current state accurately so I know which threads to follow-up on.
Thanks, Joel On Fri, Feb 06, 2015 at 12:47:46PM -0800, Jay Kreps wrote: > A problem I am having is actually understanding which KIPs are intended to > be complete proposals and which are works in progress. Joe you seem to have > a bunch of these. Can you move them elsewhere until they are really fully > done and ready for review and discussion? > > -Jay > > On Fri, Feb 6, 2015 at 12:09 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > I think we are focused on making committing new changes easier, but what > > we have seen is actually that isn't the bulk of the work (especially with > > this kind of "public interface" change where it generally has a big user > > impact). I think we actually really need the core committers and any other > > interested parties to stop and fully read each KIP and think about it. If > > we don't have time to do that we usually just end up spending a lot more > > time after the fact trying to rework things latter when it is a lot harder. > > So I really think we should have every active committer read, comment, and > > vote on each KIP. I think this may require a little bit of work to > > co-ordinate/bug people but will end up being worth it because each person > > on the project will have a holistic picture of what is going on. > > > > -Jay > > > > On Thu, Feb 5, 2015 at 11:24 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > >> Just wanted to add a few more comments on this: KIPs were suggested as > >> a process to help reach early consensus on a major change or not so > >> major (but tricky or backward incompatible) change in order to reduce > >> the likelihood of multiple iterations and complete rewrites during > >> code reviews (which is time-intensive for both the contributor and > >> reviewers); as well as to reduce the likelihood of surprises (say, if > >> a patch inadvertently changes a public API). So KIPs are intended to > >> speed up development since a clear path is charted out and there is > >> prior consensus on whether a feature and its design/implementation > >> make sense or not. > >> > >> Obviously this breaks down if KIPs are not being actively discussed - > >> again I think we can do much better here. I think we ended up with a > >> backlog because as soon as the KIP wiki was started, a number of > >> pre-existing jiras and discussions were moved there - all within a few > >> days. Now that there are quite a few outstanding KIPs I think we just > >> need to methodically work through those - preferably a couple at a > >> time. I looked through the list and I think we should be able to > >> resolve all of them relatively quickly if everyone is on board with > >> this. > >> > >> > > Its probably more helpful for contributors if its "lazy" as in "no > >> > > strong objections" . > >> > >> Gwen also suggested this and this also sounds ok to me as I wrote > >> earlier - what do others think? This is important especially if > >> majority in the community think if this less restrictive policy would > >> spur and not hinder development - I'm not sure that it does. I > >> completely agree that KIPs fail to a large degree as far as the > >> original motivation goes if they require a lazy majority but the > >> DISCUSS threads are stalled. IOW regardless of that discussion, I > >> think we should rejuvenate some of those threads especially now that > >> 0.8.2 is out of the way. > >> > >> Thanks, > >> > >> Joel > >> > >> On Thu, Feb 05, 2015 at 08:56:13PM -0800, Joel Koshy wrote: > >> > I'm just thinking aloud - I don't know what a good number would be, and > >> it > >> > is just one possibility to streamline how KIPs are processed. It largely > >> > depends on how complex the proposals are. What would be concerning is if > >> > there are 10 different threads all dealing with large KIPs and no one > >> has > >> > the time to give due diligence to each one and all those threads grind > >> to a > >> > halt due to confusion, incomplete context and misunderstandings. > >> > > >> > On Thursday, February 5, 2015, Harsha <ka...@harsha.io> wrote: > >> > > >> > > Joel, > >> > > Having only 2 or 3 KIPS under active discussion is concerning. > >> > > This will slow down development process as well. > >> > > Having a turn-around time for a KIP is a good idea but what will > >> happen > >> > > if it didn't received required votes within that time frame. > >> > > Its probably more helpful for contributors if its "lazy" as in "no > >> > > strong objections" . > >> > > Just to make sure this is only for KIPs not for regular bug fixes > >> right? > >> > > Thanks, > >> > > Harsha > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > On Thu, Feb 5, 2015, at 05:59 PM, Jiangjie Qin wrote: > >> > > > Iąm having an impression that KIP is mostly for new features but > >> not for > >> > > > bug fixes. But I agree with Joel that it might make sense to have > >> some > >> > > > big > >> > > > patches, even if they are bug fixes, to follow the KIP like process > >> which > >> > > > is more strict. > >> > > > > >> > > > Jiangjie (Becket) Qin > >> > > > > >> > > > On 2/5/15, 4:57 PM, "Gwen Shapira" <gshap...@cloudera.com > >> <javascript:;>> > >> > > wrote: > >> > > > > >> > > > >> > >> > > > >> > >> > > > >> Yes there are KIPs that are currently blocked on feedback/votes, > >> but I > >> > > > >> don't think it is an issue of not caring to comment vs having so > >> many > >> > > > >> KIPs and other code reviews in flight that people are just > >> swamped. > >> > > > >> > >> > > > >> > >> > > > >This is exactly my concern. > >> > > > >Even now important patches and features have very long development > >> and > >> > > > >review cycles due to Kafka's small and very busy committer > >> community. I > >> > > > >feel that this change takes things in the wrong direction > >> > > > > > >> > > > > > >> > > > >> Joel > >> > > > >> > >> > > > >> On Thu, Feb 05, 2015 at 04:19:54PM -0800, Gwen Shapira wrote: > >> > > > >> > Isn't requiring 3 binding votes a bit overly strict here? We > >> are > >> > > > >>talking > >> > > > >> > about patches which in can be fixed, reverted, etc. Not > >> releases, > >> > > > >>which > >> > > > >> > have legal implications. > >> > > > >> > > >> > > > >> > Why not go with usual definition: "lazy" = "No strong > >> objections for > >> > > > >>few > >> > > > >> > days"? > >> > > > >> > This means contributors will not be blocked on issues where no > >> one > >> > > > >>cares > >> > > > >> to > >> > > > >> > comment (and we had few of those). > >> > > > >> > > >> > > > >> > Gwen > >> > > > >> > > >> > > > >> > > >> > > > >> > > >> > > > >> > On Thu, Feb 5, 2015 at 4:14 PM, Joel Koshy < > >> jjkosh...@gmail.com > >> > > <javascript:;>> > >> > > > >>wrote: > >> > > > >> > > >> > > > >> > > Sorry about this - I actually meant to suggest lazy consensus > >> > > (which > >> > > > >> > > is a stronger requirement): "3 binding +1 votes and no > >> binding > >> > > > >> > > vetoes." > >> > > > >> > > > >> > > > >> > > I have updated the wiki to lazy consensus - but can change > >> it back > >> > > > >>if > >> > > > >> > > there is a reasonable objection. > >> > > > >> > > > >> > > > >> > > On Thu, Feb 05, 2015 at 06:17:44PM -0500, Joe Stein wrote: > >> > > > >> > > > +1 > >> > > > >> > > > > >> > > > >> > > > On Thu, Feb 5, 2015 at 6:16 PM, Neha Narkhede < > >> > > n...@confluent.io <javascript:;>> > >> > > > >> wrote: > >> > > > >> > > > > >> > > > >> > > > > Sounds good. > >> > > > >> > > > > > >> > > > >> > > > > On Thu, Feb 5, 2015 at 2:35 PM, Jay Kreps < > >> > > jay.kr...@gmail.com <javascript:;>> > >> > > > >> wrote: > >> > > > >> > > > > > >> > > > >> > > > > > None on my part. > >> > > > >> > > > > > > >> > > > >> > > > > > -Jay > >> > > > >> > > > > > > >> > > > >> > > > > > On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy > >> > > > >><jjkosh...@gmail.com <javascript:;> > >> > > > >> > > >> > > > >> > > wrote: > >> > > > >> > > > > > > >> > > > >> > > > > > > One amendment I would like to bring up for > >> consideration > >> > > wrt > >> > > > >> the > >> > > > >> > > KIP > >> > > > >> > > > > > > process (before we formally include it in our > >> by-laws) is > >> > > to > >> > > > >> not > >> > > > >> > > > > > > restrict the votes to be a lazy majority of the PMC, > >> and > >> > > to > >> > > > >> instead > >> > > > >> > > > > > > make it a lazy majority of committers. > >> > > > >> > > > > > > > >> > > > >> > > > > > > Our current requirement for code changes per our > >> by-laws > >> > > > >>are +1 > >> > > > >> > > from a > >> > > > >> > > > > > > committer (who is not the contributor) followed by > >> lazy > >> > > > >> approval. I > >> > > > >> > > > > > > think a lazy majority vote for more significant code > >> > > changes > >> > > > >> > > (i.e., a > >> > > > >> > > > > > > KIP) should be sufficient. > >> > > > >> > > > > > > > >> > > > >> > > > > > > Any objection to this? > >> > > > >> > > > > > > > >> > > > >> > > > > > > On Sun, Jan 18, 2015 at 10:31:08AM -0800, Jay Kreps > >> wrote: > >> > > > >> > > > > > > > Great! Sounds like everyone is on the same page > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > - I created a template page to make things > >> easier. If > >> > > > >>you > >> > > > >> do > >> > > > >> > > > > > > Tools->Copy > >> > > > >> > > > > > > > on this page you can just fill in the italic > >> portions > >> > > > >>with > >> > > > >> > > your > >> > > > >> > > > > > > details. > >> > > > >> > > > > > > > - I retrofitted KIP-1 to match this formatting > >> > > > >> > > > > > > > - I added the metadata section people asked for > >> (a > >> > > link > >> > > > >> to the > >> > > > >> > > > > > > > discussion, the JIRA, and the current status). > >> Let's > >> > > > >>make > >> > > > >> > > sure we > >> > > > >> > > > > > > remember > >> > > > >> > > > > > > > to update the current status as things are > >> figured > >> > > out. > >> > > > >> > > > > > > > - Let's keep the discussion on the mailing list > >> > > rather > >> > > > >> than > >> > > > >> > > on the > >> > > > >> > > > > > > wiki > >> > > > >> > > > > > > > pages. It makes sense to do one or the other so > >> all > >> > > the > >> > > > >> > > comments > >> > > > >> > > > > are > >> > > > >> > > > > > > in one > >> > > > >> > > > > > > > place and I think prior experience is that the > >> wiki > >> > > > >> comments > >> > > > >> > > are > >> > > > >> > > > > the > >> > > > >> > > > > > > worse > >> > > > >> > > > > > > > way. > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > I think it would be great do KIPs for some of the > >> > > > >>in-flight > >> > > > >> items > >> > > > >> > > > > folks > >> > > > >> > > > > > > > mentioned. > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > -Jay > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > On Sat, Jan 17, 2015 at 8:23 AM, Gwen Shapira < > >> > > > >> > > gshap...@cloudera.com <javascript:;> > >> > > > >> > > > > > > >> > > > >> > > > > > > wrote: > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > > +1 > >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > Will be happy to provide a KIP for the > >> > > > >>multiple-listeners > >> > > > >> > > patch. > >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > Gwen > >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > On Sat, Jan 17, 2015 at 8:10 AM, Joe Stein < > >> > > > >> > > joe.st...@stealth.ly <javascript:;>> > >> > > > >> > > > > > > wrote: > >> > > > >> > > > > > > > > > +1 to everything we have been saying and where > >> this > >> > > > >>(has > >> > > > >> > > settled > >> > > > >> > > > > > > to)/(is > >> > > > >> > > > > > > > > > settling to). > >> > > > >> > > > > > > > > > > >> > > > >> > > > > > > > > > I am sure other folks have some more feedback > >> and > >> > > > >>think > >> > > > >> we > >> > > > >> > > should > >> > > > >> > > > > > > try to > >> > > > >> > > > > > > > > > keep this discussion going if need be. I am > >> also a > >> > > > >>firm > >> > > > >> > > believer > >> > > > >> > > > > of > >> > > > >> > > > > > > form > >> > > > >> > > > > > > > > > following function so kicking the tires some to > >> > > flesh > >> > > > >> out the > >> > > > >> > > > > > > details of > >> > > > >> > > > > > > > > > this and have some organic growth with the > >> process > >> > > > >>will > >> > > > >> be > >> > > > >> > > > > healthy > >> > > > >> > > > > > > and > >> > > > >> > > > > > > > > > beneficial to the community. > >> > > > >> > > > > > > > > > > >> > > > >> > > > > > > > > > For my part, what I will do is open a few KIP > >> based > >> > > on > >> > > > >> some > >> > > > >> > > of > >> > > > >> > > > > the > >> > > > >> > > > > > > work I > >> > > > >> > > > > > > > > > have been involved with for 0.8.3. Off the top > >> of my > >> > > > >>head > >> > > > >> > > this > >> > > > >> > > > > > would > >> > > > >> > > > > > > > > > include 1) changes to re-assignment of > >> partitions 2) > >> > > > >> kafka > >> > > > >> > > cli 3) > >> > > > >> > > > > > > global > >> > > > >> > > > > > > > > > configs 4) security white list black list by > >> ip 5) > >> > > SSL > >> > > > >> 6) We > >> > > > >> > > > > > probably > >> > > > >> > > > > > > > > will > >> > > > >> > > > > > > > > > have lots of Security related KIPs and should > >> treat > >> > > > >>them > >> > > > >> all > >> > > > >> > > > > > > individually > >> > > > >> > > > > > > > > > when the time is appropriate 7) Kafka on Mesos. > >> > > > >> > > > > > > > > > > >> > > > >> > > > > > > > > > If someone else wants to jump in to start > >> getting > >> > > some > >> > > > >> of the > >> > > > >> > > > > > > security > >> > > > >> > > > > > > > > KIP > >> > > > >> > > > > > > > > > that we are going to have in 0.8.3 I think that > >> > > would > >> > > > >>be > >> > > > >> > > great > >> > > > >> > > > > > (e.g. > >> > > > >> > > > > > > > > > Multiple Listeners for Kafka Brokers). There > >> are > >> > > also > >> > > > >>a > >> > > > >> few > >> > > > >> > > other > >> > > > >> > > > > > > > > tickets I > >> > > > >> > > > > > > > > > can think of that are important to have in the > >> code > >> > > in > >> > > > >> 0.8.3 > >> > > > >> > > that > >> > > > >> > > > > > > should > >> > > > >> > > > > > > > > > have KIP also that I haven't really been > >> involved > >> > > in. > >> > > > >>I > >> > > > >> will > >> > > > >> > > > > take a > >> > > > >> > > > > > > few > >> > > > >> > > > > > > > > > minutes and go through JIRA (one I can think > >> of like > >> > > > >>auto > >> > > > >> > > assign > >> > > > >> > > > > id > >> > > > >> > > > > > > that > >> > > > >> > > > > > > > > is > >> > > > >> > > > > > > > > > already committed I think) and ask for a KIP if > >> > > > >> appropriate > >> > > > >> > > or > >> > > > >> > > > > if I > >> > > > >> > > > > > > feel > >> > > > >> > > > > > > > > > that I can write it up (both from a time and > >> > > > >> understanding > >> > > > >> > > > > > > perspective) > >> > > > >> > > > > > > > > do > >> > > > >> > > > > > > > > > so. > >> > > > >> > > > > > > > > > > >> > > > >> > > > > > > > > > long story short, I encourage folks to start > >> moving > >> > > > >>ahead > >> > > > >> > > with > >> > > > >> > > > > the > >> > > > >> > > > > > > KIP > >> > > > >> > > > > > > > > for > >> > > > >> > > > > > > > > > 0.8.3 as how we operate. any objections? > >> > > > >> > > > > > > > > > > >> > > > >> > > > > > > > > > On Fri, Jan 16, 2015 at 2:40 PM, Guozhang Wang > >> < > >> > > > >> > > > > wangg...@gmail.com <javascript:;> > >> > > > >> > > > > > > > >> > > > >> > > > > > > > > wrote: > >> > > > >> > > > > > > > > > > >> > > > >> > > > > > > > > >> +1 on the idea, and we could mutually link > >> the KIP > >> > > > >>wiki > >> > > > >> page > >> > > > >> > > > > with > >> > > > >> > > > > > > the > >> > > > >> > > > > > > > > the > >> > > > >> > > > > > > > > >> created JIRA ticket (i.e. include the JIRA > >> number > >> > > on > >> > > > >>the > >> > > > >> > > page > >> > > > >> > > > > and > >> > > > >> > > > > > > the > >> > > > >> > > > > > > > > KIP > >> > > > >> > > > > > > > > >> url on the ticket description). > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> Regarding the KIP process, probably we do not > >> need > >> > > > >>two > >> > > > >> phase > >> > > > >> > > > > > > > > communication > >> > > > >> > > > > > > > > >> of a [DISCUSS] followed by [VOTE], as Jay > >> said the > >> > > > >> voting > >> > > > >> > > should > >> > > > >> > > > > > be > >> > > > >> > > > > > > > > clear > >> > > > >> > > > > > > > > >> while people discuss about that. > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> About who should trigger the process, I think > >> the > >> > > > >>only > >> > > > >> > > involved > >> > > > >> > > > > > > people > >> > > > >> > > > > > > > > >> would be 1) when the patch is submitted / or > >> even > >> > > the > >> > > > >> > > ticket is > >> > > > >> > > > > > > created, > >> > > > >> > > > > > > > > >> the assignee could choose to start the KIP > >> process > >> > > if > >> > > > >> she > >> > > > >> > > > > thought > >> > > > >> > > > > > > it is > >> > > > >> > > > > > > > > >> necessary; 2) the reviewer of the patch can > >> also > >> > > > >>suggest > >> > > > >> > > > > starting > >> > > > >> > > > > > > KIP > >> > > > >> > > > > > > > > >> discussions. > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> On Fri, Jan 16, 2015 at 10:49 AM, Gwen > >> Shapira < > >> > > > >> > > > > > > gshap...@cloudera.com <javascript:;>> > >> > > > >> > > > > > > > > >> wrote: > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> > +1 to Ewen's suggestions: Deprecation, > >> status and > >> > > > >> version. > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > Perhaps add the JIRA where the KIP was > >> > > implemented > >> > > > >>to > >> > > > >> the > >> > > > >> > > > > > > metadata. > >> > > > >> > > > > > > > > >> > This will help tie things together. > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > On Fri, Jan 16, 2015 at 9:35 AM, Ewen > >> > > > >>Cheslack-Postava > >> > > > >> > > > > > > > > >> > <e...@confluent.io <javascript:;>> wrote: > >> > > > >> > > > > > > > > >> > > I think adding a section about deprecation > >> > > would > >> > > > >>be > >> > > > >> > > > > helpful. A > >> > > > >> > > > > > > good > >> > > > >> > > > > > > > > >> > > fraction of the time I would expect the > >> goal > >> > > of a > >> > > > >> KIP > >> > > > >> > > is to > >> > > > >> > > > > > fix > >> > > > >> > > > > > > or > >> > > > >> > > > > > > > > >> > replace > >> > > > >> > > > > > > > > >> > > older functionality that needs continued > >> > > support > >> > > > >>for > >> > > > >> > > > > > > compatibility, > >> > > > >> > > > > > > > > but > >> > > > >> > > > > > > > > >> > > should eventually be phased out. This > >> helps > >> > > Kafka > >> > > > >> devs > >> > > > >> > > > > > > understand > >> > > > >> > > > > > > > > how > >> > > > >> > > > > > > > > >> > long > >> > > > >> > > > > > > > > >> > > they'll end up supporting multiple > >> versions of > >> > > > >> features > >> > > > >> > > and > >> > > > >> > > > > > > helps > >> > > > >> > > > > > > > > users > >> > > > >> > > > > > > > > >> > > understand when they're going to have to > >> make > >> > > > >> updates to > >> > > > >> > > > > their > >> > > > >> > > > > > > > > >> > applications. > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > > Less important but useful -- having a bit > >> of > >> > > > >> standard > >> > > > >> > > > > metadata > >> > > > >> > > > > > > like > >> > > > >> > > > > > > > > >> PEPs > >> > > > >> > > > > > > > > >> > > do. Two I think are important are status > >> (if > >> > > > >>someone > >> > > > >> > > lands > >> > > > >> > > > > on > >> > > > >> > > > > > > the > >> > > > >> > > > > > > > > KIP > >> > > > >> > > > > > > > > >> > page, > >> > > > >> > > > > > > > > >> > > can they tell whether this KIP was ever > >> > > > >>completed?) > >> > > > >> > > and/or > >> > > > >> > > > > the > >> > > > >> > > > > > > > > version > >> > > > >> > > > > > > > > >> > the > >> > > > >> > > > > > > > > >> > > KIP was first released in. > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > > On Fri, Jan 16, 2015 at 9:20 AM, Joel > >> Koshy < > >> > > > >> > > > > > > jjkosh...@gmail.com <javascript:;>> > >> > > > >> > > > > > > > > >> wrote: > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > >> I'm definitely +1 on the KIP concept. As > >> Joe > >> > > > >> > > mentioned, we > >> > > > >> > > > > > are > >> > > > >> > > > > > > > > already > >> > > > >> > > > > > > > > >> > >> doing this in one form or the other. > >> However, > >> > > > >>IMO > >> > > > >> it is > >> > > > >> > > > > > fairly > >> > > > >> > > > > > > ad > >> > > > >> > > > > > > > > hoc > >> > > > >> > > > > > > > > >> > >> - i.e., a combination of DISCUSS > >> threads, jira > >> > > > >> > > comments, RB > >> > > > >> > > > > > and > >> > > > >> > > > > > > > > code > >> > > > >> > > > > > > > > >> > >> comments, wikis and html documentation. > >> In the > >> > > > >> past I > >> > > > >> > > have > >> > > > >> > > > > > had > >> > > > >> > > > > > > to > >> > > > >> > > > > > > > > dig > >> > > > >> > > > > > > > > >> > >> into a bunch of these to try and figure > >> out > >> > > why > >> > > > >> > > something > >> > > > >> > > > > was > >> > > > >> > > > > > > > > >> > >> implemented a certain way. I think KIPs > >> can > >> > > > >>help a > >> > > > >> lot > >> > > > >> > > here > >> > > > >> > > > > > > first > >> > > > >> > > > > > > > > by > >> > > > >> > > > > > > > > >> > >> providing guidelines on what to think > >> about > >> > > > >> > > (compatibility, > >> > > > >> > > > > > new > >> > > > >> > > > > > > > > APIs, > >> > > > >> > > > > > > > > >> > >> etc.) when working through a major > >> feature; > >> > > and > >> > > > >> second > >> > > > >> > > by > >> > > > >> > > > > > > becoming > >> > > > >> > > > > > > > > a > >> > > > >> > > > > > > > > >> > >> crisp source of truth documentation for > >> new > >> > > > >> releases. > >> > > > >> > > > > E.g., > >> > > > >> > > > > > > for > >> > > > >> > > > > > > > > >> > >> feature X: see relevant KIPs: a, b, c, > >> etc. > >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> On Thu, Jan 15, 2015 at 08:11:20PM > >> -0800, Jay > >> > > > >>Kreps > >> > > > >> > > wrote: > >> > > > >> > > > > > > > > >> > >> > Hey Joe, > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > Yeah I guess the question is what is > >> the > >> > > > >> definition > >> > > > >> > > of > >> > > > >> > > > > > > major? I > >> > > > >> > > > > > > > > >> agree > >> > > > >> > > > > > > > > >> > we > >> > > > >> > > > > > > > > >> > >> > definitely don't want to generate a > >> bunch of > >> > > > >> > > paperwork. > >> > > > >> > > > > We > >> > > > >> > > > > > > have > >> > > > >> > > > > > > > > >> enough > >> > > > >> > > > > > > > > >> > >> > problems just getting all the > >> contributions > >> > > > >> reviewed > >> > > > >> > > and > >> > > > >> > > > > > > checked > >> > > > >> > > > > > > > > in > >> > > > >> > > > > > > > > >> > in a > >> > > > >> > > > > > > > > >> > >> > timely fashion... > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > So obviously bug fixes would not apply > >> here. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > I think it is also pretty clear that > >> big > >> > > > >>features > >> > > > >> > > should > >> > > > >> > > > > > get > >> > > > >> > > > > > > > > >> reviewed > >> > > > >> > > > > > > > > >> > and > >> > > > >> > > > > > > > > >> > >> > discussed. To pick on myself, for > >> example, > >> > > the > >> > > > >> log > >> > > > >> > > > > > compaction > >> > > > >> > > > > > > > > work > >> > > > >> > > > > > > > > >> was > >> > > > >> > > > > > > > > >> > >> done > >> > > > >> > > > > > > > > >> > >> > without enough public discussion about > >> how > >> > > it > >> > > > >> worked > >> > > > >> > > and > >> > > > >> > > > > > why > >> > > > >> > > > > > > > > >> (imho). I > >> > > > >> > > > > > > > > >> > >> > hope/claim that enough rigour in > >> thinking > >> > > > >>about > >> > > > >> > > use-cases > >> > > > >> > > > > > and > >> > > > >> > > > > > > > > >> > operations > >> > > > >> > > > > > > > > >> > >> > and so on was done that it turned out > >> well, > >> > > > >>but > >> > > > >> the > >> > > > >> > > > > > > discussion > >> > > > >> > > > > > > > > was > >> > > > >> > > > > > > > > >> > just > >> > > > >> > > > > > > > > >> > >> > between a few people with no real > >> public > >> > > > >>output. > >> > > > >> This > >> > > > >> > > > > kind > >> > > > >> > > > > > of > >> > > > >> > > > > > > > > >> feature > >> > > > >> > > > > > > > > >> > is > >> > > > >> > > > > > > > > >> > >> > clearly a big change and something we > >> should > >> > > > >> discuss. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > If we limit ourselves to just the > >> public > >> > > > >> contracts > >> > > > >> > > the > >> > > > >> > > > > KIP > >> > > > >> > > > > > > > > >> introduces > >> > > > >> > > > > > > > > >> > the > >> > > > >> > > > > > > > > >> > >> > discussion would just be on the new > >> configs > >> > > > >>and > >> > > > >> > > > > monitoring > >> > > > >> > > > > > > > > without > >> > > > >> > > > > > > > > >> > >> really a > >> > > > >> > > > > > > > > >> > >> > discussion of the design and how it > >> works > >> > > > >>which > >> > > > >> is > >> > > > >> > > > > > obviously > >> > > > >> > > > > > > > > closely > >> > > > >> > > > > > > > > >> > >> > related. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > I don't think this should be more work > >> > > > >>because in > >> > > > >> > > > > practice > >> > > > >> > > > > > > we are > >> > > > >> > > > > > > > > >> > making > >> > > > >> > > > > > > > > >> > >> > wiki pages for any big thing anyway. > >> So this > >> > > > >> would > >> > > > >> > > just > >> > > > >> > > > > be > >> > > > >> > > > > > a > >> > > > >> > > > > > > > > >> > consistent > >> > > > >> > > > > > > > > >> > >> way > >> > > > >> > > > > > > > > >> > >> > of numbering and structuring these > >> pages. > >> > > This > >> > > > >> would > >> > > > >> > > also > >> > > > >> > > > > > > give a > >> > > > >> > > > > > > > > >> clear > >> > > > >> > > > > > > > > >> > >> call > >> > > > >> > > > > > > > > >> > >> > to action: "hey kafka people, come > >> read my > >> > > > >>wiki > >> > > > >> and > >> > > > >> > > think > >> > > > >> > > > > > > this > >> > > > >> > > > > > > > > >> > through". > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > I actually thinking the voting aspect > >> is > >> > > less > >> > > > >> > > important. > >> > > > >> > > > > I > >> > > > >> > > > > > > think > >> > > > >> > > > > > > > > it > >> > > > >> > > > > > > > > >> is > >> > > > >> > > > > > > > > >> > >> > generally clear when there is > >> agreement on > >> > > > >> something > >> > > > >> > > and > >> > > > >> > > > > > > not. So > >> > > > >> > > > > > > > > >> from > >> > > > >> > > > > > > > > >> > my > >> > > > >> > > > > > > > > >> > >> > point of view we could actually just > >> > > eliminate > >> > > > >> that > >> > > > >> > > part > >> > > > >> > > > > if > >> > > > >> > > > > > > that > >> > > > >> > > > > > > > > is > >> > > > >> > > > > > > > > >> > too > >> > > > >> > > > > > > > > >> > >> > formal, it just seemed like a good way > >> to > >> > > > >> formally > >> > > > >> > > adopt > >> > > > >> > > > > > > > > something. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > To address some of your comments from > >> the > >> > > > >>wiki: > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > 1. This doesn't inhibit someone coming > >> along > >> > > > >>and > >> > > > >> > > putting > >> > > > >> > > > > > up a > >> > > > >> > > > > > > > > patch. > >> > > > >> > > > > > > > > >> > It > >> > > > >> > > > > > > > > >> > >> is > >> > > > >> > > > > > > > > >> > >> > just that when they do if it is a big > >> thing > >> > > > >> > > introducing > >> > > > >> > > > > new > >> > > > >> > > > > > > > > >> > functionality > >> > > > >> > > > > > > > > >> > >> > we would ask for a little discussion > >> on the > >> > > > >>basic > >> > > > >> > > > > > > > > feature/contracts > >> > > > >> > > > > > > > > >> > prior > >> > > > >> > > > > > > > > >> > >> > to code review. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > 2. We definitely definitely don't want > >> > > people > >> > > > >> > > generating > >> > > > >> > > > > a > >> > > > >> > > > > > > lot of > >> > > > >> > > > > > > > > >> > these > >> > > > >> > > > > > > > > >> > >> > things every time they have an idea > >> that > >> > > they > >> > > > >> aren't > >> > > > >> > > > > going > >> > > > >> > > > > > to > >> > > > >> > > > > > > > > >> > implement. > >> > > > >> > > > > > > > > >> > >> So > >> > > > >> > > > > > > > > >> > >> > this is only applicable to things you > >> > > > >>absolutely > >> > > > >> will > >> > > > >> > > > > check > >> > > > >> > > > > > > in > >> > > > >> > > > > > > > > code > >> > > > >> > > > > > > > > >> > for. > >> > > > >> > > > > > > > > >> > >> We > >> > > > >> > > > > > > > > >> > >> > also don't want to be making proposals > >> > > before > >> > > > >> things > >> > > > >> > > are > >> > > > >> > > > > > > thought > >> > > > >> > > > > > > > > >> > through, > >> > > > >> > > > > > > > > >> > >> > which often requires writing the code. > >> So I > >> > > > >> think the > >> > > > >> > > > > right > >> > > > >> > > > > > > time > >> > > > >> > > > > > > > > >> for a > >> > > > >> > > > > > > > > >> > >> KIP > >> > > > >> > > > > > > > > >> > >> > is when you are far enough along that > >> you > >> > > know > >> > > > >> the > >> > > > >> > > issues > >> > > > >> > > > > > and > >> > > > >> > > > > > > > > >> > tradeoffs > >> > > > >> > > > > > > > > >> > >> but > >> > > > >> > > > > > > > > >> > >> > not so far along that you are going to > >> be > >> > > > >>totally > >> > > > >> > > opposed > >> > > > >> > > > > > to > >> > > > >> > > > > > > any > >> > > > >> > > > > > > > > >> > change. > >> > > > >> > > > > > > > > >> > >> > Sometimes that is prior to writing any > >> code > >> > > > >>and > >> > > > >> > > sometimes > >> > > > >> > > > > > not > >> > > > >> > > > > > > > > until > >> > > > >> > > > > > > > > >> > you > >> > > > >> > > > > > > > > >> > >> are > >> > > > >> > > > > > > > > >> > >> > practically done. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > The key problem I see this fixing is > >> that > >> > > > >>there > >> > > > >> is > >> > > > >> > > enough > >> > > > >> > > > > > new > >> > > > >> > > > > > > > > >> > development > >> > > > >> > > > > > > > > >> > >> > happening that it is pretty hard for > >> > > everyone > >> > > > >>to > >> > > > >> > > review > >> > > > >> > > > > > every > >> > > > >> > > > > > > > > line > >> > > > >> > > > > > > > > >> of > >> > > > >> > > > > > > > > >> > >> every > >> > > > >> > > > > > > > > >> > >> > iteration of every patch. But all of us > >> > > > >>should be > >> > > > >> > > fully > >> > > > >> > > > > > > aware of > >> > > > >> > > > > > > > > new > >> > > > >> > > > > > > > > >> > >> > features, the ramifications, the new > >> public > >> > > > >> > > interfaces, > >> > > > >> > > > > > etc. > >> > > > >> > > > > > > If > >> > > > >> > > > > > > > > we > >> > > > >> > > > > > > > > >> > aren't > >> > > > >> > > > > > > > > >> > >> > aware of that we can't really build a > >> > > holistic > >> > > > >> system > >> > > > >> > > > > that > >> > > > >> > > > > > is > >> > > > >> > > > > > > > > >> > beautiful > >> > > > >> > > > > > > > > >> > >> and > >> > > > >> > > > > > > > > >> > >> > consistent across. So the idea is that > >> if > >> > > you > >> > > > >> fully > >> > > > >> > > > > review > >> > > > >> > > > > > > the > >> > > > >> > > > > > > > > KIPs > >> > > > >> > > > > > > > > >> > you > >> > > > >> > > > > > > > > >> > >> can > >> > > > >> > > > > > > > > >> > >> > be sure that even if you don't know > >> every > >> > > new > >> > > > >> line of > >> > > > >> > > > > code, > >> > > > >> > > > > > > you > >> > > > >> > > > > > > > > know > >> > > > >> > > > > > > > > >> > the > >> > > > >> > > > > > > > > >> > >> > major changes coming in. > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > -Jay > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > On Thu, Jan 15, 2015 at 12:18 PM, Joe > >> Stein > >> > > < > >> > > > >> > > > > > > > > joe.st...@stealth.ly <javascript:;>> > >> > > > >> > > > > > > > > >> > >> wrote: > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > Thanks Jay for kicking this off! I > >> think > >> > > the > >> > > > >> > > confluence > >> > > > >> > > > > > > page > >> > > > >> > > > > > > > > you > >> > > > >> > > > > > > > > >> > wrote > >> > > > >> > > > > > > > > >> > >> up > >> > > > >> > > > > > > > > >> > >> > > is a great start. > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > The KIP makes sense to me (at a > >> minimum) > >> > > if > >> > > > >> there > >> > > > >> > > is > >> > > > >> > > > > > going > >> > > > >> > > > > > > to > >> > > > >> > > > > > > > > be > >> > > > >> > > > > > > > > >> any > >> > > > >> > > > > > > > > >> > >> > > "breaking change". This way Kafka can > >> > > > >>continue > >> > > > >> to > >> > > > >> > > grow > >> > > > >> > > > > > and > >> > > > >> > > > > > > > > blossom > >> > > > >> > > > > > > > > >> > and > >> > > > >> > > > > > > > > >> > >> we > >> > > > >> > > > > > > > > >> > >> > > have a process in place if we are > >> going to > >> > > > >> release > >> > > > >> > > a > >> > > > >> > > > > > thorn > >> > > > >> > > > > > > ... > >> > > > >> > > > > > > > > and > >> > > > >> > > > > > > > > >> > >> when we > >> > > > >> > > > > > > > > >> > >> > > do it is *CLEAR* about what and why > >> that > >> > > is. > >> > > > >> We can > >> > > > >> > > > > > easily > >> > > > >> > > > > > > > > >> document > >> > > > >> > > > > > > > > >> > >> which > >> > > > >> > > > > > > > > >> > >> > > KIPs where involved with this release > >> > > > >>(which I > >> > > > >> > > think > >> > > > >> > > > > > > should get > >> > > > >> > > > > > > > > >> > >> committed > >> > > > >> > > > > > > > > >> > >> > > afterwards somewhere so no chance of > >> edit > >> > > > >>after > >> > > > >> > > > > release). > >> > > > >> > > > > > > This > >> > > > >> > > > > > > > > >> > >> approach I > >> > > > >> > > > > > > > > >> > >> > > had been thinking about also allows > >> > > changes > >> > > > >>to > >> > > > >> > > occur as > >> > > > >> > > > > > > they do > >> > > > >> > > > > > > > > >> now > >> > > > >> > > > > > > > > >> > as > >> > > > >> > > > > > > > > >> > >> long > >> > > > >> > > > > > > > > >> > >> > > as they are backwards compatible. > >> > > > >>Hopefully we > >> > > > >> > > never > >> > > > >> > > > > > need > >> > > > >> > > > > > > a > >> > > > >> > > > > > > > > KIP > >> > > > >> > > > > > > > > >> but > >> > > > >> > > > > > > > > >> > >> when > >> > > > >> > > > > > > > > >> > >> > > we do the PMC can vote on it and > >> folks can > >> > > > >> read the > >> > > > >> > > > > > release > >> > > > >> > > > > > > > > notes > >> > > > >> > > > > > > > > >> > with > >> > > > >> > > > > > > > > >> > >> > > *CLEAR* understanding what is going > >> to > >> > > break > >> > > > >> their > >> > > > >> > > > > > existing > >> > > > >> > > > > > > > > >> > setup... at > >> > > > >> > > > > > > > > >> > >> > > least that is how I have been > >> thinking > >> > > about > >> > > > >> it. > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > Let me know what you think about > >> this base > >> > > > >> minimum > >> > > > >> > > > > > > approach... > >> > > > >> > > > > > > > > I > >> > > > >> > > > > > > > > >> > hadn't > >> > > > >> > > > > > > > > >> > >> > > really thought of the KIP for *ANY* > >> "major > >> > > > >> change" > >> > > > >> > > and > >> > > > >> > > > > I > >> > > > >> > > > > > > have > >> > > > >> > > > > > > > > to > >> > > > >> > > > > > > > > >> > think > >> > > > >> > > > > > > > > >> > >> more > >> > > > >> > > > > > > > > >> > >> > > about that. I have some other > >> comments for > >> > > > >> minor > >> > > > >> > > items > >> > > > >> > > > > in > >> > > > >> > > > > > > the > >> > > > >> > > > > > > > > >> > >> confluence > >> > > > >> > > > > > > > > >> > >> > > page I will make once I think more > >> about > >> > > > >>how I > >> > > > >> feel > >> > > > >> > > > > > having > >> > > > >> > > > > > > a > >> > > > >> > > > > > > > > KIP > >> > > > >> > > > > > > > > >> for > >> > > > >> > > > > > > > > >> > >> more > >> > > > >> > > > > > > > > >> > >> > > than what I was thinking about > >> already. > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > I do think we should have "major > >> changes" > >> > > go > >> > > > >> > > through > >> > > > >> > > > > > > > > confluence, > >> > > > >> > > > > > > > > >> > >> mailing > >> > > > >> > > > > > > > > >> > >> > > list discuss and JIRA but kind of > >> feel we > >> > > > >>have > >> > > > >> been > >> > > > >> > > > > doing > >> > > > >> > > > > > > that > >> > > > >> > > > > > > > > >> > already > >> > > > >> > > > > > > > > >> > >> ... > >> > > > >> > > > > > > > > >> > >> > > if there are cases where that isn't > >> the > >> > > > >>case we > >> > > > >> > > should > >> > > > >> > > > > > > > > highlight > >> > > > >> > > > > > > > > >> and > >> > > > >> > > > > > > > > >> > >> learn > >> > > > >> > > > > > > > > >> > >> > > from them and formalize that more if > >> need > >> > > > >>be. > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > /******************************************* > >> > > > >> > > > > > > > > >> > >> > > Joe Stein > >> > > > >> > > > > > > > > >> > >> > > Founder, Principal Consultant > >> > > > >> > > > > > > > > >> > >> > > Big Data Open Source Security LLC > >> > > > >> > > > > > > > > >> > >> > > http://www.stealth.ly > >> > > > >> > > > > > > > > >> > >> > > Twitter: @allthingshadoop < > >> > > > >> > > > > > > > > >> http://www.twitter.com/allthingshadoop> > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >>********************************************/ > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > On Thu, Jan 15, 2015 at 1:42 PM, Jay > >> > > Kreps < > >> > > > >> > > > > > > > > jay.kr...@gmail.com <javascript:;>> > >> > > > >> > > > > > > > > >> > >> wrote: > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > The idea of KIPs came up in a > >> previous > >> > > > >> > > discussion but > >> > > > >> > > > > > > there > >> > > > >> > > > > > > > > was > >> > > > >> > > > > > > > > >> no > >> > > > >> > > > > > > > > >> > >> real > >> > > > >> > > > > > > > > >> > >> > > > crisp definition of what they > >> were. Here > >> > > > >>is > >> > > > >> an > >> > > > >> > > > > attempt > >> > > > >> > > > > > at > >> > > > >> > > > > > > > > >> > defining a > >> > > > >> > > > > > > > > >> > >> > > > process: > >> > > > >> > > > > > > > > >> > >> > > > > >> > > > >> > > > > > > > > >> > >> > > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > >> > > > >> > >> > > > >> > >> > > > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Propo > >> > > > >>sals > >> > > > >> > > > > > > > > >> > >> > > > > >> > > > >> > > > > > > > > >> > >> > > > The trick here is to have something > >> > > > >> light-weight > >> > > > >> > > > > enough > >> > > > >> > > > > > > that > >> > > > >> > > > > > > > > it > >> > > > >> > > > > > > > > >> > >> isn't a > >> > > > >> > > > > > > > > >> > >> > > > hassle for small changes, but > >> enough so > >> > > > >>that > >> > > > >> > > changes > >> > > > >> > > > > > get > >> > > > >> > > > > > > the > >> > > > >> > > > > > > > > >> > >> eyeballs of > >> > > > >> > > > > > > > > >> > >> > > > the committers and heavy users. > >> > > > >> > > > > > > > > >> > >> > > > > >> > > > >> > > > > > > > > >> > >> > > > Thoughts? > >> > > > >> > > > > > > > > >> > >> > > > > >> > > > >> > > > > > > > > >> > >> > > > -Jay > >> > > > >> > > > > > > > > >> > >> > > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > > > >> > > > >> > > > > > > > > >> > > -- > >> > > > >> > > > > > > > > >> > > Thanks, > >> > > > >> > > > > > > > > >> > > Ewen > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> -- > >> > > > >> > > > > > > > > >> -- Guozhang > >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > -- > >> > > > >> > > > > Thanks, > >> > > > >> > > > > Neha > >> > > > >> > > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > > >> > > Joel > >> > > > >> > > > >> > > > >> > >> > > > >> > >> > > > > >> > > > >> > > >> > > >> > -- > >> > Sent from Gmail Mobile > >> > >> > >
