A problem I am having is actually understanding which KIPs are intended to be complete proposals and which are works in progress. Joe you seem to have a bunch of these. Can you move them elsewhere until they are really fully done and ready for review and discussion?
-Jay On Fri, Feb 6, 2015 at 12:09 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > I think we are focused on making committing new changes easier, but what > we have seen is actually that isn't the bulk of the work (especially with > this kind of "public interface" change where it generally has a big user > impact). I think we actually really need the core committers and any other > interested parties to stop and fully read each KIP and think about it. If > we don't have time to do that we usually just end up spending a lot more > time after the fact trying to rework things latter when it is a lot harder. > So I really think we should have every active committer read, comment, and > vote on each KIP. I think this may require a little bit of work to > co-ordinate/bug people but will end up being worth it because each person > on the project will have a holistic picture of what is going on. > > -Jay > > On Thu, Feb 5, 2015 at 11:24 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > >> Just wanted to add a few more comments on this: KIPs were suggested as >> a process to help reach early consensus on a major change or not so >> major (but tricky or backward incompatible) change in order to reduce >> the likelihood of multiple iterations and complete rewrites during >> code reviews (which is time-intensive for both the contributor and >> reviewers); as well as to reduce the likelihood of surprises (say, if >> a patch inadvertently changes a public API). So KIPs are intended to >> speed up development since a clear path is charted out and there is >> prior consensus on whether a feature and its design/implementation >> make sense or not. >> >> Obviously this breaks down if KIPs are not being actively discussed - >> again I think we can do much better here. I think we ended up with a >> backlog because as soon as the KIP wiki was started, a number of >> pre-existing jiras and discussions were moved there - all within a few >> days. Now that there are quite a few outstanding KIPs I think we just >> need to methodically work through those - preferably a couple at a >> time. I looked through the list and I think we should be able to >> resolve all of them relatively quickly if everyone is on board with >> this. >> >> > > Its probably more helpful for contributors if its "lazy" as in "no >> > > strong objections" . >> >> Gwen also suggested this and this also sounds ok to me as I wrote >> earlier - what do others think? This is important especially if >> majority in the community think if this less restrictive policy would >> spur and not hinder development - I'm not sure that it does. I >> completely agree that KIPs fail to a large degree as far as the >> original motivation goes if they require a lazy majority but the >> DISCUSS threads are stalled. IOW regardless of that discussion, I >> think we should rejuvenate some of those threads especially now that >> 0.8.2 is out of the way. >> >> Thanks, >> >> Joel >> >> On Thu, Feb 05, 2015 at 08:56:13PM -0800, Joel Koshy wrote: >> > I'm just thinking aloud - I don't know what a good number would be, and >> it >> > is just one possibility to streamline how KIPs are processed. It largely >> > depends on how complex the proposals are. What would be concerning is if >> > there are 10 different threads all dealing with large KIPs and no one >> has >> > the time to give due diligence to each one and all those threads grind >> to a >> > halt due to confusion, incomplete context and misunderstandings. >> > >> > On Thursday, February 5, 2015, Harsha <ka...@harsha.io> wrote: >> > >> > > Joel, >> > > Having only 2 or 3 KIPS under active discussion is concerning. >> > > This will slow down development process as well. >> > > Having a turn-around time for a KIP is a good idea but what will >> happen >> > > if it didn't received required votes within that time frame. >> > > Its probably more helpful for contributors if its "lazy" as in "no >> > > strong objections" . >> > > Just to make sure this is only for KIPs not for regular bug fixes >> right? >> > > Thanks, >> > > Harsha >> > > >> > > >> > > >> > > >> > > >> > > On Thu, Feb 5, 2015, at 05:59 PM, Jiangjie Qin wrote: >> > > > Iąm having an impression that KIP is mostly for new features but >> not for >> > > > bug fixes. But I agree with Joel that it might make sense to have >> some >> > > > big >> > > > patches, even if they are bug fixes, to follow the KIP like process >> which >> > > > is more strict. >> > > > >> > > > Jiangjie (Becket) Qin >> > > > >> > > > On 2/5/15, 4:57 PM, "Gwen Shapira" <gshap...@cloudera.com >> <javascript:;>> >> > > wrote: >> > > > >> > > > >> >> > > > >> >> > > > >> Yes there are KIPs that are currently blocked on feedback/votes, >> but I >> > > > >> don't think it is an issue of not caring to comment vs having so >> many >> > > > >> KIPs and other code reviews in flight that people are just >> swamped. >> > > > >> >> > > > >> >> > > > >This is exactly my concern. >> > > > >Even now important patches and features have very long development >> and >> > > > >review cycles due to Kafka's small and very busy committer >> community. I >> > > > >feel that this change takes things in the wrong direction >> > > > > >> > > > > >> > > > >> Joel >> > > > >> >> > > > >> On Thu, Feb 05, 2015 at 04:19:54PM -0800, Gwen Shapira wrote: >> > > > >> > Isn't requiring 3 binding votes a bit overly strict here? We >> are >> > > > >>talking >> > > > >> > about patches which in can be fixed, reverted, etc. Not >> releases, >> > > > >>which >> > > > >> > have legal implications. >> > > > >> > >> > > > >> > Why not go with usual definition: "lazy" = "No strong >> objections for >> > > > >>few >> > > > >> > days"? >> > > > >> > This means contributors will not be blocked on issues where no >> one >> > > > >>cares >> > > > >> to >> > > > >> > comment (and we had few of those). >> > > > >> > >> > > > >> > Gwen >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > On Thu, Feb 5, 2015 at 4:14 PM, Joel Koshy < >> jjkosh...@gmail.com >> > > <javascript:;>> >> > > > >>wrote: >> > > > >> > >> > > > >> > > Sorry about this - I actually meant to suggest lazy consensus >> > > (which >> > > > >> > > is a stronger requirement): "3 binding +1 votes and no >> binding >> > > > >> > > vetoes." >> > > > >> > > >> > > > >> > > I have updated the wiki to lazy consensus - but can change >> it back >> > > > >>if >> > > > >> > > there is a reasonable objection. >> > > > >> > > >> > > > >> > > On Thu, Feb 05, 2015 at 06:17:44PM -0500, Joe Stein wrote: >> > > > >> > > > +1 >> > > > >> > > > >> > > > >> > > > On Thu, Feb 5, 2015 at 6:16 PM, Neha Narkhede < >> > > n...@confluent.io <javascript:;>> >> > > > >> wrote: >> > > > >> > > > >> > > > >> > > > > Sounds good. >> > > > >> > > > > >> > > > >> > > > > On Thu, Feb 5, 2015 at 2:35 PM, Jay Kreps < >> > > jay.kr...@gmail.com <javascript:;>> >> > > > >> wrote: >> > > > >> > > > > >> > > > >> > > > > > None on my part. >> > > > >> > > > > > >> > > > >> > > > > > -Jay >> > > > >> > > > > > >> > > > >> > > > > > On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy >> > > > >><jjkosh...@gmail.com <javascript:;> >> > > > >> > >> > > > >> > > wrote: >> > > > >> > > > > > >> > > > >> > > > > > > One amendment I would like to bring up for >> consideration >> > > wrt >> > > > >> the >> > > > >> > > KIP >> > > > >> > > > > > > process (before we formally include it in our >> by-laws) is >> > > to >> > > > >> not >> > > > >> > > > > > > restrict the votes to be a lazy majority of the PMC, >> and >> > > to >> > > > >> instead >> > > > >> > > > > > > make it a lazy majority of committers. >> > > > >> > > > > > > >> > > > >> > > > > > > Our current requirement for code changes per our >> by-laws >> > > > >>are +1 >> > > > >> > > from a >> > > > >> > > > > > > committer (who is not the contributor) followed by >> lazy >> > > > >> approval. I >> > > > >> > > > > > > think a lazy majority vote for more significant code >> > > changes >> > > > >> > > (i.e., a >> > > > >> > > > > > > KIP) should be sufficient. >> > > > >> > > > > > > >> > > > >> > > > > > > Any objection to this? >> > > > >> > > > > > > >> > > > >> > > > > > > On Sun, Jan 18, 2015 at 10:31:08AM -0800, Jay Kreps >> wrote: >> > > > >> > > > > > > > Great! Sounds like everyone is on the same page >> > > > >> > > > > > > > >> > > > >> > > > > > > > - I created a template page to make things >> easier. If >> > > > >>you >> > > > >> do >> > > > >> > > > > > > Tools->Copy >> > > > >> > > > > > > > on this page you can just fill in the italic >> portions >> > > > >>with >> > > > >> > > your >> > > > >> > > > > > > details. >> > > > >> > > > > > > > - I retrofitted KIP-1 to match this formatting >> > > > >> > > > > > > > - I added the metadata section people asked for >> (a >> > > link >> > > > >> to the >> > > > >> > > > > > > > discussion, the JIRA, and the current status). >> Let's >> > > > >>make >> > > > >> > > sure we >> > > > >> > > > > > > remember >> > > > >> > > > > > > > to update the current status as things are >> figured >> > > out. >> > > > >> > > > > > > > - Let's keep the discussion on the mailing list >> > > rather >> > > > >> than >> > > > >> > > on the >> > > > >> > > > > > > wiki >> > > > >> > > > > > > > pages. It makes sense to do one or the other so >> all >> > > the >> > > > >> > > comments >> > > > >> > > > > are >> > > > >> > > > > > > in one >> > > > >> > > > > > > > place and I think prior experience is that the >> wiki >> > > > >> comments >> > > > >> > > are >> > > > >> > > > > the >> > > > >> > > > > > > worse >> > > > >> > > > > > > > way. >> > > > >> > > > > > > > >> > > > >> > > > > > > > I think it would be great do KIPs for some of the >> > > > >>in-flight >> > > > >> items >> > > > >> > > > > folks >> > > > >> > > > > > > > mentioned. >> > > > >> > > > > > > > >> > > > >> > > > > > > > -Jay >> > > > >> > > > > > > > >> > > > >> > > > > > > > On Sat, Jan 17, 2015 at 8:23 AM, Gwen Shapira < >> > > > >> > > gshap...@cloudera.com <javascript:;> >> > > > >> > > > > > >> > > > >> > > > > > > wrote: >> > > > >> > > > > > > > >> > > > >> > > > > > > > > +1 >> > > > >> > > > > > > > > >> > > > >> > > > > > > > > Will be happy to provide a KIP for the >> > > > >>multiple-listeners >> > > > >> > > patch. >> > > > >> > > > > > > > > >> > > > >> > > > > > > > > Gwen >> > > > >> > > > > > > > > >> > > > >> > > > > > > > > On Sat, Jan 17, 2015 at 8:10 AM, Joe Stein < >> > > > >> > > joe.st...@stealth.ly <javascript:;>> >> > > > >> > > > > > > wrote: >> > > > >> > > > > > > > > > +1 to everything we have been saying and where >> this >> > > > >>(has >> > > > >> > > settled >> > > > >> > > > > > > to)/(is >> > > > >> > > > > > > > > > settling to). >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > I am sure other folks have some more feedback >> and >> > > > >>think >> > > > >> we >> > > > >> > > should >> > > > >> > > > > > > try to >> > > > >> > > > > > > > > > keep this discussion going if need be. I am >> also a >> > > > >>firm >> > > > >> > > believer >> > > > >> > > > > of >> > > > >> > > > > > > form >> > > > >> > > > > > > > > > following function so kicking the tires some to >> > > flesh >> > > > >> out the >> > > > >> > > > > > > details of >> > > > >> > > > > > > > > > this and have some organic growth with the >> process >> > > > >>will >> > > > >> be >> > > > >> > > > > healthy >> > > > >> > > > > > > and >> > > > >> > > > > > > > > > beneficial to the community. >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > For my part, what I will do is open a few KIP >> based >> > > on >> > > > >> some >> > > > >> > > of >> > > > >> > > > > the >> > > > >> > > > > > > work I >> > > > >> > > > > > > > > > have been involved with for 0.8.3. Off the top >> of my >> > > > >>head >> > > > >> > > this >> > > > >> > > > > > would >> > > > >> > > > > > > > > > include 1) changes to re-assignment of >> partitions 2) >> > > > >> kafka >> > > > >> > > cli 3) >> > > > >> > > > > > > global >> > > > >> > > > > > > > > > configs 4) security white list black list by >> ip 5) >> > > SSL >> > > > >> 6) We >> > > > >> > > > > > probably >> > > > >> > > > > > > > > will >> > > > >> > > > > > > > > > have lots of Security related KIPs and should >> treat >> > > > >>them >> > > > >> all >> > > > >> > > > > > > individually >> > > > >> > > > > > > > > > when the time is appropriate 7) Kafka on Mesos. >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > If someone else wants to jump in to start >> getting >> > > some >> > > > >> of the >> > > > >> > > > > > > security >> > > > >> > > > > > > > > KIP >> > > > >> > > > > > > > > > that we are going to have in 0.8.3 I think that >> > > would >> > > > >>be >> > > > >> > > great >> > > > >> > > > > > (e.g. >> > > > >> > > > > > > > > > Multiple Listeners for Kafka Brokers). There >> are >> > > also >> > > > >>a >> > > > >> few >> > > > >> > > other >> > > > >> > > > > > > > > tickets I >> > > > >> > > > > > > > > > can think of that are important to have in the >> code >> > > in >> > > > >> 0.8.3 >> > > > >> > > that >> > > > >> > > > > > > should >> > > > >> > > > > > > > > > have KIP also that I haven't really been >> involved >> > > in. >> > > > >>I >> > > > >> will >> > > > >> > > > > take a >> > > > >> > > > > > > few >> > > > >> > > > > > > > > > minutes and go through JIRA (one I can think >> of like >> > > > >>auto >> > > > >> > > assign >> > > > >> > > > > id >> > > > >> > > > > > > that >> > > > >> > > > > > > > > is >> > > > >> > > > > > > > > > already committed I think) and ask for a KIP if >> > > > >> appropriate >> > > > >> > > or >> > > > >> > > > > if I >> > > > >> > > > > > > feel >> > > > >> > > > > > > > > > that I can write it up (both from a time and >> > > > >> understanding >> > > > >> > > > > > > perspective) >> > > > >> > > > > > > > > do >> > > > >> > > > > > > > > > so. >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > long story short, I encourage folks to start >> moving >> > > > >>ahead >> > > > >> > > with >> > > > >> > > > > the >> > > > >> > > > > > > KIP >> > > > >> > > > > > > > > for >> > > > >> > > > > > > > > > 0.8.3 as how we operate. any objections? >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > > On Fri, Jan 16, 2015 at 2:40 PM, Guozhang Wang >> < >> > > > >> > > > > wangg...@gmail.com <javascript:;> >> > > > >> > > > > > > >> > > > >> > > > > > > > > wrote: >> > > > >> > > > > > > > > > >> > > > >> > > > > > > > > >> +1 on the idea, and we could mutually link >> the KIP >> > > > >>wiki >> > > > >> page >> > > > >> > > > > with >> > > > >> > > > > > > the >> > > > >> > > > > > > > > the >> > > > >> > > > > > > > > >> created JIRA ticket (i.e. include the JIRA >> number >> > > on >> > > > >>the >> > > > >> > > page >> > > > >> > > > > and >> > > > >> > > > > > > the >> > > > >> > > > > > > > > KIP >> > > > >> > > > > > > > > >> url on the ticket description). >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> Regarding the KIP process, probably we do not >> need >> > > > >>two >> > > > >> phase >> > > > >> > > > > > > > > communication >> > > > >> > > > > > > > > >> of a [DISCUSS] followed by [VOTE], as Jay >> said the >> > > > >> voting >> > > > >> > > should >> > > > >> > > > > > be >> > > > >> > > > > > > > > clear >> > > > >> > > > > > > > > >> while people discuss about that. >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> About who should trigger the process, I think >> the >> > > > >>only >> > > > >> > > involved >> > > > >> > > > > > > people >> > > > >> > > > > > > > > >> would be 1) when the patch is submitted / or >> even >> > > the >> > > > >> > > ticket is >> > > > >> > > > > > > created, >> > > > >> > > > > > > > > >> the assignee could choose to start the KIP >> process >> > > if >> > > > >> she >> > > > >> > > > > thought >> > > > >> > > > > > > it is >> > > > >> > > > > > > > > >> necessary; 2) the reviewer of the patch can >> also >> > > > >>suggest >> > > > >> > > > > starting >> > > > >> > > > > > > KIP >> > > > >> > > > > > > > > >> discussions. >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> On Fri, Jan 16, 2015 at 10:49 AM, Gwen >> Shapira < >> > > > >> > > > > > > gshap...@cloudera.com <javascript:;>> >> > > > >> > > > > > > > > >> wrote: >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> > +1 to Ewen's suggestions: Deprecation, >> status and >> > > > >> version. >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> > Perhaps add the JIRA where the KIP was >> > > implemented >> > > > >>to >> > > > >> the >> > > > >> > > > > > > metadata. >> > > > >> > > > > > > > > >> > This will help tie things together. >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> > On Fri, Jan 16, 2015 at 9:35 AM, Ewen >> > > > >>Cheslack-Postava >> > > > >> > > > > > > > > >> > <e...@confluent.io <javascript:;>> wrote: >> > > > >> > > > > > > > > >> > > I think adding a section about deprecation >> > > would >> > > > >>be >> > > > >> > > > > helpful. A >> > > > >> > > > > > > good >> > > > >> > > > > > > > > >> > > fraction of the time I would expect the >> goal >> > > of a >> > > > >> KIP >> > > > >> > > is to >> > > > >> > > > > > fix >> > > > >> > > > > > > or >> > > > >> > > > > > > > > >> > replace >> > > > >> > > > > > > > > >> > > older functionality that needs continued >> > > support >> > > > >>for >> > > > >> > > > > > > compatibility, >> > > > >> > > > > > > > > but >> > > > >> > > > > > > > > >> > > should eventually be phased out. This >> helps >> > > Kafka >> > > > >> devs >> > > > >> > > > > > > understand >> > > > >> > > > > > > > > how >> > > > >> > > > > > > > > >> > long >> > > > >> > > > > > > > > >> > > they'll end up supporting multiple >> versions of >> > > > >> features >> > > > >> > > and >> > > > >> > > > > > > helps >> > > > >> > > > > > > > > users >> > > > >> > > > > > > > > >> > > understand when they're going to have to >> make >> > > > >> updates to >> > > > >> > > > > their >> > > > >> > > > > > > > > >> > applications. >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > > Less important but useful -- having a bit >> of >> > > > >> standard >> > > > >> > > > > metadata >> > > > >> > > > > > > like >> > > > >> > > > > > > > > >> PEPs >> > > > >> > > > > > > > > >> > > do. Two I think are important are status >> (if >> > > > >>someone >> > > > >> > > lands >> > > > >> > > > > on >> > > > >> > > > > > > the >> > > > >> > > > > > > > > KIP >> > > > >> > > > > > > > > >> > page, >> > > > >> > > > > > > > > >> > > can they tell whether this KIP was ever >> > > > >>completed?) >> > > > >> > > and/or >> > > > >> > > > > the >> > > > >> > > > > > > > > version >> > > > >> > > > > > > > > >> > the >> > > > >> > > > > > > > > >> > > KIP was first released in. >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > > On Fri, Jan 16, 2015 at 9:20 AM, Joel >> Koshy < >> > > > >> > > > > > > jjkosh...@gmail.com <javascript:;>> >> > > > >> > > > > > > > > >> wrote: >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > >> I'm definitely +1 on the KIP concept. As >> Joe >> > > > >> > > mentioned, we >> > > > >> > > > > > are >> > > > >> > > > > > > > > already >> > > > >> > > > > > > > > >> > >> doing this in one form or the other. >> However, >> > > > >>IMO >> > > > >> it is >> > > > >> > > > > > fairly >> > > > >> > > > > > > ad >> > > > >> > > > > > > > > hoc >> > > > >> > > > > > > > > >> > >> - i.e., a combination of DISCUSS >> threads, jira >> > > > >> > > comments, RB >> > > > >> > > > > > and >> > > > >> > > > > > > > > code >> > > > >> > > > > > > > > >> > >> comments, wikis and html documentation. >> In the >> > > > >> past I >> > > > >> > > have >> > > > >> > > > > > had >> > > > >> > > > > > > to >> > > > >> > > > > > > > > dig >> > > > >> > > > > > > > > >> > >> into a bunch of these to try and figure >> out >> > > why >> > > > >> > > something >> > > > >> > > > > was >> > > > >> > > > > > > > > >> > >> implemented a certain way. I think KIPs >> can >> > > > >>help a >> > > > >> lot >> > > > >> > > here >> > > > >> > > > > > > first >> > > > >> > > > > > > > > by >> > > > >> > > > > > > > > >> > >> providing guidelines on what to think >> about >> > > > >> > > (compatibility, >> > > > >> > > > > > new >> > > > >> > > > > > > > > APIs, >> > > > >> > > > > > > > > >> > >> etc.) when working through a major >> feature; >> > > and >> > > > >> second >> > > > >> > > by >> > > > >> > > > > > > becoming >> > > > >> > > > > > > > > a >> > > > >> > > > > > > > > >> > >> crisp source of truth documentation for >> new >> > > > >> releases. >> > > > >> > > > > E.g., >> > > > >> > > > > > > for >> > > > >> > > > > > > > > >> > >> feature X: see relevant KIPs: a, b, c, >> etc. >> > > > >> > > > > > > > > >> > >> >> > > > >> > > > > > > > > >> > >> On Thu, Jan 15, 2015 at 08:11:20PM >> -0800, Jay >> > > > >>Kreps >> > > > >> > > wrote: >> > > > >> > > > > > > > > >> > >> > Hey Joe, >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > Yeah I guess the question is what is >> the >> > > > >> definition >> > > > >> > > of >> > > > >> > > > > > > major? I >> > > > >> > > > > > > > > >> agree >> > > > >> > > > > > > > > >> > we >> > > > >> > > > > > > > > >> > >> > definitely don't want to generate a >> bunch of >> > > > >> > > paperwork. >> > > > >> > > > > We >> > > > >> > > > > > > have >> > > > >> > > > > > > > > >> enough >> > > > >> > > > > > > > > >> > >> > problems just getting all the >> contributions >> > > > >> reviewed >> > > > >> > > and >> > > > >> > > > > > > checked >> > > > >> > > > > > > > > in >> > > > >> > > > > > > > > >> > in a >> > > > >> > > > > > > > > >> > >> > timely fashion... >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > So obviously bug fixes would not apply >> here. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > I think it is also pretty clear that >> big >> > > > >>features >> > > > >> > > should >> > > > >> > > > > > get >> > > > >> > > > > > > > > >> reviewed >> > > > >> > > > > > > > > >> > and >> > > > >> > > > > > > > > >> > >> > discussed. To pick on myself, for >> example, >> > > the >> > > > >> log >> > > > >> > > > > > compaction >> > > > >> > > > > > > > > work >> > > > >> > > > > > > > > >> was >> > > > >> > > > > > > > > >> > >> done >> > > > >> > > > > > > > > >> > >> > without enough public discussion about >> how >> > > it >> > > > >> worked >> > > > >> > > and >> > > > >> > > > > > why >> > > > >> > > > > > > > > >> (imho). I >> > > > >> > > > > > > > > >> > >> > hope/claim that enough rigour in >> thinking >> > > > >>about >> > > > >> > > use-cases >> > > > >> > > > > > and >> > > > >> > > > > > > > > >> > operations >> > > > >> > > > > > > > > >> > >> > and so on was done that it turned out >> well, >> > > > >>but >> > > > >> the >> > > > >> > > > > > > discussion >> > > > >> > > > > > > > > was >> > > > >> > > > > > > > > >> > just >> > > > >> > > > > > > > > >> > >> > between a few people with no real >> public >> > > > >>output. >> > > > >> This >> > > > >> > > > > kind >> > > > >> > > > > > of >> > > > >> > > > > > > > > >> feature >> > > > >> > > > > > > > > >> > is >> > > > >> > > > > > > > > >> > >> > clearly a big change and something we >> should >> > > > >> discuss. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > If we limit ourselves to just the >> public >> > > > >> contracts >> > > > >> > > the >> > > > >> > > > > KIP >> > > > >> > > > > > > > > >> introduces >> > > > >> > > > > > > > > >> > the >> > > > >> > > > > > > > > >> > >> > discussion would just be on the new >> configs >> > > > >>and >> > > > >> > > > > monitoring >> > > > >> > > > > > > > > without >> > > > >> > > > > > > > > >> > >> really a >> > > > >> > > > > > > > > >> > >> > discussion of the design and how it >> works >> > > > >>which >> > > > >> is >> > > > >> > > > > > obviously >> > > > >> > > > > > > > > closely >> > > > >> > > > > > > > > >> > >> > related. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > I don't think this should be more work >> > > > >>because in >> > > > >> > > > > practice >> > > > >> > > > > > > we are >> > > > >> > > > > > > > > >> > making >> > > > >> > > > > > > > > >> > >> > wiki pages for any big thing anyway. >> So this >> > > > >> would >> > > > >> > > just >> > > > >> > > > > be >> > > > >> > > > > > a >> > > > >> > > > > > > > > >> > consistent >> > > > >> > > > > > > > > >> > >> way >> > > > >> > > > > > > > > >> > >> > of numbering and structuring these >> pages. >> > > This >> > > > >> would >> > > > >> > > also >> > > > >> > > > > > > give a >> > > > >> > > > > > > > > >> clear >> > > > >> > > > > > > > > >> > >> call >> > > > >> > > > > > > > > >> > >> > to action: "hey kafka people, come >> read my >> > > > >>wiki >> > > > >> and >> > > > >> > > think >> > > > >> > > > > > > this >> > > > >> > > > > > > > > >> > through". >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > I actually thinking the voting aspect >> is >> > > less >> > > > >> > > important. >> > > > >> > > > > I >> > > > >> > > > > > > think >> > > > >> > > > > > > > > it >> > > > >> > > > > > > > > >> is >> > > > >> > > > > > > > > >> > >> > generally clear when there is >> agreement on >> > > > >> something >> > > > >> > > and >> > > > >> > > > > > > not. So >> > > > >> > > > > > > > > >> from >> > > > >> > > > > > > > > >> > my >> > > > >> > > > > > > > > >> > >> > point of view we could actually just >> > > eliminate >> > > > >> that >> > > > >> > > part >> > > > >> > > > > if >> > > > >> > > > > > > that >> > > > >> > > > > > > > > is >> > > > >> > > > > > > > > >> > too >> > > > >> > > > > > > > > >> > >> > formal, it just seemed like a good way >> to >> > > > >> formally >> > > > >> > > adopt >> > > > >> > > > > > > > > something. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > To address some of your comments from >> the >> > > > >>wiki: >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > 1. This doesn't inhibit someone coming >> along >> > > > >>and >> > > > >> > > putting >> > > > >> > > > > > up a >> > > > >> > > > > > > > > patch. >> > > > >> > > > > > > > > >> > It >> > > > >> > > > > > > > > >> > >> is >> > > > >> > > > > > > > > >> > >> > just that when they do if it is a big >> thing >> > > > >> > > introducing >> > > > >> > > > > new >> > > > >> > > > > > > > > >> > functionality >> > > > >> > > > > > > > > >> > >> > we would ask for a little discussion >> on the >> > > > >>basic >> > > > >> > > > > > > > > feature/contracts >> > > > >> > > > > > > > > >> > prior >> > > > >> > > > > > > > > >> > >> > to code review. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > 2. We definitely definitely don't want >> > > people >> > > > >> > > generating >> > > > >> > > > > a >> > > > >> > > > > > > lot of >> > > > >> > > > > > > > > >> > these >> > > > >> > > > > > > > > >> > >> > things every time they have an idea >> that >> > > they >> > > > >> aren't >> > > > >> > > > > going >> > > > >> > > > > > to >> > > > >> > > > > > > > > >> > implement. >> > > > >> > > > > > > > > >> > >> So >> > > > >> > > > > > > > > >> > >> > this is only applicable to things you >> > > > >>absolutely >> > > > >> will >> > > > >> > > > > check >> > > > >> > > > > > > in >> > > > >> > > > > > > > > code >> > > > >> > > > > > > > > >> > for. >> > > > >> > > > > > > > > >> > >> We >> > > > >> > > > > > > > > >> > >> > also don't want to be making proposals >> > > before >> > > > >> things >> > > > >> > > are >> > > > >> > > > > > > thought >> > > > >> > > > > > > > > >> > through, >> > > > >> > > > > > > > > >> > >> > which often requires writing the code. >> So I >> > > > >> think the >> > > > >> > > > > right >> > > > >> > > > > > > time >> > > > >> > > > > > > > > >> for a >> > > > >> > > > > > > > > >> > >> KIP >> > > > >> > > > > > > > > >> > >> > is when you are far enough along that >> you >> > > know >> > > > >> the >> > > > >> > > issues >> > > > >> > > > > > and >> > > > >> > > > > > > > > >> > tradeoffs >> > > > >> > > > > > > > > >> > >> but >> > > > >> > > > > > > > > >> > >> > not so far along that you are going to >> be >> > > > >>totally >> > > > >> > > opposed >> > > > >> > > > > > to >> > > > >> > > > > > > any >> > > > >> > > > > > > > > >> > change. >> > > > >> > > > > > > > > >> > >> > Sometimes that is prior to writing any >> code >> > > > >>and >> > > > >> > > sometimes >> > > > >> > > > > > not >> > > > >> > > > > > > > > until >> > > > >> > > > > > > > > >> > you >> > > > >> > > > > > > > > >> > >> are >> > > > >> > > > > > > > > >> > >> > practically done. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > The key problem I see this fixing is >> that >> > > > >>there >> > > > >> is >> > > > >> > > enough >> > > > >> > > > > > new >> > > > >> > > > > > > > > >> > development >> > > > >> > > > > > > > > >> > >> > happening that it is pretty hard for >> > > everyone >> > > > >>to >> > > > >> > > review >> > > > >> > > > > > every >> > > > >> > > > > > > > > line >> > > > >> > > > > > > > > >> of >> > > > >> > > > > > > > > >> > >> every >> > > > >> > > > > > > > > >> > >> > iteration of every patch. But all of us >> > > > >>should be >> > > > >> > > fully >> > > > >> > > > > > > aware of >> > > > >> > > > > > > > > new >> > > > >> > > > > > > > > >> > >> > features, the ramifications, the new >> public >> > > > >> > > interfaces, >> > > > >> > > > > > etc. >> > > > >> > > > > > > If >> > > > >> > > > > > > > > we >> > > > >> > > > > > > > > >> > aren't >> > > > >> > > > > > > > > >> > >> > aware of that we can't really build a >> > > holistic >> > > > >> system >> > > > >> > > > > that >> > > > >> > > > > > is >> > > > >> > > > > > > > > >> > beautiful >> > > > >> > > > > > > > > >> > >> and >> > > > >> > > > > > > > > >> > >> > consistent across. So the idea is that >> if >> > > you >> > > > >> fully >> > > > >> > > > > review >> > > > >> > > > > > > the >> > > > >> > > > > > > > > KIPs >> > > > >> > > > > > > > > >> > you >> > > > >> > > > > > > > > >> > >> can >> > > > >> > > > > > > > > >> > >> > be sure that even if you don't know >> every >> > > new >> > > > >> line of >> > > > >> > > > > code, >> > > > >> > > > > > > you >> > > > >> > > > > > > > > know >> > > > >> > > > > > > > > >> > the >> > > > >> > > > > > > > > >> > >> > major changes coming in. >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > -Jay >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > On Thu, Jan 15, 2015 at 12:18 PM, Joe >> Stein >> > > < >> > > > >> > > > > > > > > joe.st...@stealth.ly <javascript:;>> >> > > > >> > > > > > > > > >> > >> wrote: >> > > > >> > > > > > > > > >> > >> > >> > > > >> > > > > > > > > >> > >> > > Thanks Jay for kicking this off! I >> think >> > > the >> > > > >> > > confluence >> > > > >> > > > > > > page >> > > > >> > > > > > > > > you >> > > > >> > > > > > > > > >> > wrote >> > > > >> > > > > > > > > >> > >> up >> > > > >> > > > > > > > > >> > >> > > is a great start. >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > The KIP makes sense to me (at a >> minimum) >> > > if >> > > > >> there >> > > > >> > > is >> > > > >> > > > > > going >> > > > >> > > > > > > to >> > > > >> > > > > > > > > be >> > > > >> > > > > > > > > >> any >> > > > >> > > > > > > > > >> > >> > > "breaking change". This way Kafka can >> > > > >>continue >> > > > >> to >> > > > >> > > grow >> > > > >> > > > > > and >> > > > >> > > > > > > > > blossom >> > > > >> > > > > > > > > >> > and >> > > > >> > > > > > > > > >> > >> we >> > > > >> > > > > > > > > >> > >> > > have a process in place if we are >> going to >> > > > >> release >> > > > >> > > a >> > > > >> > > > > > thorn >> > > > >> > > > > > > ... >> > > > >> > > > > > > > > and >> > > > >> > > > > > > > > >> > >> when we >> > > > >> > > > > > > > > >> > >> > > do it is *CLEAR* about what and why >> that >> > > is. >> > > > >> We can >> > > > >> > > > > > easily >> > > > >> > > > > > > > > >> document >> > > > >> > > > > > > > > >> > >> which >> > > > >> > > > > > > > > >> > >> > > KIPs where involved with this release >> > > > >>(which I >> > > > >> > > think >> > > > >> > > > > > > should get >> > > > >> > > > > > > > > >> > >> committed >> > > > >> > > > > > > > > >> > >> > > afterwards somewhere so no chance of >> edit >> > > > >>after >> > > > >> > > > > release). >> > > > >> > > > > > > This >> > > > >> > > > > > > > > >> > >> approach I >> > > > >> > > > > > > > > >> > >> > > had been thinking about also allows >> > > changes >> > > > >>to >> > > > >> > > occur as >> > > > >> > > > > > > they do >> > > > >> > > > > > > > > >> now >> > > > >> > > > > > > > > >> > as >> > > > >> > > > > > > > > >> > >> long >> > > > >> > > > > > > > > >> > >> > > as they are backwards compatible. >> > > > >>Hopefully we >> > > > >> > > never >> > > > >> > > > > > need >> > > > >> > > > > > > a >> > > > >> > > > > > > > > KIP >> > > > >> > > > > > > > > >> but >> > > > >> > > > > > > > > >> > >> when >> > > > >> > > > > > > > > >> > >> > > we do the PMC can vote on it and >> folks can >> > > > >> read the >> > > > >> > > > > > release >> > > > >> > > > > > > > > notes >> > > > >> > > > > > > > > >> > with >> > > > >> > > > > > > > > >> > >> > > *CLEAR* understanding what is going >> to >> > > break >> > > > >> their >> > > > >> > > > > > existing >> > > > >> > > > > > > > > >> > setup... at >> > > > >> > > > > > > > > >> > >> > > least that is how I have been >> thinking >> > > about >> > > > >> it. >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > Let me know what you think about >> this base >> > > > >> minimum >> > > > >> > > > > > > approach... >> > > > >> > > > > > > > > I >> > > > >> > > > > > > > > >> > hadn't >> > > > >> > > > > > > > > >> > >> > > really thought of the KIP for *ANY* >> "major >> > > > >> change" >> > > > >> > > and >> > > > >> > > > > I >> > > > >> > > > > > > have >> > > > >> > > > > > > > > to >> > > > >> > > > > > > > > >> > think >> > > > >> > > > > > > > > >> > >> more >> > > > >> > > > > > > > > >> > >> > > about that. I have some other >> comments for >> > > > >> minor >> > > > >> > > items >> > > > >> > > > > in >> > > > >> > > > > > > the >> > > > >> > > > > > > > > >> > >> confluence >> > > > >> > > > > > > > > >> > >> > > page I will make once I think more >> about >> > > > >>how I >> > > > >> feel >> > > > >> > > > > > having >> > > > >> > > > > > > a >> > > > >> > > > > > > > > KIP >> > > > >> > > > > > > > > >> for >> > > > >> > > > > > > > > >> > >> more >> > > > >> > > > > > > > > >> > >> > > than what I was thinking about >> already. >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > I do think we should have "major >> changes" >> > > go >> > > > >> > > through >> > > > >> > > > > > > > > confluence, >> > > > >> > > > > > > > > >> > >> mailing >> > > > >> > > > > > > > > >> > >> > > list discuss and JIRA but kind of >> feel we >> > > > >>have >> > > > >> been >> > > > >> > > > > doing >> > > > >> > > > > > > that >> > > > >> > > > > > > > > >> > already >> > > > >> > > > > > > > > >> > >> ... >> > > > >> > > > > > > > > >> > >> > > if there are cases where that isn't >> the >> > > > >>case we >> > > > >> > > should >> > > > >> > > > > > > > > highlight >> > > > >> > > > > > > > > >> and >> > > > >> > > > > > > > > >> > >> learn >> > > > >> > > > > > > > > >> > >> > > from them and formalize that more if >> need >> > > > >>be. >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > >> > > /******************************************* >> > > > >> > > > > > > > > >> > >> > > Joe Stein >> > > > >> > > > > > > > > >> > >> > > Founder, Principal Consultant >> > > > >> > > > > > > > > >> > >> > > Big Data Open Source Security LLC >> > > > >> > > > > > > > > >> > >> > > http://www.stealth.ly >> > > > >> > > > > > > > > >> > >> > > Twitter: @allthingshadoop < >> > > > >> > > > > > > > > >> http://www.twitter.com/allthingshadoop> >> > > > >> > > > > > > > > >> > >> > > >> > > > >>********************************************/ >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > On Thu, Jan 15, 2015 at 1:42 PM, Jay >> > > Kreps < >> > > > >> > > > > > > > > jay.kr...@gmail.com <javascript:;>> >> > > > >> > > > > > > > > >> > >> wrote: >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> > > > The idea of KIPs came up in a >> previous >> > > > >> > > discussion but >> > > > >> > > > > > > there >> > > > >> > > > > > > > > was >> > > > >> > > > > > > > > >> no >> > > > >> > > > > > > > > >> > >> real >> > > > >> > > > > > > > > >> > >> > > > crisp definition of what they >> were. Here >> > > > >>is >> > > > >> an >> > > > >> > > > > attempt >> > > > >> > > > > > at >> > > > >> > > > > > > > > >> > defining a >> > > > >> > > > > > > > > >> > >> > > > process: >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > >> > > > >> >> > > > >> >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Propo >> > > > >>sals >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > The trick here is to have something >> > > > >> light-weight >> > > > >> > > > > enough >> > > > >> > > > > > > that >> > > > >> > > > > > > > > it >> > > > >> > > > > > > > > >> > >> isn't a >> > > > >> > > > > > > > > >> > >> > > > hassle for small changes, but >> enough so >> > > > >>that >> > > > >> > > changes >> > > > >> > > > > > get >> > > > >> > > > > > > the >> > > > >> > > > > > > > > >> > >> eyeballs of >> > > > >> > > > > > > > > >> > >> > > > the committers and heavy users. >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > Thoughts? >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > > -Jay >> > > > >> > > > > > > > > >> > >> > > > >> > > > >> > > > > > > > > >> > >> > > >> > > > >> > > > > > > > > >> > >> >> > > > >> > > > > > > > > >> > >> >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > > >> > > > >> > > > > > > > > >> > > -- >> > > > >> > > > > > > > > >> > > Thanks, >> > > > >> > > > > > > > > >> > > Ewen >> > > > >> > > > > > > > > >> > >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> -- >> > > > >> > > > > > > > > >> -- Guozhang >> > > > >> > > > > > > > > >> >> > > > >> > > > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > > -- >> > > > >> > > > > Thanks, >> > > > >> > > > > Neha >> > > > >> > > > > >> > > > >> > > >> > > > >> > > -- >> > > > >> > > Joel >> > > > >> > > >> > > > >> >> > > > >> >> > > > >> > > >> > >> > >> > -- >> > Sent from Gmail Mobile >> >> >
