+1 on the idea, and we could mutually link the KIP wiki page with the the created JIRA ticket (i.e. include the JIRA number on the page and the KIP url on the ticket description).
Regarding the KIP process, probably we do not need two phase communication of a [DISCUSS] followed by [VOTE], as Jay said the voting should be clear while people discuss about that. About who should trigger the process, I think the only involved people would be 1) when the patch is submitted / or even the ticket is created, the assignee could choose to start the KIP process if she thought it is necessary; 2) the reviewer of the patch can also suggest starting KIP discussions. On Fri, Jan 16, 2015 at 10:49 AM, Gwen Shapira <gshap...@cloudera.com> wrote: > +1 to Ewen's suggestions: Deprecation, status and version. > > Perhaps add the JIRA where the KIP was implemented to the metadata. > This will help tie things together. > > On Fri, Jan 16, 2015 at 9:35 AM, Ewen Cheslack-Postava > <e...@confluent.io> wrote: > > I think adding a section about deprecation would be helpful. A good > > fraction of the time I would expect the goal of a KIP is to fix or > replace > > older functionality that needs continued support for compatibility, but > > should eventually be phased out. This helps Kafka devs understand how > long > > they'll end up supporting multiple versions of features and helps users > > understand when they're going to have to make updates to their > applications. > > > > Less important but useful -- having a bit of standard metadata like PEPs > > do. Two I think are important are status (if someone lands on the KIP > page, > > can they tell whether this KIP was ever completed?) and/or the version > the > > KIP was first released in. > > > > > > > > On Fri, Jan 16, 2015 at 9:20 AM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > >> I'm definitely +1 on the KIP concept. As Joe mentioned, we are already > >> doing this in one form or the other. However, IMO it is fairly ad hoc > >> - i.e., a combination of DISCUSS threads, jira comments, RB and code > >> comments, wikis and html documentation. In the past I have had to dig > >> into a bunch of these to try and figure out why something was > >> implemented a certain way. I think KIPs can help a lot here first by > >> providing guidelines on what to think about (compatibility, new APIs, > >> etc.) when working through a major feature; and second by becoming a > >> crisp source of truth documentation for new releases. E.g., for > >> feature X: see relevant KIPs: a, b, c, etc. > >> > >> On Thu, Jan 15, 2015 at 08:11:20PM -0800, Jay Kreps wrote: > >> > Hey Joe, > >> > > >> > Yeah I guess the question is what is the definition of major? I agree > we > >> > definitely don't want to generate a bunch of paperwork. We have enough > >> > problems just getting all the contributions reviewed and checked in > in a > >> > timely fashion... > >> > > >> > So obviously bug fixes would not apply here. > >> > > >> > I think it is also pretty clear that big features should get reviewed > and > >> > discussed. To pick on myself, for example, the log compaction work was > >> done > >> > without enough public discussion about how it worked and why (imho). I > >> > hope/claim that enough rigour in thinking about use-cases and > operations > >> > and so on was done that it turned out well, but the discussion was > just > >> > between a few people with no real public output. This kind of feature > is > >> > clearly a big change and something we should discuss. > >> > > >> > If we limit ourselves to just the public contracts the KIP introduces > the > >> > discussion would just be on the new configs and monitoring without > >> really a > >> > discussion of the design and how it works which is obviously closely > >> > related. > >> > > >> > I don't think this should be more work because in practice we are > making > >> > wiki pages for any big thing anyway. So this would just be a > consistent > >> way > >> > of numbering and structuring these pages. This would also give a clear > >> call > >> > to action: "hey kafka people, come read my wiki and think this > through". > >> > > >> > I actually thinking the voting aspect is less important. I think it is > >> > generally clear when there is agreement on something and not. So from > my > >> > point of view we could actually just eliminate that part if that is > too > >> > formal, it just seemed like a good way to formally adopt something. > >> > > >> > To address some of your comments from the wiki: > >> > > >> > 1. This doesn't inhibit someone coming along and putting up a patch. > It > >> is > >> > just that when they do if it is a big thing introducing new > functionality > >> > we would ask for a little discussion on the basic feature/contracts > prior > >> > to code review. > >> > > >> > 2. We definitely definitely don't want people generating a lot of > these > >> > things every time they have an idea that they aren't going to > implement. > >> So > >> > this is only applicable to things you absolutely will check in code > for. > >> We > >> > also don't want to be making proposals before things are thought > through, > >> > which often requires writing the code. So I think the right time for a > >> KIP > >> > is when you are far enough along that you know the issues and > tradeoffs > >> but > >> > not so far along that you are going to be totally opposed to any > change. > >> > Sometimes that is prior to writing any code and sometimes not until > you > >> are > >> > practically done. > >> > > >> > The key problem I see this fixing is that there is enough new > development > >> > happening that it is pretty hard for everyone to review every line of > >> every > >> > iteration of every patch. But all of us should be fully aware of new > >> > features, the ramifications, the new public interfaces, etc. If we > aren't > >> > aware of that we can't really build a holistic system that is > beautiful > >> and > >> > consistent across. So the idea is that if you fully review the KIPs > you > >> can > >> > be sure that even if you don't know every new line of code, you know > the > >> > major changes coming in. > >> > > >> > -Jay > >> > > >> > > >> > > >> > On Thu, Jan 15, 2015 at 12:18 PM, Joe Stein <joe.st...@stealth.ly> > >> wrote: > >> > > >> > > Thanks Jay for kicking this off! I think the confluence page you > wrote > >> up > >> > > is a great start. > >> > > > >> > > > >> > > The KIP makes sense to me (at a minimum) if there is going to be any > >> > > "breaking change". This way Kafka can continue to grow and blossom > and > >> we > >> > > have a process in place if we are going to release a thorn ... and > >> when we > >> > > do it is *CLEAR* about what and why that is. We can easily document > >> which > >> > > KIPs where involved with this release (which I think should get > >> committed > >> > > afterwards somewhere so no chance of edit after release). This > >> approach I > >> > > had been thinking about also allows changes to occur as they do now > as > >> long > >> > > as they are backwards compatible. Hopefully we never need a KIP but > >> when > >> > > we do the PMC can vote on it and folks can read the release notes > with > >> > > *CLEAR* understanding what is going to break their existing > setup... at > >> > > least that is how I have been thinking about it. > >> > > > >> > > > >> > > Let me know what you think about this base minimum approach... I > hadn't > >> > > really thought of the KIP for *ANY* "major change" and I have to > think > >> more > >> > > about that. I have some other comments for minor items in the > >> confluence > >> > > page I will make once I think more about how I feel having a KIP for > >> more > >> > > than what I was thinking about already. > >> > > > >> > > > >> > > I do think we should have "major changes" go through confluence, > >> mailing > >> > > list discuss and JIRA but kind of feel we have been doing that > already > >> ... > >> > > if there are cases where that isn't the case we should highlight and > >> learn > >> > > from them and formalize that more if need be. > >> > > > >> > > > >> > > /******************************************* > >> > > Joe Stein > >> > > Founder, Principal Consultant > >> > > Big Data Open Source Security LLC > >> > > http://www.stealth.ly > >> > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >> > > ********************************************/ > >> > > > >> > > On Thu, Jan 15, 2015 at 1:42 PM, Jay Kreps <jay.kr...@gmail.com> > >> wrote: > >> > > > >> > > > The idea of KIPs came up in a previous discussion but there was no > >> real > >> > > > crisp definition of what they were. Here is an attempt at > defining a > >> > > > process: > >> > > > > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > >> > > > > >> > > > The trick here is to have something light-weight enough that it > >> isn't a > >> > > > hassle for small changes, but enough so that changes get the > >> eyeballs of > >> > > > the committers and heavy users. > >> > > > > >> > > > Thoughts? > >> > > > > >> > > > -Jay > >> > > > > >> > > > >> > >> > > > > > > -- > > Thanks, > > Ewen > -- -- Guozhang
