Trying to bring attention this again. We currently have few big feature PRs going on and there is considerable discussion about the design and Implementation etcc. My intention of starting SIP is to add these details before someone goes and writes up a PR and everyone has to go through reading of design and sometimes those docs are not clear and we end up having long discussion on the PRs which should mainly about the code review itself. We should at least start making this process mandatory for any new big feature especially to the storm-core. I am less concerned about connectors and other parts which should have least resistive path and they are usually easy to review. If the devs put their thoughts and design and goes through discussion and get everyone on the same line when the PR shows up it will be less surprising and everyone involved know how the PR/Code supposed to work.
-Harsha On Fri, Jun 9, 2017, at 09:16 PM, Harsha wrote: > Arun, > For big features we did follow design doc/review. Making it > formal makes everyone to follow a process. > Again this process is not for bug fixes as we stated its about New > Features/Config Changes/Public interface changes. I don't think it puts > any extra effort for anyone who is writing detailed JIRA but by making > it formal makes everyone to add these details in a centra process. Not > everyone will look at mailing list but its easier to follow a wiki page. > We should atleast give it a try before we vote it out. > > Roshan, > Adding connector should require a SIP as well and changing any > public interfaces should be a KIP. Intention here is we've > central place where everyone can follow in detail whats the > public interface/new feature changes went in. We've changed > KafkaSpout quite a bit and there is current discussion thats > going to change it , having this documented in a central place > will make it easy to follow and recording them in release notes > as well. > > Taylor, > We can't call it a too tedious process without even giving it a > try. This has been followed to a greater success at kafka and > also Flink started the process as well > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > . > If it actually proved to more of hindrance than helping the community we > can move away from it. > > " Kafka has somewhat of a reputation for setting potentially too high a > bar. I'd rather not see that happen with this community." > Sure. But it also depends on the community. Just because some community > enforcing too high bar that doesn't mean we are trying to do it via this > process. Again we always have option if we ever veer too far in the > wrong direction to bring up and improve or remove this process. > > We should also as a community strive to have better quality and I am > hoping this will give us a chance to not only let users know what are > changes coming in but also keep the dev list to have a chance and join > the discussion. > > -Harsha > > On Jun 9, 2017, 7:18 PM -0700, Arun Iyer <[email protected]>, wrote: > I am for documenting and upfront design reviews, but maybe we should > keep it less formal and make it part of the JIRA to start with. > > Do we have any upcoming features for which we would like to see a > proposal? May be start with a couple of proposals > and see it works out before making it formal. > > > Thanks, > Arun > > > > 6/9/17, 6:49 PM, "P. Taylor Goetz" <[email protected]> wrote: > > -0 > > The KIP process feels kind of heavy. I'd rather start with a lighter > effort like improving JIRA submissions and pull requests (some pull > requests/JIRAs, even from committers and PMC members, are woefully > inadequate in terms of detail), and see how that works out. > > I share Bobby's concern that doing so might raise the bar for > contributions and potentially have a chilling effect. We don't want to > scare away contributors. Kafka has somewhat of a reputation for setting > potentially too high a bar. I'd rather not see that happen with this > community. > > I will say that I like the idea of proposals for big features, ideally > before any coding even begins -- so that others have a chance to > collaborate. But I'm hesitant to impose too much process, voting, etc. > That could scare people off. > > I think we should think carefully before going down this trail. > > -Taylor > > On Jun 9, 2017, at 8:57 PM, Priyank Shah <[email protected]> wrote: > > +1 for SIPs including a new connector. The person writing SIP can > provide details about the external system for which connector is being > written to let others know why a certain design decision was made. This > will make it easy for reviewers. > > On 6/9/17, 5:24 PM, "Satish Duggana" <[email protected]> wrote: > > +1 for SIPs. It is so useful as mentioned by others in earlier mails. It > would be very useful for new contributors and others who are looking out > for a feature design and decisions taken etc. > > Whenever a minor feature is added to a connector it may not need a > separate > SIP but the existing README can be updated with details for users. It > can > be discussed and decided apropos whether a SIP is really required for > any > enhancement which is not really big. > > > On Sat, Jun 10, 2017 at 5:13 AM, Roshan Naik <[email protected]> > wrote: > > If I am looking at the Kafka site correctly, I see that Kafka has a > total > of 167 KIPs so far. > So I assume that minor new features would not be parrt of the SIP ? > > Unlike Kafka, since Storm has a number of connectors (that keep > growing), > I am speculating the SIP process might get somewhat unwieldy if it were > to > track little changes in each of the connectors. > > Also thinking that a SIP may not be needed to justify a new connector, > but > useful if we are replacing an old connector with a new one. > > -roshan > > > > On 6/9/17, 3:19 PM, "Harsha" <[email protected]> wrote: > > Hi Bobby, > In general, a KIP is required for adding New features, > config > changes or backward-incompatible changes. Don't require > adding a KIP for bug-fixes. Devs who wants to add any > features will write up a wiki which has JIRA link, mailing > list discussion link and outline the Motivation, Public > interface changes and protocol changes etc ..a good example > here is > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 48+Delegation+token+support+for+Kafka. > They can start the discussion thread once its ready and once everyone > agrees its in a good shape, a Vote thread starts . Once there are > required votes are in one can start the PR process and get it merged > in. > Each release we can collect what features/fixes especially > to > public interfaces that went in and roll it out in release > notes. This will give a better idea for the users on what > changed and added from previous version. > We can only enforce this to new feature/config/backward > incompatible change. Having this go through the discussion > phase will give us the early feedback and potentially caught > any issues before the implementation. > Thanks, > Harsha > > On Fri, Jun 9, 2017 at 2:24 PM Bobby Evans <[email protected] > > wrote: > > Can you please explain how KIP currently works and how you would > like to see something similar in storm? > If we make the process more formal we will probably have less > people > contributing, but we will probably have better overall patches. It > is a balancing act and having never used KIP I would like to > understand it better before going all in on it. > - Bobby > > > On Friday, June 9, 2017, 4:09:38 PM CDT, Stig Døssing > <[email protected]> wrote: > > This sounds like a good idea. KIPs seem to work well for Kafka. > It's > easy > for discussions to get lost or just not seen on the mailing list. > > 2017-06-09 21:36 GMT+02:00 Harsha <[email protected]>: > > Hi All, > We’ve seen good adoption of KIP approach in Kafka > community > and would like to see we adopt the similar approach for > storm > as well. > Its hard to keep track of proposed changes and mailing list > threads to > know what all changes that are coming into and what > design/backward > incompatible changes being approved. It will be good to have > this > documented and go through discussion then Vote phase to get them > approved before we merge the PRs. This will keep everyone > informed of > what changes happened even if they are not following the mailing > list > they can go to wiki to see the list of changes went into a > release. > Community overall will be well informed of the changes as well. > Would > like to hear your thoughts. > > Thanks, > Harsha > > > > > >
