Hi all, An issue has been opened by a community member suggesting that we create a template for proposals: https://github.com/apache/incubator-druid/issues/6949
Having a template sounds convenient, and based on the discussion in this thread, I'm suggesting we adopt something based on the Kafka proposal format. I'm planning on creating such a template if there are no objections or alternative suggestions, so please take a look if you have thoughts on this. Thanks, Jon On Tue, Jan 15, 2019 at 12:07 PM Jihoon Son <[email protected]> wrote: > Good point. > If some authors raise PRs without noticing the need for a proposal, we > shouldn't ask them to close their PRs only because of the absence of the > proposal. > > "Design review" without a proposal for simple PRs would be good if we can > determine well what PRs need and what don't. > But, how do we know? Even for the same PR, someone may think it needs a > proposal but another may not. > > If someone don't notice the need for a proposal and raise a PR without it, > I'm fine with that. > However, we should still encourage writing a proposal before writing code > because we can avoid unnecessary effort. > > I think this kind of issue usually happens for first time contributors and > they will be better once they get used to Druid development. > And I believe someday even first contributors would follow this policy once > it gets settled down well in the community as Kafka community does. > > Jihoon > > On Tue, Jan 15, 2019 at 4:31 AM Roman Leventov <[email protected]> > wrote: > > > In such small PRs, authors likely won't be aware that they need to > create a > > proposal in the first place. The first reviewer just adds the "Design > > Review" tag. It's also absolutely not about considering designs and > gauging > > the proposal, it's just verifying that a configuration / parameter / HTTP > > endpoint name is reasonable and aligned with the rest of Druid. So I > think > > that a separate proposal issue for such PRs is unnecessary bureaucracy. > > > > On Tue, 15 Jan 2019 at 07:45, Jihoon Son <[email protected]> wrote: > > > > > Roman, > > > > > > > Jihoon in > > > > > > > > > https://lists.apache.org/thread.html/e007fbf362c2a870a2d88d04431789289807e00fd91d087559a01d1f@%3Cdev.druid.apache.org%3E > > > and later Gian in this thread suggested that _every_ piece of work that > > > should be labelled as "Design Review" according to the current rules > > should > > > be accompanied by an issue. I don't agree with this, there are some PRs > > as > > > small as a few dozens of lines of code, that add some configuration > > > parameter and therefore should be labelled "Design Review". I don't > > thing a > > > separate proposal issue is needed for them, and even for a little > larger > > > PRs too. > > > > > > What I'm concerned with is how people feel if their design is not > > accepted > > > even though they wrote code. Of course, as Clint said, sometimes code > > helps > > > better understanding of the proposal. But, I believe this is the case > > when > > > the proposal is quite complicated and not easy to understand without > > code. > > > Also the authors should be aware of that they might rewrite the entire > > code > > > if the design should be changed. > > > > > > If writing code is simple, I don't see why the authors don't wait until > > the > > > review for their proposal is finished. > > > > > > Jihoon > > > > > > On Fri, Jan 11, 2019 at 9:51 AM Fangjin Yang <[email protected]> wrote: > > > > > > > I agree with Gian, as an Apache committer, your responsibility is for > > the > > > > betterment of the project. I agree it is in the best interest of the > > > > project to stop thinking about what orgs people belong to. We are > all a > > > > part of the Apache software foundation, regardless of what our roles > > and > > > > titles are outside of it. > > > > > > > > On Fri, Jan 11, 2019 at 2:22 AM Roman Leventov < > [email protected]> > > > > wrote: > > > > > > > > > It's not that people from one org could abuse the project and push > > some > > > > > change, but that they have similar perspective (bubble effect) and > > some > > > > > important aspects of a large feature could escape their attention. > > > > > > > > > > I suggest it to be not a rigid rule, but a recommendation for > authors > > > of > > > > > large proposals to try to attract reviewers from other orgs. > > > > > > > > > > On Fri, 11 Jan 2019 at 02:51, Julian Hyde <[email protected]> > wrote: > > > > > > > > > > > I agree with Gian. > > > > > > > > > > > > As an Apache committer, you only have one affiliation: you are > > > working > > > > in > > > > > > the best interests of the project. > > > > > > > > > > > > Obviously, in the real world there are other pressures. But we do > > our > > > > > best > > > > > > to compensate for them. > > > > > > > > > > > > Also, as a a community we try to design our process so as to > avoid > > > > undue > > > > > > influences. For instance, when I advocate for logging cases > early, > > I > > > am > > > > > > trying to mitigate the effect of product managers and VPs of > > > > engineering, > > > > > > who like to have their say in meeting rooms rather than on public > > > > mailing > > > > > > lists. That’s just one example; if we see other influences at > play, > > > > let’s > > > > > > evolve our process to try to level the playing field. > > > > > > > > > > > > Julian > > > > > > > > > > > > > > > > > > > On Jan 10, 2019, at 10:40 AM, Gian Merlino <[email protected]> > > > wrote: > > > > > > > > > > > > > >>> What do community members think about also making a > requirement > > > > that > > > > > > >>> "Design Review" PRs and proposals are reviewed by at least > two > > > > people > > > > > > > with > > > > > > >>> different affiliation? > > > > > > >> This seems pretty reasonable to me. I haven't found anything > in > > > > Apache > > > > > > >> voting procedure docs ( > > > > https://www.apache.org/foundation/voting.html) > > > > > > that > > > > > > >> seems to explicitly forbid something like this yet at least. > > > > > > > > > > > > > > On the other hand, my understanding of the Apache Way would > mean > > > that > > > > > > this > > > > > > > kind of rule doesn't make sense. In particular from > > > > > > > https://www.apache.org/foundation/how-it-works.html: "We > firmly > > > > > believe > > > > > > in > > > > > > > hats. Your role at the ASF is one assigned to you personally, > and > > > is > > > > > > > bestowed on you by your peers. It is not tied to your job or > > > current > > > > > > > employer or company." That sentiment seems incompatible with > > making > > > > > > > explicit rules about organizational diversity in voting. I have > > > also > > > > > > heard > > > > > > > a few people say things like: people are supposed to represent > > > > > > themselves, > > > > > > > not their employers. > > > > > > > > > > > > > > Obviously, though, people's actions and opinions are influenced > > by > > > > > their > > > > > > > employer. IMO, a better way to approach a potential problem > there > > > is > > > > > that > > > > > > > if people from a particular organization end up behaving > > abusively, > > > > > then > > > > > > > PMC members from other organizations (or in extreme situations > > the > > > > > Apache > > > > > > > board itself) should tell them to knock it off. And then do > more > > > > > serious > > > > > > > things if the inappropriate behavior continues. This kind > > > > > > > > > > > > > > The other thing you brought up, promoting more shared ownership > > and > > > > > > > awareness, I am hopeful that separating proposals from PRs will > > > help > > > > > with > > > > > > > that. One reason is that it takes much less time to understand > a > > > well > > > > > > > written proposal than it takes to understand a PR. Code is very > > > clear > > > > > but > > > > > > > it is also very verbose and takes a while to read and > understand. > > > So > > > > in > > > > > > > theory changing how we operate, in this way, should promote > more > > > > > > > understanding of more people across the code base. > > > > > > > > > > > > > > On Thu, Jan 10, 2019 at 1:34 AM Clint Wylie < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > >>> > > > > > > >>> What do community members think about also making a > requirement > > > > that > > > > > > >>> "Design Review" PRs and proposals are reviewed by at least > two > > > > people > > > > > > >> with > > > > > > >>> different affiliation? > > > > > > >> > > > > > > >> > > > > > > >> This seems pretty reasonable to me. I haven't found anything > in > > > > Apache > > > > > > >> voting procedure docs ( > > > > https://www.apache.org/foundation/voting.html) > > > > > > that > > > > > > >> seems to explicitly forbid something like this yet at least. > > > > > > >> > > > > > > >> On Wed, Jan 9, 2019 at 8:52 PM Roman Leventov < > > > > [email protected]> > > > > > > >> wrote: > > > > > > >> > > > > > > >>> What do community members think about also making a > requirement > > > > that > > > > > > >>> "Design Review" PRs and proposals are reviewed by at least > two > > > > people > > > > > > >> with > > > > > > >>> different affiliation? IMO it's a good idea, because it > ensures > > > > that > > > > > > >>> different interests are taken into account. Also it pushes > > people > > > > to > > > > > > >> engage > > > > > > >>> with work done in other parts of Druid, improving shared code > > > > owning > > > > > > and > > > > > > >>> awareness. > > > > > > >>> > > > > > > >>> Or it's against the Apache Way because it's assumed that > there > > > are > > > > no > > > > > > >>> company boundaries within the community? > > > > > > >>> > > > > > > >>> On Thu, 10 Jan 2019 at 11:45, Roman Leventov < > > > > [email protected]> > > > > > > >>> wrote: > > > > > > >>> > > > > > > >>>> I see two important reasons why it makes sense to file an > > issue > > > > and > > > > > > >>>> probably announce it in the mailing list, before writing a > lot > > > of > > > > > > code, > > > > > > >>>> despite not having a clear picture of what it will be and > any > > > > > > >> performance > > > > > > >>>> data: > > > > > > >>>> 1) somebody could already work on this problem privately in > > > > > parallel, > > > > > > >> it > > > > > > >>>> allows to avoid clash in people's works > > > > > > >>>> 2) some people could quickly think about the problem field > and > > > > share > > > > > > >>>> high-level ideas that could wildly change the direction in > > which > > > > the > > > > > > >>> author > > > > > > >>>> (the person who is going to write code) will move in his > work > > > from > > > > > > >> early > > > > > > >>> on. > > > > > > >>>> > > > > > > >>>> Jihoon in > > > > > > >>>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/e007fbf362c2a870a2d88d04431789289807e00fd91d087559a01d1f@%3Cdev.druid.apache.org%3E > > > > > > >>> and > > > > > > >>>> later Gian in this thread suggested that _every_ piece of > work > > > > that > > > > > > >>> should > > > > > > >>>> be labelled as "Design Review" according to the current > rules > > > > should > > > > > > be > > > > > > >>>> accompanied by an issue. I don't agree with this, there are > > some > > > > PRs > > > > > > as > > > > > > >>>> small as a few dozens of lines of code, that add some > > > > configuration > > > > > > >>>> parameter and therefore should be labelled "Design Review". > I > > > > don't > > > > > > >>> thing a > > > > > > >>>> separate proposal issue is needed for them, and even for a > > > little > > > > > > >> larger > > > > > > >>>> PRs too. > > > > > > >>>> > > > > > > >>>> For the same reason, I also don't see the point of renaming > > > > "Design > > > > > > >>>> Review" into "Proposal", as well as separating "Design > Review" > > > > into > > > > > > >>>> "Proposal" and something like "API Review". I think a single > > > > "Design > > > > > > >>>> Review" tag handles it well. > > > > > > >>>> > > > > > > >>>> Gian mentioned an idea that PRs that follow a "Design > Review" > > > > > proposal > > > > > > >>>> issue shouldn't be "Design Review" themselves. I don't agree > > > with > > > > > > >> this, I > > > > > > >>>> think that actual code and performance data are important > > inputs > > > > > that > > > > > > >>>> should be re-evaluated at least by two people. I even think > > that > > > > > it's > > > > > > >>> very > > > > > > >>>> desirable that at least two people read _each line of > > production > > > > > code_ > > > > > > >> in > > > > > > >>>> large PRs, although it's not what was done historically in > > > Druid, > > > > > > >> because > > > > > > >>>> large bodies of newly added code, with whole new classes and > > > > > > subsystems > > > > > > >>>> added, are also coincidentally tested worse than already > > > existing > > > > > > >> classes > > > > > > >>>> and subsystems, including in production. It seems to me that > > > those > > > > > > huge > > > > > > >>>> code influxes is a major source of bugs, that could later > take > > > > years > > > > > > to > > > > > > >>>> squeeze out from the codebase. > > > > > > >>>> > > > > > > >>>> On Wed, 9 Jan 2019 at 08:24, Clint Wylie < > > [email protected]> > > > > > > wrote: > > > > > > >>>> > > > > > > >>>>> Apologies for the delayed response. > > > > > > >>>>> > > > > > > >>>>> On Thu, Jan 3, 2019 at 12:48 PM Slim Bouguerra < > > > > > > >>> [email protected]> > > > > > > >>>>> wrote: > > > > > > >>>>> > > > > > > >>>>>> I am wondering here what is the case where code first is > > > better? > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>>> I don't think it's wrong to share ideas as early as > possible, > > > and > > > > > > >> after > > > > > > >>>>> this discussion I think I am in favor of it too. I just > meant > > > > that > > > > > I > > > > > > >>> don't > > > > > > >>>>> think it's always necessarily the most productive > discussion > > > > until > > > > > > >> code > > > > > > >>>>> exists sometimes, with the types of thing I am thinking of > > are > > > > > almost > > > > > > >>>>> entirely limited to cases where things might sound good to > > > anyone > > > > > on > > > > > > >>> paper > > > > > > >>>>> but in reality need a large amount of experiments conducted > > and > > > > > > >>>>> observations collected to determine that something is > > actually > > > > > worth > > > > > > >>>>> doing, > > > > > > >>>>> which I imagine is mainly things like reworking internals > for > > > > > > >>> performance > > > > > > >>>>> improvements. > > > > > > >>>>> > > > > > > >>>>> In the case of my combined proposal PR, I needed to prove > > that > > > > the > > > > > > >>> thing I > > > > > > >>>>> was working on was a good idea... and it wasn't directly. > > But I > > > > > came > > > > > > >> up > > > > > > >>>>> with another idea during the course of experiment turned > into > > > > > > >> something > > > > > > >>>>> compelling, so an initial proposal would have looked quite > a > > > lot > > > > > > >>> different > > > > > > >>>>> than what I ended up with. Once I had proven to myself that > > it > > > > was > > > > > a > > > > > > >>> good > > > > > > >>>>> idea, then I was comfortable sharing with the wider > > community. > > > > I'm > > > > > > not > > > > > > >>>>> certain how this would play out in an always proposal first > > > > model, > > > > > > >> maybe > > > > > > >>>>> the first proposal exists, I personally reject it after > > > updating > > > > > with > > > > > > >>>>> experiment results show it's a bad idea, continue > > experimenting > > > > and > > > > > > >>> raise > > > > > > >>>>> a > > > > > > >>>>> new one after the experiments start looking promising? > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>>>>> Let's not be naive this is very rare that a contributor > will > > > > > accept > > > > > > >>> that > > > > > > >>>>>> his work is to be thrown, usually devs takes coding as > > > personal > > > > > > >>> creation > > > > > > >>>>>> and they get attached to it. > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>>> I agree, just because a handful of the committers have this > > > > > attitude, > > > > > > >> it > > > > > > >>>>> isn't fair to expect the wider community to also, that's > why > > I > > > am > > > > > in > > > > > > >>> favor > > > > > > >>>>> of formalizing the process. > > > > > > >>>>> > > > > > > >>>>> Can you please explain what is overbearing ? what can be > > > changed > > > > to > > > > > > >> make > > > > > > >>>>> it > > > > > > >>>>>> easy ? > > > > > > >>>>>> Most of the points are kind of the actual questions that > you > > > > want > > > > > to > > > > > > >>>>>> address before hand anyway isn't ? > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>>> Sorry for the confusion, I said it's "not overbearing", I > > think > > > > > it's > > > > > > >>> fine. > > > > > > >>>>> > > > > > > >>>>> What are the complaints ? > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>>>> Is this and other previous threads not a complaint about > > > opening > > > > a > > > > > > >> large > > > > > > >>>>> PR > > > > > > >>>>> without a proposal? :) I just mean that formalizing the > > > process, > > > > > even > > > > > > >>> if a > > > > > > >>>>> proposal has a reference PR opened with it near > concurrently, > > > > could > > > > > > >>>>> prevent > > > > > > >>>>> these discussions from happening in the future because > ground > > > > rules > > > > > > >> have > > > > > > >>>>> been set and we are all on the same page I guess. > > > > > > >>>>> > > > > > > >>>>> I think we should also probably consider renaming the > "design > > > > > review" > > > > > > >>>>> label > > > > > > >>>>> to "proposal" or something to make it more clear that a PR > is > > > > > > >> associated > > > > > > >>>>> with a proposal. It might also be worth considering using > > > github > > > > > > >>> projects > > > > > > >>>>> for particularly large things that involve multiple follow > up > > > > PRs, > > > > > > >> but I > > > > > > >>>>> haven't used them in much depth to know if they add > anything. > > > > > > >>>>> > > > > > > >>>>> It seems like we are all converging on agreement to do a > > github > > > > > issue > > > > > > >>>>> proposal for larger changes (I would vote for announcing > them > > > on > > > > > the > > > > > > >> dev > > > > > > >>>>> list too for more visibility), so that design review is > > > separated > > > > > > from > > > > > > >>>>> code > > > > > > >>>>> review. I guess my main concern was not wanting to > discourage > > > > > > >>>>> experimentation by walling it off behind mandatory > > discussions, > > > > but > > > > > > >> the > > > > > > >>>>> more I think about it, it doesn't technically change much > > > about > > > > > this > > > > > > >>>>> process, it just requires a more formal proposal to > accompany > > > any > > > > > > >>>>> experiments that are shared as a PR. > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>>>> On Tue, Jan 8, 2019 at 12:28 PM Gian Merlino < > > [email protected]> > > > > > > wrote: > > > > > > >>>>> > > > > > > >>>>>> I think for us, choosing to use GitHub issues as > discussion > > > > > threads > > > > > > >>> for > > > > > > >>>>>> potential 'major' contributions would be a good idea, > > > especially > > > > > if > > > > > > >> we > > > > > > >>>>>> encourage people to start them before PRs show up. > > Definitely > > > > > agree > > > > > > >>> that > > > > > > >>>>>> all contributors should go through the same process -- I > > > > couldn't > > > > > > >> see > > > > > > >>> it > > > > > > >>>>>> working well any other way. > > > > > > >>>>>> > > > > > > >>>>>> On Mon, Jan 7, 2019 at 12:23 PM Julian Hyde < > > [email protected] > > > > > > > > > > >> wrote: > > > > > > >>>>>> > > > > > > >>>>>>> Statically, yes, GitHub PRs are the same as GitHub cases. > > But > > > > > > >>>>>> dynamically, > > > > > > >>>>>>> they are different, because you can only log a PR when > you > > > have > > > > > > >>>>> finished > > > > > > >>>>>>> work. > > > > > > >>>>>>> > > > > > > >>>>>>> A lot of other Apache projects use JIRA, so there is a > > clear > > > > > > >>>>> distinction > > > > > > >>>>>>> between cases and contributions. JIRA cases, especially > > when > > > > > > >> logged > > > > > > >>>>> early > > > > > > >>>>>>> in the lifecycle of a contribution, become long-running > > > > > > >> conversation > > > > > > >>>>>>> threads with a lot of community participation. If the > Druid > > > > chose > > > > > > >> to > > > > > > >>>>> do > > > > > > >>>>>> so, > > > > > > >>>>>>> GitHub cases could be the same. > > > > > > >>>>>>> > > > > > > >>>>>>> Be careful that you do not treat “potential contributors” > > (by > > > > > > >> which > > > > > > >>> I > > > > > > >>>>>>> presume you mean non-committers) differently from > > committers > > > > and > > > > > > >> PMC > > > > > > >>>>>>> members. Anyone starting a major piece of work should > > follow > > > > the > > > > > > >>> same > > > > > > >>>>>>> process. (Experienced committers probably have a somewhat > > > > better > > > > > > >>> idea > > > > > > >>>>>> what > > > > > > >>>>>>> work will turn out to be “major”, so they get a little > more > > > > > > >> leeway.) > > > > > > >>>>>>> > > > > > > >>>>>>> Julian > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>>> On Jan 7, 2019, at 12:10 PM, Gian Merlino < > > [email protected]> > > > > > > >>> wrote: > > > > > > >>>>>>>> > > > > > > >>>>>>>> I don't think there's a need to raise issues for every > > > > change: a > > > > > > >>>>> small > > > > > > >>>>>>> bug > > > > > > >>>>>>>> fix or doc fix should just go straight to PR. (GitHub > PRs > > > show > > > > > > >> up > > > > > > >>> as > > > > > > >>>>>>> issues > > > > > > >>>>>>>> in the issue-search UI/API, so it's not like this means > > the > > > > > > >> patch > > > > > > >>>>> has > > > > > > >>>>>> no > > > > > > >>>>>>>> corresponding issue -- in a sense the PR _is_ the > issue.) > > > > > > >>>>>>>> > > > > > > >>>>>>>> I do think it makes sense to encourage potential > > > contributors > > > > to > > > > > > >>>>> write > > > > > > >>>>>> to > > > > > > >>>>>>>> the dev list or raise an issue if they aren't sure if > > > > something > > > > > > >>>>> would > > > > > > >>>>>>> need > > > > > > >>>>>>>> to go through a more heavy weight process. > > > > > > >>>>>>>> > > > > > > >>>>>>>> Fwiw we do have a set of 'design review' criteria > already > > > (we > > > > > > >> had > > > > > > >>> a > > > > > > >>>>>>>> discussion about this a couple years ago) at: > > > > > > >>>>>>>> > http://druid.io/community/#getting-your-changes-accepted. > > > So > > > > we > > > > > > >>>>>>> wouldn't be > > > > > > >>>>>>>> starting from zero on defining that. We set it up back > > when > > > we > > > > > > >>> were > > > > > > >>>>>>> trying > > > > > > >>>>>>>> to _streamline_ our process -- we used to require two > > > > non-author > > > > > > >>> +1s > > > > > > >>>>>> for > > > > > > >>>>>>>> _every_ change, even minor ones. The introduction of > > design > > > > > > >> review > > > > > > >>>>>>> criteria > > > > > > >>>>>>>> was meant to classify which PRs need that level of > review > > > and > > > > > > >>> which > > > > > > >>>>>> ones > > > > > > >>>>>>>> are minor and can be merged with less review. I do think > > it > > > > > > >> helped > > > > > > >>>>> with > > > > > > >>>>>>>> getting minor PRs merged more quickly. The list of > > criteria > > > > is, > > > > > > >>>>>>>> > > > > > > >>>>>>>> - Major architectural changes or API changes > > > > > > >>>>>>>> - HTTP requests and responses (e. g. a new HTTP > endpoint) > > > > > > >>>>>>>> - Interfaces for extensions > > > > > > >>>>>>>> - Server configuration (e. g. altering the behavior of a > > > > config > > > > > > >>>>>> property) > > > > > > >>>>>>>> - Emitted metrics > > > > > > >>>>>>>> - Other major changes, judged by the discretion of Druid > > > > > > >>> committers > > > > > > >>>>>>>> > > > > > > >>>>>>>> Some of it is subjective, but it has been in place for a > > > > while, > > > > > > >> so > > > > > > >>>>> it's > > > > > > >>>>>>> at > > > > > > >>>>>>>> least something we are relatively familiar with. > > > > > > >>>>>>>> > > > > > > >>>>>>>> On Mon, Jan 7, 2019 at 11:32 AM Julian Hyde < > > > [email protected] > > > > > > > > > > > >>>>> wrote: > > > > > > >>>>>>>> > > > > > > >>>>>>>>> Small contributions don’t need any design review, > whereas > > > > large > > > > > > >>>>>>>>> contributions need significant review. I don’t think we > > > > should > > > > > > >>>>> require > > > > > > >>>>>>> an > > > > > > >>>>>>>>> additional step for those (many) small contributions. > But > > > who > > > > > > >>>>> decides > > > > > > >>>>>>>>> whether a contribution fits into the small or large > > > category? > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> I think the solution is for authors to log a case (or > > send > > > an > > > > > > >>>>> email to > > > > > > >>>>>>>>> dev) before they start work on any contribution. Then > > > > > > >> committers > > > > > > >>>>> can > > > > > > >>>>>>>>> request a more heavy-weight process if they think it is > > > > needed. > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> Julian > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>>>>> On Jan 7, 2019, at 11:24 AM, Gian Merlino < > > > [email protected]> > > > > > > >>>>> wrote: > > > > > > >>>>>>>>>> > > > > > > >>>>>>>>>> It sounds like splitting design from code review is a > > > common > > > > > > >>> theme > > > > > > >>>>>> in a > > > > > > >>>>>>>>> few > > > > > > >>>>>>>>>> of the posts here. How does everyone feel about > making a > > > > point > > > > > > >>> of > > > > > > >>>>>>>>>> encouraging design reviews to be done as issues, > > separate > > > > from > > > > > > >>> the > > > > > > >>>>>> pull > > > > > > >>>>>>>>>> request, with the expectations that (1) the design > > review > > > > > > >> issue > > > > > > >>>>>>>>>> ("proposal") should generally appear somewhat _before_ > > the > > > > > > >> pull > > > > > > >>>>>>> request; > > > > > > >>>>>>>>>> (2) pull requests should _not_ have design review > happen > > > on > > > > > > >>> them, > > > > > > >>>>>>> meaning > > > > > > >>>>>>>>>> there should no longer be PRs with design review tags, > > and > > > > we > > > > > > >>>>> should > > > > > > >>>>>>> move > > > > > > >>>>>>>>>> the design review approval process to the issue rather > > > than > > > > > > >> the > > > > > > >>>>> PR. > > > > > > >>>>>>>>>> > > > > > > >>>>>>>>>> For (1), even if we encourage design review > discussions > > to > > > > > > >> start > > > > > > >>>>>>> before a > > > > > > >>>>>>>>>> pull request appears, I don't see an issue with them > > > running > > > > > > >>>>>>> concurrently > > > > > > >>>>>>>>>> for a while at some point. > > > > > > >>>>>>>>>> > > > > > > >>>>>>>>>> On Thu, Jan 3, 2019 at 5:35 PM Jonathan Wei < > > > > > > >> [email protected]> > > > > > > >>>>>> wrote: > > > > > > >>>>>>>>>> > > > > > > >>>>>>>>>>> Thanks for raising these concerns! > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> My initial thoughts: > > > > > > >>>>>>>>>>> - I agree that separation of design review and > > code-level > > > > > > >>> review > > > > > > >>>>> for > > > > > > >>>>>>>>> major > > > > > > >>>>>>>>>>> changes would be more efficient > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> - I agree that a clear, more formalized process for > > > > handling > > > > > > >>>>> major > > > > > > >>>>>>>>> changes > > > > > > >>>>>>>>>>> would be helpful for contributors: > > > > > > >>>>>>>>>>> - Define what is considered a major change > > > > > > >>>>>>>>>>> - Define a standard proposal structure, KIP-style > > > proposal > > > > > > >>> format > > > > > > >>>>>>>>> sounds > > > > > > >>>>>>>>>>> good to me > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> - I think it's too rigid to have a policy of "no code > > at > > > > all > > > > > > >>> with > > > > > > >>>>>> the > > > > > > >>>>>>>>>>> initial proposal" > > > > > > >>>>>>>>>>> - Code samples can be useful references for > > understanding > > > > > > >>> aspects > > > > > > >>>>>> of a > > > > > > >>>>>>>>>>> design > > > > > > >>>>>>>>>>> - In some cases it's necessary to run experiments to > > > fully > > > > > > >>>>>> understand > > > > > > >>>>>>> a > > > > > > >>>>>>>>>>> problem and determine an appropriate design, or to > > > > determine > > > > > > >>>>> whether > > > > > > >>>>>>>>>>> something is even worth doing before committing to > the > > > work > > > > > > >> of > > > > > > >>>>>>> fleshing > > > > > > >>>>>>>>> out > > > > > > >>>>>>>>>>> a proposal, prototype code is a natural outcome of > that > > > and > > > > > > >> I'm > > > > > > >>>>> not > > > > > > >>>>>>>>> against > > > > > > >>>>>>>>>>> someone providing such code for reference > > > > > > >>>>>>>>>>> - I tend to view design/code as things that are often > > > > > > >> developed > > > > > > >>>>>>>>>>> simultaneously in an intertwined way > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>>> Let's not be naive this is very rare that a > > contributor > > > > will > > > > > > >>>>> accept > > > > > > >>>>>>>>> that > > > > > > >>>>>>>>>>> his work is to be thrown, usually devs takes coding > as > > > > > > >> personal > > > > > > >>>>>>> creation > > > > > > >>>>>>>>>>> and they get attached to it. > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> If we have a clear review process that emphasizes the > > > need > > > > > > >> for > > > > > > >>>>> early > > > > > > >>>>>>>>>>> consensus building, with separate design and code > > review, > > > > > > >> then > > > > > > >>> I > > > > > > >>>>>> feel > > > > > > >>>>>>>>> we've > > > > > > >>>>>>>>>>> done enough and don't need a hard rule against having > > > some > > > > > > >> code > > > > > > >>>>>> linked > > > > > > >>>>>>>>> with > > > > > > >>>>>>>>>>> the initial proposal. If a potential contributor then > > > still > > > > > > >>>>> wants to > > > > > > >>>>>>> go > > > > > > >>>>>>>>>>> ahead and write a lot of code that may be rejected or > > > > change > > > > > > >>>>>>>>> significantly, > > > > > > >>>>>>>>>>> the risks were made clear. > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>>> Once code is written hard to think abstract. > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> I can see the validity of the concern, but I > personally > > > > don't > > > > > > >>>>> see it > > > > > > >>>>>>> as > > > > > > >>>>>>>>> a > > > > > > >>>>>>>>>>> huge risk. My impression from the Druid PR reviews > I've > > > > seen > > > > > > >> is > > > > > > >>>>> that > > > > > > >>>>>>> our > > > > > > >>>>>>>>>>> reviewers are able to keep abstract design vs. > > > > implementation > > > > > > >>>>>> details > > > > > > >>>>>>>>>>> separate and consider alternate designs when > reviewing. > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> To summarize I think it's probably enough to have a > > > policy > > > > > > >>> along > > > > > > >>>>> the > > > > > > >>>>>>>>> lines > > > > > > >>>>>>>>>>> of: > > > > > > >>>>>>>>>>> - Create more formalized guidelines for proposals and > > > what > > > > > > >>>>> changes > > > > > > >>>>>>>>> require > > > > > > >>>>>>>>>>> proposals > > > > > > >>>>>>>>>>> - Separate design and code review for major changes, > > with > > > > > > >>> design > > > > > > >>>>>>> review > > > > > > >>>>>>>>>>> first, code-level review after reaching consensus on > > the > > > > > > >>> design. > > > > > > >>>>>>>>>>> - Code before the design review is completed is just > > for > > > > > > >>>>> reference, > > > > > > >>>>>>> not > > > > > > >>>>>>>>>>> regarded as a candidate for review/merging. > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> - Jon > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>> On Thu, Jan 3, 2019 at 12:48 PM Slim Bouguerra < > > > > > > >>>>>>>>> [email protected]> > > > > > > >>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>>>>> On Thu, Jan 3, 2019 at 12:16 PM Clint Wylie < > > > > > > >>>>> [email protected]> > > > > > > >>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> I am definitely biased in this matter as an owner > of > > > > > > >> another > > > > > > >>>>> large > > > > > > >>>>>>> PR > > > > > > >>>>>>>>>>>> that > > > > > > >>>>>>>>>>>>> wasn't preceded by a direct proposal or dev list > > > > > > >> discussion, > > > > > > >>>>> and > > > > > > >>>>>> in > > > > > > >>>>>>>>>>>> general > > > > > > >>>>>>>>>>>>> I agree that proposal first is usually better, but > I > > > > think > > > > > > >> in > > > > > > >>>>> some > > > > > > >>>>>>>>>>> rarer > > > > > > >>>>>>>>>>>>> cases approaching a problem code first *is* the > most > > > > > > >>>>> appropriate > > > > > > >>>>>> way > > > > > > >>>>>>>>> to > > > > > > >>>>>>>>>>>>> have a discussion. > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> I am wondering here what is the case where code > first > > is > > > > > > >>> better? > > > > > > >>>>>>>>>>>> In general when you are writing code you have an > idea > > > > about > > > > > > >>> what > > > > > > >>>>>> you > > > > > > >>>>>>>>> want > > > > > > >>>>>>>>>>>> to change, why you want to change and why you want > to > > > > change > > > > > > >>> it. > > > > > > >>>>>>>>>>>> I do not see what is wrong with sharing this > primitive > > > > ideas > > > > > > >>> and > > > > > > >>>>>>>>> thoughts > > > > > > >>>>>>>>>>>> as an abstract proposal (at least to avoid > > overlapping) > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> I see nothing wrong with it so long as the author > > > > > > >>>>>>>>>>>>> accepts that the PR is treated as a combined > proposal > > > and > > > > > > >>>>> proof of > > > > > > >>>>>>>>>>>> concept, > > > > > > >>>>>>>>>>>>> and fair game to be radically changed via > discussion > > or > > > > > > >> even > > > > > > >>>>>>> rejected, > > > > > > >>>>>>>>>>>>> which sounds like Gian's attitude on the matter and > > is > > > > mine > > > > > > >>> as > > > > > > >>>>>> well > > > > > > >>>>>>>>>>> with > > > > > > >>>>>>>>>>>> my > > > > > > >>>>>>>>>>>>> compression stuff. > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> Let's not be naive this is very rare that a > > contributor > > > > will > > > > > > >>>>> accept > > > > > > >>>>>>>>> that > > > > > > >>>>>>>>>>>> his work is to be thrown, usually devs takes coding > as > > > > > > >>> personal > > > > > > >>>>>>>>> creation > > > > > > >>>>>>>>>>>> and they get attached to it. > > > > > > >>>>>>>>>>>> To my point you can take a look on some old issue in > > the > > > > > > >> Druid > > > > > > >>>>>> forum > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-druid/pull/3755#issuecomment-265667690 > > > > > > >>>>>>>>>>>> and am sure other communities have similar problems. > > > > > > >>>>>>>>>>>> So leaving the door open to some side cases is not a > > > good > > > > > > >> idea > > > > > > >>>>> in > > > > > > >>>>>> my > > > > > > >>>>>>>>>>>> opinion and will lead to similar issue in the > future. > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> This seems to me especially likely to happen in > cases > > > > > > >>>>>>>>>>>>> where an approach still needs proven to be a viable > > > idea > > > > > > >> *to > > > > > > >>>>> the > > > > > > >>>>>>>>>>> author*, > > > > > > >>>>>>>>>>>>> so that a much more productive discussion can be > had > > in > > > > the > > > > > > >>>>> first > > > > > > >>>>>>>>>>> place. > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> I think there is a trade off, I don't think we want > > to > > > > > > >>>>> discourage > > > > > > >>>>>>>>>>>>> experimentation by walling it off behind mandatory > > > > > > >>> discussions > > > > > > >>>>>>> before > > > > > > >>>>>>>>>>> it > > > > > > >>>>>>>>>>>>> can even start, but I do think formalizing the > > process > > > > for > > > > > > >>>>> large > > > > > > >>>>>>>>>>> changes > > > > > > >>>>>>>>>>>> is > > > > > > >>>>>>>>>>>>> a good thing, especially since we probably can't > > expect > > > > the > > > > > > >>>>> wider > > > > > > >>>>>>>>>>>> community > > > > > > >>>>>>>>>>>>> to have the same attitude towards a large PR > getting > > > > > > >>> discarded > > > > > > >>>>> as > > > > > > >>>>>> a > > > > > > >>>>>>>>>>>>> committer might. I think the Kafka approach is > > > > reasonable, > > > > > > >> a > > > > > > >>>>> bit > > > > > > >>>>>>> more > > > > > > >>>>>>>>>>>>> formal than our design review process but not > > > > overbearing. > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> Can you please explain what is overbearing ? what > can > > be > > > > > > >>>>> changed to > > > > > > >>>>>>>>> make > > > > > > >>>>>>>>>>> it > > > > > > >>>>>>>>>>>> easy ? > > > > > > >>>>>>>>>>>> Most of the points are kind of the actual questions > > that > > > > you > > > > > > >>>>> want > > > > > > >>>>>> to > > > > > > >>>>>>>>>>>> address before hand anyway isn't ? > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> Going code first > > > > > > >>>>>>>>>>>>> should be in general discouraged, but when it does > > > > happen, > > > > > > >> it > > > > > > >>>>>> seems > > > > > > >>>>>>>>>>> like > > > > > > >>>>>>>>>>>>> opening DIP/an issue/starting a mailing list thread > > or > > > > > > >>>>> whatever we > > > > > > >>>>>>> go > > > > > > >>>>>>>>>>>> with > > > > > > >>>>>>>>>>>>> to have a more high level design discussion > alongside > > > the > > > > > > >>>>>> reference > > > > > > >>>>>>> PR > > > > > > >>>>>>>>>>>>> could alleviate some of these complaints? > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> What are the complaints ? > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> +1 for "DIP" heh, I think making > > > > > > >>>>>>>>>>>>> them in the form of github issues is probably > > > > appropriate, > > > > > > >>>>> with a > > > > > > >>>>>>> dev > > > > > > >>>>>>>>>>>> list > > > > > > >>>>>>>>>>>>> thread to announce them perhaps? > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> I think github issue with [Proposal] header like > > > > > > >>>>>>>>>>>> > https://github.com/apache/incubator-druid/issues/4349 > > > is > > > > > > >> good > > > > > > >>>>> to > > > > > > >>>>>> me, > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> Thanks! > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> On Thu, Jan 3, 2019 at 10:28 AM Slim Bouguerra < > > > > > > >>>>> [email protected]> > > > > > > >>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> Thanks everyone for interacting with this thread. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> The fact that i, Roman, Jihoon and others in the > > past > > > > (FJ > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://groups.google.com/forum/#!msg/druid-user/gkUEsAYIfBA/6B2GJdLkAgAJ > > > > > > >>>>> ) > > > > > > >>>>>>>>>>>>>> raised this point indicates that PRs without a > > > proposal > > > > > > >> are > > > > > > >>>>>> indeed > > > > > > >>>>>>> an > > > > > > >>>>>>>>>>>>> issue > > > > > > >>>>>>>>>>>>>> and we need to solve it. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> Something Similar to KIP maybe called DIPs is fine > > > with > > > > > > >> me. > > > > > > >>>>>>>>>>>>>> What i strive to see is the following: > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> [Step 1] formalize what is the kind of work that > > > needs a > > > > > > >>>>> formal > > > > > > >>>>>>>>>>>>> Proposal, I > > > > > > >>>>>>>>>>>>>> think Roman and Jihoon has already covered that > > pretty > > > > > > >> well. > > > > > > >>>>> am > > > > > > >>>>>> +1 > > > > > > >>>>>>> on > > > > > > >>>>>>>>>>>>> that. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/9b30d893bdb6bb633cf6a9a700183ffb5b98f115330531a55328ac77@%3Cdev.druid.apache.org%3E > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> I am strongly in favor of the separation of > Proposal > > > > > > >> Review > > > > > > >>>>> and > > > > > > >>>>>>>>>>> (later) > > > > > > >>>>>>>>>>>>>> Code review PRs. My main reasons: > > > > > > >>>>>>>>>>>>>> Most importantly code reviewing will introduce lot > > of > > > > > > >> noise > > > > > > >>>>> and > > > > > > >>>>>>> will > > > > > > >>>>>>>>>>>>>> ultimately make the GitHub page unreadable. > > > > > > >>>>>>>>>>>>>> Avoid overlapping of work. > > > > > > >>>>>>>>>>>>>> Once code is written hard to think abstract. > > > > > > >>>>>>>>>>>>>> Separate page for Design review later can always > be > > > used > > > > > > >> it > > > > > > >>>>> as a > > > > > > >>>>>>>>>>> Design > > > > > > >>>>>>>>>>>>>> document that is readable and code free-ish. > > > > > > >>>>>>>>>>>>>> As i said the goal of this first round is to see > if > > > the > > > > > > >>>>> community > > > > > > >>>>>>>>>>> agree > > > > > > >>>>>>>>>>>>>> about such change, then make the process of design > > > more > > > > > > >>>>> inclusive > > > > > > >>>>>>>>>>> thus > > > > > > >>>>>>>>>>>>>> other contributors can submit a counter proposals. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> [Step 2] IF everybody agree about that point Step > 2 > > is > > > > to > > > > > > >>>>> define > > > > > > >>>>>>>>>>> which > > > > > > >>>>>>>>>>>>>> medium is used to Publish a primitive form of a > CODE > > > > FREE > > > > > > >>>>>> Abstract > > > > > > >>>>>>>>>>>>> Proposal > > > > > > >>>>>>>>>>>>>> containing at least the following bullet points. > > > > > > >>>>>>>>>>>>>> - The problem description and motivation > > > > > > >>>>>>>>>>>>>> - Overview of the proposed change > > > > > > >>>>>>>>>>>>>> - Operational impact (compatibility/ plans to > > > upgrades) > > > > > > >>> public > > > > > > >>>>>> API > > > > > > >>>>>>>>>>>>> changes, > > > > > > >>>>>>>>>>>>>> configuration changes, algorithm, and so on > > > > > > >>>>>>>>>>>>>> - Expected benefits and drawbacks > > > > > > >>>>>>>>>>>>>> - Rationale and alternatives > > > > > > >>>>>>>>>>>>>> - Estimate Time to Deliver if possible. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> The way i think this can be is a Github issue > where > > > > member > > > > > > >>> of > > > > > > >>>>> the > > > > > > >>>>>>>>>>>>> community > > > > > > >>>>>>>>>>>>>> will interact via comments and the author will be > > > > updating > > > > > > >>> the > > > > > > >>>>>>>>>>>>> description > > > > > > >>>>>>>>>>>>>> in the light of comments provided by the > community. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> During and near the end of the design discussions > > the > > > > > > >>> author/s > > > > > > >>>>>> can > > > > > > >>>>>>>>>>>> start > > > > > > >>>>>>>>>>>>>> writing POCs to help guide the review process this > > > > > > >> naturally > > > > > > >>>>> will > > > > > > >>>>>>> be > > > > > > >>>>>>>>>>> a > > > > > > >>>>>>>>>>>>> Pull > > > > > > >>>>>>>>>>>>>> request with actual code. > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> *Now the most important thing is that we need to > > agree > > > > > > >> that > > > > > > >>>>> any > > > > > > >>>>>>> work > > > > > > >>>>>>>>>>>> that > > > > > > >>>>>>>>>>>>>> does not align with this formal process will be > > > ignored > > > > > > >> and > > > > > > >>>>> the > > > > > > >>>>>>>>>>> author > > > > > > >>>>>>>>>>>>> will > > > > > > >>>>>>>>>>>>>> be asked to start with a DIP* > > > > > > >>>>>>>>>>>>>> *That is what i meant with “If it didn’t happen > on > > > the > > > > > > >>>>> mailing > > > > > > >>>>>>> list, > > > > > > >>>>>>>>>>>> it > > > > > > >>>>>>>>>>>>>> didn’t happen.”* > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> Thanks and happy coding! > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> On Wed, Jan 2, 2019 at 6:47 PM Gian Merlino < > > > > > > >>> [email protected]> > > > > > > >>>>>>> wrote: > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>> One of the advantages I see with a more formal > > > process > > > > is > > > > > > >>>>> (like > > > > > > >>>>>>>>>>> Kafka > > > > > > >>>>>>>>>>>>>> KIPs) > > > > > > >>>>>>>>>>>>>>> is that it levels the playing field a bit and > sets > > > some > > > > > > >>>>> ground > > > > > > >>>>>>>>>>> rules > > > > > > >>>>>>>>>>>>> for > > > > > > >>>>>>>>>>>>>>> working together. In a way it can help encourage > > > > > > >>>>> contributions > > > > > > >>>>>> by > > > > > > >>>>>>>>>>>>> making > > > > > > >>>>>>>>>>>>>> it > > > > > > >>>>>>>>>>>>>>> clear what is expected of potential contributors. > > > > > > >>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>> We have a design review process today that is not > > as > > > > > > >> formal > > > > > > >>>>> as > > > > > > >>>>>>>>>>> KIPs, > > > > > > >>>>>>>>>>>>> but > > > > > > >>>>>>>>>>>>>> is > > > > > > >>>>>>>>>>>>>>> somewhat heavier than the one you describe. Maybe > > we > > > > > > >> could > > > > > > >>>>> tweak > > > > > > >>>>>>>>>>> our > > > > > > >>>>>>>>>>>>>>> current one by starting to do design reviews > > > separately > > > > > > >>> from > > > > > > >>>>>> PRs. > > > > > > >>>>>>>>>>>> i.e., > > > > > > >>>>>>>>>>>>>> for > > > > > > >>>>>>>>>>>>>>> anything that meets our 'design review' criteria, > > do > > > > that > > > > > > >>> on > > > > > > >>>>> the > > > > > > >>>>>>>>>>> dev > > > > > > >>>>>>>>>>>>> list > > > > > > >>>>>>>>>>>>>>> or in a separate issue, and keep the PR focused > on > > > > > > >>> code-level > > > > > > >>>>>>>>>>> stuff. > > > > > > >>>>>>>>>>>>> That > > > > > > >>>>>>>>>>>>>>> way we don't end up trying to do both at once. > And > > it > > > > > > >> makes > > > > > > >>>>> it > > > > > > >>>>>>>>>>> easier > > > > > > >>>>>>>>>>>>> to > > > > > > >>>>>>>>>>>>>>> start talking about design before the code is > > ready, > > > > > > >> which > > > > > > >>>>> would > > > > > > >>>>>>> be > > > > > > >>>>>>>>>>>>>> better. > > > > > > >>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>> On Wed, Jan 2, 2019 at 6:10 PM Julian Hyde < > > > > > > >>> [email protected] > > > > > > >>>>>> > > > > > > >>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> It’s really hard to say no to a contribution > when > > > > > > >> someone > > > > > > >>>>> has > > > > > > >>>>>> put > > > > > > >>>>>>>>>>>> in > > > > > > >>>>>>>>>>>>> a > > > > > > >>>>>>>>>>>>>>>> significant amount of work. > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> The following approach is simple and works > really > > > > well: > > > > > > >>>>> Before > > > > > > >>>>>>>>>>> you > > > > > > >>>>>>>>>>>>>> start > > > > > > >>>>>>>>>>>>>>>> work, log a case, describing the problem. When > you > > > > have > > > > > > >>> some > > > > > > >>>>>>>>>>> ideas > > > > > > >>>>>>>>>>>>>> about > > > > > > >>>>>>>>>>>>>>>> design, add those to the case. When you have a > > code > > > > > > >>> branch, > > > > > > >>>>> add > > > > > > >>>>>>>>>>> its > > > > > > >>>>>>>>>>>>> URL > > > > > > >>>>>>>>>>>>>>> to > > > > > > >>>>>>>>>>>>>>>> the case. And so forth. At any point in the > > > > proceedings, > > > > > > >>>>> people > > > > > > >>>>>>>>>>> can > > > > > > >>>>>>>>>>>>>> chime > > > > > > >>>>>>>>>>>>>>>> in with their opinions. > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> In my opinion, a formal “design review” process > is > > > not > > > > > > >>>>>> necessary. > > > > > > >>>>>>>>>>>>> Just > > > > > > >>>>>>>>>>>>>>>> build consensus iteratively, by starting the > > > > > > >> conversation > > > > > > >>>>> early > > > > > > >>>>>>>>>>> in > > > > > > >>>>>>>>>>>>> the > > > > > > >>>>>>>>>>>>>>>> process. > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> Julian > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>> On Jan 2, 2019, at 12:37 PM, Gian Merlino < > > > > > > >>> [email protected] > > > > > > >>>>>> > > > > > > >>>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>> In this particular case: please consider the PR > > as > > > a > > > > > > >>>>> proposal. > > > > > > >>>>>>>>>>>>> Don't > > > > > > >>>>>>>>>>>>>>> feel > > > > > > >>>>>>>>>>>>>>>>> like just because there is code there that > takes > > a > > > > > > >>> certain > > > > > > >>>>>>>>>>>>> approach, > > > > > > >>>>>>>>>>>>>>> that > > > > > > >>>>>>>>>>>>>>>>> the approach is somehow sacred. I had to > > implement > > > > > > >>>>> something > > > > > > >>>>>> to > > > > > > >>>>>>>>>>>>>>>> crystallize > > > > > > >>>>>>>>>>>>>>>>> my own thinking about how the problem could be > > > > > > >>> approached. > > > > > > >>>>> I > > > > > > >>>>>>>>>>>> won't > > > > > > >>>>>>>>>>>>> be > > > > > > >>>>>>>>>>>>>>>>> disappointed if, as a community, we decide a > > > > different > > > > > > >>>>>>>>>>> direction > > > > > > >>>>>>>>>>>> is > > > > > > >>>>>>>>>>>>>>>> better > > > > > > >>>>>>>>>>>>>>>>> and the code all gets thrown away. That's one > of > > > the > > > > > > >>>>> reasons > > > > > > >>>>>>>>>>>> that I > > > > > > >>>>>>>>>>>>>>>> removed > > > > > > >>>>>>>>>>>>>>>>> the 0.14.0 milestone that was added to the > patch. > > > (I > > > > > > >>> don't > > > > > > >>>>>> want > > > > > > >>>>>>>>>>>> to > > > > > > >>>>>>>>>>>>>> rush > > > > > > >>>>>>>>>>>>>>>> it, > > > > > > >>>>>>>>>>>>>>>>> nor do I think that's a good idea.) > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>> In general: Sounds like we could do with some > > more > > > > > > >>>>>>>>>>> formalization > > > > > > >>>>>>>>>>>>>> around > > > > > > >>>>>>>>>>>>>>>>> what a proposal looks like, which sorts of > > changes > > > > need > > > > > > >>>>> one, > > > > > > >>>>>>>>>>> and > > > > > > >>>>>>>>>>>>> when > > > > > > >>>>>>>>>>>>>>> in > > > > > > >>>>>>>>>>>>>>>>> the dev cycle it is appropriate. FWIW I think > > > Kafka's > > > > > > >>>>> process > > > > > > >>>>>>>>>>> is > > > > > > >>>>>>>>>>>>> more > > > > > > >>>>>>>>>>>>>>> or > > > > > > >>>>>>>>>>>>>>>>> less fine, and would be okay with adopting it > for > > > > Druid > > > > > > >>> if > > > > > > >>>>>>>>>>> people > > > > > > >>>>>>>>>>>>>> like > > > > > > >>>>>>>>>>>>>>>> it. > > > > > > >>>>>>>>>>>>>>>>> Right now our standards for what requires a > > "design > > > > > > >>> review" > > > > > > >>>>>> are > > > > > > >>>>>>>>>>>>> very > > > > > > >>>>>>>>>>>>>>>>> similar to the Kafka community standards for > what > > > > > > >>> requires > > > > > > >>>>> a > > > > > > >>>>>>>>>>> KIP, > > > > > > >>>>>>>>>>>>> so > > > > > > >>>>>>>>>>>>>> we > > > > > > >>>>>>>>>>>>>>>>> have some familiarity with those concepts. > > However > > > we > > > > > > >>> don't > > > > > > >>>>>>>>>>>>> separate > > > > > > >>>>>>>>>>>>>> PR > > > > > > >>>>>>>>>>>>>>>>> review and proposal discussion as strictly as > > they > > > > do, > > > > > > >>>>> which > > > > > > >>>>>>>>>>>> seems > > > > > > >>>>>>>>>>>>> to > > > > > > >>>>>>>>>>>>>>> be > > > > > > >>>>>>>>>>>>>>>>> the foundation for the feeling of exclusion > that > > is > > > > > > >> being > > > > > > >>>>> felt > > > > > > >>>>>>>>>>>>> here. > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>> Separately: I just redid the description on > > > > > > >>>>>>>>>>>>>>>>> > > > https://github.com/apache/incubator-druid/pull/6794 > > > > to > > > > > > >>> be > > > > > > >>>>>> more > > > > > > >>>>>>>>>>>>>>>> proposal-y. > > > > > > >>>>>>>>>>>>>>>>> I followed the KIP style: > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > > > >>>>>>>>>>>>>>>> . > > > > > > >>>>>>>>>>>>>>>>> Please refresh the page and see if it looks > more > > > > > > >> useful. > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>> Gian > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>> On Wed, Jan 2, 2019 at 10:52 AM Julian Hyde < > > > > > > >>>>> [email protected] > > > > > > >>>>>>> > > > > > > >>>>>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> Slim, > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> I agree with your points that offline > > development > > > is > > > > > > >> bad > > > > > > >>>>> for > > > > > > >>>>>>>>>>>>>>> community. > > > > > > >>>>>>>>>>>>>>>>>> But I don’t think you need much mentor help. > You > > > > have > > > > > > >>>>> raised > > > > > > >>>>>>>>>>>> valid > > > > > > >>>>>>>>>>>>>>>> issues > > > > > > >>>>>>>>>>>>>>>>>> and the Druid community needs to decide what > its > > > > > > >>>>> development > > > > > > >>>>>>>>>>>>>> practices > > > > > > >>>>>>>>>>>>>>>>>> should be. > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> Julian > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> On Jan 2, 2019, at 10:29 AM, Slim Bouguerra < > > > > > > >>>>>>>>>>> [email protected]> > > > > > > >>>>>>>>>>>>>>> wrote: > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> Hello everyone and hope you all have very > good > > > > > > >>> holidays. > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> First, this email is not directed on the > author > > > or > > > > > > >> the > > > > > > >>> PR > > > > > > >>>>>>>>>>>>>>>>>>> > > > > https://github.com/apache/incubator-druid/pull/6794 > > > > > > >>> it > > > > > > >>>>>>>>>>> self, > > > > > > >>>>>>>>>>>>> but > > > > > > >>>>>>>>>>>>>> i > > > > > > >>>>>>>>>>>>>>>> see > > > > > > >>>>>>>>>>>>>>>>>>> this PR as a perfect example. > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> One of the foundation of Apache Way or what i > > > would > > > > > > >>>>> simply > > > > > > >>>>>>>>>>> call > > > > > > >>>>>>>>>>>>>> open > > > > > > >>>>>>>>>>>>>>>>>> source > > > > > > >>>>>>>>>>>>>>>>>>> community driven development is that > "Technical > > > > > > >>> decisions > > > > > > >>>>>> are > > > > > > >>>>>>>>>>>>>>>> discussed, > > > > > > >>>>>>>>>>>>>>>>>>> decided, and archived publicly. > > > > > > >>>>>>>>>>>>>>>>>>> developpement" > > > > > > >>>>>>>>>>>>>>>>>>> Which means that big technical changes such > as > > > the > > > > > > >> one > > > > > > >>>>>>>>>>> brought > > > > > > >>>>>>>>>>>>> by > > > > > > >>>>>>>>>>>>>>>> #/6794 > > > > > > >>>>>>>>>>>>>>>>>>> should have started as a proposal and round > of > > > > > > >>>>> discussions > > > > > > >>>>>>>>>>>> about > > > > > > >>>>>>>>>>>>>> the > > > > > > >>>>>>>>>>>>>>>>>> major > > > > > > >>>>>>>>>>>>>>>>>>> changes designs not as 11K line of code. > > > > > > >>>>>>>>>>>>>>>>>>> I believe such openness will promote a lot of > > > good > > > > > > >>>>> benefits > > > > > > >>>>>>>>>>>> such > > > > > > >>>>>>>>>>>>>> as: > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> - ensures community health and growth. > > > > > > >>>>>>>>>>>>>>>>>>> - ensures everyone can participate not only > the > > > > > > >> authors > > > > > > >>>>> and > > > > > > >>>>>>>>>>> his > > > > > > >>>>>>>>>>>>>>>>>> co-workers. > > > > > > >>>>>>>>>>>>>>>>>>> - ensures that the project is driven by the > > > > community > > > > > > >>> and > > > > > > >>>>>>>>>>> not a > > > > > > >>>>>>>>>>>>>> given > > > > > > >>>>>>>>>>>>>>>>>>> company or an individual. > > > > > > >>>>>>>>>>>>>>>>>>> - ensures that there is consensus (not saying > > > 100% > > > > > > >>>>>>>>>>> agreement;) > > > > > > >>>>>>>>>>>>>>> however > > > > > > >>>>>>>>>>>>>>>> it > > > > > > >>>>>>>>>>>>>>>>>>> means that all individuals will accept the > > > current > > > > > > >>>>> progress > > > > > > >>>>>>>>>>> on > > > > > > >>>>>>>>>>>>> the > > > > > > >>>>>>>>>>>>>>>>>> project > > > > > > >>>>>>>>>>>>>>>>>>> until some better proposal is put forth. > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> Personally such BIG offline PR makes me feel > > > > excluded > > > > > > >>> and > > > > > > >>>>>>>>>>>> doesn't > > > > > > >>>>>>>>>>>>>>> give > > > > > > >>>>>>>>>>>>>>>>>> me a > > > > > > >>>>>>>>>>>>>>>>>>> sense that i belong to a community at all. > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> To prevent such off list development i think > > as a > > > > > > >> Druid > > > > > > >>>>>>>>>>>> Community > > > > > > >>>>>>>>>>>>>> we > > > > > > >>>>>>>>>>>>>>>> need > > > > > > >>>>>>>>>>>>>>>>>>> to stick to the apache way “If it didn’t > happen > > > on > > > > > > >> the > > > > > > >>>>>>>>>>> mailing > > > > > > >>>>>>>>>>>>>> list, > > > > > > >>>>>>>>>>>>>>> it > > > > > > >>>>>>>>>>>>>>>>>>> didn’t happen.” > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>>> I would appreciate if some of the Apache > mentor > > > > help > > > > > > >>> with > > > > > > >>>>>>>>>>> this. > > > > > > >>>>>>>>>>>>>>>>>>> Thanks > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>> > > > > > > >>> > > > > --------------------------------------------------------------------- > > > > > > >>>>>>>>>>>>>>>>>> To unsubscribe, e-mail: > > > > > > >>> [email protected] > > > > > > >>>>>>>>>>>>>>>>>> For additional commands, e-mail: > > > > > > >>>>> [email protected] > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>> > > > > > > >> > > > > --------------------------------------------------------------------- > > > > > > >>>>>>>>>>>>>>>> To unsubscribe, e-mail: > > > > > > >> [email protected] > > > > > > >>>>>>>>>>>>>>>> For additional commands, e-mail: > > > > > > >>> [email protected] > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>>> > > > > > > >>>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> -- > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>>> B-Slim > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>> > > > > > > >>> > > > > > > > > > > > > _______/\/\/\_______/\/\/\_______/\/\/\_______/\/\/\_______/\/\/\_______ > > > > > > >>>>>>>>>>>> > > > > > > >>>>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>> > > > > > > --------------------------------------------------------------------- > > > > > > >>>>>>>>> To unsubscribe, e-mail: > [email protected] > > > > > > >>>>>>>>> For additional commands, e-mail: > > [email protected] > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>> > > > > --------------------------------------------------------------------- > > > > > > >>>>>>> To unsubscribe, e-mail: [email protected] > > > > > > >>>>>>> For additional commands, e-mail: > [email protected] > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > >
