Hi Lari, No specific reason that compares to the separated repo solution. IMO, it depends on the number of proposals to be processed. If we have more than 200 proposals per year, I will say `don't use the Pulsar main repo` with certainty I also have no objection to the separated repo solution.
> The benefit of this is that we won't be triggering workflows when modifying the PIP files. This would also open new opportunities like publishing a static website that include the rendered view of the PIP files etc. Yes, that should be the point that the separated repo is a better solution. Thanks, Penghui On Tue, Aug 30, 2022 at 1:55 AM Lari Hotari <lhot...@apache.org> wrote: > Is there a specific reason to put the PIPs in the apache/pulsar repository? > > I think that it will add unnecessary cruft to our core source code > repository. > > Could we instead create a separate repository to hold the PIP files? The > example of Rust lang has a separate repository too, > https://github.com/rust-lang/rfcs. > > I'd suggest that we create https://github.com/apache/pulsar-pips to hold > the PIPs. > > The benefit of this is that we won't be triggering workflows when > modifying the PIP files. > This would also open new opportunities like publishing a static website > that include the rendered view of the PIP files etc. > > -Lari > > > On 2022/08/29 15:38:46 PengHui Li wrote: > > ``` > > A sub-team makes final decisions about RFCs after the benefits and > > drawbacks are well understood. These decisions can be made at any time, > but > > the sub-team will regularly issue decisions. When a decision is made, the > > RFC pull request will either be merged or closed. In either case, if the > > reasoning is not clear from the discussion in thread, the sub-team will > add > > a comment describing the rationale for the decision. > > ``` > > from https://github.com/rust-lang/rfcs#reviewing-rfcs > > > > I think they merged all the approved proposals? > > > > https://github.com/rust-lang/rfcs/pulls?q=is%3Apr+is%3Amerged+ > > > > And I can also find the md files > > > > https://github.com/rust-lang/rfcs/tree/master/text > > > > Thanks, > > Penghui > > > > On Mon, Aug 29, 2022 at 11:33 PM PengHui Li <peng...@apache.org> wrote: > > > > > > The only point worth attending to while drafting the new process is > how > > > to > > > avoid neglected out-of-date proposals. If a proposal was declined, who > is > > > in charge to update its status in the readme? > > > > > > That is a good point. IMO, everyone is able to correct the proposal > status. > > > It wasn't easy for us before, only the committer could update the wiki > > > page. > > > Of course, in most cases, it is done by the author. But this is not > 100% > > > guaranteed. > > > We should add this part to the proposal process. > > > > > > > There is something appealing in not merging a PR proposal, as this > status > > > update takes care of itself (borrowing from rust RFC). > > > > > > The problem I thought of not merging the PR proposal is the proposal > looks > > > like the proposed changes never end. The author can continue to update > > > after approved. I don't want to say all the changes to the proposal > need > > > to be voted on > > > the mailing list, but we should get a chance to review the changes. > > > > > > Thanks, > > > Penghui > > > > > > On Mon, Aug 29, 2022 at 10:45 PM Asaf Mesika <asaf.mes...@gmail.com> > > > wrote: > > > > > >> I like the idea of keeping the suggestions as files in the repo since > as > > >> you mentioned, it makes the review process using PRs much more > > >> streamlined. > > >> > > >> I think keeping the status in an MD file and only there (having a > single > > >> source of truth) will make it less error-prone (people might forget to > > >> move > > >> between directories) , and also easier to have a single page to view > all > > >> proposals. > > >> > > >> The only point worth attending to while drafting the new process is > how to > > >> avoid neglected out-of-date proposals. If a proposal was declined, > who is > > >> in charge to update its status in the readme? > > >> There is something appealing in not merging a PR proposal, as this > status > > >> update takes care of itself (borrowing from rust RFC). > > >> > > >> One bonus item is that I learned through this long discussion thread > about > > >> Zulip chat :) > > >> > > >> > > >> > > >> On Mon, Aug 29, 2022 at 1:55 PM PengHui Li <peng...@apache.org> > wrote: > > >> > > >> > Hi all, > > >> > > > >> > I will merge https://github.com/apache/pulsar/pull/17270 first and > > >> draft a > > >> > proposal for > > >> > the detailed PIP process changes. And will start a new VOTE thread > for > > >> the > > >> > PIP process change. > > >> > > > >> > After the proposal(for PIP process change) is approved. I will go > back > > >> here > > >> > to discuss > > >> > how to migrate the old PIPs to the codebase(because we need to > follow > > >> > the new format of PIPs). > > >> > > > >> > Thanks, > > >> > Penghui > > >> > > > >> > On Fri, Aug 26, 2022 at 11:20 PM PengHui Li <peng...@apache.org> > wrote: > > >> > > > >> > > > Using a directory structure to organize PIP status might make > the > > >> git > > >> > > history a bit less direct because changing state is equivalent > with a > > >> > > file move instead of a line updated in a file. However, if we do > it > > >> > > that way, we could have a README.md file organizing PIP metadata > and > > >> > > linking to the actual PIP file in the directory structure. That > > >> > > README.md would also serve as the source of truth for PIP numbers > > >> > > because each PIP pointer would get its associated line number. > Then, > > >> > > concurrent PIPs would need to resolve merge conflicts just before > > >> > > merging and only then would they acquire their number. > > >> > > > > >> > > Oh, get your point, Michael. I think this solution looks better. > We > > >> can > > >> > > also > > >> > > add something in README.md and users can also get the complete > > >> > > proposal list here. In the future, maybe we can show it on the > > >> website. > > >> > > > > >> > > Best, > > >> > > Penghui > > >> > > > > >> > > On Fri, Aug 26, 2022 at 12:16 PM Michael Marshall < > > >> mmarsh...@apache.org> > > >> > > wrote: > > >> > > > > >> > >> > We should move to the codebase 100% the same as the original > > >> > >> documentation. > > >> > >> > And use another PR to make improvements for it. If it is > needed, we > > >> > >> should > > >> > >> > start with an email. > > >> > >> > We need to have historical records. > > >> > >> > > >> > >> +1 I think this is a great idea. It makes sense to copy them > verbatim > > >> > >> and then worry about updating them in a second step. > > >> > >> > > >> > >> Using a directory structure to organize PIP status might make > the git > > >> > >> history a bit less direct because changing state is equivalent > with a > > >> > >> file move instead of a line updated in a file. However, if we do > it > > >> > >> that way, we could have a README.md file organizing PIP metadata > and > > >> > >> linking to the actual PIP file in the directory structure. That > > >> > >> README.md would also serve as the source of truth for PIP numbers > > >> > >> because each PIP pointer would get its associated line number. > Then, > > >> > >> concurrent PIPs would need to resolve merge conflicts just before > > >> > >> merging and only then would they acquire their number. > > >> > >> > > >> > >> - Michael > > >> > >> > > >> > >> On Thu, Aug 25, 2022 at 9:15 PM PengHui Li <peng...@apache.org> > > >> wrote: > > >> > >> > > > >> > >> > Hi Xiangying, > > >> > >> > > > >> > >> > > We can classify the pips under these folders according to the > > >> pulsar > > >> > >> > modules, instead of just placing these pips under these > folders in > > >> an > > >> > >> > incrementing sequence number. > > >> > >> > > > >> > >> > I think it's not easy to do this. A proposal can have changes > > >> related > > >> > to > > >> > >> > multiple > > >> > >> > modules(Broker, Metadata, Client). > > >> > >> > > > >> > >> > Thanks, > > >> > >> > Penghui > > >> > >> > > > >> > >> > On Fri, Aug 26, 2022 at 9:20 AM Xiangying Meng < > > >> xiangy...@apache.org> > > >> > >> wrote: > > >> > >> > > > >> > >> > > Hi Penghui > > >> > >> > > Thanks for you start this discussion. IMO, It is also a good > way > > >> for > > >> > >> > > beginners to learn the design and implementation of each > module > > >> of > > >> > >> Pulsar. > > >> > >> > > > 3. /wiki/pip/accepted - for PIPs that have been accepted > and > > >> are > > >> > >> works in > > >> > >> > > progress > > >> > >> > > > 4. /wiki/pip/complete - for PIPs that have been completed. > > >> > >> > > > 5. /wiki/pip/rejected - for PIPs that were proposed, but > then > > >> > >> rejected or > > >> > >> > > abandoned. > > >> > >> > > We can classify the pips under these folders according to the > > >> pulsar > > >> > >> > > modules, instead of just placing these pips under these > folders > > >> in > > >> > an > > >> > >> > > incrementing sequence number. > > >> > >> > > > > >> > >> > > In this way, readers can create a new local branch dedicated > to > > >> > >> reading and > > >> > >> > > annotating proposals for themselves to read proposals they > are > > >> > >> interested > > >> > >> > > in and write their own understanding and comments anytime and > > >> > >> anywhere. > > >> > >> > > Thanks, > > >> > >> > > Xiangying Meng > > >> > >> > > > > >> > >> > > On Fri, Aug 26, 2022 at 12:23 AM PengHui Li < > peng...@apache.org> > > >> > >> wrote: > > >> > >> > > > > >> > >> > > > Hi Dave, > > >> > >> > > > > > >> > >> > > > > Let’s outline how PIPs are currently working and then > discuss > > >> > >> changes. > > >> > >> > > > > > >> > >> > > > Yes, I will prepare for the changes. > > >> > >> > > > This is the documentation for how PIPs are currently > working: > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > >> > > > >> > https://github.com/apache/pulsar/pull/17270/files#diff-a56445d038f8a3df4c74724076c62437915f091437b4e26a1c5aada7184dcffd > > >> > >> > > > The mailing list discussion: > > >> > >> > > > > > >> https://lists.apache.org/thread/m8dr0hz7qn7rkd48bcp430lcq2q3674g > > >> > >> > > > > > >> > >> > > > Anyway, I will start a new discussion with the new changes > to > > >> the > > >> > >> current > > >> > >> > > > process. > > >> > >> > > > > > >> > >> > > > > I’m not sure what is meant by putting the PIP into the > > >> > “codebase”. > > >> > >> > > > > Is the proposal to create PIPs here? > > >> > >> > > > https://github.com/apache/pulsar/tree/master/wiki > > >> > >> > > > > > >> > >> > > > Just move out from > > >> > >> https://github.com/apache/pulsar/tree/master/wiki to > > >> > >> > > > Pulsar codebase /wiki/proposals > > >> > >> > > > > > >> > >> > > > > I think that a directory structure / convention could be > used > > >> > for > > >> > >> pip > > >> > >> > > > status: > > >> > >> > > > > > >> > >> > > > > 1. /wiki/pip/discussion - for PIPs being discussed and > > >> > specified. > > >> > >> > > > > 2. /wiki/pip/proposed - for PIPs ready to be formally > > >> DISCUSSED > > >> > >> and > > >> > >> > > VOTED > > >> > >> > > > > 3. /wiki/pip/accepted - for PIPs that have been accepted > and > > >> are > > >> > >> works > > >> > >> > > in > > >> > >> > > > progress > > >> > >> > > > > 4. /wiki/pip/complete - for PIPs that have been > completed. > > >> > >> > > > > 5. /wiki/pip/rejected - for PIPs that were proposed, but > then > > >> > >> rejected > > >> > >> > > or > > >> > >> > > > abandoned. > > >> > >> > > > > > >> > >> > > > I think it's a good point, I don't see any obvious cons. > > >> > >> > > > > > >> > >> > > > Thanks, > > >> > >> > > > Penghui > > >> > >> > > > > > >> > >> > > > On Thu, Aug 25, 2022 at 11:40 PM Dave Fisher < > w...@apache.org> > > >> > >> wrote: > > >> > >> > > > > > >> > >> > > > > > > >> > >> > > > > > On Aug 23, 2022, at 10:22 AM, Rajan Dhabalia < > > >> > >> dhabalia...@gmail.com> > > >> > >> > > > > wrote: > > >> > >> > > > > > > > >> > >> > > > > > Hi, > > >> > >> > > > > > > > >> > >> > > > > >>>> I think we can move all the PIPs to the codebase > and the > > >> > new > > >> > >> > > > proposal > > >> > >> > > > > > and proposal without any reviews should happen with a > PR > > >> > first. > > >> > >> So > > >> > >> > > that > > >> > >> > > > > we > > >> > >> > > > > > can review and comment easily. > > >> > >> > > > > > > > >> > >> > > > > > I didn't understand this part. You want one to create > a PR > > >> > >> before > > >> > >> > > > > > submitting a proposal? That's clearly not a good idea > > >> because > > >> > >> if the > > >> > >> > > > PIP > > >> > >> > > > > > approach will change then the entire development effort > > >> will > > >> > be > > >> > >> > > wasted > > >> > >> > > > > and > > >> > >> > > > > > that's the whole purpose of PIP. I guess creating PIP > into > > >> an > > >> > >> issue > > >> > >> > > and > > >> > >> > > > > > discussing the issue is definitely working and it's an > > >> easier > > >> > >> way to > > >> > >> > > > > > discuss quickly rather than discussing over email > threads. > > >> > >> > > > > > > > >> > >> > > > > > Let's not change this practice without good discussion > and > > >> > >> agreement > > >> > >> > > > from > > >> > >> > > > > > the community. > > >> > >> > > > > > > >> > >> > > > > Agreed let’s have a PIP Discussion here to carefully > outline > > >> how > > >> > >> the > > >> > >> > > PIP > > >> > >> > > > > process will change. I don’t think that a new PIP should > be > > >> > overly > > >> > >> > > > planned > > >> > >> > > > > or implemented before the idea is more fully discussed > and > > >> > >> accepted. > > >> > >> > > The > > >> > >> > > > > Apache Way always works best with small incremental and > > >> > reversible > > >> > >> > > steps. > > >> > >> > > > > > > >> > >> > > > > Let’s outline how PIPs are currently working and then > discuss > > >> > >> changes. > > >> > >> > > > I’m > > >> > >> > > > > not sure what is meant by putting the PIP into the > > >> “codebase”. > > >> > >> > > > > > > >> > >> > > > > Is the proposal to create PIPs here? > > >> > >> > > > > https://github.com/apache/pulsar/tree/master/wiki > > >> > >> > > > > > > >> > >> > > > > I think that a directory structure / convention could be > used > > >> > for > > >> > >> pip > > >> > >> > > > > status: > > >> > >> > > > > > > >> > >> > > > > 1. /wiki/pip/discussion - for PIPs being discussed and > > >> > specified. > > >> > >> > > > > 2. /wiki/pip/proposed - for PIPs ready to be formally > > >> DISCUSSED > > >> > >> and > > >> > >> > > VOTED > > >> > >> > > > > 3. /wiki/pip/accepted - for PIPs that have been accepted > and > > >> are > > >> > >> works > > >> > >> > > in > > >> > >> > > > > progress > > >> > >> > > > > 4. /wiki/pip/complete - for PIPs that have been > completed. > > >> > >> > > > > 5. /wiki/pip/rejected - for PIPs that were proposed, but > then > > >> > >> rejected > > >> > >> > > or > > >> > >> > > > > abandoned. > > >> > >> > > > > > > >> > >> > > > > Regards, > > >> > >> > > > > Dave > > >> > >> > > > > > > >> > >> > > > > > > > >> > >> > > > > > Thanks, > > >> > >> > > > > > Rajan > > >> > >> > > > > > > > >> > >> > > > > > On Thu, Aug 18, 2022 at 8:27 AM PengHui Li < > > >> > peng...@apache.org> > > >> > >> > > wrote: > > >> > >> > > > > > > > >> > >> > > > > >> Hi all, > > >> > >> > > > > >> > > >> > >> > > > > >> Currently, the new proposal will be added to the issue > > >> list > > >> > >> and then > > >> > >> > > > > shared > > >> > >> > > > > >> link in the email > > >> > >> > > > > >> to request the proposal review. It's really hard to > > >> review a > > >> > >> long > > >> > >> > > > > proposal > > >> > >> > > > > >> if you want to comment > > >> > >> > > > > >> in detail. > > >> > >> > > > > >> > > >> > >> > > > > >> Here is an example: > > >> > >> > > > > >> > > >> > >> > > > > >> > > https://github.com/apache/pulsar/issues/16763#issuecomment-1219606491 > > >> > >> > > > > >> This seems very unintuitive. > > >> > >> > > > > >> > > >> > >> > > > > >> I think we can move all the PIPs to the codebase and > the > > >> new > > >> > >> > > proposal > > >> > >> > > > > and > > >> > >> > > > > >> proposal without > > >> > >> > > > > >> any reviews should happen with a PR first. So that we > can > > >> > >> review and > > >> > >> > > > > >> comment easily. > > >> > >> > > > > >> Certainly, all the votes should happen on the mailing > > >> list. > > >> > >> And we > > >> > >> > > can > > >> > >> > > > > also > > >> > >> > > > > >> discuss the > > >> > >> > > > > >> proposal on the mailing list. > > >> > >> > > > > >> > > >> > >> > > > > >> Following this way, we don't need to sync the PIPs > from > > >> the > > >> > >> issue to > > >> > >> > > > the > > >> > >> > > > > >> wiki page. > > >> > >> > > > > >> We can just add a link that points to the PIPs dir to > the > > >> > >> > > contribution > > >> > >> > > > > >> guide or README. > > >> > >> > > > > >> > > >> > >> > > > > >> We have another pain point about the duplicated PIP > > >> number. > > >> > We > > >> > >> can > > >> > >> > > > > maintain > > >> > >> > > > > >> a file, a list of > > >> > >> > > > > >> all the proposal contains the approved, in-review, > > >> drafting. > > >> > >> Before > > >> > >> > > > > >> creating a proposal, we should > > >> > >> > > > > >> have a discussion first on the mailing list, just get > > >> > feedback > > >> > >> on > > >> > >> > > the > > >> > >> > > > > >> motivation. If there are no objections, > > >> > >> > > > > >> the proposal owner can add a line to the file with > the PIP > > >> > >> number > > >> > >> > > > > through a > > >> > >> > > > > >> PR, like PIP-123: xxx (Under Discussion). > > >> > >> > > > > >> So that we can prevent the duplicated PIP number(which > > >> will > > >> > >> conflict > > >> > >> > > > if > > >> > >> > > > > >> someone merged first). > > >> > >> > > > > >> After the PR is merged, we can send out a new PR to > add > > >> the > > >> > >> > > proposal. > > >> > >> > > > > >> > > >> > >> > > > > >> Thanks, > > >> > >> > > > > >> Penghui > > >> > >> > > > > >> > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > >> > > > > >> > > > >> > > > > > >
