Hi all,
In the last 2 months, I've increased my PIP review time (I do it in
cycles), and reviewed quite a few PIPs.
My conclusion as a result of that:
It's nearly impossible to review PIPs using a mailing list.
We must fix it soon.
*Why?*
1. Let's say you review the PIP and find 10 issues. Once you quote and
comment on those ten points, you basically started 10 threads of
conversations.
After 2-3 ping pongs with quotes of quotes of quotes, it takes you forever
to read each thread properly. You find your CTRL-F to search to find your
original quote, and reply. Load it up again in your head, switching to the
PIP tab to read it again.
After 10 ping pongs, it becomes almost an impossible mission.
I can say I'm 75% tired just from the process, not from the review itself.
2. It's non collaborative by design.
After 10 ping pongs, the ability of someone to come and join the discussion
is 0. They need to go through so many replies, which are half quotes, find
the original reply, and browse to the PIP.
It's no wonder people drop off the PIP review once we cross 5-6 replies.
It's no wonder, nobody joins after 10 replies.
3. It's not open to the public. Non collaborative.
You can't just get a link, and join the review. Not only because of (1) and
(2). You need to join the mailing list. You don't get the past emails to
reply. Just joining the list is a high enough bar for many people.
I personally participated in review of proposals in OpenTelemetry in the
last 6 months, even though I'm just an occasional user. Why? They were
conducted on GitHub PR , so it was easy for me - click a link and reply.
4. All over the place
Sometimes people comment on the GitHub issue.
Sometimes on the mailing list.
Not a single place.
5. No history.
Ok, finally the author was convinced. I can't see just the changes. They
need to explicitly tell me something was changed.
6. Delete All.
They can go crazy, after 1 year, edit the GitHub issue , delete all the
text and write "Kafka is the king". No history, nobody can stop them. It's
their issue.
7. Show me all the approved PIPs
Hard to track it today, hard to maintain it updated.
8. Resolved comments
Even though you managed to read all 35 replies so far, in reply 36 you see
the author agreed to all 8 out of 10 suggestions. You have no idea of
knowing that in advance. You just wasted 1 hour.
*What do I suggest?*
PR is the main tool we have that allows multiple threaded discussion.
Git provides history. You can't delete it without approval from PMC members.
1. We'll create a folder named "pip" in the pulsar main repo. It will
contain one markdown file for each PIP. The file will be named
"pip-xxx.md".I will write below how to obtain XXX before you start.
2. To create a PIP, you grab "pip/template.md" and use it to compose your
file in the pip folder.
3. You submit this file as a PR named "PIP-xxx: short description".
4. You create "[DISCUSS] PIP-xxx: short description" in the DEV mailing
list and refer people to your PR, with short text explaining the gist of it.
5. People discuss using PR comments, each is its own threaded comment.
6. Comment was done discussion? They resolve it. This way you see what the
pending discussions are at a glance.
7. Reached consensus? Good. Send "[VOTE] PIP-xxx: short description" on DEV
mailing list.
8. PIP approved? Awesome. Push commit with link to vote.
A PMC member will merge it.
Merge == approved.
PMC members can add a PIP label.
9. Rejected? All good. Close the PR.
Closed == Rejected.
It can't be deleted. All comments are still here.
Before you start, you search Pull Requests with label PIP in GitHub (`is:pr
"PIP-" in:title`)
Take the biggest number and add 1.
It is super rare to have two people create PR at the same time.
*Show me all approved PIPs:*
Search merged PRs labeled PIP.
Look at "pip" folder
*Show me rejected PIPs:*
Search closed PRs with "PIP-" in title, or labeled PIP.
This is very similar to how Apache BK works.
WDYT?
