I have not seen any specific asks or examples that give rise to these gripes, so it's all hypotheticals as of now. Please, as Enrico said, bring up specific issues.
(1) How many reviewers (less than 1) have been involved in PIP review outside of the streamnative provider and how many of them have experience with Pulsar for more than 2 years (less than 1 or 2)? This means that not all of the reviewers are aware of the entire system or its history. If their opinions do not match the proposed solutions, then it means that a contributor who is not a streamnative provider cannot contribute to Pulsar and have the desired feature.... (2) How long it takes for streamnative contributors to move forward with improvements (less than 2 weeks) and how long it takes for other contributors (more than a couple of months or even forever). .. If I were to summarize your listed issues, they are about (1)meritocracy and (2) community. (1)People who have worked more on something know more about it. There is simply nothing that can be done to make a newcomer on the same footing as a veteran when it comes to expertise, unless the newcomer is willing to put in the time to learn and gain the understanding. There is a big difference between every idea being considered vs every idea being accepted. If one has an idea/contribution that can stand on its own merits, and has the technical justifications to back it up, then it would get accepted. I would look for other reasons, before accusing people (2) This is a community. Nobody owes anyone anything . Including providing help or support. I do not think "that guy/those guys are not helping me" or "I am not getting timely help with reviews" is a ground for griping. Everyone volunteers their time, and it is not infinite. Would it be nice if everyone helped everyone else out and had the time to do everything? Yes. Does it work that way? Not perfectly. Can it be better? Yes. But you are not going to get there blaming people. One needs to build relationships and collaboration with one's peers. That is not something that can be ordered from your peers in the community. They are all volunteers. There are ASF community rules, but rules never built a community. People and relationships do. >There are many such examples > that we can see, where contributors have requested pluggable features such > as security, rate limiting, etc. These requests were legitimate, but only > streamnative contributors will have the right to control Pulsar. > As someone who has repeatedly said no to poorly thought out features in Pulsar, what one user considers "legitimate" is debatable. Due to a lack of a clearly spelled out architectural vision, it is common to get feature requests that don't align well with Pulsar, or not well thought out, or just a niche use case for some specific user which does not fit with a general streaming platform. And for that matter, I have been ignored too - that's community for you Again, I don't know what your PIPs were, but I would not automatically assume that everything submitted as a PIP should be accepted, nor it is incumbent upon the community to make every PIP evolve, mature and get released. People volunteer time and talent - and for what interests them in Pulsar. It is entirely up to them to prioritize as they wish. -joe On Wed, Mar 6, 2024 at 12:49 PM Kalwit S <skalwit...@gmail.com> wrote: > Thanks for your reply. But I wasn’t going to go after a specific person. I > just wanted to point out this is an example of what we’ve been seeing for a > while now. And I wanted to point out why we’re not going to go with Pulsar. > Because if Streamnative is on the same trajectory as Confluent, then > there’s not a lot of value for either of us to put in a lot of time and > energy into migrating our systems. > I’ll share what we’ve seen since we started using Pulsar. Other > contributors can comment if they don’t agree. > > (1) How many reviewers (less than 1) have been involved in PIP review > outside of the streamnative provider and how many of them have experience > with Pulsar for more than 2 years (less than 1 or 2)? > This means that not all of the reviewers are aware of the entire system or > its history. If their opinions do not match the proposed solutions, then it > means that a contributor who is not a streamnative provider cannot > contribute to Pulsar and have the desired feature. On the other hand, if > the same feature is provided by a streamnative provider, then the same > enhancement could easily be added to Pulsar. There are many such examples > that we can see, where contributors have requested pluggable features such > as security, rate limiting, etc. These requests were legitimate, but only > streamnative contributors will have the right to control Pulsar. > > (2) How long it takes for streamnative contributors to move forward with > improvements (less than 2 weeks) and how long it takes for other > contributors (more than a couple of months or even forever). This means > that no matter how much time and effort you put into contributing the right > solution to Pulsar, it won’t get approved on time or will never be approved > at all because only streamnative contributors review them and this review > will wait forever for 3 approvals to merge your enhancements. Streamnative > provider isn’t motivated to review all the enhancements, but they have set > a limit for you to get certain approvals and block every small > feature required by other companies. > > Many companies have faced similar issues with Confluent. Pulsar was an > alternative solution, but unfortunately, we have come to the conclusion > that Pulsar isn’t better with a streamnative provider and reviewers. We > also found major bugs in each Pulsar release, which forced us to > continually upgrade Pulsar with little to no stability. With unstable > releases, a lazy review process, and a single provider-driven system, > Pulsar will be extremely risky to use for any company and we would rather > continue with our legacy Kafka clusters. > > On Wed, Mar 6, 2024 at 6:19 AM Dave Fisher <w...@apache.org> wrote: > > > > > > > > On Mar 6, 2024, at 4:50 AM, Lari Hotari <lhot...@apache.org> wrote: > > > > > > Hi Kalwit, > > > > > > In Apache Pulsar, as in any other Apache project, everyone is equal > > > regardless of their committer or PMC status. All project participants > > > are free to express their opinions, and, where appropriate, have the > > > community consider them when decisions are made. [1] > > > > > > I'm sorry to hear that you haven't felt this way. I believe we can find > > > a resolution. > > > > > > We hold an open community meeting every two weeks on Zoom [2]. Anyone > > > can join these meetings. The agenda is maintained in a Google Doc [3], > > > where you can add items before the meeting. During the meeting, we > > > consider all agenda items. If we run out of time, we prioritize the > > > leftover items for the next meeting. The meeting's scope is the > > > development of Apache Pulsar and the community; it is not a place to > > > get free support in using Pulsar. > > > > That’s all very nice, but the Community meeting may not be in a time that > > is easy to engage with on a global scale. > > > > Discussions on an ASF project should be on the mailing lists for three > > important reasons: > > > > 1. Individuals can participate asynchronously on a global scale. > > 2. Conversations are archived and can be reviewed years after they > > occurred. > > 3. Technical issues discussed on slack and the meeting are difficult to > > share with the community because they are in a silo. ( > > > > > > > > In remote work, there's a chance of feeling left out even when no one > > > intends to exclude you. In Apache Pulsar, we do have a problem with > > > responding to issues and pull requests in a reasonable time. It can > feel > > > disheartening when you put significant effort into making a > > > contribution, and it goes unnoticed or ignored. I understand this > > > feeling. It also feels discouraging when someone gives you feedback > > > about something minor in your contribution, which doesn't really help > > > resolve the problem you're addressing. This happens when we're busy and > > > don't spend enough time caring about others' contributions. Apache > > > Pulsar's development is handled by the community, and there's no > company > > > running this project. Therefore, when you don't get feedback, there's > no > > > company to blame. We need to resolve the problems together in the > > > Apache Pulsar project. The decision process of Apache [1] does not > > > emphasize the committer or PMC status. I hope everyone could feel > > > empowered to express their opinions and have the community consider > > > them in order to make decisions to improve. It's also good to remember > > > that it's all volunteers and if nobody volunteers to do something that > > has > > > been suggested, it won't happen. > > > > This is correct. > > > > > > > > We need to find ways to improve the Apache Pulsar project so that > > > contributors can receive feedback in a reasonable time. Currently, > > > It is possible for the contributor to join the community meeting to > > > request feedback or promote their contribution on Pulsar Slack's #dev > > channel > > > until someone responds. However, this solution where "the loudest voice > > gets > > > the most attention", doesn't scale very well and limits the Pulsar > > project. > > > It also makes contributing feel very painful and not very welcoming. > > > > This is the trouble with slack which is also time limited to 90 days of > > history. > > > > Yesterday I saw a comment about a slack channel to discuss a PIP and that > > is wrong for two reasons: > > 1. The reasoning and discussion are not shared. You have to “know” its > > happening. > > 2. The reasoning and discussion will be unavailable to anyone trying to > > understand why in a very short amount of time. > > > > If you have a problem please use the mailing list. This has been a > > standard in an ASF project for 25 years for good reasons. > > > > > > > > I agree with Enrico's suggestion of starting a new thread to address > > > these problems. It's a good way to begin constructively finding a > > > solution. Perhaps defining the problems we have in the Pulsar community > > > would be a good starting point. One clear indication of a problem is > the > > > number of open pull requests we have in apache/pulsar; currently, there > > > are 320 open pull requests. > > > > If you want to review PRs then please do so in a way that is a > > conversation. Do it in the PR or issue. The only place the discussion > > should be moved is here on this mailing list in public. > > > > > > > > I believe something good comes out of this. Looking forward to > > contributions > > > and volunteers to help fix things. > > > > > > -Lari > > > > > > 1 - Decision making in Apache projects: > > > https://community.apache.org/committers/decisionMaking.html > > > 2 - Calendar and Zoom link to meeting: > > > https://github.com/apache/pulsar/wiki/Community-Meetings > > > 3 - Meeting agenda doc: > > > > > > https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit > > > > > > On Wed, 6 Mar 2024 at 09:32, Enrico Olivelli <eolive...@gmail.com> > > wrote: > > >> > > >> Kalwit, > > >> All the committers are invited by the PMC, you can reach out to > > >> priv...@pulsar.apache.org if you have any problems. > > >> > > >> Being a committer is not only about doing code contributions, but also > > >> talking care of the project and the community. > > >> > > >> I am sorry to hear that you feel that your contributions are blocked, > > >> please start a new thread with this problem. > > >> > > >> This is a community, it is not a product managed by one single > company. > > >> > > >> Best regards > > >> Enrico Olivelli > > >> > > >> Il Mer 6 Mar 2024, 06:27 Matteo Merli <matteo.me...@gmail.com> ha > > scritto: > > >> > > >>> I will not enter into debating the long list of grievances here, > > though I > > >>> thought I needed to clarify at least 2 points: > > >>> > > >>> 1. You can ask any questions and direct any feedback to the PMC (and > if > > >>> you're not happy with the response you can take it all the way up to > > the > > >>> ASF Board), but personal attacks are not OK here > > >>> > > >>> 2. I don't think it's good looking when you're reacting to people > > >>> disagreeing with you by claiming they either are incompetent or have > > some > > >>> hidden agenda. Perhaps trying to understand why they disagreed with > you > > >>> would be more helpful. > > >>> > > >>> > > >>> Matteo > > >>> > > >>> -- > > >>> Matteo Merli > > >>> <matteo.me...@gmail.com> > > >>> > > >>> > > >>> On Tue, Mar 5, 2024 at 7:22 PM Kalwit S <skalwit...@gmail.com> > wrote: > > >>> > > >>>> Congratulations Asaf. > > >>>> > > >>>> Btw, does the Apache project have any promotion criteria for > > committers? > > >>> I > > >>>> looked at Asaf's commits at > > >>>> https://github.com/apache/pulsar/commits?author=asafm and found > that > > >>> 99% > > >>>> of the commits are simple documentation changes and 1% are related > to > > PIP > > >>>> monitoring. Most of the PIP monitoring involves adding plugins to > > >>> existing > > >>>> metrics APIs. He has also contributed to the PIP reviews, but his > > >>>> contribution is more philosophical rather than technical. Most of > his > > >>>> comments are comparing Pulsar to other projects, rather than > focusing > > on > > >>>> the internal insights that Pulsar brings to the table. Our team has > > been > > >>>> running production traffic using Apache Pulsar for over a year now. > We > > >>> have > > >>>> tried several different versions of Pulsar (which we have to > > constantly > > >>>> upgrade due to unknown issues in live production traffic) and have > > never > > >>>> seen a stable version of Pulsar. Our team has also tried to submit > > >>> multiple > > >>>> enhancements and also PIP, but most of them are bogged down by > > reviewers > > >>>> who are very new to Pulsar, might not understand messaging > correctly, > > or > > >>>> don’t find such enhancements useful for their usecases. > > >>>> I would say that most of these reviewers are brand new to Pulsar, > and > > >>>> almost all of them are from the same company that is also the > > provider of > > >>>> Pulsar. The same company controls Pulsar, prevents others from > > >>>> contributing, and avoids having non-pulsar committers. This is why > we > > >>>> wanted to replace our existing Kafka cluster with Pulsar but we see > no > > >>>> difference in Pulsar provider and Confluent because Pulsar is also > > >>> largely > > >>>> controlled by one provider and this company's reviewers are not > > >>> well-versed > > >>>> in such systems. > > >>>> In addition, we can see that almost all the reviewers are from the > > same > > >>>> company, and PIP approval requires 3+ votes, which means only > specific > > >>>> reviewers belonging to one company participate and because of that, > no > > >>> one > > >>>> can promote their improvements without the approval of the provider > > >>>> company. The Pulsar community needs to break away from the > monopolies > > of > > >>>> the provider companies, start focusing on stable releases, and let > > other > > >>>> companies make their enhancements to meet their requirements, and > > >>>> experienced contributors or Pulsar creators should be active to > > prevent > > >>>> unfairness in the community. > > >>>> > > >>>> On Tue, Mar 5, 2024 at 3:31 PM Kalwit S <skalwit...@gmail.com> > wrote: > > >>>> > > >>>>> Congratulations.! > > >>>>> > > >>>>> > > >>>>> On Tue, Feb 20, 2024 at 8:50 AM Lari Hotari <lhot...@apache.org> > > >>> wrote: > > >>>>> > > >>>>>> The Apache Pulsar Project Management Committee (PMC) has invited > > >>>>>> Asaf Mesika https://github.com/asafm to become a committer and we > > >>>>>> are pleased to announce that he has accepted. > > >>>>>> > > >>>>>> Welcome and Congratulations, Asaf Mesika! > > >>>>>> > > >>>>>> Please join us in congratulating and welcoming Asaf onboard! > > >>>>>> > > >>>>>> Best Regards, > > >>>>>> > > >>>>>> Lari Hotari > > >>>>>> on behalf of the Pulsar PMC > > >>>>>> > > >>>>> > > >>>> > > >>> > > > > >
