Thanks for your reply. But I wasn’t going to go after a specific person. I just wanted to point out this is an example of what we’ve been seeing for a while now. And I wanted to point out why we’re not going to go with Pulsar. Because if Streamnative is on the same trajectory as Confluent, then there’s not a lot of value for either of us to put in a lot of time and energy into migrating our systems. I’ll share what we’ve seen since we started using Pulsar. Other contributors can comment if they don’t agree.
(1) How many reviewers (less than 1) have been involved in PIP review outside of the streamnative provider and how many of them have experience with Pulsar for more than 2 years (less than 1 or 2)? This means that not all of the reviewers are aware of the entire system or its history. If their opinions do not match the proposed solutions, then it means that a contributor who is not a streamnative provider cannot contribute to Pulsar and have the desired feature. On the other hand, if the same feature is provided by a streamnative provider, then the same enhancement could easily be added to Pulsar. There are many such examples that we can see, where contributors have requested pluggable features such as security, rate limiting, etc. These requests were legitimate, but only streamnative contributors will have the right to control Pulsar. (2) How long it takes for streamnative contributors to move forward with improvements (less than 2 weeks) and how long it takes for other contributors (more than a couple of months or even forever). This means that no matter how much time and effort you put into contributing the right solution to Pulsar, it won’t get approved on time or will never be approved at all because only streamnative contributors review them and this review will wait forever for 3 approvals to merge your enhancements. Streamnative provider isn’t motivated to review all the enhancements, but they have set a limit for you to get certain approvals and block every small feature required by other companies. Many companies have faced similar issues with Confluent. Pulsar was an alternative solution, but unfortunately, we have come to the conclusion that Pulsar isn’t better with a streamnative provider and reviewers. We also found major bugs in each Pulsar release, which forced us to continually upgrade Pulsar with little to no stability. With unstable releases, a lazy review process, and a single provider-driven system, Pulsar will be extremely risky to use for any company and we would rather continue with our legacy Kafka clusters. On Wed, Mar 6, 2024 at 6:19 AM Dave Fisher <w...@apache.org> wrote: > > > > On Mar 6, 2024, at 4:50 AM, Lari Hotari <lhot...@apache.org> wrote: > > > > Hi Kalwit, > > > > In Apache Pulsar, as in any other Apache project, everyone is equal > > regardless of their committer or PMC status. All project participants > > are free to express their opinions, and, where appropriate, have the > > community consider them when decisions are made. [1] > > > > I'm sorry to hear that you haven't felt this way. I believe we can find > > a resolution. > > > > We hold an open community meeting every two weeks on Zoom [2]. Anyone > > can join these meetings. The agenda is maintained in a Google Doc [3], > > where you can add items before the meeting. During the meeting, we > > consider all agenda items. If we run out of time, we prioritize the > > leftover items for the next meeting. The meeting's scope is the > > development of Apache Pulsar and the community; it is not a place to > > get free support in using Pulsar. > > That’s all very nice, but the Community meeting may not be in a time that > is easy to engage with on a global scale. > > Discussions on an ASF project should be on the mailing lists for three > important reasons: > > 1. Individuals can participate asynchronously on a global scale. > 2. Conversations are archived and can be reviewed years after they > occurred. > 3. Technical issues discussed on slack and the meeting are difficult to > share with the community because they are in a silo. ( > > > > > In remote work, there's a chance of feeling left out even when no one > > intends to exclude you. In Apache Pulsar, we do have a problem with > > responding to issues and pull requests in a reasonable time. It can feel > > disheartening when you put significant effort into making a > > contribution, and it goes unnoticed or ignored. I understand this > > feeling. It also feels discouraging when someone gives you feedback > > about something minor in your contribution, which doesn't really help > > resolve the problem you're addressing. This happens when we're busy and > > don't spend enough time caring about others' contributions. Apache > > Pulsar's development is handled by the community, and there's no company > > running this project. Therefore, when you don't get feedback, there's no > > company to blame. We need to resolve the problems together in the > > Apache Pulsar project. The decision process of Apache [1] does not > > emphasize the committer or PMC status. I hope everyone could feel > > empowered to express their opinions and have the community consider > > them in order to make decisions to improve. It's also good to remember > > that it's all volunteers and if nobody volunteers to do something that > has > > been suggested, it won't happen. > > This is correct. > > > > > We need to find ways to improve the Apache Pulsar project so that > > contributors can receive feedback in a reasonable time. Currently, > > It is possible for the contributor to join the community meeting to > > request feedback or promote their contribution on Pulsar Slack's #dev > channel > > until someone responds. However, this solution where "the loudest voice > gets > > the most attention", doesn't scale very well and limits the Pulsar > project. > > It also makes contributing feel very painful and not very welcoming. > > This is the trouble with slack which is also time limited to 90 days of > history. > > Yesterday I saw a comment about a slack channel to discuss a PIP and that > is wrong for two reasons: > 1. The reasoning and discussion are not shared. You have to “know” its > happening. > 2. The reasoning and discussion will be unavailable to anyone trying to > understand why in a very short amount of time. > > If you have a problem please use the mailing list. This has been a > standard in an ASF project for 25 years for good reasons. > > > > > I agree with Enrico's suggestion of starting a new thread to address > > these problems. It's a good way to begin constructively finding a > > solution. Perhaps defining the problems we have in the Pulsar community > > would be a good starting point. One clear indication of a problem is the > > number of open pull requests we have in apache/pulsar; currently, there > > are 320 open pull requests. > > If you want to review PRs then please do so in a way that is a > conversation. Do it in the PR or issue. The only place the discussion > should be moved is here on this mailing list in public. > > > > > I believe something good comes out of this. Looking forward to > contributions > > and volunteers to help fix things. > > > > -Lari > > > > 1 - Decision making in Apache projects: > > https://community.apache.org/committers/decisionMaking.html > > 2 - Calendar and Zoom link to meeting: > > https://github.com/apache/pulsar/wiki/Community-Meetings > > 3 - Meeting agenda doc: > > > https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit > > > > On Wed, 6 Mar 2024 at 09:32, Enrico Olivelli <eolive...@gmail.com> > wrote: > >> > >> Kalwit, > >> All the committers are invited by the PMC, you can reach out to > >> priv...@pulsar.apache.org if you have any problems. > >> > >> Being a committer is not only about doing code contributions, but also > >> talking care of the project and the community. > >> > >> I am sorry to hear that you feel that your contributions are blocked, > >> please start a new thread with this problem. > >> > >> This is a community, it is not a product managed by one single company. > >> > >> Best regards > >> Enrico Olivelli > >> > >> Il Mer 6 Mar 2024, 06:27 Matteo Merli <matteo.me...@gmail.com> ha > scritto: > >> > >>> I will not enter into debating the long list of grievances here, > though I > >>> thought I needed to clarify at least 2 points: > >>> > >>> 1. You can ask any questions and direct any feedback to the PMC (and if > >>> you're not happy with the response you can take it all the way up to > the > >>> ASF Board), but personal attacks are not OK here > >>> > >>> 2. I don't think it's good looking when you're reacting to people > >>> disagreeing with you by claiming they either are incompetent or have > some > >>> hidden agenda. Perhaps trying to understand why they disagreed with you > >>> would be more helpful. > >>> > >>> > >>> Matteo > >>> > >>> -- > >>> Matteo Merli > >>> <matteo.me...@gmail.com> > >>> > >>> > >>> On Tue, Mar 5, 2024 at 7:22 PM Kalwit S <skalwit...@gmail.com> wrote: > >>> > >>>> Congratulations Asaf. > >>>> > >>>> Btw, does the Apache project have any promotion criteria for > committers? > >>> I > >>>> looked at Asaf's commits at > >>>> https://github.com/apache/pulsar/commits?author=asafm and found that > >>> 99% > >>>> of the commits are simple documentation changes and 1% are related to > PIP > >>>> monitoring. Most of the PIP monitoring involves adding plugins to > >>> existing > >>>> metrics APIs. He has also contributed to the PIP reviews, but his > >>>> contribution is more philosophical rather than technical. Most of his > >>>> comments are comparing Pulsar to other projects, rather than focusing > on > >>>> the internal insights that Pulsar brings to the table. Our team has > been > >>>> running production traffic using Apache Pulsar for over a year now. We > >>> have > >>>> tried several different versions of Pulsar (which we have to > constantly > >>>> upgrade due to unknown issues in live production traffic) and have > never > >>>> seen a stable version of Pulsar. Our team has also tried to submit > >>> multiple > >>>> enhancements and also PIP, but most of them are bogged down by > reviewers > >>>> who are very new to Pulsar, might not understand messaging correctly, > or > >>>> don’t find such enhancements useful for their usecases. > >>>> I would say that most of these reviewers are brand new to Pulsar, and > >>>> almost all of them are from the same company that is also the > provider of > >>>> Pulsar. The same company controls Pulsar, prevents others from > >>>> contributing, and avoids having non-pulsar committers. This is why we > >>>> wanted to replace our existing Kafka cluster with Pulsar but we see no > >>>> difference in Pulsar provider and Confluent because Pulsar is also > >>> largely > >>>> controlled by one provider and this company's reviewers are not > >>> well-versed > >>>> in such systems. > >>>> In addition, we can see that almost all the reviewers are from the > same > >>>> company, and PIP approval requires 3+ votes, which means only specific > >>>> reviewers belonging to one company participate and because of that, no > >>> one > >>>> can promote their improvements without the approval of the provider > >>>> company. The Pulsar community needs to break away from the monopolies > of > >>>> the provider companies, start focusing on stable releases, and let > other > >>>> companies make their enhancements to meet their requirements, and > >>>> experienced contributors or Pulsar creators should be active to > prevent > >>>> unfairness in the community. > >>>> > >>>> On Tue, Mar 5, 2024 at 3:31 PM Kalwit S <skalwit...@gmail.com> wrote: > >>>> > >>>>> Congratulations.! > >>>>> > >>>>> > >>>>> On Tue, Feb 20, 2024 at 8:50 AM Lari Hotari <lhot...@apache.org> > >>> wrote: > >>>>> > >>>>>> The Apache Pulsar Project Management Committee (PMC) has invited > >>>>>> Asaf Mesika https://github.com/asafm to become a committer and we > >>>>>> are pleased to announce that he has accepted. > >>>>>> > >>>>>> Welcome and Congratulations, Asaf Mesika! > >>>>>> > >>>>>> Please join us in congratulating and welcoming Asaf onboard! > >>>>>> > >>>>>> Best Regards, > >>>>>> > >>>>>> Lari Hotari > >>>>>> on behalf of the Pulsar PMC > >>>>>> > >>>>> > >>>> > >>> > >
