Thanks Lari. I have a pain point that is about documentation and developer awareness where we can help improve quality and suggest improvements.
1) Work on clear documentation about each thread pool and how it is used by the classes in the message path. 2) Add thread pool impacts to the PR template to help prevent performance regressions and focus on improvements. 3) identify performance bottlenecks that already exist where a thread pool change might help. Regards, Dave Sent from my iPhone > On Oct 7, 2022, at 10:17 PM, Lari Hotari <lhot...@apache.org> wrote: > > Thanks, Dave for sharing your views. I entirely agree. > > -Lari > >> On 2022/10/08 01:41:47 Dave Fisher wrote: >> To me the point of this discussion is for the community to discuss both >> improvement ideas along with current pain points in the broker. Whether or >> not the ideas and proposed solutions are breaking or incremental is yet to >> be determined. Let’s not get hung up on versions or nomenclature. >> >> All the best, >> Dave >> >> Sent from my iPhone >> >>>> On Oct 7, 2022, at 5:54 PM, Matteo Merli <matteo.me...@gmail.com> wrote: >>> >>> I actually disagree with the term "Pulsar Next Gen", because I haven't >>> seen any proposal for which that would make sense to me to be called >>> so. >>> >>> Rajan: That's the whole point of breaking it down. If you accumulate >>> many "big" changes it introduces a lot of risk for instabilities and >>> incompatibilities. Breaking it down in multiple steps helps to see the >>> incremental changes and introduced them in a phased manner. >>> >>> >>> -- >>> Matteo Merli >>> <matteo.me...@gmail.com> >>> >>>> On Fri, Oct 7, 2022 at 3:37 PM Rajan Dhabalia <rdhaba...@apache.org> wrote: >>>> >>>> Hi, >>>> >>>> Can we get the list of changes at one place which we are planning to get as >>>> part of 3.0. One thing I would like to see as a part of a major release, it >>>> CAN NOT impact existing usecases and users in any way which can force them >>>> to upgrade the client library. Applications using < 3.0 version should >>>> continue getting all the client and server side enhancements and bug fixes. >>>> Failing to provide bug-fixes and features to client < 3.0 means we are >>>> forcing them to upgrade client version by putting efforts to handle all >>>> incompatibility. and that's something we should definitely prevent because >>>> Apache Pulsar is used by many large scale business usecases and we should >>>> accommodate and motivate them to continue using Apache Pulsar. >>>> I understand as a Pulsar community we should always try to progress and >>>> build better but not at the cost of losing or reducing the Apache Pulsar >>>> community. >>>> >>>> Thanks, >>>> Rajan >>>> >>>> >>>>> On Fri, Oct 7, 2022 at 12:41 PM Lari Hotari <lhot...@apache.org> wrote: >>>>> >>>>> Thank you, Matteo. I agree that features should be delivered continuously >>>>> when that is possible. In this case, that might not apply. >>>>> >>>>> I also agree that calling this Pulsar 3.0 isn't necessarily aligned with >>>>> PIP-175 since an LTS release is when the major version is bumped. I'm fine >>>>> in calling this "Pulsar Next Gen" or something that calls out that this is >>>>> planning for making a major leap in Pulsar. >>>>> >>>>> There are several unresolved issues with PIP-45 and the Pulsar Load >>>>> balancer. The previously referred email threads contain a lot of context >>>>> to >>>>> this. Resolving the issues efficiently will most likely result in breaking >>>>> changes, which will be the reason why it deserves a major version upgrade. >>>>> >>>>> We have discussed it before that it's crucial to have a path to migrate >>>>> users when there are breaking changes. This should be covered in any of >>>>> the >>>>> solutions that are introduced. Optimally, users of Pulsar would be able to >>>>> upgrade seamlessly to Pulsar Next Gen / Pulsar 3.0, but rolling back might >>>>> not be directly supported. >>>>> >>>>> I am welcoming everyone to join this planning for the Apache Pulsar Next >>>>> Gen architecture. Please check the first email in this thread for details >>>>> of context, and start participating and contributing today. The best way >>>>> to >>>>> contribute is to participate in the email threads, since they contain >>>>> details with better context. >>>>> >>>>> -Lari >>>>> >>>>> On 2022/10/07 18:03:00 Matteo Merli wrote: >>>>>> Given the past experiences and the discussions that already happened >>>>>> around "PIP-175: Extend time based release process", the idea is to >>>>>> detach the 3.0 from "big-features" items or "incompatible changes". >>>>>> >>>>>> The changes are going to get included as they are ready, within >>>>>> feature releases, and in a fully compatible way. We don't need to >>>>>> group them together and create unnecessary risk for the release >>>>>> schedule and the users. >>>>>> >>>>>> >>>>>> -- >>>>>> Matteo Merli >>>>>> <matteo.me...@gmail.com> >>>>>> >>>>>> On Fri, Oct 7, 2022 at 10:47 AM Lari Hotari <lhot...@apache.org> wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Greeting from ApacheCon North America 2022 from New Orleans! >>>>>>> We had a great conference with a dedicated Pulsar track. Thanks to all >>>>> presenters and everyone who attended. The talks weren't recorded, but the >>>>> slides will be later on posted on the conference website [1]. >>>>>>> >>>>>>> At ApacheCon there were several presentations about "the Apache way" >>>>> and what that means in practice. Based on that, we all know that no person >>>>> is nominated as the CTO of Apache Pulsar who decides on Pulsar 3.0 and >>>>> when >>>>> that happens. It's us, the community, that serve that role together. We >>>>> come together as individuals with the Apache hat on. Everyone is equal in >>>>> the community, regardless of whether they are contributors, committers or >>>>> PMC members. >>>>>>> We welcome everyone to participate. The small detail about voting >>>>> shouldn't stop anyone from participating in any aspects of the planning >>>>> for >>>>> the roadmap. >>>>>>> >>>>>>> I'll like to get the discussions going for Pulsar 3.0. We don't need a >>>>> separate decision to start planning that. Please correct me if I'm wrong >>>>> or >>>>> if you have a different opinion. >>>>>>> >>>>>>> There are a few previous discussion threads that are related to Pulsar >>>>> 3.0 planning. >>>>>>> If you are interested in getting involved with Apache Pulsar 3.0 >>>>> planning, I think that it makes sense for you to read these threads >>>>> carefully and reply to them. Please also suggest what you think makes >>>>> sense. >>>>>>> >>>>>>> PIP-45 related: >>>>> https://lists.apache.org/thread/tvco1orf0hsyt59pjtfbwoq0vf6hfrcj >>>>>>> Pulsar Load balancer / namespace bundle related: >>>>>>> https://lists.apache.org/thread/roohoc9h2gthvmd7t81do4hfjs2gphpk >>>>>>> renaming topics: >>>>>>> https://lists.apache.org/thread/vrr75rrh4trqlp14objh3snlfvmzdrp2 >>>>>>> backpressure: >>>>> https://lists.apache.org/thread/v7xy57qfzbhopoqbm75s6ng8xlhbr2q6 >>>>>>> >>>>>>> Long list of Metadata inconsistency issues by Zac Bentley: >>>>>>> https://github.com/apache/pulsar/issues/12555 >>>>>>> That would be a good starting point to understanding the data >>>>> inconsistency issues related to current PIP-45 design. Perhaps those could >>>>> be addressed already before Pulsar 3.0? >>>>>>> >>>>>>> I'm looking forward to everyone's participation in the Apache Pulsar >>>>> 3.0 planning discussions. >>>>>>> >>>>>>> Best Regards, >>>>>>> >>>>>>> -Lari >>>>>>> >>>>>>> 1 - https://www.apachecon.com/acna2022/schedule.html >>>>>> >>>>> >>
