It seems that there are already interests in contributing to the project. Why not take more time to develop the project and get the project into a more mature stage first? There is no harm to develop the project in that way.
- Sijie On Thu, Sep 2, 2021 at 12:06 AM Lari Hotari <l...@hotari.net> wrote: > On Thu, Sep 2, 2021 at 9:51 AM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > As Sijie said it is probably too early to bring it in, as the project is > > still very new. > > On the other hand, as far as it is a single person project, I am not sure > > how many people will want to use it in production > > and we want to bring Pulsar to a wider audience. > > > > Reactive Streams is the future of streaming applications on the JVM. > It won't be a single person project. There's already Pulsar community > members that are looking forward to contributing. > Here's one example: > https://github.com/apache/pulsar/issues/10437#issuecomment-869928272 > I have also another person that has contacted me and would like to start > contributing. > > I have understood that it is a different process to accept an existing > project with multiple contributors to Apache. > Sijie and Enrico, please elaborate more about the concerns you have about > the project being new and that I'm the author of it. > I'd like to understand what problem it causes that the project is new. > Isn't that a benefit? > Let's discuss more about your concerns to gain more understanding. > > -Lari > > > > > > > > some answers to other comments inline below > > > > Enrico > > > > Il giorno gio 2 set 2021 alle ore 08:26 Rajan Dhabalia < > > rdhaba...@apache.org> > > ha scritto: > > > > > *My two cents about developing and contributing adapters/connectors for > > > Apache Pulsar. Apache Pulsar was open-sourced with Spark and Storm > > adapters > > > initially and then such adapter families evolved with multiple other > > > adapters, connectors and IO connectors (sink/source) into apache pulsar > > git > > > repo. Such adapters help pulsar for its adaption and user growth. > > However, > > > after adding more adapters in the pulsar repo, we realized two major > > > issues: One of the issues, has made Pulsar git repo bulky which > increases > > > build time and sometimes it’s difficult to build and implement > > integration > > > tests. Another issue is the maintenance of such adapters with new > > releases > > > of the dependent systems and bug fixes for multiple versions. > Therefore, > > we > > > eventually created a separate repository to manage adapters separately > > into > > > pulsar-adapters gitrepo. So, one of the takeaways from past experience, > > if > > > we absolutely need an adapter then we can use a pulsar-adapter gitrepo > > and > > > we should evaluate the feasibility of maintaining such adapters with > > their > > > different versions and their interest in the community. Thanks,Rajan* > > > > > > On Wed, Sep 1, 2021 at 10:59 PM Lari Hotari <l...@hotari.net> wrote: > > > > > > > Replies inline below: > > > > > > > > On Thu, Sep 2, 2021 at 3:37 AM Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > > > Lari, > > > > > > > > > > Thank you for bringing this up! > > > > > > > > > > In general, I would love to see this being accepted to the project. > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > However, in the past, there were other language clients contributed > > to > > > > the > > > > > project. They were not accepted because the PMC doesn't have enough > > > > > bandwidth to staff it. > > > > > > > I think that the community is still growing and also the PMC, if we have > > enough people that are willing to support this > > then I feel we can accept it. > > In particular Lari is already part of the Community as committer and he > > will take care of the project. > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-77%3A-Contribute-Supernova-to-Apache-Pulsar > > > > > is one of the examples. > > > > > > > > > > > > > I don't think that Reactive Streams is comparable to this Supernova > > > > example. That is an example of a very marginal language. > > > > > > > > Reactive Streams support is a mainstream technology that originates > > from > > > > Netflix. It's not about a single programming language. > > > > The Reactive Streams is an interoperability specification and there > are > > > > multiple implementations for the JVM. > > > > > > > > For example, this Reaction Pulsar Adapter library uses Project > Reactor > > as > > > > the implementation, but the library can be used together with Akka > > > Streams, > > > > RxJava 3, Vert.x, SmallRye Mutiny (RedHat/Quarkus) and others. > > > > > > > > This library is not about a single programming language, it's about > all > > > > languages and platforms on the JVM that support Reactive Streams. > > > > It could be an extension directly to the Pulsar Java Client. I just > > > thought > > > > that maintaining the project would be easier when it's a separate > > > > repository. > > > > If that is causing the staffing issue that you referred to being the > > > > showstopper issue, I could also restructure the project so that it > > could > > > be > > > > added as modules to apache/pulsar repository. > > > > > > > > There's a short summary in > > > > > > > > > > > > > > https://www.reactive.foundation/post/why-reactive-principles-ensure-system-scalability-with-josh-long > > > > about the value proposition of Reactive Streams and why it matters. > > > > The recent Netflix Techblog explicitly mentions Reactive Streams in > the > > > > blog post about Netflix's device management platform: > > > > > > > > > > > > > > https://netflixtechblog.com/towards-a-reliable-device-management-platform-4f86230ca623 > > > > "Back-Pressure Support - Because the processing workload varies > > > > significantly over time, the solution must first and foremost scale > > with > > > > the message load by providing back-pressure support as defined in the > > > > Reactive Streams specification — in other words, the solution should > be > > > > able to switch between push and pull-based back-pressure models > > depending > > > > on the downstream being able to cope with the message production rate > > or > > > > not." > > > > > > > > > > > > > Also, the project is still relatively new and there is only one > > > > > contributor. > > > > > > > > > > > > > > Everything has a beginning. :) > > > > I was assuming that this would be optimal that there is only one > > > > contributor and that other contributions could contaminate the IPR > and > > > make > > > > it harder to move to the Apache Pulsar project later. > > > > As mentioned above, this is more like an extension to the Pulsar Java > > > > Client and I could restructure the project so that it could be added > as > > > > modules to the apache/pulsar repository if the separate repository > > > > causes the barrier. > > > > > > > > > > > > > Hence, my recommendation is to grow the contributors for this > project > > > so > > > > > that there are enough contributors to work on that project. Once > > there > > > > are > > > > > sufficient resources and contributors, the PMC can revisit this > > > proposal > > > > > and see if the PMC is able to staff it. > > > > > > > > > > > > > Did the PMC already vote on this? :) I hope we could have the > > discussion > > > > before voting. > > > > Adding support for Reactive Streams will add a lot of value to Apache > > > > Pulsar. > > > > > > > > I'm talking about the value of Reactive Streams for Apache Pulsar > > > > application developers TODAY AT 11:05PT in > > > > my SpringOne talk "Reactive Applications with Apache Pulsar and > Spring > > > > Boot" > > > > > > > > > > > > > > https://springone.io/2021/sessions/reactive-applications-with-apache-pulsar-and-spring-boot > > > > It is free to attend, https://springone.io to register. There's also > > an > > > > AMA > > > > session immediately after the session Q&A. > > > > Please join. Come and see! > > > > > > > > I'm looking forward to hearing more voices from the Apache Pulsar > > > community > > > > in this email discussion. > > > > > > > > BR, Lari > > > > > > > > > > > > > > > > > > - Sijie > > > > > > > > > > > > > > > On Wed, Sep 1, 2021 at 4:31 AM Lari Hotari <lhot...@apache.org> > > wrote: > > > > > > > > > > > Dear Apache Pulsar community members, > > > > > > > > > > > > I have developed a Reactive Streams adapter for the Apache Pulsar > > > Java > > > > > > Client. It uses Project Reactor as the Reactive Streams > > > implementation. > > > > > > > > > > > > The repository is here: > https://github.com/lhotari/reactive-pulsar > > > > > > and library is available in Maven Central: > > > > > > > > > > > > > > > > > > > > > https://search.maven.org/search?q=g:com.github.lhotari%20a:reactive-pulsar* > > > > > > > > > > > > I would like to contribute this library to the Apache Pulsar > > project. > > > > > > I would suggest that the library is committed to a separate > > > repository, > > > > > for > > > > > > example https://github.com/apache/pulsar-reactive . > > > > > > > > > > > > # Motivation > > > > > > > > > > > > There's a need to "go reactive from end-to-end" when building > > modern > > > > > > reactive applications with platforms such as Spring Reactive. > > > > > > > > > > > > There are ways to adapt the Apache Pulsar Java client async API > > calls > > > > to > > > > > > Reactive Streams with a few lines of code. However, a lot will be > > > > missing > > > > > > and achieving the complete solution will require much more > effort. > > > > > > > > > > > > The primary goal of this library is to provide a Reactive Streams > > > > adapter > > > > > > that wraps the Apache Pulsar Java client with Reactive APIs > > > > > > for sending, reading and consuming messages. > > > > > > The secondary goal is to provide a Spring Boot starter for easily > > > > > adopting > > > > > > the power of Apache Pulsar for Spring Boot applications. > > > > > > > > > > > > It is currently out of scope for this library to build a reactive > > > > client > > > > > > all the way from the binary protocol handling. > > > > > > This library builds upon the existing Apache Pulsar Java client > and > > > > uses > > > > > > the async API for operations. > > > > > > > > > > > > # Features > > > > > > > > > > > > - Simple and intuitive reactive APIs for sending, reading and > > > consuming > > > > > > messages. > > > > > > - Non-blocking backpressure support for the provided reactive > APIs > > > > > > - Pulsar client resource lifecycle management that integrates > with > > > the > > > > > > Reactive programming model > > > > > > - Additional producer caching that caches producers. > > > > > > - Useful for API backends since the Pulsar client producer > > resource > > > > can > > > > > > be shared and cached across multiple REST calls. > > > > > > - Support for in-order parallel (concurrent) processing at the > > > > > application > > > > > > level > > > > > > - I/O intensive message processing can leverage Project > Reactor's > > > > > > capabilities to efficiently > > > > > > handle thousands of simultaneous I/O calls to external REST > > APIs > > > > and > > > > > > resources that have Reactive clients/drivers. > > > > > > - In-order parallel processing support can retain key-order at > > the > > > > > level > > > > > > of a single consumer. > > > > > > > > > > > > I'm presenting the library and it's features tomorrow on > September > > > 2nd > > > > in > > > > > > the SpringOne virtual conference, which is free to attend. > > > > > > The session details and schedule are at > > > > > > > > > > > > > > > > > > > > > > > > > > > https://springone.io/2021/sessions/reactive-applications-with-apache-pulsar-and-spring-boot > > > > > > . > > > > > > > > > > > > # Licenses > > > > > > > > > > > > The Reactive Pulsar Adapter library is under the Apache Licence > > > Version > > > > > 2.0 > > > > > > . The main dependencies are reactor-core > > > > > > (https://github.com/reactor/reactor-core) and jctools-core ( > > > > > > https://github.com/JCTools/JCTools) which are also > > > > > > ASL 2.0 licensed. > > > > > > There's also a dependency to Caffeine ( > > > > > > https://github.com/ben-manes/caffeine) > > > > > > and the Spring Boot Starter > > > > > > depends on Spring Boot and Spring Framework. These are all ASL > 2.0 > > > > > licensed > > > > > > libraries. > > > > > > The additional Spring integration testing support library depends > > on > > > > > > Testcontainers, which is also ASL 2.0 licensed. > > > > > > > > > > > > > > > > > > I'm looking forward to the discussion about this. > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Lari Hotari > > > > > > > > > > > > > > > > > > > > >
