Hi Kiryl, Thanks for mentioning pulsar-grpc. Indeed, using gRPC simplifies the implementation of the networking logic (keep-alive, reconnection, flow control,…). On the other hand, the Java gRPC implementation makes a lot of buffer copies to cleanly separate the network and app layers but that takes a toll on performance. Compared to that, the broker Pulsar protocol impl is optimized to not do copies between the consumer/producer endpoints and the bookkeeper client. So I think we could not replace completely the Pulsar protocol by gRPC. We could maybe support both but it’s a lot of work to maintain both protocols. (I kind of gave up maintaining pulsar-grpc because of the amount of work compared to the number of users, but if there’s interest I can reconsider). Another possibility would be to do a proxy instead of a low-level protocol handler. A bit like the WebSocket proxy. This would be far less work to maintain as it would use the Pulsar protocol to communicate with the brokers. It could be done as a Proxy extension. Compared to the WS proxy, this would provide type safety, discovery, and so on… As for the Admin, it’s a bit the same. It would be a bunch of work to support both gRPC and REST. You have some kind of type hinting with the OpenAPI spec that you can use to generate client SDKs (eg. with openapi-generator. I wonder what others have to say.
Christophe Le mar. 31 oct. 2023 à 19:57, Kiryl Valkovich <kiryl_valkovich@teal.tools> a écrit : > Hi! Am I understanding it right, that if this project > https://github.com/cbornet/pulsar-grpc is merged to the apache/pulsar > repo, we could easily cover non-mainstream platforms that are supported by > gRPC, but don't have ready-to-use Pulsar clients? > > https://github.com/apache/pulsar/wiki/PIP-59:-gPRC-Protocol-Handler > > Similar to already supported language-agnostic client interfaces - REST > and WebSocket. > > Actively maintained gRPC libraries I found (19, or 15 if considering JVM > languages and web as duplicates): > - [C# / .NET](https://grpc.io/docs/languages/csharp/) > - [C++](https://grpc.io/docs/languages/cpp/) > - [Dart](https://grpc.io/docs/languages/dart/) > - [Go](https://grpc.io/docs/languages/go/) > - [Java](https://grpc.io/docs/languages/java/) > - [Kotlin](https://grpc.io/docs/languages/kotlin/) > - [Node](https://grpc.io/docs/languages/node/) > - [Objective-C](https://grpc.io/docs/languages/objective-c/) > - [PHP](https://grpc.io/docs/languages/php/) > - [Python](https://grpc.io/docs/languages/python/) > - [Ruby](https://grpc.io/docs/languages/ruby/) > - [OCaml](https://github.com/dialohq/ocaml-grpc) > - [Haskell](https://github.com/awakesecurity/gRPC-haskell) > - [Elixir](https://github.com/elixir-grpc/grpc) > - [Rust](https://github.com/hyperium/tonic) > - [Scala](https://scalapb.github.io/) > - [Swift](https://github.com/grpc/grpc-swift) > - Web client: https://github.com/grpc/grpc-web > - Web client 2: https://connectrpc.com/docs/web/getting-started > > Actively maintained Pulsar libraries (8): > - Java > - C++ > - Python > - Go > - Node.js > - C# > - PHP > - Rust > > Is there any reason for not merging it to the apache/pulsar? > > I would definitely prefer to work with a statically typed gRPC client > instead of REST or WebSocket. > > By the way, the same can work for the Pulsar Admin API. Implement the gRPC > server once in Java, and we have full-featured native statically typed > (where applicable :)) Pulsar Admin clients for any platform. > > Best, > Kiryl >
