lukestephenson opened a new issue #7028:
URL: https://github.com/apache/pulsar/issues/7028


   **Is your feature request related to a problem? Please describe.**
   I'm evaluating Pulsar. A primary use case of mine is large (more than 1 
billion unique keyed messages) compacted topics. To handle these efficiently, I 
had planned to make use of multiple partitions on the brokers, as well as have 
multiple consumers with a "key shared" subscription.
   
   Currently the pulsar client rejects setting up a subscription on a compacted 
topic unless the subscription type is exclusive or failover.
   
https://github.com/apache/pulsar/blob/v2.5.2/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientImpl.java#L309-L315
   
   **Describe the solution you'd like**
   Ability to have multiple consumers sharing a subscription to a compacted 
topic (partitioned or not partitioned).
   
   **Describe alternatives you've considered**
   Allocate more resources to a single consumer. Might not be easy in 
kubernetes, we might get to a point where the pod needs more resources than the 
nodes in our cluster.  I don't believe this is the correct solution.  We should 
be building services that can scale out to many small nodes from the beginning.
   
   I'd be happy as well with each consumer being allocated a different 
partition from the topic (i.e. similar to what Kafka does).
   
   **Additional context**
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to