Hi there, sorry just checking the user list for "parallel consumer" - you can use Confluent's Parallel Consumer client library to increase your concurrency far beyond the number of partitions you have available.
For example in your case - you can process 1,000's of messages in parallel without touching the partition count. Have a look here: https://www.confluent.io/confluent-accelerators/#parallel-consumer Latest talk here: https://www.confluent.io/events/kafka-summit-europe-2021/introducing-confluent-labs-parallel-consumer-client/ Github here: https://github.com/confluentinc/parallel-consumer On Thu, Mar 21, 2019 at 9:18 AM Jérémy Thulliez <jeremy.thull...@zenika.com> wrote: > In your case, > > Increase the number of partitions on an existing topic will have an impact > on how the message will be dispatched into partitions. > > I mean if you use the default partitionner, and your message have key, the > algorithm is 'hash(key) % number of partitions' > > That means that a message with the same key, won't go into the same > partition, before and after the change. > > IMO, It was important to notice, in case you need ordering by key. > > Jeremy > -- <https://www.confluent.io> Antony Stubbs Follow us: Blog <https://confluent.io/blog?utm_source=footer&utm_medium=email&utm_campaign=ch.email-signature_type.community_content.blog> • Slack <https://slackpass.io/confluentcommunity> • Twitter <https://twitter.com/ConfluentInc> • YouTube <https://youtube.com/confluent>