Hello, We have a scenario where we will be having a large number of producers( ~50k instances - 1 Producer per instance) which will be sending data to a kafka topic. The producers could be inactive for a long time and then be asked to send a message(all at the same time) and as quickly as possible.
What is the best way to do this? Should we keep the Producer connection open throughout? Or do we create a new Producer each time we need to send a message? If the connection is not closed, I believe Kafka discards the connection after a specified timeout. Also, Does Kafka have any limit on the number of incoming connection requests? (I read a few blog posts that said that 300k connections were handled without problems) Thank you, Anirudh
