On 08.01.2019 17:11, aruna ramachandran wrote: > I need to process single sensor messages in serial (order of messages > should not be changed)at the same time I have to process 10000 sensors > messages in parallel please help me to configure the topics and partitions. >
If you want to process event in order, you only need to make sure that the messages of every sensor go into the same partition. Not every sensor needs partition of its own. Just use sensor_id or UUID or something as a key while producing. Then you can freely pick the number of partitions. Then take the retention time and throughput and try to come out at ~50GB at rest. If you need more throughput you can still use more partitions then best Jan
