Hi. I am thinking to use kafka to send/receive messages for a large number of topics - order of 100k - 1M.
It seems that the directory structure used for topics will probably not work for this usage. Also, I'm not sure if the in-memory data structures might suffer - and also it may be problematic for zookeeper. One thought I have is to modify the directory structure to be a tree of directories. Not sure what if anything might need to be done to in-memory structures or zookeeper info. Any thoughts?
