We have a single tenant application that we deploy to a kubernetes cluster in many instances. Every customer has several environments of the application. Each application lives in a separate namespace and should be isolated from other applications.
We plan to use kafka to communicate inside an environment (between the different pods). As setting up one kafka cluster per such environment is a lot of overhead and cost we would like to just use a single multi tenant kafka cluster. Let's assume we just have one topic with 10 partitions for simplicity. We can now use the environment id as a key for the messages to make sure the messages of each environment arrive in order while sharing the load on the partitions. Now we want each environment to only read the minimal number of messages while consuming. Ideally we would like to to only consume its own messages. Can we somehow filter to only receive messages with a certain key? Can we maybe only listen to a certain partition at least? Additionally we ideally would like to have enforced isolation. So each environment can only see its own messages even if it might receive messages of other environments from the same partition. I think in worst case we can make this happen by encrypting the messages but it would be great if we could filter on broker side. Christian -- -- Christian Schneider http://www.liquid-reality.de Computer Scientist http://www.adobe.com
