Hi Kafka folks, I'm investigating about using Kafka as a data ingress endpoint for our analytic service. In our scenario, Kafka cluster is behind NLB which has a public IP, and the data source service is under another public IP. Basically the data source service cannot access Kafka cluster nodes directly. And a request hitting the public IP will be routed to any broker node. If Kafka supports request forwarding, then our scenario will work. I noticed there's an article " http://grokbase.com/t/kafka/users/1441aa74t3/kafka-cluster-behind-a-hardware-load-balancer"; and Jun Rao clearly stated that Kafka does not do request forwarding.
I'm wondering if this conclusion is still true for latest Kafka version? If that, I'm wondering do you have any plan to add the support of request forwarding? If not, what's the best practice do you suggest to solve the problem? Here're what I think of: 1) Make the data source service have direct access to Kafka cluster nodes---unfortunately this is impossible for us. 2) Make all Kafka cluster nodes have public IP so that they can be accessed from our data source service. The concerns are a) we may not have enough IPs, b) the security. 3) Add a proxy service which in the same network of Kafka, to act as data ingress service. What do you suggest? I'm new to Kafka and looking forward your reply. Thanks a lot. Tony
