You can configure "advertised.host.name" for each broker, which is the name external consumers and producers will use to refer to the brokers.
On Tue, Oct 6, 2015 at 3:31 PM, Tom Brown <tombrow...@gmail.com> wrote: > Hello, > > How do you consume a kafka topic from a remote location without a dedicated > connection? How do you protect the server? > > The setup: data streams into our datacenter. We process it, and publish it > to a kafka cluster. The consumer is located in a different datacenter with > no direct connection. The most efficient scenario would be to setup a > point-to-point link but that idea has no traction with our executives. We > can setup a VPN; While functional, our IT department assures us that it > won't be able to scale. > > What we're currently planning is to expose the kafka cluster IP addresses > to the internet, and only allow access via firewall. Each message will be > encrypted with a shared private key, so we're not worried about messages > being intercepted. What we are worried about is this: how brokers refer to > each other-- when a broker directs the consumer to the server that is in > charge of a particular region, does it use the host name (that could be > externally mapped to the public IP) or does it use the detected/private IP > address. > > What solution would you use to consume a remote cluster? > > --Tom >
