Thank you all for your inputs, so going by the opinions I guess its a lot
to do with use case and how the clusters will evolve over time. Will keep
these in mind. Thanks !
On Sun, Jun 4, 2017 at 10:13 AM, Michal Borowiecki <
michal.borowie...@openbet.com> wrote:
> We are indeed running this
We are indeed running this setup in production, have been for almost 2
years now, over a gradually increasing number of deployments.
Let me clarify though:
Our clusters don't exceed 5 nodes. We're not exercising Kafka nowhere
near its limits, or bandwidth or disk I/O for that matter.
When
Hi,
I would not recommend running this kind of set up in production. Busy Kafka
brokers use up a lot of disk and network bandwidth, which zookeeper does
not deal well with. This means that a burst of traffic to 1 node carries
the risk of disrupting the ZK ensemble.
Secondly, this will cause
I'm not an expert but I prefer keeping zookeepers on the same hosts as
kafka brokers and mimic each-others topology. The reason is to minimize
the chance of e.g. kafka brokers being able to talk to one another but
zookeepers not, or vice-versa. So, I'd say I /do/ want my kafka broker
and the
Usually, the overhead comes when you have kafka and zookeeper doing the
housekeeping (i.e. Disk IO) on the same server. ZK even suggests that you
should keep their logs on totally different physical machine for better
performance. Furthermore, if a mechanical failure occurs, you might not
want
Hi,
What are the pros and cons of setting up Zookeeper on the same server as
the Kafka broker ? Earlier offsets were being written to zookeeper which
was a major overhead but with offsets being written to Kafka now, what
other requirements should be taken into consideration for setting up
