On 2018/09/14 21:00:10, Colin McCabe <cmcc...@apache.org> wrote:
> On Fri, Sep 14, 2018, at 00:46, lbari...@gmail.com wrote:
> > Hello, Colin!
> >
> > I have read the discussion on KIP-273 and want to raise this question.
> > Deploying Kafka to Kubernetes is a crucial problem in my current work.
> > And usage of ZooKeeper makes troubles, because it works unstable in
> > container.
>
> Hi Ibarikyn,
>
> I haven't heard any reports of ZooKeeper being unstable in containers. There
> are a lot of companies already deploying Kafka and ZooKeeper this way, so I
> am surprised to hear that you had trouble. Can you be a little more clear
> about what issues you are seeing with ZooKeeper in containers?
>
> > In this discussion you have mentioned that the goal is to get rid of it
> > in future releases. But since then there were no posts about it. Also, i
> > can not see such plan in a roadmap. Cuold you please tell us more about
> > work made in this stream.
>
> ZooKeeper (and etcd, and consul, and the others) were originally designed as
> configuration management systems. They're really not set up to do what Kafka
> does with them, which is use them as one half of a data management system.
> We need things like uniform access control, the ability to cache metadata
> locally, the ability to propagate only what has changed to brokers, etc.
> That requires us to manage our own metadata log. Kafka is a system for
> managing logs, so this is a natural design.
>
> There have been a lot of discussions about pluggable consensus before on the
> mailing list. Even if we wanted to keep using a configuration management
> system for metadata, pluggable consensus is a really bad idea. It means we
> would have to support multiple systems, getting correspondingly less testing
> in each. We also would have to use least common denominator APIs, since each
> system is somewhat different. And configuration would get harder for new
> users.
>
> best,
> Colin
>
Hello,
Thank you for the answer!
I can see your point and agree with some ideas (eg complexity of configuration
custom systems for users).
Some troubles using Zk that I mentioned:
- it doesnt suuport ssl (at least the version that is used in Kafka)
- cluster being unstable
- dns names confusing
But I talking about another issue. As far as I am concerned, getting rid of
dependency to Zk(and other configuration management) is the right direction.
But you did not evaluate the time when we will see it in Kafka. I would like to
know, is it WIP or smth like that. Do u have any evaluations when this feature
will be released?
BTW, how Zk-less version is supposed to work? What technology will be used as a
metadata container?
As far as I am very interested in this feature, I d like to try to contribute
my work. Could u show me issues/tickets so i can get acquainted with the
situation and maybe start work.
