Docs are not updated since 0.8 is not yet released.
Thanks,
Neha
On Tue, Nov 20, 2012 at 11:09 AM, Jason Rosenberg wrote:
> Is there a configuration doc page for 0.8 (since apparently there are some
> new settings)?
>
> Jason
>
> On Tue, Nov 20, 2012 at 10:39 AM, Jun Rao wrote:
>
>> That's righ
Is there a configuration doc page for 0.8 (since apparently there are some
new settings)?
Jason
On Tue, Nov 20, 2012 at 10:39 AM, Jun Rao wrote:
> That's right. VIP is only used for getting metadata. All producer send
> requests are through direct RPC to each broker.
>
> Thanks,
>
> Jun
>
> On
That's right. VIP is only used for getting metadata. All producer send
requests are through direct RPC to each broker.
Thanks,
Jun
On Tue, Nov 20, 2012 at 10:28 AM, Jason Rosenberg wrote:
> Ok,
>
> I think I understand (so I'll need to change some things in our set up to
> work with 0.8).
>
>
>> So the VIP is only for getting meta-data? After that, under the covers,
the producers will make direct connections to individual kafka hosts that
they learned about from connecting through the VIP
That's right.
Thanks for your questions !
On Tue, Nov 20, 2012 at 10:28 AM, Jason Rosenberg wr
Ok,
I think I understand (so I'll need to change some things in our set up to
work with 0.8).
So the VIP is only for getting meta-data? After that, under the covers,
the producers will make direct connections to individual kafka hosts that
they learned about from connecting through the VIP?
Jas
I think the confusion is that we are answering a slightly different
question then what you are asking. If I understand you are asking, "do I
need to put ALL the kafka broker urls into the config for the client and
will this need to be updated if I add machines to the cluster?".
The answer to both
On Tue, Nov 20, 2012 at 10:00 AM, Neha Narkhede wrote:
> > By requiring use of a configured broker.list for each client, means that
> > 1000's of deployed apps need to be updated any time the kafka cluster
> > changes, no? (Or am I not understanding?).
>
> The advantage is that you can configure
> By requiring use of a configured broker.list for each client, means that
> 1000's of deployed apps need to be updated any time the kafka cluster
> changes, no? (Or am I not understanding?).
The advantage is that you can configure broker.list to point to a VIP, so you
can transparently change th
Ok,
So, I'm still wrapping my mind around this. I liked being able to use zk
for all clients, since it made it very easy to think about how to update
the kafka cluster. E.g. how to add new brokers, how to move them all to
new hosts entirely, etc., without having to redeploy all the clients. The
Jason,
Auto discovery of new brokers and rolling restart of brokers are still
supported in 0.8. It's just that most of the ZK related logic is moved to
the broker.
There are 2 reasons why we want to remove zkclient from the client.
1. If the client goes to GC, it can cause zk session expiration
Trunk does not have latest 0.8 code yet. We plan to merge 0.8 back
into trunk soon, but it hasn't happened yet
Typically, the number of producers to a production Kafka clusters is
very large, which means large number of connections
to zookeeper. If there is a slight blip on the zookeeper cluster d
I checked out trunk. I guess I assumed that included the latest 0.8. Is
that not right? Am I just looking at 0.7.x+?
Honestly, I don't think it would be a positive thing not to be able to rely
on zookeeper in producer code. How does that affect the discovery of a
kafka cluster under dynamic co
In 0.8 there is no way to use zookeeper from the producer and no connection
from the client. There isn't even a way to configure a zk connection. Are
you sure you checked out the 0.8 branch?
Check the code you've got:
*jkreps-mn:kafka-0.8 jkreps$ svn info*
*Path: .*
*URL: https://svn.apache.org/re
Regarding the poducer/zk connection: if I am using zk to discover the
kafka cluster, doesn't the producer get updates if zk's knowledge of the
cluster changes? Or does it only reconsult zk if the particular kafka node
it was "getting metadata" from goes away? Should I not be using a
"zk.connect"
Starting in 0.8 there is no direct connection from the producer to zk. The
goal here is to make it easy to implement clients in non-java languages and
avoid painful zk upgrades. ZK is replaced by a "get_metadata" api (which in
terms of implementation, of course still just reads from zk--but now the
I forgot to mention, that I'm working with a recent version of the 0.8 code
(Last chaned rev: 1396425).
Jason
On Mon, Nov 19, 2012 at 1:23 PM, Jason Rosenberg wrote:
> I've been doing some testing, with an async producer.
>
> It seems, if I start up the producer, with no zk cluster present, it
I've been doing some testing, with an async producer.
It seems, if I start up the producer, with no zk cluster present, it does
what I expect, that is it waits for a limited time looking for the zk
cluster, and then gives up after the zk.connectiontimeout.ms setting
(6000ms, by default), and fails
17 matches
Mail list logo
