Thanks Jun for your quick reply.
If I understood correctly, for the first step ( fetching metadata ),
producer can still go through a VIP whereas for the second step ( once
producer gets the metadata ), producer needs to connect to the correct
instance of the broker that hosts the partition. I
In Kafka, the broker list you provide to the producer is used only for
fetching metadata, which can be served on any broker. The producer then
connects to the right brokers according to the metadata response.
Thanks,
Jun
On Mon, Mar 31, 2014 at 6:04 PM, kafka user wrote:
> If we have a kafka
What's your use case? How frequent do you expect to increase # partitions?
Thanks,
Jun
On Mon, Mar 31, 2014 at 3:27 PM, Kiran Nagasubramanian
wrote:
> Hello,
>
> I'm new to Kafka. I was wondering if it's possible to increase the
> no. of partitions of a topic during runtime(programmati
We also have a metric in the consumer that measures the # of messages a
consumer is behind.
Thanks,
Jun
On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson wrote:
> Hi,
>
> Does anyone know what the JMX metric(s) are for determining the depth of
> the kafka queue? I basically want to know the amoun
Ok. We do have a system test that hard kills the brokers. So far, we
haven't seen data loss there. However, it doesn't run very long and the
number of failures introduced is small. We will try to run a longer version
of that test and see if we can reproduce the issue locally.
Thanks,
Jun
On Fri
If we have a kafka producer connecting to a Kafka cluster behind a hardware
Load Balancer (VIP), will producer be able to send a message to a right
partition ?
Can one of the brokers in a cluster do a broker discovery to forward the
message to ?
I guess my question is whether it makes sense to use
Hello,
I'm new to Kafka. I was wondering if it's possible to increase the
no. of partitions of a topic during runtime(programmatically). I could see
that there're command line tools to achieve this. If I do not have any
other option, I was thinking of solving this by starting with excess n
getBytesIn and getBytesOut in BrokerAllTopicStat
On Mon, Mar 31, 2014 at 11:12 AM, Arnaud Lawson wrote:
> Ok thanks Guozhang . So What would the metrics for the in-traffic and
> out-traffic message count be on this page (this page shows all existing jmx
> metrics)?
>
> https://cwiki.apache.org/c
We've used bytes written as in, and bytes read as out.
Chi
On Mon, Mar 31, 2014 at 11:12 AM, Arnaud Lawson wrote:
> Ok thanks Guozhang . So What would the metrics for the in-traffic and
> out-traffic message count be on this page (this page shows all existing jmx
> metrics)?
>
> https://cwiki.a
Ok thanks Guozhang . So What would the metrics for the in-traffic and
out-traffic message count be on this page (this page shows all existing jmx
metrics)?
https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring
On Mon, Mar 31, 2014 at 1:01 PM, Guozhang Wang wrote:
>
Hi Arnaud,
Currently we do not have metric(s) for the number of messages on the kafka
broker, though we do have the metrics for the in-traffic and out-traffic
message count.
One work-around would be the ConsumerOffsetChecker tool, for which it shows
the last offset of messages on the brokers v.s.
Hi,
Does anyone know what the JMX metric(s) are for determining the depth of
the kafka queue? I basically want to know the amount of messages that are
on the queue at a certain point in time. Please let me know if there is a
way to find that out. Thanks.
--
Arnaud Lawson
,
Systems Operations E
Hi Vijay,
We implemented mutual ssl authentication in kafka for our internal use and
we have plans to it contributed back to community. But we implemented SSL over
older snapshot of version of kafka 0.8 release. We have been busy with
other projects and haven't got chance to merge our ssl changes
13 matches
Mail list logo
