Ping
Any thoughts on this?
Seems like a bug, but then again, we're not sure what the expected behavior
for regexes should be here (e.g. is there a way to whitelist topics with a
filter that looks for a leading substring, but then blocks subsequent
substrings)? E.g. apply a blacklist to a whi
Hi Jun,
Thank you for the quick feedback!
here is the issue https://issues.apache.org/jira/browse/KAFKA-1184
--
Best Regards,
Evelina
On 12/16/2013 08:18 PM, Jun Rao wrote:
It's possible to support multi-fetchers per broker in the consumer. Could
you file a jira to track that? However, typical
If there is a message, hasNext() returns true, not throwing an exception.
Thanks,
Jun
On Mon, Dec 16, 2013 at 11:29 AM, Yu, Libo wrote:
> Hi folks,
>
> For this parameters, if consumer.timeout.ms is set to 0, whenever I call
> ConsumerIterator's hasNext(),
> if there is a message available, a
What version of kafka are you on?
It seems like your producers are not seeing your broker(s). can you
confirm brokers are up?
On Mon, Dec 16, 2013 at 7:52 PM, Tom Amon wrote:
> Hi All,
>
> I have a situation where one producer/consumer is causing timeout errors on
> the Kafka broker. The excep
Hi All,
I have a situation where one producer/consumer is causing timeout errors on
the Kafka broker. The exception in the logs looks like this:
[2013-12-16 17:32:25,992] ERROR Closing socket for /10.236.67.30 because of
error (kafka.network.Processor)
java.io.IOException: Connection timed out
For anyone that's interested, I did a blog post on logs, stream processing,
and distributed systems:
http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
Hi folks,
For this parameters, if consumer.timeout.ms is set to 0, whenever I call
ConsumerIterator's hasNext(),
if there is a message available, a timeout exception will be thrown. Is my
understanding correct? Thanks.
consumer.timeout.ms
-1
Throw a timeout exception to the consumer if no mes
Any update on this guys ?.
-Original Message-
From: Seshadri, Balaji
Sent: Saturday, December 14, 2013 4:22 PM
To: users@kafka.apache.org
Subject: RE: Unable to start consumers in Tomcat
We are doing one scala consumer and one java consumer who listen on same topic
with different group
>> They will occupy 4, 5, 6?
Yes, some of the partitions can occupy 4,5,6.
>> Can I keep the zk cluster running and shut down kafka processes
before I run the partition addition tool?
No. The tool won't run properly if you shut down kafka brokers. I think you
meant shut down kafka consumer proce
For the 1st question, I don't quite get the answer.
Say I have broker 1,2,3,4,5,6. And the topic A has three partitions on broker
1,2,3.
If I add another 3 partitions for A. They will occupy 4, 5, 6?
One more question. Does the partition addition tool work
only with zookeeper? Can I keep the z
I found out the problem. The ConsumerOffsetChecker code has a regex that checks
for ip only and not hostname. Once I replaced the ip check with al .* in the
regex. I was able to run ConsumerOffsetChecker fine.
-Original Message-
From: Xuyen On
Sent: Monday, December 16, 2013 9:42 AM
To
Yes, I'm running 0.7.2. I think Guozhang Wang found the root of the problem:
The tool expected the broker info string to be in the format of host:port, and
only 1 ":" in the string.
Thanks,
Xuyen
-Original Message-
From: Jun Rao [mailto:jun...@gmail.com]
Sent: Friday, December 13, 201
Ok, that helps a lot. Our system is reporting back our broker nodes in an
unexpected format. I'll try to figure out why.
Thanks,
Xuyen
-Original Message-
From: Guozhang Wang [mailto:wangg...@gmail.com]
Sent: Friday, December 13, 2013 4:46 PM
To: users@kafka.apache.org
Subject: Re: Cons
If we have six brokers, and a topic has three partitions on broker 1, 2, 3.
Now if I add another three partitions. Will they be evenly distributed to
1,2,3 or 4,5,6?
Evenly distributed on all six brokers.
If I use reassignment tool in 0.81 with 0.8 broker, will that work and get
around the bugs?
If we have six brokers, and a topic has three partitions on broker 1, 2, 3.
Now if I add another three partitions. Will they be evenly distributed to
1,2,3 or 4,5,6?
One more question.
If I use reassignment tool in 0.81 with 0.8 broker, will that work and get
around the bugs?
Regards,
Libo
-
It's possible to support multi-fetchers per broker in the consumer. Could
you file a jira to track that? However, typically the bottleneck is not in
the fetcher, but in decompression, decoding and application logic during
the consumption.
Thanks,
Jun
On Mon, Dec 16, 2013 at 7:33 AM, Evelina wr
Hi folks!
I'm trying High-Level Consumer and wondering why it uses only 1 fetcher
per broker?
Why can't it be scaled on partition basis?
I even see the parameter '/numFetchers/' in /AbstractFetcherManager/ but
it explicitly set to 1 in /ConsumerFetcherManager/.
--
Very Best Regards,
Evelin
They will be evenly distributed across the nodes in the cluster.
Thanks,
Neha
On Dec 16, 2013 6:42 AM, "Yu, Libo" wrote:
> Assume we have three brokers and a topic already has three partitions.
> Now if I add another three partitions to this topic. Where will the three
> partitions be located af
Assume we have three brokers and a topic already has three partitions.
Now if I add another three partitions to this topic. Where will the three
partitions be located after running the script?
Regards,
Libo
-Original Message-
From: Jun Rao [mailto:jun...@gmail.com]
Sent: Sunday, Decem
Not as of yet. There was some talk of a company trying to implement client
certs and SSL/TLS and contributing it as a patch a few months back but not sure
where that is now though.
Here is the future proposal in regards to security
https://cwiki.apache.org/confluence/display/KAFKA/Security t
Hi,
Is there any support in Kafka for authentication with Zookeeper regarding
the publication of brokers endpoints etc. to ZK?
i.e. If we wanted to secure writes/reads of the /brokers node using e.g.
SASL authentication (which Zookeeper supports out-of-the-box), is there
support in Kafka for this
Here are use cases for contrast, if you haven't seen them.
https://kafka.apache.org/documentation.html#uses
Also, the most significant operational difference versus traditional
message queue brokers is that responsibility for management of where you
are in the queue lies on the client side. This
That is quite a general question, which I think grant me to give a general
answer :)
Kafka is a messaging system designed for high-throughput, so if your use
cases are beyond delivering data it may not be a good fit for you.
Guozhang
On Sun, Dec 15, 2013 at 11:57 PM, Menka wrote:
> Do we know
23 matches
Mail list logo
