Hi,
We are using kafka 0.7.
2 brokers, each broker has 10 partitions for one topic
3 consumers in one consumer group, each consumer create 10 streams.
Today, when we want to rollout new service.
After we restart one consumer we find exceptions and warning.
Hello Yonghui,
In 0.7 the consumer rebalance logic is distributed and in some corner cases
such as soft-failure-caused-consecutive rebalances some consumer may
consider the rebalance as complete while others are still trying the
rebalance process. You can check the GC logs on your consumer to
Hi all,
I'm using Kafka 0.8 and I've ran into an issue with ConsumerConnector.
Steps to reproduce:
1. Start single-broker Kafka cluster with auto.create.topic.enable set to
true
2. Start ConsumerConnector on topic (which does not yet exist) with
auto.offset.reset set to smallest.
3. Produce some
Hi,
I am hitting a strange exception while creating a topic in Kafka -
Steps to generate this-
1. Created a topic multipartition_test with 2 partitions and 2 replicas
2. Added some data to this topics and verified data is coming up for both
partitions
3. Deleted the topic. Checked only the
if placing mirror maker in the same datacenter as target cluster,
it/consumer will talks to zookeeper in remote/source datacenter. would it
more susceptible to network problems?
As for the problem commit offset without actually producing/writing msgs to
target cluster, it can be solved by
Hi all,
I am seeing some corrupt data on my 0.7.2 Kafka cluster. Every once in awhile
I'll get the following message:
Exception in thread kafka-consumer java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Unknown Source)
at java.nio.HeapByteBuffer.get(Unknown
You're probably right that it has to be GC given the CMS activity I saw in the
log but I didn't see a single concurrent mode failure, which bothers me that we
had this happen anyway... also the ZK timeout is set to a very large number...
I dunno, seems weird.
I will see what I can do about
Could you try 0.8.1.1, which fixed some bugs related to controller hanging
during shutdown.
For ZK session expiration, the #1 cause is GC. We have also seen that
transient issues with the underlying storage (e.g. raid controller reset)
can also cause ZK session expiration.
As for the data loss
Another question - If I start broker 1 back up now, with his replicas now out
of sync, will he automatically catch up with the leaders or do I have to do
something to catch him up?
--Ian
On May 12, 2014, at 10:49 AM, Jun Rao jun...@gmail.com wrote:
Could you try 0.8.1.1, which fixed some
0.8.1.1 fixed an issue when the controller could hang during soft failure.
Out-of sync replicas will catch up from the leader automatically, once
restarted.
Thanks,
Jun
On Mon, May 12, 2014 at 12:37 PM, Ian Friedman i...@flurry.com wrote:
You're probably right that it has to be GC given the
Hi
I have kafka broker running (kafka_2.9.1-0.8.1.1)
All is working.
One project requires producer is written in CSharp
I am not dot net programmer but I managed to write simple producer code
using
https://github.com/kafka-dev/kafka/blob/master/clients/csharp/README.md
the code
...
using
Could you run the DumpLogSegments tool to see if there are indeed corrupted
messages in the broker log?
Thanks,
Jun
On Mon, May 12, 2014 at 4:08 PM, Xuyen On x...@ancestry.com wrote:
Hi all,
I am seeing some corrupt data on my 0.7.2 Kafka cluster. Every once in
awhile I'll get the
The controller log in broker 1 is too late. Could you send its log
around 2014-05-12 21:24:37?
Thanks,
Jun
On Mon, May 12, 2014 at 5:02 PM, Steven Wu steve...@netflix.com wrote:
This is a three-node cluster. broker 0 lost connection to ZK. broker 1
does seem to take the controller role. but
Delete topic doesn't quite work in 0.8.1. We recently fixed it in trunk.
Could you give it a try and see if you see the same issue?
Thanks,
Jun
On Mon, May 12, 2014 at 9:39 AM, Kashyap Mhaisekar kashya...@gmail.comwrote:
Hi,
I am hitting a strange exception while creating a topic in Kafka -
14 matches
Mail list logo
