Re: Topic stuck with leader: none

If you shutdown all brokers, then bring them up one at a time does it change? Might have to do it several times. On Aug 1, 2013 8:28 PM, "Vinicius Carvalho" wrote: > Hi there! So, I setup a demo and forgot to create the topic. But since my > producer is set to create topics automatic, it created

Re: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

I'm seeing 0.8-beta1 get into this state at times, though I don't have a way to repro it. Restarting all brokers and zk repeatedly has fixed it. Again, unknown why. On Aug 1, 2013 9:06 PM, "Jun Rao" wrote: > Is this issue reproducible? > > Thanks, > > Jun > > > On Thu, Aug 1, 2013 at 8:24 AM, Nan

Re: Topic stuck with leader: none

Any error/exception in the state-change log? Thanks, Jun On Thu, Aug 1, 2013 at 8:27 PM, Vinicius Carvalho < viniciusccarva...@gmail.com> wrote: > Hi there! So, I setup a demo and forgot to create the topic. But since my > producer is set to create topics automatic, it created one. Problem is

Re: java.net.SocketException: Too many open files

If you do netstat, what hosts are those connections for and what state are those connections in? Thanks, Jun On Thu, Aug 1, 2013 at 9:04 AM, Nandigam, Sujitha wrote: > Hi, > > In producer I was continuously getting this exception > java.net.SocketException: Too many open files > even though I

Re: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

Is this issue reproducible? Thanks, Jun On Thu, Aug 1, 2013 at 8:24 AM, Nandigam, Sujitha wrote: > Sorry..I saw that error after killing brokers only...but before that no > error > > -Original Message- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Thursday, August 01, 2013 11:10 AM

Topic stuck with leader: none

Hi there! So, I setup a demo and forgot to create the topic. But since my producer is set to create topics automatic, it created one. Problem is that after when sending messages I got a LeaderNotAvailableException. I ran topic-list shell, and noticed that my topic had leader:none. So I removed and

Re: Moving raw logs from kafka into hadoop using camus

Hi Vadim, Sorry for the slow response. If your topics share commonality, you should be able to implement one decoder to handle all of them. On the other hand if your kafka data is different depending on the topic, you might need separate decoders for each topic. I don't recall if we added the a

java.net.SocketException: Too many open files

Hi, In producer I was continuously getting this exception java.net.SocketException: Too many open files even though I added the below line to /etc/security/limits.conf kafka-0.8.0-beta1-src-nofile983040 ERROR Producer connection to localhost:9093 unsuccessful (kafka.producer.Syn

RE: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

Sorry..I saw that error after killing brokers only...but before that no error -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Thursday, August 01, 2013 11:10 AM To: users@kafka.apache.org Subject: Re: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerP

Re: Consumer pauses when running many threads

Have you looked at https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3F? Thanks, Jun On Thu, Aug 1, 2013 at 7:30 AM, Sybrandy, Casey < casey.sybra...@six3systems.com> wrote: > Hello, > > We're seeing an issue running 0.7.0 where one or more of our con

Re: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

It seems that you have only one replica on broker 2 for that partition and that broker is down. That's why that partition is not available. Thanks, Jun On Thu, Aug 1, 2013 at 6:33 AM, Nandigam, Sujitha wrote: > Yes, below is the exception I see in state-change log.. > > ERROR Controller 1 epoc

Re: reprocessing messages in kafka

Kafka allows a consumer to rewind the consumption since messages are kept in the broker by a retention policy (defaults to 7 days). I am not exactly sure how Storm works. My guess is that it only checkpoints the consumer offset after all messages before that offset have been processed successfully.

Re: Kafka log4j appender infinite logging cycle question

Yes, there is this self loop, but it's not always a 1-to-1 mapping. Since you use async producers, each N messages corresponds to a Kafka produce request. So, if you encounter an error while processing the request, you add one more message (for every N messages). What you have to avoid is per messa

Consumer pauses when running many threads

Hello, We're seeing an issue running 0.7.0 where one or more of our consumers are pausing after about an hour when we have a lot of threads configured. Our setup is as follows: * 4 brokers configured for 32 threads and 32 partitions on each broker. * 2 consumers each processi

RE: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

Yes, below is the exception I see in state-change log.. ERROR Controller 1 epoch 1 initiated state change for partition [VeriSign.all.dns.hydra,0] from OfflinePartition to OnlinePartition failed (state.change.logger) kafka.common.NoReplicaOnlineException: No replica for partition [VeriSign.all.

Re: reprocessing messages in kafka

I try to resolve such behavior: suppose storm consumes messages from kafka. In case part of it's consumers crashed for any reasons and as a result didn't succeed to process the consumed messages. But if it is impossible after recover to reprocess these messages the system will not be robust and

Kafka log4j appender infinite logging cycle question

Hi, I have the following scenario: - using the Kafka log4j appender (in fact its a variant on the OOTB appender) - configuring the appender for the root log4j category - appender is using the async producer I am wondering about what happens when e.g. the producer cannot connect to any broker