Re: first message is lost

[Yonghui Zhao]

Got it， thanks Guozhang.

On Thu Nov 13 2014 at 12:04:17 AM Guozhang Wang  wrote:

> Yonghui,
>
> If consumer is not set with --from-beginning, then this scenario is
> expected: KAFKA-1006 
>
> We are still figuring what is the best way to resolve this issue.
>
> Guozhang
>
> On Wed, Nov 12, 2014 at 12:35 AM, Yonghui Zhao 
> wrote:
>
> > Hi,
> >
> > For a non-existent topic,  the consumer and producer are set up.
> > Then if the producer sends the first message, producer gets this
> exception:
> >
> > [2014-11-12 16:24:28,041] WARN Error while fetching metadata
> > [{TopicMetadata for topic test5 ->
> > No partition metadata for topic test5 due to
> > kafka.common.LeaderNotAvailableException}] for topic [test5]: class
> > kafka.common.LeaderNotAvailableException
> >  (kafka.producer.BrokerPartitionInfo)
> > [2014-11-12 16:24:28,061] WARN Error while fetching metadata
> > [{TopicMetadata for topic test5 ->
> > No partition metadata for topic test5 due to
> > kafka.common.LeaderNotAvailableException}] for topic [test5]: class
> > kafka.common.LeaderNotAvailableException
> >  (kafka.producer.BrokerPartitionInfo)
> > [2014-11-12 16:24:28,062] ERROR Failed to collate messages by topic,
> > partition due to: Failed to fetch topic metadata for topic: test5
> > (kafka.producer.async.DefaultEventHandler)
> >
> >
> > And the consumer won't get this message.  But the second message and
> later
> > is ok.
> >
> > We don't change server.properties so auto.create.topics.enable should be
> > true.
> >
> > But if we set up consumer with --from-begin flag,  such like
> > bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test5
> > * --from-begin*
> > the exception is still there but consumer will get the first message.
> >
> > My question is how to avoid the loss.
> >
> > Thanks!
> >
>
>
>
> --
> -- Guozhang
>

Re: first message is lost

[Guozhang Wang]

Yonghui,

If consumer is not set with --from-beginning, then this scenario is
expected: KAFKA-1006 

We are still figuring what is the best way to resolve this issue.

Guozhang

On Wed, Nov 12, 2014 at 12:35 AM, Yonghui Zhao 
wrote:

> Hi,
>
> For a non-existent topic,  the consumer and producer are set up.
> Then if the producer sends the first message, producer gets this exception:
>
> [2014-11-12 16:24:28,041] WARN Error while fetching metadata
> [{TopicMetadata for topic test5 ->
> No partition metadata for topic test5 due to
> kafka.common.LeaderNotAvailableException}] for topic [test5]: class
> kafka.common.LeaderNotAvailableException
>  (kafka.producer.BrokerPartitionInfo)
> [2014-11-12 16:24:28,061] WARN Error while fetching metadata
> [{TopicMetadata for topic test5 ->
> No partition metadata for topic test5 due to
> kafka.common.LeaderNotAvailableException}] for topic [test5]: class
> kafka.common.LeaderNotAvailableException
>  (kafka.producer.BrokerPartitionInfo)
> [2014-11-12 16:24:28,062] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: test5
> (kafka.producer.async.DefaultEventHandler)
>
>
> And the consumer won't get this message.  But the second message and later
> is ok.
>
> We don't change server.properties so auto.create.topics.enable should be
> true.
>
> But if we set up consumer with --from-begin flag,  such like
> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test5
> * --from-begin*
> the exception is still there but consumer will get the first message.
>
> My question is how to avoid the loss.
>
> Thanks!
>



-- 
-- Guozhang