Hi Guozhang,
The consumer, broker, and zookeeper are all on the same machine - just
testing out Kafka at the moment :)
Here are the configuration values I set:
properties.setProperty("bootstrap.servers", "localhost:9092");
properties.setProperty("group.id", "xxx-yyy-reader");
properties.setProperty("session.timeout.ms", "30000");
properties.setProperty("enable.auto.commit", "true");
properties.setProperty("auto.commit.interval.ms", "10000");
properties.setProperty("key.deserializer",
"org.apache.kafka.common.serialization.StringDeserializer");
properties.setProperty("value.deserializer",
"xxxyyy.kafka.JsonDeserializer");
properties.setProperty("client.id", "xxx-yyy-consumer");
I guess they are fairly standard.
Martin
On 18 November 2015 at 19:16, Guozhang Wang <wangg...@gmail.com> wrote:
> Hello Martin,
>
> Could you paste the consumer config values in this thread as well? And is
> the consumer co-located with the broker?
>
> Guozhang
>
> On Wed, Nov 18, 2015 at 7:40 AM, Martin Skøtt <
> martin.sko...@falconsocial.com> wrote:
>
> > Hi,
> >
> > I'm tracking the 0.9.0.0 Git tag and have a Java consumer using the new
> > API, but I'm seeing some strange issues. I run ZooKeeper and Kafka on my
> > own machine using the settings files in config/ and no authentication.
> > Build is done using Oracle JDK 8. I have 13 topics, each created with a
> > replication factor of 1.
> >
> > The first time my consumer connects to the newly created topics (before
> > anything has been written to the topic) it emits the following errors:
> >
> > [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version :
> > 0.9.0.0
> > [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId
> :
> > fe855f98263cb7b8
> > [main] ERROR
> > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
> > offset commit failed.
> > org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException: The
> > group coordinator is not available.
> > [main] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> > - Marking the coordinator 2147483647 dead.
> > [main] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> > - Attempt to join group xxx-yyy-reader failed due to obsolete coordinator
> > information, retrying.
> > [main] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> > - Marking the coordinator 2147483647 dead.
> > [main] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> > - Attempt to join group xxx-yyy-reader failed due to obsolete coordinator
> > information, retrying.
> >
> > If I restart my consumer then it starts without issue.
> >
> > When I start writing to the topics then my consumer reads data, but will
> > often log messages like these:
> >
> > [main] ERROR
> > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
> > offset commit failed.
> > org.apache.kafka.clients.consumer.internals.SendFailedException
> >
> > And occasionally these:
> > [main] INFO
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> > - Attempt to heart beat failed since member id is not valid, reset it and
> > try to re-join group.
> > [main] ERROR
> > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error
> > ILLEGAL_GENERATION occurred while committing offsets for group
> > audience-indexer-reader
> > [main] ERROR
> > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto
> > offset commit failed.
> > org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be
> > completed due to group rebalance
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:497)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:438)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:673)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:646)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:167)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(ConsumerNetworkClient.java:350)
> > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:288)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:303)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:197)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:187)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:157)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:352)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:384)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare(ConsumerCoordinator.java:236)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:209)
> > at
> >
> >
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:304)
> > at
> >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:861)
> > at
> >
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:829)
> >
> > Does anyone have any input on what might be happening?
> >
> > --
> > Martin Skøtt
> >
>
>
>
> --
> -- Guozhang
>
--
Martin Skøtt
Senior backend developer
+45 2683 7780 <+4526837780>
Copenhagen
Falcon Social
H.C Andersens Blvd. 27
1553 Copenhagen
New York City
Falcon Social
77 North 4th Street, 2nd Floor
Brooklyn, NY 11249
[image: Falcon Social]
<https://www.falconsocial.com/?utm_source=Employee%20emails&utm_medium=email&utm_content=Martin%20Sk%C3%B8tt&utm_campaign=Mail%20signature>
Social Media Management for Enterprise
