[ https://issues.apache.org/jira/browse/KAFKA-6547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358415#comment-16358415 ]
Dan commented on KAFKA-6547: ---------------------------- I think I've eliminated my client from the problem definition. I start the client first to get the group started, then I stop it. Describe shows: {code:java} kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M -Djava.security.auth.login.config=./config/kafka_client_jaas.conf" kafka-consumer-groups.sh --command-config ./config/consumer.properties --bootstrap-server kafka-server-1:9092 --group meta-printer --describe Note: This will not show information about old Zookeeper-based consumers. Consumer group 'meta-printer' has no active members. TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID payload-meta 0 836464 1060485 224021 - - - {code} Then I run my client for about 20s, with an offset commit frequency of 10s. Here is a describe after that: {code:java} kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M -Djava.security.auth.login.config=./config/kafka_client_jaas.conf" kafka-consumer-groups.sh --command-config ./config/consumer.properties --bootstrap-server kafka-server-1:9092 --group meta-printer --describe Note: This will not show information about old Zookeeper-based consumers. Consumer group 'meta-printer' has no active members. TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID payload-meta 0 974424 1060485 86061 - - - {code} Now I'm going to try to reset the group to the earliest available: {code:java} kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M -Djava.security.auth.login.config=./config/kafka_client_jaas.conf" kafka-consumer-groups.sh --command-config ./config/consumer.properties --bootstrap-server kafka-server-1:9092 --group meta-printer --reset-offsets --execute --to-earliest --topic payload-meta Note: This will not show information about old Zookeeper-based consumers. TOPIC PARTITION NEW-OFFSET payload-meta 0 0 {code} And then a subsequent describe: {code:java} kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M -Djava.security.auth.login.config=./config/kafka_client_jaas.conf" kafka-consumer-groups.sh --command-config ./config/consumer.properties --bootstrap-server kafka-server-1:9092 --group meta-printer --describe Note: This will not show information about old Zookeeper-based consumers. Consumer group 'meta-printer' has no active members. TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID payload-meta 0 974424 1060485 86061 - - - {code} Unchanged. The default server retention is forever, so no log should be pruned. But even if there was pruning, a reset to earliest should be able to at least get the offset back to the first offset shown above. The problem is not present in 0.11. I you want output of that, I'll have to set it up. Am I misunderstanding what should be happening here? Many thanks. > group offset reset and begin_offset ignored/no effect > ----------------------------------------------------- > > Key: KAFKA-6547 > URL: https://issues.apache.org/jira/browse/KAFKA-6547 > Project: Kafka > Issue Type: Bug > Components: offset manager > Affects Versions: 1.0.0 > Environment: ubuntu 16, java 1.8 > Reporter: Dan > Priority: Major > Fix For: 0.11.0.2 > > > Use of kafka-consumer-group.sh with --reset-offsets --execute <--to-earliest > or anything> has no effect in 1.0. When my group client connects and requests > a specific offset or an earliest there's no effect and the consumer is unable > to poll, so no messages, even new ones are ignored. > I installed 0.11 and these problems are not manifest. > I'm unfamiliar with the internals and put the offset manager as the possible > component, but that's a guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005)