[
https://issues.apache.org/jira/browse/KAFKA-6547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358415#comment-16358415
]
Dan commented on KAFKA-6547:
I think I've eliminated my client from the problem definition. I start the
client first to get the group started, then I stop it. Describe shows:
{code:java}
kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M
-Djava.security.auth.login.config=./config/kafka_client_jaas.conf"
kafka-consumer-groups.sh --command-config ./config/consumer.properties
--bootstrap-server kafka-server-1:9092 --group meta-printer --describe
Note: This will not show information about old Zookeeper-based consumers.
Consumer group 'meta-printer' has no active members.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
CONSUMER-ID HOST
CLIENT-ID
payload-meta 0 836464 1060485
224021 - -
-
{code}
Then I run my client for about 20s, with an offset commit frequency of 10s.
Here is a describe after that:
{code:java}
kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M
-Djava.security.auth.login.config=./config/kafka_client_jaas.conf"
kafka-consumer-groups.sh --command-config ./config/consumer.properties
--bootstrap-server kafka-server-1:9092 --group meta-printer --describe
Note: This will not show information about old Zookeeper-based consumers.
Consumer group 'meta-printer' has no active members.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
CONSUMER-ID HOST
CLIENT-ID
payload-meta 0 974424 1060485 86061
- -
-
{code}
Now I'm going to try to reset the group to the earliest available:
{code:java}
kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M
-Djava.security.auth.login.config=./config/kafka_client_jaas.conf"
kafka-consumer-groups.sh --command-config ./config/consumer.properties
--bootstrap-server kafka-server-1:9092 --group meta-printer --reset-offsets
--execute --to-earliest --topic payload-meta
Note: This will not show information about old Zookeeper-based consumers.
TOPIC PARTITION NEW-OFFSET
payload-meta 0 0
{code}
And then a subsequent describe:
{code:java}
kafka@kafka-server-1:~$ KAFKA_OPTS="-Xmx128M
-Djava.security.auth.login.config=./config/kafka_client_jaas.conf"
kafka-consumer-groups.sh --command-config ./config/consumer.properties
--bootstrap-server kafka-server-1:9092 --group meta-printer --describe
Note: This will not show information about old Zookeeper-based consumers.
Consumer group 'meta-printer' has no active members.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
CONSUMER-ID HOST
CLIENT-ID
payload-meta 0 974424 1060485 86061
- -
-
{code}
Unchanged. The default server retention is forever, so no log should be
pruned. But even if there was pruning, a reset to earliest should be able to at
least get the offset back to the first offset shown above. The problem is not
present in 0.11. I you want output of that, I'll have to set it up.
Am I misunderstanding what should be happening here?
Many thanks.
> group offset reset and begin_offset ignored/no effect
> -
>
> Key: KAFKA-6547
> URL: https://issues.apache.org/jira/browse/KAFKA-6547
> Project: Kafka
> Issue Type: Bug
> Components: offset manager
>Affects Versions: 1.0.0
> Environment: ubuntu 16, java 1.8
>Reporter: Dan
>Priority: Major
> Fix For: 0.11.0.2
>
>
> Use of kafka-consumer-group.sh with --reset-offsets --execute <--to-earliest
> or anything> has no effect in 1.0. When my group client connects and requests
> a specific offset or an earliest there's no effect and the consumer is unable
> to poll, so no messages, even new ones are ignored.
> I installed 0.11 and these problems are not manifest.
> I'm unfamiliar with the internals and put the offset manager as the possible
> component, but that's a guess.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
