[jira] [Commented] (KAFKA-6434) Kafka-consumer-groups.sh reset-offsets does not work properly for not existing group
[ https://issues.apache.org/jira/browse/KAFKA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328303#comment-16328303 ] huxihx commented on KAFKA-6434: --- [~vahid] I asked the same question before, and here was what Jason replied: "Since the protocol type is undefined for a group using manual assignment, it seems we would filter it. Maybe the coordinator ought to assume it is a consumer group if there are committed offsets. Either that or the --list output ought to include empty groups even if the protocol type is undefined." > Kafka-consumer-groups.sh reset-offsets does not work properly for not > existing group > > > Key: KAFKA-6434 > URL: https://issues.apache.org/jira/browse/KAFKA-6434 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.11.0.2 >Reporter: George Smith >Assignee: Vahid Hashemian >Priority: Major > > Our usecase: We are migrating Spark streaming app into Kafka streaming. We > want to continue processing from the last processed offsets of the Spark > streaming app. Therefore we want to define new consumer group (application > id) with given offsets. The new app was not launched yet (we don't want to > make side effects of processing into db) -> new consumer group does not exist. > I was happy to see the updated Kafka-consumer-groups.sh supports > reset-offsets method. Unfortunately it seems it's not working as expected. > {code} > kafka-consumer-groups.sh --reset-offsets --bootstrap-server localhost:9092 > --topic testTopic:0 --group testGROUP --to-offset 10 > Note: This will only show information about consumers that use the Java > consumer API (non-ZooKeeper-based consumers). > TOPIC PARTITION NEW-OFFSET > testTopic 0 10 > {code} > Now I want to check offsets for the group: > {code} > kafka-consumer-groups.sh --describe --bootstrap-server localhost:9092 --group > testGROUP > Error: Consumer group 'testGROUP' does not exist. > {code} > That's strange, isn't it? > On the other side when I use kafka-streams-application-reset.sh - the group > is obviously created - unfortunately this tool does not support given offsets > for partitions (only the beginning is supported) + it does not support > secured Kafka connection... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-6434) Kafka-consumer-groups.sh reset-offsets does not work properly for not existing group
[ https://issues.apache.org/jira/browse/KAFKA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324830#comment-16324830 ] Vahid Hashemian commented on KAFKA-6434: [~cabot] This is fixed in the current trunk (by KIP-171) and will land on 1.1.0. It appears though that {{kafka-consumer-groups.sh --list --bootstrap-server localhost:9092}} still does not output such groups because the protocol type for them is not set (the required protocol type by this command is {{consumer}}). [~hachikuji], do you think this is the correct behavior? > Kafka-consumer-groups.sh reset-offsets does not work properly for not > existing group > > > Key: KAFKA-6434 > URL: https://issues.apache.org/jira/browse/KAFKA-6434 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.11.0.2 >Reporter: George Smith >Assignee: Vahid Hashemian > > Our usecase: We are migrating Spark streaming app into Kafka streaming. We > want to continue processing from the last processed offsets of the Spark > streaming app. Therefore we want to define new consumer group (application > id) with given offsets. The new app was not launched yet (we don't want to > make side effects of processing into db) -> new consumer group does not exist. > I was happy to see the updated Kafka-consumer-groups.sh supports > reset-offsets method. Unfortunately it seems it's not working as expected. > {code} > kafka-consumer-groups.sh --reset-offsets --bootstrap-server localhost:9092 > --topic testTopic:0 --group testGROUP --to-offset 10 > Note: This will only show information about consumers that use the Java > consumer API (non-ZooKeeper-based consumers). > TOPIC PARTITION NEW-OFFSET > testTopic 0 10 > {code} > Now I want to check offsets for the group: > {code} > kafka-consumer-groups.sh --describe --bootstrap-server localhost:9092 --group > testGROUP > Error: Consumer group 'testGROUP' does not exist. > {code} > That's strange, isn't it? > On the other side when I use kafka-streams-application-reset.sh - the group > is obviously created - unfortunately this tool does not support given offsets > for partitions (only the beginning is supported) + it does not support > secured Kafka connection... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-6434) Kafka-consumer-groups.sh reset-offsets does not work properly for not existing group
[ https://issues.apache.org/jira/browse/KAFKA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324760#comment-16324760 ] Satyajit varma commented on KAFKA-6434: --- [~cabot] it is because your Consumer Group "testGROUP" is not currently Stable(Running), but it does respond and show you the new offsets for the reset-offset request made. Reset-offset would only work when the consumer group is INACTIVE(Not Running), you should be able to see "kafka-consumer-groups.sh --describe --bootstrap-server localhost:9092 --group testGROUP" result only when you have the consumer group running. > Kafka-consumer-groups.sh reset-offsets does not work properly for not > existing group > > > Key: KAFKA-6434 > URL: https://issues.apache.org/jira/browse/KAFKA-6434 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.11.0.2 >Reporter: George Smith >Assignee: Vahid Hashemian > > Our usecase: We are migrating Spark streaming app into Kafka streaming. We > want to continue processing from the last processed offsets of the Spark > streaming app. Therefore we want to define new consumer group (application > id) with given offsets. The new app was not launched yet (we don't want to > make side effects of processing into db) -> new consumer group does not exist. > I was happy to see the updated Kafka-consumer-groups.sh supports > reset-offsets method. Unfortunately it seems it's not working as expected. > {code} > kafka-consumer-groups.sh --reset-offsets --bootstrap-server localhost:9092 > --topic testTopic:0 --group testGROUP --to-offset 10 > Note: This will only show information about consumers that use the Java > consumer API (non-ZooKeeper-based consumers). > TOPIC PARTITION NEW-OFFSET > testTopic 0 10 > {code} > Now I want to check offsets for the group: > {code} > kafka-consumer-groups.sh --describe --bootstrap-server localhost:9092 --group > testGROUP > Error: Consumer group 'testGROUP' does not exist. > {code} > That's strange, isn't it? > On the other side when I use kafka-streams-application-reset.sh - the group > is obviously created - unfortunately this tool does not support given offsets > for partitions (only the beginning is supported) + it does not support > secured Kafka connection... -- This message was sent by Atlassian JIRA (v6.4.14#64029)