Re: Programmatic method of setting consumer groups offsets

2019-01-03 Thread Nimi Wariboko Jr
. From: Eric Azama Sent: Thursday, January 3, 2019 4:31 PM To: users@kafka.apache.org Subject: Re: Programmatic method of setting consumer groups offsets Adding on to Ryanne's point, if subscribe() isn't giving your consumer all of the partitions for a topic, that implies there are st

Re: Programmatic method of setting consumer groups offsets

2019-01-03 Thread Nimi Wariboko Jr
. From: Eric Azama Sent: Thursday, January 3, 2019 4:31 PM To: users@kafka.apache.org Subject: Re: Programmatic method of setting consumer groups offsets Adding on to Ryanne's point, if subscribe() isn't giving your consumer all of the partitions for a topic, that implies there are st

Re: Programmatic method of setting consumer groups offsets

2019-01-03 Thread Eric Azama
Adding on to Ryanne's point, if subscribe() isn't giving your consumer all of the partitions for a topic, that implies there are still active consumers running for that group. The consumer groups CLI command does not allow you to modify offsets for consumer groups that have active consumers. I beli

Re: Programmatic method of setting consumer groups offsets

2019-01-03 Thread Ryanne Dolan
Nimi, try using the low-level consumer API via assign() instead of subscribe(), and then commit() each partition. I suspect you are trying to commit partitions that the consumer is not subscribed to. Ryanne On Thu, Jan 3, 2019, 6:13 PM Nimi Wariboko Hi, > > Is there a guide or API that is implem

Programmatic method of setting consumer groups offsets

2019-01-03 Thread Nimi Wariboko
Hi, Is there a guide or API that is implemented that can modify stored offsets in Kafka? For example, I'm looking to set a consumer groups stored offsets in Kafka to a specific value. I know there is the `kafka-consumer-groups` CLI command, but I'm looking for a way to do so from an application