Hi Apoorv, Thanks for your response. AM1: With --by-duration, the tool obtains the current time, subtracts the duration (such as 3 hours in your example), and then calculates the offset on the topic-partition for the resulting time. It then resets the offset accordingly. It does not depend upon the share.auto.offset.reset config.
Thanks, Andrew On 2026/08/13 23:33:40 Apoorv Mittal wrote: > Hi Andrew, > Thanks for the KIP. I have one question. > > AM1: Since this KIP adds --by-duration to kafka-share-groups.sh while share > groups also accept share.auto.offset.reset=by_duration:… as a group config. I > can find share.auto.offset.reset=by_duration:PT3H (config, > kafka-configs.sh): an init-time policy the broker applies only when no SPSO > exists. I can set the config while there are active members but the config > update is not honoured. It's only honoured when a new group is specified. > Can you please clarify the expected behaviour when using > `kafka-share-groups.sh --by-duration` reset option? > > Regards, > Apoorv Mittal > > > On Thu, Aug 13, 2026 at 1:21 PM Andrew Schofield <[email protected]> > wrote: > > > Hi Aditya, > > Thanks for your reply. I'm not dismissing your concern, but I'm not > > entirely happy with a way forward yet. > > > > AK1: I wonder whether we could use a pipe here to avoid the intermediate > > file. After all, export uses stdout as the destination, so if we could use > > stdin as the source. For example, I've used '-' with --from-file below to > > indicate stdin. It could be --from-stdin or similar too. > > > > $ bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 > > --export-offsets --group CG1 --topic T1 --to-current | > > bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 > > --reset-offsets --group CG2 --from-file - --topic T1 --execute > > > > This would still allow resetting between different group types, which is > > actually the use case that got me into this, taking offsets from a consumer > > group to initialise a share group. > > > > Thanks, > > Andrew > > > > On 2026/08/12 18:12:08 Aditya Kousik wrote: > > > Hi Andrew. > > > > > > Thanks for addressing AK2-4: lgtm > > > > > > On AK1: I’m thinking from a user experience standpoint. The teams > > managing the kafka clusters and the ones using the clients diverge - but > > the Admin role may be exposed via self-service typically via API requests > > and a K8S CRD. I’m thinking of the latter case where Kafka SRE allow a thin > > wrapper for client teams to manage their own groups and topics and also > > isolating the environment where the scripts can be run. In such a case, it > > becomes cumbersome to fetch/write the csv file from a Pod that runs the > > migration ; it may not have any volume to persist it or running kubectl to > > scp the file is usually denied. So in such a case, the migrate offsets > > script could just be a single command that fetches offsets from one group > > and applies it to another (or several) without the middleman CSV. > > > > > > Thanks, > > > Aditya > > > > > > > On Aug 12, 2026, at 06:39, Andrew Schofield <[email protected]> > > wrote: > > > > > > > > Hi Aditya, > > > > Thanks for your reply. Sorry for the delay. > > > > > > > > AK1: One of the advantages of the file is that it's possible to export > > from one group type and import into another. You can also export offsets > > for multiple groups, and this changes the file format (4 columns instead of > > 3). Sadly, this is all existing behaviour that I only learnt by trial and > > error. Personally, I think that export-import is a relatively infrequent > > operation, and the KIP makes enough of a step forward. I wonder if you > > could elaborate a bit more on your idea. Feel free to push back if you > > disagree. > > > > > > > > AK2: The KIP does not mention this and I wasn't quite sure where to > > add it. You are correct. Should we improve the javadoc for the admin client? > > > > > > > > AK3: Yes, Admin is better. KIP updated. > > > > > > > > AK4: The DescribeGroupsGeneric RPC is intended as a way of getting a > > minimal set of group information for groups of any group types. It is used > > in Admin.describeGroups and this would be the initial Admin call from > > bin/kafka-XXXX-groups.sh when exporting or resetting offsets. It gives a > > way to check the group type and existence unambiguously. By contrast, if > > you use Admin.describeXXXGroups() for an existing group ID of the wrong > > type, the error code is GROUP_ID_NOT_FOUND. > > > > > > > > AK4.1: I didn't include the members intentionally. This RPC is > > agnostic to group type. I think the common subset of group attributes is > > group ID, group type, group state and coordinator. If you dig deeper, you > > find that the group types differ. For example, assignments in consumer > > groups and share groups are partitions, while in streams groups they are > > tasks. I expect we'll add connect groups one day, and until we have the > > KIP, I wouldn't want to assume the details of membership. > > > > > > > > AK4.2: This is following existing convention I'm afraid and I want to > > be consistent. If the broker does not send back a list of ACL operations, > > then this will be null. This is different than an empty set which means > > "you are not authorized", as opposed to "you don't know anything about > > authorized operations". > > > > > > > > Thanks, > > > > Andrew > > > > > > > >> On 2026/07/27 05:45:27 Aditya Kousik wrote: > > > >> A small correction. > > > >> > > > >> I meant to add a statement before the numbered comments. > > > >> > > > >> “Since the KIP motivation revolves around usability of the script, > > would you be willing to consider the following:” > > > >> > > > >> Also, typo on AK2. “…As long as there aren’t any active members in > > the group subscription”. > > > >> > > > >> -Aditya > > > >> > > > >>>> On Jul 25, 2026, at 23:53, Aditya Kousik <[email protected]> > > wrote: > > > >>> > > > >>> Hi Andrew, > > > >>> > > > >>> Thanks for this KIP, I’ve personally faced more than one scenario > > where I’ve had to use the admin client to update the offsets. > > > >>> > > > >>> The main uses cases teams face ( and TIL about the various options > > the script offers): > > > >>> Teams migrating to a new consumer group from an existing one but > > don’t want to commit to either (or duration for newer clients) auto offset > > reset policy on the new group. > > > >>> Teams want to seek past certain offsets in partitions because they > > have poison pill messages/corrupted deserialization story and they have no > > control over the data. In this case, they actually want latest offset > > strategy but most don’t even realise that it applies to streams out of > > range and are surprised/frustrated that the property doesn’t move their > > position in the queues. (I see we can do this already in the script today) > > > >>> When I migrated our clients from 2.x directly to 4.x, out of > > abundance of caution, I wrote an hourly cron job that dumps the snapshot of > > group_id,topic,partition,offset into csv files so that if I had to revert a > > few affected partitions I can trim the csv down to those partitions and > > alter them with the admin client. > > > >>> > > > >>> AK1. A single command line with args —source-group-id > > —target-group-id —export so that the intermediate CSV file can be skipped. > > > >>> AK2. Minor: The KIP does not mention the alter/list consumer group > > offsets Admin call. alter only works as long as there are active members in > > the group subscription. I was bitten by this in the past; the documentation > > was sparse around that expectation but it does make sense. > > > >>> AK3. KIP references AdminClient but I believe this is deprecated in > > favour of just Admin which is the interface that has the methods? > > > >>> AK4. How do we leverage the describe groups RPC when using the > > script/admin client? I’m trying to imagine flows other than the export > > group offsets. > > > >>> AK4.1 Can we list active members/offset state in a given group in > > addition to the GroupState? This has helped identify zombie processes to > > hunt down. The broker’s view would be useful here. (related to AK2) > > > >>> AK4.2 Could we return empty set of acl operations instead of null? > > > >>> > > > >>> Best, > > > >>> Aditya > > > >>> > > > >>> > > > >>>> On 2026/06/17 06:34:11 Andrew Schofield wrote: > > > >>>> Hi, > > > >>>> I’d like to start discussion on a small KIP to improve the > > usability of the command-line tools for resetting group offsets. When > > implementing KIP-1323 and reviewing the PRs, it became clear that the > > usability of this area of Kafka isn’t great. This KIP improves the > > usability of these tools. > > > >>>> > > > >>>> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1359%3A+Improve+usability+of+resetting+group+offsets > > > >>>> > > > >>>> Thanks, > > > >>>> Andrew > > > >>>> > > > >> > > > > > >
