>> \cc from dev
>>
>>
>> Forwarded Message ----
>> Subject: Re: KIP-122: Add a tool to Reset Consumer Group Offsets
>> Date: Thu, 23 Feb 2017 10:13:39 -0800
>> From: Matthias J. Sax
>> Organization: Confluent Inc
>> To: d...@kafka.apache.org
+1 for such a tool. It would be of great help in a lot of use cases.
On Thu, Feb 23, 2017 at 11:44 PM, Matthias J. Sax
wrote:
> \cc from dev
>
>
> Forwarded Message
> Subject: Re: KIP-122: Add a tool to Reset Consumer Group Offsets
> Date: Thu, 23 Feb 2
\cc from dev
Forwarded Message
Subject: Re: KIP-122: Add a tool to Reset Consumer Group Offsets
Date: Thu, 23 Feb 2017 10:13:39 -0800
From: Matthias J. Sax
Organization: Confluent Inc
To: d...@kafka.apache.org
So you suggest to merge "scope options" --topics, --
@Matthias about the point 9:
What about keeping only the --topic option, and support this format:
`--topic t1:0,1,2 --topic t2 --topic t3:2`
In this case topics t1, t2, and t3 will be selected: topic t1 with
partitions 0,1 and 2; topic t2 with all its partitions; and topic t3, with
only partitio
Thanks for the feedback Matthias.
* 1. You're right. I'll reorder the scenarios.
* 2. Agree. I'll update the KIP.
* 3. I like it, updating to `reset-offsets`
* 4. Agree, removing the `reset-` part
* 5. Yes, 1.e option without --execute or --export will print out current
offset, and the new off
Hi,
thanks for updating the KIP. Couple of follow up comments:
* Nit: Why is "Reset to Earliest" and "Reset to Latest" a "reset by
time" option -- IMHO it belongs to "reset by position"?
* Nit: Description of "Reset to Earliest"
> using Kafka Consumer's `auto.offset.reset` to `earliest`
I thi
Hi,
according to the feedback, I've updated the KIP:
- We have added and ordered the scenarios, scopes and executions of the
Reset Offset tool.
- Consider it as an extension to the current `ConsumerGroupCommand` tool
- Execution will be possible without generating JSON files.
https://cwiki.apach
Yeah, that's a good point.
Some of the operations might make sense on multiple partitions at once. Moving
to a timestamp might apply to all partitions, moving backwards and forwards by
N offsets might apply to all partitions.
However, moving to a specific offset ("set to offset 43") would most
ache.org, Users
Date: 02/08/2017 02:23 PM
Subject: Re: KIP-122: Add a tool to Reset Consumer Group Offsets
Great. I think I got the idea. What about this options:
Scenarios:
1. Current status
´kafka-consumer-groups.sh --reset-offset --group cg1´
2. To Datetime
´kafka-consume
Just to clarify, we'll need to allow specifying topic and partition. I
don't think we want this on ALL partitions at once.
On Wed, Feb 8, 2017 at 3:35 PM, Gwen Shapira wrote:
> That's what I'd like to see. For example, suppose a Connect task fails
> because it can't deserialize an event from a pa
That's what I'd like to see. For example, suppose a Connect task fails
because it can't deserialize an event from a partition. Stop
connector, move offset forward, start connector. Boom!
On Wed, Feb 8, 2017 at 3:22 PM, Matthias J. Sax wrote:
> I am not sure about --reset-plus and --reset-minus
>
I am not sure about --reset-plus and --reset-minus
Would this skip n messages forward/backward for each partitions?
-Matthias
On 2/8/17 2:23 PM, Jorge Esteban Quilcate Otoya wrote:
> Great. I think I got the idea. What about this options:
>
> Scenarios:
>
> 1. Current status
>
> ´kafka-consu
Great. I think I got the idea. What about this options:
Scenarios:
1. Current status
´kafka-consumer-groups.sh --reset-offset --group cg1´
2. To Datetime
´kafka-consumer-groups.sh --reset-offset --group cg1 --reset-to-datetime
2017-01-01T00:00:00.000´
3. To Period
´kafka-consumer-groups.sh -
Hi,
Just my few thoughts:
does it need to be json?
the old zkOffset tool had a nice format,
very easy to manipulate on cli
very powerfull: changes as many consumergroups/topics/partitions in one
go as you want
maybe allow -1 and -2 to indicate earliest and latest reset regardless
of what the
Yes - using a tool like this to skip a set of consumer groups over a
corrupt/bad message is definitely appealing.
B
On Wed, Feb 8, 2017 at 9:37 PM Gwen Shapira wrote:
> I like the --reset-to-earliest and --reset-to-latest. In general,
> since the JSON route is the most challenging for users, we
I like the --reset-to-earliest and --reset-to-latest. In general,
since the JSON route is the most challenging for users, we want to
provide a lot of ways to do useful things without going there.
Two things that can help:
1. A lot of times, users want to skip few messages that cause issues
and co
Thanks for the feedback!
@Onur, @Gwen:
Agree. Actually at the first draft I considered to have it inside
´kafka-consumer-groups.sh´, but I decide to propose it as a standalone tool
to describe it clearly and focus it on reset functionality.
But now that you mentioned, it does make sense to have
As long as the CLI is a bit consistent? Like, not just adding 3
arguments and a JSON parser to the existing tool, right?
On Tue, Feb 7, 2017 at 10:29 PM, Onur Karaman
wrote:
> I think it makes sense to just add the feature to kafka-consumer-groups.sh
>
> On Tue, Feb 7, 2017 at 10:24 PM, Gwen Shap
I think it makes sense to just add the feature to kafka-consumer-groups.sh
On Tue, Feb 7, 2017 at 10:24 PM, Gwen Shapira wrote:
> Thanks for the KIP. I'm super happy about adding the capability.
>
> I hate the interface, though. It looks exactly like the replica
> assignment tool. A tool everyon
Thanks for the KIP. I'm super happy about adding the capability.
I hate the interface, though. It looks exactly like the replica
assignment tool. A tool everyone loves so much that there are multiple
projects, open and closed, that try to fix it.
Can we swap it with something that looks a bit mor
Hey Jorge,
Thanks for the KIP. I have some quick comments:
- Should we allow user to use wildcard to reset offset of all groups for a
given topic as well?
- Should we allow user to specify timestamp per topic partition in the json
file as well?
- Should the script take some credential file to mak
Hi all,
I would like to propose a KIP to Add a tool to Reset Consumer Group Offsets.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+a+tool+to+Reset+Consumer+Group+Offsets
Please, take a look at the proposal and share your feedback.
Thanks,
Jorge.
22 matches
Mail list logo