Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-26 Thread Kamal Chandraprakash
Hi Ahmed, Thanks for the KIP! It will be a useful addition to know the last-stable-offset (LSO) for a partition. Motivation section is not clear to me. Can you document the scenarios on how the exposed LSO can be used for txn support/debug? -- Kamal On Thu, Mar 21, 2024 at 11:30 PM Justine

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-21 Thread Justine Olshan
Hey Ahmed, Echoing Andrew's comments to clean up the public interfaces, it was unclear to me if this new flag applies to all the options or just the "latest" option. Clarifying that and showing a few examples could help. Thanks, Justine On Thu, Mar 21, 2024 at 9:43 AM Andrew Schofield <

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-21 Thread Andrew Schofield
Hi, Glad to hear you’re better. Re-reading the KIP, I think the changes to the public interfaces are probably only the addition of the new `--isolation` flag on the `kafka-get-offsets.sh` tool. The `KafkaAdminClient.listOffsets` parameters already have what you need I think (OffsetSpec.LATEST

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-20 Thread Ahmed Sobeh
Hi Andrew, Apologies for disappearing, I had to undergo knee surgery, all good now! I adjusted the KIP as you suggested, do you think it's ready for the voting stage? On Wed, Mar 6, 2024 at 4:02 PM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Ahmed, > Looks good to me with

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-06 Thread Andrew Schofield
Hi Ahmed, Looks good to me with one remaining comment. You’ve used: bin/kafka-get-offsets.sh … --time -1 --isolation -committed bin/kafka-get-offsets.sh … --time latest --isolation -committed bin/kafka-get-offsets.sh … --time -1 --isolation -uncommitted bin/kafka-get-offsets.sh … --time

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-03-05 Thread Ahmed Sobeh
I adjusted the KIP according to what we agreed on, let me know if you have any comments! Best, Ahmed On Thu, Feb 29, 2024 at 1:44 AM Luke Chen wrote: > Hi Ahmed, > > Thanks for the KIP! > > Comments: > 1. If we all agree with the suggestion from Andrew, you could update the > KIP. > >

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-02-28 Thread Luke Chen
Hi Ahmed, Thanks for the KIP! Comments: 1. If we all agree with the suggestion from Andrew, you could update the KIP. Otherwise, LGTM! Thanks. Luke On Thu, Feb 29, 2024 at 1:32 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Ahmed, > Could do. Personally, I find the

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-02-28 Thread Andrew Schofield
Hi Ahmed, Could do. Personally, I find the existing “--time -1” totally horrid anyway, which was why I suggested an alternative. I think your suggestion of a flag for isolation level is much better than -6. Maybe I should put in a KIP which adds: --latest (as a synonym for --time -1) --earliest

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-02-28 Thread Ahmed Sobeh
Hi Andrew, Thanks for the hint! That sounds reasonable, do you think adding a conditional argument, something like "--time -1 --isolation -committed" and "--time -1 --isolation -uncommitted" would make sense to keep the consistency of getting the offset by time? or do you think having a special

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-02-27 Thread Andrew Schofield
Hi Ahmed, Thanks for the KIP. It looks like a useful addition. The use of negative timestamps, and in particular letting the user use `--time -1` or the equivalent `--time latest` is a bit peculiar in this tool already. The negative timestamps come from

[DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

2024-02-27 Thread Ahmed Sobeh
Hi all, I would like to start a discussion on KIP-1021, which would enable getting LSO in kafka-get-offsets.sh: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1021%3A+Allow+to+get+last+stable+offset+%28LSO%29+in+kafka-get-offsets.sh Best, Ahmed