Hi Greg, Is this API documented that it does not return duplicate entries? Can we also get an opinion from PMC/Committers who have KafkaConnect expertise on whether this issue is a release blocker?
If we agree that it is not a release blocker then we can have a release note clarifying this behaviour and add a reference to the JIRA that follows up on the possible solutions. Thanks, Satish. On Tue, 19 Sept 2023 at 03:29, Greg Harris <greg.har...@aiven.io.invalid> wrote: > > Hey Satish, > > After investigating further, I believe that this is a regression, but > mostly a cosmetic one. > I don't think there is significant risk of breaking clients with this > change, but it would be confusing for users, so I'd still like to get > the fix into the next RC. > I've opened a PR here: https://github.com/apache/kafka/pull/14398 and > I'll work to get it merged promptly. > > Thanks! > > On Mon, Sep 18, 2023 at 11:54 AM Greg Harris <greg.har...@aiven.io> wrote: > > > > Hi Satish, > > > > While validating 3.6.0-rc0, I noticed this regression as compared to > > 3.5.1: https://issues.apache.org/jira/browse/KAFKA-15473 > > > > Impact: The `connector-plugins` endpoint lists duplicates which may > > cause confusion for users, or poor behavior in clients. > > Using the other REST API endpoints appears unaffected. > > I'll open a PR for this later today. > > > > Thanks, > > Greg > > > > On Thu, Sep 14, 2023 at 11:56 AM Satish Duggana > > <satish.dugg...@gmail.com> wrote: > > > > > > Thanks Justine for the update. I saw in the morning that these changes > > > are pushed to trunk and 3.6. > > > > > > ~Satish. > > > > > > On Thu, 14 Sept 2023 at 21:54, Justine Olshan > > > <jols...@confluent.io.invalid> wrote: > > > > > > > > Hi Satish, > > > > We were able to merge > > > > https://issues.apache.org/jira/browse/KAFKA-15459 yesterday > > > > and pick to 3.6. > > > > > > > > Hopefully nothing more from me on this release. > > > > > > > > Thanks, > > > > Justine > > > > > > > > On Wed, Sep 13, 2023 at 9:51 PM Satish Duggana > > > > <satish.dugg...@gmail.com> > > > > wrote: > > > > > > > > > Thanks Luke for the update. > > > > > > > > > > ~Satish. > > > > > > > > > > On Thu, 14 Sept 2023 at 07:29, Luke Chen <show...@gmail.com> wrote: > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > Since this PR: > > > > > > https://github.com/apache/kafka/pull/14366 only changes the doc, > > > > > > I've > > > > > > backported to 3.6 branch. FYI. > > > > > > > > > > > > Thanks. > > > > > > Luke > > > > > > > > > > > > On Thu, Sep 14, 2023 at 12:15 AM Justine Olshan > > > > > > <jols...@confluent.io.invalid> wrote: > > > > > > > > > > > > > Hey Satish -- yes, you are correct. KAFKA-15459 only affects 3.6. > > > > > > > PR should be finalized soon. > > > > > > > > > > > > > > Thanks, > > > > > > > Justine > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 1:41 AM Federico Valeri > > > > > > > <fedeval...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Satish, this is a small documentation fix about ZK to KRaft > > > > > > > > migration, that we would like to backport to 3.5 and 3.6 > > > > > > > > branches. > > > > > Are > > > > > > > > you ok with that? > > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14366 > > > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 3:13 AM Satish Duggana < > > > > > satish.dugg...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Thanks David for the quick resolution. > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > On Tue, 12 Sept 2023 at 22:51, David Arthur > > > > > > > > > <david.art...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > Satish, > > > > > > > > > > > > > > > > > > > > KAFKA-15450 is merged to 3.6 (as well as trunk, 3.5, and > > > > > > > > > > 3.4) > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > David > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 11:44 AM Ismael Juma > > > > > > > > > > <m...@ismaeljuma.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Justine, > > > > > > > > > > > > > > > > > > > > > > Probably best to have the conversation in the JIRA ticket > > > > > > > > > > > vs > > > > > the > > > > > > > > release > > > > > > > > > > > thread. Generally, we want to only include low risk bug > > > > > > > > > > > fixes > > > > > that > > > > > > > > are > > > > > > > > > > > fully compatible in patch releases. > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 7:16 AM Justine Olshan > > > > > > > > > > > <jols...@confluent.io.invalid> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Thanks Satish. I understand. > > > > > > > > > > > > Just curious, is this something that could be added to > > > > > 3.6.1? It > > > > > > > > would be > > > > > > > > > > > > nice to say that hanging transactions are fully covered > > > > > > > > > > > > in a > > > > > 3.6 > > > > > > > > release. > > > > > > > > > > > > I'm not as familiar with the rules around minor > > > > > > > > > > > > releases, but > > > > > > > > adding it > > > > > > > > > > > > there would give more time to ensure stability. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 5:49 AM Satish Duggana < > > > > > > > > satish.dugg...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Justine, > > > > > > > > > > > > > We can skip this change into 3.6 now as it is not a > > > > > blocker or > > > > > > > > > > > > > regression and it involves changes to the API > > > > > implementation. > > > > > > > > Let us > > > > > > > > > > > > > plan to add the gap in the release notes as you > > > > > > > > > > > > > mentioned. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 12 Sept 2023 at 04:44, Justine Olshan > > > > > > > > > > > > > <jols...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > We just discovered a gap in KIP-890 part 1. We > > > > > > > > > > > > > > currently > > > > > > > don't > > > > > > > > verify > > > > > > > > > > > > on > > > > > > > > > > > > > > txn offset commits, so it is still possible to have > > > > > hanging > > > > > > > > > > > > transactions > > > > > > > > > > > > > on > > > > > > > > > > > > > > the consumer offsets partitions. > > > > > > > > > > > > > > I've opened a jira to wire the verification in that > > > > > request. > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15449 > > > > > > > > > > > > > > > > > > > > > > > > > > > > This also isn't a regression, but it would be nice > > > > > > > > > > > > > > to > > > > > have > > > > > > > > part 1 > > > > > > > > > > > fully > > > > > > > > > > > > > > complete. I have opened a PR with the fix: > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14370. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I understand if there are concerns about last minute > > > > > changes > > > > > > > > to this > > > > > > > > > > > > API > > > > > > > > > > > > > > and we can hold off if that makes the most sense. > > > > > > > > > > > > > > If we take that route, I think we should still keep > > > > > > > > verification for > > > > > > > > > > > > the > > > > > > > > > > > > > > data partitions since it still provides full > > > > > > > > > > > > > > protection > > > > > there > > > > > > > > and > > > > > > > > > > > > > improves > > > > > > > > > > > > > > the transactions experience. We will need to call > > > > > > > > > > > > > > out > > > > > the gap > > > > > > > > in the > > > > > > > > > > > > > > release notes for consumer offsets partitions > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 11, 2023 at 12:29 PM David Arthur > > > > > > > > > > > > > > <david.art...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Another (small) ZK migration issue was identified. > > > > > This one > > > > > > > > isn't a > > > > > > > > > > > > > > > regression (it has existed since 3.4), but I > > > > > > > > > > > > > > > think it's > > > > > > > > reasonable > > > > > > > > > > > to > > > > > > > > > > > > > > > include. It's a small configuration check that > > > > > > > > > > > > > > > could > > > > > > > > potentially > > > > > > > > > > > save > > > > > > > > > > > > > end > > > > > > > > > > > > > > > users from some headaches down the line. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15450 > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14367 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think we can get this one committed to trunk > > > > > > > > > > > > > > > today. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Sep 10, 2023 at 7:50 PM Ismael Juma < > > > > > > > > m...@ismaeljuma.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That sounds great. I think we should aim to only > > > > > allow > > > > > > > > blockers > > > > > > > > > > > > > > > > (regressions, impactful security issues, etc.) > > > > > > > > > > > > > > > > on > > > > > the 3.6 > > > > > > > > branch > > > > > > > > > > > > > until > > > > > > > > > > > > > > > > 3.6.0 is out. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 9, 2023, 12:20 AM Satish Duggana < > > > > > > > > > > > > > satish.dugg...@gmail.com> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Ismael, > > > > > > > > > > > > > > > > > It looks like we will publish RC0 by 14th Sep. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 8 Sept 2023 at 19:23, Ismael Juma < > > > > > > > > m...@ismaeljuma.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you have a sense of when we'll publish > > > > > > > > > > > > > > > > > > RC0? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023 at 6:27 AM David Arthur > > > > > > > > > > > > > > > > > > <david.art...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Quick update on my two blockers: > > > > > > > > > > > > > > > > > > > KAFKA-15435 is > > > > > > > > merged to > > > > > > > > > > > > > trunk and > > > > > > > > > > > > > > > > > > > cherry-picked to 3.6. I have a PR open for > > > > > > > > KAFKA-15441 and > > > > > > > > > > > > will > > > > > > > > > > > > > > > > > hopefully > > > > > > > > > > > > > > > > > > > get it merged today. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023 at 5:26 AM Ivan > > > > > > > > > > > > > > > > > > > Yurchenko < > > > > > > > > > > > > i...@ivanyu.me> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish and all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wonder if > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-14993 > > > > > > > > > > > > > > > > > should be > > > > > > > > > > > > > > > > > > > > included in the 3.6 release plan. I'm > > > > > thinking > > > > > > > > that when > > > > > > > > > > > > > > > > > implemented, it > > > > > > > > > > > > > > > > > > > > would be a small, but still a change in > > > > > > > > > > > > > > > > > > > > the > > > > > RSM > > > > > > > > contract: > > > > > > > > > > > > > throw > > > > > > > > > > > > > > > an > > > > > > > > > > > > > > > > > > > > exception instead of returning an empty > > > > > > > > InputStream. > > > > > > > > > > > Maybe > > > > > > > > > > > > it > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > > > included right away to save the > > > > > > > > > > > > > > > > > > > > migration > > > > > later? > > > > > > > > What do > > > > > > > > > > > > you > > > > > > > > > > > > > > > think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > Ivan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023, at 02:52, Satish > > > > > > > > > > > > > > > > > > > > Duggana > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Jose, > > > > > > > > > > > > > > > > > > > > > Thanks for looking into this issue and > > > > > > > resolving > > > > > > > > it > > > > > > > > > > > with > > > > > > > > > > > > a > > > > > > > > > > > > > > > quick > > > > > > > > > > > > > > > > > fix. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 7 Sept 2023 at 21:40, José > > > > > > > > > > > > > > > > > > > > > Armando > > > > > > > García > > > > > > > > > > > Sancio > > > > > > > > > > > > > > > > > > > > > <jsan...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 6, 2023 at 4:58 PM > > > > > > > > > > > > > > > > > > > > > > Satish > > > > > > > Duggana < > > > > > > > > > > > > > > > > > > > > satish.dugg...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Greg, > > > > > > > > > > > > > > > > > > > > > > > It seems > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-14273 > > > > > > > > > > > > > > > > has > > > > > > > > > > > > > > > > > > > been > > > > > > > > > > > > > > > > > > > > > > > there in 3.5.x too. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I also agree that it should be a > > > > > > > > > > > > > > > > > > > > > > blocker > > > > > for > > > > > > > > 3.6.0. > > > > > > > > > > > It > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > have > > > > > > > > > > > > > > > > > > > > > > been a blocker for those previous > > > > > releases. I > > > > > > > > didn't > > > > > > > > > > > > fix > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > because, > > > > > > > > > > > > > > > > > > > > > > unfortunately, I wasn't aware of the > > > > > issue > > > > > > > and > > > > > > > > jira. > > > > > > > > > > > > > > > > > > > > > > I'll create a PR with a fix in case > > > > > > > > > > > > > > > > > > > > > > the > > > > > > > > original > > > > > > > > > > > author > > > > > > > > > > > > > > > doesn't > > > > > > > > > > > > > > > > > > > > respond in time. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Satish, do you agree? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > >
