Is someone still working on this KIP? If no one is working on it, I'd like to update the target release on the wiki and start the vote.
Thanks, Christopher Moyer On 2025/07/16 23:19:22 "Matthias J. Sax" wrote: > I did not re-read the KIP, but base on the last comment from Lucas, it > sounds like it. > > If there is no other open points to discuss, the next step would be to > start a VOTE thread, to get the KIP approved. > > > -Matthias > > > On 7/15/25 8:09 PM, Shashwat Pandey wrote: > > Hi Matthias, > > > > Sorry this totally fell off my radar, what would be left to do? > > > > Would updating the version to latest release be enough? > > > > Regards, > > Shashwat Pandey > > > > > > On Wed, Jun 11, 2025 at 5:37 PM Matthias J. Sax <[email protected] > > <[email protected]>> wrote: > > > > Is there still interest to complete this KIP? > > > > -Matthias > > > > On 8/26/24 5:06 AM, Lucas Brutschy wrote: > > > Hi everybody, > > > > > > this KIP is looking good to me. Just note that the release version is > > > set to 3.8 and should probably be updated. > > > > > > Are we ready to open a vote on this one? I do not have any > > additional comments. > > > > > > Cheers, > > > Lucas > > > > > > On Mon, Jun 24, 2024 at 12:55 AM Shashwat Pandey > > > <[email protected] > > <[email protected]>> wrote: > > >> > > >> Totally agree on the constructors to pass in key/val/ts/headers. > > >> > > >> I was thinking about adding the method for Record as a helper > > method if > > >> there was a scenario to test both a processor and > > fixedkeyprocessor at the > > >> same time. Also, it keeps the same signature as the > > >> InternalFixedKeyRecordFactory as well. > > >> > > >> > > >> Regards, > > >> Shashwat Pandey > > >> > > >> > > >> On Sat, Jun 22, 2024 at 8:25 PM Matthias J. Sax > > <[email protected] <[email protected]>> wrote: > > >> > > >>> Thanks for the update. > > >>> > > >>> About the wiki account. Creating the account was done by Infra, but > > >>> setting permissions is on us. Fixed. > > >>> > > >>> > > >>> About the KIP: > > >>> > > >>> Should we pass-in key/value/ts/headers and mimic the > > constructors for > > >>> `TestRecord` (ie, have similar createXxx(...) overloads as factory > > >>> methods as the constructor overloads) instead of `Record` and > > >>> `TestRecord`? Or is there some benefit I am missing to pass in > > >>> `Record/TestRecord` ? > > >>> > > >>> > > >>> -Matthias > > >>> > > >>> > > >>> On 6/22/24 9:32 AM, Shashwat Pandey wrote: > > >>>> Hi Matthias, > > >>>> > > >>>> That makes sense to me! I updated the code, definitely want to > > get your > > >>>> perspective on whether or not we want to support the > > >>>> `createFixedKeyRecord(Record)` method, since we already have the > > >>>> `TestRecord` defined in the utils, it might be cleaner to just > > support > > >>> the > > >>>> `createFixedKeyRecord(TestRecord)` method. > > >>>> > > >>>> For reference - > > >>>> > > >>> https://github.com/s7pandey/kafka/ > > commit/ 8ac92509d455d8175381a9b4c83900218941bf05#diff-2a3e6e23894a888e8c2fa486e2330f42b8fb28fe2216ba182e27d3d14958457b <https://github.com/s7pandey/kafka/commit/ 8ac92509d455d8175381a9b4c83900218941bf05#diff-2a3e6e23894a888e8c2fa486e2330f42b8fb28fe2216ba182e27d3d14958457b> > > >>>> > > >>>> Also, looks like I do not have access to update the KIP, my > > confluence > > >>>> account is active now (s7pandey) but I think I need some > > permissions on > > >>> the > > >>>> actual KIP page. > > >>>> > > >>>> Shashwat > > >>>> > > >>>> On Wed, Jun 12, 2024 at 8:32 PM Matthias J. Sax > > <[email protected] <[email protected]>> > > >>> wrote: > > >>>> > > >>>>> I believe the class name was picked on purpose, to make clear > > that it > > >>>>> should not be used -- the problem is, that the class is in a > > public > > >>>>> package and is by itself public (that's unfortunately > > require, given how > > >>>>> Java works). > > >>>>> > > >>>>> Of course, it's also in the JavaDocs that the class is > > internal and > > >>>>> should not be used, but not everyone reads the JavaDocs > > necessarily, so > > >>>>> making it part of the name makes it much more explicit, what > > I believe > > >>>>> is a good thing. > > >>>>> > > >>>>> I would consider it a fix/improvement, if we could exclude > > >>>>> `InternalFixedKeyRecordFactory` from JavaDoc generation > > during the > > >>>>> release build -- but I don't think we need a KIP for this, as > > I would > > >>>>> rather consider it a bug-fix to exclude an internal class in the > > >>>>> JavaDocs build step. > > >>>>> > > >>>>> > > >>>>> -Matthias > > >>>>> > > >>>>> On 6/12/24 4:47 PM, Shashwat Pandey wrote: > > >>>>>> Hi Matthias, > > >>>>>> > > >>>>>> I think that strategy definitely works, abstracting away > > changes to > > >>>>>> FixedKeyRecord from users, I can put that new factory class > > and update > > >>>>> the > > >>>>>> KIP. > > >>>>>> > > >>>>>> This might be a discussion for another KIP, but would it > > also make > > >>> sense > > >>>>> to > > >>>>>> rename the > > >>>>>> InternalFixedKeyRecordFactory to just FixedKeyRecordFactory > > also make > > >>>>> sense? > > >>>>>> > > >>>>>> > > >>>>>> Regards, > > >>>>>> Shashwat Pandey > > >>>>>> > > >>>>>> > > >>>>>> On Mon, Jun 10, 2024 at 5:07 PM Matthias J. Sax > > <[email protected] <[email protected]>> > > >>>>> wrote: > > >>>>>> > > >>>>>>> Shaswhat, > > >>>>>>> > > >>>>>>> any updates on this KIP? -- I still think that recommending > > to use > > >>>>>>> `InternalFixedKeyRecordFactory` is not the best way to > > write test > > >>> code. > > >>>>>>> Changing `FixedKeyRecord` constructor (as I mentioned in my > > last > > >>> email) > > >>>>>>> might not be a good solution either. > > >>>>>>> > > >>>>>>> Maybe a cleaner way would be (so sidestep this problem), to > > add a new > > >>>>>>> public "factory class" into the test package to generate > > >>>>>>> FixedKeyRecords, and this factory could internally use > > >>>>>>> `InternalFixedKeyRecordFactory`? It looks cleaner to me > > from an API > > >>> POV, > > >>>>>>> and if we change anything how `FixedKeyRecord` can be > > created, it > > >>> would > > >>>>>>> become a non-user-facing / internal change to the "factory" we > > >>> provide. > > >>>>>>> > > >>>>>>> > > >>>>>>> -Matthias > > >>>>>>> > > >>>>>>> On 5/22/24 12:02 AM, Matthias J. Sax wrote: > > >>>>>>>> I was not aware of `InternalFixedKeyRecordFactory`. As the > > name > > >>>>>>>> indicates, it's considered an internal class, so not sure > > if we > > >>> should > > >>>>>>>> recommend to use it in test... > > >>>>>>>> > > >>>>>>>> I understand why this class is required, and why it was > > put into a > > >>>>>>>> public package; the way Java works, enforces this. Not > > sure if we > > >>> could > > >>>>>>>> find a better solution. > > >>>>>>>> > > >>>>>>>> Might be good to hear from others. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> -Matthias > > >>>>>>>> > > >>>>>>>> On 5/21/24 3:57 PM, Shashwat Pandey wrote: > > >>>>>>>>> Looking at the ticket and the sample code, I think it > > would be > > >>>>>>>>> possible to > > >>>>>>>>> continue using `InternalFixedKeyRecordFactory` as the > > avenue to > > >>> create > > >>>>>>>>> `FixedKeyRecord`s in tests. As long as there was a > > >>>>>>>>> MockFixedKeyProcessorContext, I think we would be able to > > test > > >>>>>>>>> FixedKeyProcessors without a Topology. > > >>>>>>>>> > > >>>>>>>>> I created a sample repo with the > > `MockFixedKeyProcessorContext` here > > >>>>> is > > >>>>>>>>> what I think the tests would look like: > > >>>>>>>>> > > >>>>>>> > > >>>>> > > >>> https://github.com/s7pandey/kafka-processor-tests/blob/main/ > > src/test/java/com/example/demo/MyFixedKeyProcessorTest.java > > <https://github.com/s7pandey/kafka-processor-tests/blob/main/src/ > > test/java/com/example/demo/MyFixedKeyProcessorTest.java> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> On Mon, May 20, 2024 at 9:05 PM Matthias J. Sax > > <[email protected] <[email protected]>> > > >>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Had a discussion on > > >>>>> https://issues.apache.org/jira/browse/KAFKA-15242 <https:// > > issues.apache.org/jira/browse/KAFKA-15242> > > >>>>>>>>>> and it was pointed out, that we also need to do > > something about > > >>>>>>>>>> `FixedKeyRecord`. It does not have a public constructor > > (what is > > >>>>>>>>>> correct; it should not have one). However, this makes > > testing > > >>>>>>>>>> `FixedKeyProcessor` impossible w/o extending > > `FixedKeyRecord` > > >>>>> manually > > >>>>>>>>>> what does not seem to be right (too clumsy). > > >>>>>>>>>> > > >>>>>>>>>> It seems, we either need some helper builder method (but > > not clear > > >>> to > > >>>>>>> me > > >>>>>>>>>> where to add it in an elegant way) which would provide > > us with a > > >>>>>>>>>> `FixedKeyRecord`, or add some sub-class to the test- > > utils module > > >>>>> which > > >>>>>>>>>> would extend `FixedKeyRecord`? -- Or maybe an even better > > >>> solution? I > > >>>>>>>>>> could not think of something else so far. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Thoughts? > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On 5/3/24 9:46 AM, Matthias J. Sax wrote: > > >>>>>>>>>>> Please also update the KIP. > > >>>>>>>>>>> > > >>>>>>>>>>> To get a wiki account created, please request it via a > > commet on > > >>>>> this > > >>>>>>>>>>> ticket: https://issues.apache.org/jira/browse/ > > INFRA-25451 <https://issues.apache.org/jira/browse/INFRA-25451> > > >>>>>>>>>>> > > >>>>>>>>>>> After you have the account, please share your wiki id, > > and we can > > >>>>> give > > >>>>>>>>>>> you write permission on the wiki. > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> -Matthias > > >>>>>>>>>>> > > >>>>>>>>>>> On 5/3/24 6:30 AM, Shashwat Pandey wrote: > > >>>>>>>>>>>> Hi Matthias, > > >>>>>>>>>>>> > > >>>>>>>>>>>> Sorry this fell out of my radar for a bit. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Revisiting the topic, I think you’re right and we > > accept the > > >>>>>>>>>>>> duplicated > > >>>>>>>>>>>> nesting as an appropriate solution to not affect the > > larger > > >>> public > > >>>>>>>>>>>> API. > > >>>>>>>>>>>> > > >>>>>>>>>>>> I can update my PR with the change. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Regards, > > >>>>>>>>>>>> Shashwat Pandey > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Wed, May 1, 2024 at 11:00 PM Matthias J. Sax < > > >>> [email protected] <[email protected]>> > > >>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Any updates on this KIP? > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On 3/28/24 4:11 AM, Matthias J. Sax wrote: > > >>>>>>>>>>>>>> It seems that `MockRecordMetadata` is a private > > class, and thus > > >>>>> not > > >>>>>>>>>>>>>> part > > >>>>>>>>>>>>>> of the public API. If there are any changes > > required, we don't > > >>>>>>>>>>>>>> need to > > >>>>>>>>>>>>>> discuss on the KIP. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> For `CapturedPunctuator` and `CapturedForward` it's > > a little > > >>> bit > > >>>>>>>>>>>>>> more > > >>>>>>>>>>>>>> tricky. My gut feeling is, that the classes might > > not need to > > >>> be > > >>>>>>>>>>>>>> changed, but if we use them within > > `MockProcessorContext` and > > >>>>>>>>>>>>>> `MockFixedKeyProcessorContext` it might be weird to > > keep the > > >>>>>>> current > > >>>>>>>>>>>>>> nesting... The problem I see is, that it's not > > straightforward > > >>>>>>>>>>>>>> how to > > >>>>>>>>>>>>>> move the classes w/o breaking compatibility, nor if we > > >>> duplicate > > >>>>>>>>>>>>>> them as > > >>>>>>>>>>>>>> standalone classes w/o a larger "splash radius". (We > > would need > > >>>>>>>>>>>>>> to add > > >>>>>>>>>>>>>> new overloads for > > MockProcessorContext#scheduledPunctuators() > > >>> and > > >>>>>>>>>>>>>> MockProcessorContext#forwarded()). > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Might be good to hear from others if we think it's > > worth this > > >>>>>>> larger > > >>>>>>>>>>>>>> changes to get rid of the nesting, or just accept > > the somewhat > > >>>>> not > > >>>>>>>>>>>>>> ideal > > >>>>>>>>>>>>>> nesting as it technically is not a real issue? > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> -Matthias > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On 3/15/24 1:47 AM, Shashwat Pandey wrote: > > >>>>>>>>>>>>>>> Thanks for the feedback Matthias! > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> The reason I proposed the extension of > > MockProcessorContext > > >>> was > > >>>>>>>>>>>>>>> more > > >>>>>>>>>>>>>>> to do > > >>>>>>>>>>>>>>> with the internals of the class (MockRecordMetadata, > > >>>>>>>>>>>>>>> CapturedPunctuator and > > >>>>>>>>>>>>>>> CapturedForward). > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> However, I do see your point, I would then think to > > split > > >>>>>>>>>>>>>>> MockProcessorContext and > > MockFixedKeyProcessorContext, some of > > >>>>> the > > >>>>>>>>>>>>>>> internal > > >>>>>>>>>>>>>>> classes should also be extracted i.e. > > MockRecordMetadata, > > >>>>>>>>>>>>>>> CapturedPunctuator and probably a new > > CapturedFixedKeyForward. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Let me know what you think! > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Regards, > > >>>>>>>>>>>>>>> Shashwat Pandey > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Mon, Mar 11, 2024 at 10:09 PM Matthias J. Sax < > > >>>>>>> [email protected] <[email protected]>> > > >>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Thanks for the KIP Shashwat. Closing this testing > > gap is > > >>> great! > > >>>>>>> It > > >>>>>>>>>>>>>>>> did > > >>>>>>>>>>>>>>>> come up a few time already... > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> One question: why do you propose to `extend > > >>>>>>> MockProcessorContext`? > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Given how the actual runtime context classes are > > setup, it > > >>>>> seems > > >>>>>>>>>> that > > >>>>>>>>>>>>>>>> the regular context and fixed-key-context are > > distinct, and > > >>>>> thus > > >>>>>>> I > > >>>>>>>>>>>>>>>> believe both mock-context classes should be > > distinct, too? > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> What I mean is that FixedKeyProcessorContext does > > not extend > > >>>>>>>>>>>>>>>> Processo [message truncated...]
