Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

like Long) to avoid this overhead? > > > > -Matthias > > On 5/15/19 6:05 PM, John Roesler wrote: >> Sounds good! >> >> On Tue, May 14, 2019 at 9:21 AM Development wrote: >>> >>> Hey, >>> >>> I think it the proposal is fina

Posted a new article about Kafka Streams

Hello Kafka Dev community, I wrote an article on implementing a custom transformer using Processor API for Kafka Streams! https://medium.com/@daniyaryeralin/utilizing-kafka-streams-processor-api-and-implementing-custom-aggregation-f6a4a6c376be

Re: Posted a new article about Kafka Streams

Bad link: https://medium.com/@daniyaryeralin/utilizing-kafka-streams-processor-api-and-implementing-custom-aggregator-6cb23d00eaa7 > On Jun 14, 2019, at 11:07 AM, Development wrote: > > Hello Kafka Dev community, > > I wrote an article on implementing a custom transformer usin

[VOTE] KIP-466: Add support for List serialization and deserialization

Hello, I want to start a vote for KIP-466 > "Add

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

Hmm the formatting got removed unfortunately. I’m sorry, it got harder to read my email. > On Jun 12, 2019, at 10:27 AM, Development wrote: > > Hi Matthias, > > Indeed, you are right. I missed your email, I had a problem with my mail > server, so I guess I didn’t receiv

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

fig instead of a >> constructor element. >> >> >> Just bounding off ideas -- maybe there are good reasons (too >> complicated?) to not support either of them. >> > > > -Matthias > > On 6/10/19 8:44 AM, Development wrote: >> Bump >> >>> On Ma

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

Bump > On May 24, 2019, at 2:09 PM, Development wrote: > > Hey, > > - did we consider to make the return type (ie, ArrayList, vs > LinkesList) configurable or encode it the serialized bytes? > > Not sure about this one. Could you elaborate? > > - atm the s

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

bump

Re: Posted a new article about Kafka Streams

ache.org/confluence/pages/viewpage.action?pageId=97553756). > It seems like there's soft interest, but maybe not yet enough to push it > over the finish line. > > Again, great job! > > Paul > > On Fri, Jun 14, 2019 at 10:33 AM Development wrote: > >>

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

eterized class structure. > Specifically, you might find > https://fasterxml.github.io/jackson-core/javadoc/2.0.0/com/fasterxml/jackson/core/type/TypeReference.html > interesting. > > Thanks, > -John > > On Mon, Jun 17, 2019 at 12:38 PM Development wrote: >> >> bump

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

object into the constructor, something like: It could be an option, but not a pretty one. What do you think of my approach to use vanilla java and canonical class name? (As described previously) Best, Daniyar Yeralin > On Jun 20, 2019, at 2:45 PM, Development wrote: > > Hi John, > > Tha

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

for your help in this KIP :) Best, Daniyar Yeralin > On Jun 21, 2019, at 1:44 PM, John Roesler wrote: > > Hey Daniyar, > > Looks good to me! Thanks for considering it. > > Thanks, > -John > > On Fri, Jun 21, 2019 at 9:04 AM Development <mailto:d..

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

out with > the syntax. > > Given these updates, I'm curious if anyone else has feedback about > this proposal. Personally, I think it sounds fine! > > -John > > On Wed, May 8, 2019 at 1:01 PM Development wrote: >> >> Hey, >> >> That worked! I certainl

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

lly I feel it would make sense to expand the > scope of this KIP a bit to include all Collections as a logical unit, but > the ROI could be low.. > > (I know of at least one instance in the unit tests where a Set serde could > be useful, and there may be more) > > On Thu, May 9,

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

o-argument static factory method for it, but it seems you could > still have a static method: > `public static Serde> List(Serde innerSerde)`. > > Thoughts? > > On Tue, May 7, 2019 at 12:18 PM Development wrote: >> >> Absolutely agree. Already pushed another comm

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

ic Serde> List(Serde innerSerde) { > return new ListSerde(innerSerde); > } > > Does that work? > > Cheers, > > Chris > > On Wed, May 8, 2019 at 10:29 AM Development <mailto:d...@yeralin.net>> wrote: > Hi John, > > I updated JIRA and KIP.

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

here. Any ideas? Best, Daniyar Yeralin > On May 9, 2019, at 2:10 PM, Development wrote: > > Hey Sophie, > > Thank you for your input. I think I’d rather finish this KIP as is, and then > open a new one for the Collections (if everyone agrees). I don’t want to > exten

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

gt; We can always add it later, but I think you're right, we need some > kind of more sophisticated support, or at least a second argument for > the inner class. > > For now, it seems like most use cases would be satisfied without > serdeFrom(...List...) > > -John >

[VOTE] KIP-466: Add support for List serialization and deserialization

Hello, I want to start a vote for KIP-466 "Add support for List serialization and deserialization”. The implementation can be found as a PR

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

rg.apache.kafka.common.serialization.Serdes#UUID, and > also, a UUID isn't a List. Did you mean that you need to use *lists of* > UUIDs? > > Thanks, > -John > > On Mon, May 6, 2019 at 11:49 AM Development wrote: > >> Hello, >> >> Starting a discussion for K

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

since the list is already in a > specific order, which the user themselves controls, it doesn't seem > strictly necessary to offer an option to sort the data during serialization. > > Thanks, > -John > > On Mon, May 6, 2019 at 8:47 PM Development wrote: > >> Hi John, &

[DISCUSS] KIP-466: Add support for List serialization and deserialization

Hello, Starting a discussion for KIP-466 adding support for List Serde. PR is created under https://github.com/apache/kafka/pull/6592 There are two topics I would like to discuss: 1. Since type for List serve needs to be declared before hand, I could

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

figs should be listed and explained in the KIP > anyway, and we can discuss further after you have updated the KIP (I can > ask more detailed question if I have any). > > > -Matthias > > On 6/21/19 2:05 PM, Development wrote: >> Yes, you are right. ByteSerializer is no

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

, we'd just stick with fully config-driven or fully source-code-driven, > not half/half. > > What do you think? > > Thanks, > -John > > > On Tue, Jul 9, 2019 at 9:58 AM Development <mailto:d...@yeralin.net>> wrote: > > > > Hi John, > > > &

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

pl`? > > > default.key.list.serde.impl -> default.list.key.serde.type > default.value.list.serde.impl -> default.list.value.serde.type > default.key.list.serde.element -> default.list.key.serde.inner > default.value.list.serde.element -> default.list.value.serde.inner > >

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

I do? Should I move my classes under org.apache.kafka.streams.kstream package instead? > On Jul 15, 2019, at 10:45 AM, Development wrote: > > Hi Matthias, > > Thank you for your input. > > I updated the KIP, made it a little more readable. > > I think the config

PR Broken builds

Hi, I’ve been working on KAFKA-8326 PR (https://github.com/apache/kafka/pull/6592 ), but I can never pass the checks. The build keeps failing with unrelated errors from SpotBugs. Seems like other people’s PRs are also affected. What’s the current

Help needed with PR #6592

Hi, I’ve been maintaining this PR "KAFKA-8326: Introduce List Serde" https://github.com/apache/kafka/pull/6592 My linters keep failing during the building, complaining about: "Disallowed import - org.apache.kafka.clients.CommonClientConfigs.” in my

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

thing merged. > > Thanks, > -john > > On Tue, Jul 16, 2019 at 2:05 PM Development wrote: >> >> Hi, >> >> Pushed new changes under my PR: https://github.com/apache/kafka/pull/6592 >> <https://github.com/apache/kafka/pull/6592> >>

[VOTE] KIP-466: Add support for List serialization and deserialization

Hi, I’d like to start a vote thread for KIP-466. This addition will introduce new serde type ListSerde. More info at: https://cwiki.apache.org/confluence/display/KAFKA/KIP-466%3A+Add+support+for+List%3CT%3E+serialization+and+deserialization Best, Daniyar Yeralin

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

Hi, Pushed new changes under my PR: https://github.com/apache/kafka/pull/6592 <https://github.com/apache/kafka/pull/6592> Feel free to put any comments in there. Best, Daniyar Yeralin > On Jul 15, 2019, at 1:06 PM, Development wrote: > > Hi John, > > I knew I was m

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

reams, there are multiple and > StreamsConfig specifies the default one of an operator does not > overwrite it. > > Thoughts? > > > Also, the KIP should explicitly mention to what classed certain configs > are added. Atm, the KIP only list parameter names, but does not state &

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

Bump > On Jul 22, 2019, at 11:26 AM, Development wrote: > > Hey Matthias, > > It looks a little confusing, but I don’t have enough expertise to judge on > the configuration placement. > > If you think, it is fine I’ll go ahead with this approach. > > Best, >

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

also need to be configured? So, by default I'd have >> to specify all six configs when I'm using Streams? >> >> I guess in the Serde, it could make use of a package-protected constructor >> for the serializer and deserializer that fixes the list type and inner type >>

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

gt;> It seems like the key/value serde configs are a better analogy than the >>> windowed serde. >>> ProducerConfig: key.serializer >>> ConsumerConfig: key.deserializer >>> StreamsConfig: default.key.serde >>> >>> Just to make sure I understan

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

d be used purely from the Java > interface, but has the drawbacks I listed above. This method, not the > constructor, is what I proposed to remove from the KIP. > > Thanks, > -John > > > On Mon, Jul 15, 2019 at 10:15 AM Development <mailto:d...@yeralin.net>> w

Re: [VOTE] KIP-466: Add support for List serialization and deserialization

aniyar, >>> >>> I'm +1 (nonbinding) >>> >>> -John >>> >>> On Tue, Jun 11, 2019 at 1:45 PM Development wrote: >>> >>>> Hello, >>>> >>>> I want to start a vote for KIP-466 < >>>&g

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

; > I guess the simplest solution is, to actually add > >>> default.list.key/value.serde.type >>> default.list.key/value.serde.inner > > to both `CommonClientConfigs` and `StreamsConfig`. > > It's not super clean, but I think it's the best we can do. Thoughts? >