Re: Kafka Streams: from a KStream, aggregating records with the same key and updated metrics ?

Thanks for sharing back your findings/code, Nicolas! -Michael On Mon, Jan 16, 2017 at 11:15 PM, Nicolas Fouché wrote: > If anyone is interested, here is my custom timestamp extractor: > https://gist.github.com/nfo/54d5830720e163d2e7e848b6e4baac20 . > > 2017-01-16 15:52

Re: Kafka Streams: from a KStream, aggregating records with the same key and updated metrics ?

If anyone is interested, here is my custom timestamp extractor: https://gist.github.com/nfo/54d5830720e163d2e7e848b6e4baac20 . 2017-01-16 15:52 GMT+01:00 Nicolas Fouché : > Hi Michael, > > got it. I understand that it would be less error-prone to generate the > final

Re: Kafka Streams: from a KStream, aggregating records with the same key and updated metrics ?

Hi Michael, got it. I understand that it would be less error-prone to generate the final "altered" timestamp on the Producer side, instead of trying to compute it each time the record is consumed. Thanks. Nicolas. 2017-01-16 10:03 GMT+01:00 Michael Noll : > Nicolas, > >

Re: Kafka Streams: from a KStream, aggregating records with the same key and updated metrics ?

Hi Eno, 2. Well, records could arrive out of order. But it should happen rarely, and it's no big deal anyway. So let's forget about the version number if it makes things easier ! 3. I completely missed out on KTable aggregations. Thanks a lot for the pointer, that opens new perspectives. ... a

Re: Kafka Streams: from a KStream, aggregating records with the same key and updated metrics ?

Hi Nicolas, There is a lot here, so let's try to split the concerns around some themes: 1. The Processor API is flexible and can definitely do what you want, but as you mentioned, at the cost of you having to manually craft the code. 2. Why are the versions used? I sense there is concern about

Kafka Streams: from a KStream, aggregating records with the same key and updated metrics ?

Hi, long long technical story, sorry for that. I'm dealing with a special case. My input topic receives records containing an id in the key (and another field for partitioning), and a version number in the value, amongst other metrics. Records with the same id are sent every 5 seconds, and the