Upgrade :)
On Tue, Jun 28, 2016 at 6:49 PM, Rohit Valsakumar <[email protected]> wrote: > Hi Jay, > > Thanks for the reply. > > Unfortunately in our case due to legacy reasons we are using > WallclockTimestampExtractor in the application for all the streams and the > existing messages in the stream probably won¹t have timestamps as they are > being produced by legacy clients. So the events are being ingested with > processing times and it may not be able to synchronize based on the > message timestamps. What do you recommend for this scenario? > > Rohit > > On 6/28/16, 5:18 PM, "Jay Kreps" <[email protected]> wrote: > >>I think you may get this for free as Kafka Streams attempts to align >>consumption across different topics/partitions by the timestamp in the >>messages. So in a case where you are starting a job fresh and it has a >>database changelog to consume and a event stream to consume, it will >>attempt to keep the Ktable at the "time" the event stream is at. This is >>only a heuristic, of course, since messages are necessarily strongly >>ordered by time. I think this is likely mostly the same but slightly >>better >>than the bootstrap usage in Samza but also covers other cases of >>alignment. >> >>If you want more control you can override the timestamp extractor that >>associates time and hence priority for the streams: >>https://kafka.apache.org/0100/javadoc/org/apache/kafka/streams/processor/T >>imestampExtractor.html >> >>-Jay >> >>On Tue, Jun 28, 2016 at 2:49 PM, Rohit Valsakumar <[email protected]> >>wrote: >> >>> Hi all, >>> >>> Is there a way to consume all the contents of a kafka topic into a >>>KTable >>> before doing a left join with another Kstream? >>> >>> I am looking at something that simulates a bootstrap topic in a Samza >>>job. >>> >>> Thanks, >>> Rohit Valsakumar >>> >>> ________________________________ >>> >>> This email and any attachments may contain confidential and privileged >>> material for the sole use of the intended recipient. Any review, >>>copying, >>> or distribution of this email (or any attachments) by others is >>>prohibited. >>> If you are not the intended recipient, please contact the sender >>> immediately and permanently delete this email and any attachments. No >>> employee or agent of TiVo Inc. is authorized to conclude any binding >>> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo >>> Inc. may only be made by a signed written agreement. >>> > > > ________________________________ > > This email and any attachments may contain confidential and privileged > material for the sole use of the intended recipient. Any review, copying, or > distribution of this email (or any attachments) by others is prohibited. If > you are not the intended recipient, please contact the sender immediately and > permanently delete this email and any attachments. No employee or agent of > TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo > Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed > written agreement.
