Re: Referential transparency issue with transform()

2018-09-24 Thread Bill Bejeck
Hi Stéphane, Can you provide log files from the crash so we can see the exact failure? Thanks, Bill On Mon, Sep 24, 2018 at 8:12 AM Stéphane. D. wrote: > Hi, > > We just stumbled upon an issue with KStream.transform() where we had a > runtime error with this code: > > ``` > DeduplicationTransf

Re: Referential transparency issue with transform()

2018-09-24 Thread Bill Bejeck
Hi Stephane, Yes, you'll want to have your TransformerSupplier to return a new instance of your Transformer Otherwise, the same Transformer instance is used across all tasks. Since the Transformer can potentially perform stateful operations with a state store, and/or you can schedule punctations u

Referential transparency issue with transform()

2018-09-24 Thread Stéphane . D .
Hi, We just stumbled upon an issue with KStream.transform() where we had a runtime error with this code: ``` DeduplicationTransformer transformer = new DeduplicationTransformer<>(...); stream.transform(() -> transformer, ...) ``` The error was: Failed to process stream task 0_0 due to the follow