Re: Difference between with and without state store cleanup streams startup

2017-04-01 Thread Matthias J. Sax
Yes. That's correct. -Matthias On 3/31/17 9:32 PM, Sachin Mittal wrote: > Hi, > Ok so basically what I understand is that there are no global offset > maintained from changelog topic at broker level. > Every local state store maintains the offset under a local checkpoint file. > > And in order t

Re: Difference between with and without state store cleanup streams startup

2017-03-31 Thread Sachin Mittal
Hi, Ok so basically what I understand is that there are no global offset maintained from changelog topic at broker level. Every local state store maintains the offset under a local checkpoint file. And in order to make sure state store rebuilds or builds its state by reading from changelog topic f

Re: Difference between with and without state store cleanup streams startup

2017-03-31 Thread Matthias J. Sax
1. The whole log will be read. 2. It will read all the key-value pairs. However, the store will contain only the latest record for each key, after state recovery finished. Both both (1) and (2): note, that changelog topics are compacted, thus, it will not read everything since you started your ap

Difference between with and without state store cleanup streams startup

2017-03-31 Thread Sachin Mittal
Hi, There are two ways to re start a streams application 1. executing streams.cleanUp() before streams.start() This cleans up the local state store. 2. Just by calling streams.start() What are the differences between two. As I understand in first case it will try to create local state store by r