Re: Are RocksDBWindowStore windows hopping or sliding?

2020-03-30 Thread Matthias J. Sax
`windowSize` has not impact on used segments: it is only used to compute the window end timestamp when you fetch a window, because the end-time is not stored explicitly (as all windows are assumed to have the same window size; i.e., it's a storage optimization to safe the bytes for window end

Re: Are RocksDBWindowStore windows hopping or sliding?

2020-03-30 Thread Sachin Mittal
Hi, I understood how window stores are implemented using rocksdb. When creating an instance of RocksDBWindowStore we pass two additional arguments: retainDuplicates windowSize I have not clearly understood the purpose of these two. Like say in my application I just create one windowed store of a

Re: Are RocksDBWindowStore windows hopping or sliding?

2020-03-01 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 If you want to put a record into multiple window, you can do a `put()` for each window. The DSL uses the store in the exact same manner for hopping window (compare the code I shared in the last reply). Even if windows are overlapping, the

Re: Are RocksDBWindowStore windows hopping or sliding?

2020-02-27 Thread Sachin Mittal
Hi, Yes I get that when I am using the apis provided by kstream I can basically use both: - Tumbling time window (non-overlapping, gap-less windows) - Hopping time window (Time-based Fixed-size, overlapping windows) I wanted to know if I am using state store directly when created using a

Re: Are RocksDBWindowStore windows hopping or sliding?

2020-02-26 Thread Matthias J. Sax
What you call "sliding window" is called "hopping window" in Kafka Streams. And yes, you can use a windowed-store for this case: In fact, a non-overlapping tumbling window is just a special case of a hopping window with advance == window-size. In Kafka Streams we have a single implementation for

Are RocksDBWindowStore windows hopping or sliding?

2020-02-26 Thread Sachin Mittal
Hi, So far how I have understood is that when we create a rocksdb window store; we specify a window size and retention period. So windows are created from epoch time based on size, say size if 100 then windows are: [0, 100), [100, 200), [200, 300) ... Windows are retained based on retention