Re: Find the running median from a data stream

2017-07-24 Thread Fabian Hueske
Hi Gabriele, I don't think you can compute the exact running median on a stream. This would require to collect all elements of the stream so you would basically need to put the complete stream into the ValueState. Even if the state is backed by RocksDB, the state for a specific key needs to fit

Find the running median from a data stream

2017-07-23 Thread Gabriele Di Bernardo
Hi guys, I want to keep track of the running median of a keyed data stream. I was considering to apply a RichMapFunction to the stream and store in a ValueState object two heaps (PriorityQueue) in order to find the running median. However, I am not really sure if this is the best approach