design question

hi all, I have a stream of incoming object versions (objects change over time) and a requirement to fetch from a datastore the last known object version in order to link it with the id of the new version, so that I end up with a linked list of object versions. all object versions contain the sam

Re: design question

This sounds like you have some per-key state to keep track of, so the 'correct' way to do it would be to keyBy the guid. I believe that if you run your environment using the Rocks DB state backend you will not OOM regardless of the number of GUIDs that are eventually tracked. Whether flink/stream p

Re: design question

cool - can you point me to some docs about how to configure Rocks DB? I searched the online docs and found nothing substantial. Also - If I'm using HDFS (S3backed ) cluster, how would that effect RocksDB? can I configure it to run on optimized SSD etc? any help is appreciated. On Sun, Apr 24, 20

Re: design question

Hi, in the Flink doc there is this: https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/state_backends.html#the-rocksdbstatebackend and this: RocksDBStateBackend

Flink CEP pattern design question

Currently I'm designing a CEP pattern to satisfy our business needs. Basically, there's two events let's call it a and b. Both a and b can have zero or multiple entries in the log. For input {a,b1,b2}, I want to get the output of {a,b1,b2} For input {b1,b2}, I want to get the output of {b1,b2} afte

Re: Flink CEP pattern design question

I think what you ask for is something like timing out greedy[1] quantifier, which is not supported. As a rather dirty workaround you could try sth like: Pattern .begin[Event]("start") .where(_._.getName == "a") .oneOrMore.opt