Re: Performance impact of many open windows at the same time

2020-05-25 Thread Aljoscha Krettek
Hi, I don't think this will immediately degrade performance. State is essentially stored in a HashMap (for the FileStateBackend) or RocksDB (for the RocksDB backend). If these data structures don't degrade with size then your performance also shouldn't degrade. There are of course some

Re: Performance impact of many open windows at the same time

2020-05-21 Thread Tzu-Li (Gordon) Tai
Hi Joe, The main effect this should have is more state to be kept until the windows can be fired (and state purged). This would of course increase the time it takes to checkpoint the operator. I'm not sure if there will be significant runtime per-record impact caused by how windows are

Performance impact of many open windows at the same time

2020-05-21 Thread Joe Malt
Hi all, I'm looking into what happens when messages are ingested with timestamps far into the future (e.g. due to corruption or a wrong clock at the sender). I'm aware of the effect on watermarking, but another thing I'm concerned about is the performance impact of the extra windows this will