Hi, with people proposing changes to the streaming part I also wanted to throw my hat into the ring. :D
During the last few months, while I was getting acquainted with the streaming system, I wrote down some thoughts I had about how things could be improved. Hopefully, they are in somewhat coherent shape now, so please have a look if you are interested in this: https://docs.google.com/document/d/1rSoHyhUhm2IE30o5tkR8GEetjFvMRMNxvsCfoPsW6_4/edit?usp=sharing This mostly covers: - Timestamps assigned at sources - Out-of-order processing of elements in window operators - API design Please let me know what you think. Comment in the document or here in the mailing list. I have a PR in the makings that would introduce source timestamps and watermarks for keeping track of them. I also hacked a proof-of-concept of a windowing system that is able to process out-of-order elements using a FlatMap operator. (It uses panes to perform efficient pre-aggregations.) Cheers, Aljoscha
