Hi Aljoscha, Thank you for the nice proposal!
I think it would make sense to allow user's to affect the readiness of the side input. I think making it ready when the first element arrives is only slightly better then making it always ready from usability perspective. For instance if I am joining against a static data set I want to wait for the whole set before making it ready. This could be exposed as a user defined condition that could also recognize bounded inputs maybe. Maybe we could also add an aggregating (merging) side input type, that could work as a broadcast state. What do you think? Gyula Aljoscha Krettek <aljos...@apache.org> ezt írta (időpont: 2017. márc. 6., H, 15:18): > Hi Folks, > > I would like to finally agree on a plan for implementing side inputs in > Flink. There has already been an attempt to come to consensus [1], which > resulted in two design documents. I tried to consolidate those two and > also added a section about implementation plans. This is the resulting > FLIP: > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API > > > In terms of semantics I tried to go with the minimal viable solution. > The part that needs discussing is how we want to implement this. I > outlined three possible implementation plans in the FLIP but what it > boils down to is that we need to introduce some way of getting several > inputs into an operator/task. > > > Please have a look at the doc and let us know what you think. > > > > Best, > > Aljoscha > > > > [1] > https://lists.apache.org/thread.html/797df0ba066151b77c7951fd7d603a8afd7023920d0607a0c6337db3@1462181294@%3Cdev.flink.apache.org%3E >
