[DISCUSS] FLIP-251: Support collecting arbitrary number of streams

2022-07-07 Thread Chesnay Schepler
Hello, I have created a FLIP for slightly extending the collect() API to support collecting an arbitrary number of streams, to make it more useful for testing complex workflows. https://cwiki.apache.org/confluence/display/FLINK/FLIP-251%3A+Support+collecting+arbitrary+number+of+streams Looki

Re: [DISCUSS] FLIP-251: Support collecting arbitrary number of streams

2022-07-08 Thread David Anderson
I've found that with our current tooling it's frustrating to try to write good end-to-end tests for real-world jobs with multiple sinks. DataStream#executeAndCollect() is okay for simple pipelines with one sink, but in my opinion we do need something like FLIP-251. The proposed interface looks goo

Re: [DISCUSS] FLIP-251: Support collecting arbitrary number of streams

2022-07-10 Thread Konstantin Knauf
Makes sense to me. Thank you, Chesnay. +1 David Anderson schrieb am Fr., 8. Juli 2022, 13:56: > I've found that with our current tooling it's frustrating to try to write > good end-to-end tests for real-world jobs with multiple sinks. > DataStream#executeAndCollect() is okay for simple pipelines

Re: [DISCUSS] FLIP-251: Support collecting arbitrary number of streams

2022-07-13 Thread Martijn Visser
+1 Op zo 10 jul. 2022 om 20:30 schreef Konstantin Knauf : > Makes sense to me. Thank you, Chesnay. +1 > > David Anderson schrieb am Fr., 8. Juli 2022, 13:56: > > > I've found that with our current tooling it's frustrating to try to write > > good end-to-end tests for real-world jobs with multipl