Re: MiniCluster with ProcessingTimeTrigger

2019-12-18 Thread Biao Liu
= IntStream.rangeClosed(1, > inputSize).boxed().collect(Collectors.toList()); > > // 1st half of pipeline > //DataStream> pipeA = env.fromCollection(listOfNumbers) > DataStream> pipeA = env.addSource(new > StreamTest.DripFeed(inputSize)) > .windowA

Re: MiniCluster with ProcessingTimeTrigger

2019-12-18 Thread John Morrow
___ From: Biao Liu Sent: Tuesday 17 December 2019 21:50 To: John Morrow Cc: user Subject: Re: MiniCluster with ProcessingTimeTrigger Hi John, The root cause is the collection source exits too fast. The window would also exit without being triggered. You could verify that by waiting

Re: MiniCluster with ProcessingTimeTrigger

2019-12-17 Thread Biao Liu
Hi John, The root cause is the collection source exits too fast. The window would also exit without being triggered. You could verify that by waiting a second before releasing the window. For example, insert a map operator between source and window operator. Blocking a second or more in the

MiniCluster with ProcessingTimeTrigger

2019-12-17 Thread John Morrow
Hi All, I'm trying to test a pipeline that consists of two Flink tasks with a MiniCluster. The 1st task has a WindowAll operator which groups items into batches every second, and the 2nd task does an async operation with each batch and flatMaps the result. I've whittled it down to the bare