Re: Flushing the result of a groupReduce to a Sink before all reduces complete

2016-10-28 Thread Paul Wilson
when to close a group > until all data is consumed. > > Please let me know if you have further questions. > > Best, Fabian > > > 2016-10-26 19:07 GMT+02:00 Paul Wilson <paulalexwil...@gmail.com>: > >> Hi, >> >> DataSet API >> Flink 1.1.3 &

Flushing the result of a groupReduce to a Sink before all reduces complete

2016-10-26 Thread Paul Wilson
Hi, DataSet API Flink 1.1.3 I have an application where I'd like to perform some mapping before batching the results and passing them to the sink. I'm performing a 'composite' key selection to group the items by their natural key as well as a batch (itemCount / batchSize). When I reduce the

Remote upload and execute

2016-09-02 Thread Paul Wilson
Hi, I'd like to write a client that can execute an already 'uploaded' JAR (i.e. the JAR is deployed and available by some other external process). This is similar to what the web console allows which consists of 2 steps: upload the JAR followed by a submit with parameters. I'm looking at the

Re: Custom Barrier?

2016-06-14 Thread Paul Wilson
... and those events are in order On 14 Jun 2016 14:04, "Paul Wilson" <paulalexwil...@gmail.com> wrote: > Hi, > > No these super-structure events only serve the purpose of defining the > boundaries of a join, and do not relate to the keys of the sub-events. > >

Custom Barrier?

2016-06-13 Thread Paul Wilson
Hi, I've been evaluating Flink and wondering if it was possible to define a window that is based on characteristics of the data (data driven) but not contained in the data stream directly. Consider 'nested events' where lower level events belong to a wider event where the wider event serves only