Re: Fixing the ExecutionConfig

2015-11-18 Thread Ufuk Celebi
I like this idea. +1 > On 18 Nov 2015, at 15:25, Stephan Ewen wrote: > > I had pretty much in mind what Aljoscha suggested. > > On Thu, Nov 12, 2015 at 11:37 AM, Aljoscha Krettek > wrote: > >> IMHO it’s not possible to have streaming/batch specific ExecutionConfig >> since the user functions

Re: Fixing the ExecutionConfig

2015-11-18 Thread Stephan Ewen
I had pretty much in mind what Aljoscha suggested. On Thu, Nov 12, 2015 at 11:37 AM, Aljoscha Krettek wrote: > IMHO it’s not possible to have streaming/batch specific ExecutionConfig > since the user functions share a common interface, i.e. > getRuntimeContext().getExecutionConfig() simply retur

Re: Fixing the ExecutionConfig

2015-11-12 Thread Aljoscha Krettek
IMHO it’s not possible to have streaming/batch specific ExecutionConfig since the user functions share a common interface, i.e. getRuntimeContext().getExecutionConfig() simply returns the same type for both. What could be done is to migrate batch/streaming specific stuff to the ExecutionEnviron

Re: Fixing the ExecutionConfig

2015-11-12 Thread Maximilian Michels
+1 for separating concerns by having a StreamExecutionConfig and a BatchExecutionConfig with inheritance from ExecutionConfig for general options. Not sure about the pre-flight and runtime options. I think they are ok in one config. On Wed, Nov 11, 2015 at 1:24 PM, Robert Metzger wrote: > I think

Re: Fixing the ExecutionConfig

2015-11-11 Thread Robert Metzger
I think now (before the 1.0 release) is the right time to clean it up. Are you suggesting to have two execution configs for batch and streaming? I'm not sure if we need to distinguish between pre-flight and runtime options: From a user's perspective, it doesn't matter. For example the serializer

Fixing the ExecutionConfig

2015-11-11 Thread Stephan Ewen
Hi all! The ExecutionConfig is a bit of a strange thing right now. It looks like it became the place where everyone just put the stuff they want to somehow push from the client to runtime, plus a random assortment of conflig flags. As a result: - The ExecutionConfig is available in batch and s