+1
No further concerns from my side either
> On 13 Jul 2015, at 18:30, Gyula Fóra <gyula.f...@gmail.com> wrote:
>
> +1
> On Mon, Jul 13, 2015 at 6:23 PM Stephan Ewen <se...@apache.org> wrote:
>
>> If naming is the only concern, then we should go ahead, because we can
>> change names easily (before the release).
>>
>> In fact, I don't think it leaves a bad impression. Global windows are
>> non-parallel windows. There are also parallel windows. Pick what you need
>> and what works.
>>
>>
>> On Mon, Jul 13, 2015 at 6:13 PM, Gyula Fóra <gyula.f...@gmail.com> wrote:
>>
>>> I think we agree on everything its more of a naming issue :)
>>>
>>> I thought it might be misleading that global time windows are
>>> "non-parallel" windows. We dont want to give a bad impression. (Also we
>>> dont want them to think that every global window is parallel but thats
>> not
>>> a problem here)
>>>
>>> Gyula
>>> On Mon, Jul 13, 2015 at 5:22 PM Stephan Ewen <se...@apache.org> wrote:
>>>
>>>> Okay, what is missing about the windowing in your opinion?
>>>>
>>>> The core points of the document are:
>>>>
>>>> - The parallel windows are per group only.
>>>>
>>>> - The implementation of the parallel windows holds window data in the
>>>> group buffers.
>>>>
>>>> - The global windows are non-parallel. May have parallel
>>> pre-aggregation,
>>>> if they are time windows.
>>>>
>>>> - Time may be operator time (timer thread), or watermark time.
>>> Watermark
>>>> time can refer to ingress or event time.
>>>>
>>>> - Windows that do not pre-aggregate may require elements in order.
>> Not
>>>> part of the first prototype.
>>>>
>>>> Do we agree on those points?
>>>>
>>>>
>>>> On Mon, Jul 13, 2015 at 4:50 PM, Gyula Fóra <gyula.f...@gmail.com>
>>> wrote:
>>>>
>>>>> In general I like it, although the main difference between the
>> current
>>>> and
>>>>> the new one is the windowing and that is still not very clear.
>>>>>
>>>>> Where do we have the full stream time windows for instance?(which is
>>>>> parallel but not keyed)
>>>>> On Mon, Jul 13, 2015 at 4:28 PM Aljoscha Krettek <
>> aljos...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> +1 I like it as well.
>>>>>>
>>>>>> On Mon, 13 Jul 2015 at 16:17 Kostas Tzoumas <ktzou...@apache.org>
>>>> wrote:
>>>>>>
>>>>>>> +1 from my side
>>>>>>>
>>>>>>> On Mon, Jul 13, 2015 at 4:15 PM, Stephan Ewen <se...@apache.org>
>>>>> wrote:
>>>>>>>
>>>>>>>> Do we have consensus on these designs?
>>>>>>>>
>>>>>>>> If we have, we should get to implementing this soon, because
>>>>> basically
>>>>>>> all
>>>>>>>> streaming patches will have to be revisited in light of this...
>>>>>>>>
>>>>>>>> On Tue, Jul 7, 2015 at 3:41 PM, Gyula Fóra <
>> gyula.f...@gmail.com
>>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> You are right thats an important issue.
>>>>>>>>>
>>>>>>>>> And I think we should also do some renaming with the
>>> "iterations"
>>>>>>> because
>>>>>>>>> they are not really iterations like in the batch case and it
>>>> might
>>>>>>>> confuse
>>>>>>>>> some users.
>>>>>>>>> Maybe we can call them loops or cycles and rename the api
>> calls
>>>> to
>>>>>> make
>>>>>>>> it
>>>>>>>>> more intuitive what happens. It is really just a cyclic
>>> dataflow.
>>>>>>>>>
>>>>>>>>> Aljoscha Krettek <aljos...@apache.org> ezt írta (időpont:
>>> 2015.
>>>>> júl.
>>>>>>> 7.,
>>>>>>>>> K,
>>>>>>>>> 15:35):
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I just noticed that we don't have anything about how
>>> iterations
>>>>> and
>>>>>>>>>> timestamps/watermarks should interact.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Aljoscha
>>>>>>>>>>
>>>>>>>>>> On Mon, 6 Jul 2015 at 23:56 Stephan Ewen <se...@apache.org
>>>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all!
>>>>>>>>>>>
>>>>>>>>>>> As many of you know, there are a ongoing efforts to
>>>> consolidate
>>>>>> the
>>>>>>>>>>> streaming API for the next release, and then graduate it
>>>> (from
>>>>>> beta
>>>>>>>>>>> status).
>>>>>>>>>>>
>>>>>>>>>>> In the process of this consolidation, we want to achieve
>>> the
>>>>>>>> following
>>>>>>>>>>> goals.
>>>>>>>>>>>
>>>>>>>>>>> - Make the code more robust and simplify it in parts
>>>>>>>>>>>
>>>>>>>>>>> - Clearly define the semantics of the constructs.
>>>>>>>>>>>
>>>>>>>>>>> - Prepare it for support of more advanced concepts, like
>>>>>>>> partitionable
>>>>>>>>>>> state, and event time.
>>>>>>>>>>>
>>>>>>>>>>> - Cut support for certain corner cases that were
>>> prototyped,
>>>>> but
>>>>>>>>> turned
>>>>>>>>>>> out to be not efficiently doable
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Based on prior discussions on the mailing list, Aljoscha
>>> and
>>>> me
>>>>>>>> drafted
>>>>>>>>>> the
>>>>>>>>>>> design documents below, which outline how the
>> consolidated
>>>> API
>>>>>>> would
>>>>>>>>>> like.
>>>>>>>>>>> We focused in constructs, time, and window semantics.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Design document on how to restructure the Streaming API:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/Streams+and+Operations+on+Streams
>>>>>>>>>>>
>>>>>>>>>>> Design document on definitions of time, order, and the
>>>>> resulting
>>>>>>>>>> semantics:
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/Time+and+Order+in+Streams
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Note: The design of the interfaces and concepts for
>>> advanced
>>>>>> state
>>>>>>> in
>>>>>>>>>>> functions is not in here. That is part of a separate
>> design
>>>>>>>> discussion
>>>>>>>>>> and
>>>>>>>>>>> orthogonal to the designs drafted here.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Please have a look and voice questions and concerns.
>> Since
>>> we
>>>>>>> should
>>>>>>>>> not
>>>>>>>>>>> break the streaming API more than once, we should make
>> sure
>>>>> this
>>>>>>>>>>> consolidation brings it into the shape we want it to be
>> in.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Greetings,
>>>>>>>>>>> Stephan
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
