Re: Changing tumbling windows inclusion

2019-05-07 Thread John Roesler
Hi Alessandro,

Interesting. I agree, messing with the record timestamp to achieve your
goal sounds too messy.

It should be pretty easy to plug in your own implementation of Windows,
instead of using the built-in TimeWindows, if you want slightly different
windowing behavior.

Does that work for you? Feel free to provide more details if you want help
brainstorming.

Thanks,
-John

On Tue, May 7, 2019 at 1:39 AM Alessandro Tagliapietra <
tagliapietra.alessan...@gmail.com> wrote:

> Hello everyone,
>
> I'm trying to window a stream of machine production data, this use case
> needs a message with timestamp ending at the tumbling window end to be
> included in the current window not the next, because the message production
> amount refers to the previous x seconds. This doesn't work because by the
> docs:
>
> Tumbling time windows are aligned to the epoch, with the lower interval
> > bound being inclusive and the upper bound being exclusive
>
>
> is there a way to have the lower bound exclusive and the upper one
> inclusive?
> Another idea is to change our timestamp extractor and remove 1 second from
> the message timestamp but it's not an option I'd like.
>
> Thank you
>
> --
> Alessandro Tagliapietra
>


Re: Changing tumbling windows inclusion

2019-05-07 Thread Alessandro Tagliapietra
Hi John,

on Slack Matthias suggested to have my own transform to window the data
myself, I'll have a look into it and the Windows implementation as you
suggested and see what I can do!

Thanks for the advice!

--
Alessandro Tagliapietra


On Tue, May 7, 2019 at 8:45 AM John Roesler  wrote:

> Hi Alessandro,
>
> Interesting. I agree, messing with the record timestamp to achieve your
> goal sounds too messy.
>
> It should be pretty easy to plug in your own implementation of Windows,
> instead of using the built-in TimeWindows, if you want slightly different
> windowing behavior.
>
> Does that work for you? Feel free to provide more details if you want help
> brainstorming.
>
> Thanks,
> -John
>
> On Tue, May 7, 2019 at 1:39 AM Alessandro Tagliapietra <
> tagliapietra.alessan...@gmail.com> wrote:
>
> > Hello everyone,
> >
> > I'm trying to window a stream of machine production data, this use case
> > needs a message with timestamp ending at the tumbling window end to be
> > included in the current window not the next, because the message
> production
> > amount refers to the previous x seconds. This doesn't work because by the
> > docs:
> >
> > Tumbling time windows are aligned to the epoch, with the lower interval
> > > bound being inclusive and the upper bound being exclusive
> >
> >
> > is there a way to have the lower bound exclusive and the upper one
> > inclusive?
> > Another idea is to change our timestamp extractor and remove 1 second
> from
> > the message timestamp but it's not an option I'd like.
> >
> > Thank you
> >
> > --
> > Alessandro Tagliapietra
> >
>