What is the best way to aggregate data over a long window

[Sachin Mittal]

Hi,
My pipeline step is something like this:

SingleOutputStreamOperator<ReducedData> reducedData =
    data
        .keyBy(new KeySelector())
        .window(
            TumblingEventTimeWindows.of(Time.seconds(secs)))
        .reduce(new DataReducer())
        .name("reduce");


This works fine for secs = 300.
However once I increase the time window to say 1 hour or 3600 the state
size increases as now it has a lot more records to reduce.

Hence I need to allocate much more memory to the task manager.

However there is no upper limit to this memory allocated. If the volume of
data increases by say 10 fold I would have no option but to again increase
the memory.

Is there a better way to perform long window aggregation so overall this
step has a small memory footprint.

Thanks
Sachin