Hi Fabian, Thank you for your suggestion. In the meantime I rethought this problem and implemented alternative solution without using windows at all. I used plain ProcessFunction with 1. Keyed state (by companyId) - to keep ratings per key 2. EventTime timers - to remove outdated ratings from state and emit recalculated score immediately
This solution gives results in real-time, windows would delay the results by 1 day I think. Regards, Krzysztof On Thu, Feb 22, 2018 at 9:44 AM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi Krzysztof, > > you could compute the stats in two stages: > > 1) compute an daily window. You should use a ReduceFunction or > AggreagteFunction here if possible to perform the computation eagerly. > 2) compute a sliding window of 90 days with a 1 day hop (or 90 rows with a > 1 row hop). > > That will crunch down the data in the first window and the second window > will combine the pre-aggregated results. > > Hope this helps, > Fabian > > 2018-02-19 16:36 GMT+01:00 Krzysztof Białek <krzysiek.bia...@gmail.com>: > >> Hi, >> >> My app is calculating Companies scores from Ratings given by users. >> Only ratings from last 90 days should be considered. >> >> 1. Is it possible to construct window processing ratings from last 90 >> days? >> I've started with *misusing* countWindow but this solution looks ugly for >> me. >> >> ratingStream >> .filter(new OutdatedRatingsFilter(maxRatingAge)) >> .keyBy(_.companyId) >> .countWindow(0L).trigger(new OnEventTrigger).evictor(new >> OutdatedRatingsEvictor(maxRatingAge)) >> .process(ratingFunction) >> >> >> 2. How to recalculate score once the rating expires (after 90 days)? >> I don't want to put artificial ratings into the stream to trigger the >> recalculation. >> >> Any idea how can I do it better? >> >> Regards, >> Krzysztof >> >> >> >
