Hi Matthias, WindowedBolt does support event time. In trident its is not yet exposed.
Hi Daniela, You could solve your use cases in different ways. One would be to have a WindowedBolt with a 1 min tumbling window, do your custom aggregation (e.g. sum) every time the window tumbles and emit the results to another bolt where you update the count in Redis. Most of your state saving could also be automated by defining a Stateful bolt that would periodically checkpoint your state (sum per device). You could also club both windowing and state into a StatefulWindowedBolt implementation. You can evaluate the options and decide based on your use cases. Take a look at the sample topologies (SlidingWindowTopology, SlidingTupleTsTopology, StatefulTopology, StatefulWindowingTopology) in storm-starter and the docs for more info. https://github.com/apache/storm/blob/master/docs/Windowing.md https://github.com/apache/storm/blob/master/docs/State-checkpointing.md -Arun On 4/10/16, 4:30 PM, "Matthias J. Sax" <mj...@apache.org> wrote: >A tumbling window (ie, non-overlapping window) is the right approach (a >sliding window is overlapping). > >The window goes into your aggregation bolt (windowing and aggregation >goes hand in hand, ie, when the window gets closed, the aggregation is >triggered and the window content is handed over to the aggregation >function). > >Be aware that Storm (currently) only supports processing time window (an >no event time windows). > >-Matthias > > >On 04/10/2016 09:56 AM, Daniela Stoiber wrote: >> Hi, >> >> thank you for your reply. >> >> How can I ensure that the latest values are pulled from Redis the sum is >> updated every minute? Do I need a sliding window with an interval of 1 >> minute? Where would this sliding window be located in my topology? >> >> Thank you in advance. >> >> Regards, >> Daniela >> >> -----Ursprüngliche Nachricht----- >> Von: Matthias J. Sax [mailto:mj...@apache.org] >> Gesendet: Samstag, 9. April 2016 12:13 >> An: dev@storm.apache.org >> Betreff: Re: Use only latest values >> >> Sounds reasonable. >> >> >> On 04/09/2016 08:34 AM, Daniela Stoiber wrote: >>> Hi, >>> >>> >>> >>> I would like to cache values and to use only the latest "valid" values >>> to build a sum. >>> >>> In more detail, I receive values from devices periodically. I would >>> like to add up all the valid values each minute. But not every device >>> sends a new value every minute. And as long as there is no new value >>> the old one should be used for the sum. As soon as I receive a new >>> value from a device I would like to overwrite the old value and to use >>> the new one for the sum. Would that be possible with the combination of >> Storm and Redis? >>> >>> >>> >>> My idea was to use the following: >>> >>> >>> >>> - Kafka Spout >>> >>> - Storm Bolt for storing the tuples in Redis and for overwriting the >>> values as soon as a new one is delivered >>> >>> - Storm Bolt for reading the latest tuples from Redis >>> >>> - Storm Bolt for grouping (I would like to group the devices per >>> region) >>> >>> - Storm Bolt for aggregation >>> >>> - Storm Bolt for storing the results again in Redis >>> >>> >>> >>> Thank you in advance. >>> >>> >>> >>> Regards, >>> >>> Daniela >>> >>> >> >> >
