It all depends on how you want to do it, and a lot of it depends on who will be
consuming the data downstream. You can have a bolt that all it does it publish
to redis, or kafka, or mysql, or what ever other store you want. You can then
choose to either have a separate spout that consumes this data from the store,
or you can just us the same stream that was going into the storage bolt, and
use that to do your other aggregations. Having a separate spout is useful if
multiple different groups want to consume that data. I.e. Group a takes the
data and cleans it up, but group b handles finances and counts the money made,
and group c handles targeting and helps see what people like. Having all three
groups coordinate writing and deploying a single topology is a pain, so having
an intermediate message broker can provide a good separation between the
different groups.
- Bobby
On Monday, January 4, 2016 1:08 AM, Daniela Stoiber
<daniela.stoi...@gmail.com> wrote:
Hello
I would like to process, join and aggregate data (with Storm Bolts) and
store this data with another Bolt into a database (e.g. Redis) to create a
new stream out of this data stored in Redis to do further aggregation on
it.
Would it be possible to create a new stream out of the processed data
stored in Redis with a new Storm Spout or would I have to use another tool
to create the new stream or would it in general be easier to do the
aggregations in Redis itself without the use of Storm.
Thank you very much in advance.
Best regards,
Daniela
