Hi, I need to coordinate between two spouts, and could use the group's insight into this. The scenario:
- One spout (let's call it event spout) receives the incoming data stream. - One spout is the "control" spout, which will receive messages from a different stream, that can impact the way that the Event Spout processes. These changes could even mean that the event spout pauses (stops processing from incoming stream until further notice). So I need to be able to trigger an action in the control spout that will affect state in the event spout. Since the spouts could end up distributed in different workers, I cannot just pass a reference. The fact that the event spout might pause makes it not feasible to use the same stream for injecting the control messages (i.e. I can send a "pause" message through the event channel, but then the spout pauses and I can't pass a "resume" message through the same channel). I'm thinking of using zookeeper or a distributed cache/shared memory system (which we already have in place for other reasons) to keep state, but I'm not having any ideas on how to actually pass the changes from control spout to event spout other than polling zookeeper/cache from the event spout. Thank you, Javier
