Hi Luis, Receiving and sending 5000 messages per second should not be a problem. Hopefully the storing and processing of those messages will keep up.
The EventBus is actor system local. It doesn't work over remoting. If you only have a fixed number of consumers, I would go with a BroadcastGroup. It uses ActorSelection under the hood broadcast messages to all the configured actor paths. B/ On 14 May 2014 at 03:53:54, Luis Medina (lu4...@gmail.com) wrote: Say that I have an actor (let's call it the ValveActor) that is taking in the Twitter stream and needs to split it (or rather copy it) so that the same stream now flows to 2 different parts of the system. Say for example that one copy of the stream is going to an actor that stores it somewhere, say the StorageActor (the details of this are irrelevant), and the 2nd copy will continue on to other actors to be used for real-time processing (let's call the first of these actors the RealTimeActor). Keep in mind that this would be happening in a cluster environment that is spread out on different machines. I have a couple of ideas in mind that I wanted to get some feedback on and perhaps learn if there is a better way of doing this. Now before anyone suggests Akka streams, yes I am aware of them and they would actually fit this use-case perfectly. The only problem is that they're not scheduled to be released until later in the year and thus the library isn't production ready yet so I need to come up with an alternative. So anyways, here are my ideas: 1. The ValveActor sends the Twitter messages to a group broadcast router which then get sent to its routees (the StorageActor and the RealTimeActor). 2. The ValveActor publishes the Twitter messages to an Event Bus which are then received by its subscribers (again, the StorageActor and the RealTimeActor). The implementation of the event bus shouldn't be complicated. Something like a LookupEventBuss would do the trick I think. The main consideration that I want to keep in mind is throughput - at one point I will be processing the Twitter firehose whose message rate is around 5000 msgs/sec (on average, this can go much higher during spikes) so the solution needs to be able to handle high messages loads without getting too bogged down. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Björn Antonsson Typesafe – Reactive Apps on the JVM twitter: @bantonsson JOIN US. REGISTER TODAY! Scala Days June 16th-18th, Berlin -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
