No 100% right ansers , u will have to test and see what will fit.. persoanlly i wud suggest Multiple spouts in one Topology and if you have N node where topology will be running then each Spout(reading from one queue) shud run N times in parallel.
if 2 Queues and say 4 Nodes then one topolgy 4 Spouts reading from Queue1 in different nodes 4 spouts reading from Queue2 in different nodes Ravi. On Sun, Oct 11, 2015 at 5:25 PM, Abhishek priya <abhishek.pr...@gmail.com> wrote: > I guess this is a question where there r no really correct answers. I'll > certainly avoid#1 as it is better to keep logic separate and lightweight. > > If your downstream bolts are same, then it makes senses to keep them in > same topology but if they r totally different, I'll keep them in two > different topologies. That will allow me to independently deploy and scale > the topology. But if the rest of logic is same I topology scaling and > resource utilization will be better with one topology. > > I hope this helps.. > > Sent somehow.... > > > On Oct 11, 2015, at 9:07 AM, Ankur Garg <ankurga...@gmail.com> wrote: > > > > Hi , > > > > So I have a situation where I want to read messages from different > queues hosted in a Rabbitmq Server . > > > > Now , there are three ways which I can think to leverage Apache Storm > here :- > > > > 1) Use the same Spout (say Spout A) to read messages from different > queues and based on the messages received emit it to different Bolts. > > > > 2) Use different Spout (Spout A and Spout B and so on) within the same > topology (say Topology A) to read messages from different queues . > > > > 3) Use Different Spouts one within eachTopology (Topology A , Topology B > and so on) to read messages from different queues . > > > > Which is the best way to process this considering I want high throughput > (more no of queue messages to be processed concurrently) . > > > > Also , If In use same Topology for all Spouts (currently though > requirement is for 2 spouts) will failure in one Spout (or its associated > Bolts) effect the second or will they both continue working separately even > if some failure is in Spout B ? > > > > Cost wise , how much would it be to maintain two different topologies . > > > > Looking for inputs from members here. > > > > Thanks > > Ankur > > > > >
