Thanks for the reply Abhishek and Ravi . One question though , going with One topology with multiple spouts ...What if something goes wrong in One spout or its associated bolts .. Does it impact other Spout as well?
Thanks Ankur On Sun, Oct 11, 2015 at 10:21 PM, Ravi Sharma <ping2r...@gmail.com> wrote: > No 100% right ansers , u will have to test and see what will fit.. > > persoanlly i wud suggest Multiple spouts in one Topology and if you have N > node where topology will be running then each Spout(reading from one queue) > shud run N times in parallel. > > if 2 Queues and say 4 Nodes > then one topolgy > 4 Spouts reading from Queue1 in different nodes > 4 spouts reading from Queue2 in different nodes > > Ravi. > > On Sun, Oct 11, 2015 at 5:25 PM, Abhishek priya <abhishek.pr...@gmail.com> > wrote: > >> I guess this is a question where there r no really correct answers. I'll >> certainly avoid#1 as it is better to keep logic separate and lightweight. >> >> If your downstream bolts are same, then it makes senses to keep them in >> same topology but if they r totally different, I'll keep them in two >> different topologies. That will allow me to independently deploy and scale >> the topology. But if the rest of logic is same I topology scaling and >> resource utilization will be better with one topology. >> >> I hope this helps.. >> >> Sent somehow.... >> >> > On Oct 11, 2015, at 9:07 AM, Ankur Garg <ankurga...@gmail.com> wrote: >> > >> > Hi , >> > >> > So I have a situation where I want to read messages from different >> queues hosted in a Rabbitmq Server . >> > >> > Now , there are three ways which I can think to leverage Apache Storm >> here :- >> > >> > 1) Use the same Spout (say Spout A) to read messages from different >> queues and based on the messages received emit it to different Bolts. >> > >> > 2) Use different Spout (Spout A and Spout B and so on) within the same >> topology (say Topology A) to read messages from different queues . >> > >> > 3) Use Different Spouts one within eachTopology (Topology A , Topology >> B and so on) to read messages from different queues . >> > >> > Which is the best way to process this considering I want high >> throughput (more no of queue messages to be processed concurrently) . >> > >> > Also , If In use same Topology for all Spouts (currently though >> requirement is for 2 spouts) will failure in one Spout (or its associated >> Bolts) effect the second or will they both continue working separately even >> if some failure is in Spout B ? >> > >> > Cost wise , how much would it be to maintain two different topologies . >> > >> > Looking for inputs from members here. >> > >> > Thanks >> > Ankur >> > >> > >> > >
