It is better to have 2 bolts - mysql bolt and mongodb bolt. Let mysql bolt forward the tuple to mongodb bolt, so in case of error it won't emit.
On 10/12/15, Ankur Garg <ankurga...@gmail.com> wrote: > So I have a situation where the tuple received on Spout has to be saved to > mysql database and mongoDb as well . > > What should be better . Using 1 bolt to save it into mysql and MongoDb or 2 > seperate Bolts (One for saving into mysql and other for saving into Mongo). > > What happens when the exception occurs while saving into mysql ? I believe > I will get acknowledgement inside the fail method in my Spout . So If I > reprocess it using 2 bolts , I believe it will again be sent to Bolt for > saving into Mongo database . > > If the above is true , will having 2 seperate bolts be of any advantage ? > how can I configure things so that Failure while inserting into mysql does > not impact inserting into MongoDb . > > Thanks > Ankur > > On Sun, Oct 11, 2015 at 10:57 PM, Ravi Sharma <ping2r...@gmail.com> wrote: > >> That depends if ur spout error has affected jvm or normal application >> error >> >> performance issue in case of lot of errors, I don't think there is any >> issue be coz of errors themselves but ofcourse if u r retrying these >> messages on failure then that means u will be processing lot of messages >> then normal and overall throughput will go down >> >> Ravi >> >> If ur topology has enabled acknowledgment that means spout will always >> receive >> On 11 Oct 2015 18:15, "Ankur Garg" <ankurga...@gmail.com> wrote: >> >>> >>> Thanks for the reply Abhishek and Ravi . >>> >>> One question though , going with One topology with multiple spouts >>> ...What if something goes wrong in One spout or its associated bolts .. >>> Does it impact other Spout as well? >>> >>> Thanks >>> Ankur >>> >>> On Sun, Oct 11, 2015 at 10:21 PM, Ravi Sharma <ping2r...@gmail.com> >>> wrote: >>> >>>> No 100% right ansers , u will have to test and see what will fit.. >>>> >>>> persoanlly i wud suggest Multiple spouts in one Topology and if you >>>> have >>>> N node where topology will be running then each Spout(reading from one >>>> queue) shud run N times in parallel. >>>> >>>> if 2 Queues and say 4 Nodes >>>> then one topolgy >>>> 4 Spouts reading from Queue1 in different nodes >>>> 4 spouts reading from Queue2 in different nodes >>>> >>>> Ravi. >>>> >>>> On Sun, Oct 11, 2015 at 5:25 PM, Abhishek priya < >>>> abhishek.pr...@gmail.com> wrote: >>>> >>>>> I guess this is a question where there r no really correct answers. >>>>> I'll certainly avoid#1 as it is better to keep logic separate and >>>>> lightweight. >>>>> >>>>> If your downstream bolts are same, then it makes senses to keep them >>>>> in >>>>> same topology but if they r totally different, I'll keep them in two >>>>> different topologies. That will allow me to independently deploy and >>>>> scale >>>>> the topology. But if the rest of logic is same I topology scaling and >>>>> resource utilization will be better with one topology. >>>>> >>>>> I hope this helps.. >>>>> >>>>> Sent somehow.... >>>>> >>>>> > On Oct 11, 2015, at 9:07 AM, Ankur Garg <ankurga...@gmail.com> >>>>> > wrote: >>>>> > >>>>> > Hi , >>>>> > >>>>> > So I have a situation where I want to read messages from different >>>>> queues hosted in a Rabbitmq Server . >>>>> > >>>>> > Now , there are three ways which I can think to leverage Apache >>>>> > Storm >>>>> here :- >>>>> > >>>>> > 1) Use the same Spout (say Spout A) to read messages from different >>>>> queues and based on the messages received emit it to different Bolts. >>>>> > >>>>> > 2) Use different Spout (Spout A and Spout B and so on) within the >>>>> same topology (say Topology A) to read messages from different queues >>>>> . >>>>> > >>>>> > 3) Use Different Spouts one within eachTopology (Topology A , >>>>> Topology B and so on) to read messages from different queues . >>>>> > >>>>> > Which is the best way to process this considering I want high >>>>> throughput (more no of queue messages to be processed concurrently) . >>>>> > >>>>> > Also , If In use same Topology for all Spouts (currently though >>>>> requirement is for 2 spouts) will failure in one Spout (or its >>>>> associated >>>>> Bolts) effect the second or will they both continue working separately >>>>> even >>>>> if some failure is in Spout B ? >>>>> > >>>>> > Cost wise , how much would it be to maintain two different >>>>> > topologies >>>>> . >>>>> > >>>>> > Looking for inputs from members here. >>>>> > >>>>> > Thanks >>>>> > Ankur >>>>> > >>>>> > >>>>> >>>> >>>> >>> >
