It could be just one job, but what is the best way to deploy many instances of this job so I could process a heavy load of messages?
Thanks, On 13 Aug 2014, at 01:39, Yan Fang <[email protected]> wrote: > *"Does one kafka-broker handle this much messages per second?"* > > I believe @Chris has better answer about this. > > > > *"I have one job that get this messages and another that reads from the > output of the first job that does some more processing."* > > Why not use one job get messages and process them? > > *" when I change a* > > *configuration of one my jobs do I need to recompile it and send the new > tar.gz to hdfs or just change the deploy/samza config and it should work."* > > No, you don't need to recompile. Change the config and run-job. It will > work. > > Thanks. > > Cheers, > > Fang, Yan > [email protected] > +1 (206) 849-4108 > > > On Tue, Aug 12, 2014 at 8:47 PM, Telles Nobrega <[email protected]> > wrote: > >> Not completely related to the topic of the question but when I change a >> configuration of one my jobs do I need to recompile it and send the new >> tar.gz to hdfs or just change the deploy/samza config and it should work. >> >> Thanks >> >> >> On Tue, Aug 12, 2014 at 11:23 PM, Telles Nobrega <[email protected]> >> wrote: >> >>> Hi, I'm running an experiment that I'm suppose to run samza with >> different >>> input rates. First I'm running with 420 messages/second and I scale up to >>> 33200 messages/second. >>> >>> Does one kafka-broker handle this much messages per second? >>> Second, what is the best way to read into samza this much messages? I >> have >>> one job that get this messages and another that reads from the output of >>> the first job that does some more processing. Is the best way to use more >>> containers and split kafka topics in partitions (the same number of >>> containers) or is there a better way to do this. >>> >>> Thanks in advance, >>> >>> -- >>> ------------------------------------------ >>> Telles Mota Vidal Nobrega >>> M.sc. Candidate at UFCG >>> B.sc. in Computer Science at UFCG >>> Software Engineer at OpenStack Project - HP/LSD-UFCG >>> >> >> >> >> -- >> ------------------------------------------ >> Telles Mota Vidal Nobrega >> M.sc. Candidate at UFCG >> B.sc. in Computer Science at UFCG >> Software Engineer at OpenStack Project - HP/LSD-UFCG >>
